Menu Close

Data Engineer (JHB)

IT – Software Development
Johannesburg – Gauteng

ENVIRONMENT:
A large Retail and Consumer Finance Concern in Joburg seeks the technical expertise of a Data Engineer whose core role will be to drive, design & build scalable ETL systems. These systems will be for a Big Data warehouse where you will implement robust & trustworthy data to support high performing ML algorithms, predictive models and support real-time data visualisation requirements across the organisation to enable self-help analytics. The ideal candidate will require a 3-year IT related Degree, have 5-10 years’ experience designing and developing data warehouses according to the Kimball methodology, is adept at design and development of ETL processes, SQL development experience & preferably SAS Data Studio and AWS experience. You must also be able to ingest/output CSV, JSON and other flat file types and any related data sources, have Retail/ Financial Services and Logistics experience and proficiency in Python or R or a willingness to learn.
 
DUTIES:
Systematic solution design of the ETL and data pipeline in line with business user specifications –
  • Ensure highest data quality assurance, data accuracy and data completeness through regular and in-depth review and testing of work.
  • Create easily understandable technical documentation that are kept up to date.
  • Conduct data design, database architecture, metadata and repository creation activities and tasks as required by business stakeholder.
  • Translate business needs into long-term architecture solutions.
  • Define, design and build dimensional databases.
  • Design the ETL pipelines.
  • Responsible for developing data warehousing blueprints, evaluating hardware and software platforms and integrating systems.
  • Evaluate reusability of current data for additional analyses.
  • Conduct data cleaning to rid the system of old, unused, or duplicate data.
  • Review object and data models and the metadata repository to structure the data for better management and quicker access.
  • Determine processes to ensure execution of relevant data application requirements for various business needs.
  • Utilise relevant templates that outlines the requirements for each step within the data modelling journey.
  • Conduct testing and quality control of databases to ensure accurate and appropriate use of data.
  • Initiate and successfully motivate improved ways of operating.
 
Develop and implement ETL pipelines aligned to the approved solution design –
  • Enhance and maintain existing ETL frameworks in line with agreed design patterns and internal governance standards to improve the EDW product offering and to remain scalable.
  • Implement the ETL pipeline in a timely manner.
  • Utilise most accurate data source to remodel into a set of data that is understandable to the end user.
  • Understand data structures to deliver data sets that are deliver to exact requirements of end user brief.
  • Ensure data is precise and is benchmarked and validated against financial records.
  • Utilise consistent data sources which result in one version of the truth.
  • Deliver on standard data marts that can be utilised for reporting and analysis which is well documented and understood by business users.
  • Translate Meta data into explanatory reports and visuals for easy understanding to end user.
  • Perform data pre-processing which includes data manipulation, transformation, normalisation, standardisation, visualisation and derivation of new variables/features, as applicable to developing specific algorithms or models.
 
Ensure data governance and data quality assurance standards are upheld –
  • Facilitate an understanding of data sources to ensure governance, procedures and standards are upheld.
  • Build data quality metrics and conduct data validation testing.
  • Follow the IT governance process when implementing a change to ensure governance standards and protocols are followed.
  • Work close with business to understand business processes and standards in order to develop data quality assurance metrics.
  • Build exception reports to help identify data quality problems.
  • Provide feedback to business owners on identified problems to ensure quality of data is rectified.
 
Deal with customers in a customer centric manner –
  • Utilise specialist knowledge to explain the data and transfer the understanding to business end user.
  • Conduct training and upskilling on new reports and or self-service analytics platforms to relevant stakeholders.
  • Communication to stakeholders to keep them abreast of current developments within the function and to manage expectations.
  • Apply the standards set out in all legislation, policy and procedure that effects the customer.
  • Deal effectively and timeously with customer complaints and ensure that complaints are resolved or escalated in line with agreed standards.
  • Deliver services and products to the customer within the parameters of the agreed SLA.
  • Recognise and celebrate customer centric behaviour within others.
  • Manage colleagues and customers’ expectations and communicate appropriately within the parameters of agreed SLAs.
  • Act in a customer centric manner that is in line with the service code and core ideology in order to meet and exceed the requirements of internal and external customers.
 
Effective Self-Management and Teamwork –
  • Actively and consistently maintain high standards of professionalism in all aspects of personal presentation and delivery.
  • Apply knowledge of the organizational systems, structures, policies and procedures to achieve results.
  • Demonstrate initiative in follow through to ensure that personal quality and productivity standards are consistently and accurately maintained.
  • Provide appropriate resolution for tasks or deadlines not met.
  • Support and drive the business’ core values.
  • Maintain a positive attitude and respond openly to feedback.
  • Take ownership for driving own career development.
  • Show commitment to teamwork and a willingness to go the extra mile to achieve team objectives.
 
REQUIREMENTS:
Qualifications –
  • 3-Year IT related Degree.
  • Post-graduate qualification (advantageous).
 
Experience/Skills –
  • 5-10 Years’ experience and understanding in designing and developing data warehouses according to the Kimball methodology.
  • Adept at design and development of ETL processes. SQL development experience, preferably SAS Data Studio and AWS experience T
  • he ability to ingest/output CSV, JSON and other flat file types and any related data sources.
  • Proficient in Python or R or willingness to learn.
  • Experience within Retail, Financial Services and Logistics environments.
  • Data Architecture, Data Modelling and Data Pipelining.
  • Solutions Architecture.
 
ATTRIBUTES:
  • Adapting and responding to change.
  • Presenting and communicating information.
  • Stakeholder Management.
  • Excellent written and verbal communication skills.
  • Analysis and Judgement.
  • Personal resilience.
  • Achieving Personal Work Goals and Objectives.
  • Customer orientation.
  • Team player.
  • Excellence orientation.
  • Responsibility and Accountability.
  • Innovative. 
  • Learning and Researching.