Menu Close

Lead Data Engineer (JHB)

IT – Software Development
Johannesburg – Gauteng

BE responsible for the installation, configuration, upgrade, administration, monitoring and maintenance of complex database environments to enable database strategies to support organisational needs as the next Lead Data Engineer sought by a large Retail and Consumer Finance Concern in Joburg. Working closely with Business Intelligence Analysts, Data Analysts and Data Scientists, you will help to implement robust & trustworthy data to support high performing ML algorithms, predictive models and support real-time data visualisation requirements across the organisation. The successful incumbent must possess a 3-Year IT related Degree, 5-10 years’ experience in Solution Architecture, Data Architecture, Business Intelligence including leading in large-scale Digital, Data and analytics and/or BI projects, Base SAS programming and SAS Data Studio & Retail SAP architecture, implementation and operations.
Execute on the Data Engineering strategy, policy and processes –
  • Determine processes to ensure execution of relevant data application requirements for various business needs.
  • Implement controls to ensure effective configuration and upgrade requirements for data requirements.
  • Implement processes that enable the management of the administration tasks required on all Data Engineering projects or initiatives.
  • Implement controls that allow the monitoring and maintenance of complex database environments and projects.
  • Participate in the design and creation of complex databases and database solutions.
  • Implement measures to ensure quality control and auditing of databases to ensure accurate and appropriate use of data.
  • Contribute to discussions that enable enhancement of database strategies.
  • Work directly with team members and business users to resolve data conflicts and inappropriate data usage.
  • Establish physical database parameters and communicate to relevant stakeholders.
  • Code database descriptions and specify identifiers of database-to-database management system or direct others in coding database descriptions.
  • Utilise development guidelines and standards to be used in application development and integration.
  • Deploy & maintain ML models as APIs for use in live environments.
  • Build high quality data assets and high-performance data services leveraging off on premise and cloud-based infrastructures.
  • Employ a variety of programming languages & tools to develop and manage data pipelines across various systems.
  • Develop data structures & processes that facilitate Data Modelling, Mining and Analytics – within an automated production environment.
  • Provide clean, usable data by employing & entrenching strong principles of data governance including but not limited to data integrity, reliability, efficiency, quality and stability.
  • Enhance and maintain existing ETL frameworks in line with agreed design patters and internal governance standards to improve our EDW product offering and to remain scalable.
  • Create robust and trustworthy ETL data pipelines.
  • Implement monitoring processes to alert the business should there be a break in the data pipeline.
  • Act with urgency to resolve the break in the data process.
  • Prepare and present potential technical solutions and innovations and advise the business on the technical and business value of the proposition with appropriate trade-offs.
Ensure that portfolio and management reporting is produced in a timely manner and aligned to the agreed communications plan –
  • Ensure end to end data processes enable the accurate view of data to end business user.
  • Filter through extensive business data sources to ensure data is sourced from most accurate source and remodelled into a set of data that is understandable to the end user.
  • Data structures are understood and deliver to end user requirements within agreed time frames.
  • Ensure that the same data sources are used to result in consistent data outputs.
  • Sign off data design and oversee the implementation of data requirement/project.
  • Implement controls to ensure that management information and management reporting is produced in a timely manner and aligned to the agreed communications plan for all relevant stakeholders.
  • Introduce processes to ensure that reporting and management information is delivered to business in agreed timeframes.
  • Manage the effective reporting and analysis requirements from the Data Engineering team.
  • Ensure monitoring of data deliverables to ensure alignment to business SLA.
  • Translate meta data into explanatory report and visuals for easy understanding to end user.
  • Ensure adherence to communication plan including call statuses.
  • Coordinate and ensure that all Data Architecture projects are designed, scoped and delivered to meet the needs of the business.
  • Design solutions for new, changed or upgraded requirements.
  • Project manage solution architecture implementation and modification activities.
  • Oversee team to ensure solution design, analysis, configuration and development tasks are executed.
Create end user forums that allow collaboration and transfer of knowledge on appropriate data sources and usage of various databases –
  • Create stakeholder understanding of available self-service data analytics.
  • Communication to stakeholders to keep them abreast of current developments within the function and to manage expectations.
  • Facilitate an understanding of data sources to ensure governance, procedures and standards are upheld.
  • Utilise specialist knowledge to explain the data and transfer the understanding to business end user.
  • Establish and share data dictionary to business stakeholders.
  • Engage and collaborate with business and IT technical functions to determine avenues for improvement in Data Engineering disciplines.
  • Conduct training and upskilling on new reports and or self-service analytics platforms to relevant stakeholders.
Manage customer centricity within area of responsibility –
  • Manage the implementation of Art of Service plans and campaigns in area of responsibility.
  • Incorporate best practice customer centric standards in the creation of policies and procedures within area of responsibility.
  • Continually measure, monitor, analyse and report on the levels of customer satisfaction for area of responsibility.
  • Develop and implement Service Level Agreements (SLA) in order to manage customers’ expectations.
  • Align current and future business processes and practices to customer requirements and legislation.
  • Develop business solutions that are aligned to customer’s value chain and needs.
  • Provide leadership in customer centricity by communicating the intent and strategy for optimisation of customer satisfaction while acting as the champion through example.
  • Recognise and celebrate customer centric behaviour within others.
  • Manage colleagues and customers’ expectations and communicate appropriately within the parameters of agreed SLAs.
  • Act in a customer centric manner that is in line with the service code and core ideology in order to meet and exceed the requirements of internal and external customers.
  • Manage the resolution of all customer complaints within the agreed standards.
Team HR Management –
  • Effectively manage self and team to deliver on agreed individual and team objectives.
  • Conduct monthly one-on-one sessions with team members.
  • Coach and mentor team members.
  • Create a culture of learning and empowerment by effectively managing and providing opportunities for training and development through the development of an Individual Development Plans (IDP).
  • Ensure that the team receives the correct training and development of skills that will enable them to perform optimally.
  • Create and manage effective performance contracts with the team that are align to the Group’s strategic objectives.
  • Create succession plan within team ensure that medium- and long-term talent requirements are catered for.
  • Drive compliance to Employment Equity legislation within the team.
  • Enhance staff retention and identify trends to reduce staff turnover within team.
  • Demonstrate abilities to anticipate and manage change.
  • Ensure that team members are customer focused and provide guidance, support or advice in situations where required.
  • Keep abreast of all Human Resource related policies and procedures.
  • Work collaboratively with cross-functional teams to design, construct, implement & maintain robust scalable data management systems and processes.
Qualifications –
  • 3-Year IT related Degree.
  • Post graduate qualification (advantageous).
Experience/Skills –
  • 5-10 Years’ experience in Solution Architecture, Data Architecture, Business Intelligence. Leading in large-scale Digital, Data and analytics and/or Business Intelligence projects.
  • Base SAS programming and SAS Data Studio.
  • Agile Methodologies experience.
  • In-depth knowledge and experience of Retail SAP architecture, implementation and operations.
  • Strong networks and knowledge of emerging trends across Data/Analytics (Big Data, Machine Learning, Deep Learning, AI).
  • Expert knowledge of complex IT environments (technical, security, business).
  • Knowledge of technology and SDLC and Agile methodology.
  • Knowledge of retail industry data models.
  • Advanced knowledge of compliance and IT governance.
  • General SAS experience
  • Leading and Supervising.
  • Project Management.
  • Conflict Management.
  • Stakeholder Management.
  • Writing and Reporting.
  • Applying expertise and knowledge.
  • Financial acumen.
  • Data Architecture, Data Modelling and Data Pipelining.
  • Solutions Architecture.
  • Deciding and initiating action.
  • Working with people.
  • Presenting and communicating information.
  • Analysing.
  • Learning and researching.
  • Planning and organising. 
  • Delivering results and meeting customer expectations.