Senior Data Engineer (12 Month Contract) (Hybrid)
IT – Analyst, Data Management
Johannesburg – Gauteng
ENVIRONMENT:
A well-established Software Engineering & QA Consultancy based in Johannesburg is actively looking for a Senior Data Engineer. This role involves the crucial tasks of developing and maintaining SQL Databases. To excel in this position, the ideal candidate should possess a B.Sc. degree in electronic and/or Computer Engineering or a related field. Furthermore, they must have a minimum of 5 years’ experience in the data engineering domain, with a strong command of SQL, proficiency in Python (preferably with pyspark experience), and an additional advantage would be familiarity with Data Warehousing, particularly Azure Synapse.
DUTIES:
- Development and maintenance of SQL Databases
- Development of data engineering and data analysis pipelines and solutions in the cloud (Azure)
- Ensure DevOps compliance for all data pipelines and workloads.
- Ensure development and maintain Power BI reports.
- Documentation of pipelines and databases.
- Troubleshooting and support to the Technical Operations team when dealing with production issues.
- Contribute to application and systems design in collaboration with the Software Architects.
- Provide mentorship and support to junior data engineers.
REQUIREMENTS:
Qualifications required:
- B.Sc. (Electronic and/or Computer Engineering) or similar qualification.
Experience required:
- Minimum of 5 years’ experience working within a data engineering environment.
Required Skill set:
- Strong SQL skills.
- Python and preferably pyspark experience.
- Data Lake experience (Azure Data Lake gen 2 experience preferred).
- NoSQL database experience (e.g., Cosmos DB, Couch Base, MongoBD, etc.).
- Azure Data Factory (ADF) experience or similar.
- Debugging and troubleshooting skills.
- Experience in writing technical documentation (architectural diagrams, release notes, etc.).
Additional Nice-to-have Skills:
- Data Warehouse experience – especially Azure Synapse.
- Databricks or Spark experience, with Azure Databricks being preferred.
- Data Science and AI/ML experience would be beneficial.
ATTRIBUTES:
- Good interpersonal and communication skills.