MS Fabric Data Engineer (JHB/CPT)
IT – Analyst, Data Management
Johannesburg – Gauteng – South Africa, Cape Town – Western Cape – South Africa
ENVIRONMENT:
YOUR passion for leading teams, driving Business Intelligence & Microsoft Fabric expertise is sought to fill the role of a MS Fabric Data Engineer wanted by a provider of tailored Financial Solutions. You will lead the initial phases of a warehouse rework project and support the ongoing expansion of a newly created data warehouse within the Microsoft Fabric platform. You will also build and lead a Data team, playing a critical role in designing, implementing, and maintaining data pipelines and ensuring high-quality data movement. The ideal candidate must preferably have Azure Data Engineer Associate or Fabric Analytics Engineer Associate/Data Engineer Associate Certifications with 5+ years’ experience working with complimentary Azure Data Technologies – Azure Synapse, Azure Data Factory, Azure Databricks, Azure SQL, etc. You also need proficiency with T-SQL, ETL/ELT tools, PySpark and Spark SQL & proven experience in using Pipelines, Lake House and Warehouse capabilities of the Microsoft Fabric platform.
DUTIES:
- Maintain and update existing reports and dashboards in Power BI to reflect evolving business requirements.
- Assist with data extractions and execute defined SQL queries.
- Perform data validation and reconciliations between reporting layers.
- Work closely with the Analytics and Engineering teams to resolve data discrepancies.
- Support documentation of reports, dashboards, and data definitions.
- Participate in data testing and user feedback processes.
- Lead the design, development, and maintenance of scalable data pipelines and infrastructure within Microsoft Fabric.
- Architect and optimize the data warehouse within the Microsoft Fabric platform.
- Oversee data ingestion, transformation, and integration processes to ensure efficient data workflows.
- Implement and maintain ETL/ELT processes, data governance, and security best practices.
- Work closely with the Analyst and business stakeholders to translate requirements into technical solutions.
- Develop the data model to be implemented to achieve the business’ analytics objectives.
- Ensure data quality, reliability, and performance across all pipelines.
- Build and lead a Data team, initially managing the existing Analyst and gradually hiring additional Engineers and Analysts as needed.
- Provide mentorship, training, and career development opportunities for team members.
- Stay updated on emerging technologies in data engineering, analytics, and Microsoft Fabric advancements.
REQUIREMENTS:
Preferred Qualifications –
One of the following Microsoft Certifications:
- Azure Data Engineer Associate (DP-203)
- Fabric Analytics Engineer Associate (DP-600)
- Fabric Data Engineer Associate (DP-700)
Experience/Skills –
- 5 + Years’ experience working with complimentary Azure Data Technologies – Azure Synapse, Azure Data Factory, Azure Databricks, Azure SQL, etc.
- Designing and developing Star Schema based data models.
- Designing and implementing analytics solutions using data warehouses and data lakes.
- Hands-on experience with ETL/ELT tools, data modelling (star schema), and performance optimization.
- Understanding of data governance, security, and compliance best practices.
- Proven experience in using Pipelines, Lake House and Warehouse capabilities of the Microsoft Fabric platform.
- Strong, proven experience in developing Semantic Models (datasets) in Power BI.
- Proficiency in T-SQL in SQL Server (On-premises or Cloud) using Stored Procedures and Functions.
- Proficiency in PySpark and Spark SQL within notebooks on the Microsoft Fabric Platform or Azure Databricks for data transformation and processing.
- Experience with use of Azure DevOps and GitHub for CI/CD deployments and use of Power BI Deployment Pipelines.
Leadership & Business Skills –
- Experience leading and managing a Data team, with a track record of hiring and mentoring data professionals.
- Strong ability to translate business requirements into technical solutions.
- Excellent communication skills to engage with stakeholders across business and technical teams.
- Experience in Project Management, Agile methodologies, and process automation.
Advantageous –
- Experience with Machine Learning models or AI-driven analytics.
- Background in Financial Services, Sales Optimization, or Regulatory Reporting.
- Experience with Premium or Embedded.