Datafin

+ 27 (0) 21 741 0400 + 27 (0) 21 741 0400   CAREERS@DATAFIN.COM

Looking for staff? OR View Vacancies

Data Engineer (CPT Hybrid)

IT – Analyst, Data Management
Cape Town – Western Cape

ENVIRONMENT:
EXTRACT & transform massive amounts of data at scale and consolidate this data into a bigger ecosystem as the next Data Engineer sought by a growing Retail Group for its Food Applications division. You will also assist in designing and implementing scalable and robust processes to support the Data Engineering capability. The successful incumbent will possess a Bachelor’s Degree in Computer Science/Business Informatics/Mathematics or similar field with 4-5 years’ experience in a similar role including a strong understanding of data structures, algorithms, and effective software design & significant experience working with structured and unstructured data at scale and different data stores such as key-value, document, columnar, etc. as well as traditional RDBMS and data warehouses. You will need proficiency in Python, Scala, Java, C, Git, SVN, Apache Spark and AWS services such as Redshift, Glue, Lambda, EMR, S3, IAM, RDS, etc.
 
DUTIES:
  • Assist in designing and implementing scalable and robust processes for ingesting and transforming large data sets.
  • Assist in the design and implementation of data pipelines from a variety of data sources and support the maintenance thereof.
  • Ingest large, complex data sets that meet functional and non-functional requirements.
  • Enable the business to solve the problem of working with large volumes of data in diverse formats, and in doing so, enable innovative solutions.
  • Build bulk and delta data patterns for optimal extraction, transformation, and loading of data.
  • Support the organisations cloud strategy and alignment to data architecture and data governance.
  • Engineer data in the appropriate formats for downstream consumption for analytics or Enterprise Applications.
  • Assist in the development of APIs to expose the data to Enterprise Applications and 3rd party vendors.
  • Assist in identifying, designing and implementing robust process improvement activities to drive efficiency and automation for greater scalability. This includes looking at new solutions and new ways of working and being on the forefront of emerging technologies.
  • Work with various stakeholders across the organisation to understand data requirements and apply technical knowledge of data management to solve key business problems.
  • Provide support in the operational environment with all relevant support teams for data services.
  • Create and maintain functional requirements and system specifications in support of data architecture and detailed design specifications for current and future designs.
  • Support test and deployment of new services and features.
 
REQUIREMENTS:
Qualifications –
  • Bachelor’s Degree in Computer Science, Business Informatics, Mathematics, Statistics or Engineering.
 
Experience/Skills –
  • 4 – 5 Years relevant Data Engineering experience.
  • A strong understanding of data structures, algorithms, and effective software design.
  • Significant experience working with structured and unstructured data at scale and different data stores such as key-value, document, columnar, etc. as well as traditional RDBMS and data warehouses.
  • Good programming, performance tuning and troubleshooting skills using programming languages such as Python, Scala, Java and C .
  • Practical experience with Apache Spark and AWS services such as Redshift, Glue, Lambda, EMR, S3, IAM, RDS, etc.
  • Experience wrangling terabytes of big, complicated, imperfect data.
  • Designing and implementing Cloud (AWS) solutions including use of APIs available.
  • Experience with Version Control systems such as Git, SVN.
  • Knowledge of Engineering and Operational Excellence using standard methodologies. Best practices in Software Engineering, Data Management, Data Storage, Data Computing and distributed systems to solve business problems with data.
  • Knowledge and understanding of Business Process Management lifecycle which covers the design, modelling, execution, monitoring, and optimization as well as business process re-engineering.
 
Advantageous –
  • Experience with DevOps architecture, implementation and operation.
  • Some experience in applying SAFe/Scrum/Kanban methodologies.
 
ATTRIBUTES:
  • Excellent verbal and written communication skills; must work well in an Agile, collaborative team environment.
  • Good problem-solving skills.
  • The ability to exercise judgment in solving technical, operational, and organisational challenges, to identify issues proactively, to present solutions and options leading to resolution.
  • Is aware of and responsive to internal and external events and influences on the technical landscape.
  • Ability to research technology-related concepts, trends, and best practices, and apply findings.
  • Appropriately derives and organises the essence of information to draw solid conclusions.
  • Proactively approaches others to obtain missing information.
  • Demonstrates a results-oriented mindset in planning and implementing activities/projects.
  • Monitors and tracks progress to ensure delivery of all planned commitments, and keeps the appropriate people informed.
  • Prepares written reports and briefs and communicates ideas clearly.
  • Speaks fluently in team meetings when presenting information.
  • Manages existing partnerships within established agreements or contracts; negotiates adjustments when mutually beneficial to do so.
  • Genuinely cultivates personal bonds with colleagues to enhance performance throughout the organisation.
  • Adjusts to work effectively within new work structures, processes, requirements, or cultures.
  • Demonstrates resourcefulness in acquiring necessary knowledge, skills, and competencies to adapt to change.