Data Engineer (SQL / Python) – (CPT)
IT – Analyst, Data Management
Cape Town – Western Cape
ENVIRONMENT:
A fast-growing Payment Service Provider that makes online commerce and digital payment acceptance easier and more accessible across all Africa, is seeking a Data Engineer to operationalize streaming platform and add more data sources (in raw form) to the data warehouse and build out more transformations that serve the business needs (using dbt and postgres native functions). The successful candidate must have at least 6 years of experience in the data field, working primarily with SQL and Python, and at least 4 to 6 years’ experience with data streaming and pub/sub models.
DUTIES:
- Operationalize streaming platform and add more data sources (in raw form) to the company’s data warehouse.
- Build out more transformations that serve the business needs (using dbt and postgres native functions).
- Build out complex models and algorithms to improve the company’s ability to help merchants and the Company to make better decisions.
- Examples could be alerting when payment processing seems down in a certain area, or advanced fraud detection.
- Allow other products to move over to event driven architectures.
- Work alongside the data team and be a part of the full software development life cycle.
- Building out the company’s core data infrastructure, that includes queuing platforms with a pub/sub architecture with a focus on security and reliability.
- Enabling other product teams to use the infrastructure by setting up guidelines, libraries, and training.
- Monitoring and improving the data warehouse, to make sure it keeps up with scale and usage.
- Show best practices in testing, documenting and the use of version control and deployment tools.
- Automate routine tasks.
REQUIREMENTS:
- At least 4 – 6 years of experience in the data field, working primarily with SQL and Python.
- At least 3 years’ experience with data streaming and pub/sub models.
- Experience with the AWS Stack (Or any other cloud environment).
- Eager to experiment with new technologies and learn new skills.
Tech Stack:
- Data: PostgreSQL, MongoDB, MySQL, Kafka
- Infra: AWS Lambda’s, Serverless, S3, API Gateway, DBT
- Tools: Jira, Confluence, Git, Postman, Pop SQL
- Automation: Gitlab pipelines, CI/CD, Terraform
- Backend: Python, Node, PHP
- Frontend: React, Angular, TypeScript, Material
- Nice To Haves:
- BSc Computer Science, technical tertiary qualification or significant work experience.
- Have worked within the FinTech/Payments industry.
- Experience working with PostgreSQL.
- Experience with Kafka implementations.
ATTRIBUTES:
- Time- and self-management skills are vital.