Advertise your job vacancies
Subscribe to email job alerts
Recruitment news
LAC deems subjective assessments 'unfair' in retrenchment process
Chloë Loubser, Mbali Mnyandu, and Josua Loots
Senior Data Engineer
Remuneration: | cost-to-company |
Location: | Cape Town |
Job level: | Senior |
Type: | Permanent |
Company: | THE SKILLS MINE (PTY) LTD |
Requirements
- Five-plus years of experience as a data engineer or relevant experience
- Proven experience in designing and implementing data pipelines
- Experience with end-to-end testing of analytics pipelines
- Expertise in managing and optimising Elasticsearch clusters, including performance tuning and scalability
- Strong proficiency with data extraction, transformation, and loading (ETL) processes
- Familiarity with data modelling and schema design for efficient data storage and retrieval
- Good programming and scripting skills using languages like Python, Scala, or Java
- Knowledge of DevOps and automation practices related to data engineering
- Experience with data engineering in an agile / scrum environment
- Familiarity with ksqlDB / Kafka or other stream processing frameworks
- Familiarity with data lakes and querying them
- Experience with integrating machine learning models into data pipelines
- Kafka / ksqlDB
- Python
- Redis
- Elasticsearch, cluster management and optimisation
- AWS S3
- PostgreSQL
- AWS
- Designing, implementing, and maintaining robust data pipelines, ensuring the efficient and reliable flow of data across systems
- Developing and maintaining Elasticsearch clusters, fine-tuning them for high performance and scalability
- Collaborating with cross-functional teams to extract, transform, and load (ETL) data into Elasticsearch for advanced analytics and search capabilities
- Troubleshooting data pipeline and Elasticsearch issues, ensuring the integrity and availability of data for analytics and reporting
- Participating in the design and development of data models and schemas to support business requirements
- Continuously monitoring and optimising data pipeline and Elasticsearch performance to meet growing data demands
- Collaborating with data scientists and analysts to enable efficient data access and query performance
- Contributing to the evaluation and implementation of new technologies and tools that enhance data engineering capabilities
Posted on 18 Sep 12:43, Closing date 18 Oct
Connecting highly talented individuals with organisations that recognise the power of talent to optimise business effectiveness.
Recent jobs by The Skills Mine | City | Date posted | |
Full-Stack Software Engineer | Cape Town | 20 Sep 15:03 | |
Senior Data Engineer | Cape Town | 18 Sep 12:43 | |
Coordinator - Marketing and Events | Johannesburg | 16 Sep 09:25 | |
Senior Back-end Engineer | Cape Town | 13 Sep 15:50 | |
Regional Sales Manager | Durban | 10 Sep 16:21 | |
Customer Success Manager | Durban | 10 Sep 13:22 | |
Web Developer and SQL Admin | Johannesburg | 10 Sep 13:20 | |
Full Stack Developer - Ruby on Rails | Johannesburg | 06 Sep 15:29 | |
Regional Product Manager - (6 Months FTC) | Johannesburg | 04 Sep 13:56 | |
HR and Talent Acquisition Business Partner | Johannesburg | 04 Sep 09:22 | |
More jobs... |
Infrastructure Support Engineer
Cape Town
New work with a leading provider of Cloud contact centre and CRM solutions in Southern Africa known for implementing some of the most advanced technology within the...
New work with a leading provider of Cloud contact centre and CRM solutions in Southern Africa known for implementing some of the most advanced technology within the...
1 day
Full-Stack Software Engineer
Cape Town
Requirements Five-plus years of experience in software engineering or development fieldsBachelor’s degree in computer science, engineering, or a related...
Requirements Five-plus years of experience in software engineering or development fieldsBachelor’s degree in computer science, engineering, or a related...
20 Sep 2024
IT Support Engineer
Cape Town, Viking Park
The Window is based in Cape Town, we specialise in IT-managed services and are looking to expand our IT support team, we are a close-knit team with great values with a...
The Window is based in Cape Town, we specialise in IT-managed services and are looking to expand our IT support team, we are a close-knit team with great values with a...
The Window
20 Sep 2024
Senior Data Engineer
Cape Town
Requirements Five-plus years of experience as a data engineer or relevant experienceProven experience in designing and implementing data pipelinesExperience with...
Requirements Five-plus years of experience as a data engineer or relevant experienceProven experience in designing and implementing data pipelinesExperience with...
18 Sep 2024
Senior Python JavaScript AWS Software Engineer
Cape Town
Are you a Senior Python JavaScript Software Engineer looking for a new opportunity in an environment that empowers salesforce to reach every customer, through an...
Are you a Senior Python JavaScript Software Engineer looking for a new opportunity in an environment that empowers salesforce to reach every customer, through an...
16 Sep 2024
Senior Back-end Engineer
Cape Town
Requirements Over five years of experience in back-end web development.In-depth expertise in PostgreSQL and RDBMS in general.Experience working with message brokers and...
Requirements Over five years of experience in back-end web development.In-depth expertise in PostgreSQL and RDBMS in general.Experience working with message brokers and...
13 Sep 2024
Senior DevOps Engineer
Tyger Valley
New work in at an on-demand delivery and digital tech shop in SA, owned by one of our major retail groups. The current need is for a DevOps engineer with key...
New work in at an on-demand delivery and digital tech shop in SA, owned by one of our major retail groups. The current need is for a DevOps engineer with key...
11 Sep 2024
Java System Integration Engineer
Cape Town
Become a member of a team enthusiastic about the challenges posed by integrating complex systems comprising diverse components and subsystems currently in search for...
Become a member of a team enthusiastic about the challenges posed by integrating complex systems comprising diverse components and subsystems currently in search for...
6 Sep 2024
Senior DevOps Engineer
Tygervalley
New work in at an on-demand delivery and Digi tech shop in SA, owed by one of our major retail groups.
New work in at an on-demand delivery and Digi tech shop in SA, owed by one of our major retail groups.
5 Sep 2024
Data Engineer - SQL DBA
Cape Town
A Fintech department, delivering financial information to the company that is timely, accurate, reliable, and relevant data originating from a multitude of external...
A Fintech department, delivering financial information to the company that is timely, accurate, reliable, and relevant data originating from a multitude of external...
4 Sep 2024
Software Developer /Data Engineer
Cape Town, Century City
A dynamic client who leverages the power of AI to analyse and interpret data with unprecedented accuracy, speed and gives actionable improvement suggestions are looking...
A dynamic client who leverages the power of AI to analyse and interpret data with unprecedented accuracy, speed and gives actionable improvement suggestions are looking...
2 Sep 2024
Mechanical Engineer
George, R 50000 - R 60000 per month
Our seeking a highly skilled and experienced Mechanical Engineer to join our team on a world-changing mission.
Our seeking a highly skilled and experienced Mechanical Engineer to join our team on a world-changing mission.
30 Aug 2024
Data Engineer
Cape Town
A fintech department, delivering financial information to the company that is timely, accurate, reliable, and relevant data originating from a multitude of external...
A fintech department, delivering financial information to the company that is timely, accurate, reliable, and relevant data originating from a multitude of external...
21 Aug 2024
Senior Data Engineer
Cape Town
A fintech department delivering timely, accurate, reliable, and relevant financial information from various external sources is seeking a senior data engineer.
A fintech department delivering timely, accurate, reliable, and relevant financial information from various external sources is seeking a senior data engineer.
5 Aug 2024
Code 14 Truck Driver
George
Truck Driver Code EC 14 with Crane Operator Certification - George Western Cape
Truck Driver Code EC 14 with Crane Operator Certification - George Western Cape
1 day