About the job Mid-level Data Engineer
Office based in Cape Town but all work is currently conducted remotely.
The successful incumbent will be responsible for sourcing and loading a wide range of data across our business so that it can be used by analysts and developers to develop data solutions to business. You need to understand and continuously seek techniques to ingest data and ensure a high degree of quality and confidence. In addition, you should be able to use the technology and seek to make use of the features therein to deliver value to the business. You will not only design and develop, but also required to coordinate with security and infrastructure as well as investigate issues, troubleshoot technical issues, and devise solutions in line with best practice. An understanding of data management solutions and a keen sense of the strategic value of information to an organization will be of importance.
DESCRIPTION OF THE POSITION:
- Design, develop and enhance the ingestion frameworks that can load data consistently with a high degree of confidence
- Load large, complex data sets and make data available for data engineers
- Source data from internal and external data sources, engaging with technical subject matter experts
- Build the infrastructure required for optimal Extraction, Transformation, and Loading (ETL) of data from a wide variety of data sources using various ‘big data’ technologies
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing models for greater scalability
- Engage with a wide range of technical stakeholders including data scientists, business analysts, other data engineers and solutions architects
- Work with data analysts and data engineers to understand the dynamic nature that is required to support the solution that needs to be built
- Follow standards and procedures for data collection, quality improvements and integration processes
KNOWLEDGE AND SKILLS:
- Strong analytical thinking and problem-solving abilities
- Ability to collect, organize, analyze, and disseminate significant amounts of information
- Strong technical and operational ability
- Deadline-driven, even in pressurized and fast-paced environments
- National Diploma in an Information Technology related discipline or Bachelor’s Degree in Computer Science, Statistics, Informatics, Information Systems or any other quantitative field (preferred)
- +-4-6 years' experience as a Data Engineer in a BI environment
- Strong data modelling / data engineering background with the ability to interpret business requirements and technical solutions to develop components of, or complete data models
- Solid background in SQL, application and information architecture and ETL principles and procedures is required
- Ability to comply to and manage data assets under a strict governance framework
- Excellent SQL skills and development using SQL and procedural extensions is required.
- Experience in ETL toolsets is required (e.g. SSIS, SAP Data Services, Informatica etc.)
- Strong Data Engineering background with a specific focus on staging high quality data
- A solid background in SQL, Information Architecture and ETL procedures
- Experience in Database technologies (e.g. SQL Server, Oracle, SAP Hana, Cloudera, Teradata or similar)
- Experience in Hadoop components (e.g. HDFS, Hive, Spark, Oozie and Impala)
- Experience with object-oriented/functional/scripting languages (e.g. Python, Unix Shell scripting, Java, Scala etc.)
- Understanding of Data warehousing (e.g. Kimball/Data Vault) and Big Data engineering principles
- Experience in agile development
**Please note : If you have not heard from us within 2 weeks, please consider your application unsuccessful.