Job Openings Senior Data Engineer

About the job Senior Data Engineer

Key Responsibilities:

  • Design, develop and implement enterprise-scale data pipelines, data ingestion and integration workflows, data transformation processes and data storage solutions across cloud and enterprise data platforms.
  • Design and implement scalable, resilient and secure data architectures, including data lakes, data warehouses, streaming platforms and time-series data platforms.
  • Integrate structured and unstructured data from multiple sources, including databases, APIs, files, streaming sources, Internet of Things (IoT) and Operational Technology (OT) systems.
  • Develop and maintain data engineering solutions using SQL, Python or equivalent programming languages, ETL frameworks, data orchestration and distributed data processing technologies.
  • Implement data modelling, version control, automated testing and Continuous Integration/Continuous Deployment (CI/CD) practices to ensure reliable data pipeline delivery.
  • Establish and maintain data quality, metadata management, data lineage, data cataloguing, access control and data security standards.
  • Implement data monitoring, governance and lifecycle management practices to ensure the reliability, integrity and security of enterprise data platforms.
  • Optimise data processing performance, troubleshoot complex data engineering issues and provide technical guidance to data engineering teams.
  • Collaborate with architects, cloud engineers, software engineers, cybersecurity teams and business stakeholders to deliver effective data engineering solutions.

Requirements:

  • At least THREE (3) years of experience as a Data Engineer or in a similar data engineering role, including the design, development and implementation of enterprise-scale data pipelines, data ingestion and integration workflows, data transformation, data storage and data processing solutions on cloud and/or enterprise data platforms.
  • Experience in designing and implementing scalable, resilient and secure data architectures, including data lakes, data warehouses, streaming platforms and/or time-series data platforms, and integrating structured and unstructured data from databases, APIs, files, streaming sources, Internet of Things (IoT) and/or Operational Technology (OT) systems.
  • Experience with relevant data engineering technologies and practices, including SQL, Python or equivalent programming languages, Extract, Transform and Load (ETL) frameworks, data orchestration, data modelling, distributed data processing, version control, automated testing and Continuous Integration/Continuous Deployment (CI/CD).
  • Experience in implementing and governing data quality, metadata management, data lineage, data cataloguing, access control, data security, monitoring and data lifecycle management, including establishing appropriate engineering standards and controls for enterprise data platforms.
  • Strong capabilities in technical solution design, performance optimisation, troubleshooting complex data-processing issues and providing technical guidance to data engineering teams, including working closely with architects, cloud engineers, software engineers, cybersecurity teams and business stakeholders.