Data Engineer

The Data Engineer is responsible for designing, developing, and maintaining scalable and automated data pipelines based on solutions recommended by the Data Engineering Lead. This role ensures data accuracy, system efficiency, and alignment with data governance, architecture standards, and business requirements.

Key Responsibilities

• Design, develop, and maintain automated data pipelines using official ETL tools and cloud platforms (e.g., AWS), based on solutions provided by the Data Engineering Lead.

• Create and maintain documentation such as Source-to-Target mappings, technical design specifications, and pipeline architecture diagrams.

• Ensure high data quality and integrity by implementing monitoring, validation, and error-handling processes throughout the pipeline lifecycle.

• Collaborate with stakeholders to understand source systems and business rules; translate them into reusable, scalable technical solutions.

• Perform unit, integration, and system testing of data flows; troubleshoot, resolve defects, and document outcomes.

• Optimize pipeline performance, environment setup, and data flow efficiency, leveraging best practices in data integration and DevSecOps.

• Provide technical expertise during design, development, deployment, and post-production phases of the pipeline lifecycle.

• Contribute to data governance initiatives by aligning with internal standards, architectural practices, and compliance guidelines.

• Estimate level of effort for new initiatives and changes; support project planning and success tracking.

• Train end users, conduct impact analysis, and support the business in leveraging data assets effectively.