Data Engineer

Key Responsibilities

1. Data Pipeline Development & Maintenance

• Design, develop, and maintain ETL/ELT pipelines across multiple data sources (internal/external).
• Optimize data workflows for performance, reliability, and scalability.
• Monitor daily batch processes and resolve data loading or quality issues in a timely manner.

2. Data Architecture & Modeling

• Develop and maintain data models (conceptual, logical, physical) supporting analytics and reporting needs.
• Ensure data structures follow best practices in performance, storage, and governance.
• Work with Data Architects to improve platform architecture and data standards.

3. Data Quality & Governance

• Build data validation rules, quality checks, and automated alerting.
• Ensure data accuracy, completeness, and consistency across pipelines.
• Implement data governance practices including documentation, lineage, and metadata management.

4. Collaboration with Stakeholders

• Work closely with Data Analysts, BI Developers, Product Owners, and business teams to understand data requirements.
• Support analytics teams with clean, structured datasets for dashboards and modelling.
• Collaborate with IT/Infrastructure on data platform performance and security.

5. Platform & Technology Operations

• Manage and optimize data storage systems (Data Lake, Data Warehouse).
• Leverage cloud platforms (Azure, AWS, GCP) or on-prem solutions as applicable.
• Support CI/CD processes for data pipelines and environment management.

Qualifications

• Bachelors degree in Computer Science, Engineering, Information Systems, or related field.
• 2–5 years experience in Data Engineering or related roles.
• Proficiency in SQL and data modelling.
• Experience with ETL/ELT tools (e.g., Databricks, ADF, SSIS).
• Programming skills: Python, PySpark or similar.
• Familiarity with cloud platforms preferred Azure
• Knowledge of data warehouse/lakehouse concepts (Delta Lake, Snowflake).
• Experience with version control (Git) and CI/CD pipelines.