Azure Data Engineer

Responsibilities

• Design and implement data pipelines using Azure Data Factory (ADF), Azure Databricks, and Azure Synapse Analytics.

• Develop and optimize ETL/ELT processes for structured and unstructured data sources.

• Ensure efficient data ingestion, transformation, and storage within Azure environments.

• Design and manage data lakes, data warehouses, and relational databases (Azure SQL, Azure Synapse, Azure Cosmos DB).

• Optimize database performance, indexing, and partitioning strategies.

• Implement data security, access control, and encryption best practices.

• Deploy and manage Azure Data Services, including Azure Blob Storage, Azure Data Lake, and Azure Functions.

• Monitor and troubleshoot data pipeline performance issues.

• Ensure data quality, integrity, and governance using Azure Purview and data cataloging tools.

• Adhere to compliance requirements and enforce data privacy best practices.

• Implement logging, monitoring, and alerting solutions for data workflows.

• Work closely with data scientists, analysts, and business teams to understand data requirements.

• Document data solutions, architecture, and best practices for future reference.

• Provide mentorship and technical guidance to junior engineers.

Qualifications

Must have

• Azure Data Engineer Associate or equivalent certification on AWS/Google

• Either Infrastructure & DevOps or Data Pipelines experience

Nice to have

• Snowflake architecture certification or practical experience

• Databricks certification or practical experience