About the job Data Engineer
About hireworks
hireworks is building a community of top talent in key international markets by unlocking unparalleled access to positions at leading U.S. based companies. As your employer, hireworks will ensure you have a seamless interview, onboarding, and employee experience - providing ongoing support and resources along the way. Established in 2023, hireworks is forging corp-to-corp relationships with leading U.S. based organizations looking to grow their teams with best-in-class talent around the world. Working with hireworks means unlocking access to a network of local peers and mentors and career opportunities through our client network.
About our Client
From strategy to execution, the company helps clients solve their most difficult challenges, delivering best-in-class data warehouses, analytics, and automation, launching amazing new products, and radically improving martech stacks to optimize every dollar spent. We work with some of the most best known brands, and they chose us because we bring a deep understanding of both business and technology. Our team helps bring focus and clarity to problems and opportunities, and we know that it's never one-size-fits-all.
It is also the creator of the leading codeless data platform powered by AI. Automated analysis, insights, anomaly detection, and predictions, all in one easy-to-use platform that will speed up every aspect of your business. From enriching data in your data warehouse to generating beautiful dashboards, our Insights Platform will help you leave spreadsheets and complicated markup languages behind forever.
About the Position
In this role, you will be a core contributor to an ongoing project, focusing on the enhancement and optimization of existing data processing pipelines and engineering solutions within the Microsoft Azure environment. You will leverage modern programming languages like Scala and technologies like Spark to process, transform, and manage our clients' data. You will work extensively within an established framework of Azure services, particularly Azure Synapse and Fabric, to ensure our data analyst clients can reliably query and maximize the value of their data.
What You’ll Do
Maintenance and further development of high-quality, resilient, and scalable data processing pipelines using Spark and Scala for loading, storing, and moving data within the current architecture
Manage, implement, and optimize data transformation logic and data structures (including tables, views, and efficient SQL on MSSQL/Synapse) within existing production environments
Develop and maintain cloud pipelines using methods like dbt or Materialized Lake Views
Review and interpret ongoing business requirements for reporting
Research data sources and field values
Build and maintain queries and views
Analyze existing queries and data processing code for performance improvements
Suggest new queries
Develop procedures and scripts for data migration
Provide timely scheduled management reporting
Investigate exceptions with regard to asset movements
About You:
At least 4 years of experience working as a Data Engineer with a strong programming background
Comfortable with Spark
Strong professional experience with Scala or Java
Experience with the Microsoft Azure ecosystem, specifically including Azure Synapse, Fabric, and MSSQL
Comfortable orchestration concepts like working with dbt or with Airflow, Kubernetes, Docker
Familiarity with data analytics and reporting from a business standpoint and comfortable setting up tables and views for PowerBi querying
Self motivated and with solid time management skills.
Team player who enjoys working with other developers, data analysts, data scientists, and business users.
Excellent communication skills (English).
A sense of humor.
Benefits:
hireworks is cultivating a growing community of top talent across Colombia. In addition to unlocking access to positions at top tier U.S. based companies, we offer a variety of benefits to enhance your experience:
Competitive Pay - compensation that reflects your experience and accomplishments
Remote Flexibility - work from anywhere in Colombia
Paid Time Off - ample vacation days to rest and recharge
Public Holidays - all Colombian federal holidays are fully paid days off