Lead Data Engineer - Infrastructure
About my client:
They are the definitive fashion shopping website and app, used by over 150M shoppers a year to discover and buy fashion. More than 8.5M products from over 12,000 brands and stores can be accessed through their website and app, offering shoppers convenience and unparalleled choice.
This is a great opportunity should you be looking to work in small, self-managing, autonomous teams with end-to-end responsibility for a specific customer-focused project.
About the Role:
They are looking for a Lead Data Engineer to head up our Data Engineering team in the Infrastructure tribe. You will be responsible for managing and growing the analytics pipeline that's used by every team in the company to join together all of the data.
You will help to set the structure for data events, manage the infrastructure and software that teams depend on to shuffle events into the data lake, liaise with engineers throughout the company on their needs for the pipeline, help maintain the tooling and developer experience of what you build, and manage the Data Engineering team.
You will also be responsible for helping shape and enforce their data governance policies that ensure events are built against and adhering to a defined schema, develop data models that heuristically analyze and warn against issues with data quality, and help to coordinate the rectification of any data problems through the development of monitoring and alerting that allows the engineering teams to own and self-serve their event pipelines.
- The analytics pipeline: building and maintaining the necessary infrastructure to move and process event data from where it’s created or ingested and into our data lake
- Working in Python on top of the AWS technology stack: utilising Kinesis, Firehose, Lambda, S3 and Glue to deliver events into Snowflake for reporting on by our BI teams
- Using infrastructure-as-code in Terraform to create and maintain scalable infrastructure to support our data lake pipelines
- Driving our data engineering chapter to liaise with other data engineers throughout the company to gather feedback and ideas for development of the analytics pipeline
- Cultivate a great developer experience that engineers can use both locally and in production to build and maintain events in our pipeline
- Create processes to help engineers monitor and improve data quality and integrity
- Perform analysis to troubleshoot and fix data and infrastructure related issues and provide recommendations for improving the pipeline to help avoid problems
- Investigate the modelling of data events to proactively identify data quality issues and find ways to autonomously introduce this feedback into production workflows
- Interest in data pipelines, analytics and event handling, and building the necessary infrastructure
- Experience supporting and mentoring other engineers and working directly with them to achieve career growth
- Able to work comfortably with the Python code and AWS infrastructure that powers our pipeline
- Experience of technical problem solving and troubleshooting and incident management
- Able to triage and manage the project workload for the team in a structured but flexible kanban-style process
- The ability to work in a supportive but autonomous environment: you’ll be expected to take stakeholder direction to develop well thought-out solutions but with the ability to have your own input on direction
What do they offer:
- You get 29 days' time off throughout the year to take a well earned rest, in addition to the 8 public bank holidays
- As a fashion company they give you £250 to spend on the site in Year 1, £500 in Year 2, £750 in Year 3 and £1000 from Year 4 onwards.
- Private Healthcare by Vitality.
- Conferences and events. They big on learning, so all employees are allocated an individual training & development budget of £1,000.
- Enhanced family leave entitlements for both parents and carers
- Discounted eye tests and glasses
- Team meet-ups, social events, sports and exercise events
- Cycle-to-work scheme
- Transport season ticket loans
Please forward a copy of your most recent CV to the following email address and an initial call will be scheduled.
Engineer Incident Management Fashion Budget REST Pipelines Healthcare Analysis Loans Sports Tests Analytics Mentoring Problem Solving Conferences Infrastructure Engineers Troubleshooting reporting Python Email Software Engineering Training Management