Data Engineer (Lead / Architect)

Job Description

• Contribute using our standard tech stack, which is Python, Linux (via WSL), PostgreSQL, SQL Server, Databricks, and spans both Azure and AWS cloud infrastructure.
  
• Participate in the design and implementation of advanced mechanisms for data ingestion and transformation, and mass parallel orchestration of network IO.
  
• Maintaining Infrastructure as Code and working with infrastructure teams to develop valuable new features for Client's Engineering Platform.
  
• Model data for a variety of purposes (operational and analytical) in collaboration with your team, other teams, and analytical data customers.
  
• Contribute to CLIENT's data platform in a way that promotes the development of a data mesh architecture for the entire enterprise.
  
• Work on indicators of data quality across our estate and improve the quality of our data along key metrics.
  
• Evaluate the quality (completeness and accuracy) of potential new data sources for specific applications.
  
• Contribute to quality automation in our CI/CD process, Optimizing for bug-free release cadence and developer delight.
  
• Maintain good quality documentation (starting with your code) for technical and non-technical audiences.
  
• Participate in a pairing-first development team, learning skills from your colleagues and giving back to the community by imparting hard-won wisdom and skills learned from past experiences.
  
• Mentoring and coaching of less experienced team members through design and code reviews
  
• Participate in a safe and honest workplace culture by participating in open conversations about technical design, implementation and interrogating decisions when appropriate.
  
• Contribution to test automation through either enhancement to the platform or additional tests
  
• Assessment of application security to ensure it complies with CLIENT Application Security Standards
  
• Evaluation of APIs to ensure that they comply with the relevant CLIENT API Standards
  

Requirements

• Lead Level: (Exp Min 6+ Years)
  
• Architect: (Exp Min 9+ Years)
  
• Ability to articulate and advocate technology solutions effectively to technical and non-technical stakeholders while applying agile methodologies
  
• Proficiency in data interrogation and presentation of analysis results as part of team outputs
  
• Extensive experience as a senior software developer or engineer with a strong focus on Python development
  
• Strong expertise in SQL databases including PostgreSQL, SQL Server, and Databricks
  
• Comfortable working directly with databases without using Object Relational Mapping tools
  
• Experience working with data at scale and the associated technical challenges
  
• Proficiency with command line tools such as git, bash or zsh, and optionally vim
  
• An undergraduate degree or demonstrable knowledge in Computer Science, Software Engineering, Logic, Applied Statistics, Actuarial Science, Data Science, or related fields is preferred
  

Skills & Ability

• You are able to grasp, articulate, and advocate for technological solutions in both technical and non-technical settings and intelligently apply agile principles as you do so.
  
• You are able to interrogate and present data and data analysis and have a willingness to present results of analysis undertaken as a part of your team's work.
  
• You have demonstrable experience as a senior software developer or engineer with a focus on Python.
  
• SQL (Postgres, SQL Server, Databricks)
  
• Comfortable working without an ORM.
  
• Familiar with the challenges of working with data*at scale*.
  
• Prefer the command line use of git, bash/zsh, and (optionally) vim.
  
• An undergraduate degree or demonstrable knowledge and experience of concepts in Computer Science, Software Engineering, Logic, Applied Statistics, Actuarial Science, Data Science, or any related discipline would be advantageous.