Job Description:
A leading Data & Analytics consultancy is looking for an AI Engineer for their practice in Sydney or melbourne
A brief job descroption is below with a more detailed one after that
Required
- Gen AI experience
- Langchain & Langgraph
- MLops
- Strong proficiency in Python
- Hands-on experience with Databricks is a nice to have
- Solid understanding of machine learning concepts
- Experience with Apache Spark
- Familiarity with SQL
- Experience building and deploying ML models in production
- Exposure to LLMs, embeddings, vector search, and RAG architectures
Preferred
- Experience with Azure
- Experience with Mosaic AI, Databricks Feature Store, and Unity Catalog
- Background in data engineering or analytics engineering
DETAILED SPECIFICATION
Role Overview
The AI Engineer is responsible for designing, building, and deploying AI-powered solutions on the Databricks Lakehouse platform. This role bridges data engineering, machine learning engineering, and applied AI, enabling scalable analytics, predictive models, and generative AI use cases across the enterprise.
The ideal candidate has hands-on experience with Databricks, strong Python and Spark skills, and practical exposure to machine learning and GenAI workloads in production environments.
Key Responsibilities
AI & Machine Learning Development
- Design, build, and deploy machine learning and AI models using Databricks Machine Learning and Mosaic AI
- Develop end-to-end ML pipelines including data preparation, feature engineering, training, evaluation, and inference
- Implement LLM-based solutions (e.g. RAG, fine-tuning, prompt engineering) using Databricks and open-source models
- Integrate ML models into downstream applications via batch and real-time inference
Data Engineering & Lakehouse Enablement
- Build scalable data pipelines using Apache Spark, Delta Lake, and Databricks Workflows
- Collaborate with data engineers to ensure high-quality, governed feature and training datasets
- Leverage Unity Catalog for secure, governed access to data and AI assets
MLOps & Productionisation
- Implement MLOps best practices using MLflow for experiment tracking, model registry, and lifecycle management
- Automate model deployment, monitoring, and retraining pipelines
- Monitor model performance, data drift, and operational metrics in production
Collaboration & Stakeholder Engagement
- Work closely with data scientists, engineers, architects, and business stakeholders
- Translate business problems into AI-driven solutions with measurable outcomes
- Contribute to AI standards, reusable frameworks, and best practices within the organisation
Required Skills & Experience
Core Technical Skills
- Strong proficiency in Python for data science and AI workloads
- Hands-on experience with Databricks (notebooks, jobs, MLflow, Delta Lake)
- Solid understanding of machine learning concepts (supervised/unsupervised learning, model evaluation)
- Experience with Apache Spark for large-scale data processing
- Familiarity with SQL for data exploration and transformation
AI & GenAI Experience
- Experience building and deploying ML models in production
- Exposure to LLMs, embeddings, vector search, and RAG architectures
- Experience using frameworks such as Hugging Face, LangChain, or similar (preferred)
Cloud & Platform
- Experience on Azure (preferred), AWS, or GCP
- Understanding of cloud security, identity, and cost management considerations for AI workloads
Preferred Qualifications
- Experience with Mosaic AI, Databricks Feature Store, and Unity Catalog
- Knowledge of CI/CD for ML pipelines
- Background in data engineering or analytics engineering
- Experience working in regulated or enterprise environments (e.g. Energy, Financial Services, Property)
What Success Looks Like
- AI solutions are production-ready, scalable, and governed
- Models deliver clear business value and are trusted by stakeholders
- AI workloads are efficiently integrated into the Databricks Lakehouse
- Best practices for MLOps, security, and cost optimisation are consistently applied
Why Join
- Work on cutting-edge AI and GenAI solutions using the Databricks Lakehouse
- Influence how AI is operationalised across the organisation
- Collaborate with experienced data and analytics professionals
- Continuous learning and exposure to emerging AI capabilities