Job Openings Databricks Data Warehouse Architect

About the job Databricks Data Warehouse Architect

Data Architect Databricks & AWS

About the Role

One of the largest companies in the global gaming industry is seeking a hands-on Data Architect with Data Warehouse Engineering expertise in Databricks (DBX) and AWS-native data services.

Youll lead the design and implementation of a new data warehouse instance for a major product linebuilding from the ground up, architecting scalable pipelines, optimizing lakehouse performance, and integrating with real-time and batch data sources across AWS.

The ideal candidate is passionate about data architecture, thrives in fast-paced environments, and has a proven record of setting up high-performance lakehouse platforms on Databricks, with a strong foundation in data warehousing principles.

⸻

Key Responsibilities

  • Design and deploy a new Databricks Lakehouse instance tailored to product-level data needs.
  • Architect and implement robust data ingestion pipelines using Spark (PySpark/Scala) and Delta Lake.
  • Integrate AWS-native services (S3, Glue, Athena, Redshift, Lambda) with Databricks for optimized performance and scalability.
  • Define data models, optimize query performance, and establish warehouse governance best practices.
  • Collaborate cross-functionally with product teams, data scientists, and DevOps to streamline workflows.
  • Maintain CI/CD pipelines for data workflows using GitOps and Infrastructure-as-Code practices (e.g., DBX).
  • Monitor data jobs and resolve performance bottlenecks or failures across environments.

⸻

Required Skills & Experience

  • Databricks / Lakehouse Architecture
  • End-to-end setup of Databricks workspaces and Unity Catalog.
  • Expertise in Delta Lake internals, file compaction, and schema enforcement.
  • Advanced PySpark/SQL skills for ETL and transformations.
  • AWS Native Integration
  • Deep experience with AWS Glue, S3, Redshift Spectrum, Lambda, and Athena.
  • Knowledge of IAM and VPC configuration for secure cloud integrations.
  • Data Warehousing & Modeling
  • Strong grasp of modern dimensional modeling (star/snowflake schemas).
  • Experience with lakehouse design patterns for mixed workloads.
  • Automation & DevOps
  • Familiarity with CI/CD for data engineering using DBX, Terraform, GitHub Actions, or Azure DevOps.
  • Proficient in monitoring tools like CloudWatch, Datadog, or New Relic for data pipelines.

⸻

Bonus / Nice to Have

  • Experience supporting gaming or real-time analytics workloads.
  • Familiarity with Airflow, Kafka, or EventBridge.
  • Exposure to data privacy and compliance frameworks (GDPR, CCPA).

⸻

Other Details

  • Location: Latin America (LATAM) or USA Remote
  • Contract Length: 1+ year
  • Client: Global gaming company

⸻

Unique Skill Keywords

  • Databricks Workspace Orchestration
  • Data Warehouse Design
  • Data Modeling
  • Delta Lake Optimization
  • Unity Catalog Implementation
  • PySpark
  • AWS Glue Job Orchestration
  • Airflow
  • S3-to-DBX Streaming Pipelines
  • Lakehouse Schema Design
  • Cross-region Data Mesh (AWS + DBX)
  • GitOps-Driven ETL
  • Product-Centric Data Layering
  • Multitenant DBX Setup
  • Redshift + Databricks Dual-query Optimization