Hong Kong, Hong Kong SAR, Hong Kong

Data Engineer

 Job Description:

About the role:

  • You are pioneering and innovative and want to be part of the cutting-edge and disruptive crypto-currency world
  • You are eager to learn new knowledge in both financial and technical fields
  • You thrive in a non-hierarchical organization with a casual working environment
  • You enjoy solving complex distributed systems challenges and optimizing streaming data pipelines
  • You value comprehensive documentation and collaborative problem-solving

As a Data Engineer you will:

  • Build real-time data pipelines using Apache Flink (PyFlink) to process high-volume logs through EC2 Vector MSK PyFlink S3 ClickHouse architecture
  • Design stream processing systems with watermark strategies, window operations, exactly-once semantics, and state management for critical data
  • Deploy and manage AWS infrastructure including Managed Flink, S3 data lakes, MSK, IAM, and CloudWatch monitoring with optimized partitioning strategies
  • Optimize performance and troubleshoot SQL queries (Flink/ClickHouse), production issues, data skew, and build Grafana dashboards for pipeline monitoring
  • Implement data quality frameworks with schema evolution, validation strategies, and translate business requirements into scalable data solutions
  • Manage DevOps processes including JAR dependencies, Docker containerization, Kubernetes deployments, and comprehensive documentation

Qualifications: 

  • University degree in Computer Science, Software Engineering or related disciplines
  • Apache Flink/PyFlink with watermarks, state management, and window operations
  • Python (pandas, polars, boto3) and SQL (Flink SQL, ClickHouse)
  • Kafka/AWS MSK and message streaming concepts
  • AWS services: Managed Flink, S3, MSK, CloudWatch, IAM
  • Docker, Kubernetes, and containerization fundamentals
  • Grafana monitoring and production troubleshooting experience
  Required Skills:

Data