Data Engineer
Job Description:
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Google ‘big data’ technologies.
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- Work with stakeholders including the BSA, Report developers, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Additional responsibilities include troubleshooting, maintenance, and optimization or enhancement of existing processes.
- Partner with engineering leads and architects to define & coordinate technical design.
- Design and code reviews to ensure standards and quality level for the build
- Performance tuning of data pipelines jobs to meet SLA
- Prepare technical documentations on the deliverables
- Identify, define and implement best practices for process improvements for SDLC management.
- Overall 8+ years’ experience in a professional software development in a data centric environment.
- Must have 4 to 6 years of experience using SQL, Python and big data infrastructure such as Hadoop Eco System, HDFS, Hive
- Must be hands-on and have working experience in SQL, Python and PySpark
- Must have 2+ years of experience with GCP platform (Google Bigquery, Dataflow, Dataproc).
- Must be well versed with Data Modeling techniques like Star & snow flake schema, how to bring efficiency in data modeling for efficient querying data for analysis, and
- Working experience with real-time & stream processing systems like kafka, pub sub and NoSQL & indexing technologies.
- Working experience in using ETL tool like Informatica is a big plus.
- Hands-on Knowledge of Configuration Management with tools like Ansible/Chef/Puppet
- Strong design skills with a proven track record of success on large/highly complex projects preferably in the area of Enterprise Apps and Integration.
Please send your most updated CV in MS WORD FORMAT to anu@quinnox-solutions.com
Kindly indicate your availability, current and expected remuneration package.
We regret that only shortlisted candidates will be notified.
Quinnox Solutions Pte Ltd (License Number: 06C3126)
Registered EA Personnel (Reg. No.:R1441433)
Required Skills:
Hive Ansible Data Modeling Best Practices ETL Performance Tuning Architects Data Structures Chef NoSQL Resume SDLC Hadoop Pipelines Big Data Structures Developers Deliverables Reviews Engineer Analysis Optimization Infrastructure Troubleshooting Integration Databases Software Development Python Software SQL Design Engineering Maintenance Management