Data Engineer
Bangalore
Overview:
We are hiring a Lead Engineer to lead the migration of data pipelines from Hadoop to BigQuery. The ideal candidate should have strong experience in Google Cloud Platform (GCP), specifically with BigQuery and Dataproc, as well as proficiency in Linux/Unix environments.Additionally, the candidate should possess expertise in either PySpark, Scala, or Java, and have hands-on experience with Apache Airflow and unit testing. Secondary skills in Shell Scripting, Kafka, and GCP Looker are highly beneficial.
Responsibilities:
Mandatory Skills:
- GCP Data Engineer Certification
- Experience in pipeline migration from Hadoop to BigQuery.
- Proficiency with Google Cloud Platform (GCP), BigQuery and Dataproc.
- Hands on experience in any Query Language (SQL/HQL)
- Strong skills in Linux/Unix environments.
- Knowledge of PySpark, Scala or Java.
- Strong understanding of unit testing principles.
Optional Skills:
- Proficiency in Shell Scripting.
- Experience with Apache Airflow for workflow management.
- Familiarity with Kafka.
- Knowledge of GCP Looker.
Requirements:
- Bachelor’s degree (in any field).
- MSc/BE/Masters with specialization in IT/Computer Science is desirable.
- 4-7 years of work experience.
- Adapt/ Experience working in multi-channel delivery projects is desirable.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration abilities.
- 4+ years of Experience in Software Development.
AmazonBangalore
supply chain defects and develops root cause fixes to improve free cash flow and (c) supports operational process for WW Retail teams.
Our team of high caliber software developers, applied scientists, data engineers, product managers and Business Intelligence...
GenpactBangalore
digital operations services, and expertise in data, technology, and AI.
Inviting applications for the role of SeniorPrincipal Consultant- Data Engineer!
Responsibilities
• Operate ongoing business relationship management sessions to review operational...
AccentureBangalore
Project Role : Data Engineer
Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load...