Manager | GCP Data Engineer | Ahmedabad | Engineering | Data Modernization & Migration

Deloitte

8–15 yrs Ahmedabad Full Time Work from office
Deloitte logo
Posted : 1 week ago
Actively hiring

Job description

Lead the development and refinement of cloud-based data platforms as a GCP Data Engineering Manager. This role requires extensive expertise in Google Cloud Platform (GCP) data services and hands-on performance engineering for large-scale Spark and Dataproc workloads.

Leverage your experience to architect and deliver robust data solutions, driving innovation and efficiency within our data modernization and migration initiatives. This is an opportunity to shape the future of our cloud data infrastructure.

Responsibilities

Design and implement data solutions using a range of GCP services including Cloud Run, Cloud Functions (Python), Cloud SQL PostgreSQL, Pub/Sub, GCS, and BigQuery. Manage and optimize orchestration workflows with Cloud Composer/Managed Airflow, focusing on FinOps, capacity planning, and cost governance.

Provide expert technical leadership and architectural guidance to engineering teams. Drive performance engineering for Dataproc workloads by executing test waves, troubleshooting failures using Spark UI and platform diagnostics, and analyzing workload profiles for cost-performance benchmarks. Define VM sizing and recommend improvements for Big Data workloads.

Qualifications

A minimum of 8 to 15 years of dedicated experience in Data Engineering, with a significant focus on GCP platforms. Proficiency in Python, Spark, Dataproc, GCS, BigQuery, and Cloud Data Architecture is essential.

Experience with Cloud Run, Cloud Functions, Cloud SQL PostgreSQL, and Pub/Sub is required. Strong administrative and optimization skills for Cloud Composer/Airflow, including FinOps, are crucial. Candidates should possess proven abilities in Spark UI debugging, YARN diagnostics, and workload benchmarking. Excellent stakeholder management, team leadership, and solution design capabilities are expected.

Essential Skills

GCPData EngineeringPythonSparkDataprocGCSBigQueryCloud Data ArchitectureCloud RunCloud FunctionsCloud SQL PostgreSQLPub/SubCloud ComposerAirflowFinOpsSpark UI DebuggingYARN DiagnosticsWorkload BenchmarkingStakeholder ManagementTeam LeadershipSolution Design

Highlights

  • Actively hiring

More Details

RoleManager | GCP Data Engineer | Ahmedabad | Engineering | Data Modernization & Migration
DepartmentData Engineering
Employment TypeFull Time, Work from office

About the Company

Deloitte logo

Deloitte

IT Consulting

Manager | GCP Data Engineer | Ahmedabad | Engineering | Data Modernization & Migration at Deloitte | SkillMX | SkillMX