Manager | GCP Data Engineer | Ahmedabad | Engineering | Data Modernization & Migration
Deloitte
Deloitte
Lead the development and refinement of cloud-based data platforms as a GCP Data Engineering Manager. This role requires extensive expertise in Google Cloud Platform (GCP) data services and hands-on performance engineering for large-scale Spark and Dataproc workloads.
Leverage your experience to architect and deliver robust data solutions, driving innovation and efficiency within our data modernization and migration initiatives. This is an opportunity to shape the future of our cloud data infrastructure.
Design and implement data solutions using a range of GCP services including Cloud Run, Cloud Functions (Python), Cloud SQL PostgreSQL, Pub/Sub, GCS, and BigQuery. Manage and optimize orchestration workflows with Cloud Composer/Managed Airflow, focusing on FinOps, capacity planning, and cost governance.
Provide expert technical leadership and architectural guidance to engineering teams. Drive performance engineering for Dataproc workloads by executing test waves, troubleshooting failures using Spark UI and platform diagnostics, and analyzing workload profiles for cost-performance benchmarks. Define VM sizing and recommend improvements for Big Data workloads.
A minimum of 8 to 15 years of dedicated experience in Data Engineering, with a significant focus on GCP platforms. Proficiency in Python, Spark, Dataproc, GCS, BigQuery, and Cloud Data Architecture is essential.
Experience with Cloud Run, Cloud Functions, Cloud SQL PostgreSQL, and Pub/Sub is required. Strong administrative and optimization skills for Cloud Composer/Airflow, including FinOps, are crucial. Candidates should possess proven abilities in Spark UI debugging, YARN diagnostics, and workload benchmarking. Excellent stakeholder management, team leadership, and solution design capabilities are expected.
Deloitte
IT Consulting