GCP Data Engineer
Infosys
Infosys
Join Infosys as a Lead Consultant in our Data & Analytics Unit, focusing on cutting-edge data engineering and analytics initiatives for global clients. You'll work with cloud-native data platforms, modern analytics ecosystems, and AI-driven solutions as part of large-scale enterprise data modernization programs. This role offers continuous learning, certifications, and career growth opportunities within our dynamic data and analytics practice.
Design and develop scalable GCP data engineering solutions and cloud-native data platforms. Build robust batch and real-time data pipelines using BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Python, PySpark, and Databricks. Optimize Databricks, Delta Lake, Spark, and Structured Streaming for large-scale processing, ensuring high-throughput streaming with advanced data handling. Develop complex SQL transformations, data models, and analytics-ready BigQuery datasets, orchestrating pipelines with Cloud Composer/Airflow and implementing CI/CD and Terraform.
Ensure data quality, governance, security, monitoring, and production support. Collaborate with architects, stakeholders, and cross-functional teams in an Agile environment. Troubleshoot and optimize end-to-end GCP, Databricks, and streaming data pipelines.
Requires a minimum of 9-12 years of experience in GCP Data Engineering, with strong hands-on expertise in scalable cloud data platforms. Proficiency in Google Cloud Storage, BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Composer, and other GCP services is essential.
Experience with batch and real-time streaming technologies like Pub/Sub, Dataflow/Apache Beam, Spark Structured Streaming, or Kafka is crucial. Demonstrated coding proficiency in Python, PySpark, and SQL, including debugging and performance optimization, is required.
Key technical skills include Databricks with Delta Lake and Structured Streaming, handling TB/PB-scale datasets, data modeling for analytics, and implementing CI/CD with Terraform. Strong troubleshooting, observability, production support, and cloud cost-optimization skills are vital. Excellent client communication, stakeholder management, and problem-solving abilities are expected.
Infosys
Information Technology & Services