GCP Data Engineer

Infosys

7–9 yrs Noida Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Infosys is seeking a Senior Consultant for its Data & Analytics Unit, focusing on cutting-edge data engineering initiatives. This role involves working with global clients on cloud-native data platforms and modern analytics ecosystems. You will contribute to large-scale enterprise data modernization programs, leveraging AI-driven solutions and benefiting from continuous learning and career growth opportunities within Infosys' data and analytics practice.

Responsibilities

Design and develop scalable GCP Data Engineering solutions and cloud-native data platforms. Build robust batch and real-time data pipelines using tools like BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Python, PySpark, and Databricks. Optimize Databricks, Delta Lake, Spark, and Structured Streaming for large-scale data processing. Implement high-throughput streaming solutions with effective handling of late-arriving data and duplicates. Manage high-volume TB/PB-scale data with a focus on performance, scalability, reliability, and cost optimization. Develop complex SQL transformations and data models for analytics-ready datasets. Orchestrate pipelines using Cloud Composer/Airflow and implement CI/CD with Terraform for Infrastructure as Code. Ensure data quality, governance, security, monitoring, alerting, and production support. Collaborate effectively with stakeholders and cross-functional teams in an Agile environment.

Qualifications

Requires strong hands-on GCP Data Engineering experience with scalable cloud data platforms. Expertise in Google Cloud Storage, BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Composer, Cloud Functions, Cloud Monitoring, IAM, and Secret Manager. Proven experience with batch and real-time streaming technologies such as Pub/Sub, Dataflow/Apache Beam, Spark Structured Streaming, or Kafka. Proficient in Python, PySpark, and SQL, including debugging and performance tuning. Experience building high-volume, low-latency ingestion and transformation pipelines with robust error handling. Strong experience with Databricks, Delta Lake, Structured Streaming, Auto Loader, and Spark performance optimization. Ability to handle TB/PB-scale datasets, including partitioning, clustering, schema evolution, and data lifecycle management. Solid understanding of Data Lake, Lakehouse, and modern cloud data-platform architectures. Practical data modeling experience for analytics and reporting. Experience with CI/CD tools (Cloud Build, GitHub Actions, Jenkins) and Terraform for Infrastructure as Code. Excellent troubleshooting, observability, production support, and cloud cost-optimization skills are essential. Strong client communication, stakeholder management, and problem-solving abilities are also required.

Essential Skills

GCPData EngineeringGoogle Cloud StorageBigQueryDataflowDataprocPub/SubCloud ComposerCloud FunctionsCloud MonitoringIAMSecret ManagerBatch ProcessingReal-time StreamingApache BeamSpark Structured StreamingKafkaPythonPySparkSQLDatabricksDelta LakeStructured StreamingAuto LoaderData ModelingCI/CDCloud BuildGitHub ActionsJenkinsTerraformInfrastructure as CodeObservabilityCloud Cost OptimizationAirflow

Highlights

  • Actively hiring

More Details

RoleGCP Data Engineer
IndustryInformation Technology & Services, Management Consulting
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

Information Technology & Services

GCP Data Engineer at Infosys | SkillMX | SkillMX