Technical Specialist (IN)

HCLTech

8+ yrs India Full Time Remote
HCLTech logo
Posted : today
Actively hiring

Job description

Seeking a skilled Senior GCP Data Engineer with extensive experience in Google Cloud Platform, Python, PySpark, and SQL. This role focuses on designing, developing, and maintaining scalable data platforms for both batch and streaming data. The ideal candidate will transform complex data into reliable, secure assets, demonstrating strong technical leadership and a commitment to production engineering excellence. Your expertise will be crucial in translating business needs into robust cloud data solutions.

Responsibilities

Architect and build scalable data pipelines and platforms on GCP for batch and near-real-time data. Develop efficient ETL/ELT solutions utilizing Python, PySpark, SQL, and various GCP services. Design data models (dimensional, normalized, denormalized, lakehouse) to meet analytical and operational demands. Implement advanced SQL transformations and create reusable data processing components. Optimize Spark jobs, BigQuery queries, and resource utilization for peak performance and cost-effectiveness. Ingest data from diverse sources, ensuring schema evolution and incremental processing. Establish automated data quality controls, lineage, and observability. Apply stringent security and governance standards, including IAM and encryption. Develop comprehensive testing strategies and integrate them into CI/CD pipelines. Utilize infrastructure-as-code for consistent environment deployment. Address and resolve complex production issues, conducting root-cause analysis. Collaborate with cross-functional teams to deliver maintainable solutions. Lead technical reviews, uphold engineering standards, and mentor team members.

Qualifications

Requires 8+ years of professional experience in data engineering, data warehousing, big data, or cloud platform development. Must have strong production experience with data solutions on Google Cloud Platform. Advanced proficiency in Python for data processing, automation, and framework development. Advanced skills in PySpark and Apache Spark, covering transformations, performance tuning, and distributed processing. Expert SQL knowledge, including complex queries, optimization, and data modeling. Hands-on experience with BigQuery, Cloud Storage, Dataproc, Dataflow, Pub/Sub, Cloud Composer, or Apache Airflow. Solid understanding of ETL/ELT patterns, batch/streaming architectures, data lakes, data warehouses, and orchestration. Familiarity with Git, automated testing, CI/CD, logging, monitoring, and production support. Working knowledge of GCP IAM, networking, encryption, data governance, and secure data architecture. Strong analytical, problem-solving, communication, documentation, and leadership skills. Bachelor's degree in a relevant field or equivalent practical experience.

Essential Skills

GCPPythonPySparkSQLBigQueryCloud StorageDataprocDataflowPub/SubCloud ComposerApache AirflowETLELTData WarehousingBig DataGitCI/CDGCP IAM

Highlights

  • Actively hiring

More Details

RoleTechnical Specialist (IN)
Employment TypeFull Time, Remote

About the Company

HCLTech logo

HCLTech

IT Consulting