Technical Lead (IN)

HCLTech

6–8 yrs India Full Time Remote
HCLTech logo
Posted : today
Actively hiring

Job description

Seeking a seasoned Senior GCP Data Engineer with 6-8 years of extensive experience. This role requires deep hands-on proficiency in GCP, BigQuery, Airflow, Astronomer, SQL, Python, GitHub, and CI/CD.

The ideal candidate possesses a thorough technical understanding of GCP data services, enabling them to independently design, develop, optimize, and support robust, scalable cloud-based data solutions.

Responsibilities

Key duties involve designing and maintaining scalable data pipelines and ETL/ELT solutions within GCP. You will craft complex, production-ready Apache Airflow DAGs from scratch, managing dependencies, scheduling, error handling, and monitoring.

Responsibilities include building and managing Airflow workloads via Astronomer, optimizing BigQuery datasets and queries, and developing reusable data engineering components using Python and SQL. A strong grasp of GCP architecture and services is essential for selecting appropriate tools. You'll manage GitHub repositories, implement CI/CD pipelines, and ensure engineering best practices for code quality, testing, and security.

Troubleshooting complex issues across BigQuery, Airflow, Python, SQL, and GCP environments is a critical aspect. You will also collaborate with cross-functional teams to translate requirements into scalable solutions and provide technical guidance to junior engineers through code reviews and mentorship.

Qualifications

This position demands 6-8 years of dedicated experience in Data Engineering, with significant hands-on application of GCP-based data solutions. A strong technical foundation across the Google Cloud Platform ecosystem and cloud-based data architectures is mandatory.

Essential skills include advanced expertise in Google BigQuery (complex SQL, data modelling, partitioning, clustering, query optimization) and Apache Airflow (designing complex DAGs). Proficiency with Astronomer for Airflow development and management is also required. Advanced SQL and Python skills for data engineering tasks are crucial.

Candidates must demonstrate a strong understanding of GCP services like BigQuery, Cloud Storage, Pub/Sub, Dataflow, Cloud Run, Cloud Functions, Secret Manager, and IAM. Hands-on experience with GitHub, including Git workflows, CI/CD principles, and version control, is vital. Familiarity with ETL/ELT, data warehousing, and data modelling is expected.

Excellent troubleshooting and analytical abilities are needed to independently resolve complex technical problems. The capacity to provide technical leadership, conduct code reviews, and mentor junior engineers is highly valued. Experience with Terraform/IaC and Docker are considered beneficial.

Essential Skills

GCPBigQueryAirflowAstronomerSQLPythonGitHubCI/CDCloud StoragePub/SubDataflowCloud RunCloud FunctionsSecret ManagerIAMETLELTData WarehousingData Modelling

Good to Have

TerraformIaCDocker

Highlights

  • Actively hiring

More Details

RoleTechnical Lead (IN)
DepartmentData Engineering
Employment TypeFull Time, Remote

About the Company

HCLTech logo

HCLTech

IT Consulting