Track Lead - Extract Transform Load (ETL), Data Lakes (IN)

HCLTech

5–9 yrs India Full Time Work from office
HCLTech logo
Posted : today
Actively hiring

Job description

Lead the charge in modernizing our data infrastructure by migrating an existing Oracle data warehouse to Databricks on the Allianz Data Platform. This senior role requires deep expertise in Databricks, SQL, and data warehousing, with a focus on implementing cutting-edge data engineering practices.

We're looking for someone passionate about building AI-enabling data platforms and ensuring the highest standards of data quality and performance. Join us to drive innovation and shape the future of our data landscape.

Responsibilities

Analyze the current Oracle data warehouse and PostgreSQL sources, designing the optimal target model within Databricks.

Develop and refine ETL/ELT pipelines using Spark SQL and PySpark to seamlessly migrate core data warehouse tables and datamarts.

Translate complex Oracle SQL/PL-SQL logic into efficient Databricks Spark SQL and PySpark code, ensuring functional parity and peak performance.

Maintain the integrity, performance, and reliability of the new Databricks data warehouse, empowering advanced analytics and AI/ML initiatives.

Collaborate effectively with Enterprise Architecture, the ADP platform team, and business stakeholders to uphold architectural standards and best practices.

Create comprehensive documentation for migration strategies, data models, and operational procedures to ensure long-term manageability and knowledge sharing.

Qualifications

Possess 5-9 years of experience in data engineering, with a strong foundation in data warehousing, ETL/ELT, and performance tuning.

Demonstrate advanced proficiency with Databricks, including Spark SQL, PySpark, Delta Lake, job orchestration, and cluster management.

Showcase solid experience in data warehousing principles, dimensional modeling, and working with Oracle DWH environments, including SQL and PL/SQL.

Experience integrating PostgreSQL as a data source into a central DWH/Lakehouse is essential.

Ability to reverse-engineer and translate existing ETL logic (e.g., SAS jobs) into Databricks pipelines is crucial.

Exceptional analytical, logical, and communication skills, coupled with strong stakeholder management abilities in a global, multi-vendor setting.

An understanding of how well-structured data fuels advanced analytics and AI/ML capabilities is highly valued.

Essential Skills

DatabricksSpark SQLPySparkDelta LakeETLData WarehousingDimensional ModellingOracleSQLPL/SQLPostgreSQLData LakesData MigrationSASStakeholder ManagementData QualityPerformance TuningLogical ReasoningAnalytical SkillsCommunication Skills

Highlights

  • Actively hiring

More Details

RoleTrack Lead - Extract Transform Load (ETL), Data Lakes (IN)
DepartmentData Engineering
Employment TypeFull Time, Work from office

About the Company

HCLTech logo

HCLTech

IT Consulting