Senior Azure Data Engineer – Snowflake & Databricks

Persistent

8–14 yrs Pune, Bengaluru, Chennai Full Time Hybrid (office + remote)
Persistent logo
Posted : today
Actively hiring

Job description

Join a leading digital engineering and enterprise modernization partner as a Senior Azure Data Engineer. This AI-led firm leverages deep technical expertise and industry experience to empower clients. They collaborate with global industry leaders, including Fortune 50 companies and top financial institutions.

This role is central to designing, developing, and optimizing enterprise-scale data platforms. These platforms are crucial for advanced analytics, business intelligence, and digital transformation initiatives. The ideal candidate will possess extensive technical skills in cloud-based data architectures, ETL/ELT frameworks, data warehousing, and large-scale data processing.

You will work closely with architects, business stakeholders, analytics teams, and engineering groups. The goal is to deliver scalable, secure, and high-performance data solutions across Azure and Snowflake ecosystems. This is a full-time opportunity based in Pune, Bengaluru, or Chennai.

Responsibilities

Design and develop robust, scalable data pipelines utilizing Azure Databricks and PySpark. Build and refine ETL/ELT workflows to efficiently process diverse data types.

Develop and manage Snowflake data warehouse solutions, focusing on data modeling, transformation, and performance optimization. Create reusable frameworks for data ingestion, orchestration, validation, and transformation processes.

Integrate data from various enterprise sources into Azure Data Lake and Snowflake environments. Design and implement modern data lakehouse architecture solutions.

Optimize data processing workloads for peak performance, scalability, reliability, and cost-effectiveness. Implement comprehensive enterprise data quality, governance, monitoring, and security controls.

Develop scalable data models and support critical business intelligence and analytics requirements. Collaborate effectively with business stakeholders, solution architects, and analytics teams to deliver impactful data-driven solutions.

Troubleshoot and resolve production issues related to data pipelines and cloud platforms. Participate actively in architecture discussions and propose improvements for data platforms. Conduct code reviews and champion data engineering best practices. Implement CI/CD pipelines and deployment automation. Support cloud migration and modernization initiatives.

Generate and maintain essential technical documentation, design specifications, and operational runbooks. Mentor junior engineers and contribute to technical leadership within the team.

Qualifications

A minimum of 8–14 years of dedicated experience in Data Engineering, Analytics, and Data Warehousing projects is essential.

Demonstrate strong hands-on proficiency with Azure Databricks, PySpark, Spark SQL, and Delta Lake. Showcase proven expertise in Snowflake, including data loading, transformation, optimization, clustering, and data sharing.

Possess strong SQL development and query optimization skills. Experience designing and implementing large-scale ETL/ELT pipelines is required. A solid understanding of data warehousing concepts, dimensional modeling, and data architecture principles is crucial.

Familiarity with Azure services such as Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Key Vault, and Azure DevOps is expected. Expertise in designing and implementing Lakehouse architectures and processing large-scale structured and unstructured datasets is a must.

Develop a strong understanding of data governance, metadata management, and data quality frameworks. Experience implementing enterprise security controls within cloud data platforms is vital. Hands-on experience with Git, branching strategies, and source control management is necessary.

Proficiency in implementing CI/CD pipelines and DevOps practices for data engineering solutions. Knowledge of performance tuning, workload optimization, and cloud cost management is beneficial. Familiarity with monitoring, logging, and operational support of enterprise data platforms is expected.

Exhibit strong analytical, troubleshooting, and problem-solving capabilities. Experience leading technical discussions and collaborating with cross-functional teams is key. Excellent communication and stakeholder management skills are required. Experience working in Agile and Scrum delivery environments is preferred.

Demonstrate the ability to mentor and guide junior engineers while actively contributing to technical excellence. This role is for an experienced professional and not open to freshers.

Essential Skills

Azure DatabricksSnowflakePySparkSpark SQLDelta LakeSQLAzure Data Factory (ADF)Azure Data Lake Storage (ADLS)Azure Key VaultAzure DevOpsGitCI/CDDevOps

Highlights

  • Actively hiring

More Details

RoleSenior Azure Data Engineer – Snowflake & Databricks
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Persistent Systems logo

Persistent Systems

IT Consulting

Senior Azure Data Engineer – Snowflake & Databricks at Persistent | SkillMX | SkillMX