Senior Azure Data Engineer - Snowflake & Databricks

Persistent

8–14 yrs Pune, Hyderabad, Chennai Full Time Hybrid (office + remote)
Persistent logo
Posted : today
Actively hiring

Job description

Join a leading AI-led digital engineering firm as a Senior Azure Data Engineer, specializing in Snowflake and Databricks. This role focuses on designing, developing, and optimizing enterprise-scale data platforms to drive advanced analytics and digital transformation. You will leverage deep expertise in cloud data architectures and large-scale data processing to build high-performance solutions within Azure and Snowflake environments. Collaboration with architects, business stakeholders, and engineering teams is key to delivering innovative data solutions.

We offer a dynamic work environment with a focus on talent development, cutting-edge technologies, and a people-centric culture. Benefit from competitive compensation, growth opportunities, and comprehensive health and insurance coverage. Our commitment to diversity and inclusion ensures a supportive and accessible workplace for all.

Responsibilities

Design and implement robust data pipelines using Azure Databricks and PySpark, optimizing ETL/ELT workflows for diverse data types. Develop and manage Snowflake data warehouses, focusing on data modeling and performance tuning. Create reusable frameworks for data ingestion, orchestration, and validation, integrating data from multiple sources into Azure Data Lake and Snowflake. Implement data lakehouse architectures and optimize workloads for performance, scalability, and cost efficiency. Ensure enterprise-grade data quality, governance, monitoring, and security controls. Develop scalable data models to meet business intelligence and analytics needs. Troubleshoot production issues and participate in architecture discussions to recommend platform enhancements.

Enforce data engineering best practices through code reviews. Implement CI/CD pipelines and automation for data engineering projects. Support cloud migration and modernization initiatives. Create technical documentation and operational runbooks. Mentor junior engineers and contribute to technical leadership.

Qualifications

A minimum of 8 to 14 years of experience in Data Engineering, Analytics, and Data Warehousing is required. Essential technical skills include strong hands-on experience with Azure Databricks, PySpark, Spark SQL, Delta Lake, and Snowflake (data loading, transformation, optimization, clustering). Proficiency in SQL development and query optimization is crucial.

Demonstrated expertise in designing and implementing large-scale ETL/ELT pipelines and a solid understanding of data warehousing concepts, dimensional modeling, and data architecture principles are expected. Experience with Azure services such as Azure Data Factory, Azure Data Lake Storage, and Azure DevOps is necessary. Familiarity with Lakehouse architectures, processing large-scale datasets, data governance, and enterprise security controls within cloud platforms is also required.

Proficiency in Git and implementing CI/CD pipelines with DevOps practices is important. Knowledge of performance tuning, workload optimization, cloud cost management, and monitoring/logging for enterprise data platforms is beneficial. Strong analytical, troubleshooting, problem-solving, and stakeholder management skills are vital. Experience in Agile/Scrum environments and the ability to mentor junior engineers are highly valued. Experience leading technical discussions and collaborating with cross-functional teams is a key aspect of this role.

Essential Skills

Azure DatabricksPySparkSnowflakeSQLETLData WarehousingAzure Data FactoryAzure Data Lake StorageGitCI/CDData Engineering

Good to Have

DatabricksSpark SQLDelta LakeAzure Key VaultAzure DevOpsLakehouse ArchitectureData GovernanceMetadata ManagementDevOpsScrum

Highlights

  • Actively hiring

More Details

RoleSenior Azure Data Engineer - Snowflake & Databricks
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Persistent Systems logo

Persistent Systems

IT Consulting

Senior Azure Data Engineer - Snowflake & Databricks at Persistent | SkillMX | SkillMX