Senior Azure Data Engineer

Persistent

6–10 yrs Pune, Bengaluru, Chennai Full Time Hybrid (office + remote)
Persistent logo
Posted : today
Actively hiring

Job description

Join a leading digital engineering and enterprise modernization partner as a Senior Azure Data Engineer. This AI-led firm leverages deep technical expertise and industry experience to empower clients with future-ready solutions. You will be instrumental in designing, developing, and optimizing scalable cloud-based data platforms for enterprise analytics, BI, and digital transformation. This role offers the opportunity to build modern data architectures and develop enterprise-grade ETL/ELT pipelines, driving data-driven decision-making across a global organization.

This position requires hands-on experience with Azure cloud services, Snowflake data warehousing, and large-scale data processing frameworks, contributing to the company's consistent growth and recognition as a leader in IT services and product engineering.

Responsibilities

Design, develop, and maintain robust data pipelines utilizing Azure Databricks and PySpark. Build and refine ETL/ELT workflows for diverse datasets, including structured, semi-structured, and unstructured data. Develop and manage Snowflake data warehouse solutions, focusing on data ingestion, transformation, and performance optimization. Create reusable frameworks for data ingestion, orchestration, transformation, validation, and monitoring. Integrate data from various enterprise applications, databases, APIs, and cloud platforms into Azure Data Lake and Snowflake.

Design and implement advanced data lake and Lakehouse architectures. Optimize data processing workloads for maximum performance, reliability, scalability, and cost efficiency. Implement best practices for data quality, governance, security, and monitoring. Collaborate with business stakeholders, solution architects, data analysts, and engineering teams to deliver impactful data-driven solutions. Troubleshoot and resolve production issues swiftly and efficiently.

Qualifications

A minimum of 6 to 10 years of experience in Data Engineering and Cloud Data Platform development is essential. Demonstrate strong hands-on expertise in Azure Databricks, PySpark, Spark SQL, and Delta Lake. Proven experience with Snowflake, encompassing data loading, transformation, query optimization, data sharing, and performance tuning, is critical. Possess strong SQL expertise for complex query development, optimization, troubleshooting, and performance management.

Experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Azure Key Vault, and Azure DevOps is required. A good understanding of Data Warehousing principles, Dimensional Modeling, ETL/ELT frameworks, and Lakehouse architecture is expected. Experience building and supporting enterprise-scale data pipelines and analytics platforms is a plus. Knowledge of data governance, data quality, metadata management, and security best practices is valuable. Familiarity with Git, CI/CD pipelines, source control management, and Agile methodologies is beneficial. A strong understanding of distributed computing and cloud-native data architectures is important, alongside excellent analytical, troubleshooting, debugging, and problem-solving skills. Effective communication and stakeholder management abilities are key to success in this role.

Essential Skills

Azure DatabricksSnowflakePySparkSQLAzure Data FactoryAzure Data Lake Storage Gen2Azure Key VaultAzure DevOpsSpark SQLDelta LakeData WarehousingDimensional ModelingETLELTLakehouse ArchitectureGitCI/CDDevOpsAgile

Good to Have

AWS CloudDatabricks

Highlights

  • Actively hiring

More Details

RoleSenior Azure Data Engineer
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Persistent Systems logo

Persistent Systems

IT Consulting

Senior Azure Data Engineer at Persistent | SkillMX | SkillMX