Lead Azure Data Engineer
Persistent
Persistent
Join Persistent as a Lead Azure Data Engineer and drive the design, development, and optimization of modern cloud-based data platforms. This role is key to building scalable enterprise data solutions and enabling advanced analytics, reporting, and business intelligence across the organization. You will leverage your expertise in Azure Databricks, Snowflake, PySpark, and SQL to lead impactful data engineering initiatives.
Persistent is a leading AI-led digital engineering partner, known for its deep technical expertise and industry experience. We collaborate with many industry leaders, including Fortune 50 companies and top technology firms, helping them anticipate future needs and achieve competitive advantages. Our commitment to client success and agility has fueled consistent growth, supported by a global team of over 29,500 professionals.
Design and build scalable enterprise data pipelines using Azure Databricks and PySpark, optimizing ETL/ELT frameworks for large datasets. Develop and manage Snowflake data warehouse solutions, focusing on data modeling and performance. Implement Lakehouse architectures on Azure and Snowflake, creating reusable frameworks for data ingestion, transformation, and validation.
Integrate diverse enterprise data sources into Azure Data Lake and Snowflake. Optimize data processing for performance, cost, and reliability. Establish robust data governance, security, and quality frameworks. Collaborate with cross-functional teams to deliver business-driven data solutions and lead troubleshooting efforts for production issues.
Bring 12-16 years of experience in Data Engineering and cloud data platforms, with strong hands-on skills in Azure Databricks, PySpark, Spark SQL, and Delta Lake. Proven expertise in Snowflake, including data loading, transformations, and query optimization, alongside advanced SQL skills. Familiarity with Azure Data Factory, ADLS Gen2, Azure Key Vault, and Azure DevOps is essential.
A solid understanding of Data Warehousing, Dimensional Modeling, ETL/ELT, and Lakehouse architectures is required. Experience with CI/CD, DevOps practices, and Agile methodologies is crucial. You should possess strong analytical, debugging, and problem-solving abilities, coupled with excellent communication and leadership qualities.
Persistent Systems
Information Technology & Services