Senior Azure Data Engineer
Persistent
Persistent
Join a leading digital engineering and enterprise modernization partner as a Senior Azure Data Engineer. This AI-led firm leverages deep technical expertise and industry experience to empower clients with future-ready solutions. You will be instrumental in designing, developing, and optimizing scalable cloud-based data platforms for enterprise analytics, BI, and digital transformation. This role offers the opportunity to build modern data architectures and develop enterprise-grade ETL/ELT pipelines, driving data-driven decision-making across a global organization.
This position requires hands-on experience with Azure cloud services, Snowflake data warehousing, and large-scale data processing frameworks, contributing to the company's consistent growth and recognition as a leader in IT services and product engineering.
Design, develop, and maintain robust data pipelines utilizing Azure Databricks and PySpark. Build and refine ETL/ELT workflows for diverse datasets, including structured, semi-structured, and unstructured data. Develop and manage Snowflake data warehouse solutions, focusing on data ingestion, transformation, and performance optimization. Create reusable frameworks for data ingestion, orchestration, transformation, validation, and monitoring. Integrate data from various enterprise applications, databases, APIs, and cloud platforms into Azure Data Lake and Snowflake.
Design and implement advanced data lake and Lakehouse architectures. Optimize data processing workloads for maximum performance, reliability, scalability, and cost efficiency. Implement best practices for data quality, governance, security, and monitoring. Collaborate with business stakeholders, solution architects, data analysts, and engineering teams to deliver impactful data-driven solutions. Troubleshoot and resolve production issues swiftly and efficiently.
A minimum of 6 to 10 years of experience in Data Engineering and Cloud Data Platform development is essential. Demonstrate strong hands-on expertise in Azure Databricks, PySpark, Spark SQL, and Delta Lake. Proven experience with Snowflake, encompassing data loading, transformation, query optimization, data sharing, and performance tuning, is critical. Possess strong SQL expertise for complex query development, optimization, troubleshooting, and performance management.
Experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Azure Key Vault, and Azure DevOps is required. A good understanding of Data Warehousing principles, Dimensional Modeling, ETL/ELT frameworks, and Lakehouse architecture is expected. Experience building and supporting enterprise-scale data pipelines and analytics platforms is a plus. Knowledge of data governance, data quality, metadata management, and security best practices is valuable. Familiarity with Git, CI/CD pipelines, source control management, and Agile methodologies is beneficial. A strong understanding of distributed computing and cloud-native data architectures is important, alongside excellent analytical, troubleshooting, debugging, and problem-solving skills. Effective communication and stakeholder management abilities are key to success in this role.
Persistent Systems
IT Consulting