Azure Data Engineer – Snowflake & Databricks
Persistent
Persistent
Join our dynamic team as an Azure Data Engineer and play a pivotal role in architecting and developing cutting-edge data solutions. You will leverage your expertise in Azure Databricks, Snowflake, PySpark, and SQL to build robust, scalable data platforms that drive enterprise analytics and business intelligence. This is an opportunity to work with large-scale datasets and modern cloud technologies within an innovative environment.
Design, develop, and maintain high-performance data pipelines using Azure Databricks and PySpark. Build and optimize ETL/ELT workflows for diverse data types. Develop and manage Snowflake data warehouses, focusing on data modeling and performance. Integrate data from various enterprise systems into Azure Data Lake and Snowflake. Implement best practices for data quality, governance, monitoring, and security. Support the design of modern data lakehouse architectures and troubleshoot data-related issues. Collaborate with stakeholders to deliver data-driven solutions and participate in Agile ceremonies.
A minimum of 4-10 years of experience in Data Engineering and Analytics is essential. Demonstrated hands-on proficiency with Azure Databricks, PySpark, Spark SQL, and Delta Lake is required. Expertise in Snowflake, including data ingestion, transformation, optimization, and data sharing, is crucial. Strong SQL development skills with a focus on query optimization are necessary. Familiarity with Azure services such as ADF, ADLS, Key Vault, and Azure DevOps is expected. A solid understanding of data warehousing concepts, dimensional modeling, data lake, and lakehouse architectures is vital. Experience with Git, CI/CD pipelines, and Agile methodologies is also a key requirement.
Persistent Systems
IT Consulting