Snowflake Data Engineer
Persistent
Persistent
Join our dynamic team as a Senior Data Engineer with a strong focus on Azure Databricks and data engineering excellence. You will be instrumental in designing, developing, and optimizing scalable data pipelines within the Azure ecosystem. This role is key to enhancing our data capabilities and fostering data-driven decision-making across the organization. We are an AI-led, platform-driven digital engineering partner committed to helping clients anticipate future needs and achieve a competitive advantage.
Our work spans across industry leaders, including top banks and healthcare innovators. We thrive on a disruptor's mindset, client success, and agility, maintaining consistent growth and recognition for innovation and inclusion. Our global team of over 26,500 professionals in 18 countries drives transformation for market leaders. Recognized for AI and Data Excellence and Talent Readiness in Data, Analytics, and AI.
Design and implement robust ETL/ELT data pipelines using Azure Databricks, ensuring high performance and scalability. Collaborate with cross-functional teams to gather requirements and translate them into technical specifications. Optimize existing data pipelines for performance and efficiency, handling large datasets effectively. Utilize Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS) for data ingestion and storage. Implement Delta Lake and Lakehouse architecture for enhanced data reliability and accessibility. Conduct performance tuning and optimization of data processes for timely delivery. Maintain version control and CI/CD practices using Git and Azure DevOps. Stay current with the latest trends and best practices in cloud-based data engineering.
Possess 4+ years of hands-on experience with Azure Databricks and strong proficiency in PySpark and Python programming. Demonstrate a solid understanding of SQL and data modeling techniques, along with experience in building ETL/ELT data pipelines. Have hands-on experience with Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS), and familiarity with Delta Lake and Lakehouse architecture. Exhibit knowledge of performance optimization techniques for large datasets and experience with Git, CI/CD, and Azure DevOps. A strong understanding of cloud-based data engineering concepts is essential. Preferred skills include experience with data visualization tools, machine learning concepts, and data governance practices.
Persistent Systems
IT Consulting