Azure Data Engineer
Persistent
Persistent
Join a dynamic AI-led digital engineering team as an Azure Data Engineer. You will architect and implement robust, scalable data solutions leveraging Azure, Databricks, and PySpark to drive critical business and healthcare initiatives. This role is pivotal in developing modern data platforms and advanced analytics that provide clients with a significant competitive advantage.
Persistent is a global leader in enterprise modernization, renowned for its technical expertise and commitment to client success. We empower market leaders across finance, technology, and healthcare, fostering innovation and growth. Our diverse, global team thrives in a dynamic environment, consistently delivering exceptional results and driving industry recognition.
Design, develop, and maintain scalable data pipelines utilizing Azure Databricks and PySpark. Build and enhance complex ETL/ELT workflows for large-scale data processing. Develop high-performance data solutions using Azure data services and cloud-native technologies.
Investigate and resolve data quality issues, pipeline failures, and production incidents. Perform root cause analysis and troubleshoot data processing and platform-related challenges. Monitor, optimize, and tune Spark workloads for enhanced performance, scalability, and reliability.
Create and maintain efficient SQL queries, data models, and transformation logic. Collaborate with stakeholders, architects, and engineering teams to define requirements and deliver effective data solutions. Support production deployments and provide L2/L3 support for critical data platforms.
Implement data governance, monitoring, and operational best practices. Ensure adherence to coding standards, security policies, and data engineering best practices. Drive continuous improvements in data processing frameworks, automation, and platform stability.
A minimum of 4-8 years of experience in Data Engineering, Big Data, and Data Integration projects is required. You should possess strong hands-on experience with Azure Databricks and PySpark, coupled with solid expertise in designing and developing enterprise-scale ETL/ELT pipelines.
Proficiency in SQL, data analysis, and database concepts is essential. Experience with Azure cloud services, modern data architectures, and Apache Spark for distributed processing is expected. A strong understanding of Delta Lake, Data Lake, and Lakehouse architectures is crucial.
Troubleshooting production issues, data inconsistencies, and performance bottlenecks is a key skill. Familiarity with data governance, data quality, and metadata management practices is beneficial. Experience with CI/CD pipelines, DevOps, and deployment automation is highly desirable.
Excellent analytical, problem-solving, and debugging skills are necessary. Strong communication and stakeholder management abilities are key. A Bachelor's degree in Engineering, Computer Science, Information Technology, or a related field is required.
Persistent Systems
IT Consulting