Azure Data Engineer
Persistent
Persistent
Persistent seeks a skilled Azure Data Engineer to enhance our data engineering team. This role demands deep expertise in Azure Cloud, Databricks, PySpark, Python, and Kafka, with practical experience in developing robust, scalable data platforms. The ideal candidate will combine technical acumen with problem-solving and collaborative abilities to create and maintain enterprise-level data solutions that drive business intelligence, analytics, and digital transformation.
We are a leading AI-driven digital engineering and modernization partner, dedicated to helping clients stay ahead in a dynamic market. Our innovative solutions and deep industry knowledge provide a distinct competitive edge. We collaborate with numerous industry leaders globally, including Fortune 50 companies and major financial institutions. Recognized for innovation, leadership, and inclusion, Persistent is experiencing significant growth, fueled by our global team of over 27,500 members across 18 countries.
Design, develop, test, and maintain scalable data engineering solutions on Azure Cloud. Build and optimize batch and real-time data processing pipelines using Databricks and PySpark. Develop robust data ingestion, transformation, and integration frameworks for diverse data types. Translate business and technical requirements into secure, scalable, and maintainable data solutions. Implement streaming data solutions utilizing Kafka and modern event-driven architectures. Optimize data pipelines for peak performance, reliability, and scalability. Collaborate closely with cross-functional teams including architects, product owners, business analysts, QA, and DevOps. Ensure data quality, consistency, governance, and compliance throughout the data lifecycle. Support CI/CD implementation and automation for data engineering workflows. Create and maintain comprehensive technical documentation and reusable assets.
A minimum of 5-8 years of experience in Data Engineering, Big Data, or Cloud Data Platform development is essential. Demonstrate strong hands-on proficiency with Azure Cloud services and data engineering solutions. Extensive experience with Azure Databricks and PySpark is a must. Possess strong programming skills in Python for data processing and automation tasks. Proven experience building and supporting both real-time and batch data pipelines. Hands-on experience with Kafka and event-streaming architectures is required. Good understanding of data modeling, data warehousing, and ETL/ELT concepts. Experience with API integration, data ingestion frameworks, and cloud-native architectures is beneficial. Solid knowledge of software development best practices, version control, testing, and deployment methodologies. Demonstrated ability to troubleshoot production issues and optimize application/data pipeline performance. Understanding of data governance, data quality, security, and compliance principles. Experience working in Agile development environments and cross-functional teams is preferred. Excellent analytical, problem-solving, communication, and documentation skills are critical. Ability to manage deliverables independently while collaborating effectively with stakeholders.
Persistent Systems
IT Consulting