Azure Data Engineer

Persistent

5–8 yrs Pune Full Time Hybrid (office + remote)
Persistent logo
Posted : today
Actively hiring

Job description

Persistent seeks a skilled Azure Data Engineer to enhance our data engineering team. This role demands deep expertise in Azure Cloud, Databricks, PySpark, Python, and Kafka, with practical experience in developing robust, scalable data platforms. The ideal candidate will combine technical acumen with problem-solving and collaborative abilities to create and maintain enterprise-level data solutions that drive business intelligence, analytics, and digital transformation.

We are a leading AI-driven digital engineering and modernization partner, dedicated to helping clients stay ahead in a dynamic market. Our innovative solutions and deep industry knowledge provide a distinct competitive edge. We collaborate with numerous industry leaders globally, including Fortune 50 companies and major financial institutions. Recognized for innovation, leadership, and inclusion, Persistent is experiencing significant growth, fueled by our global team of over 27,500 members across 18 countries.

Responsibilities

Design, develop, test, and maintain scalable data engineering solutions on Azure Cloud. Build and optimize batch and real-time data processing pipelines using Databricks and PySpark. Develop robust data ingestion, transformation, and integration frameworks for diverse data types. Translate business and technical requirements into secure, scalable, and maintainable data solutions. Implement streaming data solutions utilizing Kafka and modern event-driven architectures. Optimize data pipelines for peak performance, reliability, and scalability. Collaborate closely with cross-functional teams including architects, product owners, business analysts, QA, and DevOps. Ensure data quality, consistency, governance, and compliance throughout the data lifecycle. Support CI/CD implementation and automation for data engineering workflows. Create and maintain comprehensive technical documentation and reusable assets.

Qualifications

A minimum of 5-8 years of experience in Data Engineering, Big Data, or Cloud Data Platform development is essential. Demonstrate strong hands-on proficiency with Azure Cloud services and data engineering solutions. Extensive experience with Azure Databricks and PySpark is a must. Possess strong programming skills in Python for data processing and automation tasks. Proven experience building and supporting both real-time and batch data pipelines. Hands-on experience with Kafka and event-streaming architectures is required. Good understanding of data modeling, data warehousing, and ETL/ELT concepts. Experience with API integration, data ingestion frameworks, and cloud-native architectures is beneficial. Solid knowledge of software development best practices, version control, testing, and deployment methodologies. Demonstrated ability to troubleshoot production issues and optimize application/data pipeline performance. Understanding of data governance, data quality, security, and compliance principles. Experience working in Agile development environments and cross-functional teams is preferred. Excellent analytical, problem-solving, communication, and documentation skills are critical. Ability to manage deliverables independently while collaborating effectively with stakeholders.

Essential Skills

AzureDatabricksPySparkPythonKafkaAPI IntegrationData ModelingData WarehousingETLELTCI/CDDevOpsAgile

Highlights

  • Actively hiring

More Details

RoleAzure Data Engineer
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Persistent Systems logo

Persistent Systems

IT Consulting

Azure Data Engineer at Persistent | SkillMX | SkillMX