Azure Data Engineer

Persistent

5–8 yrs Pune Full Time Hybrid (office + remote)
Persistent logo
Posted : today
Actively hiring

Job description

Join a leading digital engineering partner focused on AI and platform-driven solutions to empower clients and foster innovation. We collaborate with industry leaders across finance, healthcare, and technology, consistently delivering growth and recognized for our leadership in innovation and inclusion. Our global team of over 27,500 professionals in 18 countries drives transformation and maintains our position as a rapidly growing IT services brand.

We are seeking a skilled Azure Data Engineer to enhance our data engineering capabilities. This role is pivotal in designing, developing, and maintaining robust, scalable data platforms. The ideal candidate possesses strong Azure Cloud expertise, proficiency in Databricks, PySpark, Python, and Kafka, crucial for building advanced data solutions that support business intelligence and digital transformation efforts.

Responsibilities

Architect, build, and maintain scalable data engineering solutions on Azure Cloud. Develop and optimize data processing pipelines, both batch and real-time, leveraging Databricks and PySpark. Create and implement frameworks for ingesting, transforming, and integrating diverse data types. Translate business and technical requirements into secure, scalable, and maintainable data solutions. Utilize Python and cloud-native services to develop and support data processing workflows. Implement streaming data solutions using Kafka and modern event-driven architectures. Conduct code reviews, perform unit and integration testing, and provide production support. Continuously optimize data pipelines for enhanced performance, reliability, and scalability. Collaborate effectively with cross-functional teams including architects, product owners, business analysts, QA, and DevOps. Ensure data quality, consistency, governance, and compliance across the data lifecycle. Support CI/CD implementation and automation for data engineering workloads. Develop and maintain comprehensive technical documentation and reusable assets.

Qualifications

Accumulate 5-8 years of experience in Data Engineering, Big Data, or Cloud Data Platform development. Possess strong hands-on experience with Azure Cloud services and data engineering tools. Demonstrate extensive expertise in Azure Databricks and PySpark. Showcase robust programming skills in Python for data processing and automation. Experience building and supporting both real-time and batch data pipelines. Proficiency with Kafka and event-streaming architectures. Solid understanding of data modeling, data warehousing, and ETL/ELT principles. Familiarity with API integration, data ingestion frameworks, and cloud-native architectures. Strong grasp of software development best practices, including version control, testing, and deployment. Proven ability to troubleshoot production issues and optimize application and data pipeline performance. Understanding of data governance, data quality, security, and compliance. Experience working within Agile development environments and cross-functional teams. Exhibit strong analytical, problem-solving, communication, and documentation skills. Ability to manage tasks independently while collaborating effectively with stakeholders.

Essential Skills

AzureDatabricksPySparkPythonKafkaETLELTAPI IntegrationCI/CDAgile

Highlights

  • Actively hiring

More Details

RoleAzure Data Engineer
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Persistent Systems logo

Persistent Systems

IT Consulting

Azure Data Engineer at Persistent | SkillMX | SkillMX