Azure Data Engineer

Persistent

5–8 yrs Pune Full Time Hybrid (office + remote)
Persistent logo
Posted : today
Actively hiring

Job description

Join a leading digital engineering and enterprise modernization partner focused on AI and platform-driven solutions. We empower clients with advanced capabilities and innovative strategies. Our extensive experience with Fortune 50 companies and top financial institutions across the US and India highlights our commitment to client success and agility in dynamic markets.

We are currently seeking a skilled Azure Data Engineer to enhance our data engineering team. The ideal candidate possesses deep expertise in Azure Cloud, Databricks, PySpark, Python, and Kafka, with a proven track record in building robust, scalable data platforms. This role demands a blend of technical acumen, problem-solving prowess, and collaborative spirit to architect, develop, and maintain enterprise-grade data solutions essential for business intelligence, analytics, and digital transformation.

This full-time position is based in Pune and requires 5 to 8 years of relevant experience.

Responsibilities

In this role, you will be instrumental in designing, developing, testing, and maintaining scalable data engineering solutions within the Azure Cloud environment. You will build and optimize both batch and real-time data processing pipelines using Databricks and PySpark, and develop sophisticated frameworks for ingesting, transforming, and integrating structured and unstructured data.

Translate complex business and technical requirements into secure, scalable, and maintainable data solutions. You will also develop and support data processing workflows leveraging Python and cloud-native services, and implement streaming data solutions with Kafka, embracing modern event-driven architectures. Responsibilities include conducting code reviews, unit testing, integration testing, debugging, and providing production support.

Key duties also involve optimizing data pipelines for peak performance, reliability, and scalability, and collaborating closely with architects, product owners, business analysts, QA, DevOps teams, and data governance stakeholders. Ensuring data quality, consistency, governance, and compliance throughout the data lifecycle is paramount. You will support CI/CD implementation for data engineering workloads and create comprehensive technical documentation, reusable assets, and operational procedures. Contributing to engineering best practices, innovation, and continuous improvement is also expected.

Qualifications

We are looking for professionals with 5-8 years of dedicated experience in Data Engineering, Big Data, or Cloud Data Platform development. A strong command of Azure Cloud services and data engineering solutions is essential.

Candidates must demonstrate extensive experience with Azure Databricks and PySpark, coupled with robust programming skills in Python for data processing and automation. Proven experience in building and supporting both real-time and batch data pipelines is required, along with hands-on expertise in Kafka and event-streaming architectures. A solid understanding of data modeling, data warehousing, and ETL/ELT concepts is necessary.

Familiarity with API integration, data ingestion frameworks, and cloud-native architectures is beneficial. Strong knowledge of software development best practices, including version control, testing, and deployment methodologies, is a must. Experience in troubleshooting production issues and optimizing application and data pipeline performance is also key. An understanding of data governance, data quality, security, and compliance principles is important. Prior experience in Agile development environments and cross-functional teams is preferred. Excellent analytical, problem-solving, communication, and documentation skills are vital, along with the ability to manage deliverables independently while collaborating effectively.

Essential Skills

AzureDatabricksPySparkPythonKafkaAPI integrationData modelingData warehousingETL/ELTCloud-native architecturesVersion controlTestingDeployment methodologiesCI/CDAgile

Highlights

  • Actively hiring

More Details

RoleAzure Data Engineer
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Persistent Systems logo

Persistent Systems

IT Consulting

Azure Data Engineer at Persistent | SkillMX | SkillMX