Data Engineer
Persistent
Persistent
Join a dedicated team focused on enhancing the healthcare system. This role offers the opportunity to collaborate with skilled professionals to develop innovative solutions addressing a wide range of health consumer needs.
As an experienced Data Engineer, you will play a crucial role in designing, developing, and maintaining scalable data pipelines. Your expertise will be vital in ensuring data quality, resolving pipeline issues, and optimizing data processing frameworks.
Key responsibilities include designing and developing robust data pipelines using Databricks and PySpark. You will investigate and resolve data quality issues, pipeline failures, and production incidents, performing root cause analysis for any data processing or framework-related problems.
Monitoring and optimizing Spark workloads for performance, scalability, and reliability is essential. You will collaborate with cross-functional teams to understand business requirements and implement effective data solutions, providing L2/L3 support for critical data platforms and driving continuous improvements.
This position requires 4 to 6 years of dedicated experience in Data Engineering and Big Data technologies, with strong hands-on expertise in Databricks. Proven experience in analyzing and resolving production support issues, framework failures, and data discrepancies is expected.
Candidates should possess strong SQL and data analysis skills, with experience in complex ETL/ELT pipelines and distributed data processing. Proficiency in troubleshooting, debugging, and performance tuning, along with experience on cloud platforms like Azure, is also a requirement.
Persistent Systems
Information Technology & Services