Senior Data Engineer
Persistent
Persistent
Join Persistent as a Senior Data Engineer and contribute to improving the healthcare system. This role offers a collaborative environment with skilled professionals focused on addressing diverse health consumer needs through innovative data solutions.
Be a key player in a dynamic team, leveraging your expertise to design and implement cutting-edge data strategies. We are looking for individuals passionate about making a tangible difference in healthcare delivery.
Design, develop, and maintain robust, scalable data pipelines utilizing Databricks and PySpark. Investigate and resolve critical issues including data quality anomalies, pipeline failures, and production incidents. Conduct thorough root cause analysis for data processing and framework-related challenges. Monitor and optimize Spark workloads to ensure peak performance, scalability, and reliability. Collaborate with cross-functional teams to translate business requirements into effective data solutions. Provide essential L2/L3 support for critical data platforms and manage production deployments. Establish and uphold high standards for data engineering, code quality, monitoring, and operational excellence. Champion continuous improvements across data processing frameworks and platform stability.
A minimum of 6 years of dedicated experience in Data Engineering and Big Data technologies is essential. Demonstrate strong, hands-on expertise with Databricks. Proven experience in analyzing and resolving production support issues, framework failures, and data discrepancies. Possess advanced SQL and data analysis capabilities. Experience with complex ETL/ELT pipelines and distributed data processing is required. Proficiency in troubleshooting, debugging, and performance tuning of data systems. Familiarity with cloud platforms, particularly Azure, is necessary. Experience with Kafka Connect is highly valued.
Persistent Systems
IT Consulting