Senior Data Engineer
Persistent
Persistent
Join a forward-thinking team focused on enhancing the healthcare system for everyone. As a Senior Data Engineer, you will collaborate with exceptional individuals to develop innovative solutions addressing diverse health consumer needs. Persistent Systems, an AI-led digital engineering partner, empowers clients with cutting-edge solutions and deep industry expertise. Recognized for innovation and growth, we are committed to client success and fostering an agile, dynamic environment.
Design, build, and maintain robust, scalable data pipelines utilizing Databricks and PySpark. Investigate and resolve data quality issues, pipeline failures, and production incidents. Conduct root cause analysis for data processing and framework-related problems. Monitor and optimize Spark workloads for peak performance, scalability, and reliability. Collaborate with cross-functional teams to translate business needs into effective data solutions. Provide L2/L3 support for critical data platforms and manage production deployments. Establish and uphold best practices for data engineering, code quality, monitoring, and operational excellence. Drive continuous improvements in data processing frameworks and platform stability.
Bring over 6 years of dedicated experience in Data Engineering and Big Data technologies. Possess strong, hands-on expertise with Databricks. Demonstrate proven experience in analyzing and resolving production support issues, framework failures, and data discrepancies. Exhibit excellent SQL and data analysis capabilities. Have experience with complex ETL/ELT pipelines and distributed data processing. Showcase proficiency in troubleshooting, debugging, and performance tuning. Previous experience with cloud platforms, particularly Azure, is essential.
Persistent Systems
IT Consulting