Join Optum, a global leader in health optimization, and contribute to improving health outcomes for millions. As part of our inclusive culture, you will collaborate with talented peers, access comprehensive benefits, and discover exciting career development opportunities.

This role is instrumental in advancing health optimization on a global scale. We invite you to make a significant impact on the communities we serve, fostering a spirit of Caring, Connecting, and Growing together.

Design, develop, and deploy scalable data pipelines utilizing Azure Databricks, Spark SQL, and Python. Build robust data ingestion, transformation, and curation processes from diverse enterprise sources. Craft and optimize complex SQL queries for enhanced performance.

Manage and maintain Airflow DAGs for seamless orchestration, scheduling, and monitoring. Collaborate closely with stakeholders to define requirements and refine backlogs. Implement rigorous data quality validations and exception handling.

Support Delta Lake and Lakehouse architectures within Databricks. Drive data platform modernization and cloud migration initiatives. Develop reusable frameworks and automation solutions to boost engineering efficiency. Troubleshoot pipeline failures and performance bottlenecks, adhering to coding standards and security best practices. Contribute to CI/CD pipelines and automated deployments.

Collaborate with cross-functional teams to support analytics, reporting, and AI initiatives. Prepare technical documentation and operational runbooks. Provide technical recommendations and participate in architecture discussions.

Stay abreast of emerging technologies and cloud services. Comply with all company policies and security standards.

A Bachelor's Degree in Engineering, Computer Science, Information Technology, MCA, or equivalent experience is required.

Possess at least 4 years of hands-on experience in Azure Databricks and modern data engineering principles. Experience working within Agile/Scrum delivery frameworks is essential.

Key technical proficiencies include Microsoft Azure, Azure Data Factory, Azure Key Vault, Azure DevOps/GitHub, CI/CD pipelines, data ingestion frameworks, data transformation, batch processing, data quality implementation, query optimization, and performance tuning. Expertise in Azure Databricks, Apache Spark, Spark SQL, SQL Development, Python Programming, Apache Airflow (DAG Development), Delta Lake, Azure Data Lake Storage, ETL/ELT, Data Warehousing Concepts, and Data Modeling Fundamentals is crucial.