Azure Databricks Engineer
Capgemini
Capgemini
Join a global leader in technology transformation and help shape the future of data. Capgemini empowers its employees to grow their careers through collaborative work and innovative solutions. We are seeking an Azure Databricks Engineer to unlock the value of technology for leading organizations, building a more sustainable and inclusive world.
Design and develop robust end-to-end data pipelines utilizing Azure Data Factory and Databricks. Build and optimize ETL workflows for ingesting, transforming, and loading data into Azure Synapse Analytics and ADLS. Implement comprehensive data validation and quality frameworks to ensure data reliability. Write efficient PySpark and SQL code for complex transformations, aggregations, and performance tuning. Manage secure access protocols using Azure Key Vault and role-based permissions, adhering to compliance best practices. Collaborate within an Agile/Scrum environment to deliver solutions aligned with business objectives.
A minimum of 4 to 12 years of hands-on experience with Azure Databricks, complemented by strong PySpark expertise, is essential. Proficiency in Azure Data components, including Azure SQL Database, Azure SQL Warehouse, and Synapse Analytics, is required. Experience in Python/PySpark/Scala/Hive programming is a must. Familiarity with building CI/CD pipelines within data environments is also necessary.
Capgemini
Information Technology & Services