Azure Databricks Engineer

Capgemini

4–12 yrs Bengaluru, Chennai, Pune, Mumbai Full Time Hybrid (office + remote)
Capgemini logo
Posted : today
Actively hiring

Job description

Join a global leader in technology transformation and help shape the future of data. Capgemini empowers its employees to grow their careers through collaborative work and innovative solutions. We are seeking an Azure Databricks Engineer to unlock the value of technology for leading organizations, building a more sustainable and inclusive world.

Responsibilities

Design and develop robust end-to-end data pipelines utilizing Azure Data Factory and Databricks. Build and optimize ETL workflows for ingesting, transforming, and loading data into Azure Synapse Analytics and ADLS. Implement comprehensive data validation and quality frameworks to ensure data reliability. Write efficient PySpark and SQL code for complex transformations, aggregations, and performance tuning. Manage secure access protocols using Azure Key Vault and role-based permissions, adhering to compliance best practices. Collaborate within an Agile/Scrum environment to deliver solutions aligned with business objectives.

Qualifications

A minimum of 4 to 12 years of hands-on experience with Azure Databricks, complemented by strong PySpark expertise, is essential. Proficiency in Azure Data components, including Azure SQL Database, Azure SQL Warehouse, and Synapse Analytics, is required. Experience in Python/PySpark/Scala/Hive programming is a must. Familiarity with building CI/CD pipelines within data environments is also necessary.

Essential Skills

Azure DatabricksPySparkSQLDelta LakeAzure Data FactoryAzure Data LakeETLAzure Synapse AnalyticsADLSAzure Key VaultAgileScrumAzure SQL DatabaseAzure SQL WarehouseScalaHiveCI/CD

Highlights

  • Actively hiring

More Details

RoleAzure Databricks Engineer
IndustryInformation Technology & Services
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Capgemini logo

Capgemini

Information Technology & Services

Azure Databricks Engineer at Capgemini | SkillMX | SkillMX