Seeking a skilled Databricks Consultant to drive data modernization and migration initiatives. This role focuses on developing and implementing robust data engineering solutions leveraging the Azure Databricks platform. You will be instrumental in designing, building, and maintaining scalable data pipelines, transforming data with Spark, and ensuring optimal data structure within Delta Lake environments for advanced analytics.

This is a hybrid role based in Mumbai, offering a dynamic environment for impactful data solutions.

Design and implement scalable ETL/ELT pipelines using Azure Databricks, orchestrating workflows with Azure Data Factory. Develop and maintain data processing solutions using PySpark and Spark SQL within Databricks notebooks. Manage and optimize data within Delta Lake, leveraging Delta tables and schema enforcement for data integrity. Integrate diverse data sources, including SQL Server, Oracle, and others, into the Azure Databricks ecosystem. Contribute to data cataloging and documentation for enhanced data discoverability. Troubleshoot and resolve complex issues related to Spark jobs, data pipelines, and data quality.

A minimum of 3 to 5 years of dedicated experience in data engineering is essential, with a strong focus on Azure Databricks. Demonstrated proficiency in PySpark or Spark SQL for efficient data transformation. Hands-on experience with Azure Data Lake Storage (ADLS Gen2) and Azure Data Factory (ADF). Solid understanding of SQL and experience with relational databases such as SQL Server, Oracle, or PostgreSQL. Excellent problem-solving, analytical, and communication skills are crucial for success in agile, cross-functional team environments.