Seeking a skilled Azure Databricks Developer to join our team. This role focuses on building and optimizing data pipelines and transformations within the Azure ecosystem.

Leverage your expertise in Databricks and PySpark to develop scalable and efficient data solutions. You will be instrumental in creating robust data models and ensuring data quality and consistency.

Develop and maintain data ingestion frameworks using ADF and Databricks. Implement data transformations from Bronze to Silver layers, adhering to architectural standards. Architect and implement data quality checks, schema validation, and contract rules. Build and optimize high-throughput ELT/ETL pipelines for data ingestion, transformation, and curation. Integrate data from various on-premise and cloud systems, APIs, and third-party sources. Utilize PySpark for complex transformations, prioritizing performance and modularity. Construct orchestration workflows in ADF, including pipelines, triggers, and linked services. Develop dimensional models (star/snowflake schemas), fact tables, and dimensions, including SCD handling. Translate Silver datasets into analytics-ready models with consistent KPI definitions and business logic. Ensure consistency, reusability, and scalability of data models across domains.

Demonstrated experience in building and maintaining ingestion frameworks with ADF and Databricks. Proficiency in implementing data transformations and data quality measures. Strong ability to architect and optimize ELT/ETL pipelines for structured, semi-structured, and unstructured data. Experience integrating data from diverse sources, including cloud systems and APIs. Advanced skills in PySpark for developing scalable transformation scripts on Databricks. Expertise in leveraging Delta Lake features such as ACID transactions, schema enforcement, and time travel. Proven ability in performance tuning Databricks jobs and handling performance bottlenecks. Familiarity with Databricks Genie is a plus. Experience with CI/CD integrations for Databricks jobs and notebooks is required.

This role is based at our Birlasoft Office in Hinjawadi, Pune, India.