Seeking a seasoned Databricks Data Engineer with extensive experience in Databricks, PySpark, Azure Data Factory (ADF), and SQL. This role focuses on designing, building, and optimizing advanced data platforms and analytics solutions within the Azure ecosystem.

Core responsibilities include developing enterprise-grade data pipelines for efficient data ingestion from various sources, implementing complex data transformations, and supporting scalable analytical initiatives. The ideal candidate will possess a strong understanding of modern data engineering principles and best practices.

Architect and maintain robust data ingestion frameworks utilizing ADF, Databricks, and Spark.

Implement data transformations from Bronze to Silver layers according to architectural guidelines.

Design and integrate comprehensive data quality checks, schema validation, and contract rules.

Build and optimize high-throughput ELT/ETL pipelines for ingestion, transformation, and curation of diverse data types.

Integrate data seamlessly from on-premise, cloud systems, APIs, and third-party sources.

Develop scalable transformation scripts in PySpark on Databricks, leveraging advanced optimizations and Delta Lake features for ACID transactions, schema enforcement, and time travel.

Implement complex transformations, ensuring performance efficiency and code modularity.

Construct orchestration workflows in ADF, including pipelines, triggers, and integration runtimes.

Build dimensional models (star/snowflake schemas), fact tables, dimensions, and handle surrogate keys and SCDs.

Translate Silver datasets into analytics-ready models with consistent KPI definitions and business logic.

Ensure consistency across domains, promoting reusability and scalability of models.

Perform performance tuning on Databricks clusters, addressing bottlenecks in configuration, shuffle operations, joins, and parallelization.

Collaborate with platform teams on Databricks cluster management, jobs, notebooks, and CI/CD integrations.

Implement data quality checks, audit mechanisms, and validation frameworks within Unity Catalog.

Enforce Unity Catalog standards, including naming conventions, metadata policies, and access controls (RBAC/ABAC).

Manage changes like column derivation logic modifications, backward compatibility, and impact analysis.

Partner with business stakeholders and data stewards to define business rules and validate metrics.

Collaborate closely with IT and business teams to gather data requirements and deliver production-ready solutions.

Engage with architects, product owners, and cross-functional engineering teams to align technical delivery with business goals.

Provide mentorship and guidance to junior engineers as needed.