Seeking a skilled Databricks Data Engineer with strong proficiency in Databricks, PySpark, Azure Data Factory (ADF), and SQL. This role focuses on architecting, developing, and optimizing modern data platforms and analytical solutions within the Azure ecosystem.

Key responsibilities involve building robust enterprise-grade data pipelines, managing data ingestion from various sources, implementing complex data transformations, and supporting scalable analytics. The ideal candidate will drive innovation in data architecture and delivery.

Develop and maintain data ingestion frameworks using ADF, Databricks, and Spark. Implement data transformations from Bronze to Silver layers, ensuring alignment with architectural standards.

Architect comprehensive data quality checks, schema validation, and contract rules. Build and optimize high-throughput ELT/ETL pipelines for ingesting, transforming, and curating diverse data types.

Integrate data from on-premise, cloud-based systems, APIs, and third-party sources. Utilize PySpark for complex transformations, prioritizing performance and code modularity.

Construct ADF orchestration workflows, including pipelines, triggers, linked services, and parameterized datasets. Familiarity with Databricks Genie is beneficial.

Develop dimensional models (star/snowflake schemas), fact tables, dimensions, and handle surrogate keys and SCDs. Translate Silver datasets into analytics-ready models with consistent KPI definitions and business logic.

Ensure consistency across domains with common dimensions and conformed models, prioritizing reusability and scalability.

Engineer scalable transformation scripts using PySpark on Databricks, leveraging advanced optimizations like caching, partitioning, and Delta Lake features for ACID transactions, schema enforcement, evolution, and time travel.

Conduct performance tuning, addressing bottlenecks in cluster configuration, shuffle operations, joins, and parallelization. Collaborate with platform teams on Databricks cluster management, job scheduling, notebook organization, and CI/CD integration.

Implement data quality checks, audit mechanisms, and validation frameworks within Unity Catalog. Enforce Unity Catalog standards, including naming conventions, metadata policies, and access controls (RBAC/ABAC).

Manage changes such as column derivation logic updates, backward compatibility, and impact analysis. Partner with business stakeholders and data stewards to define business rules and validate metrics.

Collaborate closely with IT and business teams to understand data requirements and deliver production-ready solutions. Work with architects, product owners, and cross-functional engineering teams to align technical delivery with business objectives. Mentor junior engineers as needed.