Seeking a skilled Databricks Data Engineer with expertise in Databricks, PySpark, ADF, and SQL. This role focuses on architecting, developing, and optimizing modern data platforms and analytical solutions within the Azure environment. Key to this position is a proven track record in building enterprise-grade data pipelines, facilitating data ingestion from various sources, implementing complex transformations, and supporting scalable analytics initiatives.

Key responsibilities include executing data engineering tasks such as building ingestion frameworks (ADF/Databricks/Spark), implementing Bronze -> Silver transformations, and designing data quality checks. You will construct and optimize robust ELT/ETL pipelines, integrate data from diverse on-premise and cloud systems, and develop complex transformations using PySpark. Building orchestration workflows in ADF and familiarization with Databricks Genie are also essential.

Further duties involve designing and implementing curated data models in the Gold layer, including dimensional models and fact/dimension tables. This requires translating Silver datasets into analytics-ready models with consistent KPI definitions and business logic, ensuring cross-domain consistency and model reusability.

Expertise in Databricks and PySpark engineering is crucial, involving the development of scalable transformation scripts with advanced optimizations and the implementation of Delta Lake features. Performance tuning and collaboration with platform teams for cluster management are also part of this role.

Ensuring data governance and quality through the implementation of data quality checks, audit mechanisms, and validation frameworks within Unity Catalog is vital. This includes enforcing Unity Catalog standards, naming conventions, metadata policies, and access controls.

Finally, you will collaborate closely with IT and business teams to understand data requirements, deliver production-ready solutions, and align technical delivery with business objectives. Mentoring junior engineers will also be part of your responsibilities.

This role requires hands-on expertise in Databricks, PySpark, Azure Data Factory (ADF), and SQL. Experience in building enterprise-level data pipelines, ingesting data from diverse sources, and implementing complex transformations is necessary. Strong skills in developing scalable transformation scripts using PySpark on Databricks, including advanced optimizations and Delta Lake features, are essential.

Familiarity with Azure Data Factory workflows, including pipelines, triggers, and linked services, is expected. Proficiency in designing and implementing curated data models, such as dimensional models (star/snowflake schemas), fact tables, and dimensions, is also required.

Knowledge of data governance principles, including data quality checks, audit mechanisms, and validation frameworks, particularly within Unity Catalog, is important. Experience with enforcing naming conventions, metadata policies, and access controls (RBAC/ABAC) is necessary.

Candidates should possess the ability to collaborate effectively with customer IT and business teams, understand data requirements, and deliver reliable solutions. Experience in performance tuning for Databricks clusters and Spark jobs is also beneficial. Guidance and mentoring of junior engineers would be an advantage.