Seeking a highly skilled Data Engineer Advisor with 8-10 years of experience to spearhead the modernization and scaling of our enterprise data platform. This role requires a hands-on technical lead capable of driving innovation while actively contributing code. You will lead the migration, integration, and development of data pipelines, transitioning from legacy SQL Server/SSIS environments to our modern Azure Databricks and PySpark cloud infrastructure.

As the principal architect and developer for end-to-end ELT/ETL pipelines, you will ensure optimal performance, robust data governance, and seamless scalability. Your expertise will be crucial in shaping the future of our data capabilities.

Key responsibilities include leading the migration of legacy SSIS packages to cloud-native Azure Databricks/PySpark Lakehouse environments and architecting Modern Data Warehouse (MDW) and Medallion architecture systems within Azure.

You will mentor and guide a team of data engineers, conducting thorough code reviews and establishing development standards. Collaboration with stakeholders, including Data Architects, Business Analysts, and DevOps, will be essential for translating business needs into technical solutions.

Develop and optimize high-performance ETL/ELT pipelines using PySpark on Azure Databricks, and maintain existing SSIS packages and T-SQL code. Implement Delta Lake features, manage data quality frameworks, and enforce CI/CD automations using Azure DevOps or GitHub Actions.

Required expertise includes 8-10 years of hands-on experience with SSIS, SQL Server, Azure Databricks, and PySpark. Advanced proficiency in Python and PySpark core APIs is essential, alongside mastery of SQL Server, advanced T-SQL writing, and extensive SSIS configuration and troubleshooting.

Experience with Azure Data Factory for orchestration, deep understanding of relational and dimensional data modeling (Star and Snowflake architectures), and competency with Git, CI/CD, and secure data handling mechanisms are mandatory.

We are looking for a proven coding lead who can balance management and hands-on execution. Excellent communication skills for cross-functional collaboration and strong debugging/RCA abilities are vital. Preferred candidates will hold Azure Data or Databricks certifications.