Seeking a highly skilled Data Engineer Advisor with 8-10 years of experience to lead the modernization and scaling of our enterprise data platform. This role requires a deep technical understanding and hands-on coding ability, focusing on driving initiatives related to SSIS, SQL Server, Azure Databricks, and PySpark. You will provide technical leadership, manage data pipeline migrations, integrations, and development, and architect end-to-end ELT/ETL processes for optimal performance and scalability.

This is a key position to guide our transition to modern cloud infrastructure while ensuring seamless operation of existing systems.

Lead the migration of legacy SSIS packages to Azure Databricks/PySpark Lakehouse environments. Architect Modern Data Warehouse (MDW) and Medallion architecture systems in Azure. Mentor data engineers, conduct code reviews, and establish development standards. Collaborate with stakeholders to translate business needs into technical designs. Develop and optimize batch and real-time ETL/ELT pipelines using PySpark on Azure Databricks. Maintain and refactor existing SSIS packages and T-SQL code. Implement Delta Lake features for transaction management, schema evolution, and time-travel. Orchestrate workflows using Azure Data Factory (ADF) with various data sources. Optimize Spark configurations for reduced cloud compute costs. Enforce data quality frameworks and implement CI/CD using Azure DevOps or GitHub Actions.

Requires 8-10 years of experience with a strong background in SSIS, SQL Server, Azure Databricks, and PySpark. Advanced proficiency in Python and PySpark core APIs (Spark SQL, DataFrames). Mastery of SQL Server, advanced T-SQL, and extensive SSIS package experience. Hands-on experience with Azure Data Factory (ADF) for orchestration. Expert understanding of relational and dimensional data modeling (Star, Snowflake). Competency with Git, CI/CD, and secure data handling practices (Azure Key Vault). Proven ability to split time between technical execution and leadership/design. Excellent communication skills for executive and technical audiences. Strong debugging and root-cause analysis skills are essential.