Seeking an experienced Business Intelligence Senior Specialist with 9-11 years of expertise in SSIS, SQL Server, Azure Databricks, and PySpark. This role requires a hands-on Tech Lead to spearhead the modernization and scaling of our enterprise data platform. You will provide strong technical leadership while actively contributing through coding. Key responsibilities include leading the migration, integration, and development of data pipelines across legacy SQLServer/SSIS environments and modern cloud infrastructure built on Azure Databricks and PySpark. You'll serve as the principal architect and developer for end-to-end ELT/ETL pipelines, ensuring optimal performance, robust data governance, and scalability.

Lead the transition of legacy SQL Server Integration Services (SSIS) packages to contemporary, cloud-native Azure Databricks / PySpark Lakehouse environments. Architect and implement end-to-end Modern Data Warehouse (MDW) and Medallion architecture systems (Bronze, Silver, Gold layers) within Azure. Mentor and guide a team of data engineers, performing rigorous code reviews, enforcing design patterns, and establishing development standards. Collaborate effectively with stakeholders, including Data Architects, Business Analysts, and DevOps teams, to translate complex business requirements into detailed technical blueprints.

Develop and optimize high-performance, scalable batch and real-time ETL/ELT pipelines using PySpark on Azure Databricks. Maintain and refactor existing on-premise SSIS packages and complex T-SQL code, including stored procedures and views, ensuring business continuity during transition. Implement Delta Lake features such as transaction management, schema evolution, and time-travel querying. Orchestrate workflows by integrating Azure Data Factory (ADF) with on-premise databases, cloud storage (ADLS Gen2), and Databricks activities.

Optimize Spark configurations and cluster utilization to significantly reduce cloud compute costs and job execution times. Enforce data quality frameworks with automated validation, error reconciliation, and operational audit logging. Implement CI/CD automations using Azure DevOps or GitHub Actions for environment management, code repositories (Git), and deployment pipelines.

Requires deep, production-level knowledge of the Azure Big Data Stack, including Azure Databricks, Delta Lake, Unity Catalog, and Azure Data Lake Storage (ADLS Gen2). Possess advanced proficiency in Python and PySpark core APIs (Spark SQL and DataFrames). Mastery of SQL Server, advanced T-SQL writing, and extensive experience configuring, deploying, and troubleshooting SSIS packages are essential. Hands-on experience with Azure Data Factory (ADF) for dependency mapping and scheduling is required. An expert understanding of relational (OLTP) and dimensional (OLAP) schemas, particularly Star and Snowflake architectures, is necessary. Competency with Git branching, CI/CD automated release deployments, and secure data handling mechanisms (e.g., Azure Key Vault, network isolation) is expected.

Demonstrated track record as a coding lead, balancing management/design with hands-on execution. Excellent cross-functional communication skills, capable of discussing data strategies with both executives and junior technical staff. Strong debugging and root-cause analysis (RCA) skills for multi-tier platform failures are critical. Preferred certifications include Microsoft Certified: Azure Data or Databricks.