Tech Lead
NTT DATA
NTT DATA
Seeking a highly experienced and deeply technical Data Tech Lead with 9 to 11 years in SSIS, SQL Server, Azure Databricks, and PySpark. This role is pivotal in modernizing and scaling our enterprise data platform. You will provide technical leadership while remaining hands-on with coding, driving the migration, integration, and development of data pipelines across legacy and cloud environments. As the principal architect and developer for end-to-end ELT/ETL processes, you'll ensure optimal performance, robust data governance, and seamless scalability.
This is an opportunity to shape the future of our data infrastructure, bridging existing systems with cutting-edge cloud solutions. You will be instrumental in transforming our data capabilities and empowering data-driven decision-making across the organization.
Lead the transition of legacy SQL Server Integration Services (SSIS) packages to modern, cloud-native Azure Databricks / PySpark Lakehouse environments. Architect and implement end-to-end Modern Data Warehouse (MDW) and Medallion architecture systems (Bronze, Silver, Gold layers) on Azure. Mentor a team of data engineers through rigorous code reviews and the establishment of development standards. Collaborate with stakeholders, including Data Architects and Business Analysts, to translate complex business needs into actionable technical blueprints.
Develop and optimize high-performance batch and real-time ETL/ELT pipelines using PySpark on Azure Databricks. Maintain and refactor existing SSIS packages and complex T-SQL code, ensuring business continuity. Implement Delta Lake features for enhanced data management. Orchestrate workflows using Azure Data Factory (ADF) to integrate on-premise databases with cloud storage and Databricks. Optimize Spark configurations to reduce cloud compute costs and improve job execution times. Enforce data quality frameworks and implement CI/CD automations for streamlined development and deployment.
Must possess 9 to 11 years of hands-on experience with SSIS, SQL Server, Azure Databricks, and PySpark. Advanced proficiency in Python and PySpark core APIs is essential. Mastery of SQL Server, advanced T-SQL, and extensive experience with SSIS package configuration and troubleshooting are required. Solid experience with Azure Data Factory (ADF) for orchestration is necessary. Expert understanding of relational and dimensional data modeling, including Star and Snowflake architectures, is key. Competency with Git branching, CI/CD, and secure data handling mechanisms is expected. Proven ability to balance management, design, and hands-on coding is crucial. Excellent cross-functional communication and strong root-cause analysis skills are vital.
NTT DATA
IT Consulting