Senior Technical Lead (IN)
HCLTech
HCLTech
Seeking a Senior Technical Lead with expertise in Databricks and Azure to architect, develop, and maintain a robust and secure data platform. This role operates at the nexus of infrastructure, data engineering, and cloud automation, crucial for establishing an enterprise-grade platform supporting analytics and AI initiatives.
This position requires a proactive approach to cloud infrastructure management and a deep understanding of how to leverage Databricks and Azure services for scalable data solutions. The ideal candidate will drive innovation and efficiency in our data platform operations.
Key responsibilities include automating the provisioning of Databricks and Azure resources via Terraform, following Infrastructure-as-Code (IaC) principles. You will design and build modular Terraform components for scalable deployments and implement CI/CD pipelines using Azure DevOps for automated infrastructure management.
Additionally, you will provision new infrastructure using Databricks Terraform modules for standardized, automated, and governed deployments of catalogs, job policies, and clusters. Enabling self-service infrastructure provisioning through ServiceNow integration is also a core task. Troubleshooting platform issues, monitoring Azure resource limits and cluster performance, and managing FinOps integration by onboarding Databricks cost data are critical functions. Staying abreast of industry best practices, especially concerning Terraform and its Azure/Databricks providers, is essential.
A degree in Computer Science, Information Technology, or a related field is required, coupled with proven experience as a Platform Engineer or Cloud Engineer. Strong proficiency in Microsoft Azure and its services, along with extensive knowledge of Infrastructure-as-Code (IaC) using Terraform, is necessary. Familiarity with the Databricks Provider and experience setting up Azure Data Lake Gen2 with Terraform are highly valued.
Knowledge in data management, including archiving and backup, and experience with system interface setup and authentication mechanisms are important. Excellent problem-solving, communication, and collaboration skills are essential for working effectively in a fast-paced environment. Fluency in English is mandatory; German language skills are considered a plus. Certifications in Microsoft Azure or Databricks are welcomed.
HCLTech
IT Consulting