Senior Technical Lead (IN)

HCLTech

7+ yrs India Full Time Remote
HCLTech logo
Posted : today
Actively hiring

Job description

We are seeking a seasoned Databricks Platform Administrator with over 7 years of IT experience to manage, secure, and optimize Databricks environments. This role focuses on administering Databricks workspaces, clusters, security, governance, performance, and integrations across major cloud platforms like Azure, AWS, and GCP.

Enterprise needs highlight a strong demand for expertise in Databricks Lakehouse, Unity Catalog, governance, monitoring, and cloud administration.

Responsibilities

Key responsibilities include administering Databricks workspaces, configuring cluster policies, and managing resource allocation.

Implement and maintain Unity Catalog for robust data governance and access control. Set up RBAC, SSO, SCIM provisioning, and identity integrations with Azure AD/Entra ID or equivalent.

Monitor platform usage, performance, costs, and system health, while also performing platform upgrades, patching, and capacity planning. Secure connectivity through network configuration, VNet injection, and Private Link.

Manage secrets, key vault integrations, and data security controls. Support CI/CD pipelines using tools like Azure DevOps, GitHub Actions, or Jenkins. Troubleshoot various platform and job execution issues, and develop operational dashboards and alerts to ensure compliance with security and governance standards.

Qualifications

This position requires 7+ years of IT experience, with a minimum of 3 years dedicated to Databricks Administration. Deep experience is needed in Databricks Workspace Administration, Cluster Management, Unity Catalog, Databricks SQL, Delta Lake, Job & Workflow Management, and MLflow Administration.

Proficiency with cloud platforms such as Azure (preferred), AWS, and GCP is essential. A strong understanding of Python, PySpark, SQL, and REST APIs is also required, alongside experience in implementing platform security, governance, and monitoring solutions.

Familiarity with CI/CD tools like Jenkins, GitHub, and Azure DevOps is necessary. Knowledge of Infrastructure as Code, with a preference for Terraform, is highly valued. Preferred qualifications include Databricks certifications, FinOps experience, and exposure to MLOps and AI platform administration.

Essential Skills

DatabricksDatabricks Workspace AdministrationCluster ManagementUnity CatalogDatabricks SQLDelta LakeJob & Workflow ManagementMLflow AdministrationAzureAWSGCPPythonPySparkSQLREST APIsPlatform SecurityData GovernanceMonitoringCI/CDJenkinsGitHub ActionsAzure DevOpsInfrastructure as CodeTerraform

Good to Have

Databricks Certified Data Engineer Associate/ProfessionalFinOpsCloud Cost OptimizationMLOpsAI Platform AdministrationData LineageCompliance Frameworks

Highlights

  • Actively hiring

More Details

RoleSenior Technical Lead (IN)
Employment TypeFull Time, Remote

About the Company

HCLTech logo

HCLTech

IT Consulting

Senior Technical Lead (IN) at HCLTech | SkillMX | SkillMX