Databricks Administration

Infosys

3–5 yrs Bengaluru Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Join Infosys as a Senior Associate Consultant, contributing to our Data & Analytics Unit. This role focuses on administering and optimizing enterprise-level Databricks platforms to support large-scale data analytics, AI, and ML initiatives. You will play a key role in platform governance, security, automation, and cost management, ensuring reliability and compliance.

As a seasoned professional, you will be instrumental in defining and implementing best practices for Databricks administration. This includes managing multi-workspace environments, enforcing platform standards, and leading governance strategies for critical components like Unity Catalog. Your expertise will directly impact the efficiency and effectiveness of our data operations.

This position offers a dynamic environment where you can leverage your technical skills and leadership abilities. You will have the opportunity to mentor junior team members and act as a senior technical point of contact, fostering a culture of continuous learning and improvement within the team.

Responsibilities

Oversee and manage Databricks accounts and multi-workspace environments across the entire SDLC, from development to production. Define and implement robust platform standards for clusters, jobs, SQL warehouses, and workspace configurations. Lead the Unity Catalog governance strategy, managing catalogs, schemas, permissions, and external data locations. Design and deploy enterprise monitoring, alerting, and compliance dashboards, ensuring operational service level objectives are met. Manage high-severity incidents, conduct thorough root-cause analyses, and implement preventive measures. Establish and maintain security, audit, and regulatory compliance controls for the Databricks platform. Drive cost governance, capacity forecasting, and performance optimization for all data workloads. Develop and lead Infrastructure-as-Code strategies using Terraform, APIs, and CI/CD pipelines for automation. Standardize CI/CD practices, Databricks Asset Bundles, and DevOps operating models. Collaborate with cloud platform, security, and data engineering teams on architectural and governance decisions. Develop and maintain platform documentation, onboarding standards, and the operational maturity roadmap.

Qualifications

Possess 2-5 years of dedicated experience in Databricks Administration or Big Data Platforms. Demonstrate strong, hands-on experience with Databricks functionalities including Workspace, Clusters, Jobs, and Notebooks. Exhibit experience with Apache Spark, utilizing PySpark or Scala for distributed data processing. Showcase a solid understanding of cloud platforms, with a preference for Azure, but AWS and GCP also considered. Familiarity with data storage systems such as ADLS, S3, and Blob Storage is essential. Possess a good grasp of data engineering concepts and ETL/ELT pipeline design. Be familiar with security best practices, including IAM roles, data encryption, and private endpoints. Demonstrate proficiency in SQL and scripting languages like Python and Shell. Have experience in performance tuning and cost optimization for Databricks clusters and workloads. A Bachelor of Engineering degree is required for this role.

Essential Skills

DatabricksPythonPySparkSpark SQLAWSAzureGCPUnity CatalogDelta Live TablesServerless SQLTerraformCI/CDDevOpsApache SparkADLSS3Blob StorageETLELTSQLShell Scripting

Good to Have

Scala

Highlights

  • Actively hiring

More Details

RoleDatabricks Administration
IndustryInformation Technology & Services, Data Analytics, AI / Machine Learning
DepartmentData Science, Cloud Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

Information Technology & Services

Databricks Administration at Infosys | SkillMX | SkillMX