Azure Administration
Infosys
Infosys
Join Infosys as a Consultant in our Data & Analytics Unit, focusing on Azure Databricks administration. This role is key to managing, operating, and optimizing enterprise Azure Databricks platforms. You will ensure the high availability, security, performance, and cost-efficiency of our data platforms, supporting critical data engineering, analytics, and AI/ML initiatives.
We seek a proactive individual with extensive experience in Azure Databricks, Azure infrastructure, and robust security practices. You'll collaborate closely with cloud, data, and business teams to build and maintain a stable, scalable data ecosystem. This is an excellent opportunity to contribute to digital transformation at a global scale.
Key responsibilities include administering Azure Databricks workspaces across development, QA, and production environments. This involves configuring and managing clusters, including autoscaling and auto termination. You will also set up environments, perform upgrades, and ensure the operational stability of Databricks services.
Integration with various Azure services such as ADLS Gen2, Azure SQL, Synapse, and Event Hub is crucial. You'll manage VNet injected workspaces, private endpoints, and secure network configurations. User and access management using Azure Active Directory, along with implementing data governance via Unity Catalog, will be integral to your role.
Cost optimization strategies, including cluster right-sizing and instance pool configuration, are essential. You'll monitor usage, provide cost reports, and analyze workload patterns for performance tuning. Deploying and managing infrastructure using Terraform or ARM templates, supporting CI/CD pipelines, and automating operational tasks will streamline our processes.
Troubleshooting cluster failures, job issues, and performance bottlenecks, followed by root cause analysis, is a significant part of this role. Providing Level 2/3 support in a 24x7 model and maintaining comprehensive platform documentation are also expected. Collaboration with data engineers, data scientists, and security teams for continuous improvement is vital.
The ideal candidate possesses 5+ years of experience in data platform or cloud administration, with a strong emphasis on Azure Databricks. Hands-on experience with core Azure services like ADLS Gen2, Azure Networking, Azure Monitor, and Azure AD is mandatory.
A solid understanding of Apache Spark architecture and performance tuning is required. Proficiency in Python, PowerShell, or Bash scripting, along with experience in Infrastructure as Code tools such as Terraform or ARM templates, is essential for success in this role.
Valuable additions include experience with Unity Catalog, data lineage, Delta Lake, and CI/CD pipelines using Azure DevOps or GitHub Actions. Familiarity with FinOps practices in Azure and relevant certifications (Azure Databricks, Microsoft Azure Administrator/Solutions Architect) are highly beneficial.
We are looking for individuals with strong analytical and troubleshooting skills, excellent communication abilities, and the capacity to work independently in a dynamic, 24x7 support environment. A proactive, detail-oriented approach focused on operational excellence is key.
Infosys
IT Consulting