Senior Engineer - Data Engineering

Altimetrik

10+ yrs Hyderabad Full Time Remote
Altimetrik logo
Posted : 1 week ago
Actively hiring

Job description

Altimetrik, a digital business services company, is seeking a seasoned L4 Databricks Platform Administrator with extensive AWS expertise. This role is central to ensuring the robust performance, security, and scalability of our Databricks environments.

You will be instrumental in supporting our data science and engineering teams, providing advanced administration, troubleshooting, and optimization for the Databricks platform.

Responsibilities

Key responsibilities include administering Databricks workspaces on AWS, managing user access, configuring clusters, and optimizing resource allocation. You will monitor platform performance, implement best practices for governance, and manage Databricks SQL warehouses and jobs.

Seamless integration with AWS services like S3, IAM, EC2, and VPC is crucial. This involves configuring networking and security, optimizing data storage, and adhering to AWS security best practices for Databricks environments.

You will implement and maintain robust security policies, ensure compliance with regulations, and manage access controls. Proactive troubleshooting of platform issues, performance tuning, and supporting data teams are also core duties.

Developing automation scripts, implementing infrastructure-as-code using Terraform or CloudFormation, and establishing CI/CD pipelines for Databricks deployments will be key. Collaboration with data engineers, scientists, and AWS/Databricks support teams is essential, along with participation in on-call rotations.

Qualifications

A Bachelor's degree in Computer Science, Information Technology, or a related field is required, alongside a minimum of 10 years of experience in data platform administration, with a strong emphasis on Databricks on AWS.

Extensive experience with AWS services and architecture, cluster management, performance tuning, and security best practices is expected. Proficiency in infrastructure-as-code tools like Terraform and experience with CI/CD pipelines are also necessary.

Technical skills should include strong Databricks administration capabilities, deep knowledge of AWS services, and proficiency in scripting languages such as Python and Bash. Familiarity with SQL, data warehousing concepts, data governance, security principles, and Spark technologies is also vital.

Excellent problem-solving, analytical, communication, and collaboration skills are a must. The ability to work independently, manage multiple priorities, and maintain a high level of attention to detail is crucial for success in this role. Preferred certifications include AWS Certified Solutions Architect and Databricks certifications.

Essential Skills

DatabricksAWSPythonTerraformCI/CDSQLBashSpark

Good to Have

ApacheAzure DevOpsData LineagePower BIAgile

Highlights

  • Actively hiring

More Details

RoleSenior Engineer - Data Engineering
DepartmentData Engineering
Employment TypeFull Time, Remote

About the Company

Altimetrik logo

Altimetrik

IT Consulting