Azure Databricks Lead

Birlasoft

6–10 yrs Noida, Mumbai, Pune, Bengaluru, Hyderabad, Chennai Full Time Hybrid (office + remote)
Birlasoft logo
Posted : 1 week ago
Actively hiring

Job description

Seeking an experienced Azure Databricks Lead to drive the design, development, and deployment of innovative data solutions. This role focuses on leveraging Azure services to build robust and scalable data pipelines, integrating diverse data sources to meet growing data demands.

The ideal candidate will have a strong background in modern data warehousing techniques, utilizing the Azure stack including Azure Data Lake and Azure Databricks, coupled with expertise in PySpark for efficient data processing. This position offers an exciting opportunity to contribute to cutting-edge data projects in a collaborative environment.

Responsibilities

Lead the end-to-end design, development, and deployment of data solutions using Azure Services, with a focus on Databricks and PySpark.

Develop and maintain scalable data pipelines and create new integrations for increasingly complex and voluminous data.

Design and implement modern data warehouse solutions leveraging Azure Data Lake and Azure Databricks, utilizing PySpark for processing.

Develop batch processing and integration solutions, effectively processing both structured and non-structured data.

Create detailed technical specifications and application interface designs.

Qualifications

A minimum of 6 to 10 years of professional experience is required, with a proven track record in designing, developing, and deploying solutions using Azure Services such as Databricks, PySpark, SQL, and Data Factory.

Demonstrated expertise in building scalable data pipelines and integrating new data sources is essential.

Candidates must possess in-depth skills in Databricks, PySpark, and SQL, along with experience in data modeling.

Experience in creating technical specifications and application interface designs is a key requirement.

Familiarity with developing modern data warehouse solutions using Azure Stack (Azure Data Lake, Azure Databricks) and PySpark is expected.

Ability to collaborate effectively with BI, analytics, and business teams is crucial for success in this role.

Essential Skills

Azure DatabricksPySparkSQLData ModellingData FactoryAzure Data LakeBatch Processing

Highlights

  • Actively hiring

More Details

RoleAzure Databricks Lead
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Birlasoft logo

Birlasoft

IT Consulting

Azure Databricks Lead at Birlasoft | SkillMX | SkillMX