Databricks - Technical Lead-Data Engg
Birlasoft
Birlasoft
Join our dynamic team as a Technical Lead specializing in Data Engineering. This role focuses on designing, developing, and deploying robust data solutions using cutting-edge Azure technologies. You will be instrumental in shaping our data infrastructure and driving data-driven innovation.
We seek an experienced professional to lead the charge in building and optimizing scalable data pipelines. Your expertise will be crucial in integrating new data sources and ensuring our systems can handle increasing data volumes and complexity with efficiency.
Key responsibilities include performing design, development, and deployment using Azure Services like Databricks, PySpark, SQL, and Data Factory. You will develop and maintain scalable data pipelines, build new data source integrations, and create technical specification designs.
This role also involves developing modern data warehouse solutions leveraging Azure Stack (Azure Data Lake, Azure Databricks) and PySpark. You will develop batch processing and integration solutions, adeptly handling both structured and non-structured data. Collaboration with BI & analytics teams and business stakeholders is essential.
A minimum of 2 to 3 years of project experience with Azure Databricks is required. Additionally, a minimum of 2-3 years of experience in Azure Data Factory (ADF) and a minimum of 2+ years of experience with PySpark are essential qualifications for this position.
Demonstrated in-depth skills with Databricks and PySpark, along with strong SQL proficiency, are critical. The ability to collaborate effectively with BI & analytics teams and business stakeholders is also a key requirement.
Birlasoft
IT Consulting