Databricks + PySpark - Technical Lead-Data Engg
Birlasoft
Birlasoft
Seeking a skilled Technical Lead with expertise in Data Engineering to drive the design, development, and deployment of data solutions.
This role focuses on leveraging Azure services like Databricks and PySpark to build and maintain scalable data pipelines and integrations.
The ideal candidate will possess strong analytical skills and a passion for modern data warehousing concepts.
Design, develop, and deploy data solutions using Azure services including Databricks, PySpark, SQL, and Data Factory.
Construct and manage robust data pipelines, integrating new data sources to accommodate growing data volumes and complexity.
Create detailed technical and application interface specifications.
Build modern data warehouse solutions utilizing Azure Stack components like Azure Data Lake and Azure Databricks, coupled with PySpark.
Develop batch processing and integration solutions, adept at handling both structured and non-structured data.
Collaborate effectively with BI and analytics teams, as well as business stakeholders.
Possess a minimum of 2-3 years of project experience with Azure Databricks.
Have at least 2-3 years of experience working with Azure Data Factory (ADF).
Demonstrate 2+ years of hands-on experience with PySpark.
Exhibit in-depth proficiency in Databricks, PySpark, and SQL.
Birlasoft
IT Consulting