A leading Assurance team is searching for a Databricks/Data Engineer Lead to architect and implement robust, scalable data pipelines and analytics solutions on Azure. This role is integral to the Edward Jones engagement, requiring expert-level Databricks, Spark, Python, and Azure data engineering expertise.

Join a dynamic team focused on leveraging data to drive impactful insights and solutions within the Sustainability & Emerging Assurance domain. This is an excellent opportunity to shape data strategies and mentor emerging talent.

Design, develop, and maintain advanced Databricks-based data pipelines and transformation workflows. Build highly scalable Spark applications tailored for large-scale data processing and sophisticated analytics. Develop robust Python-based data engineering solutions for both batch and real-time streaming workloads. Architect and implement comprehensive data solutions utilizing Azure Data Factory, Azure Synapse, or comparable Azure services. Champion data quality, ensure lineage tracking, and maintain governance across all data products. Implement and manage DevOps practices for seamless data pipeline deployment, continuous monitoring, and automation. Foster collaboration with data scientists, ML engineers, and key business stakeholders. Provide mentorship to junior data engineers and actively contribute to establishing technical standards and best practices.

A minimum of 6 years of dedicated data engineering experience is essential. Candidates with an MCA & BSc background are highly preferred for this role. Demonstrate advanced proficiency in Databricks, including notebooks, Delta Lake, and Unity Catalog. Possess advanced skills in Databricks Spark development, specifically PySpark and Spark SQL. Showcase advanced proficiency in Python programming for data engineering tasks. Exhibit advanced knowledge and practical application of Azure data services like Azure Data Factory, Synapse, and ADLS. Maintain intermediate or higher proficiency in DevOps tools and Continuous Integration/Continuous Deployment (CI/CD) practices. Solid understanding and practical experience with data modeling, data warehousing concepts, and ETL/ELT patterns. Prior experience within the Financial Services domain is considered a significant advantage.