Join EY and contribute to building a better working world. EY's Consulting Services offers a unique, industry-focused environment where you can leverage deep industry experience with strong functional and technical capabilities. Our Data and Analytics team tackles complex issues and capitalizes on opportunities to deliver enhanced business outcomes for clients across financial services and capital markets.

This Senior Data & AI Architect role offers a compelling opportunity to blend deep technical ownership of enterprise Lakehouse and data platform architecture with hands-on experience in building Agentic AI and RAG solutions on Databricks. The ideal candidate will excel at designing large-scale, production-grade data pipelines, architecting modern AI-driven solutions, and engaging directly with clients on strategic proposals and solution design.

Architect and deliver scalable Data & AI solutions on the Databricks Lakehouse platform, encompassing ingestion, transformation, storage, governance, and consumption. Design and optimize large-scale distributed data processing pipelines using Hadoop, Apache Spark, and PySpark. Build reusable, production-grade Python frameworks for batch and streaming data engineering. Define and implement cloud data architecture on AWS/Azure, integrating with native Databricks services. Own end-to-end ETL/ELT processes, including ingestion, transformation, orchestration, and data quality. Design dimensional data models to support analytical use cases. Build Agentic AI solutions on Databricks, including LLM integration and enterprise data integration. Architect and implement RAG solutions on Databricks. Establish and enforce engineering best practices, including Git-based version control and CI/CD. Lead performance optimization and production deployment of Databricks jobs. Participate in strategic RFP/RFI responses and technical proposals. Act as a trusted technical advisor in client workshops, translating business needs into robust technical architectures. Mentor engineering teams on Databricks, Spark, cloud architecture, and AI/RAG practices.

Possess over 13 years of experience in Data Engineering/Data Architecture, with robust hands-on expertise in Databricks, Hadoop, Spark/PySpark, and cloud data platforms. Demonstrate strong command of Databricks Lakehouse, Delta Lake, Databricks SQL, and Unity Catalog, including Jobs/Workflows and production deployments. Exhibit deep knowledge of the Hadoop ecosystem and HDFS, coupled with extensive Apache Spark and PySpark experience for large-scale distributed data processing. Showcase strong Python proficiency for building scalable, production-grade data pipelines. Possess significant experience with AWS and/or Azure, including cloud storage, IAM, networking, security, and monitoring, with seamless Databricks integration. Demonstrate expertise in ETL/ELT, batch/streaming processing, data ingestion, transformation, orchestration, and data quality, with a proven ability to design scalable pipeline architectures. Strong SQL skills and experience in dimensional modeling (star/snowflake schemas, fact/dimension models, SCD) are essential. Hands-on experience designing and building Agentic AI solutions with Databricks, including LLMs, tool/function calling, agent workflows, and enterprise data integration is required. Strong experience building RAG solutions using Databricks, including document ingestion, chunking, embeddings, vector search, retrieval, grounding, and evaluation is crucial. Knowledge of Git, branching strategies, pull requests, and code reviews, along with CI/CD experience across multiple environments is necessary. Ability to participate in strategic RFP/RFI responses, solution shaping, technical proposals, and effort estimations is expected. Strong client-facing and communication skills are vital to translate business requirements into technical solutions and communicate architectural decisions effectively.