EY - GDS Consulting - AI and DATA - Databricks Kafka Data Engineer - Senior
EY
EY
Join EY and embark on a career path as unique as you are. Leverage global scale, support, an inclusive culture, and cutting-edge technology to reach your full potential. We value your unique voice to help EY evolve and grow. Contribute to building an exceptional experience for yourself and fostering a better working world for everyone.
EY is searching for a Senior Consultant specializing in Databricks and Kafka Data Engineering. This role requires a deep understanding of modern Data Lakehouse architectures, scalable data platform design, enterprise-grade data pipeline development, and the delivery of analytics-ready data products. The ideal candidate will possess extensive technical expertise across the entire data engineering lifecycle, from ingestion and transformation to modeling and consumption, all while championing engineering best practices, automation, and effective stakeholder engagement.
Spearhead the design and implementation of advanced Data Lakehouse platforms utilizing Databricks on AWS. Develop and optimize robust ETL/ELT data pipelines and integrate Kafka streaming solutions. Architect comprehensive end-to-end data solutions for reporting, BI, advanced analytics, and AI use cases. Establish and enforce data platform standards, governance controls, and best engineering practices. Design conceptual, logical, and physical data models, including dimensional models with Fact and Dimension tables. Implement CI/CD processes using Azure DevOps and GitHub for efficient data engineering workflows. Provide essential technical leadership and mentorship to engineering teams. Collaborate closely with stakeholders and delivery teams to translate requirements into scalable, high-quality solutions.
Demonstrate expert-level proficiency in Databricks, Apache Spark (PySpark/Spark SQL), Delta Lake, and Kafka Streaming. Possess strong skills in processing semi-structured data formats like JSON and XML, and extensive experience with AWS Cloud Services. Proven experience in ETL/ELT design and development, Data Lakehouse Architecture, and Data Modeling (including Dimensional Modeling). Solid understanding of Data Warehousing concepts, SQL, and performance optimization techniques. Experience in Real-Time Data Processing and Data Pipeline Orchestration is crucial. Proficiency in DevOps tools such as GitHub, Azure DevOps, and CI/CD implementation, including Databricks Asset Bundles (DAB). Familiarity with Agile Delivery Methodologies is expected. Possess strong analytical and problem-solving abilities, coupled with excellent communication and stakeholder management skills. Ability to thrive in a global delivery environment with a strong ownership mindset focused on quality and client value is essential. A Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field is required.
EY Global Delivery Services ( EY GDS)
Management Consulting