EY - GDS Consulting - AI and DATA - Databricks Kafka Data Engineer - Senior

EY

5–7 yrs Bengaluru Full Time Hybrid (office + remote)
EY logo
Posted : today
Actively hiring

Job description

Join EY and embark on a career path as unique as you are. Leverage global scale, support, an inclusive culture, and cutting-edge technology to reach your full potential. We value your unique voice to help EY evolve and grow. Contribute to building an exceptional experience for yourself and fostering a better working world for everyone.

EY is searching for a Senior Consultant specializing in Databricks and Kafka Data Engineering. This role requires a deep understanding of modern Data Lakehouse architectures, scalable data platform design, enterprise-grade data pipeline development, and the delivery of analytics-ready data products. The ideal candidate will possess extensive technical expertise across the entire data engineering lifecycle, from ingestion and transformation to modeling and consumption, all while championing engineering best practices, automation, and effective stakeholder engagement.

Responsibilities

Spearhead the design and implementation of advanced Data Lakehouse platforms utilizing Databricks on AWS. Develop and optimize robust ETL/ELT data pipelines and integrate Kafka streaming solutions. Architect comprehensive end-to-end data solutions for reporting, BI, advanced analytics, and AI use cases. Establish and enforce data platform standards, governance controls, and best engineering practices. Design conceptual, logical, and physical data models, including dimensional models with Fact and Dimension tables. Implement CI/CD processes using Azure DevOps and GitHub for efficient data engineering workflows. Provide essential technical leadership and mentorship to engineering teams. Collaborate closely with stakeholders and delivery teams to translate requirements into scalable, high-quality solutions.

Qualifications

Demonstrate expert-level proficiency in Databricks, Apache Spark (PySpark/Spark SQL), Delta Lake, and Kafka Streaming. Possess strong skills in processing semi-structured data formats like JSON and XML, and extensive experience with AWS Cloud Services. Proven experience in ETL/ELT design and development, Data Lakehouse Architecture, and Data Modeling (including Dimensional Modeling). Solid understanding of Data Warehousing concepts, SQL, and performance optimization techniques. Experience in Real-Time Data Processing and Data Pipeline Orchestration is crucial. Proficiency in DevOps tools such as GitHub, Azure DevOps, and CI/CD implementation, including Databricks Asset Bundles (DAB). Familiarity with Agile Delivery Methodologies is expected. Possess strong analytical and problem-solving abilities, coupled with excellent communication and stakeholder management skills. Ability to thrive in a global delivery environment with a strong ownership mindset focused on quality and client value is essential. A Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field is required.

Essential Skills

DatabricksApache SparkPySparkSpark SQLDelta LakeKafka StreamingJSON processingXML processingAWSETLELTData Lakehouse ArchitectureData ModelingDimensional ModelingData WarehousingSQLPerformance OptimizationReal-Time Data ProcessingData Pipeline OrchestrationGitHubAzure DevOpsCI/CDDatabricks Asset Bundles (DAB)Agile

Good to Have

Insurance domain knowledgeBanking domain knowledgeFinancial Services domain knowledgeAI/ML workload support on Databricks

Highlights

  • Actively hiring

More Details

RoleEY - GDS Consulting - AI and DATA - Databricks Kafka Data Engineer - Senior
IndustryManagement Consulting
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

EY Global Delivery Services ( EY GDS) logo

EY Global Delivery Services ( EY GDS)

Management Consulting

EY - GDS Consulting - AI and DATA - Databricks Kafka Data Engineer - Senior at EY | SkillMX | SkillMX