Senior Consultant | Databricks | Mumbai | Engineering
Deloitte
Deloitte
Join Deloitte's Technology & Transformation practice as a Senior Databricks Data Engineer and help organizations unlock the full potential of their data. You will be instrumental in designing, developing, and maintaining robust data solutions on the Databricks platform, driving data-driven decision-making across diverse clients. This role offers a unique opportunity to collaborate with leading organizations and contribute to shaping the future of the region's economy.
Deloitte empowers individuals to bring their whole selves to work, fostering an environment where innovation, growth, and making a meaningful impact are paramount. Become part of a dynamic team focused on data modernization and leverage cutting-edge technologies to transform vast datasets into actionable insights.
Design, develop, and maintain scalable data engineering solutions using Databricks, PySpark, and SQL for enterprise data platforms. Implement and optimize secure, reliable, and performant data architectures. Build and manage data pipelines for diverse data types from multiple sources, supporting both batch and near real-time processing. Utilize Databricks capabilities including Delta Lake, DLT, Unity Catalog, Workflows, Serverless Compute, Genie, and Mosaic AI. Develop data solutions aligned with Lakehouse, Medallion, Data Warehouse, and Data Mart standards. Optimize Spark jobs, SQL queries, clusters, and pipelines for enhanced performance, scalability, reliability, and cost efficiency. Manage and monitor Databricks clusters and resource utilization across cloud environments. Collaborate with stakeholders to translate business requirements into technical solutions. Implement enterprise data governance, security, and compliance standards. Leverage cloud-native services (Azure, AWS, GCP) for integrated data solutions. Deploy code using GitHub, CI/CD pipelines, and Agile methodologies. Work with large-scale datasets, implementing robust security, data quality, and optimization techniques. Perform system integration and testing to ensure data accuracy and reliability. Analyze, troubleshoot, and resolve defects and production incidents promptly. Document technical designs, data models, and operational procedures for platform sustainability.
Possess 3-7 years of experience in data engineering, with a strong focus on Databricks. Demonstrate hands-on expertise with Databricks, PySpark, and SQL. Experience with cloud platforms such as Azure, AWS, or GCP is essential. Proficiency in Delta Lake, Delta Live Tables (DLT), Unity Catalog/Data Governance, Databricks Workflows, and Serverless Compute is required. Solid understanding of Data Architecture, Data Modeling, Lakehouse Architecture, Data Warehousing, and the Medallion Framework. Familiarity with GitHub, CI/CD pipelines, Agile delivery, and DevOps practices is necessary. Strong analytical, troubleshooting, and performance tuning skills are crucial. A Bachelor’s or Master’s degree in any field is required. Experience with ETL/ELT processes, data warehousing, and data modeling concepts is expected. Knowledge of Python or SQL and Delta Lake is mandatory. Understanding of DevOps principles and practices, along with excellent problem-solving and communication skills, are vital.
Deloitte
Management Consulting