Senior Consultant | Databricks | Mumbai | Engineering

Deloitte

3–7 yrs Mumbai Full Time Work from office
Deloitte logo
Posted : 1 week ago
Actively hiring

Job description

Join Deloitte's Technology & Transformation practice as a Senior Databricks Data Engineer and help organizations unlock the full potential of their data. You will be instrumental in designing, developing, and maintaining robust data solutions on the Databricks platform, driving data-driven decision-making across diverse clients. This role offers a unique opportunity to collaborate with leading organizations and contribute to shaping the future of the region's economy.

Deloitte empowers individuals to bring their whole selves to work, fostering an environment where innovation, growth, and making a meaningful impact are paramount. Become part of a dynamic team focused on data modernization and leverage cutting-edge technologies to transform vast datasets into actionable insights.

Responsibilities

Design, develop, and maintain scalable data engineering solutions using Databricks, PySpark, and SQL for enterprise data platforms. Implement and optimize secure, reliable, and performant data architectures. Build and manage data pipelines for diverse data types from multiple sources, supporting both batch and near real-time processing. Utilize Databricks capabilities including Delta Lake, DLT, Unity Catalog, Workflows, Serverless Compute, Genie, and Mosaic AI. Develop data solutions aligned with Lakehouse, Medallion, Data Warehouse, and Data Mart standards. Optimize Spark jobs, SQL queries, clusters, and pipelines for enhanced performance, scalability, reliability, and cost efficiency. Manage and monitor Databricks clusters and resource utilization across cloud environments. Collaborate with stakeholders to translate business requirements into technical solutions. Implement enterprise data governance, security, and compliance standards. Leverage cloud-native services (Azure, AWS, GCP) for integrated data solutions. Deploy code using GitHub, CI/CD pipelines, and Agile methodologies. Work with large-scale datasets, implementing robust security, data quality, and optimization techniques. Perform system integration and testing to ensure data accuracy and reliability. Analyze, troubleshoot, and resolve defects and production incidents promptly. Document technical designs, data models, and operational procedures for platform sustainability.

Qualifications

Possess 3-7 years of experience in data engineering, with a strong focus on Databricks. Demonstrate hands-on expertise with Databricks, PySpark, and SQL. Experience with cloud platforms such as Azure, AWS, or GCP is essential. Proficiency in Delta Lake, Delta Live Tables (DLT), Unity Catalog/Data Governance, Databricks Workflows, and Serverless Compute is required. Solid understanding of Data Architecture, Data Modeling, Lakehouse Architecture, Data Warehousing, and the Medallion Framework. Familiarity with GitHub, CI/CD pipelines, Agile delivery, and DevOps practices is necessary. Strong analytical, troubleshooting, and performance tuning skills are crucial. A Bachelor’s or Master’s degree in any field is required. Experience with ETL/ELT processes, data warehousing, and data modeling concepts is expected. Knowledge of Python or SQL and Delta Lake is mandatory. Understanding of DevOps principles and practices, along with excellent problem-solving and communication skills, are vital.

Essential Skills

DatabricksPySparkSQLAzureAWSGCPDelta LakeDelta Live TablesUnity CatalogData GovernanceDatabricks WorkflowsServerless ComputeData ArchitectureData ModelingLakehouse ArchitectureData WarehousingMedallion FrameworkGitHubCI/CDAgileDevOpsPythonETLELT

Good to Have

GenieMosaic AIDatabricks Data Intelligence Platform

Highlights

  • Actively hiring

More Details

RoleSenior Consultant | Databricks | Mumbai | Engineering
IndustryManagement Consulting
DepartmentData Engineering, Consulting
Employment TypeFull Time, Work from office

About the Company

Deloitte logo

Deloitte

Management Consulting