T&T - Engineering - Consultant - Statistical Analysis System Data engineer - Multiple Location

Deloitte

3+ yrs Pune, Bengaluru, Hyderabad, Bhubaneswar, Chennai, Coimbatore, Ahmedabad Full Time Hybrid (office + remote)
Deloitte logo
Posted : 1 week ago
Actively hiring

Job description

Deloitte's Technology & Transformation practice empowers clients to extract maximum value from their data. Our global team offers strategic guidance and implementation services to help companies manage diverse data sources, transforming them into actionable insights for informed decision-making and a competitive edge. We address opportunities across business intelligence, data management, performance management, and advanced analytics, including big data, cloud, AI, and machine learning.

This role focuses on managing client relationships and ensuring the successful delivery of programs and accounts. You will be the primary liaison between Deloitte and clients, aligning goals, timelines, budgets, and expected outcomes to drive client satisfaction and business growth.

Responsibilities

Migrate SAS frameworks to Databricks/PySpark, designing and developing PySpark pipelines from SAS macros. Implement validation frameworks between SAS and Spark outputs, and manage the DevOps lifecycle with Git/Bitbucket integration, code reviews, and production deployments.

Design, build, and maintain scalable data pipelines on AWS, and partner with cross-functional teams to translate business requirements into technical delivery plans. Build and optimize data lakes using S3, Glue Catalog, and Lake Formation, and work with Redshift/Snowflake for data warehousing solutions.

Implement data quality checks, monitoring, logging, and CI/CD deployments. Collaborate with analytics and business teams to deliver high-quality datasets. Design, develop, and deploy solutions using various tools, design principles, and conventions.

Understand existing processes and facilitate changing requirements through structured change control. Maintain comprehensive documentation for solutions, test procedures, and scenarios during UAT and Production phases. Coordinate with process owners and business teams to comprehend current processes and design automation workflows.

Qualifications

A strong foundation in migrating SAS frameworks to Databricks/PySpark is essential, along with proficiency in Python (PySpark) and SQL for data transformation.

Demonstrated experience in designing and developing PySpark pipelines from SAS macros and implementing validation frameworks between SAS and Spark outputs. Expertise in managing the DevOps lifecycle, including Git/Bitbucket integration, code reviews, and production deployments.

Proven ability to design, build, and maintain scalable data pipelines on AWS. Experience with Redshift/Snowflake for data warehousing solutions and building/optimizing data lakes using S3, Glue Catalog, and Lake Formation.

Solid understanding of data quality checks, monitoring, logging, and CI/CD deployment processes. Familiarity with data modeling, data lake principles, and best practices is expected. Experience with Spark (EMR/Glue) and distributed data processing is required.

Essential Skills

SASDatabricksPySparkPythonSQLAWSDevOpsGitBitbucketS3Glue CatalogLake FormationRedshiftSnowflakeCI/CDData WarehousingData PipelinesData ModelingData LakeCloudFormation

Good to Have

Terraform

Highlights

  • Actively hiring

More Details

RoleT&T - Engineering - Consultant - Statistical Analysis System Data engineer - Multiple Location
IndustryEngineering
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Deloitte logo

Deloitte

Engineering