Deloitte's Technology & Transformation practice empowers companies to harness the full potential of their data. We provide strategic guidance and implementation services, helping organizations integrate data from diverse sources into actionable insights for informed decision-making. Our expertise spans business intelligence, data management, performance management, and advanced analytics, including big data, cloud, cognitive, and machine learning technologies.

This Senior Consultant role focuses on SAS framework data engineering, requiring strong client relationship management and program delivery oversight. You will be the primary liaison between our organization and clients, ensuring project goals, timelines, budgets, and expected outcomes are met, ultimately driving client satisfaction and business growth.

Design and develop robust PySpark pipelines, migrating SAS macros to Databricks. Implement validation frameworks between SAS and Spark outputs. Manage the DevOps lifecycle, integrating with Git/Bitbucket, conducting code reviews, and overseeing production deployments.

Build and maintain scalable data pipelines on AWS. Develop and optimize data lakes using S3, Glue Catalog, and Lake Formation. Implement data quality checks, monitoring, logging, and CI/CD processes. Collaborate with analytics and business teams to deliver high-quality datasets. Design, develop, and deploy solutions using various tools and design principles.

A minimum of 3 years of hands-on experience migrating SAS frameworks to Databricks/PySpark is essential. Proficiency in Python (PySpark), SQL, and data transformation logic is required. Experience with Redshift, Athena, and Glue Data Catalog is expected.

Strong knowledge of Spark (EMR/Glue) and distributed data processing is necessary. Familiarity with Terraform/CloudFormation for Infrastructure as Code is a plus. A solid understanding of data modeling, data lake principles, and best practices is crucial. Ability to understand existing processes and facilitate changes through a structured control process. Maintain comprehensive documentation for solutions and testing procedures.