Join NTT DATA as a Data Engineer Senior Consultant and play a pivotal role in migrating datastores from on-premise DataLake to an AWS-hosted LakeHouse. This is a high-impact project for Goldman Sachs, requiring end-to-end data migration expertise. You'll collaborate with a dedicated factory team, ensuring seamless transitions and data integrity.

This role demands a proactive approach to refactoring logic, migrating extraction processes, and ensuring robust data transfer. You will engage directly with internal clients and data owners, acting as a key technical liaison to facilitate sign-offs and confirm that migrated assets align with business objectives. Become an integral part of transforming data infrastructure with cutting-edge cloud solutions.

Key responsibilities include refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment. You will execute physical data migration, ensuring data integrity and acting as a technical liaison for stakeholder engagement. Additionally, translate and optimize legacy SQL and Spark-based consumption patterns for Snowflake and Iceberg, understanding usage patterns to deliver required data products. A rigorous approach to data validation and reconciliation is crucial, ensuring migrated data is functionally equivalent to production flows. You will also work with other internal data management platforms and demonstrate an aptitude for learning new workflows and language constructs.

We are seeking candidates with a Bachelor's or Master's degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field. A minimum of 3-5 years of hands-on coding experience in a collaborative environment is required, along with troubleshooting skills and basic scripting experience. Professional proficiency in Python or Java is essential, alongside deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices. Experience with K8s deployment is also a plus.

Core data engineering competencies include sophisticated understanding of Temporal Data Modeling (e.g., SCD Type 2), Schema Management (e.g., Apache Iceberg), Performance Optimization (data partitioning and clustering), and Architectural Theory (balancing Normalization vs. Denormalization, Natural vs. Surrogate Keys). Familiarity with technologies such as Kafka, ANSI SQL, Spark, JSON, Avro, Parquet, Hadoop, Snowflake, and Apache Iceberg is expected.