Data Engineer Senior Consultant

NTT DATA

5+ yrs Bengaluru Full Time Hybrid (office + remote)
NTT DATA logo
Posted : 1 week ago
Actively hiring

Job description

Join NTT DATA as a Senior Data Migration Engineer, a crucial role focused on modernizing data infrastructures. You will be instrumental in migrating complex Informatica PowerCenter workflows to an advanced data stack, leveraging Python, dbt, and Airflow. This position offers the opportunity to make a significant impact on large-scale projects, requiring a blend of deep technical expertise and hands-on delivery.

As a key technical contributor, you will design and implement robust extract/load pipelines and transformations. Your work will ensure the delivery of production-quality, functionally equivalent data solutions at an enterprise level, driving innovation and client success. We are looking for passionate individuals eager to grow within a forward-thinking organization.

Responsibilities

Translate intricate Informatica PowerCenter workflows, including sessions, mappings, and parameterizations, into equivalent Python EL scripts and dbt transformation models.

Develop and maintain high-performance Python EL pipelines for loading substantial data volumes, including billion-row tables, into target data warehouses. Utilize tools like SQLAlchemy, cx_Oracle, pyodbc, and Oracle Data Pump or SQL Server BCP for efficient data movement.

Design and build dbt transformation models, adhering to best practices such as model layering, macros, incremental strategies, and snapshot patterns. Implement dbt-native tests and custom Python unit tests to guarantee transformation accuracy and data quality.

Orchestrate end-to-end Python EL and dbt script execution using Airflow DAGs, ensuring functional equivalence with source Informatica workflows. Contribute to GitLab CI/CD pipelines for code quality, automated testing, and deployment.

Conduct peer code reviews, offer constructive technical guidance, and troubleshoot performance issues or data discrepancies during SIT and UAT. Play a role in developing essential technical documentation, runbooks, and handover materials for seamless project transitions.

Qualifications

A minimum of 5 years of experience with Informatica PowerCenter, encompassing mappings, sessions, workflows, and parameterization.

At least 3 years of hands-on experience with dbt Core, including expertise in model layering, macros, testing, and incremental materializations.

Demonstrated experience with Airflow, covering DAG authoring, task dependencies, and integration with scheduling tools.

5+ years of extensive Python experience, with a focus on production-level parallelism and batch data processing.

Proficiency in SQL, including complex joins, window functions, and aggregations on large datasets.

Experience in high-volume data movement using Oracle Data Pump (expdp/impdp) or SQL Server BCP / BULK INSERT.

Preferred qualifications include experience with SQLAlchemy, cx_Oracle, or pyodbc for database connectivity, and experience mentoring junior engineers in offshore delivery settings.

Essential Skills

Informatica PowerCenterPythonSQLdbt CoreAirflowOracle Data PumpSQL Server BCP

Good to Have

SQLAlchemycx_OraclepyodbcMentoring

Highlights

  • Actively hiring

More Details

RoleData Engineer Senior Consultant
DepartmentData Engineering, Consulting
Employment TypeFull Time, Hybrid (office + remote)

About the Company

NTT DATA logo

NTT DATA

IT Consulting