Data Migration Engineer (SWP Workstream)
NTT DATA
NTT DATA
Join our team as a Senior Data Migration Engineer, a crucial hands-on role driving a large-scale data migration. You will be instrumental in designing and implementing robust solutions using Python for data extraction and loading, dbt for transformations, and Airflow for orchestration.
This position demands deep expertise in migrating complex Informatica PowerCenter workflows to a modern data stack, ensuring production-quality, functionally equivalent output at an enterprise level.
Translate intricate Informatica workflows into equivalent Python EL scripts and dbt transformation models. Develop and maintain Python EL pipelines for high-volume data landing into target warehouses, leveraging tools like SQLAlchemy and Oracle Data Pump.
Design and implement dbt transformation models following best practices, including model layering and incremental strategies. Create dbt-native and custom Python unit tests to ensure transformation accuracy and data quality. Orchestrate end-to-end processes using Airflow DAGs and contribute to GitLab CI/CD pipelines for code deployment and testing.
Conduct peer code reviews, provide constructive feedback, and troubleshoot performance issues or data discrepancies during testing phases. Contribute essential technical documentation and runbooks for seamless knowledge transfer.
A minimum of 5 years of experience with Informatica PowerCenter, covering mappings, sessions, workflows, and parameterization, is essential. You should also possess at least 3 years of hands-on experience with dbt Core, including model layering, macros, tests, and incremental materializations.
Required skills also include 3+ years of Airflow experience for DAG authoring and task dependencies, and 5+ years of Python experience with a focus on parallelism and batch data processing. Proficiency in SQL, including complex joins and window functions for large datasets, and experience with high-volume data movement tools like Oracle Data Pump or SQL Server BCP are critical.
Experience with SQLAlchemy, cx_Oracle, or pyodbc for Python database connectivity is preferred. Additionally, experience mentoring junior engineers in offshore delivery models would be a valuable asset.
NTT DATA
IT Consulting