Data Senior Engineer

NTT DATA

Fresher Bengaluru Full Time Work from office
NTT DATA logo
Posted : today
Actively hiring

Job description

Join our team as a Data Senior Engineer and play a crucial role in modernizing our data pipelines. You will be instrumental in translating complex existing workflows into efficient, scalable Python and dbt solutions.

This position offers an exciting opportunity to work with large datasets and cutting-edge technologies, contributing to robust data transformations and reliable data warehousing. You'll be a key player in enhancing our data infrastructure and ensuring its performance and accuracy.

Responsibilities

Translate intricate Informatica workflows into equivalent Python EL scripts and dbt transformation models. Develop and maintain Python EL pipelines to ingest high-volume data into the data warehouse using tools like SQLAlchemy and bulk export utilities. Design and implement dbt transformation models, adhering to best practices for layering, macros, and incremental strategies. Create dbt-native tests and custom Python unit tests to ensure transformation accuracy and data quality. Build Airflow DAGs for end-to-end orchestration of Python EL and dbt scripts. Contribute to GitLab CI/CD pipelines for automated testing and deployment of dbt and Airflow code. Conduct peer code reviews and offer constructive technical feedback. Troubleshoot performance bottlenecks and data discrepancies during SIT and UAT phases. Contribute to the creation of technical documentation and runbooks.

Qualifications

Proven experience in assessing and translating complex Informatica workflows. Strong proficiency in developing Python EL pipelines, handling high-volume data. Expertise in designing and developing dbt transformation models, incorporating best practices. Experience with dbt testing and Python unit testing for data validation. Skilled in developing Airflow DAGs for workflow orchestration. Familiarity with GitLab CI/CD for data pipelines. Ability to troubleshoot performance issues and data discrepancies. Excellent communication and collaboration skills for peer reviews and documentation.

Essential Skills

PythondbtSQLAlchemycx_OraclepyodbcOracle Data PumpSQL Server BCPAirflowGitLab CI/CD

Highlights

  • Actively hiring

More Details

RoleData Senior Engineer
Employment TypeFull Time, Work from office

About the Company

NTT DATA logo

NTT DATA

IT Consulting

Data Senior Engineer at NTT DATA | SkillMX | SkillMX