Data Engineer - Tester
NTT DATA
NTT DATA
Join NTT DATA, a global leader in business and technology services, committed to accelerating client success and making a positive societal impact. We are expanding our team with a passionate Data Engineer - Tester in Bengaluru, India. This role is pivotal in ensuring the quality and integrity of our data pipelines and solutions.
As a forward-thinking organization serving 75% of the Fortune Global 100, NTT DATA offers unparalleled opportunities for growth and innovation. We are at the forefront of AI and digital infrastructure, empowering organizations to navigate the digital future with confidence and sustainability.
We prioritize local hiring to ensure responsive and effective client support. While remote and hybrid options are often available, they are subject to client needs. Flexibility is key to meeting the evolving demands of both clients and employees.
Develop automated test suites for complex ETL/ELT pipelines, leveraging PySpark and DataTester.
Ensure data accuracy, completeness, and consistency across source systems and AWS S3 data lakes.
Design and maintain scalable, reusable data testing frameworks for schema validation, null checks, and data reconciliation.
Conduct thorough unit, integration, and regression testing for distributed data processing applications.
Identify, document, and track data anomalies, performance issues, and pipeline failures.
Collaborate closely with Data Engineers and Platform teams, implementing test-driven development practices.
Proficiency in data testing frameworks like DataTester for automating data quality checks.
Strong skills in PySpark and Apache Spark for large-scale data manipulation and querying.
Deep understanding of AWS S3 storage structures, partitioning, and file formats (Parquet, Delta, Avro).
Solid experience in functional and non-functional data testing, including migration and regression testing.
Advanced SQL skills for relational database querying and data warehousing verification.
Experience with Git and CI/CD pipelines for automated test execution is essential.
NTT DATA
Information Technology & Services