Data Engineer senior Consultant
NTT DATA
NTT DATA
NTT DATA seeks a highly skilled Senior Data Engineer Consultant to join our dynamic team in Bangalore, India. This role is crucial for designing, building, and supporting robust event-driven data pipelines. You will work with established architectural standards to develop secure, reusable components using a variety of AWS services and programming languages.
The position demands accountability for data accuracy, efficient processing, lineage, and overall pipeline performance throughout the ingestion-to-serving lifecycle. We are looking for innovative individuals eager to grow with a forward-thinking organization.
Develop and maintain event-driven and batch data ingestion pipelines, ensuring rigorous validation of files and events through transformation and persistence.
Build secure, reusable components using AWS services like S3, Lambda, SQS, and EventBridge, with a focus on Python or Java and strong SQL proficiency.
Implement comprehensive error handling, retry mechanisms, dead-letter queues, and replay capabilities to ensure data integrity and idempotency.
Create automated tests, participate in code reviews, and support CI/CD pipelines for seamless deployments and robust code quality.
Collaborate closely with architects, engineers, and subject matter experts to refine requirements and ensure successful project outcomes.
A minimum of 5–7 years in data engineering or backend engineering, with at least 3 years of hands-on AWS development experience.
Strong expertise in AWS S3, Lambda, SQS, EventBridge, CloudWatch, IAM, KMS, and Secrets Manager.
Proficiency in Python and strong SQL skills; Java experience is a plus. Experience with PostgreSQL or Amazon Aurora PostgreSQL is required.
Demonstrated ability in building event-driven pipelines, schema validation, transformations, and managing data quality and lineage.
Familiarity with Git-based development, CI/CD practices, and troubleshooting distributed systems using logs and metrics.
NTT DATA
IT Consulting