AWS Glue, Python
Infosys
Infosys
Join a dynamic team focused on transforming raw data into valuable, analytics-ready assets to drive informed decision-making. In this role, you will architect and refine cloud-based data integration workflows using AWS Glue and Python. You'll collaborate closely with engineers, analysts, and stakeholders to build and maintain scalable, reliable data pipelines.
This is an environment that champions clean engineering, continuous improvement, and shared responsibility. Your insights into performance optimization, automation, and data quality are highly valued. If you thrive on solving complex data challenges, building robust ETL processes, and learning modern AWS data services while contributing to significant outcomes, this role offers a compelling blend of hands-on development and problem-solving.
Design, build, and maintain robust ETL/ELT pipelines for batch and incremental data processing using AWS Glue and Python. Develop and optimize AWS Glue Jobs (PySpark/Python), focusing on job parameters, bookmarks, retries, and performance tuning. Implement comprehensive data ingestion, transformation, and validation logic to ensure data accuracy, completeness, and consistency. Integrate pipelines seamlessly with core AWS services like S3, IAM, and CloudWatch for secure, observable, and scalable workflows. Troubleshoot and resolve job failures by analyzing logs and metrics, and implement effective fixes to enhance stability and runtime efficiency. Collaborate with cross-functional teams to understand requirements, define data mappings, and deliver well-documented solutions. Adhere to engineering best practices, including code reviews, version control, and the creation of reusable, modular coding patterns.
Possess a Bachelor’s degree or equivalent qualification (e.g., BE/BTech/MSc/MCA/MTech). Bring 3–5 years of professional experience in data engineering, ETL development, or data integration. Demonstrate strong, hands-on expertise with AWS Glue and Python for constructing production-grade data pipelines. Showcase working knowledge of fundamental AWS concepts, including security essentials (IAM), storage strategies, and monitoring practices. Exhibit a strong ability to debug data pipeline issues and deliver reliable, well-documented solutions.
Infosys
IT Consulting