Aws Developer
Infosys
Infosys
Seeking a skilled AWS Glue Developer / Data Engineer to architect, build, and refine robust ETL/ELT data pipelines within the AWS ecosystem. This role demands significant practical experience with AWS Glue, Python, PySpark, SQL, data lake principles, and various AWS data services.
Key responsibilities include designing, developing, and maintaining scalable ETL/ELT pipelines using AWS Glue. You will craft AWS Glue jobs leveraging Python, PySpark, and SQL for processing extensive datasets. Responsibilities also encompass building data ingestion and transformation flows for diverse data types, configuring AWS Glue components, and integrating with AWS services like S3, Redshift, and IAM. You will implement data quality checks, validations, and reconciliation, alongside optimizing Glue/Spark jobs for performance, cost, and reliability. Experience with incremental loads, partitioning, error handling, and security controls is essential. The role involves troubleshooting pipeline issues, migrating legacy workloads to AWS, and developing automated deployment processes. Collaboration with stakeholders to translate requirements into data solutions and documentation of technical designs are also crucial.
This position requires a minimum of 5 years in Data Engineering or ETL development, with a strong track record of building production-ready AWS Glue and PySpark solutions. You should possess experience managing large datasets and complex transformations in AWS environments. Essential technical skills include proficiency in AWS Glue, Python/PySpark/Apache Spark, SQL, Amazon S3, and a solid understanding of Data Lake/Cloud Data Warehouse concepts, ETL/ELT design, and performance optimization. Experience with AWS IAM and AWS Redshift is also expected. A Bachelor of Engineering or Bachelor of Technology degree is required. While not mandatory, an AWS Certified Data Engineer – Associate or a similar AWS certification is preferred.
Infosys
IT Consulting