Aws Developer
Infosys
Infosys
Seeking a seasoned AWS Glue Developer and Data Engineer to spearhead the design, development, and optimization of robust, scalable ETL/ELT data pipelines within the AWS ecosystem. This role demands deep expertise in AWS Glue, Python, PySpark, SQL, and foundational data lake technologies, complemented by a solid understanding of various AWS data services.
This position offers an opportunity to work with cutting-edge cloud data platforms and contribute to critical data initiatives. The ideal candidate will be instrumental in building efficient and reliable data solutions that drive business insights.
Key responsibilities include designing and maintaining ETL/ELT pipelines using AWS Glue, developing sophisticated jobs with PySpark and SQL for large-scale data processing, and building ingestion/transformation pipelines for diverse data sources. You will configure and manage AWS Glue components, integrate with AWS services like S3 and Redshift, and implement rigorous data validation and quality checks.
Performance optimization, cost efficiency, and robust error handling are paramount. You will also be involved in troubleshooting pipeline issues, implementing security controls, and migrating legacy ETL workloads to AWS. Collaboration with cross-functional teams and documentation of technical designs are essential aspects of this role.
A strong foundation in Data Engineering and ETL development, with a minimum of 3 years of overall experience, is required. Proven hands-on expertise in developing production-grade AWS Glue, Python, and PySpark solutions is essential. The role necessitates experience in handling high-volume datasets and complex transformations within AWS environments.
Familiarity with Data Lake and Cloud Data Warehouse concepts, along with performance optimization and troubleshooting skills, is crucial. While not mandatory, an AWS Certified Data Engineer – Associate or similar AWS certification is advantageous. A Bachelor of Engineering or Bachelor of Technology degree is a prerequisite for this role.
Infosys
Information Technology & Services