AWS Data Engineer
Infosys
Infosys
Seeking a skilled AWS Data Engineer to design, build, and manage robust data pipelines on the AWS platform. This role requires a strong foundation in data engineering principles, with hands-on experience in PySpark, SQL, and various ETL frameworks. The ideal candidate will be adept at working with extensive structured and semi-structured datasets to drive data solutions.
As an AWS Data Engineer, you will play a pivotal role in our Data & Analytics Unit, contributing to innovative projects within Infosys Limited. This is an opportunity to leverage your expertise in a dynamic environment, working on challenging data-related tasks and collaborating with cross-functional teams to meet business needs.
Key responsibilities include architecting and implementing scalable data pipelines within AWS, and developing efficient ETL/ELT workflows for data ingestion and transformation.
You will build and optimize data processing solutions using PySpark and SQL, ensuring data accuracy through rigorous validation and quality checks. Integrating data from diverse sources into cloud-based platforms and troubleshooting pipeline issues are also core aspects of this role.
Collaboration with business, analytics, and engineering teams to translate data requirements into actionable engineering solutions will be essential. You will also contribute to optimizing data pipelines and SQL queries for maximum performance and scalability.
A proven track record in AWS Data Engineering services such as S3, Glue, Lambda, EMR, Redshift, Athena, and IAM is essential.
Candidates must possess strong programming expertise in PySpark and a deep understanding of SQL, including advanced query writing and data manipulation.
Hands-on experience in developing ETL/ELT pipelines and a solid grasp of data warehousing, data lakes, and data modeling concepts are required. Experience managing large datasets and optimizing distributed processing workloads is also crucial.
While not mandatory, proficiency in Python, familiarity with Git, CI/CD, orchestration tools, Agile practices, and prior exposure to cloud migration projects are highly advantageous.
Infosys
IT Consulting