AWS Data Engineer
Infosys
Infosys
As an AWS Data Engineer, you will leverage extensive experience in building and managing large-scale cloud data platforms on AWS. Your expertise will be crucial in designing, developing, and deploying robust data solutions that drive client success.
This role involves a deep understanding of modern data architectures, including Data Lakes and Lakehouses, to optimize data storage, processing, and retrieval for analytical, operational, and reporting needs.
Join our consulting team to tackle complex client challenges, diagnose issues, and craft innovative solutions that facilitate seamless deployment and ensure client satisfaction.
Design and implement scalable data ingestion and transformation pipelines on AWS, ensuring high volume and low latency.
Develop and maintain real-time and streaming data processing solutions using AWS services and associated technologies.
Optimize data pipelines and queries for performance, cost-efficiency, and scalability, especially for TB/PB-scale datasets.
Implement CI/CD practices for data engineering workflows using tools like AWS CodePipeline, CodeBuild, and GitHub Actions.
Collaborate with clients to understand their needs, diagnose problem areas, and design innovative data solutions.
Participate in solution design, configuration, and pilot testing, providing expertise to resolve any design-related queries.
Proven hands-on experience with core AWS data services including S3, Glue, Lambda, Redshift, Lake Formation, CloudWatch, and IAM.
Proficiency in real-time and streaming data processing with technologies like Kinesis, Kafka, or equivalents.
Strong coding skills in Python, PySpark, and SQL, with a focus on optimization.
Demonstrated experience in building high-volume, low-latency data ingestion and transformation pipelines.
Expertise in Databricks, Delta Lake, Structured Streaming, and Spark performance tuning.
Experience managing large-scale datasets (TB/PB) including partitioning, file optimization, schema evolution, and lifecycle management.
Solid understanding of Data Lake, Lakehouse, and Modern Data Platform architectures.
Practical experience in data modeling for analytical, operational, and reporting workloads.
Familiarity with CI/CD pipeline implementation using AWS CodePipeline, CodeBuild, CodeDeploy, GitHub Actions, or Jenkins.
Excellent troubleshooting skills for production issues, performance bottlenecks, and cost optimization.
Strong client communication, stakeholder management, and problem-solving abilities.
Infosys
IT Consulting