AWS Data Engineer

Infosys

5–7 yrs Bengaluru Full Time Work from office
Infosys logo
Posted : today
Actively hiring

Job description

As an AWS Data Engineer, you will leverage extensive experience in building and managing large-scale cloud data platforms on AWS. Your expertise will be crucial in designing, developing, and deploying robust data solutions that drive client success.

This role involves a deep understanding of modern data architectures, including Data Lakes and Lakehouses, to optimize data storage, processing, and retrieval for analytical, operational, and reporting needs.

Join our consulting team to tackle complex client challenges, diagnose issues, and craft innovative solutions that facilitate seamless deployment and ensure client satisfaction.

Responsibilities

Design and implement scalable data ingestion and transformation pipelines on AWS, ensuring high volume and low latency.

Develop and maintain real-time and streaming data processing solutions using AWS services and associated technologies.

Optimize data pipelines and queries for performance, cost-efficiency, and scalability, especially for TB/PB-scale datasets.

Implement CI/CD practices for data engineering workflows using tools like AWS CodePipeline, CodeBuild, and GitHub Actions.

Collaborate with clients to understand their needs, diagnose problem areas, and design innovative data solutions.

Participate in solution design, configuration, and pilot testing, providing expertise to resolve any design-related queries.

Qualifications

Proven hands-on experience with core AWS data services including S3, Glue, Lambda, Redshift, Lake Formation, CloudWatch, and IAM.

Proficiency in real-time and streaming data processing with technologies like Kinesis, Kafka, or equivalents.

Strong coding skills in Python, PySpark, and SQL, with a focus on optimization.

Demonstrated experience in building high-volume, low-latency data ingestion and transformation pipelines.

Expertise in Databricks, Delta Lake, Structured Streaming, and Spark performance tuning.

Experience managing large-scale datasets (TB/PB) including partitioning, file optimization, schema evolution, and lifecycle management.

Solid understanding of Data Lake, Lakehouse, and Modern Data Platform architectures.

Practical experience in data modeling for analytical, operational, and reporting workloads.

Familiarity with CI/CD pipeline implementation using AWS CodePipeline, CodeBuild, CodeDeploy, GitHub Actions, or Jenkins.

Excellent troubleshooting skills for production issues, performance bottlenecks, and cost optimization.

Strong client communication, stakeholder management, and problem-solving abilities.

Essential Skills

AWS S3AWS GlueAWS LambdaAWS RedshiftAWS Lake FormationAWS CloudWatchAWS IAMAmazon Kinesis Data StreamsKinesis FirehoseKinesis AnalyticsAmazon KafkaPythonPySparkSQLDatabricksDelta LakeStructured StreamingSpark Performance TuningCI/CDAWS CodePipelineCodeBuildCodeDeployGitHub ActionsJenkinsAWS Core servicesAWS Database TechnologiesAWS

Good to Have

Amazon Dynamo DB

Highlights

  • Actively hiring

More Details

RoleAWS Data Engineer
DepartmentData Engineering, Consulting
Employment TypeFull Time, Work from office

About the Company

Infosys logo

Infosys

IT Consulting

AWS Data Engineer at Infosys | SkillMX | SkillMX