AWS Data Engineer
Infosys
Infosys
Join Infosys Limited as a Senior Consultant in our Data & Analytics Unit, focusing on building cutting-edge AWS Data Engineering solutions. This role is ideal for experienced professionals seeking to leverage their expertise in large-scale cloud data platforms and contribute to innovative data solutions.
We are looking for individuals passionate about designing, developing, and optimizing robust data pipelines and modern data architectures within the AWS ecosystem. If you thrive in a collaborative environment and are adept at problem-solving, this is an excellent opportunity to advance your career.
Key responsibilities include designing and developing scalable AWS data engineering solutions and cloud-native data platforms. You will build batch and real-time data pipelines using AWS Glue, PySpark, Databricks, S3, Kinesis, Redshift, and Kafka/MSK. Optimizing Databricks, Delta Lake, and Spark workloads for large-scale data processing with PySpark and Databricks transformations is crucial.
Implement streaming solutions using Amazon Kinesis for near real-time data ingestion. You'll also handle high-volume data pipelines, focusing on performance, scalability, and cost optimization, alongside developing complex SQL transformations and data models. Adhering to CI/CD, Infrastructure as Code, and DevOps best practices is essential.
A strong foundation in AWS Data Engineering with expertise across S3, Glue, Lambda, Redshift, Lake Formation, CloudWatch, and IAM is required. Proficiency in real-time and streaming data processing using Kinesis or Kafka is key. Candidates must possess hands-on coding skills in Python, PySpark, and SQL, with strong optimization capabilities.
Experience with Databricks, Delta Lake, Structured Streaming, and Spark performance tuning is essential. You should be comfortable handling TB/PB-scale datasets, understanding partitioning, file optimization, schema evolution, and data lifecycle management. A solid grasp of Data Lake, Lakehouse, and Modern Data Platform architectures is necessary. Practical data modeling experience for analytical, operational, and reporting workloads is expected.
Infosys
IT Consulting