Data Engineer-Data Platforms-AWS

IBM

Fresher Pune, Chennai, Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and collaborate with global industry leaders to shape their hybrid cloud and AI strategies. You will leverage IBM's technology and Red Hat's robust solutions to drive significant client impact. Curiosity is key here; we encourage innovation and exploration to deliver tangible results. Our supportive culture fosters continuous career development while valuing your unique contributions.

We are searching for a seasoned AWS Data Engineer to architect, develop, and maintain robust, secure, and high-performance data platforms within the AWS ecosystem. This role demands extensive practical experience with AWS services, distributed data processing frameworks, and SQL-based analytics. The ideal candidate will partner closely with architects, application teams, and business stakeholders to deliver enterprise-grade data solutions.

Responsibilities

Design, build, and manage scalable data pipelines and comprehensive data platforms on AWS. Develop and operate serverless and containerized applications using AWS Lambda, EC2, and EKS. Create batch and large-scale data processing solutions leveraging Apache Spark. Implement efficient data ingestion, transformation, and analytics processes with AWS Glue and Athena. Craft complex, high-performance SQL queries for advanced analytics, reporting, and data validation. Optimize and maintain data storage solutions using AWS S3 and RDS. Manage AWS infrastructure through Infrastructure as Code (IaC) using CloudFormation. Integrate AWS security best practices across IAM, VPC networking, data encryption, and access controls. Develop clean, maintainable, and scalable code in Python and/or Scala. Monitor, troubleshoot, and enhance the performance, cost-effectiveness, and reliability of data workloads. Collaborate with diverse teams to support crucial analytics, reporting, and regulatory requirements.

Qualifications

Essential qualifications include strong, hands-on experience with core AWS services such as S3, Lambda, EC2, EKS, AWS Glue, Athena, RDS, VPC, IAM, and CloudFormation. Proficiency in writing complex SQL queries, joins, aggregations, and user-defined functions, alongside expertise in performance tuning and query optimization, is crucial. A solid understanding of AWS Security, including IAM policies, VPC security, encryption, and least-privilege access, is required. Demonstrated programming experience with Python and/or Scala is a must, as is substantial experience with Apache Spark for large-scale data processing. Excellent communication skills are needed to articulate technical concepts to non-technical audiences. A foundational knowledge of data engineering principles like ETL/ELT, data modeling, and distributed systems is expected.

Essential Skills

AWSS3LambdaEC2EKSAWS GlueAthenaRDSVPCIAMCloudFormationSQLPythonScalaApache SparkETLELTData ModelingDistributed SystemsData Pipelines

Good to Have

DBTSnowflakeDatabricksData LakeLakehouseApache AirflowAWS EMRAWS KinesisManaged Streaming for Apache KafkaAWS RedShiftAuroraDynamoDBAWS DMSAWS DatabrewRedShift Spectrum

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-AWS
IndustrySoftware Development
DepartmentData Engineering, Data & Analytics
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Software Development

Data Engineer-Data Platforms-AWS at IBM | SkillMX | SkillMX