Data Engineer-Data Platforms-AWS

IBM

7–9 yrs Pune Full Time Work from office
IBM logo
Posted : yesterday
Actively hiring

Job description

Join IBM Consulting as a Data Engineer, focusing on Data Platforms within the AWS ecosystem. You will play a key role in designing, developing, and maintaining robust data engineering solutions, collaborating with leading companies to shape their hybrid cloud and AI transformations. This position is based in one of our IBM Consulting FutureNow Centers, offering deep technical and industry expertise to clients globally.

Responsibilities

As a Data Engineer, you will craft and manage batch and real-time data pipelines leveraging a variety of AWS services. Key duties include designing and building scalable data layers on AWS RedShift, Aurora, and DynamoDB, alongside data migration using AWS DMS. You will also be responsible for scheduling and overseeing data services on the AWS platform and integrating open-source technologies like Apache Airflow and dbt with Spark/Python or Spark/Scala.

Qualifications

This role requires extensive experience (7-9 years) with AWS services including EMR, Glue, Kinesis, RedShift, Aurora, and DynamoDB for data pipeline development and data layer creation. Proficiency in migrating data using AWS DMS and managing data services on AWS is essential. Candidates should be adept at utilizing open-source technologies such as Apache Airflow, dbt, and Spark (Python or Scala) within the AWS environment. A Bachelor's degree is required, with a Master's degree being preferred.

Essential Skills

AWS EMRAWS GlueGlue CatalogAWS KinesisAWS RedShiftAWS AuroraAWS DynamoDBAWS DMSApache AirflowdbtSparkPythonScala

Good to Have

AWS DatabrewLambda FunctionsPythonRedShift Spectrum

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-AWS
IndustryIT Services and IT Consulting
DepartmentData Science
Employment TypeFull Time, Work from office

About the Company

IBM logo

IBM

IT Services and IT Consulting

Data Engineer-Data Platforms-AWS at IBM | SkillMX | SkillMX