Data Engineer-Data Platforms-AWS
IBM
IBM
Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies. As a Data Engineer on our Data Platforms team, you'll be instrumental in developing and maintaining advanced data engineering solutions within the AWS ecosystem.
This role offers a dynamic environment where curiosity and innovation drive success. You'll collaborate globally, leveraging robust IBM technology and Red Hat to accelerate client impact and foster your long-term career development in a culture of growth and empathy.
You will design, build, and operate robust batch and real-time data pipelines using key AWS services like EMR, Glue, Glue Catalog, and Kinesis. Responsibilities include creating comprehensive data layers on RedShift, Aurora, and DynamoDB, and migrating data efficiently using AWS DMS.
Further tasks involve managing data services on the AWS platform, developing pipelines for Data Warehouses and Datalakes with AWS Kinesis and MSK, and effectively utilizing open-source technologies such as Apache Airflow and dbt with Spark/Python or Scala.
We seek expertise in designing, building, and operating batch and real-time data pipelines on AWS. Essential experience includes working with AWS EMR, Glue, Glue Catalog, and Kinesis, as well as creating data layers with AWS RedShift, Aurora, and DynamoDB.
Proficiency in open-source technologies like Apache Airflow and dbt, along with Spark/Python or Spark/Scala, is crucial. Familiarity with AWS Glue Databrew, Lambda functions for data processing, and RedShift Spectrum for data analytics is highly advantageous.
IBM
Data Analytics