Data Engineer-Data Platforms-AWS
IBM
IBM
Join IBM Consulting to shape hybrid cloud and AI journeys for leading companies worldwide. You'll collaborate closely with clients, leveraging IBM technology, Red Hat, and strategic partnerships to drive significant impact. Curiosity is key here; we encourage challenging the status quo and creating innovative solutions. Our growth-focused, empathetic culture supports your long-term career development and values your unique contributions.
As a Data Engineer specializing in Data Platforms on AWS, you will be instrumental in advising on, developing, and maintaining robust data engineering solutions within the AWS Cloud ecosystem.
Your role involves designing, building, and operating both batch and real-time data pipelines using a variety of AWS services. You will create essential data layers on AWS RedShift, Aurora, and DynamoDB, and manage data migration using AWS DMS.
Key responsibilities include developing data pipelines for Data Warehouses and Datalakes, utilizing services like AWS Kinesis and Managed Streaming for Apache Kafka. You'll also manage data services on the AWS Platform and leverage open-source technologies such as Apache Airflow and dbt, alongside Spark with Python or Scala.
We are looking for expertise in the AWS toolset, including AWS EMR, AWS Glue, Glue Catalog, and Kinesis for pipeline development. Experience with creating data layers on AWS RedShift, Aurora, and DynamoDB, and migrating data with AWS DMS is essential.
Proficiency in open-source technologies like Apache Airflow and dbt, along with Spark (Python/Scala) on AWS, is required. Familiarity with AWS Glue Databrew for data preparation and quality, Lambda functions with Python for processing and transformation, and RedShift Spectrum for warehousing and analytics is also expected.
IBM
Information Technology & Services