Data Engineer-Data Platforms-AWS
IBM
IBM
Join IBM Consulting and embark on a career shaping hybrid cloud and AI journeys for leading global companies. We foster a culture of curiosity, innovation, and continuous growth, supporting your long-term career development.
As a Data Engineer specializing in Data Platforms on AWS, you will be instrumental in designing, developing, and maintaining robust data engineering solutions within the AWS Cloud ecosystem. Your expertise will drive the creation and operation of efficient batch and real-time data pipelines.
Your primary responsibilities will include architecting and building scalable batch and real-time data pipelines utilizing key AWS services like EMR, Glue, and Kinesis. You will be responsible for creating essential data layers on RedShift, Aurora, and DynamoDB, alongside data migration using DMS.
Additionally, you will develop sophisticated data pipelines for Data Warehouses and Datalakes, leveraging services such as AWS Kinesis and MSK. Managing and scheduling data services on the AWS platform will be crucial for ensuring seamless operations and integration of our data engineering solutions.
We seek individuals proficient in leveraging open-source technologies like Apache Airflow and dbt, along with Spark using Python or Scala on the AWS platform to enhance our data engineering solutions.
Familiarity with the comprehensive AWS toolset, including EMR, Glue, and Kinesis, is essential for designing and operating data pipelines. Experience with AWS Databrew for data preparation and quality, and knowledge of Lambda functions with Python for data processing and integration are highly valued. Exposure to RedShift Spectrum for data warehousing and analytics capabilities is also beneficial.
IBM
Information Technology & Services