Data Engineer-Data Platforms-AWS
IBM
IBM
Embark on a rewarding career in IBM Consulting, where you'll cultivate enduring client relationships and engage in global collaboration. You will partner with industry-leading companies, guiding them through their hybrid cloud and AI transformations. Leveraging robust IBM technology, Red Hat, and the support of our strategic partners, you'll be equipped to drive significant change and accelerate client success. At IBM Consulting, curiosity is the engine of achievement. We foster an environment where challenging the status quo, exploring novel ideas, and crafting innovative solutions are encouraged to deliver tangible results. Our culture prioritizes continuous growth and empathy, focusing on your long-term career development while celebrating your unique contributions.
As a Data Engineer focused on Data Platforms within the AWS ecosystem, you will be instrumental in advising on, developing, and maintaining sophisticated data engineering solutions. Your role involves designing, building, and operating both batch and real-time data pipelines utilizing a range of AWS services. Key responsibilities include developing and managing data layers on AWS RedShift, Aurora, and DynamoDB, alongside data migration tasks using AWS DMS. You will also schedule and oversee data services on the AWS platform, ensuring optimal performance of data engineering solutions. This includes developing robust data pipelines for Data Warehouses and Data Lakes, leveraging AWS Kinesis and Managed Streaming for Apache Kafka.
This role requires a strong command of AWS services such as EMR, Glue, Glue Catalog, and Kinesis for building and operating data pipelines. Experience with AWS RedShift, Aurora, and DynamoDB for data layer creation and migration via AWS DMS is essential. Proficiency in utilizing open-source technologies like Apache Airflow and dbt, along with Spark/Python or Spark/Scala on the AWS platform, is expected. Familiarity with AWS Glue Databrew for data preparation and quality, Lambda functions with Python for processing and transformation, and RedShift Spectrum for warehousing and analytics tasks is also crucial.
IBM
Data Analytics