Data Engineer-Data Platforms-AWS
IBM
IBM
Embark on a career with IBM Consulting, where you'll cultivate enduring client relationships and collaborate globally. You'll empower leading organizations across diverse industries to navigate their hybrid cloud and AI strategies. Leveraging robust IBM technology, Red Hat, and support from our strategic partners, you'll be equipped to drive significant transformations and deliver impactful client outcomes. Curiosity is the cornerstone of success at IBM Consulting. We foster an environment that encourages challenging the status quo, exploring innovative concepts, and developing groundbreaking solutions for tangible results. Our growth-oriented and empathetic culture prioritizes your long-term career advancement while celebrating your unique talents and experiences.
As a Data Engineer focused on Data Platforms within the AWS ecosystem, your role involves advising on, developing, and maintaining sophisticated data engineering solutions.
You will be responsible for designing, building, and operating both batch and real-time data pipelines, utilizing a range of AWS services to ensure seamless integration and optimal performance.
Key duties include creating data layers on AWS RedShift, Aurora, and DynamoDB, and managing data migration via AWS DMS. You'll also schedule and oversee data services on the AWS Platform, develop pipelines for Data Warehouses and Datalakes using AWS Kinesis and Managed Streaming for Apache Kafka, and integrate open-source technologies like Apache Airflow and dbt with Spark/Python or Spark/Scala on AWS.
Demonstrate experience in designing, building, and operating batch and real-time data pipelines with AWS services such as EMR, Glue, Glue Catalog, and Kinesis.
Possess experience in creating data layers using AWS RedShift, Aurora, and DynamoDB, and migrating data with AWS DMS.
Showcase proficiency in scheduling and managing data services on the AWS Platform.
Highlight experience developing batch and real-time pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka.
Emphasize experience with open-source technologies like Apache Airflow and dbt, alongside Spark/Python or Spark/Scala on AWS.
Familiarity with AWS Glue Databrew for data preparation and quality tasks is a plus.
Exposure to Lambda functions with Python for data processing and transformation is beneficial.
Experience with RedShift Spectrum for data warehousing and analytics tasks is valued.
IBM
Information Technology & Services