Data Engineer-Data Platforms-AWS

IBM

Fresher Pune, Chennai, Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Embark on a career with IBM Consulting, where you'll build lasting client relationships and collaborate globally. You will empower leading companies across diverse industries in their hybrid cloud and AI transformation. Leveraging strategic partnerships, robust IBM technology, and Red Hat, you'll be equipped to drive significant change and accelerate client success. At IBM Consulting, a curious mindset is key. We encourage challenging the status quo, exploring novel ideas, and crafting innovative solutions that deliver tangible results. Our growth-oriented and empathetic culture fosters your long-term career development while valuing your unique contributions.

Responsibilities

As a Data Engineer focused on Data Platforms within the AWS ecosystem, you will advise on, develop, and maintain data engineering solutions. This includes designing, building, and operating batch and real-time data pipelines using various AWS services. Key responsibilities involve creating data layers on AWS RedShift, Aurora, and DynamoDB, migrating data with AWS DMS, and managing data services on the AWS Platform.

You will develop batch and real-time pipelines for Data Warehouses and Data Lakes, utilizing AWS Kinesis and Managed Streaming for Apache Kafka. The role requires leveraging open-source technologies like Apache Airflow and dbt, along with Spark/Python or Spark/Scala on the AWS Platform.

Qualifications

This role requires hands-on experience with AWS services including EMR, Glue, Glue Catalog, and Kinesis for designing and operating data pipelines. You should have experience with AWS RedShift, Aurora, and DynamoDB for data layer creation and data migration using AWS DMS.

Familiarity with open-source technologies such as Apache Airflow and dbt, along with Spark/Python or Spark/Scala on the AWS Platform is essential. Experience in scheduling and managing data services on AWS, proficiency with AWS Glue Databrew for data preparation and quality, and knowledge of Lambda functions with Python for data processing are expected.

An understanding of RedShift Spectrum for data warehousing and analytics tasks is also beneficial for this position.

Essential Skills

AWS EMRAWS GlueGlue CatalogAWS KinesisAWS RedshiftAmazon AuroraAmazon DynamoDBAWS DMSApache KafkaApache AirflowdbtSparkPythonScalaAWS LambdaRedshift SpectrumAWS Glue Databrew

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-AWS
IndustryInformation Technology & Services
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Information Technology & Services

Data Engineer-Data Platforms-AWS at IBM | SkillMX | SkillMX