Data Engineer-Data Platforms-AWS

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies. As a Data Engineer on our Data Platforms team, you'll be instrumental in developing and maintaining advanced data engineering solutions within the AWS ecosystem.

This role offers a dynamic environment where curiosity and innovation drive success. You'll collaborate globally, leveraging robust IBM technology and Red Hat to accelerate client impact and foster your long-term career development in a culture of growth and empathy.

Responsibilities

You will design, build, and operate robust batch and real-time data pipelines using key AWS services like EMR, Glue, Glue Catalog, and Kinesis. Responsibilities include creating comprehensive data layers on RedShift, Aurora, and DynamoDB, and migrating data efficiently using AWS DMS.

Further tasks involve managing data services on the AWS platform, developing pipelines for Data Warehouses and Datalakes with AWS Kinesis and MSK, and effectively utilizing open-source technologies such as Apache Airflow and dbt with Spark/Python or Scala.

Qualifications

We seek expertise in designing, building, and operating batch and real-time data pipelines on AWS. Essential experience includes working with AWS EMR, Glue, Glue Catalog, and Kinesis, as well as creating data layers with AWS RedShift, Aurora, and DynamoDB.

Proficiency in open-source technologies like Apache Airflow and dbt, along with Spark/Python or Spark/Scala, is crucial. Familiarity with AWS Glue Databrew, Lambda functions for data processing, and RedShift Spectrum for data analytics is highly advantageous.

Essential Skills

AWSData EngineeringData PipelinesData WarehousingSparkPythonScalaApache AirflowAWS GlueAWS KinesisAWS EMRAWS RedshiftAmazon DynamoDBData QualityData PreparationData TransformationSchedulingOpen Source

Good to Have

dbtAWS LambdaRedshift SpectrumApache KafkaAWS DMSAWS Glue Databrew

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-AWS
IndustryData Analytics
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Data Analytics

Data Engineer-Data Platforms-AWS at IBM | SkillMX | SkillMX