Data Engineer-Data Platforms-AWS

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting to shape hybrid cloud and AI journeys for leading companies worldwide. You'll collaborate closely with clients, leveraging IBM technology, Red Hat, and strategic partnerships to drive significant impact. Curiosity is key here; we encourage challenging the status quo and creating innovative solutions. Our growth-focused, empathetic culture supports your long-term career development and values your unique contributions.

As a Data Engineer specializing in Data Platforms on AWS, you will be instrumental in advising on, developing, and maintaining robust data engineering solutions within the AWS Cloud ecosystem.

Responsibilities

Your role involves designing, building, and operating both batch and real-time data pipelines using a variety of AWS services. You will create essential data layers on AWS RedShift, Aurora, and DynamoDB, and manage data migration using AWS DMS.

Key responsibilities include developing data pipelines for Data Warehouses and Datalakes, utilizing services like AWS Kinesis and Managed Streaming for Apache Kafka. You'll also manage data services on the AWS Platform and leverage open-source technologies such as Apache Airflow and dbt, alongside Spark with Python or Scala.

Qualifications

We are looking for expertise in the AWS toolset, including AWS EMR, AWS Glue, Glue Catalog, and Kinesis for pipeline development. Experience with creating data layers on AWS RedShift, Aurora, and DynamoDB, and migrating data with AWS DMS is essential.

Proficiency in open-source technologies like Apache Airflow and dbt, along with Spark (Python/Scala) on AWS, is required. Familiarity with AWS Glue Databrew for data preparation and quality, Lambda functions with Python for processing and transformation, and RedShift Spectrum for warehousing and analytics is also expected.

Essential Skills

AWSData PipelinesData EngineeringAWS GlueAWS EMRKinesisApache KafkaRedshiftAuroraDynamoDBApache AirflowdbtSparkPythonScalaData WarehousingData LakeData ProcessingData TransformationAWS LambdaRedshift SpectrumAWS Glue Databrew

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-AWS
IndustryInformation Technology & Services
DepartmentData Engineering, Software Development
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Information Technology & Services

Data Engineer-Data Platforms-AWS at IBM | SkillMX | SkillMX