Data Engineer-Data Platforms-AWS

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting as a Data Engineer focused on Data Platforms within the AWS ecosystem. This role involves collaborating with leading companies to shape their hybrid cloud and AI strategies. You will develop and maintain robust data engineering solutions, leveraging IBM's technology and strategic partnerships to drive significant client impact.

At IBM Consulting, curiosity and innovation are highly valued. We encourage challenging the status quo and exploring new ideas to create effective solutions. Our supportive culture prioritizes long-term career development, recognizing and celebrating your unique skills and contributions.

Responsibilities

Design, build, and manage batch and real-time data pipelines utilizing AWS services like EMR, Glue, and Kinesis. Develop data layers on RedShift, Aurora, and DynamoDB, and perform data migration using AWS DMS. Schedule and manage data services efficiently on the AWS platform. Create comprehensive data pipelines for Data Warehouses and Data Lakes, incorporating AWS Kinesis and Managed Streaming for Apache Kafka.

Utilize open-source technologies such as Apache Airflow and dbt, along with Spark/Python or Spark/Scala on the AWS platform. Gain experience with AWS Glue Databrew for data preparation and quality. Apply Lambda functions with Python for data processing and transformation. Work with RedShift Spectrum for advanced data warehousing and analytics tasks.

Qualifications

Proven experience in designing and developing batch and real-time data pipelines on AWS. Expertise in AWS services including EMR, Glue, Glue Catalog, Kinesis, RedShift, Aurora, and DynamoDB. Familiarity with data migration tools like AWS DMS. Proficiency in using open-source technologies such as Apache Airflow and dbt. Experience with Spark/Python or Spark/Scala. Exposure to AWS Lambda functions and RedShift Spectrum is beneficial.

Understanding of data warehousing, data lakes, data preparation, and data quality concepts is essential. The ability to schedule and manage data services on the AWS platform is required. Experience with AWS Glue Databrew and Managed Streaming for Apache Kafka is a plus.

Essential Skills

AWSData PipelinesData EngineeringData WarehousingData ProcessingPythonSparkApache KafkaAWS GlueAWS KinesisAWS EMRAmazon RedshiftAmazon DynamoDBApache Airflowdbt

Good to Have

ScalaAWS DMSAWS LambdaAWS Redshift SpectrumAWS Glue DatabrewManaged Streaming for Apache Kafka

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-AWS
IndustryInformation Technology & Services, Cloud Computing
DepartmentData Engineering, Software Development
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Information Technology & Services

Data Engineer-Data Platforms-AWS at IBM | SkillMX | SkillMX