Data Engineer-Data Platforms-AWS

IBM

Fresher Pune, Chennai, Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and shape the future of hybrid cloud and AI journeys with leading companies worldwide. We foster a culture of curiosity and innovation, empowering you to challenge norms and create impactful solutions. Your long-term career development is our priority, valuing your unique skills.

As a Data Engineer focused on Data Platforms within the AWS ecosystem, you will be instrumental in advising on, developing, and maintaining robust data engineering solutions. This role demands expertise in designing, building, and operating both batch and real-time data pipelines.

This position offers a hybrid work model, allowing for collaboration while maintaining flexibility, and is based in multiple cities across India, including Pune, Chennai, and Bangalore.

Responsibilities

Key responsibilities include designing, building, and operating batch and real-time data pipelines using a variety of AWS services such as EMR, Glue, Glue Catalog, and Kinesis. You will create data layers on AWS RedShift, Aurora, and DynamoDB, managing data migration with AWS DMS.

Further duties involve scheduling and managing data services on the AWS Platform for efficient operation. You will develop pipelines for Data Warehouse and Datalake, leveraging AWS Kinesis and Managed Streaming for Apache Kafka. The role also requires the utilization of open-source technologies like Apache Airflow, dbt, Spark with Python or Scala on the AWS Platform.

Qualifications

This role requires extensive experience with the AWS toolset, including EMR, Glue, Glue Catalog, and Kinesis for pipeline development. You should have a strong background in creating data layers on RedShift, Aurora, and DynamoDB, and migrating data using AWS DMS.

Proficiency in open-source technologies such as Apache Airflow and dbt, along with Spark/Python or Spark/Scala on AWS, is essential. Experience in scheduling and managing data services on the AWS Platform is also required. Familiarity with AWS Glue Databrew, Lambda functions with Python, and RedShift Spectrum for data warehousing and analytics tasks is beneficial.

Essential Skills

AWSData EngineeringData PipelinesETLPythonScalaSparkSQLAmazon RedshiftApache KafkaAWS GlueAWS KinesisApache AirflowdbtAWS DMSAWS LambdaAWS Elastic MapReduceRedshift Spectrum

Good to Have

Data QualityData PreparationData TransformationAmazon DynamoDBAWS Glue Databrew

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-AWS
IndustryInformation Technology & Services
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Information Technology & Services

Data Engineer-Data Platforms-AWS at IBM | SkillMX | SkillMX