Data Engineer-Data Platforms-AWS

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Embark on a career with IBM Consulting, where innovation thrives on long-term client relationships and global collaboration. You'll partner with leading companies across diverse industries, guiding their hybrid cloud and AI transformations. Leveraging robust IBM technology, Red Hat, and strategic partner support, you'll possess the resources to drive significant change and accelerate client success. Curiosity is the engine of achievement at IBM Consulting; we encourage challenging the status quo, exploring novel ideas, and developing groundbreaking solutions.

Our growth-focused and empathetic culture prioritizes your long-term career development while celebrating your unique talents and contributions. As a Data Engineer specializing in Data Platforms on AWS, you will be instrumental in advising on, developing, and maintaining sophisticated data engineering solutions within the AWS Cloud ecosystem. Your role will involve designing, building, and operating both batch and real-time data pipelines using a variety of AWS services.

Responsibilities

Key responsibilities include designing and developing robust data pipelines using AWS services like EMR, Glue, Glue Catalog, and Kinesis, ensuring seamless integration and operational efficiency. You will also create essential data layers on AWS RedShift, Aurora, and DynamoDB, migrating data effectively with AWS DMS.

Furthermore, you will manage data services on the AWS Platform, ensuring optimized performance. The role also involves developing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka. Proficiency in open-source technologies such as Apache Airflow and dbt, along with Spark/Python or Spark/Scala on the AWS Platform, is crucial for supporting these data engineering solutions.

Qualifications

Essential experience includes working with AWS services (EMR, Glue, Glue Catalog, Kinesis) for data pipeline development. You should be adept at creating data layers on AWS RedShift, Aurora, and DynamoDB, and migrating data using AWS DMS.

Familiarity with open-source technologies like Apache Airflow and dbt, and Spark/Python or Spark/Scala on AWS is required. Experience in scheduling and managing data services on AWS is also necessary. Exposure to AWS Glue Databrew for data preparation and quality, Lambda functions with Python for data processing and transformation, and RedShift Spectrum for warehousing and analytics are key aspects of this role.

Essential Skills

AWS GlueAWS KinesisAWS EMRApache KafkaApache AirflowPythonSparkData WarehousingData EngineeringData PipelinesAWS LambdaAmazon RedshiftdbtAmazon DynamoDB

Good to Have

AWS DMSAWS Redshift SpectrumAWS Glue DatabrewScala

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-AWS
IndustryInformation Technology & Services, Data Analytics
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Information Technology & Services

Data Engineer-Data Platforms-AWS at IBM | SkillMX | SkillMX