Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and collaborate globally to shape hybrid cloud and AI transformations for leading companies. Leverage IBM technology, Red Hat, and strategic partnerships to drive innovation and client success. This role focuses on designing, building, and maintaining data engineering solutions within Google's Cloud ecosystem.

We are seeking a Data Engineer with expertise in Google's data platforms. You will be instrumental in developing robust batch and real-time data pipelines, managing data migration, and optimizing data layer design. Your curiosity will drive you to challenge norms and create impactful solutions, supported by a culture of growth and empathy dedicated to your long-term career development.

Responsibilities

Key responsibilities include designing and developing comprehensive batch and real-time data pipelines for Data Warehouses and Data Lakes using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable.

You will build and maintain data engineering solutions leveraging Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python. Furthermore, you'll manage data platform operations through Google Cloud Scheduler and Cloud Composer (Airflow).

Implement efficient data migration solutions using Google services and optimize the data layer with BigQuery, BigTable, and Cloud Spanner for effective data storage and retrieval.

Qualifications

A strong understanding of Google's Cloud ecosystem is essential, including services like Google DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB. Experience in developing and managing batch and real-time data pipelines using Google services and open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala is required.

Proficiency in Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python is necessary for building and maintaining data engineering solutions. Familiarity with scheduling and managing data platforms via Google Cloud Scheduler and Cloud Composer (Airflow) is also important.

Experience with data layer design using services like BigQuery, BigTable, and Cloud Spanner for efficient data storage and retrieval is expected. Knowledge of open-source technologies and experience implementing data migration solutions are key to this role.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryBigTableDataProcDataFlowPubSubCloud ComposerApache AirflowSparkPythonHadoopApache BeamScaladbtCloud StorageCloud SpannerCloudSQLAlloyDBData MigrationSchedulingProgramming Language

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

IT Consulting

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX