Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies worldwide. Embrace a culture of curiosity, innovation, and continuous growth, where your unique skills drive meaningful client impact. This role focuses on designing, building, and maintaining robust data engineering solutions within Google's Cloud ecosystem.

As a specialized Data Engineer, you will leverage Google's advanced services to create efficient batch and real-time data pipelines, facilitate data migration, and architect sophisticated data layers. Your expertise will be crucial in transforming data platforms and accelerating client success.

Responsibilities

Design and develop scalable batch and real-time data pipelines using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable for Data Warehouses and Data Lakes.

Build and maintain data engineering solutions leveraging Google Cloud Storage, BigTable, DataProc (Spark/Hadoop), and DataFlow (Apache Beam/Python).

Manage and schedule data platform operations using Google Cloud Scheduler and Cloud Composer (Airflow) for optimal pipeline performance.

Implement comprehensive data migration strategies to ensure seamless data transfer between systems.

Design and optimize data layers using BigQuery, BigTable, and Cloud Spanner for efficient data storage and retrieval.

Qualifications

Demonstrated experience in designing, building, and maintaining data engineering solutions on Google Cloud Platform, utilizing services such as DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB.

Proven ability in developing and managing batch and real-time data pipelines for Data Warehouses and Data Lakes, incorporating Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala.

Proficiency in working with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python).

Familiarity with scheduling and managing data platforms using Google Cloud Scheduler and Cloud Composer (Airflow).

Experience in data layer design with Google services like BigQuery, BigTable, and Cloud Spanner.

Exposure to open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala is highly valued.

Experience implementing data migration solutions using Google services.

Expertise in Cloud Composer (Airflow) for platform scheduling and management.

Essential Skills

Google Cloud PlatformBigQueryDataFlowPub/SubBigTableDataProcCloud ComposerApache AirflowSparkPythonScalaHadoopApache BeamdbtCloud StorageCloud SQLAlloyDBCloud Spanner

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
IndustryInformation Technology & Services
DepartmentData Science
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Information Technology & Services

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX