Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and shape the future of hybrid cloud and AI journeys for leading companies worldwide. You'll collaborate closely with clients, leveraging robust IBM technology, Red Hat, and strategic partnerships to drive meaningful change and accelerate client impact. At IBM Consulting, curiosity is key to success. We encourage challenging the norm, exploring new ideas, and creating innovative solutions.

This role focuses on designing, building, and maintaining data engineering solutions within Google's Cloud ecosystem. It requires expertise in utilizing various Google services for both batch and real-time data pipelines, data migration, and data layer design.

Responsibilities

As a Data Engineer specializing in Google's data platforms, you will:

- Design and develop robust batch and real-time data pipelines for Data Warehouses and Data Lakes using Google services like DataProc, DataFlow, Pub/Sub, BigQuery, and BigTable. - Build and maintain sophisticated data engineering solutions leveraging Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam or Python. - Efficiently schedule and manage data platform operations using Google Cloud Scheduler and Cloud Composer (Airflow) to ensure optimal data pipeline performance. - Develop and implement seamless data migration solutions using Google services to ensure smooth data transfer between systems. - Design and optimize data layers using Google services such as BigQuery, BigTable, and Cloud Spanner for efficient data storage and retrieval.

Qualifications

We are seeking professionals with:

- Deep expertise in designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem, including services like DataProc, DataFlow, Pub/Sub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB. - Proven experience in developing and managing batch and real-time data pipelines for Data Warehouses and Data Lakes, utilizing Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala. - Proficiency with Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam or Python. - Understanding of data platform management, including scheduling and managing operations with Google Cloud Scheduler and Cloud Composer (Airflow). - Experience in data layer design using Google services such as BigQuery, BigTable, and Cloud Spanner. - Familiarity with open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala. - Experience implementing data migration solutions using Google services.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryDataFlowPub/SubBigTableCloud SpannerCloud SQLAlloyDBDataProcSparkHadoopApache BeamPythonScalaApache AirflowdbtCloud ComposerCloud SchedulerData MigrationData Storage

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

IT Consulting

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX