Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting to build lasting client relationships and drive innovation in hybrid cloud and AI. You'll collaborate globally, shaping client journeys with robust IBM technology and Red Hat support. Curiosity is key; we encourage challenging the status quo and creating impactful solutions. Our growth-focused culture supports your long-term career development while valuing your unique contributions.

As a Data Engineer specializing in Google's data platforms, your role involves designing, building, and maintaining data engineering solutions within the Google Cloud ecosystem. This position requires expertise in leveraging various Google services for both batch and real-time data pipelines, data migration, and effective data layer design.

Responsibilities

Design and develop robust batch and real-time data pipelines for Data Warehouses and Datelakes utilizing Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable.

Build and maintain scalable data engineering solutions using Google Cloud Storage, BigTable, BigQuery DataProc (with Spark and Hadoop), and Google DataFlow (with Apache Beam or Python).

Manage the data platform efficiently through scheduling and orchestration using Google Cloud Scheduler and Cloud Composer (Airflow).

Implement effective data migration solutions, ensuring seamless data transfer between various systems using Google services.

Design and optimize the data layer with Google services such as BigQuery, BigTable, and Cloud Spanner for efficient data storage and retrieval.

Qualifications

Demonstrated experience in designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem, including services such as Google DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB.

Proven track record in developing and managing batch and real-time data pipelines for Data Warehouses and Datelakes, utilizing Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala.

Proficiency in working with Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python for building and maintaining data engineering solutions.

Understanding of data platform management, including scheduling and managing operations using Google Cloud Scheduler and Cloud Composer (Airflow).

Experience in data layer design using Google services such as BigQuery, BigTable, and Cloud Spanner, ensuring optimal data storage and retrieval. Familiarity with implementing data migration solutions using Google services.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryDataFlowPubSubBigTableSparkPythonHadoopCloud SchedulerCloud ComposerAirflowData MigrationCloud SpannerCloud SQLAlloyDBApache BeamdbtScalaGoogle Cloud Storage

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
IndustryTechnology
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Technology

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX