Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Shape the future of hybrid cloud and AI journeys with IBM Consulting. This role focuses on designing, building, and maintaining robust data engineering solutions within Google's comprehensive cloud ecosystem. You'll leverage cutting-edge Google services to create efficient batch and real-time data pipelines, manage data migrations, and optimize data layer designs for leading companies.

We foster a culture of curiosity and continuous growth, encouraging you to explore new ideas and develop innovative solutions. Your long-term career development is a priority, and your unique skills and experiences are highly valued as we collaborate worldwide to drive significant client impact.

Responsibilities

As a Data Engineer specializing in Google's data platforms, your key responsibilities will include:

- Designing and developing scalable batch and real-time data pipelines using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. - Building and maintaining sophisticated data engineering solutions with Google Cloud Storage, BigTable, BigQuery, DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). - Scheduling and managing data platform operations through Google Cloud Scheduler and Cloud Composer (Airflow) for seamless pipeline execution. - Implementing effective data migration strategies and solutions across various systems. - Designing and optimizing data layers using BigQuery, BigTable, and Cloud Spanner for efficient data management.

Qualifications

We are seeking individuals with extensive experience in:

- Designing, building, and maintaining data engineering solutions on Google Cloud, utilizing services such as DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB. - Developing and managing batch and real-time data pipelines for Data Warehouses and Datelakes with Google services and open-source tools like Apache Airflow, dbt, Spark/Python, or Spark/Scala. - Proficiently working with Google Cloud Storage, BigTable, BigQuery, DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). - Understanding data platform management, including scheduling and orchestration with Google Cloud Scheduler and Cloud Composer (Airflow). - Experience in data layer design using BigQuery, BigTable, and Cloud Spanner. - Utilizing open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala for data pipeline development. - Developing and implementing data migration solutions using Google services. - Expertise with Cloud Composer (Airflow) for platform management.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryDataFlowPubSubBigTableSparkHadoopApache BeamPythonCloud StorageCloud ComposerApache AirflowdbtScalaCloud SpannerCloudSQLAlloyDB

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

IT Consulting