Data Engineer-Data Platforms-Google

IBM

Fresher Hyderabad Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting to shape hybrid cloud and AI journeys for leading global companies. We foster long-term client relationships and encourage innovation, curiosity, and growth. As a Data Engineer focused on Google's data platforms, you'll design, build, and maintain robust data solutions within the Google Cloud ecosystem.

This role emphasizes expertise in batch and real-time data pipelines, data migration, and data layer architecture on Google Cloud. You will leverage various Google services to drive impactful client solutions and accelerate business transformation.

Responsibilities

Design and develop sophisticated batch and real-time data pipelines for Data Warehouses and Datelakes using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable.

Build and maintain data engineering solutions by utilizing Google Cloud Storage, BigTable, BigQuery DataProc (with Spark/Hadoop), and Google DataFlow (with Apache Beam or Python).

Manage data platform operations, including scheduling, using Google Cloud Scheduler and Cloud Composer (Airflow) for efficient pipeline execution.

Implement comprehensive data migration strategies using Google services to ensure seamless data transfer.

Optimize data layers with services such as BigQuery, BigTable, and Cloud Spanner for enhanced data storage and retrieval.

Qualifications

Proven experience in designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem, including DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB.

Demonstrated ability in developing and managing batch and real-time data pipelines for Data Warehouses and Datelakes, leveraging Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala.

Proficiency with Google Cloud Storage, BigTable, BigQuery DataProc, and Google DataFlow (with Apache Beam or Python) for building and maintaining data engineering solutions.

Familiarity with scheduling and managing data platforms using Google Cloud Scheduler and Cloud Composer (Airflow) for efficient operations.

Solid understanding of data layer design using BigQuery, BigTable, and Cloud Spanner for effective data storage and retrieval.

Experience with open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala is essential.

Expertise in developing and implementing data migration solutions using Google services.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryDataFlowPubSubBigTableCloud StorageSparkPythonHadoopApache BeamApache AirflowCloud ComposerData MigrationData StorageSchedulingdbt

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

IT Consulting

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX