Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

This role focuses on designing, constructing, and maintaining robust data engineering solutions within Google's comprehensive cloud ecosystem. You will leverage various Google services for both batch and real-time data pipelines, manage data migration processes, and architect efficient data layers. Curiosity and a drive for innovation are key to succeeding here, as you'll be encouraged to explore new ideas and create impactful solutions that deliver tangible results for leading companies navigating their hybrid cloud and AI journeys.

Responsibilities

Design and develop scalable batch and real-time data pipelines for data warehouses and data lakes using Google Cloud services. Build and maintain data engineering solutions by utilizing Google Cloud Storage, BigTable, BigQuery, DataProc with Spark/Hadoop, and DataFlow with Apache Beam or Python. Ensure efficient data pipeline operations by scheduling and managing the data platform using Google Cloud Scheduler and Cloud Composer (Airflow). Develop and implement seamless data migration solutions between systems. Optimize the data layer for efficient storage and retrieval using services like BigQuery, Big Table, and Cloud Spanner.

Qualifications

Demonstrated experience in designing, building, and maintaining data engineering solutions on Google Cloud, encompassing services like DataProc, DataFlow, PubSub, BigQuery, Big Table, Cloud Spanner, CloudSQL, and AlloyDB. Proven track record in developing and managing batch and real-time data pipelines for Data Warehouses and Datalakes, employing Google services and open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala. Proficiency in Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python. Understanding of data platform management, including scheduling and orchestration with Google Cloud Scheduler and Cloud Composer (Airflow). Experience with data layer design using BigQuery, Big Table, and Cloud Spanner. Familiarity with open-source technologies is a plus.

Essential Skills

Google Cloud PlatformBigQueryDataFlowPubSubBigTableDataProcSparkHadoopApache BeamPythonCloud ComposerCloud SchedulerCloud SpannerCloudSQLAlloyDBdbtScalaData StorageData MigrationProgramming Language

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
IndustryData Analytics
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Data Analytics

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX