Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Shape the future of hybrid cloud and AI journeys with IBM Consulting. Collaborate closely with leading companies across diverse industries, driving meaningful change and client impact. At IBM, curiosity fuels innovation; we encourage challenging norms and creating groundbreaking solutions. Our culture fosters long-term career development, valuing your unique skills and experiences.

As a Data Engineer specializing in Google's data platforms, you will be instrumental in designing, building, and maintaining robust data engineering solutions within Google's Cloud ecosystem. This role demands proficiency in leveraging various Google services for both batch and real-time data pipelines, data migration, and comprehensive data layer design.

Responsibilities

Your core responsibilities will include designing and developing sophisticated batch and real-time data pipelines for Data Warehouses and Data Lakes using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable.

You will also be responsible for developing and maintaining scalable data engineering solutions using Google Cloud Storage, BigTable, BigQuery DataProc (with Spark and Hadoop), and Google DataFlow (with Apache Beam or Python). Efficiently manage data platform operations through scheduling and orchestration with Google Cloud Scheduler and Cloud Composer (Airflow).

Furthermore, you will implement and optimize data migration solutions, ensuring seamless data transfer. Design and refine the data layer using services such as BigQuery, BigTable, and Cloud Spanner for optimal data storage and retrieval.

Qualifications

A strong foundation in Google's Cloud ecosystem is essential, encompassing the design, development, and maintenance of data engineering solutions. This includes hands-on experience with Google DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB.

Demonstrated experience in developing and managing batch and real-time data pipelines for Data Warehouses and Data Lakes is crucial. Proficiency with open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala is expected.

Working knowledge of Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python is required. Familiarity with data platform management, scheduling, and orchestration using Google Cloud Scheduler and Cloud Composer (Airflow) is also necessary.

Expertise in data layer design using BigQuery, BigTable, and Cloud Spanner for efficient data storage and retrieval, along with experience in data migration solutions using Google services, is vital for success in this role.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryDataFlowPubSubBigTableCloud ComposerApache AirflowSparkPythonHadoopScaladbtCloud StorageCloud SpannerCloud SQLAlloyDBDataProc

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
IndustryData Analytics
DepartmentData Engineering, Software Development
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Data Analytics

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX