Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and help leading companies shape their hybrid cloud and AI futures. This role focuses on designing, building, and maintaining robust data engineering solutions within Google's Cloud ecosystem. You'll leverage Google's advanced services to create efficient batch and real-time data pipelines, manage data migrations, and optimize data layer design.

This is an opportunity to work with cutting-edge technology and drive meaningful change for clients. IBM Consulting fosters a culture of curiosity and growth, supporting your long-term career development while valuing your unique contributions.

Responsibilities

Key responsibilities include designing and developing both batch and real-time data pipelines using Google services like DataProc, DataFlow, Pub/Sub, BigQuery, and BigTable. You will build and maintain data engineering solutions utilizing Google Cloud Storage, BigTable, and DataProc with Spark/Hadoop, or DataFlow with Apache Beam/Python.

Further duties involve managing data platforms through Google Cloud Scheduler and Cloud Composer (Airflow), implementing data migration solutions, and optimizing the data layer with services such as BigQuery, BigTable, and Cloud Spanner for efficient storage and retrieval.

Qualifications

We are seeking candidates with extensive experience in Google's Cloud ecosystem, specifically in designing, building, and maintaining data engineering solutions. Proficiency in developing and managing batch and real-time data pipelines for Data Warehouses and Data Lakes is essential, utilizing Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala.

Candidates should demonstrate experience with Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam or Python. Familiarity with scheduling and managing data platforms using Cloud Scheduler and Cloud Composer (Airflow) is also required. Experience with data layer design using BigQuery, BigTable, and Cloud Spanner, as well as implementing data migration solutions, is crucial for success in this role.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryDataFlowPub/SubBigTableSparkHadoopApache BeamPythonScalaCloud ComposerApache AirflowdbtCloud StorageCloud SpannerCloud SQLAlloyDBData MigrationData Platform ManagementScheduling

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
IndustryTechnology
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Technology

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX