Data Engineer-Data Platforms-Google

IBM

Fresher Hyderabad Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and contribute to long-term client relationships by shaping their hybrid cloud and AI initiatives. Leverage IBM technology, Red Hat, and strategic partnerships to drive significant change and client impact. Our culture fosters curiosity, innovation, and continuous growth, valuing your unique contributions.

This role focuses on Google's data platforms, requiring you to design, build, and maintain robust data engineering solutions within the Google Cloud ecosystem. You will be instrumental in developing batch and real-time data pipelines, managing data migrations, and optimizing data layers.

Responsibilities

Key responsibilities include designing and developing batch and real-time data pipelines for data warehouses and data lakes using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. You'll build and maintain data engineering solutions utilizing Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam or Python.

Furthermore, you will manage data platform operations through Google Cloud Scheduler and Cloud Composer (Airflow). Implementing efficient data migration solutions and optimizing data layers with services such as BigQuery, BigTable, and Cloud Spanner are also core aspects of this role.

Qualifications

We are seeking individuals with expertise in Google's cloud ecosystem for data engineering, including services like DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB. A strong background in developing and managing batch and real-time data pipelines for data warehouses and data lakes using Google services and open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala is essential.

Proficiency in Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python) is required. Familiarity with scheduling and managing data platforms using Google Cloud Scheduler and Cloud Composer (Airflow) is also expected. Experience with data layer design using BigQuery, BigTable, and Cloud Spanner for efficient data storage and retrieval, along with implementing data migration solutions using Google services, is crucial.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryBigTableDataFlowPubSubDataProcCloud ComposerApache AirflowSparkPythonHadoopApache BeamCloud StorageCloud SpannerCloudSQLAlloyDBdbtScala

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

IT Consulting

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX