Data Engineer-Data Platforms-Google

IBM

Fresher Hyderabad Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and contribute to long-term client relationships, shaping hybrid cloud and AI journeys. Leverage IBM technology, Red Hat, and strategic partnerships to drive significant change and client impact. Our culture fosters curiosity, innovation, and growth, supporting your career development while valuing your unique contributions.

As a Data Engineer focused on Google's data platforms, you will be instrumental in designing, building, and maintaining robust data engineering solutions within the Google Cloud ecosystem. This role demands proficiency in leveraging various Google services for both batch and real-time data pipelines, data migration strategies, and effective data layer design.

Responsibilities

Your core responsibilities will involve architecting and developing efficient batch and real-time data pipelines for Data Warehouse and Datalake environments using Google services like DataProc, DataFlow, PubSub, BigQuery, and Big Table.

You will build and manage data engineering solutions with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). Ensuring smooth data platform operations through scheduling and management using Google Cloud Scheduler and Cloud Composer (Airflow) is also key.

Additionally, you will implement comprehensive data migration solutions and design/optimize data layers using services such as BigQuery, Big Table, and Cloud Spanner for optimal data storage and retrieval.

Qualifications

Demonstrate expertise in the Google Cloud ecosystem, including designing, building, and maintaining data engineering solutions with services like Google DataProc, DataFlow, PubSub, BigQuery, Big Table, Cloud Spanner, CloudSQL, and AlloyDB.

Possess a strong background in developing and managing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing Google services alongside open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala.

Showcase proficiency with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). Familiarity with data platform management, data migration solutions, and Cloud Composer (Airflow) for scheduling and operational efficiency is essential. Experience with data layer design using BigQuery, Big Table, and Cloud Spanner is also required.

Essential Skills

PythonSparkHadoopData EngineeringData PipelinesApache AirflowData PlatformOpen SourceApache BeamBigTableReal-time DataGoogle BigQueryCloud StorageData MigrationSchedulingGoogle Cloud

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

IT Consulting

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX