Data Engineer-Data Platforms-Google

IBM

Fresher Hyderabad Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Embark on a career shaping hybrid cloud and AI journeys for leading companies. This role focuses on designing, building, and maintaining robust data engineering solutions within Google's Cloud ecosystem. You will leverage cutting-edge Google services to create efficient batch and real-time data pipelines, manage data migrations, and optimize data layer architecture.

Responsibilities

Key responsibilities include designing and developing batch and real-time data pipelines using services like DataProc, DataFlow, PubSub, and BigQuery. You will build and maintain data engineering solutions leveraging Google Cloud Storage, BigTable, and DataFlow with Apache Beam or Python. Managing data platforms through Cloud Scheduler and Cloud Composer (Airflow) is crucial, as is implementing seamless data migration solutions. Optimizing the data layer with BigQuery, BigTable, and Cloud Spanner ensures efficient data storage and retrieval.

Qualifications

This role requires hands-on experience with Google's Cloud ecosystem, including services such as DataProc, DataFlow, PubSub, BigQuery, and BigTable. Proficiency in developing and managing data pipelines with technologies like Spark/Python and Apache Airflow is essential. Experience with data layer design and implementing data migration solutions using Google services is expected. Familiarity with open-source technologies like dbt and Cloud Composer (Airflow) for platform management is a significant advantage.

Essential Skills

Data EngineeringData PipelinesGoogle BigQueryGoogle Cloud StorageBigTableData MigrationSparkPythonHadoopApache BeamApache AirflowdbtCloud ComposerSQL

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
IndustrySoftware
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Software

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX