Data Engineer-Data Platforms-Google

IBM

Fresher Hyderabad Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and shape the future of hybrid cloud and AI. You'll build long-term client relationships, collaborate globally, and drive meaningful change using robust IBM technology and Red Hat. Our culture fosters curiosity, innovation, and your long-term career development, valuing your unique contributions.

This role focuses on designing, building, and maintaining data engineering solutions within Google's Cloud ecosystem. You will leverage various Google services for both batch and real-time data pipelines, data migration, and data layer design.

Responsibilities

As a Data Engineer, your key responsibilities will involve designing and developing robust batch and real-time data pipelines for data warehouses and data lakes. This includes utilizing Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable.

You will also develop and maintain data engineering solutions using Google Cloud Storage, BigTable, and DataProc with Spark/Hadoop, or DataFlow with Apache Beam/Python. Furthermore, you'll manage the data platform through Google Cloud Scheduler and Cloud Composer (Airflow), ensuring efficient pipeline operations. Implementing data migration solutions and optimizing the data layer with services such as BigQuery, BigTable, and Cloud Spanner are also critical aspects of this role.

Qualifications

We are seeking individuals with extensive experience in Google's Cloud ecosystem, including designing and building data engineering solutions. Proficiency in developing and managing batch and real-time data pipelines for data warehouses and data lakes using Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala is essential.

Demonstrated experience working with Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam/Python is required. Knowledge of scheduling and managing data platforms using Google Cloud Scheduler and Cloud Composer (Airflow) is also expected. Understanding data layer design with Google services like BigQuery, BigTable, and Cloud Spanner, alongside experience in implementing data migration solutions, will be highly valued.

Essential Skills

Google Cloud PlatformBigQueryDataflowPub/SubBigtableDataProcCloud StorageCloud ComposerApache AirflowSparkHadoopPythonApache BeamData MigrationData EngineeringData PipelinesData Platform ManagementData Layer DesignOpen Source TechnologiesdbtCloud SpannerCloudSQLAlloyDB

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
IndustrySoftware Development
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Software Development

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX