Data Engineer-Data Platforms-Google

IBM

Fresher Hyderabad Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies globally. We foster long-term client relationships through close collaboration, leveraging robust IBM technology and Red Hat. Curiosity is key to our success; we encourage challenging norms and innovating for real results. Our growth-focused culture prioritizes your career development while valuing your unique contributions.

This role focuses on designing, building, and maintaining data engineering solutions within Google's Cloud ecosystem. You'll expertly utilize various Google services for batch and real-time data pipelines, data migration, and data layer design, driving impactful client outcomes.

Responsibilities

Design and develop robust batch and real-time data pipelines for Data Warehouses and Datasets. This involves leveraging Google services such as DataProc, DataFlow, PubSub, BigQuery, and BigTable.

Build and maintain sophisticated data engineering solutions using Google Cloud Storage, BigTable, BigQuery with Spark/Hadoop, and Google DataFlow with Apache Beam or Python.

Schedule and meticulously manage the data platform using Google Cloud Scheduler and Cloud Composer (Airflow), ensuring optimal data pipeline performance.

Develop and implement seamless data migration solutions across various systems using Google's suite of services.

Design and optimize the data layer using Google services like BigQuery, BigTable, and Cloud Spanner for efficient data storage and retrieval.

Qualifications

Demonstrate extensive experience in designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem, including services like DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB.

Possess proven expertise in developing and managing batch and real-time data pipelines for Data Warehouses and Datasets, utilizing Google services alongside open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala.

Exhibit strong proficiency with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python) for building and sustaining data engineering solutions.

Understand the intricacies of scheduling and managing data platforms through Google Cloud Scheduler and Cloud Composer (Airflow) for efficient pipeline operations.

Showcase practical experience in data layer design using BigQuery, BigTable, and Cloud Spanner to ensure optimized data storage and retrieval.

Be adept at developing and implementing data migration solutions using Google services for smooth data transfers.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryDataFlowPubSubBigTableCloud ComposerApache AirflowSparkPythonHadoopApache BeamCloud Storage

Good to Have

dbtScalaCloud SpannerCloudSQLAlloyDB

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
IndustrySoftware
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Software

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX