Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting and contribute to long-term client relationships, shaping hybrid cloud and AI journeys. Leverage IBM technology, Red Hat, and strategic partnerships to drive significant change and client impact. Our culture fosters curiosity, innovation, and growth, supporting your career development while valuing your unique contributions.
As a Data Engineer focused on Google's data platforms, you will be instrumental in designing, building, and maintaining robust data engineering solutions within the Google Cloud ecosystem. This role demands proficiency in leveraging various Google services for both batch and real-time data pipelines, data migration strategies, and effective data layer design.
Your core responsibilities will involve architecting and developing efficient batch and real-time data pipelines for Data Warehouse and Datalake environments using Google services like DataProc, DataFlow, PubSub, BigQuery, and Big Table.
You will build and manage data engineering solutions with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). Ensuring smooth data platform operations through scheduling and management using Google Cloud Scheduler and Cloud Composer (Airflow) is also key.
Additionally, you will implement comprehensive data migration solutions and design/optimize data layers using services such as BigQuery, Big Table, and Cloud Spanner for optimal data storage and retrieval.
Demonstrate expertise in the Google Cloud ecosystem, including designing, building, and maintaining data engineering solutions with services like Google DataProc, DataFlow, PubSub, BigQuery, Big Table, Cloud Spanner, CloudSQL, and AlloyDB.
Possess a strong background in developing and managing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing Google services alongside open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala.
Showcase proficiency with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). Familiarity with data platform management, data migration solutions, and Cloud Composer (Airflow) for scheduling and operational efficiency is essential. Experience with data layer design using BigQuery, Big Table, and Cloud Spanner is also required.
IBM
IT Consulting