Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting and contribute to shaping the hybrid cloud and AI journeys of leading global companies. This role offers a dynamic environment focused on long-term client relationships and worldwide collaboration. Leverage IBM technology, Red Hat, and strategic partnerships to drive significant client impact. We encourage curiosity, innovation, and continuous professional growth, valuing your unique contributions.
As a Data Engineer focusing on Google's data platforms, you will architect, develop, and maintain robust data engineering solutions within the Google Cloud ecosystem. This includes designing and implementing batch and real-time data pipelines, managing data migrations, and optimizing data layer design across various Google services.
Key tasks involve building data pipelines using services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. You will also develop data engineering solutions utilizing Google Cloud Storage and manage data platforms through Cloud Scheduler and Cloud Composer for efficient pipeline operations.
This role requires deep expertise in Google's data platforms, including extensive experience with services such as DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB. Proven ability to design and manage batch and real-time data pipelines for data warehouses and datalakes using open-source technologies like Apache Airflow, dbt, Spark (Python/Scala), and Apache Beam is essential.
Familiarity with data migration strategies and optimizing data layer design using BigQuery, BigTable, and Cloud Spanner is also expected. Experience with Cloud Composer for scheduling and managing data platform operations is a significant advantage.
IBM
IT Consulting