Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies worldwide. Our collaborative environment, fueled by curiosity and innovation, empowers you to challenge norms and create impactful solutions. We prioritize your long-term career growth, valuing your unique contributions.
As a Data Engineer specializing in Google's data platforms, you'll be instrumental in designing, building, and maintaining robust data engineering solutions within the Google Cloud ecosystem. This role demands expertise in leveraging various Google services for both batch and real-time data pipelines, data migration strategies, and effective data layer design.
Key responsibilities include designing and developing batch and real-time data pipelines using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. You will build and maintain data engineering solutions utilizing Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python.
You'll also schedule and manage the data platform via Google Cloud Scheduler and Cloud Composer (Airflow), implement data migration solutions for seamless system transfers, and optimize the data layer using BigQuery, Big Table, and Cloud Spanner for efficient data handling.
This role requires demonstrated experience in designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem, including services like Google DataProc, DataFlow, PubSub, BigQuery, Big Table, Cloud Spanner, CloudSQL, and AlloyDB.
Candidates should have experience developing and managing batch/real-time data pipelines for Data Warehouses and Datelakes using Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala. Proficiency with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python) is essential. Experience with data layer design using BigQuery, Big Table, and Cloud Spanner, along with data migration solutions, is also expected.
IBM
Data Analytics