GCP Data Architect

Persistent

14+ yrs Bengaluru Full Time Hybrid (office + remote)
Persistent logo
Posted : today
Actively hiring

Job description

This role focuses on leading the end-to-end data architecture for the IAM Data Modernization program. The project involves migrating an on-premises SQL data warehouse to a data lake on Google Cloud Platform (GCP), enabling advanced analytics, reporting, and GenAI use cases. The initiative aims to establish a single source of truth for enterprise-wide data-driven decision-making.

The program modernizes enterprise IAM data platforms, transitioning from on-premises systems to a scalable, metadata-driven architecture. The current phase builds a generic metadata-driven ETL framework for ingesting data into OpenShift/S3, with future integration into GCP/BigQuery for advanced analytics and GenAI.

Responsibilities

Key responsibilities include owning the data architecture across OpenShift, S3, PostgreSQL, and GCP/BigQuery. You will define and guide the implementation of a metadata-driven ETL framework for diverse source types. Architecting reusable metadata models and providing architecture direction for MVC-based orchestration are crucial. Design scalable ingestion patterns for various scenarios and guide the design of S3 data lake zones with robust error handling.

Define data quality, schema validation, and operational resilience standards. Provide technical leadership for GCP/BigQuery architecture, including integration, schema mapping, and performance optimization. Review and guide CI/CD automation for deployments and configuration management. Offer technical guidance on Python, PySpark, SQL, and BigQuery. Support UAT and production readiness through validation, reconciliation, and performance testing. Partner with stakeholders to finalize architecture decisions and documentation.

Qualifications

A minimum of 14 years of overall IT experience is required, with substantial expertise in Data Architecture, Big Data Engineering, Cloud Data Platforms, and ETL/ELT Frameworks. Experience with Data Lake Architecture and Enterprise Data Modernization programs is essential.

Hands-on experience with PySpark for ETL/ELT, data transformation, and performance optimization is a must. Solid understanding of distributed data processing concepts is expected. Strong experience designing data platforms on Google Cloud Platform (GCP), along with experience in Data Lakes, data warehousing, and large-scale migration programs, is critical.

Proven experience designing and implementing data lake architectures and strong knowledge of Cloud Storage (GCS) design are necessary. Familiarity with Java and Security is also required for this hybrid role in Bengaluru, working 3 days a week from the client office.

Essential Skills

PySparkGCPData ArchitectureBig Data EngineeringCloud Data PlatformsETL/ELT FrameworksData Lake ArchitectureEnterprise Data ModernizationDevOps CI/CDOpenShiftSQLdata modelingperformance tuningPostgreSQLBigQueryGitHub ActionsJFrogHarnessLiquibaseHelmJavaSecuritySplunkVDSActive DirectoryMongoDBGrafana

Highlights

  • Actively hiring

More Details

RoleGCP Data Architect
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Persistent Systems logo

Persistent Systems

IT Consulting

GCP Data Architect at Persistent | SkillMX | SkillMX