Senior Technical Architect (IN)

HCLTech

Fresher India Full Time Work from office
HCLTech logo
Posted : today
Actively hiring

Job description

Join our team as a Senior Technical Architect, focusing on designing and deploying sophisticated end-to-end data pipelines within cloud environments, with a preference for GCP. This role involves building scalable workflows for data ingestion, transformation, and processing, leveraging distributed technologies like Spark, Flink, or Storm.

You will develop advanced ELT/ETL and migration pipelines, contributing to the modernization of legacy systems like Datastage. The position requires expertise in diverse database technologies, including relational, NoSQL, MPP, and columnar stores such as BigQuery, Redshift, Azure SQLDW, HBase, and MongoDB.

Furthermore, you'll implement real-time streaming and messaging-based pipelines using Kafka, Pulsar, or Pub/Sub. A key aspect of this role is constructing optimized, scalable data models to support various consumption needs, utilizing strategies like partitioning, sharding, bucketing, and aggregation. You'll also be responsible for performance tuning and optimization across storage, compute, and query layers.

Responsibilities

Design and implement comprehensive end-to-end data pipelines on cloud platforms, prioritizing GCP.

Develop highly scalable data ingestion, transformation, and processing workflows using distributed computing frameworks such as Spark, Flink, and Storm.

Create robust ELT/ETL processing and migration solutions, including the decommissioning and modernization of legacy Datastage systems.

Work extensively with a broad spectrum of database technologies, encompassing relational, NoSQL, MPP, and columnar stores like BigQuery, Redshift, Azure SQLDW, HBase, and MongoDB.

Implement and manage streaming and messaging-based pipelines utilizing Kafka, Pulsar, or Pub/Sub.

Construct optimized and scalable data models designed for diverse consumption patterns, employing techniques such as partitioning, sharding, bucketing, and aggregation.

Apply performance tuning and optimization strategies across all layers of data storage, compute, and querying.

Qualifications

Proven experience in designing and delivering end-to-end data pipelines on cloud platforms, with a strong preference for GCP.

Expertise in building scalable data ingestion, transformation, and processing workflows using distributed technologies (e.g., Spark, Flink, Storm).

Demonstrated ability to develop robust ELT/ETL processing and migration pipelines, including experience with legacy system modernization.

Proficiency in working with a variety of database technologies: relational, NoSQL, MPP, and columnar stores (e.g., BigQuery, Redshift, Azure SQLDW, HBase, MongoDB).

Experience implementing streaming and messaging-based pipelines (e.g., Kafka, Pulsar, Pub/Sub).

Skilled in building optimized, scalable data models for diverse consumption needs, applying partitioning, sharding, bucketing, and aggregation.

Strong capability in performance tuning and optimization across storage, compute, and query layers.

Essential Skills

GCPSparkFlinkStormELTETLDatastageBigQueryRedshiftAzure SQLDWHBaseMongoDBKafkaPulsarPub/SubData ModelingPerformance Tuning

Highlights

  • Actively hiring

More Details

RoleSenior Technical Architect (IN)
Employment TypeFull Time, Work from office

About the Company

HCLTech logo

HCLTech

IT Consulting

Senior Technical Architect (IN) at HCLTech | SkillMX | SkillMX