Senior Data Engineer

Persistent

8–12 yrs Hyderabad Full Time Hybrid (office + remote)
Persistent logo
Posted : today
Actively hiring

Job description

Join our dynamic team as a Senior Data Engineer, specializing in Oracle-based analytical platforms and Apache Spark. This role is pivotal for large-scale data modernization, migration, and analytics initiatives. You will be instrumental in building high-performance data pipelines, optimizing extensive data processing, and integrating enterprise data platforms with advanced cloud analytics solutions.

We are an AI-led digital engineering partner committed to client success and innovation. Our global team thrives on a disruptor's mindset, driving growth and transforming industries for market leaders worldwide. Recognized for innovation and leadership, we offer a collaborative environment where talent development is paramount.

Responsibilities

Design and develop robust, large-scale data processing solutions leveraging Apache Spark (PySpark/Scala) and Python. Build and maintain scalable ETL/ELT pipelines crucial for analytical and reporting workloads. Optimize query performance, Spark jobs, and end-to-end data processing for high-volume batch and near real-time operations. Implement sophisticated data integration, replication, and ingestion strategies using technologies like Oracle GoldenGate, Kafka, and CDC frameworks. Collaborate cross-functionally with application, data, and analytics teams to architect scalable data models and processing frameworks. Lead performance tuning efforts across Oracle, Spark, and distributed data processing environments.

Qualifications

A minimum of 8 years of dedicated experience in Data Engineering or a closely related field. Demonstrated hands-on expertise with Spark (PySpark and/or Scala) and Python is essential. Possess a deep understanding of Oracle databases, particularly in OLAP, data warehousing, and analytical contexts. Proven ability to tune and optimize Spark applications, complex SQL queries, large-scale ETL/ELT processes, and high-volume batch workloads. Strong foundational skills in Data Modeling are required. Experience with enterprise-scale data integration technologies such as Goldengate or CDC is necessary. Familiarity with Exadata Server and large-scale Oracle analytics platforms is a plus. Experience with GCP services like Google BigQuery, Cloud Dataproc, Cloud Storage, Pub/Sub, and Dataflow is advantageous. Ability to integrate Oracle with cloud data platforms, data lakes, Kafka, Hadoop, or Google BigQuery. Exposure to Azure IaaS, OCI, or multi-cloud data architectures is beneficial. Experience supporting extensive Data Migration and modernization programs is expected.

Essential Skills

Apache SparkPySparkScalaPythonETLELTSQLOracle GoldenGateKafkaCDCData ModelingData MigrationData ModernizationGoogle BigQueryCloud DataprocCloud StoragePub/SubDataflowHadoop

Good to Have

Exadata ServerAzure IaaSOCI

Highlights

  • Actively hiring

More Details

RoleSenior Data Engineer
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Persistent Systems logo

Persistent Systems

IT Consulting

Senior Data Engineer at Persistent | SkillMX | SkillMX