Data Engineer
Persistent
Persistent
Join Persistent as a Data Engineer and leverage your expertise in Oracle-based analytical platforms and Apache Spark. This role is pivotal in driving large-scale data modernization, migration, and analytics projects. You'll be instrumental in crafting high-performance data pipelines and optimizing complex data processing environments.
We are a leading AI-led Digital Engineering partner, empowering clients with innovative solutions. Our work spans across Fortune 50 companies and top financial institutions, offering a dynamic environment for growth and learning.
This position offers a competitive compensation package, professional development opportunities, and a culture that values talent and well-being. We embrace diversity and offer flexible work arrangements to support a balanced lifestyle.
Design, develop, and optimize robust data processing solutions utilizing Apache Spark (PySpark/Scala) and Python.
Construct and maintain scalable ETL/ELT pipelines to support critical analytical and reporting needs.
Enhance query performance, Spark jobs, and overall data processing efficiency for high-volume batch and near real-time data flows.
Implement data integration, replication, and ingestion strategies using tools like Oracle GoldenGate and Kafka.
Collaborate with cross-functional teams to develop scalable data models and processing frameworks.
Lead performance tuning efforts across Oracle, Spark, and distributed data processing systems.
A minimum of 5 years of dedicated experience in Data Engineering or a closely related field is required.
Proficiency in Spark (PySpark and/or Scala) and Python is essential for this role.
Demonstrate a deep understanding of Oracle RDBMS, including its application in OLAP, Warehousing, and analytical workloads.
Possess proven experience in optimizing Spark applications, complex SQL queries, large-scale ETL/ELT processes, and high-volume batch processing.
Strong data modeling skills are a must, along with experience in enterprise-scale data integration technologies like Goldengate or Kafka.
Persistent Systems
IT Consulting