Join Iris Software, recognized as one of India's Top 25 Best Workplaces in the IT industry. We are a rapidly expanding IT services company focused on becoming our clients' most trusted technology partner and a premier destination for top industry professionals.

Our vision is to empower enterprise clients across financial services, healthcare, transportation, and professional services through technology-enabled transformation. We specialize in complex, mission-critical applications leveraging cutting-edge technologies in Application & Product Engineering, Data & Analytics, Cloud, DevOps, MLOps, Quality Engineering, and Business Automation.

At Iris, we believe in a culture where your potential is valued, your voice matters, and your contributions create real impact. We offer cutting-edge projects, personalized career development, continuous learning, and mentorship to support your personal and professional growth. "Build Your Future. Own Your Journey." is more than a slogan; it's our commitment to providing opportunities for you to shape your own success story.

Design and implement scalable Big Data solutions utilizing Apache Spark, Hadoop, and Azure Databricks. Define robust data processing architectures, transformation strategies, and engineering standards aligned with business objectives.

Lead the development of distributed data processing pipelines using Spark (Scala or PySpark) and other Hadoop ecosystem technologies. Optimize SQL and Hive-based data processing solutions for enhanced performance and scalability. Architect and fine-tune Azure Databricks solutions for large-scale data engineering and analytics.

Design and deploy data Lakehouse solutions with Apache Hudi or Apache Iceberg. Establish comprehensive frameworks for data ingestion, transformation, validation, and reconciliation to ensure data reliability. Drive performance tuning initiatives across Spark jobs, Databricks workloads, Hive queries, and Hadoop environments.

Review data engineering solutions to ensure adherence to architectural, performance, and engineering standards. Conduct detailed root cause analysis to troubleshoot complex data processing, performance, and platform issues. Mentor team members on Spark, Hadoop, Databricks, Hudi/Iceberg, SQL optimization, and best practices in Big Data engineering.

Collaborate effectively with various teams and stakeholders to support end-to-end data platform delivery. Champion continuous improvement initiatives focused on scalability, performance, reliability, and operational efficiency.

This role requires 7-8 years of experience in Big Data engineering. Essential skills include expertise in Data Ingestion Tools (Sqoop), Hadoop (HDFS + YARN), Azure Databricks, and a strong understanding of the Hadoop Ecosystem Fundamentals (HBase + Impala), Hive, and Scala.

Proficiency in Apache Hudi is mandatory. While PySpark is an advantageous additional skill, a strong foundation in the core technologies is paramount. The ideal candidate demonstrates strong ownership, collaborates effectively with diverse teams, and promotes quality-focused engineering through proactive validation and optimization.

Candidates should possess strong analytical thinking to tackle complex data engineering challenges, exhibit adaptability in dynamic technology environments, and communicate clearly regarding delivery status, risks, and opportunities. A high level of attention to detail is expected across all aspects of data architecture, processing design, testing, and implementation. Encouraging continuous improvement and supporting knowledge sharing are key attributes for success in this role.