Join Iris Software, a rapidly growing IT services company recognized among India's Top 25 Best Workplaces in the IT industry. We are seeking a Senior Big Data Engineer with 6-7 years of experience to contribute to our award-winning culture that values talent and ambition.

At Iris, we aim to be our clients' most trusted technology partner and a prime destination for top industry professionals. With a global team across India, the U.S.A., and Canada, we empower enterprise clients through technology-driven transformations in financial services, healthcare, transportation & logistics, and professional services.

Our work involves complex, mission-critical applications utilizing cutting-edge technologies such as Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation. We believe in empowering our employees with ownership of their career paths, fostering a supportive environment for personal and professional growth through challenging projects, development opportunities, and mentorship.

Design and implement scalable Big Data solutions leveraging Apache Spark, Hadoop, and Azure Databricks. Define robust data processing architectures, transformation strategies, and engineering standards aligned with business objectives.

Lead the development of distributed data processing pipelines using Spark (Scala or PySpark) and the Hadoop ecosystem. Architect and optimize SQL and Hive-based data processing for enhanced performance and scalability.

Architect and optimize Azure Databricks solutions for large-scale data engineering and analytics. Design and implement data Lakehouse solutions using Apache Hudi or Apache Iceberg, establishing frameworks for data ingestion, transformation, validation, and reconciliation to ensure data reliability.

Drive performance tuning initiatives across Spark jobs, Databricks workloads, Hive queries, and Hadoop processing environments. Review data engineering solutions to ensure adherence to architecture, performance, and engineering standards.

Troubleshoot complex data processing, performance, and platform issues with detailed root cause analysis. Mentor team members on Spark, Hadoop, Databricks, Hudi/Iceberg, SQL optimization, and Big Data engineering best practices.

Collaborate effectively with various teams and stakeholders to support end-to-end data platform delivery. Drive continuous improvement initiatives focused on scalability, performance, reliability, and operational efficiency.

Candidates should possess 6-7 years of experience in Big Data engineering.

Mandatory skills include proficiency in Data Ingestion Tools (Sqoop), Hadoop (HDFS + YARN), Azure Databricks, Hadoop Ecosystem Fundamentals (HBase + Impala), Hive, Scala, and Apache Hudi.

Additional skills in PySpark are a plus.

Behavioral competencies such as strong ownership, effective collaboration, quality-focused engineering, analytical thinking, adaptability, effective communication, and high attention to detail are essential. Encouraging continuous improvement and supporting knowledge sharing are also key.