Join Iris Software, a rapidly expanding IT services company recognized among India's Top 25 Best Workplaces in the IT industry. We are seeking a Data Engineer to contribute to our vision of being a trusted technology partner. Our work involves complex, mission-critical applications using cutting-edge technologies. At Iris, we offer a culture that values your talent, ambition, and provides a launchpad for your career growth.

We foster an environment where your potential is recognized, your voice is heard, and your contributions make a real impact. Benefit from engaging projects, tailored career development, continuous learning opportunities, and mentorship designed to support your professional and personal advancement.

Design and implement scalable data engineering solutions, focusing on PySpark and modern distributed data processing. Define architectures for data ingestion, transformation, and processing to meet business needs. Develop and optimize Snowflake or Delta Lake on Databricks solutions for enterprise-scale platforms. Lead the creation of high-performance batch and streaming data pipelines. Architect event-driven data solutions using Apache Kafka or Amazon Kinesis, establishing robust standards and patterns.

Architect workflow orchestration using Apache Airflow or Databricks Workflows, ensuring reliable pipeline execution through monitoring and scheduling. Drive data quality, validation, and governance practices. Design solutions based on Lakehouse architecture principles, incorporating data observability and platform engineering standards to enhance scalability and operational visibility. Develop business-focused data products by improving data quality, discoverability, and usability.

Promote the responsible use of AI-assisted engineering to boost development productivity, testing, and quality. Review data pipeline designs and implementations to ensure adherence to standards. Troubleshoot complex data processing and streaming issues through root cause analysis. Mentor team members on key technologies and data engineering best practices. Collaborate with cross-functional teams for end-to-end data platform delivery.

This role requires strong proficiency in mandatory skills including Apache Airflow, PySpark, Apache Kafka, and Delta Lake on Databricks. Experience with Snowflake and Amazon Kinesis is also essential for designing event-driven architectures and optimizing data solutions.

Key competencies include a strong foundation in Data Science and Machine Learning with Python and Apache Spark, alongside expertise in Database Programming using SQL. You should be skilled in Data Quality & Validation and possess knowledge of CI/CD development tools. Familiarity with GCP services is a plus.

Behavioral attributes such as strong ownership, effective collaboration, quality-focused engineering, and analytical thinking are crucial. Adaptability to evolving technologies and excellent communication skills are expected. A high degree of attention to detail and a commitment to continuous improvement are vital for success in this role.