Join Iris Software, recognized as one of India's Top 25 Best Workplaces in the IT industry. We are a rapidly expanding IT services company committed to being our clients' most trusted technology partner and a launchpad for top industry professionals to realize their full potential. With over 4,300 associates across India, the U.S.A., and Canada, we empower enterprise clients through technology-enabled transformations in financial services, healthcare, transportation & logistics, and professional services. Our expertise spans mission-critical applications utilizing cutting-edge technologies like Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation.

At Iris, we believe in "Build Your Future. Own Your Journey." Our culture values your potential, amplifies your voice, and ensures your work creates real impact. We provide cutting-edge projects, personalized career development, continuous learning, and mentorship to foster your personal and professional growth. Explore more about life at Iris through our "inside look" video.

Key responsibilities include designing scalable data engineering solutions using PySpark and modern distributed data processing frameworks. You will define data ingestion, transformation, and processing architectures aligned with business goals, and design/optimize Snowflake or Delta Lake on Databricks solutions for enterprise-scale data platforms. This role involves leading the implementation of high-performance batch and streaming data pipelines, and designing event-driven data architectures using Apache Kafka or Amazon Kinesis.

You will define data streaming standards, integration frameworks, and scalable processing patterns. Architecting workflow orchestration solutions using Apache Airflow or Databricks Workflows, and establishing monitoring, scheduling, and operational controls for reliable pipeline execution are crucial. Driving data quality, validation, reconciliation, and governance practices across data engineering solutions is also a key part of this role. You will design solutions following modern Lakehouse architecture, data observability, and platform engineering standards to enhance scalability, reliability, and operational visibility.

Further responsibilities include driving the development of business-focused data products by improving data quality, discoverability, usability, documentation, and trusted data consumption. You will promote the responsible use of AI-assisted engineering capabilities to boost development productivity, testing, documentation, and quality. Reviewing data pipeline designs and implementations to ensure adherence to standards, and troubleshooting complex data processing and streaming platform issues through root cause analysis are essential. Mentoring team members on core technologies and best practices, and collaborating with various teams for end-to-end data platform delivery are also expected.

Mandatory skills for this role include PySpark, Apache Kafka, Databricks Workflows, and Delta Lake on Databricks. Essential competencies encompass Data Science and Machine Learning (Apache Spark, Python, Databricks), Data Engineering (Data Quality & Validation, Apache Kafka), Big Data (Pyspark), and Database Programming (SQL). Strong behavioral competencies are required, including demonstrating strong ownership, effective collaboration with teams and stakeholders, and promoting quality-focused engineering through proactive validation and optimization.

Additionally, strong analytical thinking is needed to evaluate complex data engineering and platform challenges. Adaptability in managing evolving technologies, data ecosystems, and business requirements is crucial. Experience with communication and collaboration is also a key requirement. While not explicitly stated as mandatory, understanding and working with Snowflake, Amazon Kinesis, and Apache Airflow would be highly beneficial for designing event-driven architectures and workflow orchestration.