Join Iris Software, recognized as one of India's Top 25 Best Workplaces in the IT industry, and contribute to building your most impactful career. We are a rapidly expanding IT services company committed to being a trusted technology partner and a premier destination for top industry talent.

Our global presence spans India, the U.S.A., and Canada, where we empower enterprise clients across financial services, healthcare, transportation & logistics, and professional services. We specialize in complex, mission-critical applications, leveraging cutting-edge technologies in Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation.

Key responsibilities include designing scalable data engineering solutions with PySpark and distributed processing frameworks, and defining robust data ingestion, transformation, and processing architectures. You will architect and optimize Snowflake or Delta Lake on Databricks, and lead the implementation of high-performance batch and streaming data pipelines.

Further duties involve designing event-driven architectures using Apache Kafka or Amazon Kinesis, and establishing standards for data streaming and integration. You will architect workflow orchestration solutions with Apache Airflow or Databricks Workflows, ensuring reliable pipeline execution through monitoring and scheduling controls. Driving data quality, validation, and governance practices, while adhering to modern Lakehouse architecture principles and data observability standards, is essential.

You will also develop business-focused data products by enhancing data quality, discoverability, and usability. Promoting responsible AI-assisted engineering for improved development productivity and quality, and reviewing pipeline designs for adherence to standards, are critical. Troubleshooting complex platform issues, mentoring team members, and collaborating with various stakeholders for end-to-end platform delivery are also key components of this role.

We are seeking a Senior Data Engineer with 7-8 years of experience. Essential mandatory skills include Amazon Kinesis, Apache Spark, Data Quality & Validation, PySpark, SQL, Apache Airflow, and Delta Lake on Databricks. Proficiency in Python and basic understanding of DevOps & CI/CD are also required.

Successful candidates will demonstrate strong ownership, effective collaboration with diverse teams and stakeholders, and a commitment to quality-focused engineering. The role demands robust analytical thinking for complex data engineering challenges and adaptability in a dynamic technological landscape.

Effective communication regarding delivery status, risks, and opportunities, coupled with high attention to detail in architecture, design, testing, and implementation, is crucial. Encouraging continuous improvement, supporting knowledge sharing and mentoring, and balancing technical priorities with business needs are expected. Innovation through modern practices and AI-assisted development is highly valued.