Senior Data Engineer (Noida, UP, India)
Iris Software
Iris Software
Join Iris Software, recognized as one of India's Top 25 Best Workplaces in the IT industry. We are a rapidly expanding IT services company committed to being a trusted technology partner and a preferred destination for top industry professionals. With a global presence across India, the U.S.A., and Canada, we empower enterprise clients through technology-driven transformations in financial services, healthcare, transportation & logistics, and professional services. Our expertise spans critical application development, data analytics, cloud solutions, DevOps, and business automation using cutting-edge technologies.
At Iris, your career is a journey of growth. Our philosophy, "Build Your Future. Own Your Journey.," emphasizes employee ownership and provides the right opportunities to shape individual career paths. We foster an environment where talent is valued, voices are heard, and work delivers tangible impact. Through challenging projects, tailored development, continuous learning, and mentorship, we support your professional and personal evolution.
Spearhead the design of scalable data engineering solutions leveraging PySpark and modern distributed processing frameworks. Define robust data ingestion, transformation, and processing architectures aligned with business needs.
Develop and optimize Snowflake or Delta Lake on Databricks for enterprise-scale data platforms. Lead the implementation of high-performance batch and streaming data pipelines.
Design event-driven data architectures using Apache Kafka or Amazon Kinesis, establishing clear data streaming standards and integration frameworks. Architect workflow orchestration using Apache Airflow or Databricks Workflows.
Implement comprehensive monitoring, scheduling, and operational controls for reliable pipeline execution. Drive data quality, validation, and governance practices.
Architect data engineering solutions based on Lakehouse principles, emphasizing data observability and platform engineering for enhanced scalability and reliability. Foster the development of business-focused data products by improving data quality, discoverability, and usability.
Champion the responsible use of AI-assisted engineering to boost development productivity, testing, and documentation. Review pipeline designs and implementations to ensure adherence to performance and scalability standards.
Conduct detailed root cause analysis for complex data processing, workflow, and streaming issues. Mentor team members on key technologies and data engineering best practices, collaborating with stakeholders for seamless end-to-end data platform delivery.
Mandatory skills include proficiency in Databricks Workflows, PySpark, Apache Kafka, and Delta Lake on Databricks. Experience with Snowflake, Amazon Kinesis, and Apache Airflow is also essential.
Demonstrate strong ownership and a commitment to data engineering excellence. Collaborate effectively with diverse teams and business stakeholders to ensure successful project delivery.
Promote quality-focused engineering through proactive validation, optimization, and a drive for continuous improvement. Apply strong analytical thinking to effectively tackle complex data engineering and platform challenges.
Iris Software
Information Technology & Services