NTT DATA is seeking a skilled Data Engineer Advisor to join our dynamic team in Bangalore. This role is instrumental in designing, building, and maintaining robust data pipelines and products within the Databricks Lakehouse platform. You will focus on data ingestion, transformation, governance, and optimization, ensuring adherence to engineering best practices and platform standards. Collaborate closely with architects, analysts, and business stakeholders to deliver high-quality data solutions that power analytics, reporting, and AI initiatives.

Key responsibilities include developing and maintaining scalable data pipelines within Databricks, understanding and leveraging its latest data processing capabilities, and creating ingestion frameworks for both batch and streaming workloads. You will build reliable, reusable data products, implement medallion architecture patterns, and develop Databricks notebooks, workflows, and jobs. A significant part of the role involves optimizing performance and cost-efficiency, supporting platform monitoring, and troubleshooting. You will also contribute to deployment automation using Terraform and CI/CD pipelines, manage code repositories, and enhance platform automation. Ensuring data quality, supporting data lineage and governance, and adhering to security standards are crucial, alongside producing comprehensive technical documentation.

This position requires a minimum of 3 years of experience in Data Engineering roles, with strong hands-on experience and a Databricks engineering certification. Proficiency in Databricks, PySpark, SQL, Delta Lake, AWS, Terraform, Git, and CI/CD is essential. Experience with modern ETL/ELT development is also a requirement. Desirable skills include Unity Catalog, Kafka, streaming technologies, data quality frameworks, and machine learning pipelines. Experience with AWS and Terraform or Infrastructure-as-Code is expected.