Join our team as a skilled Databricks Developer, focusing on the creation and upkeep of robust data pipelines and analytical solutions. This role is pivotal for driving enterprise data integration and analytics initiatives. You will leverage your deep expertise in Apache Spark, Python, SQL, and cloud data engineering technologies to build scalable systems.

We are seeking professionals adept at transforming raw data into actionable insights. Your work will directly contribute to enhancing our data capabilities and supporting critical business functions through innovative data solutions.

Key responsibilities include designing, developing, and maintaining ETL/ELT pipelines within the Databricks environment. You will build scalable data processing frameworks using Apache Spark (PySpark) and construct both batch and streaming data pipelines. Integrating data from diverse sources, such as databases, APIs, and cloud storage, will be a core function. Implementing data transformation, cleansing, and validation processes is essential, as is developing and optimizing Delta Lake tables.

Furthermore, you will create reusable Databricks notebooks, workflows, and jobs. Monitoring and troubleshooting pipeline performance and failures are crucial. Collaboration with business analysts, architects, and stakeholders to define data requirements is expected. Adhering to coding standards, version control, and CI/CD best practices, along with active participation in code reviews and technical discussions, will ensure project success.

A strong foundation in Databricks, Apache Spark/PySpark, Python, and SQL is mandatory for this role. Proven experience in ETL/ELT development and working with Delta Lake is required. A solid understanding of data warehousing concepts and familiarity with Git for version control are essential.

Preferred qualifications include experience with major cloud platforms like Microsoft Azure (Azure Databricks, ADLS Gen2, Azure Data Factory), AWS (S3, Glue, EMR), or Google Cloud (BigQuery, Cloud Storage). Experience with streaming technologies like Spark Structured Streaming and orchestration tools such as Airflow is highly valued. A grasp of data governance and security concepts would be beneficial.