We are seeking a Custom Software Engineer with 2-5 years of experience to join our team. This role involves leading the design, development, and configuration of applications, serving as a primary point of contact.

An exciting opportunity exists for a Lead Data Engineer focused on designing, building, and enhancing applications within Palantir Foundry to meet evolving business requirements. This position requires a minimum of 6 years of relevant work experience.

As a Custom Software Engineer, you will be responsible for the complete lifecycle of data pipelines and applications on Palantir Foundry, including design, development, testing, and ongoing support.

Key responsibilities include configuring and customizing Workshop to create efficient workflows and robust ontologies. You will also develop dynamic and interactive forms in Workshop using TypeScript for user-driven data entry and validation. Collaboration with data engineers and stakeholders is crucial for successful application deployment and operation, ensuring data-related technical issues are resolved and requirements are met through effective data pipeline design.

A strong foundation in PySpark, Python, and SQL is essential, with a proven ability to write and optimize SQL and Spark jobs.

Hands-on experience with Palantir Foundry services such as Data Connection, Code Repository, Contour, Data Lineage, and Health Checks is required. Familiarity with Workshop, Ontology, and Slate is highly beneficial. Expertise in data engineering and building ELT/ETL data pipelines, including data refinement and quality checks on Palantir Foundry, is expected. Experience in ingesting data from various external systems using data connections and sync mechanisms is also important.

Proficiency in TypeScript for customizing Workshop forms, including validation and data binding with Ontology, is a must. A good understanding of Spark Architecture with experience in performance tuning and code optimization is valuable. The role involves managing both structured and unstructured data across formats like CSV, JSON, Parquet, and ORC, along with developing scalable architectures for large datasets. Knowledge of data loading mechanisms and CDC strategies is beneficial.

We value experience with test-driven development and CI/CD workflows, alongside proficiency in version control software like Git and major hosting services (e.g., Azure DevOps, GitHub, Bitbucket, GitLab). Adhering to code best practices to enhance readability, maintainability, and quality is expected.