Unlocking the Power of Palantir Foundry

Insights and Differentiators

Disclosures: The opinions and ideas expressed in this article are solely my own. This article was written in my personal capacity. ~5% of this article was AI-generated. Unsourced claims are anecdotal based on my 20+ years as a software engineer and technology executive. I am one of two certified global Foundry ambassadors.

Palantir Foundry is still in its early stages of adoption as it did not emerge as a mature platform until early 2021. As such, many potential customers have questions about where it sits in their technology stack, the main value proposition, and how it can integrate with existing systems. This article will provide a blueprint for answering those questions based on real-world experience integrating Foundry in some of the world's largest organizations.

Context

The current landscape of enterprise software is complex and riddled with challenges. Solutions are slow, have enormous failure rates, and are often built on technology that is deprecated the day it ships. The market is also saturated. Over 70% of a typical organization's software is now SaaS. This proliferation of SaaS solutions often results in expensive and cumbersome data silos. Without a unified data landscape, it will be impossible to make data-driven decisions in the execution space, which is where 90% of the value levers are found.

Home-grown solutions have the unintended side effect of vendor-locking organizations to tribal knowledge that has long since walked out the door. They also have failure rates between 75–90%. The reason for this should be obvious but is almost always never addressed in the planning stages.

1. Going from POC to production is hard — User experience is the tip of the iceberg regarding software. But it’s often the only consideration when measuring complexity. Every production software application will need operations (GitOps, DevSecOps, etc.), observability, compliance, and a lot more if it’s going to be successful. If you’d like to learn more about the complexity and how companies like Google manage it, read “Software Engineering at Google” by Hyrum Wright, Titus Winters, and Tom Manshreck. Or read this summary.
2. Lack of qualified talent — There aren’t enough high-quality engineers worldwide to build quality software at your organization. The world's top talent is consolidated in FAANG companies, and that isn’t going to change anytime soon.
3. You’re in a different business — Chances are your enterprise is not in the software business. This means investments in technology, while they often sound good on paper, yield negative returns. Solutions take too long to develop (windows close rapidly as the market and technology landscape evolve) and suffer from high failure rates. See points one and two above.

Companies need a full-stack solution to build a data grid that operationalizes data across bespoke applications that deliver value without massive investments in engineering resources.

Palantir Differentiators

1. Time to Value — Palantir delivers complete solutions with a measurable ROI in weeks or months. If you are facing a crisis or can't wait a year or more to deliver solutions using traditional platforms and tools, Palantir Foundry is the right solution.
2. Full Stack — Foundry manages the dozens of tools and services required to operationalize your data. This drives complexity and cost down, letting you focus on solutions to business problems, not technical issues. Foundry is the right solution if your business is resource-constrained or lacks software engineers.
3. Extend Legacy Systems — Almost every company has legacy systems, often in the form of mainframes or monoliths, that the business depends on but can’t or won’t continue to fund development. Palantir Foundry, coupled with Foundry Container Engine, provides a novel approach to extending the usefulness of the systems and amplifying the value of their underlying data. If your most important data and business logic are locked in legacy systems, Foundry is the solution you’ve been looking for.
4. Empower Subject Matter Experts — The people who understand your business should be empowered to build bespoke solutions that solve their most pressing issues. Foundry’s low/no code AI-enabled tools let you do just that. This helps solve the engineering bottleneck problem that often blocks companies from extracting value from their data. Foundry is an ideal solution to upskill your workforce and accelerate ML/AI adoption and data-driven decision-making.

Architecture

Palantir Foundry is a data processing layer that sits on top of your existing infrastructure to simplify data integration, modeling, and application-building. It is not a replacement for virtual computing, durable object storage, or advanced cloud services. It’s built on an open and extensible architecture that builds on the technology investments you’ve already made. This type of abstraction is often referred to as sky compute and is illustrated below.

Palantir Foundry enables you to deploy multi-cloud applications leveraging Apollo (CI/CD solution for container-based workloads) and its advanced data synchronization tools. It also enables data harmonization.

Foundry is flexible and can be used exclusively as a data mesh powering your cloud-native applications (see this architecture from Merck KGaA) or a full-stack solution for creating data products and their applications.

Core Solution Components

Palantir Foundry ships with dozens of modules and hundreds of microservices to build and deploy solutions for your business. However, there are four core modules that every organization will use most often:

1. Data Sources and Syncs — Sources and Syncs allow you to connect and regularly import and export data from hundreds of supported types. This includes ERPs, CRMs, cloud storage, JDBC, APIs, and more. It shows that Palantir spent over ten years developing its data integration tools. They even support software-defined data integrations (SDDI), which automatically transforms source systems (like SAP) into canonical forms for supply chain management.
2. Pipeline Builder — Similar to Alteryx, Pipeline Builder is a visual tool for performing data transformations. You can clean and join all your data sources integrated into Foundry and export to data sets used in your ontology. Pipeline Builder also contains AIP Assists for creating transforms from natural language and explaining transforms for documentation.
3. Ontology Manager — The Ontology Manager allows you to model your harmonized data into an ontology of connected objects. It includes describing your data model for AI applications, connecting objects using graphs, performing actions against the data, and leveraging a micro-frontend architecture in your applications.
4. Workshop — Workshop is a low/node code application building tool that allows you to rapidly build and deploy applications leveraging your company's models and data products. It uses React under the hood for a modern look and feel with support for mobile devices. Citizen engineers can perform development without managing complex deployment and infrastructure code.

Commonly used Foundry modules also include Contour, Quiver, Object Explorer, Code Authoring, Code Workbooks, and ML Objectives. For a complete overview of the platform, please refer to the documentation. For additional resources, visit learn.palantir.com and Ontologize.

The Artificial Intelligence Platform

Palantir recently announced the launch of its Artificial Intelligence Platform (AIP). AIP delivers on the promise of making AI secure, accessible, and easy to integrate into your business. The core components of AIP are as follows:

1. AIP Assists — These virtual AI assistants do everything from providing in-platform help to generating data pipelines and applications from natural language descriptions.
2. AIP Terminal — Terminal is enterprise chat that secures your data and ensures nothing leaks outside your organization. Terminal makes it easy to ground models in your organization's data using a point-and-click interface and incorporates an advanced reasoning engine with observability by default. Observing the decision-making process allows the operator to determine if the generated content is correct.
3. AIP Logic — AIP logic allows you to construct agents that perform anything from alert remediation and scheduling to other common enterprise tasks. Agents are emerging as language programming interfaces (LPI) that will one day replace API orchestration as the default for composing workflows.
4. AIP Automate — Automate orchestrates agents created in AIP logic and allows humans to gate the decision-making process with the ability to approve or reject AI-generated proposals and scenarios.
5. Application Integrations and Developer Tools — AIP includes low-level building tools such as privately hosted model endpoints, tool interfaces, and more, enabling developers to build the hard parts of AI applications to empower citizen engineers downstream. The newly released OSDK also enabled developers to include AIP-powered solutions into existing applications that are not hosted in Foundry.

Companies that fail to leverage AI over the next three to five years will be severely disadvantaged. It won’t be a repeat of the enterprise boondoggle where marginal software is yielding marginal advantage. It will be the difference that exists today between FAANG companies and everyone else. Palantir’s differentiator in leveraging AI is the ease and speed at which you can build applications. The need for software engineers and AI experts is minimal or zero.

Industry Example: Supply Chain Execution

Supply chain execution (SCE) focuses on optimizing asset utilization, managing costs, and ensuring timely and accurate deliveries. It heavily depends on tangible resources like forklifts and freight carriers and intangible assets like up-to-minute data and timely communications. Day-to-day challenges arise from disruptions in supply-demand balance, and to address them, managers need visibility into the entire chain to spot and address issues such as bottlenecks. Companies achieve this visibility using specific applications that guide and monitor supply chain tasks, like automatic inventory rebalancing, truck roll optimization, and cost-to-serve modeling.

Palantir's unique ability to connect and operationalize diverse data sources stands out in this space. While obvious in the context of SCE, these benefits are not limited to SCE. This results from Foundry's ability to drive the marginal integration costs of data integration, application development, and artificial intelligence to effectively zero.

Some of the core SEC modules available in Foundry include:

1. Transportation Optimization
2. Order Management
3. Scheduling
4. Warehouse Management

Many of the manual tasks performed in Foundry’s SEC modules can now be automated with AIP. Many UI components can altogether be eliminated and replaced with AIP Logic and AIP Automate, leveraging the ontology-backed objects and functions under the hood. Canonical Ontologies, like the one used in Foundry’s SEC solutions, are becoming infinitely more valuable as they enable these types of AI-backed workflows.

Alternatives

Before diving into the alternatives, it’s important to understand the role of data platforms. Databrick and Snowflake are two popular platforms for data platforms for building ML/AI solution components. They are not (as of today) capable of building the same type of applications and workloads enabled by Foundry or its alternatives. Foundry often consumes the work produced in these platforms (data products and models). For more on how Foundry extends your data platforms, read this article.

1. Microsoft Fabric — Fabric is still in the early stages of its release and lacks many required features, such as version control, before it’s ready for primetime. That said, Fabric will be a powerful platform for the modern era of enterprise software and a great alternative to Foundry.
2. Home-Grown Solution — Don’t do it.

Conclusion

Palantir Foundry has rapidly positioned itself as an instrumental player in data processing, analytics, and AI. Its capacity to seamlessly integrate with existing tech stacks and add value in short time frames makes it an enticing solution for organizations eager to capitalize on their data without the burdensome intricacies often encountered in traditional platforms. Among its key differentiators, Palantir ensures rapid ROI, mitigates technical complexities, rejuvenates legacy systems, and critically empowers subject matter experts with tools that bypass the usual engineering bottlenecks.

If you want to learn more, message me on LinkedIn. Thank you!