A recent SupplyChainBrain article raised important questions about the trustworthiness of artificial intelligence models built to manage supply chains, from the standpoint of cybersecurity. But there’s another way of thinking about the topic: how the data that fuels those models impacts accuracy and utility. In other words, can we trust them to provide useful intelligence?

You’ve no doubt heard the “garbage in, garbage out” axiom. In the kind of analysis most businesses are comfortable doing today, we think of data in terms of quality: If you analyze data that hasn’t been standardized, deduplicated, and otherwise quality-assured, you’ll end up with worthless “insights” from whatever model that data feeds.

That’s true in AI models as well, but there's also more to the story. If you’re building one to provide insight into your supply chain, you have to think about more than just quality. For example, what data sources will you use to fuel the model?

The starting point for most organizations is internal data. But that’s far from a slam dunk: First of all, not all organizations have the critical mass of data necessary to train a large language model (LLM) to the point that it can provide useful insights. And even those that do might want to inform their model with data that their organization simply doesn’t track — for example, with weather insights from NOAA, which are essential to understanding supply chain behavior.

An important point to keep in mind is that many organizations (especially those founded more than a decade ago) store data in formats that aren’t conducive to LLM training. When that’s the case, the organization must first standardize and unify its various data sources to prepare them for fueling an AI model.

A helpful metaphor here is thinking about a car. If data is the new oil (as The Economist claimed back in 2017), and the AI model to provide supply chain insight is the vehicle you’re building, you must first refine your data into something like gasoline so that it’s actually usable. You also have to think about the other fluids (aka other data sources) that are essential to keeping the car in good working order: motor oil, windshield wiper fluid, coolant and the like.

This brings us to the second consideration worth contemplating when weighing a supply chain AI’s trustworthiness: Who is building and querying your AI model?

Continuing the car metaphor, we can think of the move from traditional supply chain tracking to that powered by AI as akin to the transition from horses to cars. The latter let ordinary people do more, faster.

But while the mode of transportation or supply chain analysis is accessible and usable by ordinary people, it still requires expert building and maintenance. Most industrial leaders contemplating AI-powered supply chain analysis recognize that they’ll need workers with a specialized skill set to build the AI model in the first place. In many places, that’s underway, as IT teams learn how to create the kind of neural networks that power LLMs.

It’s also important to recognize that you’ll likely need something akin to prompt engineers to query the model once it’s built. These professionals will be something like AI super users; they’ll become skilled at asking the right questions and figuring out how to elicit the best responses from the language model. The process will become circular: As an organization improves its prompts, it will learn where the holes are in the model, and use that knowledge to further refine it.

Again, drawing out our automotive metaphor: Where once we relied on farriers (the craftsmen who shoed horses’ hooves, and in our modern-day equivalent those who know how to query SQL databases), in a world of AI-powered models, we’ll lean on mechanics, or those who can train and refine a custom model.

While cybersecurity will always be a key component of trust in digital tools, the question of trustworthiness in AI-driven supply chain models is much larger than that of cybersecurity. The ways we build, maintain and interact with AI models have a huge impact on their ability to provide valuable insights, which means the importance of data governance and transparent operations is greater than ever.

Jason Hehman is a client partner and Industry 4.0 vertical lead at TXI.