Is an AI answer a source?

It is the evening before a charity’s annual report is due. Someone pastes a year of notes into an AI assistant. They ask for a paragraph on what the programme achieved for participants. A minute later there is a tidy paragraph. It even includes a percentage of participants who say they feel more confident. Nobody can say where that figure came from.

The figure might be right. But a language model writes what sounds plausible, and that is not always what is true. It may cite nothing, or cite something that does not exist. And because the prose sounds so sure of itself, hardly anyone stops to check.

We keep asking ourselves whether an answer like that can count as a source. For us, a source is something you can open, such as a report, a survey or notes from a conversation. A source works like a till receipt. With a receipt you can go back to the shop. An AI answer is a total without a receipt, a summary of material you never get to see. Even so, we understand why people use it. The report has to go in, and there is never enough time.

We sometimes wonder what this means for learning. A funder’s board wants to know what grantees do. But a report is not only for the funder. Participants, residents, partners and the local council all have a right to know, or a stake in it. Reports drafted with AI may read more smoothly. Whether they tell any of these readers more about what actually happened, we honestly do not know.

We once ran a small test. We asked several AI assistants, a few times each, how many homes there are in the Netherlands. Each time, every one of them came back with a slightly different number. Without a source, there is no telling which one to use.

We have not got this figured out either. When we use AI, we ask the model to back every claim with a word-for-word quote from the source. Then we go and find that quote ourselves. Anything we cannot find, we drop. It yields less than we would like, and it takes time. Nor does it solve everything. What it does show us is where the evidence is thin.

What would we like to see? Every figure in a report saying where it comes from, whoever or whatever wrote the text. And funders, grantees, participants and partners talking about it together, because they all want to learn what works. We are not there ourselves yet. But of one thing we are sure. A report that is honest about its doubts is worth more than a smooth text full of certainty.