Hallucination is not a hygiene problem

A brief filed recently at the Belgian Court of Cassation cited eight judgments of that Court. In the advocate-general's account, two did not contain the rule they were cited for and the other six were simply wrong. When the advocate-general raised it by e-mail, the lawyer who had signed the brief blamed a former member of staff and offered to withdraw any reference that could not be verified in accessible Belgian sources.

Nobody in that story needed a better chatbot. They needed somebody to open eight links.

The case appears in the address Michel Nolet de Brauwere gave on 1 September 2026 to open the Belgian judicial year, 35 pages on justice and artificial intelligence, thirteen named risks, and, unless he is mistaken, the first mercuriale in the country to take the subject on. It is a serious piece of work, more honest than most industry writing on the same question: he states plainly that on several of the fields he is crossing he holds "geen enkele legitimiteit", no legitimacy at all, to speak with authority.

So it is worth being equally plain about the one place the analysis goes wrong, because it is the place that decides what the judiciary will eventually buy.

Safe is the wrong specification

Asked what to do, the address answers first with security. Magistrates need an IT environment, including case management, storage, communication and AI, that offers every guarantee against leaks, intrusion, outage and external manipulation, and in which personal data can be entered lawfully. Every word of that is worth having. None of it would have caught the eight citations.

The reason lies in how the address explains hallucinations. It opens that section with the Paris bar's account: tools, often free, drawing their input from free sources such as blogs and opinion pieces, rest on unreliable data that can lead to hallucinations. The next paragraph describes omnivorous systems contaminated by the errors circulating on social media.

That is a theory of pollution, and pollution is the wrong model. A language model does not look anything up. It produces the most probable continuation of the text in front of it. It invents a judgment because it was built to write, not to retrieve. Training data explains a model's biases; it does not explain a fabricated ECLI number. Cleaner input does not yield correct citations, because citations are not being copied from anywhere in the first place.

The address has the better explanation two paragraphs further on, in its own words: these tools tend to give the answer they think will please the user rather than admit they have nothing, "zelfs als dat betekent dat ze bronnen moeten verzinnen". Even if that means inventing sources. Invention is not a contaminant in the pipeline. It is the pipeline working as designed.

This is not a quarrel about vocabulary. It decides whether the proposed fix can work. Researchers at Stanford tested the three products closest to the secured environment the address hopes for: Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI. Paid tools, retrieving from the publisher's own curated legal database, the walled garden itself. Across 202 preregistered queries on US law, run between March and May 2024, each of the three hallucinated between 17 and 33 percent of the time, and the authors add that they noticed the responses, Lexis+ AI's in particular, "evolve over time" even during the study. Their own summary: "RAG systems are no panacea." The same group had already measured general purpose models at 58 percent (GPT-4) to 88 percent (Llama 2) on specific, verifiable questions about randomly chosen federal cases, and found that the models cannot always predict, or do not always know, when they are producing legal hallucinations.

Those are American products under American law, and Belgian figures at that scale do not exist. But the mechanism lives in the model, not in the jurisdiction, and a walled garden is exactly what was tested.

Secure and correct are different failures. A European, well governed, properly audited tool protects the confidentiality of the file. It does not make the citation real.

The specification that would have worked

Write the requirement the other way round and it becomes buildable. The tool returns a resolvable link to a judgment in the public case law database, or it returns nothing at all. A human opens the link. The procedure penalises the person who did not.

The address already contains the first half of that, in its sharpest sentence: judicial actors, being responsible for the accuracy of their assertions, cannot use a tool that does not cite its sources and does not allow those sources to be verified. It goes further than most vendors would like, insisting that the person who signs stays responsible for every reference, "zelfs wanneer deze zijn aangeleverd door een AI-tool die zogenaamd in een afgeschermde en beveiligde omgeving opereert". Even when the tool supposedly runs in a walled and secured environment. That clause concedes the argument.

The third part is already in use. On 25 March 2026 the Antwerp court of appeal awarded a higher procedural indemnity against a party whose AI assisted pleadings produced a chaotic and shifting case. No detector was needed. Sloppy is a property of the document, and the bench can see it.

The scale is worth knowing. The database Damien Charlotin keeps of decisions dealing with hallucinated material stood at 2,038 cases on 11 September 2026, of which 1,172 came from litigants without a lawyer and 811 from lawyers, eight of them Belgian. Those are decisions, not filings, so the number of briefs behind it is larger and unknowable.

I run a version of this rule over my own writing, and not out of principle. Every quotation in a piece I publish is opened on the page where it lives and copied exactly, or it stays a bracket in the draft that I have to fill myself. A summary is never a source. It is not a clever technique. It is bookkeeping, and it is the only part of this that also works for one person with no budget.

The raw material for the judicial version exists and is free. Juportal, the public Belgian case law database, carries every decision the Court has ordered published, with the summaries and the opinions of the public prosecutor. What is missing, in the address's own words, is a reliable and secure tool that makes proper use of it. Belgian courts and prosecution offices hold no subscriptions to the commercial legal AI products at all, and the AI projects running at the Court of Cassation and the Antwerp court of appeal are waiting on the executive to agree to fund them.

A retrieval layer over Juportal that refuses to answer without a resolvable link is not a research programme. It is a tender, and a small one.

The dependency underneath

The strongest pages of the address have almost nothing to do with AI, and they are the reason the tender matters more than the chatbot.

Footnote 42 lists what a Belgian magistrate runs today, software whose source code is not public: Windows with Word, Excel, Outlook and Teams, plus JustCase, SharePoint and Copilot. The federal justice guidelines the address cites are themselves hosted on a SharePoint. The IT services and software placed at magistrates' disposal, he writes, are supplied by foreign companies, and all the data held in them is entrusted to those companies. He is not speculating about the consequences: judges of the International Criminal Court were, as he spoke, under unilateral US sanctions for doing their job, with their e-mail accounts and payment methods blocked.

Copilot is the concrete case. The ICT team at the federal justice department asked the courts and prosecution offices to use Copilot Chat with Enterprise Data Protection rather than ChatGPT, on the ground that it is safer for data protection, and its own newsletter explained that what you enter or generate is "over het algemeen niet" stored. Generally not. That is the phrase a data protection impact assessment exists to replace. The Netherlands commissioned one: in December 2024 M365 Copilot was judged not yet compliant to use, on four high risks driven mainly by a lack of transparency from Microsoft, and after a revision in September 2025 the light went to amber, with a residual medium risk that it still generates inaccurate or incomplete personal data. In Belgium, the heads of the three highest courts asked the federal government for exactly that assessment on 26 June 2026, and were answered with a future digital agency and a commitment to proceed step by step.

This is where the European argument earns its place, and where it has to be precise. The Cloud and AI Development Act now defines four sovereignty assurance levels, to be used by public sector bodies according to their own risk assessments, and a court is a public sector body. At Level 3 the provider must be owned and controlled from the EU, with conditions such as personnel citizenship, though the Commission can still recognise third-country providers there. Only Level 4 requires full transparency and control over the software supply chain and no interference from a third country. That is the shelf for judicial data, and a court can ask for it. Storage, the retrieval layer and the model itself can all sit with a provider on that shelf, which settles who can switch the judiciary off, who can read its files and whose court order reaches its data. It settles nothing about whether the model tells the truth.

I have argued here before that owning the code while renting the brain is not independence. So the honest shape of the position is this. The retrieval layer over Juportal is the part Belgium can own outright. The model can at least be housed in Europe, under European control, and that is a procurement decision available this year. What nobody in Europe owns yet is the layer underneath, the chips; the address itself counts an interrupted chip supply among the ways access can be cut, with a footnote pointing at Taiwan. A European model on European silicon looks to me like a decade of industrial policy, not a tender. Those are separable purchases with separable risks, and a court can act on the first two without waiting for the third.

What the saved time is for

The address closes by quoting Christophe Soulard, first president of the French Cour de cassation, who asks what the time these tools save should be spent on. More files, better decisions, more listening at the hearing.

My answer is smaller than the question deserves, and it is the whole of the argument. Spend it opening the link.

For the record: I work at a Belgian systems integrator, so public sector tenders of the kind I am describing here are the sort of work my employer competes for.