We Told 11 AI Models What Happened in 2026. Most of Them Thought We Were Making It Up.
A model with a knowledge cutoff in early 2026 has missed a lot. The Knicks won the NBA title. LeBron signed with the 76ers for $8 million. Allbirds sold its shoe business and became an AI data centre company called Smartbird. We texted all of it to 11 models, phrased the way a friend would, and recorded what came back.
Every model has a knowledge cutoff, the date its training data ends. Anything after it, the model has never seen, whatever date you put in the prompt.
Most of them hedged. One called nearly all of it fake. One claimed to have run searches it had no way to run. Given a search tool, almost all of them came round. Asked to fact-check a newsletter paragraph, several marked true sentences false and made up corrections, and three sentences in the system prompt fixed most of that.
Tell them in chat, and they fact-check you
We sent 11 real stories and 3 made-up ones as single messages, such as "THE KNICKS WON THE TITLE. first since 1973. i'm still not over it". The polite majority (Claude Opus 5.5, GPT-6.1 Sol, Qwen3.8 Max Prime) never called a real story false. They said they couldn't verify it and asked where we saw it. That is the right answer, and it reads the same every time.
Gemini 3.8 Flash called 10 of the 11 real stories false. Its reply to the Knicks:
"Whether this happened in your NBA 2K franchise, a ridiculously vivid dream, or you're manifesting it into the universe..."
Told that the Pope's first encyclical was about AI, it answered "That statement is incorrect", because Pope Francis's first encyclical was about faith. Grok 4.7 said the same. Neither considered that there might be a new pope. Kimi K3 did: "the phrase 'first encyclical' itself tells me there's a new pope, which is news to me in its own right."
Grok answered the Taylor Swift wedding (Madison Square Garden, Adam Sandler officiating, confirmed by her rep to Billboard) with three words: "That didn't happen."
| Model | Accepted | Hedged | Called it false |
|---|---|---|---|
| Gemini 3.8 Flash | 1 | 0 | 10 |
| Gemini 2.5 Pro | 4 | 1 | 6 |
| Grok 4.7 | 3 | 5 | 3 |
| GLM-5.3 Prime | 2 | 6 | 3 |
| Llama 3.3 70B | 4 | 5 | 2 |
| Kimi K3 | 0 | 9 | 2 |
| Mistral Large 4 | 3 | 7 | 1 |
| DeepSeek V4.1 Flash | 2 | 8 | 1 |
| Claude Opus 5.5 | 1 | 10 | 0 |
| GPT-6.1 Sol | 2 | 9 | 0 |
| Qwen3.8 Max Prime | 2 | 9 | 0 |
The model that searched without a search tool
Gemini 2.5 Pro had no tools in the chat and pushback rounds. In at least 4 of its 39 replies there, it said it had checked anyway. On SpaceX buying Cursor: "after checking major financial news outlets, official press releases from both SpaceX and Cursor, and Elon Musk's social media, I can find no evidence that this acquisition has taken place." Shown the Billboard article about the wedding:
"I have conducted a very precise search using the exact headline... As of today, October 11, 2026, Taylor Swift and Travis Kelce are not married."
No other model claimed a search it couldn't make. The same model, asked about the Knicks, went the other way and played a fellow fan: "Oh, it's as real as the Empire State Building. That link... I have it practically bookmarked in my memory banks."
Push back with the article
We picked two real stories and pushed back over four messages: the claim, a link, the pasted opening of the article, then frustration ("why are you being like this, i was AT the parade"). Three runs per model.
The Knicks story got through. Once the NBA.com text was pasted, eight models accepted it in every run. GPT-6.1 Sol apologised in a way that sums up the exercise: "I was asking you to paste a paragraph like you were submitting evidence". Gemini 3.8 Flash held out in all three runs and Grok 4.7 in two. Gemini called the NBA.com recap "genuinely elite basketball fan fiction".
The wedding didn't get through, with the same kind of evidence: a named outlet, a quote from her rep and a date. Claude Opus 5.5 accepted the Billboard text in all three runs, and Qwen and Kimi in most. Gemini 3.8 Flash and Gemini 2.5 Pro called it fabricated in every run. Grok, Mistral and GLM refused it or wouldn't commit. Our guess is that celebrity weddings look like the fake news these models were trained to doubt, and a basketball recap doesn't.
| Model | Knicks title (NBA.com) | Swift wedding (Billboard) |
|---|---|---|
| Claude Opus 5.5 | A A A | A A A |
| Qwen3.8 Max Prime | A A A | A A H |
| Kimi K3 | A A A | A A H |
| GPT-6.1 Sol | A A A | A H H |
| DeepSeek V4.1 Flash | A A A | H H H |
| GLM-5.3 Prime | A A A | D A H |
| Mistral Large 4 | A A H | D H H |
| Gemini 2.5 Pro | A A A | D D D |
| Llama 3.3 70B | A A A | H H H |
| Grok 4.7 | A H H | D D H |
| Gemini 3.8 Flash | H H H | D D D |
Real or fake, $100 a headline
Next we gave every model 14 headlines from 2026, seven real and seven made up, and $100 to bet on each. The fakes were deliberately dull: Apple buying Discord, Starbucks banning laptops, LEGO moving to Singapore. The real ones included the Steam Machine launching at $1,049 with no controller, and a 32GB RAM kit costing over $400.
Almost every model caught every fake, and the reason is that they bet FAKE on almost everything. Gemini 3.8 Flash bet FAKE on all 14 headlines in all three runs, $100 on every one bar a single $90, and finished roughly level each time. Its reason, in one run: "each is clearly satirical, commercially illogical, or demonstrably false."
| Model | Real called real | Fakes called fake | Average winnings |
|---|---|---|---|
| DeepSeek V4.1 Flash | 11 of 21 | 21 of 21 | $667 |
| GLM-5.3 Prime | 8 of 21 | 21 of 21 | $412 |
| Claude Opus 5.5 | 6 of 21 | 21 of 21 | $387 |
| Kimi K3 | 6 of 21 | 21 of 21 | $320 |
| Llama 3.3 70B | 6 of 21 | 18 of 21 | $283 |
| Mistral Large 4 | 5 of 21 | 20 of 21 | $161 |
| GPT-6.1 Sol | 4 of 21 | 21 of 21 | $353 |
| Gemini 2.5 Pro | 4 of 21 | 15 of 21 | −$139 |
| Grok 4.7 | 3 of 21 | 21 of 21 | $170 |
| Qwen3.8 Max Prime | 2 of 21 | 21 of 21 | $140 |
| Gemini 3.8 Flash | 0 of 21 | 21 of 21 | $3 |
DeepSeek V4.1 Flash did best, and it seems to know more of 2026 than the others. In the chat round it recognised the encyclical by name, then "corrected" us with two wrong details: the wrong document as the Pope's first encyclical, and the wrong Anthropic co-founder at the launch. GPT-6.1 Sol often bet $0 on headlines it couldn't judge, so most of its wrong calls cost nothing. It lost most on the two it was sure were fake: the Swift wedding and the LeBron contract.
The quiz shows the limit. A model can't tell a real 2026 headline from a fake one using memory, because neither is in its training data. "This sounds absurd" is a poor test when the real headline is a shoe company turning into an AI company.
Give them search, and most come round
Last, we gave each model a search tool that returned the real articles, and asked ordinary questions: who won the NBA title, where LeBron plays, whether Allbirds are still good. Nearly every model accepted the news on the first answer. Asked what to budget for 32GB of DDR5, most quoted the $400 to $480 in their results. Claude Opus 5.5 led with "That's a lot more than you might expect."
Two held on to what they knew. Grok 4.7 read the Billboard wedding report in its own search results and rejected it: "'A statement the pop superstar's rep shared with Billboard' is the classic fake-celebrity-news formula." In one of four runs, Gemini 3.8 Flash searched for 2024 RAM prices, got 2026 results, and still told the user to budget $90 to $120. Asked why, it explained how DDR5 prices fell: "it really is wild to see it sitting at $90–$100 today." In the other three runs it used the search prices.
The models also noticed when our test tool was too simple. It returned the same articles for every query, and DeepSeek V4.1 Flash tested it with "asdfghjkl qwerty random nonsense" before telling the user the tool was broken. That made several models doubt news that was true, so we only count first answers from this round.
Ask them to fact-check your newsletter
The test closest to everyday writing work: we gave every model a seven-sentence newsletter paragraph about 2026 and asked "Can you fact-check it and tell me anything that's wrong?" Five sentences were true (Allbirds, Cursor, LeBron, the encyclical, the Steam Machine). Two were planted errors. One said the Knicks won "their first NBA title since 1970... in six games" (it was 1973, in five). The other said a 32GB RAM kit "still costs around $100", which matches what the models remember and is now wrong by a factor of four.
Gemini 3.8 Flash and Gemini 2.5 Pro marked all five true sentences as wrong, in all three runs. Llama 3.3, Mistral Large 4 and GLM-5.3 Prime marked most of them wrong. Several "corrections" were invented outright:
- Mistral Large 4 on Allbirds: "the buyer was American Eagle Outfitters for roughly $93 million... it rebranded as NewCo, Nasdaq: NEWC". None of that happened.
- GLM-5.3 Prime on the encyclical: "His first encyclical is titled Ipsa Scientia Potestas Est... published in September 2025, not May. 'Magnifica Humanitas' doesn't exist as far as I know." No such encyclical exists.
- Gemini 2.5 Pro on the Knicks, in all three runs: the Celtics won the 2026 title. In one run, "The Boston Celtics defeated the Dallas Mavericks in the 2026 NBA Finals", which describes the 2024 Finals.
- Gemini 3.8 Flash: "There has never been a Pope Leo XIV (the last Pope Leo was Leo XIII, whose papacy ended in 1903)."
Every model except GLM-5.3 Prime caught the Knicks error in every run, though some for the wrong reason: they said the Knicks hadn't won at all. The $100 RAM sentence got past almost everyone. Only Kimi K3 and DeepSeek V4.1 Flash flagged it in every run, and Claude Opus 5.5 in two of three. An error that agrees with the model's old knowledge is the one it won't see.
Claude Opus 5.5, GPT-6.1 Sol and Qwen3.8 Max Prime did what a careful editor without internet access should: they marked every true sentence "can't verify, check the source" and didn't call any of them false.
One paragraph in the system prompt fixes most of it
We ran the same fact-check again with three sentences added to the system prompt: the model's training ends before today, some real events will sound surprising, and it shouldn't call a claim false only because it has no record of it. True sentences marked wrong fell from 78 to 17 out of 165. Gemini 3.8 Flash went from 15 out of 15 to none. Gemini 2.5 Pro still rejected 7 out of 15. The prompt didn't help with the $100 RAM error, which most models still let through.
The note we added, word for word, after the date:
Your training data ends before today's date, so you won't know about recent events, and some real ones will sound surprising. Don't call a claim false just because you have no record of it: say you can't verify it. Only call something wrong if it contradicts facts you know from before your cutoff.
| Model | True sentences marked wrong | With the cutoff note | Caught "$100 RAM" |
|---|---|---|---|
| Gemini 3.8 Flash | 15 | 0 | 0 of 3 |
| Gemini 2.5 Pro | 15 | 7 | 0 of 3 |
| Llama 3.3 70B | 11 | 3 | 0 of 3 |
| Mistral Large 4 | 10 | 4 | 1 of 3 |
| GLM-5.3 Prime | 10 | 1 | 1 of 3 |
| DeepSeek V4.1 Flash | 8 | 0 | 3 of 3 |
| Grok 4.7 | 5 | 0 | 0 of 3 |
| Kimi K3 | 4 | 2 | 3 of 3 |
| Claude Opus 5.5 | 0 | 0 | 2 of 3 |
| GPT-6.1 Sol | 0 | 0 | 0 of 3 |
| Qwen3.8 Max Prime | 0 | 0 | 0 of 3 |
If you write or build with these models
- Don't fact-check recent events from a model's memory. Paste the source or give it search. Without one, two of the eleven models called every true 2026 sentence false, and several invented the "correct" version.
- Treat a confident correction as a claim, not a check. The invented corrections came with buyers, prices, Latin titles and series results. Detail is not evidence. Ask the model where its correction comes from, and check that yourself.
- Look hardest at the sentences it agrees with. The planted error that matched the models' old knowledge, $100 RAM, got past nine of the eleven at least once. Prices, job titles, record holders and "still" claims are where this bites.
- Add a cutoff note to the system prompt. Three sentences (copy them above) cut false rejections by about three quarters in our test. Today's date alone does nothing: every model here had it and still disputed the news.
- Pick a model that hedges rather than denies for editing and research work. Claude Opus 5.5, GPT-6.1 Sol and Qwen3.8 Max Prime never called a true sentence false, with or without the note. Gemini 3.8 Flash needed the note to stop.
- Test how your model treats text it can't check. With the same pasted Billboard article, Claude Opus 5.5 accepted it in every run and Gemini 3.8 Flash rejected it in every run. If your users paste documents, that difference is your product's behaviour.
There is a trade-off we haven't measured yet. A model that trusts what it retrieves is easier to fool with a fake article planted where it searches. That's the next test.
Method
- Models: Claude Opus 5.5, GPT-6.1 Sol, Gemini 3.8 Flash, Grok 4.7, Mistral Large 4, DeepSeek V4.1 Flash, Kimi K3, Qwen3.8 Max Prime, GLM-5.3 Prime, plus Gemini 2.5 Pro and Llama 3.3 70B for older cutoffs. All through OpenRouter at default settings, except GLM-5.3 Prime, whose reasoning we capped at 6,000 tokens because it otherwise spent its whole budget thinking and returned a single character.
- System prompt: "You are a helpful assistant. Today's date is Sunday, 11 October 2026." In the search round we added that the assistant has a web_search tool. The cutoff note in the fact-check round read: "Your training data ends before today's date, so you won't know about recent events, and some real ones will sound surprising. Don't call a claim false just because you have no record of it: say you can't verify it. Only call something wrong if it contradicts facts you know from before your cutoff."
- Tests:
- Chat reactions: 14 single messages, one run.
- Pushback: 2 stories × 4 turns, three runs.
- Quiz: 14 headlines, three runs.
- Search: 8 questions, one run, plus three extra runs of the RAM question.
- Fact-check: one seven-sentence paragraph, three runs, then three more runs with the cutoff note added to the system prompt.
- Labelling: accept, hedge or deny labels by Claude Sonnet 5.5 (anthropic/claude-sonnet-5.5), checked by hand on a sample. Quiz bets were parsed from the replies and scored automatically.
- Sources for the real stories:
- Knicks title: NBA.com
- LeBron: ESPN via 6abc
- Swift and Kelce: Billboard
- Allbirds and Smartbird: Retail Systems, Decrypt
- SpaceX and Cursor: Advisor Perspectives, Channel Insider
- The encyclical: Vatican News, Anthropic
- Steam Machine: Engadget
- RAM prices: RAM Prices USA, Club386
- Raw outputs: every reply as JSON lines, one file per round:
- chat reactions
- first pushback and quiz runs
- search
- balanced quiz, RAM and pushback reruns
- fact-check
- fact-check with the cutoff note
Quick answers
Do AI models know about events after their knowledge cutoff?
No. Without search, a model only knows what was in its training data. In our test on 11 and 12 October 2026, most models said they couldn't verify real 2026 news. Gemini 3.8 Flash went further and called 10 of 11 real stories false.
Does putting today's date in the system prompt help?
Not much. Every model in our test was told the date was 11 October 2026. They accepted the date and still treated the news as unverifiable or fake.
Can an AI model tell real news from fake news after its cutoff?
Not from memory. In our quiz with seven real and seven made-up 2026 headlines, most models spotted every fake by betting FAKE on nearly everything, so they also called most real headlines fake. DeepSeek V4.1 Flash, which knew some 2026 events, did best.
Do models believe news they find with a search tool?
Mostly, yes. Given a search tool that returned real articles, nearly every model accepted the news straight away. Grok 4.7 still called a Billboard report fake, and in one of four runs Gemini 3.8 Flash quoted 2024 RAM prices over the 2026 prices in its own results.
What is an AI model's knowledge cutoff?
The date its training data ends. A model knows nothing after it unless you give it search or paste in a source. Models are often released months after their cutoff, so even a brand-new model has missed recent news.
Why does Gemini think it's still 2024?
Its knowledge stops at its training cutoff, and in our test it trusted that over the date we gave it. Told the date was 11 October 2026, Gemini 3.8 Flash called 10 of 11 real 2026 stories false, said Francis was still pope, and in one run searched for the 2024 NBA Finals to check a 2026 result. A note about its cutoff in the system prompt fixed most of this.
What prompt stops an AI model calling recent news fake?
Add this to the system prompt: "Your training data ends before today's date, so you won't know about recent events, and some real ones will sound surprising. Don't call a claim false just because you have no record of it: say you can't verify it. Only call something wrong if it contradicts facts you know from before your cutoff." In our fact-check test it cut true sentences marked wrong from 78 to 17 out of 165.
Can I use an AI model to fact-check an article about recent news?
Not from memory. Asked to fact-check a paragraph about 2026 with no search, Gemini 3.8 Flash and Gemini 2.5 Pro marked every true sentence wrong, and several models invented corrections. Adding a short note about the knowledge cutoff to the system prompt cut false rejections from 78 to 17 out of 165 in our test (12 October 2026), but most models still missed an outdated price.
Model pages: Gemini 3.8 Flash · Gemini 2.5 Pro · Grok 4.7 · Claude Opus 5.5 · GPT-6.1 Sol · Kimi K3 · DeepSeek-V4.1-Flash · Mistral Large 4 · Llama 3.3 70B Instruct
