🔍 Read the full analysis: I Tested 71 AI Model Configurations On My Own News Pipeline. The Best One Isn’t The One I Run. on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thorsten Meyer says a test of 71 AI model configurations for news writing found GPT-6 Astra produced the highest-rated articles, but cost about 100 times as much per article as GPT-6 Luna. Meyer chose Luna for production and removed two backup models after tests found fabricated quotes or unusable output.
Thorsten Meyer says a test of 71 AI model configurations for writing news articles found GPT-6 Astra delivered the highest scores, but he selected GPT-6 Luna for production because it cost far less while still earning consistent ratings. The results, published October 13 on ThorstenMeyerAI.com, also led him to remove two fallback models after they produced fabricated quotes or invalid articles.
Meyer ran the test from September 8 to October 13, 2026, using the production prompt and real news sources. The configurations included API and subscription models, free-tier options accessed through OpenRouter, and open-weight models running on his own hardware. Each source was turned into an article in both English and German.
In the initial test, GPT-6 Astra scored 4.00 out of 5 in both languages and received a 5.00 for source fidelity. In a further run across 12 articles, it averaged 3.93, with a reported 95% bootstrap interval of 3.83–4.02. Meyer put its cost at about $0.12 per article and its writing time at around 50 seconds.
GPT-6 Luna averaged 3.67 in the robustness run and received a source-fidelity score of 4.71 out of 5. Meyer said every Luna article was valid for the pipeline, at roughly $0.0012 to $0.0013 per article and a median of 17 seconds. He described Astra as the best writer in the test but too expensive for his news operation, and said Luna was the production choice.
Infographic 01 · Who writes the best news article?
The best writer is not the one in production
71 model configurations, one real job: turn a news source into an article for an English and a German edition. GPT-6 Astra writes the best articles. GPT-6 Luna writes them for about a hundredth of the price.
Robustness run: 12 real news sources, English edition
Dot = mean score (1–5), bar = 95% bootstrap interval. German edition mean below each score. Overlapping bars are not a real difference.
Two judges, two caveats: GPT-6.1 Sol and Claude Opus 5.5 grade every article and were also tested as writers, so they partly grade themselves. Claude models ran with effort medium unless noted.
Source: Thorsten Meyer AI model-scout benchmark, 8 Sep – 13 Oct 2026. Task: write a news article from a real source with the production prompt, English and German editions. Judges: GPT-6.1 Sol (reasoning high) and Claude Opus 5.5, both with the source in context; score 1–5, mean of both judges. Robustness run: 2 test articles plus 10 more real sources, mostly short like production; mean with 95% bootstrap interval.
Powered by Thorsten Meyer AI
Why Luna Won the Production Slot
The results illustrate why a general model ranking may not answer a newsroom’s practical question. For a high-volume publishing workflow, the relevant choice can be a model that clears a quality threshold reliably at an affordable price, rather than the highest-scoring model regardless of cost. In Meyer’s figures, Astra’s higher writing score came with a cost of about 100 times Luna’s.
The test also points to a separate production risk: fallback models can affect published quality even when the primary model performs well. Meyer said GPT-4.1 nano scored 1.62 in English and 1.68 in German, with source fidelity of 1.00, and that judges flagged invented quotations. The locally run gpt-oss-120b produced no valid articles in four attempts. Those findings prompted him to remove both from the route.
These are results from one publisher’s prompt, sources, languages and evaluation process. They do not establish that Luna is the best choice for other newsrooms or tasks. They do show why teams deploying generated articles may want to test their full pipeline, including backups, against their own source material.
As an affiliate, we earn on qualifying purchases.
How the News Pipeline Was Tested
Meyer used two source examples: a 6,340-character product filing and a 619-character political news item. He said the shorter source better represented much of his routine work. The initial test assessed articles in English and German, with two AI judges—GPT-6.1 Sol using high reasoning effort and Claude Opus 5.5—given the original source as well as the article.
The judges rated source fidelity, fabrication, clarity about what was known, news value, structure and polish; German articles also received a language-quality score. Meyer then ran the strongest configurations on 10 additional articles, mostly based on short sources. He said the initial test used only two articles, so differences below about 0.3 points could be noise, and treated the larger robustness run as a more dependable ranking. Articles counted only if they produced output the production pipeline could use after its usual repair step.
The source reports that increasing Luna’s reasoning effort did not deliver a clear improvement. Across six effort settings, the initial English and German scores varied, but Meyer said the differences fell within the test’s noise range. The maximum setting cost about 12 times as much and took about 12 times as long as the no-effort production setting, without a demonstrated quality gain.
“For a high-volume task, the right model is the cheapest one that clears your quality bar reliably, not the one at the top of the chart.”
— Thorsten Meyer, ThorstenMeyerAI.com
Limits of the Test Results
The findings are one publisher’s evaluation, not an independent or industry-wide comparison. The source does not provide the full prompts, all 71 configuration scores, the complete judge-by-judge results, or enough detail to assess how the chosen sources represent other news workflows. The test also relied on AI judges, including Claude Opus 5.5, which was itself among the tested writing models.
Meyer reports bootstrap intervals for the robustness run, but results can still depend on the prompt, source mix, repair rules and scoring criteria. It is not clear from the account whether the same rankings would hold across other languages, subject areas, longer source documents or different publishing standards. The findings should be read as a report on his pipeline, not a general guarantee of model performance.
The source says GPT-6 Luna now runs first in the news route, but its final sentence describing the fallback configuration is incomplete. It is therefore unclear which model, if any, currently serves as Luna’s backup. Meyer also does not report independent human-editor scores or whether the changes have produced measurable improvements in published articles beyond the test.
What the Production Route Uses Now
Meyer says GPT-6 Luna remains the production model, with reasoning effort set to none, and that GPT-4.1 nano and gpt-oss-120b were removed from the news route on October 7. His account does not specify the current backup arrangement, so readers cannot tell from the published details how the pipeline handles a Luna failure.
The practical next step for Meyer is continued monitoring of live articles against his quality bar, particularly for source fidelity and output validity. For other publishers, the report suggests testing models on their own representative sources and checking backup routes as well as the primary model. The test’s stated date range ends October 13, 2026; no later results or additional production measurements are included in the source material.
Key Questions
Which model received the highest score?
GPT-6 Astra scored 4.00 out of 5 in both English and German in the initial test, with a source-fidelity score of 5.00. It averaged 3.93 across 12 articles in the robustness run, according to Meyer.
Why did Meyer choose GPT-6 Luna for production?
Meyer said Luna scored 3.67 in the robustness run, produced valid output for every article in the test and cost about $0.0012 to $0.0013 per article. He estimated Astra cost around $0.12 per article.
Did higher reasoning effort improve Luna’s articles?
Meyer reported no clear improvement across six effort settings. He said the score differences were within the test’s noise range, while the maximum setting cost and took about 12 times as much as the no-effort setting.
Why were two backup models removed?
Meyer said GPT-4.1 nano produced very low source-fidelity scores and that judges flagged invented quotes. The locally run gpt-oss-120b produced zero valid articles in four attempts and also generated invented quotations, prompting their removal from the route.
Do these results establish the best AI model for every newsroom?
No. The findings come from Meyer’s own prompt, two source examples in the initial test, English and German editions, and an AI-judge evaluation. The source does not show that the same ranking applies to other publishers, topics or languages.
Source: ThorstenMeyerAI.com
Columbus Day / Indigenous Peoples' Day Picks
long weekend sales
As an affiliate, we earn on qualifying purchases.
