VigilSAR, a specialized defense-ISR software platform, has released its latest public LLM leaderboard, showcasing how various language models perform in intelligence, surveillance, and reconnaissance tasks. Unlike typical AI benchmarks, this one emphasizes trustworthy reasoning, reporting, and restraint— qualities critical for operational environments rather than general trivia.
Launched with a set of 14 models evaluated across 300 tasks, the scores were finalized on July 17, 2026. The results are fully public, but the task set remains private. This deliberate choice prevents models from training on the evaluation data, ensuring the scores reflect genuine capabilities rather than memorization. A public leaderboard displays aggregate results, alongside confidence intervals and held-out score gaps to maintain transparency about model generalization.
Among the findings, Claude Fable 5 leads with a score of 67.77, firmly in Band A. Notably, a new entry, Moonshot’s Kimi K3, debuted at #3 with a score of 64.65. This model is classified in Band B and outperforms every GPT and Gemini model on the board, highlighting its strong capabilities for defense-ISR tasks. The leaderboard emphasizes bands rather than precise ranks, reflecting the overlap of confidence intervals and the inherent uncertainty in the measurement.
Why does VigilSAR keep the task set private? The site explains that “vendor claims are not evidence,” and the evaluation aims to determine which models can truly handle critical defense-ISR workloads. The assessment is designed to be vendor-neutral and independent, focusing on models the operators themselves use or consider deploying. Furthermore, the leaderboard includes cost-per-correct-answer economics, giving a practical perspective on model deployment.
For tech readers, this benchmark offers insight into how model deployment realities influence scoring — notably, one locally runnable model is labeled as “sovereign-deployable,” meaning it can be operated securely on-premises, a crucial factor for defense applications. The presence of VigilSAR as an independent evaluator underscores the importance of honest, transparent benchmarking in high-stakes environments.
As an affiliate, we earn on qualifying purchases.
