Part of the DGEval benchmark. DGEval: all results and updates
Every model we had tested answered from what it learned in training, with no way to check its answer against an authoritative dangerous goods source. This update asks what happens when a system can check Hazcheck systems before it answers.
We ran the same four DGEval sections against Hazcheck Assist, which uses tool calls to consult Hazcheck systems before reasoning over what comes back, and compared it with the frontier models. Hazcheck Assist is available as part of Hazcheck products, not for individual use.
87.4% overall, ten points above the best frontier model
Hazcheck Assist scored 87.4% overall, about ten points above the best frontier model, Gemini 3.1 Pro at high thinking (77.0%). The advantage is narrowest on multiple choice, where frontier models were already strong, and widest on regulatory recall, where the best score rose from 36.8% to 60.7%.
View data table
| Measure | Hazcheck Assist | Gemini 3.1 Pro (High) |
|---|---|---|
| Multiple choice | 98.1% | 95.8% |
| Open-ended questions | 93.4% | 86.0% |
| DGL lookup | 97.1% | 89.3% |
| Regulatory recall | 60.7% | 36.8% |
| Overall | 87.4% | 77.0% |
Against Gemini 3.1 Pro: multiple choice 98.1% vs. 95.8%, Dangerous Goods List lookup 97.1% vs. 89.3%, open-ended questions 93.4% vs. 86.0%, and regulatory recall 60.7% vs. 36.8%. On multiple choice, 98.1% is also 14.3 points above the corrected human practitioner baseline of 83.8%.
Regulatory recall, by citation depth
Regulatory recall can be scored at three levels: the right chapter, the right section, and the exact subsection a practitioner would cite on a declaration. Hazcheck Assist found the right chapter 92.9% of the time and the right section 87.2% of the time. At the exact subsection it scored 38.4%, against a frontier average of 4.6%: the largest margin, but also the area with the most room to improve.
View data table
| Measure | Hazcheck Assist | Best frontier model | Frontier average |
|---|---|---|---|
| Right chapter | 92.9% | 79.4% | 50.8% |
| Right section | 87.2% | 56.3% | 27.2% |
| Exact subsection | 38.4% | 14.3% | 4.6% |
In Hazcheck products, the citation gap is closed. Hazcheck Assist is used alongside Hazcheck Validate, which checks shipments against the IMDG Code requirements, and the system flags or links to the relevant provisions of the official IMDG Code as users work. So users are pointed to the governing regulation rather than relying on the model to recall it.
The largest gains are on the highest-stakes fields
On the Dangerous Goods List fields the first study flagged as highest-stakes, Hazcheck Assist scored 100% on Stowage Code and Stowage Category (frontier average 53.2% and 50.3%) and 91.7% on Segregation Group (33.0%). Segregation Code is the exception, at 76% against a 34.9% frontier average.
View data table
| Measure | Frontier average | Hazcheck Assist |
|---|---|---|
| Marine Pollutant | 86.2% | 100.0% |
| Primary Hazard Class | 87.2% | 100.0% |
| Packing Group | 79.7% | 100.0% |
| Emergency Schedule | 72.9% | 100.0% |
| Proper Shipping Name | 60.6% | 96.8% |
| Stowage Code (operational) | 53.2% | 100.0% |
| Stowage Category (operational) | 50.3% | 100.0% |
| Segregation Code (operational) | 34.9% | 76.0% |
| Special Provisions (operational) | 35.3% | 100.0% |
| Segregation Group (operational) | 33.0% | 91.7% |
What it costs
The results cost more in money and time: $22.36 and 7.1 hours per 1,000 questions, in a sequential, unoptimised run that includes the tool calls. That compares with $10.80 and 4.3 hours for Gemini 3.1 Pro. Both figures should fall as tool calls are run in parallel.
What this means for Hazcheck users
Across all four sections, and on the stowage and segregation fields the first study flagged as highest-stakes, Hazcheck Assist is the strongest system we have tested. In Hazcheck products it works alongside Hazcheck Validate, which checks shipments against IMDG Code requirements, and the system flags or links to the relevant provisions of the official IMDG Code. That combination, accurate dangerous goods knowledge plus the governing regulation in front of the user, is what general-purpose AI alone does not provide.
Limits and next steps
These are NCB Hazcheck’s own results for its own system: single-run point estimates on the same 1,678 questions, with open-ended answers marked by Gemini 2.5 Flash. Exact-subsection citation (38.4%) and Segregation Code (76%) show where the model itself has most room to improve; in Hazcheck products, users are also pointed to the governing IMDG Code provisions through Hazcheck Validate and links to the official IMDG Code. We will re-run DGEval as the system is refined and as new models are released.