What it asks
How often do wrong LLM outputs pass property and metamorphic checks, the checks that need no exact oracle? A wrong output that passes is a silent failure. Method
In property-based testing, a test states what must hold for every input in a range and runs on inputs chosen at random. Metamorphic testing derives new test cases from ones that passed, and can be used without a test oracle, a way to tell correct behaviour from incorrect.
How it is studied
The checks’ verdicts are compared against exact oracles, across a ladder of open-weight models. Method This page does not name the models, the tasks or the checks yet.
Where it stands
In preparation. Method No result is claimed here until the paper or a preprint is public.
My part
I am the paper’s first author.
