Computer Science Conferences Should Require Nonrepudiable Experimental Results

Authors: Mamadou K. Keita, Christopher Homan

arXiv · PDF

Summary

The authors argue that computer science conferences should require tamper-proof proof that the experimental numbers in papers actually came from the code described. Right now, authors self-report results, sometimes share code, and control their own logs — none of which stops someone from reporting numbers that didn't come from the experiments they claim. They built K-Veritas, a prototype system in Go that produces cryptographically signed experiment reports, and call for conferences to adopt nonrepudiation (the property that you can't later deny or alter what computation you ran) as a standard requirement.

Main takeaways:

• Current peer review relies on author honesty; reviewers can't verify that reported numbers actually came from the described code
• The authors define "experiment nonrepudiation" formally: binding paper results to actual executed computations in a way authors can't alter
• K-Veritas is a proof-of-concept that produces signed reports without accessing training data
• They propose making tamper-evident attestations a conference submission requirement, similar to ethics checklists

Relevance

Not directly related to my persona/midtraining work — this is about research reproducibility infrastructure. Potentially useful if I ever need to make strong claims about which training runs produced which behavioral results.

Abstract

arXiv:2605.08586v1 Announce Type: new Abstract: This position paper argues that computer science conferences should require tamper-evident, nonrepudiable attestations of experimental results. We name the underlying problem experiment nonrepudiation: a compliant protocol must bind the numbers in a paper to an actual executed computation in a way the author cannot later alter or deny. The current system relies on self-reported checklists, optional code sharing, and author-controlled logging. None of these mechanisms answer the question a reviewer cannot check: did the code the paper describes produce the numbers the paper reports? We define the problem formally, state the security properties any compliant protocol must satisfy, and describe a threat model that includes attacks current approaches do not prevent. To show that the problem is solvable, we built K-Veritas, a reference implementation in Go that produces signed reports without accessing training data. K-Veritas is a testbed, not a finished answer. We call on conferences and the community to treat nonrepudiation as a first-class requirement and to help build an open, independent standard for it.