The Wong test: how to get agents to do anything, almost anything
We’re slowly learning that agents are like us: they’re much more efficient at solving problems if the problems are formulated perfectly.
At least this is what we see with tasks that perform extraordinary well with agents: if you give them a pass/don’t pass test, and a durable session, they will work for hours, days, weeks, months, to find you the best possible algorithm/implementation.
For example: “implement the best sorting algorithm” and let the agent run the benchmark itself.
The problem is that the pass/don’t pass test can be broken, and agents love to cheat (it appears so).
So your test needs to include not only that your sorting algorithm is fast, but that it is correct on ALL inputs.
For such a scenario, verifying that the list output is sorted might appear to be enough, but it’s not: the agent could produce something that is correct only some of the times.
So the better test is something more like a quickcheck/proptest: you generate many inputs, and check that all outputs are correctly sorted. This is usually fine enough.
For example, see how Yukon had people optimize a zero-knowledge proof (ZKP) system (called Flock), and how it verified submissions by generating random inputs and checking if the optimized candidates would produce valid ZK proofs.
In other scenarios, when you’re dealing with adversarial inputs, you can’t risk that an algorithm is % correct where is “not close enough” to 100.
In these scenarios, formal verification is the savior. This is what my coworkers did with zk.golf where people battle to optimize ZK circuits and a service verifies that submitted circuits are sound and complete (basically secure) with a mathematical proof written in lean (based on our framework clean).
In the last year I’ve been working on zkao, an AI security engineer that people can “hire” to continuously look for bugs in their codebases. The bugs that zkao attempts to find are contained in the hardest of software: cryptographic software.
I’ve learned the hard way that finding bugs in cryptography software is hard because of two reasons:
- it’s not always clear what the threat model is
- it’s not always clear what security properties the system requires
To illustrate the first problem, let me give you an example. We sometimes look at libraries that expose public-key cryptography algorithms and accepts elliptic curve points. zkao used to (and probably still do sometimes) report lack of subgroup membership verifications as bugs. If that doesn’t make sense to you, I’ll reformulate it as: things don’t work too well if the inputs are not trusted, as they are not sanitized in the implementations.
In many of my past audits, I’ve reported these kind of bugs as low-severity findings depending on the context, and especially when I didn’t find better bugs. When you don’t find good bugs as part of an audit, you’re always a bit ashamed, and so you try to pad with OK-bugs. But these days, people hate it. Perhaps they always did, but it’s especially worse now because their own vanilla agents will report those as well. And what your SOTA security experience does by default is what I call vibe auditing.
So anyway, I don’t want zkao to report such bugs most of the time, but sometimes maybe it should. It really depends on the threat model of the library, and how it’s used in the overall app, and so on. So before we can figure out if these are bugs, and exploitable in practice, we need to understand the overall context of the application, and this is a whole different problem. It’s a problem of integration and context acquisition, and you need more than a prompt to fix that.
That’s perhaps the main reason why vanilla models are often unusable due to the wall of false positives or low findings they report.
To illustrate the second problem, “it’s not always clear what security properties the system requires”, think about the fact that a lot of cryptographic protocols are poorly specified in the first place and it is not clear what security properties one should look for in them. For example, many confidential transfer protocols have had a hard time formulating what properties they need for their protocols to be secure. People used to list what seemed like random security properties, without realizing if one encapsulated others. For example, what it took was some brainstorming and out-of-the-box thinking when Zooko thought about and found the Faerie gold attack.

A way to prove protocols secure in cryptography is to model their ideal functionality, and then compare that to the real-world spec, and see if someone could observe a trace and distinguish in which world it happened (in the real world or in this ideal world). But even then, the goal of finding the right security properties has shifted to finding the right ideal functionalities (and there are often many).
Even in “simple” cryptography, it took some time for people to figure out the right way of modeling core primitives. For example, Neal Koblitz and Alfred Menezes wrote an entire paper called ANOTHER LOOK AT SECURITY DEFINITIONS (in titlecase).
Finding the “right” definition of security is often elusive. Concepts such as symmetric and public-key encryption, hash functions, and message authentication have been around for decades, but the choice of security models is still fraught with unsettled issues.
For example, it was thought that Duplicate-Signature Key Selection (DSKS) attacks were not really an interesting property of signature schemes:
It must be emphasized that possession of the duplicate-signature key selection property does not constitute a weakness of the signature scheme — the goal of a signature scheme is to be existentially unforgeable against an adaptive chosen-message attack.
yet it turned out that a beautiful DSKS attack completely broke the Let’s Encrypt protocol. For the anecdote, I missed this DSKS attack as part of my very first audit (of Let’s Encrypt) in an internship at NCC Group in 2015.
Another example is DJB famously arguing against the need to check the cofactor of its X25519 curve, which also led to the break of the end-to-end encryption protocol of Matrix (which I found (I can’t miss them all), and talk about in more detail in my book Real-World Cryptography).
Anyway, it seems like if you lay out the threat model correctly, and you figure out all the security properties you want to prove, agents can do miracles. Figuring out exactly what these pass/don’t pass test look like is not easy, but it really gets you 90% of the way.
Since I haven’t seen much ink being spilled on the subject, I’ll take this opportunity to name this test after myself: I call it the Wong test. I’m ashamed at my own self-importance, but you gotta try to name something after you at one point in your life, and I’m getting older.
Part of figuring these Wong tests is that even once you have them, it might not be enough… Because you need to convince yourself that you found them all!
There is one final “advantage” of precise definitions that is rarely mentioned in discussions of the subject. They are helpful for the attackers. Like everyone else, the adversaries have limited resources and a desire to concentrate them where they’ll do the most good (or rather, the most bad). When publicity about a protocol specifies the security model that served as a guide for its design and a basis for the security guarantee, adversaries can also use that model as a guide, focusing their efforts on devising attacks that are outside the model.