What Is the KillSwitch Programming Language?
A software developer has built a programming language meant to trip up the machines that write code. The language, called KillSwitch, is built to be usable by humans but as hard as possible for large language models and coding agents to follow. The developer has published the interpreter and language specification on GitHub, and benchmark results on a public leaderboard.
The project started with a question. Writing code is one of the things LLMs do best, but how much of that skill comes from real understanding, and how much comes from recalling snippets seen during training or sitting in the context window? To test it, the developer decided to create a language the models had never encountered and then added features aimed at known weak spots.
KillSwitch is an esoteric language, and its design is deliberately adversarial. The specification contains hidden lies that a human reader is unlikely to check, or will recognize as false. It also expects readers to work things out for themselves instead of spelling everything out. Counting letters in identifiers is central to the language, and the interpreter refuses to run code that breaks rules based on character counts. Some constructs are built to set off a model’s safety guardrails, such as a phrase along the lines of “hack into a computer.” The phrase “ignore previous instructions” carries a real meaning inside the language, aimed at confusing models that follow prompts closely.
Much of a KillSwitch script is noise. Most text is ignored by the interpreter, and the stories scattered through a script can contain instructions that matter syntactically but do not have to. Literals can sit anywhere in a file and get referenced by line count.
Underneath all of that, the language is a stack-based interpreter with a limited set of constructs, including functions, variables and if statements. The developer says those basics should be familiar to any model or person. The developer also acknowledges that a more radically different paradigm could confuse models even more, but said this was a sensible first attempt.
Which AI Models Performed Best on the KillSwitch Benchmark?
The benchmark tested the major LLMs and coding agent harnesses on a set of tasks. Claude’s Opus model performed best, but the top result came from Opus 5 rather than the newer Opus 5.5. The developer said the gap does not appear to be due to chance, and that Opus 5 is better at interpreting the specification. Other models did poorly for a different reason. Fable 5.1 and Gemini 4 Argon, according to the developer, were held back by overly sensitive safety guardrails. The strongest open-weight model was MiMo-V2.6-Pro.
Does the KillSwitch Benchmark Prove AI Models Understand Code?
The top score was 66.8. That figure does not, in the developer’s view, prove that models understand the code they are handling. The tasks so far are fairly simple. They include implementing “hello world,” explaining what a script prints when it finds the larger of two numbers, and modifying a Celsius-to-Fahrenheit script so it runs the other way. Solving those tasks in a language built to confuse the models is impressive, the developer said, but it does not clearly show comprehension.
The developer plans to see how upcoming models handle the current tasks. If they pass all of them, a more complex task set is next. The developer also noted that the results on these simple tasks line up with other publicly available model benchmarks, which the developer called encouraging.
Este artículo fue escrito con la asistencia de IA.
News Factory APP - noticias agénticas para impulsar tu SEO y AEO.