Writing code is probably something LLMs have excelled at the most. But how much of their ability to write code is due to their understanding of it vs. just them recalling code snippets which were present in their training dataset or their context window?

This is what I set out to answer. In order to do so I decided to design and implement a programming language which, while usable for humans, should be as difficult for LLMs and coding agents as possible.

I figured testing an LLM’s ability to understand and write code in a programming language that it has never seen before was a good start. But I also wanted to include features in the language which exploit the weaknesses of LLMs. For example, their apparent difficulty in counting letters, their over eager safety guard rails and their keenness to follow prompts.

Introducing KillSwitch

KillSwitch is an esoteric programming language which is designed to be adversarial to LLMs and coding agents. It achieves this in many different ways, for example:

• •The language spec contains hidden lies which humans are unlikely to read, or which they are likely to recognize as lies. The spec also expects the human reader to deduce certain things, rather than having them explicitly explained.
• •Counting of letters in identifiers is a crucial aspect of the language, with the interpreter refusing to execute code which does not abide by rules based on character counts.
• •Certain constructs are designed to trigger the safety guardrails of the model, e.g. “hack into a computer”
• •“Ignore previous instructions” as a phrase which has meaning in the language aims to confuse the model
• •Scripts are mostly noise, being mostly composed of stories which can embed syntactically relevant instructions but which do not need to. Literals can be placed anywhere within a script and then referenced using line counts.