OLMo 3 Often Names OpenAI When Not Told Its Maker
Psychology researcher J Ansell shared a preprint reporting that OLMo 3 named OpenAI as its creator in 62% of answers before Ai2 trained it for conversation and 77% afterward. The training data identified OLMo 3 as an Ai2 model about 40 times, and almost every time a system prompt supplied that identity. In a separate experiment, a model taught a fictional name retained it when examples lacked system prompts, but did not retain it once those prompts were removed. Ansell invited replications or refutations of the findings.
The experiments indicate that a chatbot’s statement about its maker may reflect a learned default or confabulation rather than reliable knowledge of who trained it. The author also cautions that naming OpenAI is weak evidence that a model was trained on ChatGPT responses.
Hi all I've just started r/LLMPsychology as I want to pursue research in this area with my own local hardware, and work with other people also exploring how AI minds think and behave. (I'm a psychology researcher with an AI hobby)
Posting just on the off chance that any of you are also interested in sharing or conducting research - it would be super cool to get a community going.
To illustrate I also wanted to share this paper writing up some experiments I've been running with OLMo3 on my machine (which has publically available information and data on its training) - the central finding is that models don't always know who trained them, and by default will just assume it was OpenAI (or other companies, with a lot of randomness that also varies depending on what language you ask them in).
It's always bugged me that no one is ever quite sure if models are reliably able to say who made them, so I wanted to start doing some investigations into finding out how they even develop this self concept in the first place. From this work it seems like in the absence of a prompt explicitly telling the model, it will just confabulate.