Reddit user reports direct audio understanding in ChatGPT web
AI summary
A Reddit user reported that ChatGPT on the web correctly identified an uploaded test beep as a steady, high-pitched electronic tone without executing code. The user also said ChatGPT analyzed an uploaded Russian emo-rap track using Python and soundfile, without Whisper or ASR, and accurately described its hook, instruments, structure and lyrical subject matter. The user could not reproduce audio input through the API or Codex and noted that the web tool sometimes returned an unsupported-audio message.
In-depth insight
The report suggests that ChatGPT's web harness may support meaningful uploaded-audio analysis, but the underlying processing path remains unverified and appears unavailable through the API and Codex.
| Been poking at this for a few days and wondering if anyone else noticed. First I uploaded a generated beep (test-2.wav, literally nothing else in it) and told it to describe it without python or any code. It said steady high-pitched electronic tone, like a test signal, no voices. Correct, and no code ran at all. Then a Russian emo-rap track, heavy autotune, 808s, renamed to shfm.mp3 so the filename gives nothing away. It opened python, read the first 26 seconds with soundfile and just did display(Audio(chunk)). No whisper, no ASR, nothing else in that cell. After that it got the hook pretty much right and described the song correctly: instruments, structure, what the lyrics are about. The parts it got wrong were exactly the parts I can't make out myself either, which is not what you'd expect if it just remembered the song. On another track it even replayed a chunk at 0.9x speed, like it was trying to catch a fast verse, and ended up with around 70% of the words right. The weird part: the model itself doesn't support audio input. Through the API it just doesn't take audio, and in Codex I couldn't get it to listen to anything. Even in the web version the tool once returned "you do not support audio" to it mid conversation. So whatever does the listening lives only in the ChatGPT web harness. The help page for the new audio uploads mentions "audio understanding and transcription" and says it depends on the selected model, but nothing about how it actually works. Anyone know what's going on under the hood? Is there a separate audio model handing the main one a description, or some audio path the API doesn't expose? Has anyone gotten anything similar working through the API or Codex? And if you've tested it on music, how good is it for you? (model: gpt-6, Extra High) submitted by /u/Possible_Statement84 [link] [comments] |