Imnimo t1_jdi9wze wrote on March 24, 2023 at 4:14 PM

Surely the Alternative Uses Test is all over the place in the LLM training data?

currentscurrents t1_jdj9tsl wrote on March 24, 2023 at 8:04 PM

Oh, definitely. I just checked ChatGPT and it's both aware of the existence of the test and can generate example question/answer pairs. This is a general problem when applying human psychology tests to LLMs.

It does help that this test is open-ended and has no right answer. You can always come up with new objects to ask about.