Viewing a single comment thread. View all comments

maskedpaki t1_jds3ate wrote

Try getting human programmers to do those problems. Guarantee many will fail too.

13

enryu42 OP t1_jds49mm wrote

Well, they do, and quite successfully, this is what these sites are about...

Of course if you ask some frontend engineer to solve some math-y problem, they'll be confused. But this is simply because they lack knowledge, and GPT4 evidently doesn't have this issue. Moreover, I doubt any human programmer will have troubles with the "Beginner" problems, regardless of their specialization.

15

farmingvillein t1_jdsfaq5 wrote

> Moreover, I doubt any human programmer will have troubles with the "Beginner" problems, regardless of their specialization.

FWIW, I think you overestimate humans. Particularly those who haven't actively been practicing leetcode-style coding. E.g., many of the problems are specified in "competition language", not "human-friendly language" (where "human-friendly", e.g., is something you'd be happy to see in a design doc). (Should that matter to GPT-4? I dunno.)

I do think it is fair though to say that, with some baseline level of practice (which is potentially the relevant comparison point), a lot of people would probably nail the "beginner" tests.

6

Narootomoe t1_jds6i3w wrote

Thats a good way to put it I don't think I've seen yet, may I steal it?

"If a human had instant recall to all the knowledge GPT4 has, it wouldn't stumble on any of these problems", something like that

3

red75prime t1_jdtqsmj wrote

Does GPT-4 have instant recall of all of its training data? I doubt it. It probably has some emergent structures akin to episodic memory, but it seems to have trouble distinguishing its memories from its hallucinations, so it's not a fully functional episodic memory (it lacks metamemory or something like that).

1

robobub t1_jdst1oo wrote

> Moreover, I doubt any human programmer will have troubles with the "Beginner" problems, regardless of their specialization.

Have you not heard about how many fail to pass FizzBuzz interview questions?

3