Viewing a single comment thread. View all comments

robobub t1_jdsria4 wrote

Reply to comment by liqui_date_me in [D] GPT4 and coding problems by enryu42

While GPT-4 is autoregressive, it takes into account the tokens it has chosen to generate incrementally. So it is only limited to O(1) if it attempts to answer with the correct answer immediately. It can in theory take O(m) steps, where m is the number of intermediate tokens it predicts.

3

robobub t1_jdwa5wf wrote

Ill add this:

If it is possible for GPT to do 1+1, it can do a large number of them incrementally. It's not smart enough to do it all the time by planning ahead, (you'll have more success if you encourage GPT to have a train of thought reasoning here and here) but it's possible.

1