Viewing a single comment thread. View all comments

sumane12 t1_j9j0pi7 wrote

My guy, correct me if I'm wrong, but doesn't it outperform humans, in everything but social sciences?...

177

turnip_burrito t1_j9j2sg5 wrote

Yes, and it does it with only 0.4% the size of GPT3, possibly enough to run on a single graphics card.

It uses language and pictures together instead of just language.

229

Neurogence t1_j9jef7k wrote

What is the "catch" here? It sounds too good to be true

54

WithoutReason1729 t1_j9jmd05 wrote

The catch is that it only outperforms large models in a narrow domain of study. It's not a general purpose tool like the really large models. That's still impressive though.

111

Ken_Sanne t1_j9jxg68 wrote

Can It be fine tuned ?

7

WithoutReason1729 t1_j9jxy78 wrote

You can tune it to another data set and probably get good results, but you have to have a nice, high quality data set to work with.

18

Ago0330 t1_j9lm5ty wrote

I’m working on one that’s trained on JFK speeches and Bachlorette data to help people with conversation skills.

21

Gynophile t1_j9msb3s wrote

I can't tell if this is a joke or real

7

Ago0330 t1_j9msg1r wrote

It’s real. Gonna launch after GME moons

10

ihopeshelovedme t1_j9npl0j wrote

Sounds like a viable AI implementation to me. I'll be your angel investor and throw some Doge your way or something.

2

Borrowedshorts t1_j9ka0ta wrote

I don't think that's true, but I do believe it was finetuned on the specific dataset to achieve the SOTA result they did.

5

InterestingFinish932 t1_j9m2xhe wrote

It chooses the correct answer from multiple choices. it isn't actually comparable to chatGtp.

4

em_goldman t1_j9jzamt wrote

That’s so cool!! That’s how humans remember things, too

5

gelukuMLG t1_j9kftza wrote

does that prove that parameters aren't everything?

3

dwarfarchist9001 t1_j9knt85 wrote

It was shown recently that for LLMs ~0.01% of parameters explain >95% of performance.

5

gelukuMLG t1_j9kxnj4 wrote

But higher parameters allow for broader knowledge right? You can't have a 6-20B model have broad knowledge as a 100B+ model, right?

1

Ambiwlans t1_j9lab3g wrote

At this point we don't really know what is bottlenecking. More params is an easyish way to capture more knowledge if you have the architecture and the $$... but there are a lot of other techniques available that increase the efficiency of the parameters.

9

dwarfarchist9001 t1_j9lb1wl wrote

Yes but how many parameters must you actually have to store all the knowledge you realistically need. Maybe a few billion parameters is enough to store the basics of every concept known to man and more specific details can be stored in an external file that the neural net can access with API calls.

5

turnip_burrito t1_j9kgb2q wrote

We already knew parameters aren't everything, or else we'd just be using really large feedforward networks for everything. Architecture, data, and other tricks matter too.

3

soapyshinobi t1_j9jo9rw wrote

It's outperforming religion for some now.

https://www.businessinsider.com/rabbi-chat-gpt-ai-sermon-deathly-afraid-2023-2

16

sumane12 t1_j9jz889 wrote

Atleast AI can make accurate predictions for the next character in a line of text, which is better than any religion has predicted 🤣

27

Fedude99 t1_j9ps9w4 wrote

Religion is just anything you have faith ("belief") in without understanding the belief justification chains (or even that there is such a thing as different kinds of links in belief justification chains).

Thus, modern atheists are religious as well as they don't actually understand the Science (tm) and "logic" that shapes their beliefs, and they end up in culture war battles no different from early religious wars.

Modern science can no longer even predict what a man or woman is, which is just as simple as predicting what color the sky is. As an atheist myself, it's important to acknowledge the win religion has on this one.

3

sumane12 t1_j9ptb8u wrote

Great post. I was about to counter that religion would require some kind of worship, but there's religions such Buddhism that requires no such worship.

2

SnooHabits1237 t1_j9rglfg wrote

Whst makes you think that atheists don’t understand the logic behind their beliefs? Religion is based off of myth and atheism is evidence based and logical.

1

SnooHabits1237 t1_j9rgs3w wrote

Whst makes you think that atheists don’t understand the logic behind their beliefs? Religion is based off of myth and atheism is evidence based and logical.

1

ninjasaid13 t1_j9js0zk wrote

"ChatGPT might be really great at sounding intelligent, but the question is, can it be empathetic? And that, not yet at least, it can't," added Franklin.

He admitted there's a chance.

5

gthing t1_j9kgkhy wrote

It's good at faking empathy, just like humans.

22

monsieurpooh t1_j9nh885 wrote

Anyone who's a staunch opponent of the idea of philosophical zombies (to which I am more or less impartial) could very well be open to the idea that ChatGPT is empathetic. If prompted well enough, it can mimic an empathetic person with great realism. And as long as you don't let it forget the previous conversations it's had nor exceed its memory window, it will stay in character and remember past events.

2

Bakagami- t1_j9j6f40 wrote

yup and correcr me if I'm wrong, but those aren't average humans either, those are experts in their fields

3

Cryptizard t1_j9j6j7x wrote

You are wrong. It’s not experts. It’s randos on mechanical Turk.

48

Bakagami- t1_j9j7a63 wrote

rip, they should've included expert performance as well then

15

Artanthos t1_j9jhm3l wrote

You are setting the bar as anything less than perfect is failure.

By that standard, most humans would fail. And most experts are only going to be an expert in one field, not every field, so they would also fail by your standards.

14

Bakagami- t1_j9jid4u wrote

Wtf are you talking about. It's a benchmark, it's to compare performance. I'm not setting any bar, and I'm not expecting it to beat human experts immediately.

9

SgathTriallair t1_j9knp1a wrote

Agreed. Stage one was "cogent", stage two was "as good as a human", stage three is "better than all humans". We have already passed stage 2 which could be called AGI. We will soon hit stage 3 which is ASI.

−1

jeegte12 t1_j9mocmt wrote

we are a million miles away from AGI.

0

Cryptizard t1_j9j80kk wrote

But then they wouldn’t be able to say that the AI beats them and it wouldn’t be as flashy of a publication. Don’t you know how academia works?

3

Bakagami- t1_j9j8djw wrote

No. I haven't seen anyone talking about it because it beat humans, it was always about it beating GPT-3 with less than 1B parameters. Beating humans was just the cherry on top. The paper is "flashy" enough, including experts wouldn't change that. Many papers do include expert performance as well, it's not a stretch to expect it.

17

Cryptizard t1_j9j8qk5 wrote

The human performance number is not from this paper, it is from the original ScienceQA paper. They are they ones that did the benchmarking.

1

IluvBsissa t1_j9j7tmn wrote

Are you joking or serious ?

1

Cryptizard t1_j9j7x5v wrote

Serious, read the paper.

8

IluvBsissa t1_j9j81ht wrote

My disappointment is unmeasurable and my day is ruined.

7

coumineol t1_j9jdp9z wrote

Really? So the time has come where a small-scale AI model being smarter than "ordinary" humans is not impressive.

22

olivesforsale t1_j9jpxi4 wrote

Awe is so last December - impatience is the new mode. They teased us with the future, now we expect it ASAP!

13

Cryptizard t1_j9jxvg2 wrote

It's not ordinary humans, it's people on mechanical turk who are paid to do them as fast as possible and for as little money as possible. They are not motivated to actually think that hard.

5

coumineol t1_j9k4pf5 wrote

That's prejudice. You don't know that.

−1

Cryptizard t1_j9ka13l wrote

No it is economics, they make less money the longer they stop and think about it.

5

Starshot84 t1_j9jne8z wrote

Only because ai isn't as morally flexible as most ppl are

1