IluvBsissa

IluvBsissa t1_j9j9ml9 wrote

Reply to comment by astonzhang in [R] Multimodal Chain-of-Thought Reasoning in Language Models - Amazon Web Services Zhuosheng Zhang et al - Outperforms GPT-3.5 by 16% (75%->91%) and surpasses human performance on ScienceQA while having less than 1B params! by Singularian2501

Dr. Zhang, thank you so much. Please can you tell us more about your model's performance ? How would it do on standard MMLU ? Can it be improved by increasing parameters count ? The paper didn't mention if the human testers were average human or experts ?

6

IluvBsissa t1_j9j8ubb wrote

Reply to What. The. ***k. [less than 1B parameter model outperforms GPT 3.5 in science multiple choice questions] by Destiny_Knight

I don't get it. Why are they comparing their model's performance to regular humans and not experts, like every other papers ? Does it mean these tests are "average difficulty" ? I read somewhere that GPT3.5 had a 55.5% score on MMLU, while PalM was at 75 and human experts 88.8. How would this CoT model perform on standards benchmarks, then ? I feel scammed rn.

7

IluvBsissa t1_j3w2mnf wrote

Reply to comment by V_Shtrum in Escape Velocity from Bullshit Jobs by maxtility

https://www.vox.com/2018/5/8/17308744/bullshit-jobs-book-david-graeber-occupy-wall-street-karl-marx

This interview clarifies a few points. I think Bullshit Jobbers are also random positions created by rich rentier to artificially inflate their company, thus their pride. "I manage a consulting agency of over 300 people", sounds really nice at a fancy dinner, even though half employees are underpaid young interns.

1

IluvBsissa t1_j3w1bi1 wrote

Reply to comment by V_Shtrum in Escape Velocity from Bullshit Jobs by maxtility

Yes, that's his definition.

I would maybe add : if a machine can do your job better than you, and at a lower cost, you have a Bullshit Job. Eg : Pharmacists, data entry mf, tellers...some people love their bs jobs tho, e.g sugar industry Lobbyists who get paid millions each year to veto new national health measure in Congress.

I think a few accademic tried to venture a lil bit in these new waters, but didn't generate any new insights. I think we should ask them on Reddit.

2

IluvBsissa t1_j3vn0wd wrote

Reply to comment by TheSecretAgenda in Escape Velocity from Bullshit Jobs by maxtility

As someone who worked in corporations, I have to say it : yes, Bullshit Jobs exist. Even more so in the US and Japan, the Kings of unnecessary bureaucratic bullshits and forms filling. According to McKinsey (2016), 69% of administrative jobs could be automated. But we refuse to do so, and keep using tons of papers instead of digitalizing all info in the cloud. Maybe when Boomers and X finally f**k off the economy, we may see some changes, but I doubt it. We need a bloody revolution.

3