Submitted by EducationalCicada t3_10vgrff in MachineLearning
harharveryfunny t1_j7knqfa wrote
Reply to comment by bartturner in [N] Google: An Important Next Step On Our AI Journey by EducationalCicada
OpenAI trained GPT on Microsoft Azure - it has zero to do with Google's TPU. While the "Attention Is All You Need" paper did come out of Google, it just built on models//concepts that came before. OpenAI have proven themselves plenty capable of innovating.
bartturner t1_j7l64gq wrote
> OpenAI trained GPT on Microsoft Azure - it has zero to do with Google's TPU.
Geeze. ChatGPT would NOT exist if not for Google because the underlying tech was invented by Google.
OpenAI uses other people's stuff instead of inventing things themselves like Google.
Many of the big AI breakthroughs from the last decade+ have come from Google. GANs is another perfect example.
https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)
The TPUs are key in being able to bring a large language model to market at scale. Not training but the inference aspect.
harharveryfunny t1_j7lu67f wrote
What underlying are you talking about? Are you even familiar with the "Attention" paper and it's relevance here? Maybe you think OpenAI use Google's Tensorflow? They don't.
GANs were invented by Ian Goodfellow while he was a student at. U.Montreal, before he ever joined Google.
No - TPUs are not key to deploying at scale unless you are targeting Google cloud. Google is a distant 3rd in cloud marketshare behind Microsoft and Amazon. OpenAI of course deploy on Microsoft Azure, not Google.
bartturner t1_j7lugv5 wrote
Geeze. Who do you think invented Transformers?
https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)
NO!!! GANs were invented by Ian while he was working at Google. It is a pretty interesting story.
The vast majority of the major AI breakthroughs from the last decade+ came from Google.
OpenAI really does NOT do R&D. THey more use the R&D from others and mostly Google.
harharveryfunny t1_j7lvevz wrote
https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf
See page 1 footnote : "Goodfellow did this work as a UdeM student".
bartturner t1_j7lwvdt wrote
Ha! Go listen to Lex's podcast. Ian explains it all and it was ALL while working at Google.
harharveryfunny t1_j7ly2jz wrote
And then he travelled back in time to go write that paper at U.Montreal ?
Anyways, Schmidhuber was the real inventor ;-)
bartturner t1_j7lza3y wrote
Go listen to the podcast and Ian explains it all. Plus no Schmidhuber was NOT the inventor. It was Ian.
Go listen to the podcast and get back to me.
The key AI R&D from the last decade plus has all come from Google. Not from OpenAI and most definitely not from Microsoft.
Viewing a single comment thread. View all comments