What a waste of time

fine tuning is just additional training, so if it works from scratch it works with fine tuning. And no it may not be as effective as other methods but the poster was claiming it was impossible

Train one from scratch

Why don’t you actually implement a transformer from scratch and then speak more confidently, this is like talking to a virgin about sex.

Fine tuning is additional training, there are lots of ways of doing that and sometimes it’s absolutely ideal, there are tradeoffs

Again you are using an incredibly limited definition of fine tuning based on what the open ai api allows, which once again tells me you don’t know ML.

Fine tuning is ANY additional training on a foundational model, this can be MLM training on the model base or selectively training the subsequent layers.

OF COURSE this can add knowledge as you are doing the same training that got it knowledge in the first place. Glad to see you jumped on the chatgpt band wagon last week, build a transformer from scratch and come talk to me

They will absolutely reshape the world in the next 5 years, all I'm saying is in its current state I haven't found it helpful. I'm sure in the next couple of years it's the main thing I will use

This is a fine approach but fine tuning can and does add knowledge to models, please quit saying that

Lol this guy doesn’t understand ML, you are absolutely adding knowledge to the model

Huh? I think that depends on the fine tuning you are talking about. Fine tuning can absolutely add knowledge to a model

Fair enough then give them problems to solve and measure their output. This feels like “90% of dentists claim crest improves your dental health”

I’ll take an independent study into consideration but today I find it more of a novelty

I doubt it, I do pretty standard engineering, whats more likely is there is selection bias in the survey and people are overestimating it due to hype.

I'd love to see an actual double blind study.

I’m and MLE and I’ve used it a bunch, it’s hardly ever actually useful. It gets close but it’s not there and it’s faster to google almost every time.

It will be useful in probably a year or two, but it needs to understand how to run its own experiments. Anyone who actually thinks this is useful right now is just buying hype

Yeah I don’t buy a survey, could be heavily biased

I don’t even roll yet but that 40% number, I would love to see how they calculated it.

I’ve tried gpt 4 on a lot of problems and it fails 9/10 times and I would be faster just googling it.

This stuff will be amazing it’s just not quite yet

So WebGPT doesn’t quite do this, it uses a JavaScript library to simplify web pages to basic text

Dex is the closest that comes to mind.

With how deep the python ecosystem is, and how fast LLMs are moving, the next language for ML will likely be English

Please without the transformer we would never be able to scale, not to mention all of this being built on BERT as well. Then a bunch of companies scaled it further including Google

https://arxiv.org/pdf/1706.03762.pdf the paper that made all this possible.

Google has also been leading in research around transformers and NLP for some time. Not that they don’t in ways share from each other

Yeah right, OpenAI is built on google research, and cool you worked a half functioning chat or into the worst messaging and search app, congrats

I think Google wins this race in the end, seeing ChatGPT be plugged into crappy Microsoft products tells me where it is heading

Nah foundational models will be replaced with distributed ones

Such an insane point of view, yeah you can’t criticize virtue signaling without virtue signaling….

Again the dumbest thing I’ve seen on Reddit today and that says a lot.

Try and use common sense rather than arguing yourself in a circle

Ah so there is no way to criticize virtue signaling without you yourself virtue signaling.

Thanks for the dumb comment of the day

Right, what China is doing is about a million times worse than Elon but I’m sure the virtue signaling won’t go there