Viewing a single comment thread. View all comments

Zermelane t1_j2d8bms wrote

> This week, Philip Wang, the developer responsible for reverse-engineering closed-sourced AI systems including Meta’s Make-A-Video, released PaLM + RLHF, a text-generating model that behaves similarly to ChatGPT

Oh, yeah, he does that, quite a lot, too. Basically, most of the cool ML things that come out have a simple enough core that if you are a brilliant enough engineer (and Phil Wang is), and familiar enough with the common concepts that they tend to be built on, you can reimplement it on top of a ML framework in an evening or two.

Think of it as basically being an executable version of the mathematics describing the model. It would take not just GPUs and data, but also still a whole bunch of engineering and code, to actually get from this to a trained model.

Unrelatedly, the same guy made This Person Does Not Exist a few years back, so that might be what you also know him for.

10

ThePlanckDiver t1_j2dloch wrote

With all due respect to lucidrains (whose work is great), this article is clickbait; this is like saying there’s an open-source version of the Empire State Building, which is in fact just the blueprint.

Anyone is free to build it, sure, just bring your own bricks (data) and mortar (FLOPS).

(Those CarperAI and LAION initiatives mentioned at the end sound interesting, but honestly, after all the premature hype for BigScience’s BLOOM and Meta’s OPT and co. which in reality turned out to be a snooze, I’ll reserve my celebrations for when they deliver something that’s actually useful.)

5

mkultra500000 t1_j2eg48m wrote

Honestly. The recipe for these models that will allow for a successfully trained model are a pretty big deal. It’s not a small thing.

3

mkultra500000 t1_j2edn4j wrote

The mathematics that describe the model are the weights and measures that are the model.

Are you saying that it would require training or are you saying that just getting an interface running that talks to the model is what’s left?

1