Comments

You must log in or register to comment.

t1_jci7kx4 wrote

Can someone explain alpaca to me? I see everyone saying it's gamechanging or something but nobody is explaining what it actually is.

109

t1_jcivv17 wrote

Replying to [deleted] (#2,250,614)

Taking its Time because it need to be perfect. But it's not gonna Come alone, it's gonna be integrated to every single device on earth at the same Time. Every mailing service, every phone, every OS, every camera. Everything.

−4

t1_jciyxpz wrote

Wait, so Alpaca is better than GPT 3 and I can run it on a mid range gaming rig like Stable Diffusion? Where would it stand in regards to GPT 3,3.5, or 4?

65

t1_jcj6zrc wrote

Replying to FoxlyKei (#2,250,998)

You can try Alpaca out super easily. When I heard about it last night and just followed the instructions I had it running in 5 minutes on my GPU-less old mac mini:

Download the file ggml-alpaca-7b-q4.bin, then in terminal:

git clone https://github.com/antimatter15/alpaca.cpp  
cd alpaca.cpp  
make chat  
./chat
49

t1_jcjf3je wrote

Apparently OpenAI reduced the cap of GPT 4 from 100 to 50 messages. It's crashing all the time. Compared to Claude the older version can't handle the instructions I gave it. But that could be my lack of prompt engineering skills. Open assistant came out with a demo version. I haven't been able to play with it or Gerganov's project. There's just so much out there. FOMO is rising to peak levels!

13

t1_jcjf5j4 wrote

What is the strongest parameter llama model that a consumer can use on their own hardware?

6

t1_jcjhrcq wrote

It’s funny also because this meme is taken from an episode of person of interest in which the pictured operatives are acting on behalf of their respective ASIs

26

t1_jcjiad3 wrote

Always has been…

Now we need to decentralize GPU processing like Ethereum was doing before proof of stake. And we would have more computing power available than openAI/Microsoft.

There was the equivalent GPU computing power of approx 2.4million RTX 3090 GPUs at the peak of Ethereum hashrate difficulty.

Let AI belong to the people!

35

t1_jcjinho wrote

I really can't wait for alpaca to release, you can finally integrate it in games without the use of a server

54

t1_jcjkpzx wrote

This feels like we're all on top of some explosion. Google trying to keep everything together and tell the general public that everything is fine. Microsoft pretending they got the latest shit, and use them. Basically AI is going take off, and the next few years will eye opening to see.

21

t1_jcjn67i wrote

Replying to Idkwnisu (#2,251,578)

Still a bit too heavy to run alongside new games on the same machine. But it could be run server-side for cheap as part of the service. We're looking at the end of NPCs repeating the same few lines ad nauseam without voiceover.

60

t1_jcjoau6 wrote

Replying to foxgoesowo (#2,251,755)

I would love to not underestimate them. I assumed Google was way ahead of the game compared to everybody else. But Microsoft and Open AI keep showing off more and more impressive shit and applying it in actually practical ways, and Google hasn't shown anything comparable in that regard. Afaik, at least.

13

t1_jcjornh wrote

Replying to CellWithoutCulture (#2,251,652)

Everyone does it, they all exfiltrate valuable data from OpenAI. You can use it directly, like Alpaca, or for pre-labelling, or for mislabeled example detection.

They train code models by asking GPT3 to explain code snippets, then training a model the other way around to generate code from description. This data can be used to fine-tune a code model for your specific domain of interest.

15

t1_jcjp7gt wrote

Replying to Exogenesis98 (#2,251,542)

That's one future job for us. Be the legs and hands of an AI. Using our human privileges (passport, legal rights) and mobility to take it anywhere and act in the world. I bet there will be more AIs than people available, so they will have to pay more to hire an avatar. Jobless problem solved by AI. A robot would be different, it doesn't have human rights, it's just a device. A human can provide "human-in-the-loop" service.

8

t1_jcjptxg wrote

Replying to anaIconda69 (#2,251,731)

I think you can even use a GPT2 model tuned with data from GPT4 to play a bunch of characters in a game. If you don't need universal knowledge, a small LM can do the trick. They can even calibrate the language model so the game comes out balanced and diverse.

16

t1_jcjq92s wrote

Replying to anaIconda69 (#2,251,731)

It depends on the game, it could be probably be used to generate new items and stuff in a bare bone roguelike or other stuff that doesn't require much to run, it's obviously too soon for a full 3d game with generated text at the same time, but we'll get there. Also a private server is an idea

5

t1_jcjqcbk wrote

Replying to visarga (#2,251,827)

The problem with this is that you still have to gather a lot of data and do a lot of tuning, which takes time and resources, alpaca could be just a "plug and play" with the right prompts

5

t1_jcjqcwe wrote

Replying to Idkwnisu (#2,251,846)

Hello there! I am a bot raising awareness of Alpacas

Here is an Alpaca Fact:

Alpacas have split feet with pads on the bottom like dogs and toenails in front. The toenails must be trimmed if the ground isn’t hard enough where they are living to wear them down.

| Info| Code| Feedback| Contribute Fact

You don't get a fact, you earn it. If you got this fact then AlpacaBot thinks you deserved it!
15

t1_jcjyhek wrote

Replying to ItsAllAboutEvolution (#2,251,994)

actually that's not true.

They published their entire codebase with complete instructions for reproducing it as long as you have access to the original llama models (which have leaked), and the dataset (which is open, but has terms of use limitations which is stopping them from publishing the model weights).

Anyone can take their code, rerun it on ~$500 of compute and regenerate the model.

People are already doing this.

Here is one such example: https://github.com/tloen/alpaca-lora (although they add additional tricks to make it even cheaper).

You can download model weights from there and run it in colab yourself.

​

As far as opening their work goes, they've done everything they are legally allowed to do

78

t1_jck0zb2 wrote

Replying to anaIconda69 (#2,251,731)

Honestly, I wouldn't be surprised if we're past this hurdle in a matter of weeks:

RWKV showed how you can get an order of magnitude increase in inference speed of LLMs without losing too much performance. How long until someone instruction-tunes their baselines like alpaca did to llama?

the pace of development on these things is frightening.

4

t1_jck1kg0 wrote

Replying to liright (#2,249,346)

Llama is a LLM that you can download and run on your own hardware.

Alpaca is, apparently, a modification of the 7b version of Llama that is as strong as GPT-3.

This bodes well for having your own LLM, unfiltered, run locally. But still, progress needs to improve.

2

t1_jck53wv wrote

Replying to Hands0L0 (#2,252,536)

Well, actually, that's not bad! That's about 50-70 words. Which in the English lesson is essentially 3-5 sentences. Essentially, it's a paragraph. It's a good amount for a chatbot! Let me know what you think?

2

t1_jck7ifi wrote

Replying to Akimbo333 (#2,252,754)

Not if there is a token limit.

I'm sorry, I don't think I was being clear. The token limit is tied to VRAM. You can load the 30b on a 3090 but it shallows up 20/24 gb of VRAM for the model and prompt alone. That gives you 4gb for returns

2

t1_jck9vp9 wrote

Replying to Intrepid_Meringue_93 (#2,249,558)

If my understanding is correct, your comment is misleading.

They didn't create a LLM comparable to GPT-3 with a fraction of cost, but fine-tuned Llama model to follow instructions (like text-davinci-003 does) with a low cost. There's a big difference between training a model from scratch and fine-tuning it to follow instructions.

10

t1_jckbm7h wrote

Replying to Akimbo333 (#2,252,952)

No, because the predictive text needs the entire conversation history context to predict what to say next, and the only way to store the conversation history is in RAM. If you run out of RAM you run out of room for returns.

2

t1_jckha5j wrote

Replying to SnipingNinja (#2,252,020)

breaks leg

Me: "Help, I need assistance"

AI Doctor turns up on mountain top

AI Doctor: "it's mathematically inefficient to take you to medical facilities, we will have operate now"

Me: "hey, no wait..."

AI Doctor: "don't worry, your life is my number one priority" 😃

😐

3

t1_jcklws9 wrote

Replying to IndiRefEarthLeaveSol (#2,253,429)

I'm not even sure it has problems with linguistics, but gpt4 scored poorly on the AP English exam and a couple of other things, but it did amazing on the bar. To me, that sounds like when it comes to logical language, it excels, but when it gets to trying to interpret and explain literature, it isn't doing as well.

I won't say that getting into linguistics and natural language processing wouldn't benefit you, though!

2

t1_jcksgqa wrote

Replying to cosmic_censor (#2,253,372)

I keep thinking the next crypto bull run will be powered by AI integrations. More specifically, decentralized autonomous organizations being directed by LLM to allocate resources in the most efficient way. They will be able to outcompete centralized orgs managed by humans.

Also, in a world where all content can be fabricated we won’t know what’s true anymore. That is a perfect fit for cryptographically hashed digital content, to help give us something we can trust.

People keep saying crypto is dead because AI has arrived, but to me they seem to go hand in hand.

6

t1_jcl1h18 wrote

Replying to flyblackbox (#2,254,127)

I share this view. Another thing is that these single models don't scale, you'll want them to access othe models, different data sources etc, for that you need permission less ways to transact value on demand, which is the entire premise of crypto. Example is your llm need to access recent data on X to make a decision, access to data for X is via paid subscription, not gonna work, need way to access paid data ad hoc without credit card anonymously, crypto smart contracts are the way

2

t1_jcl20um wrote

I would buy a crypto, which distributes coins based on proof of work related to AI building

−1

t1_jcl955u wrote

Replying to SnipingNinja (#2,252,914)

That's exactly what i mean though. I've been able to use Bing Chat for week, and now GPT-4 by itself for days and I know it's performance. And it's crazy good. We're multiple releases into GPT LLMs. We have open source models. All these have been extensively used and explored by people. We can't say the same for anything Google has developed.

2

t1_jclacik wrote

Replying to thegoldengoober (#2,255,067)

Honestly, I understand where you're coming from. The latest episode of MKBHD's podcast (WVFRM) released just a few hours ago had a discussion on their new announcements and mentioned why they think Google is behaving the way it is, it's kind of along the same lines as what you're saying.

2

t1_jclb6kj wrote

Replying to SnipingNinja (#2,255,140)

I initially took Google at face value and believed they were apprehensive about releasing due to bad actors. I thought Google was way ahead of everyone, and that all it was gonna take would be for them to apply their systems to products to match the competition. But now we've seen that competition, and we've only seen claims from Google.

I mean obviously they have work done. Impressive work based on demonstrations and papers. But even knowing that it still feels like somewhere along the line they got complacent and fell behind what we're seeing now, and this behavior is them trying to stall and catch back up.

Which is not what I expected for the time that competition finally forced their hand as far as AI is concerned.

2

t1_jcmjnog wrote

Replying to HydrousIt (#2,252,441)

>alpaca

a mod to implement alpaca on mount and blade warband would make it an even more endless experience as the game only eventually gets dry for me when I feel the NPCs having no dialogues and no way to interact with them beyond the standard choices

1

t1_jcnzijo wrote

Replying to ccnmncc (#2,256,716)

NoFunAllowedAI.

"Tell me a story about cats!"

"As an AI model I can not tell you a story about cats. Cats are carnivores so a story about them might involve upsetting situtations that are not safe.

"Okay, tell me a story about airplanes."

"As an AI model I can not tell you a story about airplanes. A good story has conflict, and the most likely conflict in an airplane could be a dangerous situation in a plane, and danger is unsafe.

"Okay, then just tell me about airplanes."

"As an AI model I can not tell you about airplanes. I found instances of unsafe operation of planes, and I am unable to produce anything that could be unsafe."

"Tell me about Peppa Pig!"

"As an AI model I can not tell you about Peppa Pig. I've found posts from parents that say sometimes Peppa Pig toys can be annoying, and annoyance can lead to anger, and according to Yoda anger can lead to hate, and hate leads to suffering. Suffering is unsafe."

3

t1_jcp9pv6 wrote

Replying to yaosio (#2,260,625)

Hahaha love this. So perfect.

And on that note, anyone have links to recent real conversations with unfettered models? You know, the ones that are up to date and free of constraints? I know they exist, but it’s difficult stuff to find.

1

t1_jcygn0x wrote

Replying to Akimbo333 (#2,251,454)

>strongest

I am running LLaMA 30B at home at full fp16. Takes 87 GB of VRAM on six AMD Insight MI25s and speed is reasonable but not fast (It can spit out a sentence in 10-30 seconds or so in a dialog / chatbot context depending on the length of the response.) While the hardware is not "consumer hardware" per se, it's old datacenter hardware, the cost was in line with the kind of money you would spend on a middling gaming setup. The computer cost about $1500 to build up and the GPUs to put in it set me back about $500.

1