wywywywy t1_jdm16va wrote on March 25, 2023 at 12:01 PM

> I wanted to train Meta's LLaMA model on this data, but considering their license, I'm not sure if that is the best way. Suggestions will be appreciated.

If we ignore OpenAI's licence, is it ok to perhaps ignore Meta's licence as well? Or is that going too far

> The trained model will be open source, under MIT License.

Is the dataset going to be open source as well? So that other people can use it to train other models.

wywywywy t1_jcaxwnu wrote on March 15, 2023 at 3:22 PM

Reply to [D] What do people think about OpenAI not releasing its research but benefiting from others’ research? Should google meta enforce its patents against them? by [deleted]

I'm more worried that other big players (Meta Google Alibaba Nvidia IBM etc) will follow suite and start withholding information :(

wywywywy t1_jb97nl6 wrote on March 7, 2023 at 11:20 AM

Reply to [R] Created a Discord server with LLaMA 13B by ortegaalfredo

Nice one.

With dual 3090s, I think 30b should be possible in 8bit?

wywywywy t1_j9b2kqu wrote on February 20, 2023 at 4:57 PM

Reply to comment by xrailgun in [D] Large Language Models feasible to run on 32GB RAM / 8 GB VRAM / 24GB VRAM by head_robotics

So, not scientific at all, but I've noticed that checkpoint file size * 0.6 is pretty close to actual VRAM requirement for LLM.

But you're right it'd be nice to have a table handy.

wywywywy t1_j9ar2tk wrote on February 20, 2023 at 3:40 PM

Reply to comment by xrailgun in [D] Large Language Models feasible to run on 32GB RAM / 8 GB VRAM / 24GB VRAM by head_robotics

I did test larger but it didn't run. I can't remember which ones, probably GPT-J. I recently got a 3090 so I can load larger models now.

As for quality, my use case is simple (writing prompt to help with writing stories & articles) and nothing sophisticated, and they worked well. Until ChatGPT came along. I use ChatGPT instead now.

wywywywy t1_j9apjs3 wrote on February 20, 2023 at 3:29 PM

Reply to [D] Large Language Models feasible to run on 32GB RAM / 8 GB VRAM / 24GB VRAM by head_robotics

I had a 3070 with 8GB and I managed to run these locally through KoboldAI.

Meta OPT 2.7B
EleutherAI GPT-Neo 2.7B
BigScience Bloom 1.7B

wywywywy t1_j4los24 wrote on January 16, 2023 at 4:35 PM

Reply to [D] The Illustrated Stable Diffusion (Video) by jayalammar

Very nice video. Clear & concise explanation

wywywywy t1_j32xwg5 wrote on January 5, 2023 at 6:19 PM

Reply to comment by samobon in [News] AMD Instinct MI300 APU for AI and HPC announced by samobon

Saying that though, it's nice to see AMD trying. The monopoly is extremely unhealthy.

wywywywy t1_j18a6g2 wrote on December 22, 2022 at 12:00 PM

Reply to comment by maizeq in [D] Running large language models on a home PC? by Zondartul

I haven't tried it myself but Intel has their own dist of Python and they also have their own Pytorch extension. They seem to be quite usable from looking at some of the github comments.

wywywywy t1_j151o6u wrote on December 21, 2022 at 6:46 PM

Reply to [D] Running large language models on a home PC? by Zondartul

You could run a cut-down version of such models. I managed to run inference on OPT 2.7B, GPT-Neo 2.7B, etc on my 8GB gpu.

Now that I've upgraded to a used 3090, I can run OPT 6.7B, GPT-J 6B, etc.

wywywywy t1_iv3wpty wrote on November 5, 2022 at 2:21 AM

Reply to [P] Learn diffusion models with Hugging Face course 🧨 by lewtun

So if this turns out to be successful, can we expect more free classes? Or is this a one time thing?