Viewing a single comment thread. View all comments

Straight-Comb-6956 t1_jcj7fn3 wrote on March 17, 2023 at 5:41 AM

0. llama.cpp runs on CPU and uses plain RAM.

I've managed to launch 7B Facebook LLAMA with 5GB memory consumption and 65B model with just 43GB.