They are running a stripped down version of facebook's LLM. https://github.com/antimatter15/alpaca.cpp is one of the first projects demonstrating it.

So how are people running LLMs locally on laptops and phones? It seems the training costs are unimaginable but you can optimize the models to run on consumer hardware.