Submitted by nick7566 t3_yyr0cs in singularity
visarga t1_ix0ji8d wrote
Reply to comment by Kolinnor in Why Meta’s latest large language model survived only three days online by nick7566
> it was utter trash and excessively arrogant
Galactica is a great model for citation retrieval. It has innovations in citation learning and beats all other systems. Finding good citations is a time consuming task when writing papers.
It also has a so called <work> token that triggers additional resources such as a calculator or Python interpreter. This is potentially very powerful, combining neural and symbolic reasoning.
Another interesting finding from this paper is that a smaller, very high quality dataset can replace a much larger, noisy dataset. So there's a trade-off here between quality and quantity, it's not sure which direction has the most payoff.
I'd say the paper was targeted for critique because it comes from Yann LeCunn's AI institute. Yann has some enemies on Twitter since a few years ago. They don't forget or forgive. There's a good video on this topic by Yannic Kilcher.
And by the way, the demo still lives on HuggingFace: https://huggingface.co/spaces/lewtun/galactica-demo
Viewing a single comment thread. View all comments