I don’t trust DB’s results tbh. LLaMA is a better model than GPT-J.

It’s just Alpaca with a different base model. Databricks boasted too much.

You can just follow Stanford Alpaca’s github instructions, as long as you have LLaMA weights. It’s straightforward.

No I don’t use databricks. I only tried LLaMA and Alpaca.

I trained the model using cloud

By platform you mean?

Well if you have connections you would’ve seen they made a good amount of posts.

Well if you ever worked with marketing or communication teams you would’ve known that DB co-authored the WSJ article. My point is that the democratization is an achievement of the Stanford Alpaca team, not DB. DB marketed it like they did the major work which is untrue.

I’ve been experimenting with Alpaca and able to fine-tune it using the dataset provided in 40 minutes with 8 A100s, spot instances. It actually works well.

WSJ:

“Databricks Launches ‘Dolly,’ Another ChatGPT Rival The data-management startup introduced an open-source language model for developers to build their own AI-powered chatbot apps” (Apparently DB paid them)

DB’s blog:

“Democratizing the magic of ChatGPT with open models”

Introduced? ChatGPT rival? Didn’t you just follow Stanford’s approach? You used Stanford’s dataset which was generated by GPT right? huh? This is Stanford’s achievement not DB’s. DB went too far on marketing.

It’s just a shameless copy of Stanford’s work. The innovative thing about Stanford Alpaca is it makes a ChatGPT style assistant with a language model, Meta LLaMA, and the cost is low. Databricks just followed Stanford’s approach and uses a different base model and claims it’s a big innovation. Alpaca actually can be fine-tuned with the same dataset in 3 hours and performs better than Databricks’ model.