No_Combination_6429 t1_jd20q4w wrote on March 21, 2023 at 7:19 AM

Could you please provide the source Code for the fine-tuning? Also did you use the LoRa approach?

juliensalinas OP t1_jd2owfz wrote on March 21, 2023 at 12:29 PM

Sure. Here's the repo I used for the fine-tuning: https://github.com/kingoflolz/mesh-transformer-jax. I used 5 epochs, and appart from that I kept the default parameters in the repo.

I haven't tried the LoRa approach yet. Do you think it could improve quality?

Thanks for sharing! As far as I know LoRa approach increases efficiency, not so sure about quality Wiki. Maybe the paper can help you further.

Thx!