Viewing a single comment thread. View all comments

yaosio t1_iu2za10 wrote

Reply to comment by 0xWTC in [R] "Re3: Generating Longer Stories With Recursive Reprompting and Revision" - Generating stories of 2000+ words (or even much longer) by 0xWTC

Diffusion for language models provides more coherent output according to various studies I've found. I'm surprised nobody's talking about it considering all the hype about diffusion for image generators. I guess it's not as cool as it sounds. The paper doesn't compare it to GPT models which should have told me something.

https://arxiv.org/abs/2205.14217

https://github.com/xiangli1999/diffusion-lm

There's also a new method that's even faster than diffusion.

https://www.assemblyai.com/blog/an-introduction-to-poisson-flow-generative-models/

I hope you have good luck on your text generating endeavors!

7

DigThatData t1_iu3eb42 wrote

computer vision often overshadows NLP. Hard to compete when something novel is making the rounds with pretty pictures to go with it.

4

0xWTC OP t1_iu3kyag wrote

the paper is actually using GPT-3, as far as I understand. It's hard to compare since you physically can't really generate a 2000 word article with GPT-3 (one shot)

thanks I will look into it

2