Viewing a single comment thread. View all comments

tinylobsta OP t1_j6haurw wrote

Hey people’s, co-creator of NF here. NF is created using generative algorithms and machine learning techniques for the dialogue, audio, speech, and pretty much everything other than the prefabricated 3D assets.

We started this about four years ago before OpenAI and SD kinda swept the landscape and are thinking now how to incorporate pieces of those into it.

We’re still thinking about where to take it next, so ideas and feedback always appreciated.

33

blueSGL t1_j6i4rli wrote

> We started this about four years ago before OpenAI and SD kinda swept the landscape and are thinking now how to incorporate pieces of those into it.

Elevenlabs can be used for voice cloning and synthesis https://beta.elevenlabs.io/s

SD is being used to create textures directly in blender using the depth map feature of 2.0 https://www.youtube.com/watch?v=EzpSj8JOb1c

and you can likely find a prompt for GPT3 to create Seinfeld scripts https://www.youtube.com/watch?v=1onxri0duN0 (there are some other voice synth tools linked in the description)

10

fastinguy11 t1_j6ifgbc wrote

eleven labs is way to fucking expensive for a always on show, that would be 528 dollars per day

8

EskNerd t1_j6l5mee wrote

$360/day by my math (50k characters/hour at $0.30/1k characters), but still crazy expensive.

1

S3ndD1ckP1cs t1_j6hobil wrote

I love the concept! I could see this developing with newer, better AIs to create a never ending Saturday Night Live of continuously generated skits interrupted by humorous ads.

You guys are definitely onto something.

8

nocloudno t1_j6i6zae wrote

I like the idea of the skits not being funny but the ads are

8

SWATSgradyBABY t1_j6jg46k wrote

No matter how many times ads ruin every new experience we get you are always in the comments begging for more.

1

Readswere t1_j6hxqvf wrote

Yes... it could be excellent, a zeitgeist machine with irreverant takes... an organic TV channel

I can't believe AI image generation has gone this far, but I don't know how strong the 'templates'/algorithms this uses.

Incredible!

1

ziplock9000 t1_j6hq4ye wrote

So all of the 3D assets were created by humans?

2

tinylobsta OP t1_j6ilizy wrote

Yup, at the time we created this, there was no good tech in place for the artwork. With SD, we think there’s a path to replacing our existing art pipeline with something generative, but it’s still being figured out. The problem is that 3D assets take a long time for models to create, and we run in near real time. But we’ll get there.

2

MarginCalled1 t1_j6iu0ye wrote

I'd think the ideal way to handle this would be to create a large enough buffer, say a couple hours. It wouldn't technically be 'live' but very close to it and allow for 3d artwork generation.

Additionally you may be able to save previously generated content for reuse going forward, for example if I wanted to create a character named Daffy, instead of drawing him each time you could have AI generate him in every possible motion then refer to that going forward. That would save a ton of compute and shave a lot of time off the processing requirements.

1

tinylobsta OP t1_j6jp1jb wrote

We've considered this -- the show is actually on about a 2m delay, but otherwise, it's entirely live. You can't see it in the iteration I have streaming rn, but the entire show is configurable... if you want less of one character, we can do that. Want more of one setting? We can do that, too! More lines per character? etc.

It was a design decision we made so that the audience (in the future) can morph the narrative of the show. We actually monitor the Twitch chat and can pick up keywords to help shape the narrative (without defining it, the generative stuff does all that). So we wanted to keep to the 2m per scene concept. Might need to do something like that in the future (batching), though, if time-to-create keeps being a constraint for 3D models.

6

MarginCalled1 t1_j6jw5db wrote

I'd assume that your time-to-create would gradually become faster and faster with how quickly this particular technology is moving and with general hardware tech advancement as well.

You guys are on the cutting edge, some might say you are a little ahead of your time. Regardless it's welcome innovation and all I can do is wish you the absolute best. Fascinating stuff

2

SWATSgradyBABY t1_j6jhb6b wrote

Every possible motion is not practical but I like where you are going.

1

bemmu t1_j6ik5ik wrote

What is this actually running on, is it Unity?

1

SkaldCrypto t1_j6jdbni wrote

This was dope I got a laugh when it talked about waking up everyday cause you know, it's infinite.

1

ImpossibleSnacks t1_j6jn4wk wrote

This is super fascinating, thanks for this. Obviously it’ll get better as you incorporate newer tech.

One thing I’d do is pay attention to advancements in music generation like MusicLM and once that tech matures I’d make a version of 80s MTV where all of the music and music videos are AI generated. Sitcoms take a bit of time investment but back in the day you could pop in and out of MTV for a few minutes and catch a couple videos, I think that really works well with this “forever” model. And offering 80s MTV and 90s sitcoms would get a ton of attention and traffic.

1