Viewing a single comment thread. View all comments

ebolathrowawayy t1_j1zpmyo wrote

Reply to comment by lambolifeofficial in ChatGPT Could End Open Research in Deep Learning, Says Ex-Google Employee by lambolifeofficial

I don't see how unless open source figures out a way to distribute training across machines which afaik is incredibly inefficient/impossible right now. It seems that most progress is due to testing out ideas on $100mil worth of hardware iteratively. Oh and also having massive data on a scale that open source will never have access to.

1

onyxengine t1_j1zsjhj wrote

Distributed training will get better

6

Artanthos t1_j22jjeb wrote

Will it get better faster than the closed systems improve?

1

onyxengine t1_j23qkpz wrote

Eventually yes it will become way faster than “closed systems”, because it will be in the cloud on the best machines. Cloud hosting services are clearly incentivized to make distributed training for open source communities affordable and accessible.

2

Artanthos t1_j24dri4 wrote

The cloud? That is owned and operated by the companies developing the closed systems?

The best machines? While competing against companies with billions of dollars of dedicated funding?

You need to reevaluate your logic.

1