>GPT4 - 100 Trillion Parameters

I must be out of the loop. Where is that rumor from?

Before you go through this work, you know that there are existing datasets, right?

See: https://github.com/EBazarov/nsfw_data_source_urls and https://github.com/alex000kim/nsfw_data_scraper for example.

If you want to train a NSFW classifier, use the existing sets first. And use a pre-trained Imagenet classifier first and fine tune it. This will get you 90+% of the way there. It would make sense for you to have your own testing set to make sure that it works for your use-case (CVAT or VoTT work fine), but goodness, don't start from scratch.

It will be hard to get a free source of data for publications like that, because they are not free.

However, there is GDELT: https://www.gdeltproject.org/ (wikipedia: https://en.wikipedia.org/wiki/GDELT_Project ) It's a project that collects events and other data from a variety of sources.

Are there still farmers? Of course there are.

There will still be software engineers, but their jobs will be very different. Someone will need to define the inputs and outputs ,what it is supposed to do, to interact with. The actual code will mostly be written by AI, but it's direction will be determined by people.

Perhaps eventually even that will go away, but if that's the case, no job is safe.

V1 refers the the mammalian primary visual cortex. http://www.scholarpedia.org/article/Area_V1

Cells in V1 respond to simple features such as lines of various orientations, certain simple frequencies, colors, etc. the article discusses it more.

The first layer of a CNN does something similar.

It doesn't make sense to have a blanket ban on articles posted on it.

This is very interesting, if somewhat dense and hard to follow if you don't have some of the background.

I recommend reading an article they reference: A Mathematical Framework for Transformer Circuits https://transformer-circuits.pub/2021/framework/index.html

If nothing else, that paper will explain that OV means output-value:

>Attention heads can be understood as having two largely independent
computations: a QK (“query-key”) circuit which computes the attention
pattern, and an OV (“output-value”) circuit which computes how each
token affects the output if attended to.

Nice. Keep it up. I'm looking forward to November's digest already.

Certainly in terms of ML all you have to do is look at NeurIPS or other top-level conference to see that they are publishing more. But I got the impression that u/nillouise was discussing AGI which most ML is not. In terms of the singularity (this sub), I don't see it and it is because of the technical focus on applications.

While I agree that the hype of humanoid robots is currently overblown, I can't say that the article itself was particularly enlightening.

It points out that the number of industrial robots are increasing rapidly, but it's not clear what jobs they are doing exactly, what areas that are being quickly replaced by robots and what areas within industry that are not, what areas are ripe for replacement in the near future and which will remain the province of humans, and why. Humans are replaced by robots when 1. the technology is there, so they can and 2. they are cheaper than humans. Both of these are changing: the technology improves, human costs change, but neither of these is smooth, and they interact, since the availability of a robot can cause the cost of a human to decrease, and you need people to fix robots (currently).

Also, the flat out statements that humanoid robots will not, for example, make you a cup of tea and other 'it will never happen' similar comments seems foolish. There is no principled reason that humanoid robots won't be successful; the only questions are the timeline and the economics.

For example, sex bots already exist (not me, of course, don't be silly). As the technology improves and costs go down, there will be more of them. So the scifi future of people having sex with robots already exists, it's just not very evenly distributed (with apologies to Gibson).

Fair enough, the analogy is flawed; let me try again. Then you think you should have access to all books, all music, all movies, etc. because they do not have the scarcity of a rocket and can be easily copied?

I'm a huge fan of open source, and I especially appreciate that so much of the software and tooling in ML and AI is open source. At the same time, DeepMind can release or not whatever they want, you are not entitled to it. But fear not, someone will release an open source work-alike soon enough. Perhaps HuggingFace, maybe some other group.

Gato didn't perform especially well compared with other networks trained (individually) on each test. It was a demonstration of generalization, and not particularly well suited for any particular task. The authors themselves called it a proof-of-concept.

>We should be able to use this model. Everyone should be able to enjoy the benefits of AI.

If you want to use a model, build and train it yourself. If someone builds a rocket, you don't get to go for a ride. Why do you think that you should just be able to use a model that someone else designed and trained?

Nice story dude, but you are really r/lostredditors

I understand that. I don't even bother understanding what the latest developments in javascript libraries are or what release we're in with Java. I can barely keep up with changes in Python, ML libraries, and things specific to my work. Since (as I mentioned) I do applied stuff, I have to go between handling large data (dask, BigQuery, etc.) and ML frameworks (sagemaker, vertex ai, etc.) as well as the underlying algorithms. It's too much.

>Why do you even need to understand everything? FOMO?

FOMO is part of it. I'm interested in ML in general, and think that AI is coming (really unsure when) and I want to be part of all of it. I have this fear that someone is going to do something very important and I just won't know about it.

There is also the expectation (at least where work) that when you are a 'Machine Learning Engineer' that you have a pretty good grasp on the field as a whole. You don't want someone to say 'What do you think of Random Forests, how do they work?' and you go 'What's a Random Forest?' (hyperbolic example). I kind of sucks when you are the 'ML Guy' in a pretty big company so people come to you with (random) questions.

Finally, I'm old enough that I think that when ML started, I did know most of it. Having been raised on Duda and Hart and then was around when backprop made it's second renaissance, I remember when Elements of Statistical Learning came out. So, perhaps it's a sadness that the field has blossomed beyond me.

here's a link to the actual paper: https://journals.lww.com/cogbehavneurol/Fulltext/9900/Consciousness_as_a_Memory_System.19.aspx

"[O]our theory of consciousness rejects the idea that consciousness
initially evolved in order to allow us to make sense of the world and
act accordingly, and then, at some later point, episodic memory
developed to store such conscious representations. Our theory is that
consciousness developed with the evolution of episodic memory simply—and
owerfully—to enable the phenomena of remembering"