That sounds sensible, or at least it might brings costs down, but it's just kind of nuts.

70% automation?

Not when powerful AI is being fine-tuned to maximize a reward (money).

This is the whole reinforcement learning alignment problem, just with human+AI instead of AI by itself. Unaligned incentives (money vs. human well-being).

In addition, there's also a large risk of somebody accidentally making it evil. We should probably stop training on data that has these narratives in it.

We shouldn't be surprised when we train a model on X, Y, Z and it can do Z. I'm actually surprised that so many people are surprised at ChatGPT's tendency to reproduce (negative) patterns from its own training data.

The GPTs we've created are basically split personality disorder AI because of all the voices on the Internet we've crammed into the model. If we provide it a state (prompt) that pushes it to some area of its state space, then it will evolve according to whatever pattern that state belongs to.

tl;dr: It won't take an evil human to create evil AI. All it could take is some edgy 15 year old script kid messing around with publicly-available near-AGI.

I wish OpenAI hadn't ever released ChatGPT. Also all those "democratize AI" guys screaming for research labs to release their AI for public use. What a mess. Now we're gonna end up with third party non-research corporations trying to use AI to make money to our long term detriment, and probably get royally rekt by some corpo money-making AI.

I don't think it's really worth worrying about. AI won't be able to do your job until AGI, and automation will be brittle and weak until then. By the time AGI rolls around, all information workers will lose their jobs almost simultaneously.

That makes sense.

There is no point in simulating every atom, you're right.

Also such an "always on atom level" Earth-simulating machine would be larger than Earth itself, which seems like a waste of resources.

In curious too. When you're thinking, doesn't an entire sentence of internal dialogue flash through your mind in an instant? But then there are long periods with no internal dialogue in between. I wonder what it averages out to.

AI will not replace us quite that quickly.

No, the actual definition is the point in time when technological progress makes predictions of the future useless. Usually in context of AI, but it could also be due to other technology.

In a human brains I'd guess it's a mix of both things. A more reflexive response not requiring labeling, and a response to many different kinds of post-labeled signals relating to the door. Not sure how much of each though.

But it has a way higher probability of being conscious then.

I think if we had the right resources, this would make a hell of a research paper and conference talk.

Spooky model magic.

> style is just a desperate grasp that humans have something to offer that ai doesn't.

I think this is the correct assessment. But we may choose human media/products/services for other subjective reasons, like a "journey, not the destination" mindset or social connection.

That was entertaining but... dafuq?

It seems way more likely to me that we get aligned AGI or unaligned AGI than whatever that is lol

I agree with all of this, but just to be a bit over-pedantic on one bit:

> Models cant speak or hear when they want to Its just not part of their programming.

As you said it's not part of their programming, in today's models. In general though, it wouldn't be too difficult to construct a new model that judges at each timestep based on both external stimuli and internal hidden states when to speak/interrupt or listen intently. Actually at first glance such a thing sounds trivial.

It's still not idiotic.

DeepMind's decision makes sense. Even in light of other teams making different decisions.

Do you realize that algorithmically, it is much easier to test approaches on finite state games and later scale up to games with infinite states?

"Idiocy"

Okay buddy, let's see you advance the field of AI in a smarter way.

A modification: I think the optimal future is one where all our personal AI are kept in some bounds by the programming of a superior, autonomous, human-aligned ASI. Not sure what the bounds are thpugh. It can figure that out by discussions with us.

Yes it was the election, and certainly not anything else that happened that year.

Clearly

That's also interesting. It's like building a specialized "wariness" or "discernment" layer into the agent.

This really makes one wonder which kinds of pre-main and post-main processes (like other LLMs) would be useful to have.

That's good.

Maybe also in the future, for an extra layer of safety, when you can several LLMs together, you can use separate LLMs "judges". The judges can have memory refreshed every time you interact with the main one, and can screen the main LLM for unwanted behavior. They can do this by taking the main LLM's tentative output string as their own input, and use that to stop the main LLM from misbehaving.