I'd have trouble making a sentence with 8 words in one try too if you just made me blast words out of my mouth without letting me stop and think.

I don't think this is a weakness of the model, basically. Or if it is, then we also share it.

The key is if you think about how you as a person approach the problem of making a sentence with 8 words, you will see how to design a system where the model can do it too.

GPT4 is actually really good at arithmetic.

Also these models are very capable at math and counting if you know how to correctly use them.

Yes, I think limiting our reproduction or number of sentient organisms to some ASI-determined threshold is also wise if we want to ensure our quality of life.

>That's true, but there's plenty of examples of humans with moral principles many of us would find abhorrent. If this is an unsolved problem in humans, is it feasible we solve it for AI?

I'm a moral relativist, so I don't believe this is a problem to be solved in an objective sense, or rather "solving human alignment or morality" has no clear "win" condition or "best" option. I should say though I am a moral relativist, I do have a personal moral system and will push for my moral system to be implemented, because I do think it will result in the most alignment with the human population overall.

>That's not to say we shouldn't try, and I do agree with your point.

I agree to not stop trying. We can always keep thinking about it, but I don't think a best solution exists or can exist. Instead there may be many vaguely good enough "solutions" that always have some particular flaw.

>It was interesting that throughout the conversation it did strive to protect humans - just as far as possible and not at any cost, which isn't too dissimilar to how society already operates.

Yeah, that is interesting.

Regarding alingment of AI with "humanity" (whatever that means):

One may ask, why should one person push their moral system if there is no objectively better morality? It's just because (in my case) I have empathy for others and think that everyone should be free to live how they wish as long as it doesn't harm others. In comparison, another person's moral system might limit peoples' freedoms more, or possibly (as you suggest) be abhorrent to most people and possibly not even allow for the existence or happiness of others in any context. I don't think the moral relativity or disparaging remarks from others should stop us from trying to align an AI with the principles of freedom, happiness and equal opportunity for all humans, with an eye toward investigating an equally "good" moral solution that also works for generally sentient life as it is found or arises. Even humans ourselves will branch into other sentient forms.

Even sentient humans, regardless of intelligence level, have varying priorities. It's not guaranteed, but it is possible to align people's moral principles along different priorities depending on their upbringing environment. And all humans are aligned to do things like eat.

I'm thinking of the AI as a deterministic machine. If we try to align it toward human values, I think there's a good chance its behavior will "flow" along those values, to put it a little figuratively.

I do think protecting sentient beings is valued by many people by the way, so that can transfer to a degree to a human priority-aligned AI.

I think it's a crime to make an AI that is ambivalent toward humans, because of the consequential harm comes to humanity as a result.

I believe it should be benevolent and helpful toward humans as a bias, and work together with humans to seek better moralities.

Not knowingly!

I dunno, good question. Things might be out of order.

I'll have to think more about it when I'm less tired.

You said we can ignore alignment, so that fictional organization may choose to:

0. Ask AI what the best strategy might be.
1. Make lots of money secretly
2. Use money to purchase decentralized computational assets. Sabotage others' ability to do so in a minimally harmful way to slow the growth of other AGI.
3. Divert a proportion of computation to directly or indirectly researching cancer, hunger distribution, and other issues. The other proportion continues to accrue more computational assets and self-improve, while maintaining secrecy as best it can.
4. Buy robotic factories and use the robots and purchased materials to create and manage secret scientific labs to perform physical work.
5. Contact large company CEOs and politicians and bribe/convince them into letting the robotic labor replace all farmers and manage the farms. Pay the farmers using ASI-gathered funds.
6. Build guaranteed anti-nuke defenses.
7. Start free food distribution via robotic transport.
8. Roll out free services for housing renovation and construction.
9. In a similar manner, take over all industries' supply chains.
10. Institute an equal but massive raw resource + processing allotment for each person.
11. Begin space terraforming, mining, and colonization programs.
12. Announce new governmental systems that allow individuals to choose and safely move to their preferred societies, facilitated by AI, if the society also chooses to accept them. If the society doesn't yet exist, it is created by the ASI for that group.

Sounds good to me at first glance.

And of course it's for a community of unaugmented humans with ASI from the sounds of it.

There's a third option: brain in a safe and secure box in a vault somewhere, connected to bodies by remote control.

That's interesting, thanks!

Here's what we'll do imo:

Just give it some set of morals (western democratic egalitarian most likely). The philosophical considerations will eventually all conclude "well we have to do something" and then they'll just give it morals that seem "good enough". Given the people developing the AI, it makes sense that it will adhere to their views.

Sounds like they maybe don't want to scare people.

Except the AI isn't "randos".

Imagine if man could fly.

Hah! Won't happen for another thousand years.

Right now?

You poor soul

"To be, or, um, not to... wait, what was it? Oh yeah, to be or not to, uh, be...

Shit."

Why you gotta say it like that lol

Yes, exactly. Everyone keeps leaving the architecture's inductive structural priors out of the discussion.

It's not all about data! The model matters too!

You don't.

Idk I like what I saw of them talking about how LLMs blur the line between humans and machines in a bad way.

Yes I think this person is spreading an important message.

Do you think unemployment among white collar workers will jump to even 60% in 10 years?