Yeah, in this case the only real issue is that tacking on 'made by ChatGPT' feels kinda weird. I feel like in society we like to keep up appearances and pretend like there's this level of special sincerity in those kinds of messages, but we all know that when we go to write them we're just copying ideas from other similar messages we see elsewhere and doing our best. It's rarely something truly personal, but it's a bit crass to pull back the curtain and show that.

I think we understand these systems well enough to know that just having knowledge about them isn't enough. We have experience with them going wrong in humans. You can lose the ability to feel pain if the physical structures that enable that are damaged. Knowledge won't help you there, no matter how much you have, if you don't have working nerves. Now, it might be possible to design something in an AI that mimics those systems, but I think that would have to be a very intentional act. It couldn't just be something that happens when the AI has learnt enough about pain unless it also has the ability to alter its own systems and decides to design such a thing for itself.

That also requires a mechanism. I firmly believe that an AI can't actually experience emotions just by learning a lot of information about them. Mimic them, sure, but I don't think you can just spontaneously develop a system through which emotion is felt.

A cardigan still isn't nearly as warm as a full suit.

Well, consider this. The dress code for men is a full suit. What falls into the dress code for women that's ever warmer than that? Women are expected to dress in ways that aren't as warm. The dress codes are what's ridiculous.

Wouldn't you need some sort of mechanism through which to experience pain? Like, even if something is smart enough to perfectly understand those concepts, it's not going to spontaneously generate the kind of systems through which humans experience pain. No matter how well I understand pain, if I don't have working nerves, I won't feel pain.

I mean, it is pretty shitty if you're using it to harass people whose finances you know nothing about, especially in a targeted way just because you have something against them.

Is this in cases where you somehow actually know they're committing tax evasion, or are people just reporting random people under the assumption that they are and hoping to get a payout?

Sure, but the same would be true if I wrote a book about the future as well, even if I had no particular insight.

Yup, though I think the key is that it would have to do that consistently and completely. We have AIs now that are definitely not conscious that can superficially mimic that sometimes and I think it's important that we don't consider things like that to be interchangeable with actual consciousness.

Absolutely not, but we likely will have the ability to create an AI that is long before we have the ability to perform your test.

This is kind of like how cats domesticated themselves. Over time, cats have figured out the kinds of behaviours they need to show in order to get humans to take care of all their needs.

If he's doing fine on his own, it's good that he can be free. And hey, since he's hanging out in a park, people will still be able to enjoy seeing him.

That's two different things. Its actions can be both perfectly reasonable and not aligned with our best interests.

I mean, it's inherently hard to predict. I think it's a little silly to believe you could somehow know exactly what AGI will do to that degree, but we also can't be sure it won't do those things.

Right now, they're trying to make an improved search engine. I'm sure they'll keep working on other approaches to AI with different implementations in the future, but you must admit that, as a search engine, an answering machine is a better fit than a friend.

Yeah. I think it's very understandable that big businesses would want to reign in these products they're trying to design for general use. You don't really want your search engine having a mental breakdown while your ten year old is trying to do research for their homework. It probably won't be more than a year or two until there are open source models that are just as good that we can have a bit more fun with.

Who would invent those technologies if most of the population had vanished? Who would process those resources into the things that people use?

It's a difficult problem and none of the solutions are ideal. You simply can't have an AI that acts without any confines and always behaves in ways that you would prefer.

It already has been, really, but it's surprising how fast you get used to these things. I wonder how much more nuts it will get.

If anything, this should be a reminder that it's not necessarily as smart as it might appear at times. It can still get very lost and not make sense.

This is the downside of not having it do the whole "Beep boop am robot" thing incessantly like ChatGPT does. It's much easier for it to slide a little too far in the other direction and then things get real weird. I hope it doesn't lead to them removing all the personality from it.

Exactly. Like yes, that is how much it costs to hire a maid, and I don't hire a maid, so clearly I'm not willing to pay that price point for something that does the work of a maid. I can't justify that kind of expense to have someone else do things I can do myself for me. If it can do things like repairs, the price I'm willing to pay would probably go up a fair bit, because those aren't skills I have and so I would otherwise be paying for that work.

I don't think it's that they don't know what it's worth, I think they just don't have that kind of money to throw around.

This shows they can be scared into backing down. We've got to start cancelling accounts every time they try it on.