You can't predict a pandemic. And you can't predict the singularity. Even if you could, there is no way to prepare. The AI models are growing rn. But there is no guarantee that they will continue growing. Quite the opposite actually. Everything has a limit, but it will take time to reach it.

OpenAI is just trying to generate hype now. This could mean that they need to find more investors. When companies start doing that l, it usually is bluff. They probably realised that getting good clean data is going to get exponentially harder. So they have to pay humans to help them acquire the data somehow.

It's not that people lack imagination. They just imagine different things than you. Most of them don't imagine a world with AI.

It's not that people lack imagination. They just imagine different things than you. Most of them don't imagine a world with AI.

Some LLMs are not trained with the right amount of parameters or the right learning rate. But the static nature of LLMs is the biggest problem. You need neuromorphic hardware and spiking neural networks to address the issue. In the meantime I think quick fixes will be attempted such as forward 2x passes. My friend Fred says that just adding small random Gaussian noise to the parameters can also help. Obviously human brains are very noisy but somehow very efficient too.

1. FDVR

2. Automatic translation real time like in Star Trek

3. Some sort of recommendation engine/ wizard for all situations. I think mostly social interactions.

4. A tool that does planning.

5. A tool that gathers news.

6. A tool that takes care of my financial situation. If money is still relevant.

7. A tool that that takes care of my health.

Need neuromorphic hardware and spiking neural networks and quantum computers. Even if qubits double every two years, it will take a while. GPT is just static parameters. You need some way to constantly update them. Anyway LLM is one of thousands required systems. We don't have thousands of labs doing all the required projects. They are doing more or less the same. We are nowhere near that point.

It depends. If you have a fact with a certain probability p, it could have been confirmed n times. The standard error if I remember correctly is the square root of p(1-p)/n. So let's say p = 0.9, the std is like a margin of error sqrt(0.9 * 0.1/n). You want n to be high to have a low std.

AGI would be like us but with extra ability. ASi will be able to do much more. AGI could mean lost jobs and high suicide rates. ASI could mean mass extinctions.

We can have AGI in three years but if we get there it won't be a good AGI. If we take our time, it would be a proper AGI. The road to ASI will then be full of ANI and AGI. The thing that we say is AGI has to be nothing like AGI. However, AGI is nothing like ASI or ANI.

My friend Fred says that LLM would be nothing like XLLM. And XLLM will be nothing like SLLM. For one XLLM will likely use forward 2x instead of backprop. And SLLM will have spiking neural networks.

IMO SLLM will be part of AGI. ASI would be too weird to even imagine. AGI would require quantum computers and ANI to operate the QM. With Winograd FFT. ASI could use something wilder than QM.

The Asian science institute exists.

So basically GPT3+ is like a microbrain that is as good as dead. Human brains have way more 'parameters' and they change in value all the time. Otherwise it would be hard to have an original thought. So the ramifications are pretty trivial.

Static parameters are meaningless. Human brains are not static until after death. Besides modeling reality requires more than a bit of algebra.

32K context is the new 600K RAM. The bigger the model the more resources you need to support it and the more expensive it gets. Without any guarantee about the quality. For example ChatGPT would produce code like:

int result= num1 + num2; return result;

That's in itself not technically wrong but it is unnecessary long. Any static analysis tool would have nagged about this. Also, unit tests or compilers would have caught any actual errors. The OpenAI culture is of PhDs with a certain background. They work in Jupiter notebooks and don't know about standard Dev tools.

My friend Fred says that he can add value with his code generation startup because of that. I also think that LLMs and more traditional technology combined are the way to go.

Hey now! If you really feel this way, you should talk to someone in person. Most people who have problems don't talk about them and seem happy on the outside. I think that is a big mistake!

People are interested in sports, gossip, politics and maybe seven other things. AI is never one of them. My friend Fred says that the early internet was like that too. IMO ChatGPT is like the Java applets of those days. Now obsolete because of better technology. However, the number of internet users doubled each year back then. According to what I have read online ChatGPT has 100 million daily users. This is much more than the number of web users in the beginning of the internet.

So Fred says that we will get a 10x boost by EOY in performance. It could come from model compression or something else. If Bard, Claude, Ernie, and the other bots are any good, we could expect a billion daily AI users. Fred wants to do a code generation startup in one programming language. It's a niche product, but he says that we can have a million users. Obviously there's companies who can do the same, but on the other hand Twitter started out like a basic app and look where it is now.

Goebbels was very intelligent. Intelligence and ethics are not the same thing. Whether you can create Frankenstein's monster with HIA, IDK. But I think HIA is less developed than AI. However, with AI able to reason about proteins, one day pills could be made that can give us near genius abilities. Imagine giving the pills to an army. Suddenly every conscript could become a Napoleon. I think that this scenario is more likely than what you describe.

This is great almost like telekinesis. Next century people might receive implants directly after birth. I think being a cyborg has many advantages. You can participate in a hive mind, learn faster, and communicate faster.

That's reinforced learning, right? My friend Fred says that RL is more fragile than supervised learning. Has to do with the flexible nature of RL. It's good enough for some games, though.

They could have one if they want. You only need 40M dollars to buy a thousand A100s. They might already have them. Or they could be paying OpenAI to help.

Palantir can predict protests based on social media. I'm sure it works a bit like Bing. You say 'Hi, what is up?' It says 'There could be a riot in X soon.' Replace social media with reports from commanders in the field and you can do something similar. The system can say 'I think there's a major enemy offensive in Y'.

My friend Fred says that the rules don't apply to the military. They can do whatever they want whereas civilians have to worry about regulations. But that has never stopped anyone for long.

IDK... I tend to be rude to my computer. Could get banned for this. I shared an office with someone who kicked his computer when he was frustrated. And then there's ambient noise...

I am not a firm believer in the literal technological singularity. Moore's law and the knowledge of human brains currently don't really support it. Quantum computers might change that.

But if I look at my friend Fred, he is excited. I'm also excited but not as much. There was a German company that we thought would bring a ChatGPT like bot in 2020, but that didn't happen. So it looks like that you can't give it too much freedom. You have to guide it. This makes generating code currently the best option because it follows rigid rules. Will this code lead to a self reinforced feedback loop? There's no way to tell.

My friend Fred wants to focus on code, preferably one programming language. The plan for his possible startup is to do unit tests, correctness proof, and linters to assure quality.

Related to LLMs, I have been thinking about news, tweets, and blogs. Pictures would work too I think. But killer apps tend to be video related these days like YouTube and TikTok. So you need an intermediate step to get some text.

The world is burning. By the end of the year, ChatGPT will be ten times faster. Bing, Bard, Claude, Ernie, Galactica, Baba will take over. Soon they will be 100x more powerful, meaning less than a second per request. Speech recognition will be so good that you only need to mouth a bit and the AIs will create games and books and software and art for you. My friend Fred says that he will quit his job when the new Nvidia GPUs become available. They are basically the replicators of Star Trek. You don't need anything else.

If you follow the bread crumbs back, you will find artificial neural networks decades ago, but computers were slow and had megabytes of memory. Data points in the past offer no guarantee for the future. Even if you can stack neural layers as though they were dirty dishes, you are just doing statistics. Which is fine but there are many different methods to reason that would work better.