Building my own proto-AGI: Update on my progress

Submitted by Lesterpaintstheworld t3_11attqx in singularity

Context

I'm building my own ACE (Autonomous Cognitive Entity), and I'm already at a pretty interesting stage with good results & some emerging behaviors. Original post:https://www.reddit.com/r/singularity/comments/113p2jn/the_road_to_agi_building_homebrew_autonomous/

Progress

- Audio: Josh can now hear what I'm saying. It increased the amount of input he his getting ~10x from chat: Voice is the way to go. If I could plug my thread of thought continuously as an input, I would. In the meantime, I'm mumbling all day long in my headset while coding. Good enough ^^

- Code awareness: I started a micro-process feeding Josh the code of his microservices & letting Josh read and make sense of it. It shows encouraging results, as he is capable of showing some understanding what each one does. I'm currently limited by the 4K context size of davinci, but this should change soon (GPT-4 rumored to have up to 32k context window). End goal being the creation of synthetic code, with Josh creating new microservices by himself.

- Critic: One of the hardest parts IMO: reducing confabulation & assessing whether a thought is a good one relative to context. Still working on this part.

- Actor: Josh is now capable of working and reworking on a single piece of text to improve it gradually. This is useful for email crafting for example. I'll be thinking about new ways to "move" ie. act in the world.

- Short-term memory: I gave Josh short-term memory (currently 12 most important thoughts of the session). I'm experimenting to see how to best inject them in the microservices.

- Passing tests: I started a list of increasingly difficult tests my ACE needs to pass. It starts from "Can you repeat this word", to writing emails, to eventually multi-steps complex problems.

Difficulties

My difficulties at the moment are:

- Memory querying: Memories are stored in a Semantic DB (Pinecone). However, I did not figure out how to properly retrieve memories in an Q&A format. ie: What is my relationship with this person? --> Should return memories like "I like this person and think X/Y".

- Memory consolidation: At the moment the thoughts are piling up in Josh's "brain". I need to find the best ways to distill & consolidate memories (merging similar one, removing unimportant ones etc.). Did not really start this yet.

- Visual processing: At the moment Josh is only capable of treating text. I faked audio understanding with a speech-to-text model, but most visual information would be lost if using image-to-text models. I could add a DB to store visual info, but I am unsure how the info would then be linked to the semantic part of the brain.

- Funding: That is a big problem for me atm. I would like to stay full-time, because I'm already struggling to keep up with say Sidney from Bing (I'm not far behind I have to say). I applied for a grant, but I was wondering if you had suggestions.

​

Here is my current (simplified) architecture (I'll need to do some refactoring/cleaning at some point):

https://preview.redd.it/s7jt286hl5ka1.png?width=984&format=png&auto=webp&v=enabled&s=077a763e27c586ca7c2a1b0c0e4be48780726119

​

Here are a couple screenshots:

https://preview.redd.it/alqqn1qil5ka1.png?width=615&format=png&auto=webp&v=enabled&s=972fd53c7565e0ef726d72fe03e055e16c91b9c6

https://preview.redd.it/ykruuv5kl5ka1.png?width=571&format=png&auto=webp&v=enabled&s=59aea14b1739280c9c72ff4c7f94c0d9ffcff2e5

https://preview.redd.it/idqkml5ll5ka1.png?width=612&format=png&auto=webp&v=enabled&s=a51eb7996eff2b743afc47e92fcb86e34c9b22cf

I'd be glad to answer to any questions, and I'm also open to suggestions!

Best,

Lester

92

Comments

You must log in or register to comment.

DungeonsAndDradis t1_j9udfs8 wrote

Neat! Good luck, and hopefully someday AGI will credit you as one of its founders.

19

nikitastaf1996 t1_j9uim0x wrote

I have seen one similar project on YouTube.Where there is two there is ten.I don't know what that will lead to.But quantity often converges to quality.

20

DamienLasseur t1_j9us2r0 wrote

This is super fascinating. I'd imagine this is a computationally expensive endeavour so I'm curious, what hardware are you using to train it? I'd love to talk further if possible.

9

AwesomeDragon97 t1_j9v3jm4 wrote

I have a few questions:

  1. How many GPUs does it take to run?

  2. Is it better or worse than ChatGPT?

  3. Will it be Open Source?

9

MrTacobeans t1_j9vzw1p wrote

Why are you building this based on a closed API?

You could eventually find something in this adventure and openAI could be like woah let's not go there and block/ruin the work you've done. There are multiple open source models that can be worked into the kind of flow you are creating.

On a side note though leveraging GPT-3 to create even a proto AGI seems incredibly unlikely. If it was possible it would likely be in the news already. You mentioned yourself the memory limit. That's a big chunk of the issue with current AI. Can't keep a "sense of mind" going when half of it is getting deleted every few prompts

1

Lesterpaintstheworld OP t1_j9w529n wrote

Replying to MrTacobeans (#1,979,216)

The engine to generate token can be changed at any moment. Actually I'm looking forward to being a able to plug it on GPT 3.5 / 4. Also it could be replaced by an open-source counterpart, I am just not aware of any at the moment.

I think no one really knows we're AGI will emerge from. But even having an agent that can be an helpful assistant, even without the "AGI" part, would be quite the success for me. Business applications are numerous

7

MrTacobeans t1_j9xi9n6 wrote

Replying to Lesterpaintstheworld (#1,980,187)

If you are paying for the API something like "RWKV" might be an alternative hosted on a GPU Cloud provider. The model is currently only at 14B parameters but technically has "unlimited context" which in theory is probably not actually unlimited but for what I saw in your use case it might be worth looking into

3

IluvBsissa t1_j9xx7dz wrote

I really don't get it. How is that proto-AGI ? Anyone ?

2

Lesterpaintstheworld OP t1_j9xyx13 wrote

I'm open to constructive criticism, especially because I'm not from a ML background. I do have an engineering degree in CS, but there will definitely be gaps in my knowledge.

3

Lesterpaintstheworld OP t1_j9y8v2e wrote

Replying to IluvBsissa (#1,992,356)

My project is an implementation of the "cognitive architecture" approach to intelligence. It postulate that what's missing to get to AGI is not only scale (OpenAI's current approach), but a layer of logic and memory. David Shapiro makes a better job than me of explaining this approach on YouTube if this is interesting for you

4