Submitted by jsonathan t3_106q6m9 in MachineLearning
Comments
phobos_0 t1_j3hwuqe wrote
Dude this is dope
jsonathan OP t1_j3hwx5l wrote
Right now, this is just a simple demo of what’s possible with AI-driven debugging. But I’d like to build it out so that instead of just explaining errors, Adrenaline provided a ChatGPT-style assistant that can answer questions about your error, and teach you during the debugging process.
This is open-source, so if anyone’s interested in contributing, here’s the GitHub repository: https://github.com/shobrook/adrenaline
uoftsuxalot t1_j3hz4nm wrote
Not to take anything away from this project, but it’s just an api call to gpt3 with prompt “fix this error {error}”. I thought there was some training and fine tuning, but I guess LLMs can do it all now a days
jsonathan OP t1_j3i0txg wrote
Yeah, right now it’s just a thin wrapper around GPT-3, but there’s a lot that could be done to improve it, like using static code analysis to build a better prompt or even training a more specialized model (like this).
satireplusplus t1_j3i1grq wrote
LLMs are our new overlords, it's crazy
coolcake t1_j3i1nzd wrote
amazing
ddproxy t1_j3i21my wrote
Does it come with an animated assistant in the shape of a paperclip?
cgk001 t1_j3i2vsa wrote
Limited by the 4k token max in api call?
anikinfartsnacks t1_j3i68yq wrote
whowasphones t1_j3iexz6 wrote
GoofAckYoorsElf t1_j3ilywu wrote
This is all great. The only problem is that I can't use it due to non-disclosure and IP protection of my employer. As long as I have to send code over the web, it's a no-no.
VectorSpaceModel t1_j3is0it wrote
TAs around the world are rejoicing
_swnt_ t1_j3itpq1 wrote
That's would be an actually useful paperclip 😂
naiq6236 t1_j3j0ofz wrote
This would be a game changer dude
[deleted] t1_j3j2nsf wrote
[deleted]
Accomplished-Low3305 t1_j3j6b0v wrote
It would be nice to have some metric to evaluate how good is GPT-3 solving bugs. In my experience it only works fine for simple bugs, such as using an incorrect variable.
yerop82726 t1_j3j6hp9 wrote
Nice one. Keen to see the vscode extension!
Jbonez87 t1_j3jc3a7 wrote
This is pretty cool dude!
outthemirror t1_j3jg8l7 wrote
If this was a pycharm/vscode plugin….
sublimegeek t1_j3jj9gv wrote
Shared with my discord
[deleted] t1_j3jjc7v wrote
[deleted]
ginger_beer_m t1_j3jlhaj wrote
How did you deal with incorrectness from ChatGPT?
StuntHacks t1_j3jmwtj wrote
Maybe we should make more of em
jsonathan OP t1_j3joesx wrote
I didn't. Adrenaline won’t always correctly fix your error, but it can at least give you a starting point.
jsonathan OP t1_j3jokpy wrote
What could go wrong?
ItsAllJustASickGame t1_j3jv7vh wrote
Bruuuuuh is this generally reliable? And if so, where can I get it?!
jsonathan OP t1_j3k1brg wrote
You can use it here: https://useadrenaline.com
RKHS t1_j3k2134 wrote
This is a fairly useless example. It's simply a rewording of the error. Do you have any examples that are non trivial?
troubletmill t1_j3k2i82 wrote
This is really good mate.
ItsAllJustASickGame t1_j3k3ipw wrote
Awesome thank you!
kelkulus t1_j3k8w2w wrote
Well for one, he's not using ChatGPT. GPT-3 is not the same.
Soundwave_47 t1_j3k9npf wrote
Anecdotally, it is comparable.
davidswelt t1_j3kg6vo wrote
OK, how did you evaluate it? How do you tell it's working well or not?
Eastern_Care_6369 t1_j3kipko wrote
Can you turn this into an IOS app
ksblur t1_j3kkft7 wrote
Just wait till you see what the rate for management will be after LLMs come for their jobs.
Managers are mostly people-interaction-managers, and LLMs are already 10x better at that than they are at creating novel code.
fnetma t1_j3kmesf wrote
An AI debugger would be very helpful. I actually see that as a use case...
scaredandconfussled t1_j3koss8 wrote
Don't give me hope like that.
KuzonFire11 t1_j3kx7zk wrote
u/jsonathan Shoot me a dm, I'd love to do a VSCode ext.
--algo t1_j3kxv2l wrote
How do you deal with source code hosting?
Funny_Willingness433 t1_j3l00n8 wrote
That is good.
lucidrage t1_j3l0695 wrote
They become self replicating.
danielswrath t1_j3l1fkb wrote
GPT-3 has the same problem though. ChatGPT is a successor of GPT-3, so it's not the same but it's not extremely different either.
2Punx2Furious t1_j3l26ui wrote
Even fine-tuning the prompt could get much better results. Prompt engineering is important.
2Punx2Furious t1_j3l29nx wrote
And it's not even AGI yet. The singularity is closer than a lot of people think.
Eastern_Incident4922 t1_j3l2own wrote
Can i try this
GoofAckYoorsElf t1_j3l2sls wrote
A cloud hosted GitLab with customer managed keys. We have a very detailed IP and security agreement with our cloud provider.
[deleted] t1_j3l3a9f wrote
[deleted]
[deleted] t1_j3l3f45 wrote
[deleted]
jsonathan OP t1_j3l3xm5 wrote
I haven’t used this yet, but my understanding is it can explain code and generate unit tests. It can’t explain and fix errors.
jsonathan OP t1_j3l41za wrote
Yep! Try it out here: https://useadrenaline.com/
[deleted] t1_j3l47br wrote
[deleted]
Datafuse_Analytics t1_j3l6uck wrote
This is awesome..
Mikatron3000 t1_j3l71ns wrote
Not sure if this is already there but it might be worth adding some license information here since sending closed source code over an open sourced API / model might become a no no in the future legally. I guess that would be the problem of making this an Intellij / vscode plugin
0x2113 t1_j3l7vqh wrote
That just means we'd have more paperclips. I see no downside here.
NavinF t1_j3lggbf wrote
Glum-Bookkeeper1836 t1_j3lhgfx wrote
I'm not sure if we know this for certain, but it appears to be davinci instruct 3 with a custom prompt prefix.
Glum-Bookkeeper1836 t1_j3lhkhz wrote
Paperclip stan
devinhedge t1_j3lhp0h wrote
This is cool. How do we give feedback to the training engine so that it improves over time?
ImPetarded t1_j3lo80s wrote
...it didn't end like we thought it would in the movies. There were no killer machines....there were paper clips, trillions of them....
Own-Cherry6760 t1_j3m1ifz wrote
Which model are you using from openAI ?
TrueBirch t1_j3mcg7g wrote
I don't think AGI will ever happen, but with enough task-specific applications, the difference may become academic.
EarthAdmin t1_j3mgzbo wrote
Would love this to be a VSCode plug-in! Happy to drop our OpenAI api key in there.
2Punx2Furious t1_j3mopda wrote
Yeah, I see a lot of goalpost-moving, but in the end, it depends on how you define "AGI", some people have varying definitions. I think even a language model can become AGI eventually.
iamnotlefthanded666 t1_j3muxea wrote
Why don't you think AGI will ever happen?
TrueBirch t1_j3mw92i wrote
There are some things that are incredibly hard. Imagine you work on a farm. You toss the keys to the ATV to a 17yo farmhand who's never worked for you before. You say, "Head over to field 3 and tell me if it's dry enough to plow. You can see where it is on this paper map. Radio back using this handheld." The farmhand duly drives the ATV to field 3, sees that it's muddy, picks up the radio, and says, "Sorry boss, field 3's a no-go."
We're a long way from a robotic farmhand being able to perform those skills, certainly not for a price comparable to a farm laborer.
You could definitely train an application-specific AI to monitor fields and report on their moisture levels. You could even have an algorithm that schedules all of your farm equipment based on current conditions and other factors. So it's not that AI can't revolutionize how we work, it's just that it'll be different from true AGI.
TrueBirch t1_j3mwk67 wrote
Check out this comment. Some things that we take for granted from low-wage humans are incredibly hard for computers and robots. Think about valet parking. Our society doesn't think "Oh my goodness, valet parkers are geniuses!!!" But it's really really hard to build a robot that can do what they do.
HoneyEatingPunkKid t1_j3n0ubs wrote
vscode extension pls
LetterRip t1_j3n91mt wrote
I'd do GLM-130B
> With INT4 quantization, the hardware requirements can further be reduced to a single server with 4 * RTX 3090 (24G) with almost no performance degradation.
https://github.com/THUDM/GLM-130B
I'd also look into pruning/distillation and you could probably shrink the model by about half again.
2Punx2Furious t1_j3nzumw wrote
> We're a long way from a robotic farmhand being able to perform those skills, certainly not for a price comparable to a farm laborer.
If we get AGI, we automatically get that as well, by definition. Those you listed are all currently hard problems, yes, but an AGI would be able to do them, no problem.
The issue is, will AGI ever be achieved, and if yes, when?
I think the answer to the first one is simple, the second one not as much.
The answer (in very short) is: Most likely yes, unless we go extinct first. Because we know that general intelligence is possible, so I see no reason why it shouldn't be possible to replicate artificially, and even improve it, and several, very wealthy companies are actively working on it, and the incentive to achieve it is huge.
As for the when, it's impossible to know until it happens, and even then, some people will argue about it for a while. I have my predictions, but there are lots of disagreeing opinions.
I don't know how someone even remotely interested in the field could think it will never happen for sure.
As for my prediction/opinion, I actually give it a decent chance of it happening in the next 10-20 years, with probability increasing every year until the 2040s. I would be very surprised if it doesn't happen by then, but of course, there is no way to tell.
datamakesmydickhard t1_j3o73d6 wrote
Has it really come to this
2Punx2Furious t1_j3o7hps wrote
Yes, it's been like this for a while now.
keepthepace t1_j3o7vy8 wrote
I was going to argue that employees will be able to bullshit their automated manager easily but well, it is not like humans are much better at handling it.
keepthepace t1_j3o8avv wrote
I am willing to be that 99% of the code is overprotected and no one in OpenAI would spend valuable time looking at it.
These protections mostly exist to justify some bullshit jobs within the company.
cloudedleopard42 t1_j3p7pr3 wrote
is it possible to fine tune GPT for static code analysis ? if yes...what would be the training set looks like?
GoofAckYoorsElf t1_j3po3ti wrote
Probably. I'm still getting fired if I do something like that without permission.
TradeApe t1_j3rra6b wrote
If they can automate huge chunks of super busy cargo harbors, they can automate valet parking...and they won't even need AGI for that. Hell, valet parking will likely become obsolete once full self driving is here.
People also didn't think AI will make artists obsolete...but here we are.
Think_Olive_1000 t1_j3tmuvt wrote
Why the fuck are you coding on an iPhone - if you're going to use a phone at least be android
TrueBirch t1_j3xlp2e wrote
Artists are hardly obsolete. Photoshop didn't make them obsolete and generative AI won't either. And I say that as someone who has extensively used Stable Diffusion for work and personal projects.
Regarding valets, I'm referring to the ability to toss your keys to a robot and have it drive your car. Even when true self driving cars are first produced (which always seems to be ten years away), we'll be a long way away from a robot being able to park a non-automated car. That's just one example of a task that seems really easy for humans but is shockingly hard for robots. Folding laundry is another one, which is especially relevant since I'm ignoring the fact that my dryer just finished a load.
the_night_question t1_j47coor wrote
Really cool!!!!!
LucasLeroy19 t1_j4cj9v0 wrote
My question to you is, why the name Adrenaline? How did you come up with that?
TrueBirch t1_j4jd8af wrote
A true AGI has way too many edge cases to be possible in the timeframe you describe. It's also not necessary to create AGI in order to make a lot of money from AI. You can find the specific jobs that you want to replace and create a task-specific AI to do it.
eldenrim t1_j4kmltf wrote
I'm curious how you feel about the following:
There are humans that can't do the task you outlined. Why use it as a metric for AGI? Put in other words, what about a "less intelligent" AGI, that crawls before it walks? An AGI equivalent to a human with lower IQ, or some similar measurement that correlates with not being capable of the same things as those in your example?
Second, if an A.I can do 80% of what a human can, and a human can do 10% of what an A.I can, would you still claim the system isn't an AGI? As in, if humans can do X, A.I can do X * 100 things, but there's a venn diagram with some things unique to humans and many things unique to A.I, does it not count because you can point to human examples of tasks it cannot complete?
Finally, considering a human system has to account for things irrelevant to an AGI (body homeostasis with heart rate and such, immune system, etc) and an AGI can build on code before it, what do you see as the barrier to AGI? Is it not a matter of time?
TrueBirch t1_j4kv71p wrote
I think "AGI" is a silly concept overall and never really happening. Computers are good at doing things in different ways from humans. Rather than chasing AGI, you can make a lot more of an impact by leveraging a computer's strengths and avoiding its weaknesses.
For my example, I picked an occupation with an average salary south of $30,000/year (source). I'm not saying everybody can do it, but the market puts a price on this kind of labor that suggests many people can do it. A true AGI system could replicate how a low-salary human does a job. In reality, a computerized system would use a few wireless sensors that call home instead of physically driving around looking at fields.
Similarly, consider meter readers, another low-wage job. Imagine what it would take to create a robot that could drive from house to house, get out of the car, find the power meter, gently move anything blocking it, and take a reading. Instead, utilities use smart meters that call home. It's cheaper, more reliable, and simpler.
It's beyond hard to create a true AGI system, and there are plenty of ways to make tons of money with application-specific systems.
2Punx2Furious t1_j4kyyhq wrote
True that you don't need AGI to disrupt everything. But I don't think the edge cases matter, it's not like it will be coded manually.
eldenrim t1_j4l7ilw wrote
I'm currently interested in ML to alleviate the suffering of my disabled partner and myself, I just enjoy theoretical discussion with AGI.
Maybe making money will come later. :)
TrueBirch t1_j4lb7fg wrote
>I don't think the edge cases matter
Being able to handle those weird edge cases is what distinguishes AGI from the kinds of AI that companies are currently developing...
2Punx2Furious t1_j4lbgjj wrote
Yes, I'm saying the fact that there are edge cases doesn't matter, because it's not us who have to address them. As we get closer and closer to AGI, it will get better at handling them, we won't have to find them, and code solutions for them. I think it will be an emergent quality of AGI.
TrueBirch t1_j4ldkum wrote
I'm talking about where the funding is going. Anything remotely approaching AGI would require billions and billions of dollars of funding.
eldenrim t1_j4lfc8z wrote
So you don't think that repeatedly making narrow AI, and then at some point bundling them together, is a valid way to get to AGI?
TrueBirch t1_j4qdbf2 wrote
It'll be something entirely new, but not capable of doing everything that my toddler can do. Systems will be designed to avoid those weaknesses. Again, think about replacing meter readers with cheap sensors instead of expensive robots.
aminostfx t1_j51kpiw wrote
This is awesome. But i have an idea to make it even better. What if we train a RL agent to write code without errors and actually make sure there is no bugs in the code. The environment used to train the RL would be the compiler. We can start first with support python only and supporting other languages later on. DM if you’re interested to colab on this project.
jsonathan OP t1_j3hwo11 wrote
Try it out here: https://useadrenaline.com