The current codex-davinci model from openAI still blows me away.

I basically asked it nicely to write me a vscode plugin that takes the selected text, prompts the user for the instructions and sends it off to the edit-api endpoint and replaces the text with the response. Including the changes to the package json needed to expose the setting where you put the api key and and a prompt if the key setting is empty to fill the setting.

All that in around seven prompts and in only 2 of them I had to make some changes as it fucked up a bracket in one and one where it forgot to read the apikey setting first before checking it.

It's not perfect, you still need to be able to code to check for errors, but it's already more helpful than some of my colleagues.