- Did you use any fine-tuning with this or is this standard GPT4?
- Does quality of response improve with any additional examples?
- Does this perform well for specific languages?
It's interesting because we came to a different conclusion with Autopilot[0] - context and learning is incredibly important for result quality, and gpt4 doesn't (yet) support fine-tuning but will soon, and we'll definitely be taking advantage of that. Not just for quality, but also for speed (less time spent gathering context and processing input tokens).
My view is, everyone has access to chatgpt and github copilot, and so the idea is to provide value in excess of what chatgpt/copilot can do. Part of that is embedding it in the UI, but (especially for internal tools, which tend to be shorter) the improvement isn't huge over copy/paste or using copilot in vs code.
However, beyond UI integration, we can intelligently pull context on related files, connected DBs/resources, SDKs you're using, and so on. And that's something chatgpt can't do (for now). The quality of response, from what we saw, dramatically improved with the right docs and examples pulled in.
And yes, gpt4 does much better on JS (React specifically) and Python. It's just whatever it's trained on, and there's a ton more JS/Python code out there.
To be able to click a button and send in script, Context and error message. I belive is a huge changes not a small change. Speeding ut the turnaround time a lot.
I will fins out the next 2 weeks at least. I hopw this will change how i program
No need to fine-tune since windmill uses normal code without custom syntax so gpt4 corpus works very well. That's one of the benefit of not using DSL, most tooling work well out-of-the-box including vscode copilot if you were to use windmill using the GitHub/vscode integration.
