Some time ago, I discovered an interesting study carried out by researchers from the University of Quebec on the security of the code generated by ChatGPT, the language model developed by OpenAI. You might wonder what they found? Well, hang in there, because the results are amazing!
The researchers asked ChatGPT to generate 21 programs and scripts in different languages, and only five of them were secure on the first attempt. After insisting that ChatGPT fix its own mistakes, they managed to get seven more secure codes.
Part of the problem seems to stem from the fact that ChatGPT does not take into account an “ adversarial ” code execution model. In other words, it does not consider that the code it generates could be used for malicious purposes. Additionally, ChatGPT refuses to create offensive code, but will gladly create vulnerable code, which the authors consider an ethical inconsistency.
The researchers also found that one of ChatGPT’s “answers” as a silver bullet to security concerns was to only have valid inputs, which isn’t really realistic in the real world. Also, the pattern never provides helpful hints to make code more secure unless specifically instructed to remedy problems. And to ask him that, you have to know what to ask him exactly, which means that you yourself must be familiar with the language and the vulnerabilities, in advance.
Concluding their study, the researchers believe that ChatGPT, in its current form, poses a risk and that AI falls victim to Dunning Krüger syndrome. Students and developers should be aware that code generated with this type of tool may be insecure. Additionally, the pattern’s behavior is unpredictable, as it can generate secure code in one language and vulnerable code in another.
In short, if you use ChatGPT or another similar tool (Github Copilot…etc) to generate code, keep in mind that you should not take at face value that the code provided is secure. Be vigilant and make sure to check and test the code for any vulnerabilities. And don’t forget, as Gaston Lagaffe says: ” Safety first!”
