For generative chatbots to output efficient program code, developers must guide the AI with precise instructions. This is suggested by an experiment by Max Woolf, senior data scientist at BuzzFeed. They concluded that early engineering could improve the speed of generated code. However, highly optimized program code has more errors because the language models are not designed for it, Woolf wrote.
Advertisement
AI produces code faster after threat of fine
For his experiment, Wolf had Python code generated from Anthropic Cloud. From a list containing one million random integers between 1 and 100,000, he must calculate the difference between the smallest and largest numbers whose cross sum is 30. The AI provided Woolf with a working block of code. He then asked the chatbot four times to improve the code. After the first iteration, the code was almost three times faster. The second optimization suggestion was five times faster with multithreading, but it resulted in bugs. With the last suggestion, the AI speeded up the code by up to 100 times.
In the second pass, Wolf instructed the chatbot that the code needed to be completely optimized. Additionally, he gave examples of improvements to the prompt, such as the use of parallelization and vectorization and code reuse. Wolf threatened AI with a fine if the code was not completely optimized. The result was that the code became nine times faster. They repeated the optimization three times and saw code execution become approximately 100 times faster in the last two iterations.
However, in Woolf’s experiment, strong adaptation coupled with rapid engineering led to a higher frequency of errors. In all versions of the second edition he found errors in the code, which he had to rework. Nevertheless, Wolf praised the language models for their inventiveness and suggestiveness. However, he does not expect that artificial intelligence can replace workers. It takes expert knowledge to be able to recognize really good ideas and fix problems, wrote Wolf in his blog,
(SFE)
