The popularity of coding assistants as companions in software development is growing. The fact that artificial intelligence dominates programming has been known and accepted since the latest ChatGPT – but in practice questions arise not only about the quality of the code, but also about how specifically developers benefit from the use of AI in their daily tasks. The Cloudflight company surveyed more than 1,000 participants as part of its annual Cloudflight Coding Competition. The conclusion from the survey is this: AI does not help everyone and is not suitable for every task.
Advertisement
Helpful for routine tasks
target of April 2024 among the participants of the Cloudflight Coding Contest The aim of the study was to find out how the performance of programming with AI support differs from that of developers working traditionally. Of the 1,040 participants, most of whom were generally willing to use AI in their work, 46 percent said they have saved time by using coding assistants such as ChatGPT, GitHub Copilot, Gemini, Find or JetBrains AI. However, a good fifth (22 percent) complained that they have wasted more time.
According to CloudFlight, this self-assessment of developers is also reflected in the direct speed comparison of the coding competition, in which participants had to create business logic under time pressure. In the task, which was divided into seven levels, those who programmed without the help of AI completed levels 2 and 3 on average 11 to 16 minutes faster than those supported by coding assistants. Only a few participants reached the highest competition level (up to level 6) – and only without the help of AI. CloudFlight CEO Dr. Martin Andres focuses primarily on the demanding task: “AI support is currently particularly useful for routine tasks; current AI assistants, on the other hand, can rarely create more complex and comprehensive logic without errors on their own.”
Internal testing with GitHub Copilot
CloudFlights already gained comparable insights during an internal test with GitHub Copilot, in which around 70 developers participated. It became clear that the coding assistant was unsuitable for less technical roles, but most participants saved an average of 40 minutes of working time per day during the four-week test period through the use of AI. The coding assistant proved to be helpful mainly in routine tasks, such as unit and integration testing, as well as in creating documentation or database schemas. According to those responsible at CloudFlights, the AI assistants are still reaching their limits with larger code bases that require more extensive contextual knowledge – especially due to the comparatively small underlying models.
More information about the Cloudflight Coding Contest
The programming contest was originally launched in 2007 at the Johannes Kepler University Linz Cloudflight Coding Contest (not to be confused with the CCC – Chaos Computer Club; editor’s note) takes place twice a year in different cities around the world and online. According to the organizer flight of clouds It is one of Europe’s largest competitions with thousands of participants regularly. Nearly 5,000 programmers took part in the April 2024 edition in 46 locations in 14 countries as well as online, to solve algorithmic puzzles set by the organisers. In the Cloudflight Coding Competition, participants have two tracks, School and Classic, in which teams or individuals can compete in any programming language.
(Map)
