Bad news for developers? This AI is getting very good at writing code


Deciding which programming language to learn is a big question for developers today because of the huge time investment it takes. But this question could become redundant in a future where artificial intelligence (AI) models do all the heavy lifting of understanding the description of a problem and coding a solution.

Researchers from Google’s AI-focused DeepMind unit say its AlphaCode system can express solutions to code problems that score mid-level in programming contests held by new programmers. These competitions require humans to understand a problem described in natural language and then code an algorithm effectively.

in a new non-peer-reviewed article, DeepMind researchers detail how AlphaCode achieved an average ranking of 54.3% as the top entrants in 10 previously held programming contests with over 5,000 entrants. The contests were hosted on the Codeforces code contest platform.

SEE: Employers desperately need data scientists as demand soars

DeepMind claims that AlphaCode is the first AI code generation system that performs at a competitive level in code competitions for human developers. The research could improve programmer productivity and help non-programmers express a solution without knowing how to write code.

Human competitors and therefore AlphaCode had to analyze the description of a challenge or puzzle and quickly write a program to solve them. It’s harder than training a model using GitHub data to solve a simple coding challenge.

Like humans, AlphaCode needed to understand a multi-paragraph natural language description of the problem, background narrative details, and a description of the desired solution in terms of input and output.

To solve the problem, the competitor would have to create an algorithm and then implement it efficiently, which might require selecting, for example, a faster programming language like C++ over Python to overcome these constraints.

AlphaCode’s pre-training dataset included 715 GB of code pulled from files pulled from GitHub repositories written in C++, C#, Go, Java, JavaScript/TypeScript, Lua, Python, PHP, Ruby, Rust, and Scala. The team refined the model using datasets of competitive programming problems extracted from Codeforces and similar datasets.

The boost DeepMind gave AlphaCode was achieved by combining large-scale transformer models. Examples of these include OpenAI’s GPT-3 and Google’s BERT language model. DeepMind used transformer-based language models to generate code and then filter the output to a small set of “promising programs” that were submitted for evaluation.

“At evaluation time, we create a massive amount of C++ and Python programs for each problem, orders of magnitude larger than previous work,” The AlphaCode team at DeepMind explains in a blog post.

“Then we filter, group and re-rank these solutions into a small set of 10 candidate programs which we submit for external evaluation. This automated system replaces the competitor’s trial and error process of debugging, compiling, passing testing and finally submission.”

DeepMind shows how AlphaCode codes a solution to a given problem here.

DeepMind considers a few potential drawbacks to what it is trying to achieve. For example, the models can generate code with exploitable weaknesses, including “unintentional vulnerabilities originating from outdated code or intentional vulnerabilities injected by malicious actors into the training set”.

SEE: Web developer or CTO, which are the tech professions whose salaries are increasing the fastest?

There are also environmental costs. Training the model took “hundreds of petaFLOPS days” in Google’s data centers. But generating longer-term AI code “could lead to systems that can write themselves and improve recursively, rapidly leading to increasingly advanced systems.”

There is a risk that automation will reduce developer demand, but DeepMind points out the limitations of current code completion tools that greatly improve programming productivity, but until recently they were limited to suggestions from a single line and limited to certain languages ​​or short code snippets.

However, DeepMind stresses that its work is far from being a threat to human programmers, but that its systems must be able to develop problem-solving capabilities to help humanity.

“Our exploration of code generation leaves a lot of room for improvement and hints at even more exciting ideas that could help programmers improve their productivity and open the door to people who don’t currently write code,” say the authors. DeepMind researchers.


About Author

Comments are closed.