AI Case Study

Uizard uses deep learning to generate lines of code from images of graphical user interfaces with 77% accuracy

Uizard, a Danish startup, aims to automate software development with RNNs for GUI development using screenshots of the interface. The code works across various platforms - . iOS and Android native mobile interfaces, and multi-platform web-based HTML/CSS interfaces.

Industry

Technology

Software And It Services

Project Overview

"Uizard's program imports paper or digital mock-ups into Sketch and generates a working prototype.

In order to program the neural network, the team had to overcome three main problems. The first being computer vision - that a computer will not automatically understand the given scene and cannot naturally identify the objects present, their positions, and characteristics (i.e. buttons and labels). Secondly, they had to overcome a language problem – the issue of teaching the network to understand text, so it could create accurate samples. Finally, the deep learning program had to be trained to understand the link between code, text and the corresponding images, so it could connect all three while generating new code."

Reported Results

According to Uizard, the platform develops code with 77% accuracy enabling faster deployment

Technology

"Pix2Code can be used to generate the code (DSL) for a user interface, and we can use Code2Pix to see how close the captions are at capturing the real image. Pix2Code would be the generator and Code2Pix would be the discriminator. Together, they could form a GAN which might be able to generate code for GUIs more accurately than Pix2Code by itself."

Function

Information Technology

Development

Background

Uizard automates GUI software development.

Benefits

Data

Computer wire frame mock-ups, GUI