AI Case Study
Uizard uses deep learning to generate lines of code from images of graphical user interfaces with 77% accuracy
Uizard, a Danish startup, aims to automate software development with RNNs for GUI development using screenshots of the interface. The code works across various platforms - . iOS and Android native mobile interfaces, and multi-platform web-based HTML/CSS interfaces.
Industry
Technology
Software And It Services
Project Overview
"Uizard's program imports paper or digital mock-ups into Sketch and generates a working prototype.
In order to program the neural network, the team had to overcome three main problems. The first being computer vision - that a computer will not automatically understand the given scene and cannot naturally identify the objects present, their positions, and characteristics (i.e. buttons and labels). Secondly, they had to overcome a language problem – the issue of teaching the network to understand text, so it could create accurate samples. Finally, the deep learning program had to be trained to understand the link between code, text and the corresponding images, so it could connect all three while generating new code."
Reported Results
According to Uizard, the platform develops code with 77% accuracy enabling faster deployment
Technology
"Pix2Code can be used to generate the code (DSL) for a user interface, and we can use Code2Pix to see how close the captions are at capturing the real image. Pix2Code would be the generator and Code2Pix would be the discriminator. Together, they could form a GAN which might be able to generate code for GUIs more accurately than Pix2Code by itself."
Function
Information Technology
Development
Background
Uizard automates GUI software development.
Benefits
Data
Computer wire frame mock-ups, GUI