AI Case Study

Researchers at UC Merced train algorithm to create ground-level images from satellite pictures outperforming conventional interpolation method used for classifying land

Xueqing Deng and colleagues at the University of California, Merced are using a generative adversarial network consisting of two neural networks, the generator and the discriminator, to create images of the ground from satellite pictures. This is useful for geographers who have to classify land according to its use. The algorithm has been trained to associate ground-level images with their overhead view. The discriminator evaluates the images the generator creates and its feedback is used to improve the images produced. The system was tested with 4,000 overhead images and their corresponding ground truth images. In comparison to the current method used by geographers which is correct in 65% of cases, this technique could correctly determine land use 73% of the time.

Industry

Industrials

Construction And Engineering

Project Overview

Xueqing Deng and colleagues at the University of California, Merced "have trained a machine-learning algorithm to create ground-level images simply by looking at satellite pictures from above.

The technique is based on a form of machine intelligence known as a generative adversarial network. This consists of two neural networks called a generator and a discriminator.

Deng and co trained the discriminator using real images of the ground as well as satellite images of that location. So it learns how to associate a ground-level image with its overhead view.
The team then trained the discriminator with 16,000 pairs of overhead and ground-level images.

The next step was to start generating ground-level images. The generator was fed a set of 4,000 satellite images of specific locations and had to create ground-level views for each, using feedback from the discriminator. The team tested the system with 4,000 overhead images and compared them with the ground truth images.

The results make for interesting reading. The network produces images that are plausible given the overhead image, if relatively low in quality. The generated images capture basic qualities of the ground, such as whether it shows a road, whether the land is rural or urban, and so on. “The generated ground-level images looked natural although, as expected, they lacked the details of real images,” say Deng and co.

Now Deng and co’s generative adversarial networks provide an entirely new way to determine land use. When geographers want to know the ground-level view at any location, they can simply create the view with the neural network based on a satellite image.

Reported Results

When comparing the conventional method of interpolation with the new technique, the latter "turns out to correctly determine land use 73 percent of the time, while the interpolation method is correct in just 65 percent of cases."

Technology

"The technique is based on a form of machine intelligence known as a generative adversarial network. This consists of two neural networks called a generator and a discriminator.

The generator creates images that the discriminator assesses against some learned criteria, such as how closely they resemble giraffes. By using the the output from the discriminator, the generator gradually learns to produce images that look like giraffes.

In this case, Deng and co trained the discriminator using real images of the ground as well as satellite images of that location. So it learns how to associate a ground-level image with its overhead view."

Function

R And D

Core Research And Development

Background

With photography and drones it is currently possible to capture high quality images of land from the sky. "Researchers are working on the inverse problem: given a satellite image of Earth’s surface, what does that area look like from the ground? How clear can such an artificial image be?

One important task for geographers is to classify land according to its use, such as whether it is rural or urban. Ground-level images are essential for this. However, existing databases tend to be sparse, particularly in rural locations, so geographers have to interpolate between the images, a process that is little better than guessing."

Benefits

Data

"The team use as ground truth the LCM2015 ground-cover map, which gives the class of land at a one-kilometer resolution for the entire UK. However, the team limits the data to a 71x71-kilometer grid that includes London and surrounding countryside. For each location in this grid they downloaded a ground-level view from an online database called Geograph. The team then trained the discriminator with 16,000 pairs of overhead and ground-level images."