A search engine for the planet? Try the demo

Play ball! GeoVisual Search finds baseball stadiums. (Image: Descartes Labs)

Where are all the windmills on Earth? Or oil derricks? How about baseball stadiums?

You could scan through the millions of satellite images snapped by hundreds of satellites now circling the planet. Or you could try Descartes Labs’ demo search engine.

Satellites are snapping images of the Earth every day. Alongside Planet Inc. and DigitalGlobe satellites, constellations are planned from companies such as Urthecast and Astro Digital (the latter launched its first pair of satellites in July). But how do we make use of all of that data in an organized, searchable way?

New Mexico startup Descartes Labs has created a cloud-based supercomputing platform to apply machine intelligence to massive data sets, using satellite imagery to model complex systems on the planet.

While Descartes started by focusing on forestry and agriculture, its new tool Geovisual Search allows users to find similar-looking objects of any kind all over the globe. Just click anywhere on the map and a red tile appears, enabling users to search for similar objects. “To do this, we use deep learning, a form of artificial intelligence that is loosely inspired by the structure of the brain,” Descartes Labs explains.

“Last year, a team at Carnegie Mellon University applied the principles of visual search to seven cities around the world in a demo called Terrapattern. We were impressed with their work and wondered: could we do this not just for a few cities, but for the entire globe?”

Terrapattern was designed as a prototype for scanning geographical areas for specific visual features. Its focus is on helping people identify, characterize and track indicators that have not been detected or measured previously, and which have sociological, humanitarian, scientific or cultural significance. So far, it focuses only on specific cities: Pittsburgh, San Francisco, New York City, Detroit, Berlin, Miami and Austin.

Terrapattern locates cul-de-sacs in Pittsburgh. (Image: Terrapattern)

Inspired by Terrapattern, Descartes goes farther. The company has built three demo maps on three different scales.

• The continental United States — This map uses aerial imagery at 1-meter per pixel from the U.S. National Agriculture Imagery Program (NAIP). The high-resolution imagery enables detection of smaller items such as orchards.
• China — This map uses satellite imagery at 4-meter resolution from Planet. Though the resolution isn’t as high as the NAIP map over the U.S., Planet’s satellites will soon be providing daily pictures of the globe. In this map, you’ll be able to find solar farms and stadiums.
• The entire world — This map uses Landsat 8 and is at 15-meter resolution. Though much coarser than the other maps, you’ll be able to find larger scale objects such pivot irrigation and suburbs.

Every time you click on a tile, GeoVisual Search looks over the entire map for visually similar tiles. At this point, GeoVisual Search isn’t trying to get an accurate count of objects such as windmills. Instead, a search will return the top results, up to 1,000.

However, Descartes Labs’ research on teaching the computer visual patterns is an important step on the road to counting objects accurately, the company said in a blog announcing the search engine.

“We use a type of artificial intelligence called deep learning, which is loosely inspired by neurons and the structure of the brain. For every tile on the map we run it through a deep learning algorithm that creates a fingerprint for that tile. Basically, you can think of it as abstracting some of the qualities of that tile in a way that allows the computer to begin representing the image like a human does: with colors, edges, and other features of the image. When you click on something, we compare every other image to that fingerprint and try to return the ones that look like each other.

“Our research will start to focus on object detection at scale: how do we look for wind turbines, derricks, oil tanks, buildings and other important objects all over the planet. For these objects, we’ll use the underlying principles of visual similarity to teach the computer what a wind turbine looks like in all of its forms and then try to do an accurate count of all the turbines globally. Obviously this is a very difficult task, but we think we’ve got the science to tackle this problem.

“Once we’ve counted objects, we can start looking at maps through time and see what changes — how many new wind turbines are there and where are they, for example.”

Descartes is inviting geospatial developers to take part in the search engine’s development. “If you have ideas about what you’d like to do with GeoVisual Search today and have a team of developers who are experts at machine learning and/or geospatial data, drop us a line for early access to our underlying platform.”

Descartes is evolving the demo, so a release date hasn’t yet been set. Read about the tech behind the demo.