R&D

The Air the Artist Wanted to Paint - A New Way to Appreciate Painting with AI [˚Curation]

2019.03.08

[Project Overview]

 ˚Curation, planned and produced by Dentsu Lab Tokyo, employs AI to estimate the air temperature and humidity in an image, then produce an exhibition piece that offers a new way to interact with painted art. This article presents the whole story, from the details behind the work and the AI applied, to the production process itself.

[Concept Development]

 Countless landscapes have been painted around the world. But what if we could use the power of AI to really feel the space and the air depicted in such a painting?

 The project inspired by this musing set a topic to tackle: whether AI can estimate and reproduce the air temperature and humidity in a painted landscape. The project moved forward with a machine learning model designed and developed by a Dentsu Lab Tokyo technologist.

Meanwhile, development progressed from the aforementioned musing to an actual experiment while exchanging ideas with a planner and art director. This led to the idea of using the air temperature and humidity of the exhibit’s environment to produce an installation for curating the painting.

  “The place you’re in right now may contain the very same air as a famous painting.”

 The sensations we have in this spot where we stand now—the warmth or cold, the feel of the humidity—are what we get from ˚Curation, a work that uses AI to curate a painting that shares its very air with the viewer.

curation_02のコピー

[APPROACH AND FINISHED PROJECT]

This piece was exhibited at Art Fair Tokyo, which was held in March 2019.

 The score for air temperature and humidity obtained from a meteorological data API was inputted into HERA, an independently developed AI, so that the viewer can explore a painted work that provides sensations that feel natural in the surrounding air. By displaying this moment by moment in a display patterned like a frame floating in midair, the viewer can admire the piece while also experiencing the same air within the piece at that moment. In addition, by displaying the painting’s captions in a sub-monitor, we produced a way to view works with the same interface one uses to experience a regular art museum.

By creating a previously nonexistent relationship between the work and the viewer, we offered a form of curation that is impossible with human hands.

curation_説明図

curation_differenceのコピー

[ABOUT PROJECT HERA]

 HERA is an AI development project that performs machine learning on a set of over 7 million images tagged with air temperature and humidity. It then takes the inputted landscape image to estimate that location’s air temperature and humidity.

In order to develop the AI, first, outdoor photos available in the public domain were collected. The geotag data appended to these images were then cross-referenced with global meteorological data to produce a dataset connecting weather with the images. Deep learning was then applied to study the dataset and build a machine learning model that takes an inputted image to estimate the air temperature and humidity in the scenery the image depicts.

 The ˚Curation exhibit used open data made available by art museums from around the world to gather images of 500,000 paintings. We then utilized an independently developed tool to sort out landscapes by hand and gather images of landscapes painted by various artists from across the globe.

 The collected landscape data was fed into the model to make estimates about the air in the paintings. The results were turned into a database which is used to detect the most similar meteorological environment for the location for ˚Curation, thus presenting the painting in a way that may create a deeper connection with the depicted location.

Dentsu Lab Tokyo led the team that developed the project’s concept and designed and installed the AI. The project continues to develop concepts with an eye toward applications in a variety of fields, from art to advertising.

credit
Creative Director: Kazuyoshi Ochi (Dentsu Lab Tokyo)|Art Director: Naomi Okamura (Dentsu Lab Tokyo) |Creative Technologist: Ryoya Sugano (Dentsu Lab Tokyo)| Creative Technologist: Shintaro Murakami (Dentsu Lab Tokyo) | Creative Technologist: Toru Urakawa (Dentsu Lab Tokyo)| Creative Producer: Kohei Ai (Dentsu Lab Tokyo)

Dentsu Lab Tokyo