ARTICLE

DATE
February 8th,
2016
007

Loop Between Imagination And Creation

KAME KAI

Loop Between Imagination And Creation

Qosmo / コズモ


Nao Tokui
Founder / CEO of Qosmo Inc. PhD (Engineering)
He received his Ph.D. from the University of Tokyo researching artificial intelligence and human-computer interaction. He has been actively working in the field of interactive art and music based on his research expertise and insights. After pursuing his research and creative interest in several laboratories in Japan and Europe, he founded Qosmo in 2009 with his fellow researchers and artists to deploy their ideas in the real world and market.
His current vision is to build simple softwares fostering human creativity and imagination; invaluable in our fast-changing, complex world.


Toru Urakawa
Born 1988. After completing studies at the Department of Pure and Applied Mathematics at the Graduate School of Fundamental Science and Engineering, Waseda University, he graduated from the 2014 Fall Class at the School for Poetic Computation (SFPC). URAKAWA entered Qosmo Inc. while still enrolled at university, and has been involved in the production of mobile applications, and the design of stage production systems as a programmer. In recent years his manifold activities have been inspired by a particular interest in articles of daily use, and their translation into media that function in everyday life.


Mr. Tokui:
I am Tokui from Qosmo.

First of all, we are going to give you a brief introduction about the company. Qosmo (kɒzmoʊ) was founded in 2009. Mr. Urakawa and I are going to give you the introduction today.

A common question we often receive is about the company name. It was named after a term “cosmos”. Cosmos can be a name of flower, but it also means the whole universe/ the whole universe regarded as a complex and orderly system. We found the way these meanings being poles apart interesting and decided to use it as our company name. If you check the English pronunciation, it is pronounced as cozmo instead of cosmo. So the official pronunciation of our company name should be cozmo, but we still hear many people call us cosmo-san, cosmo-san.
This part-timer who worked with us in the past had kept on calling cosmo-san for the whole three years, lol.

Job description

There are three main pillars for our jobs, which are sound, interaction and data.
First, as far as data is concerned, because of my past research in AI, we now have more exposure to a creative work from AI approach. Let me show you some examples.

Sound of Honda

As you are probably aware of this already, Mr. Sawai who used to be in the team back then took charge of the sound system of Qosmo.

Honda Smile Mission / Petiteco

This is another project we worked with Mr. Sugano, which was to make a vehicle as a radio personality.
Ms. Ohrai of Dentsu Lab Tokyo designed the car. Lately, Pepper is known for having a conversation in a natural language; we mounted a similar mechanism on a vehicle. We came up with a conversational engine which enables collecting conversations from Twitter to analyze and perform a natural conversation.

N-Building.app

Let’s talk about iPhone apps next. The project was to think about the 21st Century’s “kanban (signboard)” and “street sign” together with Mr. Naoki Terada, the architect and Alex (a.k.a. Alexander Reeder) who currently runs a company called “art and program”.
The project started when Mr. Terada crated a building with a gigantic print of QR code on the wall. The mechanism was designed to be extremely simple: by scanning this QR code with a mobile phone it transfers to the website. However since it was not good enough, Qosmo came up with the AR mechanism. By reading the QR code it overlays a graphic to make the building appear as if inside the building is transparent. We came up with devices such as showing a character in it which connects you to the Twitter account of an actual tenant in the building when you tap, as well as popping balloons out of a character showing the thoughts of people around the building.
This was made back in 2010, which means we have been working on things like this for quite some time. Back then optimization was the toughest part as the machine power of iPhone CPU was not so good as today.

Salyu music

This is an iPhone app created as a music video for an artist called Salyu. We planned and made it with Mr. Shingo Ohno of Wieden+Kennedy. In a nutshell, this music video is a VJ app because an image as if you are seeing through a camera turns into a music video at a real time instead of showing a pre-set image. From the perspective of computing power, this was another tough part for us, too.

Mr. Tokui:
So these are the three pillars of our core business. And so much for the company introduction. From now on, we would like to share about our own projects and interests with you.

Mr. Urakawa

Urakawa:
Hello, I am Urakawa. Nice to meet you.
I was born in 1988 in Saitama prefecture. Since I was a junior in college, I had been a part-timer at Qosmo for four years.

My first project was of a hotel in Kyoto called “Hotel Kanra”. The project was to use about 100 LED bars in the lobby to sense human movement and conversation volume to reflect as a light output. The following project was of Boredoms back in 2011. On the stage there is an empty space in the center surrounded by six drums with Kinect installed above. It was to obtain and information of hand movement and head height, pass it to the program written by Mr. Sawai, and generate it as the sound.

Mr. Tokui:
It adds effects with the hand movement, taking out synthesizers, etc. Pretty much everything like this has been already explored with Kinect, but doing something on a large scale as such back in 2011 was quite challenging.

Mr. Urakawa:
Lastly, Sound of Honda (iPhone app). It was to sense the car movement to recreate an engine sound of Honda’s former exceptional car. I had rented a car for two straight weeks to do the coding inside the car. This, too, is a combined project of data and sound.

While working on these projects, I graduated college in 2013 and became a full-timer.

In 2014, I was given about three months for my learning and development, and participated a program called School for Poetic Computation held in the US.

Mr. Tokui:
I flatter my company being so generous letting one of its only three employees go and do whatever at his will for whole three months with pay. What a cool company! LOL

Mr. Urakawa:
I concur with you indeed.

SFPC is a development member of openFrameworks. And it is like some sort of a 10-week school/educational program held by NY-based artists such as Zachary Lieberman who is well-known with his own works. Things made with the SFPC were exhibited and sold in ICC kids program last year. I will come back to this later.

If I were to describe my interest with words, my recent interpretation has been something like “creating a new daily life from a mathematical point of view”.

Mirabeau

A collaboration project of Mr. Takamasa Aoki and Mr. Ryoichi Kurokawa. I was a high schooler when I saw it and found this hard-to-explain kind of organic movement was interesting. Of course the sound is cool, too. I had started to feel like coming up with things like that. The work was announced nearly a decade ago in 2007, but it is still up to date.

Things like this made me realize I won’t be able to realize what I really want to do without knowing how to digitalize and express physical movements with math or computer. And that’s how I ended up being here today.

8112[1]
Mr. Urakawa:
Now let’s go back to talk about the ICC exhibition we mentioned earlier, this is a project called binary card game. As you can see, it is a card game with information of black and white only. As you know, computer involves only two symbols that shows high/low voltages but it can express various information. We started the project seeking for a way to re-express this idea as a table game we have in our daily life. Anybody can create and share a rule to play this game. By holding workshop for adults and kids, we have been continuing our activity to help them learn what binary is and create games for real, too. Even though leaving all the decision-making to somebody else, anyone both young and old can come up with a game and play it. That’s how a project is designed.

Mr. Tokui:
Being an interactive company we sent our employee to learn about interaction, but contrary to our expectation he came back with something analogue. We found this rather interesting.

Mr. Urakawa:
Also, another project we are focusing on right now is apparel-related.
Take a look at these two noise images here. A pattern appears when you put them together. Two sheets of noise are created with the image information, each of which makes no sense alone. But it begins making sense when they are put together. We are working on reproducing this mechanism with clothing. However, one problem arises when we try to do it with clothing. It is the incapability of restoring the original information with a subtle gap in position. As it takes quite accuracy which is hard to realize with clothing, we have been working on various experiments and research.

Mr. Tokui:
The interesting part of this experience is the way information pops out of a sheet of paper but not on the screen, isn’t it?

Mr. Urakawa:
You are right. Creating new things with something we are familiar in our daily life such as card games and apparels. And it would be great if such things transform into something new through mathematical viewpoints, I guess.
s_IMG_1881

Mr. Tokui

Mr. Tokui:
Now let me talk about something with a keyword “generative and delegative”.
As we mentioned earlier, music is the core part of Qosmo’s business. I on the other hand cannot play any instruments at all. Why do I work in the music-related business now? It’s because my interest is originated from thinking about expression of AI and relationship between computer and human beings. I chose music as the media.
First, when it comes to creating music by computer, I was truly convinced how useless computer is. I am saying this because fundamentally computer can do nothing other than being ordered to do. It of course allows us to create music by typing with MIDI keyboard, cut and paste with a music editorial software. However it got me thinking, without being able to play any instruments at all, what could I possibly do with a computer without any musical knowledge?
In a sense, I want to “live comfortably”. Although I cannot play any instruments, I can do programming. So I started thinking if there is anyway I could program “a mechanism of entering certain input for creating a musical output”.
As a basic stance, I have an expectation to “be surprised by computer”. Earlier I said I want to “live comfortably” not because I want to automatically compose with a computer but to seek for a way to use computer for assisting my composition and growing my ideas. In other words, I would like to “gain new awareness” by using computer. Apparently, it is me who does the creation. The thing is if there is anyway I could make an output by obtaining “awareness” with the help of computer.

Generative and delegative. I explained to you these are the two major theme in my thoughts. Generative is what has been on my mind for the longest time, however it was not until recently did I realize the significance of delegative as the other keyword. By the way, the word Delegative is a coinage.

First of all, let me introduce you the work that has inspired me in terms of “Generative”. It was a CG work by Karl Sims in 1994.

Evloed Virutal Creatures

Although it seems like a creature, it was not thoroughly made with CG software. All the creator did was coming up with a simple rule. Let me show you first, I created a structure with some bones to put square boxes together, and joints connecting those bones. Generate the massive number of those randomly. Simulate the movements in a virtual physical space. This joint moves randomly, too. Because it is random they do not work smoothly of course, but some of them should work relatively better than the others. The next generation model is created by multiplying those successful samples. Repeating this process over time creates a creature-like movements as mentioned before. The same mechanism of the biological evolution. The law of the survival of the fittest. The interesting thing about this is that it creates movements Karl Sims had not expected or those not existing in today’s animate nature though looking creature-like. We felt the roman in the possible shape of biological evolution though it did not happen for real on this planet.

Sodaconstructor
This work is a JAVA app made in 1998. This has a similar mechanism as well, which was created by manually designating the joint/muscle movements. The respective muscle/joint contraction is described with sine wave, which also makes a very unique movement. By looking at it, the way it is being organic appeared somehow interesting to me. A complex event as such can be created by adding a mechanism of evolution, the law of the survival of the fittest, and alternation of generations- I got attracted to it and have come to create projects of this kind for my own.

Boids
Another work I referred to was a famous work called Boids made in 1989. It is a simulation regarding the movement of a flock of birds. The simulation accurately captures its movement of avoiding obstacles such as poles. This, too, involves a simple mechanism. While keeping a certain distance from an object near you, go along with other birds to the same direction for some reason or other, and head to the position which is a center of gravity of the flock. Such a complex movement is created with the combination of the above three rules only. This simulation was used in one of the actual Batman movie for the movement of bats in one scene. As I had occasions to observing tings like these, I have come to think of coming up with a way of incorporating this notion into music.


■SONAPHARE
This is a project I created by reflecting it. I am going to give you a brief explanation. The respective object within the space has a role of making noise and effect, and such parameter is linked to the position within the 3D space. So the sound changes by moving inside the space. A complex movement is created by simulating physical 3D space through a reciprocal action between objects. Thus the way something different pops out of my image seems interesting to me. It is to release the control a little bit instead of controlling 100%. Something like gardening. You cast seeds for your own, but you cannot completely control what grows out of them. Despite the intention of how you want it grow after planting, but you never know how it would bloom. The CG seems very primitive today, but I suppose it was up to date back then. With the mechanism, we held a collaboration with artists such as Nujabes.

As for my personal interest, I have a motivation of creating tools. Because I don’t know how to play music instruments, I would like to create supporting tool. Instead of creating an output by a computer, create one with a computer. That is a kind of tool I would like to make. I aim to create something that cannot be done solely by myself or computer.

■NHK SP NEXT WORLD
Our recent work based on this notion was NHK SP’s NEXT WORLD. The project was to create a theme song for the program aired last spring. Mr. Manabe of Rhizomatiks introduced it to me, so I assisted the system design. It is to work on evolving the music over the website, which is played during the program intermission. Speaking of evolution, music evaluation is harder in reality because it cannot be quantified objectively. So for this case, the law of the survival of the fittest model was applied to the music evolution in the form of web voting.. It has the mechanism of people visiting the website vote for it.

Delegative

Now, let me talk about a keyword called “Delegative”. In the past, I used to focus on how to obtain an output I could not imagine from a small input. But in addition, I have come to the conclusion if there is anyway I could produce an interesting output by letting go of the control of inputs that are already in hand.

■2045
There is a DJ event currently going on called 2045. It is a club event to think about the way to express DJ/VJ of the time after Big Data by using AI which I have been working on with Mr. Manage of Rhizomatiks. Generally, people who do DJ are of the selfish race, lol. Their motivation comes from the desire to have many people listen to the music they enjoy to listen. DJ involves selecting and mixing music, and this project is to see what if the selection part was excluded.
My ultimate goal is to play Back to Back with a computer. Back to Back is one of the styles where two DJ play music one after another. Like the way of jazz chase, the song I play is responded by another. I wonder if I could do that with a computer. I hope to do that with a robot eventually. Back to Back is interesting because of the surprise I receive from the other DJ with an unexpected response as well as musical discovery. I am seeking for a way to replace them with a computer.

Today, Internet is filled with musical data such as YouTube, Spotify, etc., and I feel there are way too many to choose. So I am seeking for a way to provide a new form of DJ with the software. What I do right now is letting software do the song selection. By obtaining and analyzing lots of the past DJ playlists, I let it choose the next song for the one I play. As a result of analyzing nearly 1.5 million playlists, we were able to categorize them in various clusters. Based on this cluster I combine algorisms and test to select some songs similar to the one played now, or those that are similar but with faster tempo, etc. Other than that, we distribute a dedicated app testing to see how much customers are in the groove as well as obtaining the movements on the floor by using iBeacon as an experiment. I could possibly DJ in a wise way, lol: If I could gain approval and iTunes data of the customers in advance, I would forecast the preference of today’s customer and play Beyoncé to bring them back from the bar to the floor.

As an ultimate goal, I would like to play Back to Back with AI software by using an analogue turn table by the end of the year. So the mixing part is going to be “released” as well.

Recent Experiments

As I don’t have anything I could introduce as my work, they are just the test case, but it is about applying fold neural network of so called Deep Learning. Generally, fold neural network is used for image analyses, we have challenged automatic coloring here. It is like an image of coloring based on an analysis identifying a certain shape as eye to recognize the pattern as a human face. The original image of Seven Samurai is on the left. The one on the right is the same image after being analyzed and colored. Originally it was a black and white movie, but the way it is finished gives you an idea what it would be like in color. The interesting thing about the computer is the way it gets confused about the variety of so many colors for clothing and ends up expressing it in gray for the most part. The same experiment can be done easily if you use the disclosed model.

One last thing. This project may seem like a joke, but it was a work presented at an event called eAT the other day. The title is “Scene analysis and application by the fold neural network around the reuse of ethnological image materials of the late Showa era”. It is to show some hard-core images of the classic laser disc karaoke to AI to analyze and create lyrics from them, and letting people sing the song whether he likes it or not, lol. The project was born while I was holding a conversation with Mr. Sugano of Dentsu Lab Tokyo. What you see at the upper left is the caption of the scene generated as a result of image analysis. I was not the one who created the analytical part of the image; an topical algorithm called Neural Talk is incorporated. It is to search word data obtained through an image by using this with a thesaurus, pass them to J-pop lyrics database, and obtain lyrics that has a rhyme with the searched words. This is quite an unreasonable demand, but the most amazing part of all is that it really works for people singing with it, lol.

Thus, coming from the same mechanism of generating an output from an input, we could come up with something interesting by applying the latest Deep Learning etc. in addition to a movement similar to a complex system by an evolutionary system or simple physical simulation. We will further follow this direction.

Here’s the final summary of our thoughts: Things like the earlier karaoke project, for instance, a generated word itself does not sound so funny, but what makes it funny is the way people read too much into the output they see to grow their own imaginations. When I think about how to make a system that leaves some space for imagination and fantasies, letting go of the control appears to be the key to me.

While we love overwhelming with gorgeous production and rich information, we would like to pursue an airy style like wabi sabi (aesthetic sense in Japanese art emphasising quiet simplicity and subdued refinement). We hope to continue working on systems and projects that produces a loop of imagination and creation in order to help computer and people giving good influences to one another. In this respect, Qosmo, the company name deriving from the term cosmos with two extreme meanings, convinced us again today what a great name it is indeed, lol.
Thank you.

Q&A

Question:
Originally, I have been interested in AI and things like that, and I can feel such strong attraction coming from the part where human attractiveness and room for imagination meet. The more the world matures, the better the accuracy becomes. And the more spots for creation get filled, the fewer the room becomes for human imagination/fantasy to penetrate. Please share your thoughts on this.
Mr. Tokui:
I agree with you.
The same applies to Karaoke, too. Without keeping this comfort balance by exceeding or falling below the expectation for accuracy, it becomes uninteresting, I guess. Well, the current karaoke system is way too much disappointing though, lol. Nevertheless, depending on how a creator looks at this, a new expression could be created as a result of improved degree of perfection. It is up to a designer how he/she wants to leave some room in it, etc.
In my view, this is similar to the time when openCV was launched. In the beginning we found the computerized face recognition interesting. But today, as you can see, it is something already in common that we use to create something with.
Question:
Thanks so much, the discussion went very fruitful. “Delegative” was the keyword that touched my heart the most for this occasion. The most critical part of general software is how easily it can be made. So this very much refreshed my mind. Since this concept has not been accepted in public, it made a perfect sense to me as a new keyword. Conversely, do you have a good guideline to widely promote the delegative notion as a new interface?
Mr. Tokui:
It’s very tough. So long as it is my software or work, I could keep it as my work by delegating the discretion within my own framework, but you are right. Generally speaking it is a tough part for sure. Two types of things exist: one without knowing an outcome and the other with foreseeable completed version to some extent. The way you look at it will vary depending on them.
Question:
What occasion would be the best to ask Qosmo a job consultation? While there are varieties of companies and creators out there, in what occasion do you wish to have them contact you?
Mr. Tokui:
One occasion would be when the job requires exploring some sort of experiments without knowing an outcome or a completed version. Our strength as a small team is being able to perform tests promptly to make improvements. It would be great if we could work together to experiment and create something. On another note this might sound surprising to you, but we are a group of romanticists, so we love romantic works, lol. It would be nice if we could start thinking and working together from a conceptual part.

Mr. Urakawa:
Please allow me to add more information to that. We as a company are a group of members who share the ambition of creating software and algorisms, as well as creating something new with them. We hope to receive a consultation from a customer for a work like that.

Mr. Tokui:
Also, majority of the works we deal with lately are interesting but not profitable, it would be nice if we could work on a lucrative work every once in a while, lol.
Thank you very much and we look forward to working with you soon.

s_IMG_2116

NEW POSTS

Dentsu Lab Tokyo