Technological contribution to the Four-Company Joint Research Project
MIXI was one of four companies to participate in the Alter 3 project: a joint research effort started in February 2019 that looked to expand the possibilities for communication between humans and androids. We donated Alter 3 to the Ikegami Laboratory of the Graduate School and College of Arts and Sciences of the University of Tokyo in August 2022. The simulator our company developed continues to be used by them for research.
Summary of the Four-Company Joint Research Project
This project followed our group’s mission to make the world more vibrant through communication. The project was launched by members of the four following companies: Kimura Koki of MIXI providing the Alter 3 Simulator that was made using technology cultivated through MIXI’s communication services, world-renowned android researchers Hiroshi Ishiguro and Kohei Ogawa of Osaka University, ALife (Artificial Life) researchers Takashi Ikegami, Itsuki Doi, Atsushi Masumori, and Norihiro Maruyama of the University of Tokyo, and Takehito Masui of Warner Music Japan providing the demonstration space.
For the four-company joint project, the android made use of theoretical designs made by the Ikegami Laboratory at the University of Tokyo to maximize the realism of its expressions. It also was the first android to use the ALIFE Engine™, Alternative Machine, Inc.’s newly developed dynamic-generation engine. And starting with Scary Beauty by Keiichiro Shibuya, a musician and the originator of the android opera, the curator of the National Museum of Emerging Science and Innovation, Maholo Uchida, planned to exhibit Alter 3 around the world. In August 2021, Alter 3 had performed ina special program of the New National Theater, Tokyo featuring world-renowned conductor Kazushi Ono, writer Masahiko Shimada, and musician Keiichiro Shibuya and it showed how human-android communication and entertainment may evolve.
Project Members
Koki Kimura
Koki Kimura: The President and Representative Director of MIXI, Inc. and provider of the Alter 3 Simulator (simulates Alter 3’s movements in a virtual space). Koki Kimura worked for a variety of companies, including those in the electrical equipment and mobile content industries. He joined mixi (now MIXI, Inc.) in 2008. He oversaw operations consulting for mixi’s Games Department and worked on several communications-based games, including “Sunshine Ranch.” Afterward, he helped launch the Monster Strike Project. He was made a corporate officer in November 2014, appointed to the board of directors in June 2015, then became the director and corporate officer of sports in April 2018. In June 2018, he became the president and representative director of MIXI, Inc, and the director and corporate officer of sports. He is currently the president and representative director of MIXI as of April 2020.
Hiroshi Ishiguro
Hiroshi Ishiguro: A leading researcher on androids and doctor of engineering.
Robotics engineer, distinguished professor of the Department of Systems Innovation, Graduate School of Engineering Science, Osaka University, director of ATR Hiroshi Ishiguro Laboratories, and ATR fellow. He has conducted research and development of intelligent robots and perceptual information infrastructure, aiming to realize the base of the next generation of information and robotics. He was awarded the Osaka Culture Prize in 2011.
Takashi Ikegami
Takashi Ikegami: Physics professor and graduate of the School of Science, University of Tokyo whose research was focused on artificial life. He is also a professor of the Graduate School of Arts and Sciences, the University of Tokyo. His research focuses on complex systems, especially artificial life, and aims to construct a theory of life from the perspective of dynamics. While working as an editor for international journals including “BioSystems”, “Artificial Life”, and “Adaptive Behavior”, he has worked with Keiichiro Shibuya, evala, Kenshu Shintsubo on art activities such as filmachine (2006), Mind Time Machine (2010), and Long Good bye (2016).
Takehito Masui
Takehito Masui: Executive producer of Warner Music Japan. He provided a concert demonstration of Alter 3.
He is a music producer who produced many Japanese music albums that have topped the charts. He was also in charge of the production of this project. As the executive producer of Scary Beauty, he provided a demonstration space for the project and explored new possibilities for human-android entertainment.
Artists Participating in the Project
Keiichiro Shibuya
Keiichiro Shibuya: Director and Composer for Scary Beauty. Graduate of the Department of Music at Tokyo University of the Arts with a major in music composition. He established the music label ATAK in 2002, releasing innovative electronic musical works both domestically and internationally. His work has appeared in the music of numerous films and sound installations. In 2012, he released the world’s first humanless Vocaloid opera, titled THE END and starring Hatsune Miku with computer-generated graphics and music, at the Yamaguchi Center for Arts and Media. This piece was first publicly performed at the Théâtre du Châtelet in Paris, with ongoing demand and public performances worldwide to this day.
He has worked with artist Hiroshi Sugimoto and robot researcher Hiroshi Ishiguro at the Palais de Tokyo and has presented many collaborations with stars such as Jeremie Belingard at the Palais Garnier. He collaborated with complex systems researcher Takashi Ikegami for 15 years on the development of noise and stereophonic sound. He is currently expanding his activities from Tokyo and Paris.
What is Alter 3?
Alter 3 has a body that exposes its inner machinery and a face that displays neither age nor gender in the hopes of creating a completely new lifeform that evokes the imagination of the viewer. World-renowned names in the fields of androids and artificial lifeforms from Osaka University and the University of Tokyo came together to create Alter 2 and continued researching to get to this point. This research aimed to find if an artificial lifeform could achieve realistic movement through interaction with its environment, possibly answering the fundamental question of what life is. We hoped to speed up the research in realistic movements for Alter 3 with the development of past Alters and past research as a base. With research focused on lifelike movement technology, we aimed for a design that could be moved around the world and used stably and continuously. Unlike its previous incarnation, both of Alter 3’s eyes are cameras, and it can speak from its mouth. Along with its more human-like sensor system and expressiveness, we’ve increased how expressive the area around its mouth can be, allowing it to sing. And with improved output compared to its earlier iteration, Alter 3 can move its body with a higher degree of responsiveness and dynamic movements. Finally, its increased portability allows it to be disassembled, shipped via plane, and reassembled with ease.
Sharing Our Technical Expertise With the World
MIXI had developed the Alter 3 Simulator–software that recreates the look and design of Alter 3–allowing for virtual movement simulations that do not require Alter 3 to be there physically.
Alter 3 was one of a kind and performs in locations all around the world.
Along with allowing the simulation of movements that have not yet been performed by Alter 3, these simulations allowed us to minimize the risk of damage during tests using Alter 3.
With a highly-accurate simulator, we could run movement tests and production checks, detecting any collisions without relying on Alter 3’s physical body. We believe that this may contribute greatly both physically and temporally in the development of algorithms for the ALIFE Engine, checks for large-scale productions in which rehearsals may not be possible, and the advancement of research and planning.
The main body of Alter 3 moves based off of commands sent through the Alter 3 application (ALIFE Engine System). It was necessary for the simulator to reproduce the movements that would be made if the main body were sent the same commands.
To achieve this, we created a 3d model of Alter 3 and developed the simulator using technologies we’ve cultivated in our various businesses in the software that our company has actively used–the game engine Unity(*1).
The Trajectory of Altar 3
| March 13, 2019 | Düsseldorf, Germany Performed “Scary Beauty” for the opening of “Hi, Robot! Das Mensch Maschine Festival” Organizer: tanzhaus-nrw Location: Robert-Schumann-Saal |
|---|---|
| March 28 – May 5 | Düsseldorf, Germany Exhibited at “KÖRPERWENDE: from Nam June Paik to Hiroshi Ishiguro” Organizer: tanzhaus-nrw Location: NRW-Forum Düsseldorf |
| May 16 – August 26, 2019 | London, England Exhibited at “AI: More Than Human” Organizer: Barbican Centre Location: Barbican Centre, London |
| November 19, 2019 – March 29, 2020 | Roppongi, Tokyo Exhibited at “Future and the Arts: AI, Robotics, Cities, Life – How Humanity Will Live Tomorrow” Alter 3 was on display until the end of December 2019. It was then replaced by Alter 1 in 2020. Organizer: Mori Art Museum Location: Mori Art Museum (Roppongi Hills Mori Tower 53F) |
| January 31, 2020 | Sharjah, United Arab Emirates Performed the android opera “Scary Beauty” at “Inter-Resonance: Inter-Organics Japanese Performance and Sound Art” Organizer: Sharjah Art Foundation Location: Sharjah Performing Arts Academy |
| March 2, 2020 | Performed in NTT DOCOMO’s special video “Bakuso Orchestra” |
| March 27, 2020 | Performed at TOKYO ALIFE 2020 Ver.0 Organizer: ALIFE Lab. Location: SHIBUYA QWS (or via DOMMUNE) |
| July 24, 2020 | Performed in NHK E tv “lalala♪ Classic” |
| December 10, 2020 | Performed MUTEK Japan’s “Kairai Kagura” Organizer: MUTEK Japan Location: Shibuya Stream Hall |
| March 11 – 13, 2021 | Participatory experiment event “Kikai Ningen ni Nattemo Watashi ha Watashi?” (“Is a Machine Me Still Me?”) Organizer: Miraikan, Tokyo University Graduate School of Arts of Sciences Ikegami Laboratory Location: Miraikan |
| August 21 – 22, 2021 | Performed “Super Angels” Organizer: Agency for Cultural Affairs, Japan Arts Council, NEW NATIONAL THEATRE, TOKYO Location: New National Theatre, Tokyo |
| March 2, 2022 | Performed Android Opera “Mirror” Keichiro Shibuya Organizer: Japan Pavilion, EXPO2020 Dubai Location: EXPO2020 Dubai Jubilee Stage |
| August 1, 2022 | MIXI, Inc. donated Artificial life × android “Alter 3” to Ikegami Laboratory at the University of Tokyo, Graduate School of Arts and Sciences, Department of General Systems Studies |
What is Scary Beauty?
The android opera Scary Beauty was the first performance at the National Museum of Emerging Science and Innovation in Odaiba, Tokyo, held in July 2018 as a public program for ALIFE 2018 (The International Society for Artificial Life), a gathering of artificial life researchers from around the world. In this performance, android Alter 3 used its AI to become the conductor of a 30-human orchestra, even singing as an accompaniment. Keiichiro Shibuya managed the composition and the piano, with world-renowned researchers in the field of artificial life and androids from the University of Tokyo and Osaka University joining the project to help with project planning. For the actual performance, the sudden changes in tempo and intensity that went beyond human imagination and its accompanying expressions were all decided by Alter 3 itself. Its potential to create something new or disastrous out of these musical compositions creates a “curious, frightening sense of beauty.”
Alter 3 shows off a large number of expressions as the conductor of “Scary Beauty.” This is due particularly to Alter 3 having working eyes and double the torque in its movements, completely changing the expressions Alter 2 had as a conductor. And though Alter 2 seemed like a genderless conductor, Alter 3’s increased speed made it seem monstrous, amplifying its power and eeriness, further contrasting its beauty.
For more information, please follow the link below.
http://scarybeauty.com
