VIRTUAL, AUGMENTED AND MOBILE KYOTO
MIT Architecture 4.570 | 02/2018 |
Instructor: Prof. Takehiko Nagakura, Dr. Daniel Tsai, Prof. Howard Burns
Team member: Min Guo
Project Overview
This project is a revised sequel to the five previous Digital Heritage subjects, in which we conducted fieldwork using onsite digital recording and 3D captures and developed digital representation prototypes, while studying historic architectural places and urban designs.
Kyoto Impression with paintings
Utagawa Hiroshige was a Japanese ukiyo-e artist, considered the last great master of that tradition. His style is more people-friendly, beautiful, and has the style of sketches.
We started with the paintings of Hiroshige, specifically look at the paintings of Kiyomizu Temple , which is an standing out Buddhist temple in eastern Kyoto. We also used paintings of other artists as a supplement. Throughout the ages, many artists have painted Kiyomizu Temple in their eyes with brushes.
From 2D to 3D Experience
We hope that by using VR technology, the original plane paintings can be transformed into a three-dimensional, immersive experience.
Photogrammetric Model
Two technologies are highlighted in our project, photogrammetry and panoramic video. Photogrammetry is the science of making measurements from photographs (Lowe,1999; Zisserman, 2001). One application of this process is computation of a 3D representation from an appropriate set of photographs of an object. Contemporary photogrammetric modeling software uses photographs to create both 3D form and texture applied on its surfaces.
In the process of the research of Kiyomizu Temple, we used photogrammetry to scan some typical Japanese style buildings, accessories, landscape sketches and patterns, and input the final results into the computer for calculation. Through recap photo and photoscan these two software, we restored the scanned photos from different angles to 3D models with high-definition texture. Typically, the bronze sculpture at the entrance of the Temple is a bronze cast dragon, which is placed on the top of the sink.
One benefit of Photogrammetry is that it can use resources from internet.
This example is the Longmen Grottoes generated by photos downloaded from different websites.
Panoramic Video
Panoramic video can record a 360 degree spherical or cylindrical view for each video frame. While the video is running or when paused, the viewer can freely look up, down, and completely around, as well as close-up and away. It can play interactively within a desktop viewer like YouTube, on a head mounted equipment such as the Oculus, or through a special panoramic projection system to get a more immersive experience.
Test on Harvard Science Center Plaza
Onsite Record
Site Investigation
Our site visit tour is starting form Annex to Museum of Kyoto and end in Kiyomizu Temple. The total distance is about 4 kilometers, along which we visited several places like Soiree, Francois and Times. Above is the map that shows the route of our site investigation.
a. Start at Annex to Museum of Kyoto (aka Bunpaku, and former Bank of Japan): Free admission
b. Super-block between San-jo (third street) and Shi-jo (fourth street)
c. Hanamikoji alley in Gion
d. sloped Promenade of Kiyomizu Temple
e. Kiyomizu Temple: Find the viewpoint of the trade-mark photo angle of this temple on deck
A New Narrative
After restoring the 3D model and events on the site, we tried to create more new experiences with the materials we found. Through Hiroshige's painting, we find that the tea house and the river in the painting are elements that do not exist in today's environment. Therefore, we combined the virtual scene in the painting with the real scene to reflect the design theme of the project: transforming 2D artwork into 3D immersion experience.
Create Immersive Experience
According to the new narrative line, we used Unity3D to create a series of roaming and interactive scenes, and try to restore the landscape of Kiyomizu Temple in these scenes. We chose HTC Vivek as the main VR device and set up three different interaction modes in the whole scene.
1. Fixed-route roaming, users observe and experience scenes by wearing VR helmet
2. Play 360 videos when triggered, users trigger events at specific locations, and access 360 videos
3 Using HTC handles to jump (Teleport) and interact with specific objects
Recreating Unique Kiyomizu Experience
The experience of 2D to 3D is the theme of the whole design, so we set several key spots in the scene. At these key spots, we can see the scenes from the same angle as the artists’ paintings. Users can make their own experience journey of Kiyomizu Temple by observing and choosing these scenes.
We hope that artistic creation, data technology and human emotional experience can be integrated into one, so as to make design more vitality. Combining new technology and experience mode, physical space with virtual space, we could add new elements in the past tour routes. In this way we could achieve the purpose of history education. On the other hand, we could also use this experience itself as a game, thus the site will become the actual place of the game to increase the fun of the tour.
