MASK v2.0 – Unity!

Unity3D version of MASK!

Since it’s much more stable with App than with Webpage, maybe it’s worth a try to make a unity version of MASK collection. Hmm such a sweet dilemma!

Codes of accessing camera on the phone… omg so simple!

WebCamDevice[] devices = WebCamTexture.devices;
mCamera = new WebCamTexture ();

//for(var i=0; i<devices.Length; i++){
//	Debug.Log(devices[i].name);

if (devices.Length > 0) {
	mCamera.deviceName = devices[0].name;
	displayPlan.GetComponent<Renderer>().material.mainTexture = mCamera;
	mCamera.Play ();

MASK v2.0 – Local server(*2) network!

Screen Shot 2015-09-18 at 8.40.12 PM

Quick notes on how I made local Peer.js server, and also closed network!

When making my thesis, I used cloud hosted PeerServer with API key, so the project was constrained by internet access. In order to make the project more robust and stable, I decided to create PeerServer on my own laptop, and let devices (mobile phones) access the laptop (server) through Wi-Fi.

Here are the quick notes on how I did those (OMG such a googling / debugging hell!):


Create PeerServer accompanying another WebSocket system

Peer.js documented well about how to create your own PeerServer, even with examples of combining with existing express app, which is perfect for me. But because I have my own Node server as well, I struggled for a while to realize that, I should create two servers for each websocket with different ports!!! Finding this out through non-stop console.log and try & error!

Server codes

var express = require('express');
var app = express();
var http = require('http');
var server = http.createServer(app);
var port = process.env.PORT || 7000;

// peer_server
var ExpressPeerServer = require('peer').ExpressPeerServer;
var peerExpress = require('express');
var peerApp = peerExpress();
var peerServer = require('http').createServer(peerApp);
var options = { debug: true }
var peerPort = 9000;

app.get('*', function(req, res){
    res.sendFile(__dirname + req.url);
peerApp.use('/pp', ExpressPeerServer(peerServer, options));



Client side sudocode

var peerConnect = function() {			
	/* WebRTC - Peer.js */
	peer = new Peer( ID(), { secure: false, host: 'YOUR_IP', port: 5000, path:'/pp', debug:true} );

	// Get an ID from the PeerJS server		
	peer.on('open', function(id) {
		console.log('My peer ID is: ' + id);
		peer_id = id;

		// Now we can connect to my socket server						

	peer.on('call', function(incoming_call) {
		console.log("Got a call!");
		incoming_call.answer(my_stream); // Answer the call with an A/V stream.
		// do what you want~~~


MacbookPro as server, accessing by devices through Wi-Fi with router

  1. Connect Router with MacbookPro
  2. Click Wi-Fi symbol to expand the list –> Create Network…Screen Shot 2015-09-19 at 12.43.25 PM
  3. Setup name and passwordScreen Shot 2015-09-19 at 12.44.53 PM
  4. System Preferences… –> SharingScreen Shot 2015-09-19 at 12.46.06 PM
    • Setup computer name
    • Click Internet Sharing
    • Share your connection from: Ethernet
    • To computer using: Wi-Fi
    • Click Wi-Fi Options –> setup Wi-Fi name and passwordScreen Shot 2015-09-19 at 12.48.59 PM
    • Click on box next to Internet Sharing to startScreen Shot 2015-09-19 at 12.50.45 PM
  5. @ your device’s Wi-Fi list, choose the Wi-Fi shared by MacbookPro and enter password
  6. Voila! Now your device and MacbookPro have same IP address, thus the device and go to the page that MacbookPro serves with <IP_address>:7000/index.html.

Summer Throwback Three: We Are What We Pretend To Be Inc. established!

The third major experiment I had this summer was a street car journey. At first it’s just another experiment with Unity, but then it leads me to an idea for the project I want to do this year, as the Research Fellow at ITP.

I decided to establish an incorporation, called WE ARE WHAT WE PRETEND TO BE Inc..

It’s inspired by the quotes of writer Kurt Vonnegut:

“We are what we pretend to be, so we must be careful about what we pretend to be.”

I discovered this when I was doing research for my thesis, and it inspired me a lot to shape the idea, and to make the MASK series.

WE ARE WHAT WE PRETEND TO BE Inc.(WAWPRE) is an entity for my series experiments about the relationship between our free will and unfree (uncontrollable) will. Can I change my unfree will by change my free will? How can I shift self identity by ultering my and also others perspective?


The first product of WE ARE WHAT WE PRETEND TO BE Inc. is MASK, a customized VR headset that helps the user become what they want to be.



The second service of WE ARE WHAT WE PRETEND TO BE Inc. is THE TRAM.

We Are What We Pretend To Be

It’s an installation that provided customized virtual reality experience that users (aka passengers) can design their tram journey based on their mood, destination, favorite color, and cloth to wear. Also from the cutout hole on the cardboard box, which is usually covered with curtain, I will take a picture of the passengers, so the passengers will be able to see themselves in the VR.

WE ARE WHAT WE PRETEND TO BE Inc. THE TRAM takes you to wherever you want, fulfill whatever desire you want, therefore you become whomever you want. You set yourself free.

I joined Internet Yami-Ichi in New York this summer, and displayed my little handcraft Home Depot cardboard box as an installation + experience seller. For the software, I used Unity and started developing it on and off since August, since I was notified that I was selected to join the event. As for the box, I made it with recycled materials I found in junk shelf in one night before the event! Holy cow SO NERVOUS.

It’s really a successful experiments, for first ever user test! The users used it for a long time, average at least more than 1 minute! Which is a huge difference for me.  I think that’s because the tram keeps moving, the scenery outside the window keeps changing, plus there are ghost passengers getting on the bus when the tram stops at the tram station, the user is expecting things to happen, thus enjoys the VR experience for the longer time.

Screen captures with different customized setup


Contents development

It’s a subtraction process, because of limited time haha

You can escape into different spaces, under different circumstances.

Different customized options & graphic design

Ideas finalizing


Internet Yami-Ichi 

IMG_20150912_131813 IMG_20150912_133938 IMG_20150912_14281211997881_10207456961200394_772412524_nIMG_20150912_182150 IMG_20150912_184804


Thank you all my lovely first ever passengers<3 <3 <3

Let me thank you with this passengers all together gif <3<3<3 Thank you guys!!!



Summer Throwback Two: Big Mac Index VR

In this summer, I was interning in The Economist Media Lab. It’s a tight team of Ron Diorio, Frank J. Andrejasich, and Ziv Schneider. The goal of this internship is exploring how The Economist can use virtual reality to enhance the editorial piece, reach broader audience, and create unusual experience.

Because of my previous 3D data visualization experiments, The Economist is interested how they can do data viz in VR, since data representation is one of their forte. Luckily, I could choose what I want to create, so I picked Big Mac Index. Why? Because it is BURGER dude. BURGER.

About Big Mac Index:

One of the most famous index of The Economist. The earliest data starts from September 1986, and it has been used as a semi-humorous illustration of purchasing power parity (PPP). It makes exchange-rate theory a bit more digestible.

For this, I use the every half a year data since 2000, and the countries include: Argentina, Australia, Brazil, Britain, Canada, Chile, China, Czech Republic, Denmark, Euro area, Hong Kong, Hungary, Indonesia, Japan, Malaysia, Mexico, New Zealand, Poland, Russia, Singapore, South Africa, South Korea, Sweden, Switzerland, Taiwan, Thailand, and United States.


There are total three scenes in this Big Mac Index VR: 1) Intro, 2) Map, 3) Comparison Mode.


With bunch of graphics and little texts, the opening shows how to navigate and interact in the virtual world, and gives an intro about the Index. It’s like the traditional motion graphic voice-over video, yet it’s 3D and interactable.



Map is the major part of Big Mac Index VR. There are two kinds of data: first one is “How many Big Mac you can buy with $10 value equivalent in the currency?”, second one is: “The valuation of the currency compared with US Dollar”. For each one, I use burger tower and color of map to represent it. Throughout different time periods, the height of the burger tower and the color of the map change.


There are also two modes for how to display the data, one is automatic time passing by (default), another one is being designated. By tilting the headset 90 degree clockwise, the menu is toggled up to be chosen from. Once it’s not in auto mode, the time period appears in the sky for be chosen, and the map only shows data for specific year.



Still in progress :) It’s for clear comparison in small group with selected countries.



Summer Throwback One: Facebook Land

Originally is a birthday present to Andy S. But it ends up to be an Facebook post exploration journey. Below is the instructions:

  • Start the journey
  • Log into Facebook with token
  • Hear happy birthday wishes from Yelling Kid
  • Your latest 20 posts from the news feed show up, include: name, contents, and picture.
  • Be checked out by Andy the Candy
  • Press Space key to have confusing conversations with Andy the Candy
  • Andy the Candy starts walking
  • Follow Andy the Candy to a post
  • Press Space key to view the content
  • Andy the Candy greets you when you are back
  • Andy the Candy leads you to another post
  • (Loop)

Very fun experiments! Possible next steps: More posts you browse, the darker the environment it becomes, and eventually shut down the application :>

Just don’t fall for FB too much yo ^_<


Materials –> 3D-like 2D gif images

WTF Moment_04

Was fabricating my Duet mask,

and someone showed me this

Screen Shot 2015-04-21 at 1.27.52 PM

WTFFFFFFF. SO SIMILAR. And it’s fabricated better too!

Have to find a way to make it looks different…

\ _ /  Fighting!!

MASK_04 – Duet

Fourth of MASK collection – Duet


It’s a remedy mask for two people who hesitate to talk to each other. Separated from outside world with the mask, users sit with legs crossed, face to face, and communicate with each other through smiles. It’s for the eager of communicating without seeing each other’s face yet still feeling the intimacy. In the virtual world, users are surrounded by huge monsters, who will help the remediation process.


  • Two-people, foldable accordion origami mask
  • Portable intimate space
  • Self-made surface mounted circuit board as Muscle Sensor * 2
    • One for each users
    • For detecting the muscle on face. In this scenario, will be used to detect Zygomaticus major muscle, which is associated with smiling


Content Test


Fabrication Process

Small scale paper testing (Thank Origami master Uttam for the help!) 

Then use mylar sheet for the final material. Got an angel helper Gladys!

Test putting it on two people! Need to add paper in the bottom part to increase the distance between two users, allowing the space for sitting with legs crossed face to face comfortably



Hand recognition

  • FINALLY working…
  • Work good with handfist :) recognition, very finicky with handopen :(  recognition


  • Won’t be used in my project. Just procrastinated like crazy.
  • Issues to solve
    • Transform position from 2d to 3d
    • Recognition stability
    • Screenshot_2015-04-13-21-42-49

Read The Rest

« Older Entries Newer Entries »