VR

Making a VR app in three.js is pretty simple. You basically just have to tell three.js you want to use WebXR. If you think about it a few things about WebXR should be clear. Which way the camera is pointing is supplied by the VR system itself since the user turns their head to choose a direction to look. Similarly the field of view and aspect will be supplied by the VR system since each system has a different field of view and display aspect.

Let's take an example from the article on making a responsive webpage and make it support VR.

Before we get started you're going to need a VR capable device like an Android smartphone, Google Daydream, Oculus Go, Oculus Rift, Vive, Samsung Gear VR., an iPhone with a WebXR browser.

Next, if you are running locally you need to run a simple web server like is covered in the article on setting up.

If the device you are using to view VR is not the same computer you're running on you need to serve your webpage via https or else the browser will not allow using the WebXR API. The server mentioned in the article on setting up called Servez has an option to use https. Check it and start the server.

The note the URLs. You need the one that is your computer's local ipaddress. It will usually start with 192, 172 or 10. Type that full address, including the https:// part into your VR device's browser. Note: Your computer and your VR device need to be on the same local network or WiFi and you probably need to be on a home network. note: Many cafes are setup to disallow this kind of machine to machine connection.

You'll be greeted with an error something like the one below. Click "advanced" and then click proceed.

Now you can run your examples.

If you're really going to do WebXR development another thing you should learn about is remote debugging so that you can see console warnings, errors, and of course actually debug your code.

If you just want to see the code work below you can just run the code from this site.

The first thing we need to do is include the VR support after including three.js

import * as THREE from '/build/three.module.js';
+import {VRButton} from '/examples/jsm/webxr/VRButton.js';

Then we need to enable three.js's WebXR support and add its VR button to our page

function main() {
  const canvas = document.querySelector('#c');
  const renderer = new THREE.WebGLRenderer({canvas});
+  renderer.xr.enabled = true;
+  document.body.appendChild(VRButton.createButton(renderer));

We need to let three.js run our render loop. Until now we have used a requestAnimationFrame loop but to support VR we need to let three.js handle our render loop for us. We can do that by calling WebGLRenderer.setAnimationLoop and passing a function to call for the loop.

function render(time) {
  time *= 0.001;

  if (resizeRendererToDisplaySize(renderer)) {
    const canvas = renderer.domElement;
    camera.aspect = canvas.clientWidth / canvas.clientHeight;
    camera.updateProjectionMatrix();
  }

  cubes.forEach((cube, ndx) => {
    const speed = 1 + ndx * .1;
    const rot = time * speed;
    cube.rotation.x = rot;
    cube.rotation.y = rot;
  });

  renderer.render(scene, camera);

-  requestAnimationFrame(render);
}

-requestAnimationFrame(render);
+renderer.setAnimationLoop(render);

There is one more detail. We should probably set a camera height that's kind of average for a standing user.

const camera = new THREE.PerspectiveCamera(fov, aspect, near, far);
+camera.position.set(0, 1.6, 0);

and move the cubes up to be in front of the camera

const cube = new THREE.Mesh(geometry, material);
scene.add(cube);

cube.position.x = x;
+cube.position.y = 1.6;
+cube.position.z = -2;

We set them to z = -2 since the camera will now be at z = 0 and camera defaults to looking down the -z axis.

This brings up an extremely important point. Units in VR are in meters. In other words One Unit = One Meter. This means the camera is 1.6 meters above 0. The cube's centers are 2 meters in front of the camera. Each cube is 1x1x1 meter large. This is important because VR needs to adjust things to the user in the real world. That means we need the units used in three.js to match the user's own movements.

And with that we should get 3 spinning cubes in front of the camera with a button to enter VR.

click here to open in a separate window

I find that VR works better if we have something surrounding the camera like room for reference so let's add a simple grid cubemap like we covered in the article on backgrounds. We'll just use the same grid texture for each side of the cube which will give as a grid room.

const scene = new THREE.Scene();
+{
+  const loader = new THREE.CubeTextureLoader();
+  const texture = loader.load([
+    'resources/images/grid-1024.png',
+    'resources/images/grid-1024.png',
+    'resources/images/grid-1024.png',
+    'resources/images/grid-1024.png',
+    'resources/images/grid-1024.png',
+    'resources/images/grid-1024.png',
+  ]);
+  scene.background = texture;
+}

That's better.

click here to open in a separate window

Note: To actually see VR you will need a WebXR compatible device. I believe most Android Phones can support WebXR using Chrome or Firefox. For iOS you might be able to use this WebXR App though in general WebXR support on iOS is unsupported as of May 2019.

To use WebXR on Android or iPhone you'll need a VR Headset for phones. You can get them for anywhere from $5 for one made of cardboard to $100. Unfortunately I don't know which ones to recommend. I've purchased 6 of them over the years and they are all of varying quality. I've never paid more than about $25.

Just to mention some of the issues

  1. Do they fit your phone

    Phones come in a variety of sizes and so the VR headsets need to match. Many headsets claim to match a large variety of sizes. My experience is the more sizes they match the worse they actually are since instead of being designed for a specific size they have to make compromises to match more sizes. Unfortunately multi-size headsets are the most common type.

  2. Can they focus for your face

    Some devices have more adjustments than others. Generally there are at most 2 adjustments. How far the lenses are from your eyes and how far apart the lenses are.

  3. Are they too reflective

    Many headsets of a cone of plastic from your eye to the phone. If that plastic is shinny or reflective then it will act like a mirror reflecting the screen and be very distracting.

    Few if any of the reviews seem to cover this issue.

  4. Are the comfortable on your face.

    Most of the devices rest on your nose like a pair of glasses. That can hurt after a few minutes. Some have straps that go around your head. Others have a 3rd strap that goes over your head. These may or may not help keep the device at the right place.

    It turns out for most (all?) devices, you eyes need to be centered with the lenses. If the lenses are slightly above or below your eyes the image gets out of focus. This can be very frustrating as things might start in focus but 45-60 seconds later the device has shifted up or down 1 millimeter and you suddenly realize you've been struggling to focus on a blurry image.

  5. Can they support your glasses.

    If you wear eye glasses then you'll need to read the reviews to see if a particular headset works well with eye glasses.

I really can't make any recommendations unfortunately. Google has some cheap recommendations made from cardboard some of them as low as $5 so maybe start there and if you enjoy it then consider upgrading. $5 is like the price of 1 coffee so seriously, give it try!

There are also 3 basic types of devices.

  1. 3 degrees of freedom (3dof), no input device

    This is generally the phone style although sometimes you can buy a 3rd party input device. The 3 degrees of freedom mean you can look up/down (1), left/right(2) and you can tilt your head left and right (3).

  2. 3 degrees of freedom (3dof) with 1 input device (3dof)

    This is basically Google Daydream and Oculus GO

    These also allow 3 degrees of freedom and include a small controller that acts like a laser pointer inside VR. The laser pointer also only has 3 degrees of freedom. The system can tell which way the input device is pointing but it can not tell where the device is.

  3. 6 degrees of freedom (6dof) with input devices (6dof)

    These are the real deal haha. 6 degrees of freedom means not only do these device know which way you are looking but they also know where your head actually is. That means if you move from left to right or forward and back or stand up / sit down the devices can register this and everything in VR moves accordingly. It's spookily and amazingly real feeling. With a good demo you'll be blown away or at least I was and still am.

    Further these devices usually include 2 controllers, one for each hand and the system can tell exactly where your hands are and which way they are oriented and so you can manipulate things in VR by just reaching out, touching, pushing, twisting, etc...

    6 degree of freedom devices include the Vive and Vive Pro, the Oculus Rift and Quest, and I believe all of the Windows MR devices.

With all that covered I don't for sure know which devices will work with WebXR. I'm 99% sure that most Android phones will work when running Chrome. You may need to turn on WebXR support in about:flags. I also know Google Daydream will also work and similarly you need to enable WebXR support in about:flags. Oculus Rift, Vive, and Vive Pro will work via Chrome or Firefox. I'm less sure about Oculus Go and Oculus Quest as both of them use custom OSes but according to the internet they both appear to work.

Okay, after that long detour about VR Devices and WebXR there's some things to cover

  • Supporting both VR and Non-VR

    AFAICT, at least as of r112, there is no easy way to support both VR and non-VR modes with three.js. Ideally if not in VR mode you'd be able to control the camera using whatever means you want, for example the OrbitControls, and you'd get some kind of event when switching into and out of VR mode so that you could turn the controls on/off.

If three.js adds some support to do both I'll try to update this article. Until then you might need 2 versions of your site OR pass in a flag in the URL, something like

https://mysite.com/mycooldemo?allowvr=true

Then we could add some links in to switch modes

<body>
  <canvas id="c"></canvas>
+  <div class="mode">
+    <a href="?allowvr=true" id="vr">Allow VR</a>
+    <a href="?" id="nonvr">Use Non-VR Mode</a>
+  </div>
</body>

and some CSS to position them

body {
    margin: 0;
}
#c {
    width: 100%;
    height: 100%;
    display: block;
}
+.mode {
+  position: absolute;
+  right: 1em;
+  top: 1em;
+}

in your code you could use that parameter like this

function main() {
  const canvas = document.querySelector('#c');
  const renderer = new THREE.WebGLRenderer({canvas});
-  renderer.xr.enabled = true;
-  document.body.appendChild(VRButton.createButton(renderer));

  const fov = 75;
  const aspect = 2;  // the canvas default
  const near = 0.1;
  const far = 5;
  const camera = new THREE.PerspectiveCamera(fov, aspect, near, far);
  camera.position.set(0, 1.6, 0);

+  const params = (new URL(document.location)).searchParams;
+  const allowvr = params.get('allowvr') === 'true';
+  if (allowvr) {
+    renderer.xr.enabled = true;
+    document.body.appendChild(VRButton.createButton(renderer));
+    document.querySelector('#vr').style.display = 'none';
+  } else {
+    // no VR, add some controls
+    const controls = new OrbitControls(camera, canvas);
+    controls.target.set(0, 1.6, -2);
+    controls.update();
+    document.querySelector('#nonvr').style.display = 'none';
+  }

Whether that's good or bad I don't know. I have a feeling the differences between what's needed for VR and what's needed for non-VR are often very different so for all but the most simple things maybe 2 separate pages are better? You'll have to decide.

Note for various reasons this will not work in the live editor on this site so if you want to check it out click here. It should start in non-VR mode and you can use the mouse or fingers to move the camera. Clicking "Allow VR" should switch to allow VR mode and you should be able to click "Enter VR" if you're on a VR device.

  • Deciding on the level of VR support

    Above we covered 3 types of VR devices.

    • 3DOF no input
    • 3DOF + 3DOF input
    • 6DOF + 6DOF input

    You need to decide how much effort you're willing to put in to support each type of device.

    For example the simplest device has no input. The best you can generally do is make it so there are some buttons or objects in the user's view and if the user aligns some marker in the center of the display on those objects for 1/2 a second or so then that button is clicked. A common UX is to display a small timer that will appear over the object indicating if you keep the marker there for a moment the object/button will be selected.

    Since there is no other input that's about the best you can do

    The next level up you have one 3DOF input device. Generally it can point at things and the user has at least 2 buttons. The Daydream also has a touchpad which provides normal touch inputs.

    In any case if a user has this type of device it's far more comfortable for the user to by able to point at things with their controller than it is to make them do it with their head by looking at things.

    A similar level to that might be 3DOF or 6DOF device with a game console controller. You'll have to decide what to do here. I suspect the most common thing is the user still has to look to point and the controller is just used for buttons.

    The last level is a user with a 6DOF headset and 2 6DOF controllers. Those users will find an experience that is only 3DOF to often be frustrating. Similarly they usually expect to be able to virtually manipulate things with their hands in VR so you'll have to decide if you want to support that or not.

As you can see getting started in VR is pretty easy but actually making something shippable in VR will require lots of decision making and design.

This was a pretty brief intro into VR with three.js. We'll cover some of the input methods in future articles.