Canvas:3D

Contents

• Notes from 3D session at Boston Devloper Day, March 2007
• Historical notes (original Canvas 3D documentation)

Future Directions

David Humphrey, notes from Boston DevDay 2007

Role for canvas3d in the existing web

Canvas3d is a binary extension (written in C++) and accessible via script (i.e., JavaScript) that provides access to a 3D graphics context. Currently (April 2, 2007) it only works with Windows, but will work cross-platform. It uses hardware acceleration via OpenGL.

The current plan is to release the extension and watch what happens in the wild instead of trying to over-design APIs now and optimize too early. Other attempts to do 3D on the web are overreaching, and suffer from initial complexity. With canvas3d, the thinking is to start small and add/optimize as it becomes clear how people will use it.

One point made repeatedly is that we probably shouldn't be thinking about "Doom in the browser" as much as mixing 2D/3D seamlessly. It will be important to integrate the existing 2D web with new 3D content. It should be progressive enhancement vs. complete re-write of the web. Creating 3D worlds in the web is less important than figuring out how to add 3D to the toolset/metaphors/technologies we already have on-line.

What we need in order to get there

Since canvas3d exposes a raw OpenGL-based 3D-context, one of the first things to consider doing is writing various helper APIs in JavaScript:

• Scene graph Library
• Texture Buffers
• Matrix Libraries
• Model Loading
• Hit Testing
• Animation
• Compile shader files with GSL compiler and use, exposing via JavaScript

Some of this will be slow initially. However, once it's clear that a JS-only route won't work for some things (i.e., don’t assume it won't), slow things could be moved to C++/XPCOM and exposed to script via a separate extension (e.g., matrix libraries).

Important ideas to consider for 3D content in the web context:

• Introduction of spatial cues to the web
• Camera doesn't move with depth
• It needs to be possible to attach scripts to the 3D content and push events/etc into the browser.
• Need to figure out the right balance for hiding/showing content within the 3D scene. Consider how XBL hides its shadow tree/content from the "outer" DOM, as an example. Content will need access to some portion of the internal scene so that items can be linkable, portions of the scene expose themselves as <image> elements.

The role of CSS

Leveraging the existing web is key, including using CSS to do things like:

• Font-Depth, which could be used to move font in/out on the z-plane
• Lighting properties (light0…m)
• Scale for hovering
• Layering text in-front of, behind

Declarative animations for arbitrary content would also be possible down the road. For example, in CSS:

button:hover { animation: bounce 2; }

Doing this would cause button to "bounce" two times in its current position (i.e., grow to a certain size, then shrink back without changing it's (x,y) position).

Examples of 3D/2D existing together

Example 1: On-line store

Imagine a site that allows the user to browse/purchase items on-line, perhaps chairs. A typical page might contain a table with pictures of each chair, followed by a description, part number, price, etc. Each piece of furniture could live in a separate <canvas /> element, and therefore be its own 3D scene.

Moving the mouse over a chair might bring the chair closer (e.g., zoom toward) and allow the user to interact with it (e.g, rotate it). However, the 3D content in the <canvas /> element would not be a black box. Other content on the page could interact with it, and vice versa. Doing things with the chair inside the <canvas /> might cause events to propagate out into the web page and affect other <canvas /> objects (e.g., changing the preferred color might change all chairs to the same color) or other 2D elements in the page's content (e.g., update a price in a label).

Example 2: Adding 3D interaction for image display

Consider using a photo service like Flickr.com in order to build a 3D spatial image browser/slideshow. A combo 2D/3D web app could be created to showcase many of the things that are possible using cavas3d. One of the important things to get right would be to improve on the current Flash-based model used by Flickr itself. Flash is graphically rich, but breaks the web in that content within the embedded Flash element is not linkable/accessible to other page content. A canvas solution would need to insure that images in the 3D view were still <image> elements, so that other parts of the page could add listeners or otherwise interact with them. However, these <image> elements could behave in interesting 3D ways. With Flickr’s current slideshow view, you select a thumbnail image and it loads that image into view. Instead, if you select an image it could progressively scale-up to become the largest image (point-and-zoom) and then fall-back again when not in focus. Near/far spatial organization could also be added in order to allow the user to put images in order of importance along the z-plane or using relative size.

Beyond canvas rectangles

Eventually in Firefox (post version 3) it will be possible to do 3D transforms for content (e.g., making a button seem to grow or rotate when you hover over it). But for now, 3D will be confined to the canvas rectangle(s).