Canvas:3D: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Added notes from Boston DevDay)
No edit summary
 
(8 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= Canvas 3D Context =
'''NOTE: This is old content!  Canvas 3D has now become WebGL, a standard being developed by the Khronos Group.  See http://www.webgl.org/ for more information!'''
<i>[[VladVukicevic]], work in progress</i>


== Rationale ==
The Canvas 3D project aims to add support for low-level hardware accelerated (where possible) 3D rendering via the HTML <tt>canvas</tt> element.  Some of the original ideas and thoughts surrounding the 3D canvas are available on [[Canvas:3D/Historical|historical notes page]], and more recent thoughts and plans are at various other documents below.


Since <tt>&lt;canvas&gt;</tt> provides basically a rectangular immediate-mode drawing surface, extending it to programmatic 3D rendering is straightforward.  [http://www.opengl.org/ OpenGL] is the natural accepted cross-platform 3D API to follow; it has a well-defined and rigorous specification that would be well beyond my ability to recreate. However, OpenGL itself (2.0 in its latest incarnation) is extremely sprawling, including many redundant ways and now-obsolete methods.
The Canvas 3D context(s) will be delivered in the form of a Firefox addon, with the possibility of shipping by default with Firefox 3.  Please see the Roadmap for schedule and release plans.  Discussion around both the development of the canvas 3D addon as well as for applications taking advantage of the 3D capabilities can take place at the [https://labs.mozilla.com/forum/index.php/board,9.0.html|Canvas 3D Discussion Forum at Mozilla Labs].


Instead, I plan to follow [http://www.khronos.org/opengles/ OpenGL ES], which is a version of the OpenGL spec pared down for implementation in embedded systems.  It removes much of the dead weight of OpenGL, e.g. rendering via <tt>glBegin</tt>, <tt>glVertex[234][dfis][v]</tt>, etc.; rendering quads and polygons (since they can be emulated through triangles trivially); and similar.
== Contents ==


[http://www.khronos.org/opengles/1_X/ OpenGL ES 1.1.4] is the current plan, which presents a traditional fixed-function OpenGL API.  I would like to, at some point, support [http://www.khronos.org/opengl/2_X/ OpenGL ES 2.0], which provides maps today's modern programmable hardware; however, implementing 2.0 can be done an incremental step over implementing the fixed-function API.
* [https://labs.mozilla.com/forum/index.php/board,9.0.html Canvas 3D Discussion Forum at Mozilla Labs]
* [[Canvas:3D/Roadmap|Roadmap]]
* [[Canvas:3D/DevDayBoston0307|Notes from 3D session at Boston Developer Day, March 2007]]
* [[Canvas:3D/Historical|Historical notes (original Canvas 3D documentation)]]
* [http://www.c3dl.org/ JavaScript library implementation work]
* [http://hg.mozilla.org/users/vladimir_mozilla.com/canvas3d/ Firefox extension source code]


One of the difficulties of OpenGL is that it is extremely low level, and in many ways looks strange when exposed as a web API.  Where appropriate, the addition of helper objects and functions will be done to allow for speed and ease of use -- however, as with OpenGL on the desktop, it should be possible to create a richer API on top of OpenGL in whatever language is being used to interact with the 3D canvas context (e.g. JavaScript, Python, etc.).
== Related Links ==


== Security Considerations ==
* [http://www.opengl.org/ OpenGL Home Page]
 
* [http://khronos.org/opengles/ OpenGL ES Home Page and Specifications]
The OpenGL API is complex and stateful; much of the API, out of permance requirements, deals with pointers to arbitrary data types.  For example, the function <tt>glVertexPointer(GLint size, GLenum type, GLsizei stride, const GLvoid *ptr)</tt> can take bytes, integers, floats, doubles, etc. as valid <tt>ptr</tt> types, based on the given <tt>type</tt>.  After a vertex pointer is set, calling <tt>glDrawArrays(GLenum mode, GLint first, GLsizei count)</tt> to actually draw the given vertex/texture/etc. pointer arrays will require a different size of array depending on whether one is drawing TRIANGLES, TRIANGLE_STRIPS, TRIANGLE_FANS, etc. -- for example, assuming the appropriate types are used, TRIANGLES will need 3*count floats from vertices, 2*count floats from texcoords (assuming they're specified), 4*count floats from a color array, etc.
 
OpenGL implementations usually handle an error by simply crashing; no size is passed when the various pointer arrays are set (the <tt>size</tt> parameter refers to the number of coordinates per element, e.g. 3 for a 3D vertex).  This is obviously not acceptable for a 3D Canvas Context executing within a web browser; thus the implementation must be extrmely careful to track array sizes and the like and to check them before calling in to any underlying OpenGL implementation.
 
== Performance ==
 
Initial testing reveals that performance can be quite good; a pbuffer context is created for the canvas, and copying into the front buffer is performed when SwapBuffers() is executed.  Because all drawing is done through arrays, there isn't a lot of method call overhead for drawing operations.
 
== Implementation Notes ==
 
- vertex/color/texcoord array data
- matrix manipulation (load/push/pop/translate/rotate/scale/ortho/frustum/lookAt/matrixmode)
- drawing via drawarrays or drawelements (indexes)
- texturing from <img> (2D POT textures only, no TEXTURE_RECTANGLE or 1D/3D supported)
 
== Future Directions ==
 
<i>David Humphrey, notes from Boston DevDay 2007</i>
 
=== Role for canvas3d in the existing web ===
 
Canvas3d is a binary extension (written in C++) and accessible via script (i.e., JavaScript) that provides access to a 3D graphics context.  Currently (April 2, 2007) it only works with Windows, but will work cross-platform.  It uses hardware acceleration via OpenGL.
 
The current plan is to release the extension and watch what happens in the wild instead of trying to over-design APIs now and optimize too early.  Other attempts to do 3D on the web are overreaching, and suffer from initial complexity.  With canvas3d, the thinking is to start small and add/optimize as it becomes clear how people will use it.
 
One point made repeatedly is that we probably shouldn't be thinking about "Doom in the browser" as much as mixing 2D/3D seamlessly. It will be important to integrate the existing 2D web with new 3D content. It should be progressive enhancement vs. complete re-write of the web.  Creating 3D worlds in the web is less important than figuring out how to add 3D to the toolset/metaphors/technologies we already have on-line.
 
=== What we need in order to get there ===
 
Since canvas3d exposes a raw OpenGL-based 3D-context, one of the first things to consider doing is writing various helper APIs in JavaScript:
 
* Scene graph Library
* Texture Buffers
* Matrix Libraries
* Model Loading
* Hit Testing
* Animation
* Compile shader files with GSL compiler and use, exposing via JavaScript
 
Some of this will be slow initially.  However, once it's clear that a JS-only route won't work for some things (i.e., don’t assume it won't), slow things could be moved to C++/XPCOM and exposed to script via a separate extension (e.g., matrix libraries).
 
Important ideas to consider for 3D content in the web context:
 
* Introduction of spatial cues to the web
* Camera doesn't move with depth
* It needs to be possible to attach scripts to the 3D content and push events/etc into the browser.
* Need to figure out the right balance for hiding/showing content within the 3D scene.  Consider how XBL hides its shadow tree/content from the "outer" DOM, as an example.  Content will need access to some portion of the internal scene so that items can be linkable, portions of the scene expose themselves as <code><image></code> elements.
 
=== The role of CSS ===
 
Leveraging the existing web is key, including using CSS to do things like:
 
* Font-Depth, which could be used to move font in/out on the z-plane
* Lighting properties (light0…m)
* Scale for hovering
* Layering text in-front of, behind
 
Declarative animations for arbitrary content would also be possible down the road.  For example, in CSS:
 
<pre>
button:hover { animation: bounce 2; }
</pre>
 
Doing this would cause button to "bounce" two times in its current position (i.e., grow to a certain size, then shrink back without changing it's (x,y) position).
 
=== Examples of 3D/2D existing together ===
 
==== Example 1: On-line store ====
 
Imagine a site that allows the user to browse/purchase items on-line, perhaps chairs.  A typical page might contain a table with pictures of each chair, followed by a description, part number, price, etc.  Each piece of furniture could live in a separate <code><canvas /></code> element, and therefore be its own 3D scene. 
 
Moving the mouse over a chair might bring the chair closer (e.g., zoom toward) and allow the user to interact with it (e.g, rotate it).  However, the 3D content in the <code><canvas /></code> element would not be a black box.  Other content on the page could interact with it, and vice versa.  Doing things with the chair inside the <code><canvas /></code> might cause events to propagate out into the web page and affect other <code><canvas /></code> objects (e.g., changing the preferred color might change all chairs to the same color) or other 2D elements in the page's content (e.g., update a price in a label).
 
==== Example 2: Adding 3D interaction for image display ====
 
Consider using a photo service like Flickr.com in order to build a 3D spatial image browser/slideshow.  A combo 2D/3D web app could be created to showcase many of the things that are possible using cavas3d.  One of the important things to get right would be to improve on the current Flash-based model used by Flickr itself.  Flash is graphically rich, but breaks the web in that content within the embedded Flash element is not linkable/accessible to other page content.  A canvas solution would need to insure that images in the 3D view were still <code><image></code> elements, so that other parts of the page could add listeners or otherwise interact with them.  However, these <code><image></code> elements could behave in interesting 3D ways.  With Flickr’s current slideshow view, you select a thumbnail image and it loads that image into view.  Instead, if you select an image it could progressively scale-up to become the largest image (point-and-zoom) and then fall-back again when not in focus.  Near/far spatial organization could also be added in order to allow the user to put images in order of importance along the z-plane or using relative size.
 
=== Beyond canvas rectangles ===
 
Eventually in Firefox (post version 3) it will be possible to do 3D transforms for content (e.g., making a button seem to grow or rotate when you hover over it).  But for now, 3D will be confined to the canvas rectangle(s).

Latest revision as of 20:35, 25 January 2010

NOTE: This is old content! Canvas 3D has now become WebGL, a standard being developed by the Khronos Group. See http://www.webgl.org/ for more information!

The Canvas 3D project aims to add support for low-level hardware accelerated (where possible) 3D rendering via the HTML canvas element. Some of the original ideas and thoughts surrounding the 3D canvas are available on historical notes page, and more recent thoughts and plans are at various other documents below.

The Canvas 3D context(s) will be delivered in the form of a Firefox addon, with the possibility of shipping by default with Firefox 3. Please see the Roadmap for schedule and release plans. Discussion around both the development of the canvas 3D addon as well as for applications taking advantage of the 3D capabilities can take place at the 3D Discussion Forum at Mozilla Labs.

Contents

Related Links

Retrieved from "https://wiki.mozilla.org/index.php?title=Canvas:3D&oldid=196582"