MDN/Development/Emscripten

Project statement

Emscripten is an LLVM to JavaScript compiler. It takes LLVM bytecode (which can be generated from C/C++ using Clang, or any other language that can be converted into LLVM bytecode) and compile that into JavaScript, which can be run on the web (or anywhere else JavaScript can run). Using Emscripten, you can

• Compile C and C++ code into JavaScript and run that on the web
• Run code originated in languages like Python on the web as well, for example by compiling CPython from C to JavaScript

This is a really cool and very useful technology, and it is essential for the success of some of Mozilla's current initiatives, such as web gaming. But the uses of Emscripten go beyond just building amazing games, therefore I have decided that Emscripten should have its own landing page on MDN, covering Emscripten in a more complete fashion.

Emscripten currently has a wiki on github that contains some great content, but this content could use some updating and reorganization, and we have lots of other ideas for content too, to tell the full story of how useful Emscripten is to interested parties such as C++ developers.

This document outlines the plans for an MDN resource covering Emcripten.

Where will this project live?

The main host for this docs project will be an MDN zone, with its landing page at https://developer.mozilla.org/en-US/docs/Emscripten.

We will also have a professional looking external site, say at emscripten.org, with nice links to demos etc., maybe a dozen pages all in all. This will be the "landing page", but will link back to MDN for the meat of the content.

What content should it have?

Although Emscripten can be used to compile a number of different languages into JavaScript, it would make sense to start with C++, as that is the best supported and most useful target laguage for our current projects. The following quote from Jukka provides some good context for the purpose of these docs:

My experience and perspective is mainly from the "Game developer who has written a native game in C/C++ for Android/iOS/PC and wants to port it to FxOS/browser" viewpoint.

Proposed structure

https://developer.mozilla.org/en-US/docs/Emscripten

• - Introducing Emscripten
• - Web abilities and limitations
• Product manual (covers latest version, currently 1.7.8)
  • - Download and install
  • - Workflow and tools
  • - Techniques
  • - Reference
• - Demos
• - Older product manual versions
  • - 1.7.1
    • - Download and install
    • - Workflow and tools
    • - Techniques
    • - Reference
  • - 1.5.1
    • - Download and install
    • - Workflow and tools
    • - Techniques
    • - Reference

Introducing Emscripten (always cover the latest version of Emscripten)

An introduction to Emscripten, including how it works, how to get it set up and start working with it, and some beginner's tutorials. Throughout this section we can reference more detailed guides in later sections.

• What is Emscripten and how does it work?
• The nature of Emscripten compiled JS - brief summary
  • non-threading
  • async main loop
  • async file loading
  • memory model
  • networking
  • performance
• Your first steps with Emscripten
  • Download and install (https://github.com/kripken/emscripten/wiki/Emscripten-SDK, https://github.com/kripken/emscripten/wiki/Getting-started-on-Mac-OS-X, https://github.com/kripken/emscripten/wiki/Getting-Started-on-Ubuntu-12.10, https://github.com/kripken/emscripten/wiki/Using-Emscripten-on-Windows)
  • Hello world (https://github.com/kripken/emscripten/wiki/Tutorial, present A minimal GL rendering sample that shows how to get stuff up and running
    • Generating the code
    • The asm.js module
    • The heap
    • linking
    • Emscripten boilerplate
    • Quick guide to Emscripten compiling, including some essential compiler flags
• Quickstart guide to putting Emscripten in your toolchain
• Next steps
  • Emscripten APIs
  • Working out what C++ stuff is supported well in Emscripten, and what isn't
  • What problems should you solve on the C++ side?
  • What should be written in JS and glued on. How is this done?
  • https://github.com/kripken/emscripten/wiki/Library-Support ?
  • https://github.com/kripken/emscripten/wiki/CodeGuidelinesAndLimitations
  • Solving specific problems (mini case studies - these don't need to be long, just present minimal examples and link to further resources in the techniques section?)
    • 2d graphics
    • 3d graphics
    • sound
    • file-system access
    • input
    • networking

Web abilities and limitations (probably doesn't need to be version-specific)

This could be a single article but may well be multiple articles under a single section as the document resource grows. The main purpose here is to provide native developers trying to research the feasibility of running their project on the web with some answers.

• How does the web differ to running native code? (https://github.com/kripken/emscripten/wiki/Emscripten-browser-environment)
  • non-threading
  • async main loop
  • async file loading
  • memory model
  • https://github.com/kripken/emscripten/wiki/Browser-limitations
  • networking
  • performance
• Performance on the web
  • How does it compare to native?
  • What are the bottlenecks?
  • 3D on the web is slow; how come?
  • What kind of sacrifices need to be made when porting to the web, in terms of performance?
• Browser support for Emscripten compiled code
  • What does Emscripten rely on?
  • What browsers currently support this combination?

Download and install (could be versioned)

• Win
• Mac
• Linux
• Troubleshooting
• Other useful tool links

Workflow and tools (could be versioned)

As a native developer, how can I get up and running with Emscripten, including it as an additional development target that fits into my existing toolchain?

• Emscripten in your development toolchain
  • An ideal toolchain
  • Other alternatives
• Emscripten in your build system (https://github.com/kripken/emscripten/wiki/Building-Projects)
  • https://github.com/kripken/emscripten/wiki/Code-Generation-Modes
  • Reworking your existing build system to target emscripten
  • Co-maintaining both native and emscripten targets simultaneously
  • Deploying and running your app on a mobile device
  • Installable app packaging
  • Mobile runtime quirks
• Debugging Emscripten code output (https://github.com/kripken/emscripten/wiki/Debugging)

Techniques for working with Emscripten (could be versioned)

More detailed tutorials than you got with the hello world section in the introductory material. Generally these should help developers solve specific problems with their code, and porting it across. It might be an idea to, at each point, say what C++ component is being used, and what JavaScript API/construct it is being compiled into, making it easier to understand what is going on whether you are a C++ person or a JavaScript person.

Generally the C++ devs won't be initially familiar with the JS specs, and if they are presented e.g. with the Gamepad API or Web Audio API, they'll reject those docs since they are foremost looking for C/C++ solutions to their problems. This is partly because they have just learned about this Emscripten C/C++ compiler that they are now harnessing, and they are right now getting their initial problems solved in the C/C++ side without a single line of JS. Another reason is that devs might not know the JS syntax at all, or if they do, there's a considerable amount of new C/C++ <-> JS interop machinery that one needs to learn, and how to integrate JS into the native build arch, and so on, so being able to develop against JS side APIs will generally come much later during the porting cycle, when the landscape has been charted sufficiently enough to know that one can't (yet) solve all the problems with just native C/C++ code alone.

• C++ libraries supported by Emscripten - for each one, cover writing code that works best with Emscripten compilation
  • SDL
  • OpenAL
  • EGL (see https://github.com/kripken/emscripten/wiki/EGL-Support-in-Emscripten for a good example)
  • OPENGL (https://github.com/kripken/emscripten/wiki/OpenGL-support)
  • GLES2
  • https://github.com/kripken/emscripten/wiki/GC
  • https://github.com/kripken/emscripten/wiki/Linking
  • others...
• Architecting around common problems - provide a case study for each? (Currently we do have some examples of "how do I do X in Emscripten" as part of the unit test suite, but naturally that's not a good format for reading as a documentation/learning material. Some of those could be hoisted or used as a basis for a better set of example apps.)
  • Common C++ libraries/technologies that Emscripten doesn't handle so well, provide an article for each?
  • non-threading
  • async main loop (async compilation is already covered at https://developer.mozilla.org/en-US/docs/Games/Techniques/Async_scripts. Perhaps include some extra explanation about what's going on inside the JS engine?)
  • async file loading (https://github.com/kripken/emscripten/wiki/Filesystem-Guide, https://github.com/kripken/emscripten/wiki/Filesystem-API)
  • memory model
  • networking
  • performance
  • https://github.com/kripken/emscripten/wiki/Asm-pointer-casts
  • https://github.com/kripken/emscripten/wiki/Synchronous-Virtual-XHR-Backed-File-System-Usage
• Emscripten APIs (see https://github.com/kripken/emscripten/blob/master/system/include/emscripten/emscripten.h)
  • Embind (https://github.com/kripken/emscripten/wiki/embind)
  • y
  • z
• Combining handspun JavaScript with Emscripten code (https://github.com/kripken/emscripten/wiki/Interacting-with-code)
  • Developing a JS library to solve problem X
  • Gluing it to your Emscripten application
• Benchmarking, profiling and optimization
  • e.g. minimizing data sizes (https://github.com/kripken/emscripten/wiki/Compressing-Downloads)
  • optimize web transfers?
  • optimizing generated code (https://github.com/kripken/emscripten/wiki/Optimizing-Code, https://github.com/kripken/emscripten/wiki/Optimizing-the-source-code)
• Creating the launch experience
  • Creating a start page
  • Launching the application into full screen or windowed mode.

Reference documentation (could be versioned)

• Compiler reference
  • Compiler flags list (The most important compiler flags are mentioned in https://github.com/kripken/emscripten/wiki/Tutorial and https://github.com/kripken/emscripten/wiki/Optimizing-Code)
• Links to relevant C/C++ library documentation, with annotations relevant to Emscripten usage
• Links to relevant JS/API documentation, with annotations relevant to Emscripten usage
• Links to other relevant docs - Graphics, GLES2?
• Links to relevant tool documentation
• Emscripten FAQ (https://github.com/kripken/emscripten/wiki/FAQ)

Demos (does this need to be versioned?)

• a
• b
• c
• etc.

What should we talk to?

Alon, Jukka, Vlad, Martin Best, Dave Herman, Luke Wagner, the rest of the games team!