MDN/Development/Emscripten

Project statement

Emscripten is an LLVM to JavaScript compiler. It takes LLVM bytecode (which can be generated from C/C++ using Clang, or any other language that can be converted into LLVM bytecode) and compile that into JavaScript, which can be run on the web (or anywhere else JavaScript can run). Using Emscripten, you can

• Compile C and C++ code into JavaScript and run that on the web
• Run code originated in languages like Python on the web as well, for example by compiling CPython from C to JavaScript

This is a really cool and very useful technology, and it is essential for the success of some of Mozilla's current initiatives, such as web gaming. But the uses of Emscripten go beyond just building amazing games, therefore I have decided that Emscripten should have its own landing page on MDN, covering Emscripten in a more generic fashion.

Emscripten currently has a wiki on github that contains some great content, but this content could use some updating and reorganization, and we have lots of other ideas for content too, to tell the full story of how useful Emscripten is to interested parties such as C++ developers.

This document outlines the plans for an MDN resource covering Emcripten.

Where will this project live?

The obvious place would be a landing page at https://developer.mozilla.org/en-US/docs/Emscripten.

What content should it have?

Although Emscripten can be used to compile a number of different languages into JavaScript, it would make sense to start with C++, as that is the best supported and most useful target laguage for our current projects. The following quote from Jukka provides some good context for the purpose of these docs:

My experience and perspective is mainly from the "Game developer who has written a native game in C/C++ for Android/iOS/PC and wants to port it to FxOS/browser" viewpoint.

Introducing Emscripten

An introduction to Emscripten, including how it works, how to get it set up and start wqorking with it, and some beginner's tutorials

• What is Emscripten and how does it work?
  • The nature of Emscripten compiled JS
• Your first steps with Emscripten
  • Download and install
  • Hello world

Web abilities and limitations

This could be a single article but may well be multiple articles under a single section as the document resource grows. The main purpose here is to provide native developers trying to research the feasibility of running their project on the web with some answers.

• How does the web differ to runing native code?
  • non-threading
  • async main loop
  • async file loading
  • memory model
  • networking
  • performance
• Performance on the web
  • How does it compare to native?
  • What are the bottlenecks?
  • 3D on the web is slow; how come?
  • What kind of sacrifices need to be made when porting to the web, in terms of performance?
• Browser support for Emscripten compiled code
  • What does Emscripten rely on?
  • What browsers currently support this combination?
• C++ landscape support for Emscripten
  • Common C++ libraries/technologies that Emscripten handles successfully
  • Common C++ libraries/technologies that Emscripten doesn't handle so well
    • What needs to be rewritten in C++
    • What is better written from scratch on the JavaScript side?

Integrating Emscripten into your workflow

As a native developer, how can I get up and running with Emscripten, including it as an additional development target that fits into my existing toolchain?

• Emscripten in your development toolchain
  • An ideal toolchain
  • Other alternatives
  • A minimal GL rendering sample that shows how to get stuff up and running?
• Emscripten in your build system
  • Reworking your existing build system to target emscripten
  • Co-maintaining both native and emscripten targets simultaneously
  • Deploying and running your app on a mobile device
• Debugging Emscripten code output

Techniques for working with Emscripten

More detailed tutorials than you got with the hello world section in the introductory material.

Reference documentation

• Compiler reference
• Links to relevant C/C++ library documentation, with annoatations relevant to Emscripten usage
• Links to relevant JS/API documentation, with annotations relevant to Emscripten usage
• Links to other relevant docs - Graphics, GLES2?
• Links to relevant tool documentation

(disbelief/solving blockers) 10. I have learned about these hard problems (non-threading, async main loop, async file loading, memory model, networking, performance, ...) that I am now able to relate to my own codebase having attempted a port. What can I do? How do I architect around these problems?

(port development) 11. Where is the reference documentation for C/C++ library/api X in Emscripten?

(port development) 12. Where is the reference documentation for JS library/api/spec X?

(port development) 13. How do I develop a JS library to solve X and glue it to my C/C++ game?

(port development) 14. How do I manage FxOS app packaging and mobile runtime?

(finalization) 15. How do I do benchmarking, profiling and optimizations? How to minimize data sizes and optimize web transfers?

(finalization) 16. How do I interact/glue the C/C++ game experience with the browser environment? Web site development around the game shell page, etc?

Generally the devs are initially not familiar with the JS specs, and if they are presented e.g. with the Gamepad API or Web Audio API, they'll reject those docs since they are foremost looking for C/C++ solutions to their problems. This is partly because they have just learned about this Emscripten C/C++ compiler that they are now harnessing, and they are right now getting their initial problems solved in the C/C++ side without a single line of JS. Another reason is that devs might not know the JS syntax at all, or if they do, there's a considerable amount of new C/C++ <-> JS interop machinery that one needs to learn, and how to integrate JS into the native build arch, and so on, so being able to develop against JS side APIs will generally come much later during the porting cycle, when the landscape has been charted sufficiently enough to know that one can't (yet) solve all the problems with just native C/C++ code alone.

I think an important angle to cover, that we currently have a big glaring hole at in the current Emscripten docs at https://github.com/kripken/emscripten/wiki#technical-documentation is the answers to questions #3, #4 and #11. We should have strong documentation pages on each individual API/library we support directly in Emscripten, since a huge part of the Emscripten work (and one that stands us out most compared to nacl!) goes into making sure that existing C/C++ technologies like SDL, OpenAL, EGL, GLES2, etc. are directly supported. For each of these apis that are there, we should have a page "Support for X in Emscripten", similar to we now have for EGL https://github.com/kripken/emscripten/wiki/EGL-Support-in-Emscripten , since that will help people at the initial feasibility study phase and to deciding positively on whether they will take on the effort to try a port.

A second set of docs we should have more flesh in, is the set of emscripten-specific APIs that we have been developing. We don't have much of those, but generally this header file is the only doc available https://github.com/kripken/emscripten/blob/master/system/include/emscripten/emscripten.h .

Third, the set of big problems that everyone porting a native game will eventually run into (item #10 in the list above). Until we have good fixes on those, we should have good prose-like technical documentation that describes the problems with examples of what will not work, and how to refactor the code so that it is JS-friendly, etc.

Currently we do have some examples of "how do I do X in Emscripten" as part of the unit test suite, but naturally that's not a good format for reading as a documentation/learning material. Some of those could be hoisted or used as a basis for a better set of example apps.