NuwaTemplateProcess

Nuwa is a mechanism to fork content processes from a template process. The template process is called Nuwa process. The processes forked from Nuwa process are parasited processes. The basic idea of Nuwa is running Nuwa process like a normal content process, but freeze all its threads after running into a stable state. All threads of Nuwa process will be frozen at a freeze point. The freeze points can be poll, epoll, pthread_mutex_lock, pthread_cond_wait, ... etc, functions that block a thread and release CPU to other threads or processes. For most programs, include B2G, the threads are not always busy. They usually sleep at some point by calling one of above functions. Nuwa process is frozen at freeze points to keep itself at some stable state. Then, Nuwa process forks itself, and its children are used as content processes. All threads are rebuilt and restarted from their freeze points at parasited processes.

Freeze Points

Nuwa make wrappers for all functions being freeze points. All these wrappers are in BionicGlue.cpp. We use --wrap arguments to make GNU LD redirect all function calls of freeze points to respective wrappers. In most time, these wrapper will call respective back functions. For example, the wrapper of pthread_mutex_lock() will call pthread_mutex_lock() to do the task. These wrappers will be blocked on a lock, sThreadFreezeLock, once Nuwa process run into a stable state. We say Nuwa process is frozen if all threads are blocked at these wrappers.

Threads

For Linux system, all threads are not surviving after fork. So, we need to create a new thread and running the frozen context for each thread in Nuwa process; parasite the frozen context on the new thread. To parasite the frozen context on the new thread, we reuse the stack of the frozen thread in the new thread. Each process forked from Nuwa process owns independent memory space, so it does not inference each other to reuse the stack of frozen stack.

 pthread_attr_setstack(attr, stack_ptr, stack_size)

is called to assign a stack for a thread created with the given attributes.

The context of a thread is comprised by registers and a stack. The registers are saved and restored by calling setjmp() and longjmp(). setjmp() is called at freeze point just before freezing the thread. longjmp() is called just at beginning of the new thread. Nuwa passes a special start routine for calling pthread_create(). The routine will call longjmp() ASAP to parasite the context of frozen thread on the new thread.

TLS

Thread local storage, aka TLS, is a part of a thread. TLS provides a separated storage for each thread, the data stored in TLS are accessed through a key. Nuwa process will remember all keys ever been used, and read and remember the associated data just before freeze point for every thread, then restore them at parasited threads. See SaveTLSInfo(), RestoreTLSInfo(), __wrap_pthread_key_create(), and __wrap_pthread_key_delete() in BionicGlue.cpp.