This article will focus on the internal workings of the Inter-Process Communication (IPC) implementation in FirefoxOS (Boot2Gecko). It is still in development and might change a lot along the way. The main purpose of this article is, to give a detailed insight of the internal working and implementaiton. To read more about the security aspect of the IPC, please check out this article (TODO: link to other article). Any feedback or suggestion is greatly appreciated.

Architecture

Process architecture

In FirefoxOS we have a Multi-process Architecture, where the apps on the phone are running in a different process, which has the least amount of priviledges. On a running system we have a signle parent process called b2g. The b2g process has a child process called nuwa, this process will be used to fork the app processes. Whenever a new app is being started on the phone, b2g tells nuwa that we need a new process. All child processes have the least amount of privileges possible. For every action they want to perform (that needs more privileges), they need to go through the parent process (b2g). This is where the Inter-process Communication (IPC) comes into place. Each child has a connection, using an IPC channel, to the parent which will be used for their communication. The process layout is illustrated on the right.

Setup

Now that you have a rough overview of how everything is layed out, we will go into more detail on how this communication actually works. We use Unix sockets, created with the socketpair system call, to send messages across the process boundary. For writing and reading, the system calls sendmsg and recvmsg are being used. Each process has its own dedicated thread that handles the socket operations, it is called IOLoop. Each IOLoop thread has its own outgoing message queue, which is used by the main thread to send a new message across the channel.

IOLoop

An IOLoop thread is created for the parent during b2g startup, and for each child process after nuwa forks.

Parent

In the parent process (b2g) the IOLoop is created very early on, more precisely it happens in the NS_InitXPCOM2 function. This is the code snippet that actually initiates the thread start:

...
scoped_ptr<BrowserProcessSubThread> ioThread(
    new BrowserProcessSubThread(BrowserProcessSubThread::IO));
...
ioThread->StartWithOptions(options)
...

ioThread is an instance of BrowserProcessSubThread which inherits from base::Thread, itself a sublcass of PlatformThread::Delegate.

StartWithOptions is actually defined in base::Thread, the call will lead to a couple of more calls which will eventually end up at pthread_create. The function started in the new thread is ThreadFunc. ioThread object is passed along the calls and ioThread->ThreadMain() is called in the new thread.

ThreadMain is now running in the IOLoop thread, it will call Init() of the current instance and also creates a MessageLoop instance and call the Run() method. We will come back to that part later on, since it is not part of the thread startup any more.

...
// The message loop for this thread.
MessageLoop message_loop(startup_data_->options.message_loop_type);
...
// Let the thread do extra initialization.
// Let's do this before signaling we are started.
Init();
...
message_loop.Run();
...

Child

For the child IOLoop thread spawn, we have to look at the nuwa process as a child of b2g, but also at the forked processes of nuwa.

How b2g spawns nuwa will be covered later on, for right now we assume that the nuwa process already exists. Once nuwa is created, we eventually reach the XRE_InitChildProcess function, this function will be responsible for creating the IOLoop thread right along this line.

process = new ContentProcess(parentHandle);

In the ContentProcess constructor, the ProcessChild constructor is called, which leads to a call of the ChildProcess constructor. The important part to note here is, that the ChildProcess constructor gets passed a new instance of IOThreadChild. Within the constructor, the Run() is called on the passed IOThreadChild object:

ChildProcess::ChildProcess(ChildThread* child_thread)
    : child_thread_(child_thread),
    ... 
{
	...
		child_thread_->Run();
}

From there, the StartWithOptions function is called. At this point, it follows the same code path as for the main IOLoop startup the only exception is, that it is a IOThreadChild instance and not a BrowserProcessSubThread, so just check out the illustration above.

Since explaining the entire call flow with just text is kind of hard to understand, here a little illustration of it:

Now we covered the case for the nuwa process. All future children will be forked from nuwa, and since fork only copies the thread it was called in to the new process, all threads so far will be lost.

We want to have all threads from nuwa (along with the IOLoop), in the forked process. In order to do that, pthread_create is not called directly, instead, the call is routed to __wrap_pthread_create which wraps the real pthread_create. The purpose of the wrapper function, is to maintain a static list of startup information for all created threads (sAllThreads). This list will be copied to the new process, the new process will then call RecreateThreads to restore all threads, based on the information maintained in the list.

Channel

In order to be able to send and receive messages, we have to create a channel between the parent and the child. This section will cover the classes that are being used for doing that, the actual setup between parent and child will be covered once we get to the process spawning part.

It pretty much comes down to the creation of an IPC::Channel instance (the process of creating the instance will be covered later on). This class has two important attributes:

channel_impl_ which is the actual implementation of the channel (platform specific)
listener_ this is used to pass incomming messages to

the posix class for the channel_impl_ object can be found here (Channel::ChannelImpl).

Channel::ChannelImpl has the following important attributes:

pipe_ the file descriptor of the pipe (created by socketpair) for the parent side
client_pipe_ the client end of the pipe
listener_ the object that receives the incoming messages
output_queue_ a queue where all outgoing messages are pushed to

Channel::ChannelImpl has two overloaded construtor which can be used to create an object. One of them takes a file descriptor as the first argument which will be stored in pipe_. The more interesting constructor is the one which takes a channel_id (can also be empty). Both of them also take a Mode and a Listener* pointer as second and third argument. Mode just specifies if we are the server or the client.

When the constructor with the channel_id is called, CreatePipe will be called from there. We have to distinguish two different cases from here:

Mode == MODE_SERVER

In this case, socketpair will be called. One end of the pipe will be stored in pipe_ the other in client_pipe_. If channel_id is not empty, we insert a new entry in a PipeMap where we associate client_pipe_ with the given channel_id.

Mode != MODE_SERVER

In this case, we call ChannelNameToClientFD, which looks inside the PipeMap for an entry with the given channel_id. The result will be stored in pipe_.

In the end, EnqueueHelloMessage is called which will push the inital hello message onto the output queue.

After the object creation is completed, the Connect method can be called, this method will tell libevent to notify us whenever something has been written to pipe_ and is ready to be received.

OnFileCanReadWithoutBlocking is the callback for this event. This function will then call a function to read the message from the file descriptor and then the message will be passed to the OnMessageReceived function inside the listener_ (this will be covered later).

Here a short illustration of the call flow:

Spawning

In the previous section, we learned how IOLoop is created and how a channel is created. Throughout the last sections, we made the assumption that a process has already been started. This section will cover how those processes actually get started and how they connect to the IPC::Channel. We will again, have to distinguish between the nuwa process and the children of nuwa.

At this point I would suggest reading the IPDL Tutorial, because from this point on we will reference to some of the classes generated from those IPDL files.

Nuwa

creating the process

Throughout the initialization phase of the b2g process, an instance of the singleton class PreallocatedProcessManagerImpl will be created. This instance is mainly accessed through a couple of static functions defined in the PreallocatedProcessManager class. The purpose of this manager is to keep track of pre-allocated processes, this will be explained in more detail in the #Preallocated seciton.

The implementation class has two important attributes:

mSpareProcesses which is an array that contains the preallocated processes (will be important later on)
mPreallocatedAppProcess which will be the nuwa process

This initialization happens inside the ContentParent::StartUp function, when exuting the following code:

...
// Try to preallocate a process that we can transform into an app later.
PreallocatedProcessManager::AllocateAfterDelay();
...

This call will lead to the creation of the one and only instance of PreallocatedProcessManagerImpl is created (inside the PreallocatedProcessManagerImpl::Singleton function). Right after the constructor call, the Init function is invoked. Following the call flow from there, we will end up in Enable. Enable will then schedule the nuwa fork, with a 1 second delay (DEFAULT_ALLOCATE_DELAY), by calling ScheduleDelayedNuwaFork. This gives the b2g process enough time to finish its initalization.

As soon as the delay time has passed, the DelayedNuwaFork function is called inside the main thread. Inside the function, we will call ContentParent::RunNuwaProcess which returns a pointer to a ContentParent object, this object represents our nuwa process.

Inside the ContentParent constructor, a couple of interesting things happen.

we insert the new ContentParent into the global static list called sContentParent
we create a GeckoChildProcessHost instance
we call the LaunchAndWaitForProcessHandle method of GeckoChildProcessHost

The LaunchAndWaitForProcessHandle method will schedule a task inside the IOLoop thread. In the IOLoop thread, RunPerformAsyncLaunch is called. After a few calls, we will end up in the LaunchApp function. This is where the forking happens. After the fork, it will call execve in the child to re-execute itself.

connect to the channel

We covered the actual spawning, what's left is the part where the parent (b2g) and the child (nuwa) connect to the same IPC channel. We have two important calls for that on the parent side, one is made before the child was spawned, the other after the spawn. The first one is in the RunPerformAsynchLaunch function, before actually calling PerformAsynchLaunch (the position is marked with a 'x' in the above diagram), we call InitializeChannel this will call CreateChannel. At this point a new IPC::Channel object is created, so please checkout the #Channel section above.

The GeckoChildProcessHost object created inside the ContentParent constructor, serves as the listener_ inside the IPC::Channel object. So GeckoChildProcessHost will supply the OnMessageReceived function. There is nothing done there, it just saves all the incoming messages. At this point we can consider the parent process to be connected to the channel. This was the first important call.

The second one is called as soon as LaunchAndWaitForProcessHandle returned (nuwa process is running at this point). Since the current OnMessageReceived handler doesn't do any good, we will have to assign a new listener_. In order to do that, this is being executed (mSubprocess is an instance of GeckoChildProcessHost which is created in ContentParent):

Open(mSubprocess->GetChannel(), mSubprocess->GetOwnedChildProcessHandle());

User:Tedd/ipc-doc-preview

Contents