Embedding/IPCLiteAPI: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(26 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Lite-weight Embedding IPC API for Mozilla based applications with native UI. ==
== Lite-weight Embedding API for Mozilla based applications with native UI. ==
<p>Basic architecture is very close to [http://trac.webkit.org/wiki/WebKit2 WebKit2], and [http://browser.garage.maemo.org/docs/browser_paper.html MicroB]
<p>Basic architecture is very close to [http://trac.webkit.org/wiki/WebKit2 WebKit2], and [http://browser.garage.maemo.org/docs/browser_paper.html MicroB]
== Story ==
=== Mozilla has bunch of technologies integrated recently: ===
* [https://wiki.mozilla.org/Platform/GFX/OffMainThreadCompositing OMTC - Off Main thread compositing - keeping Gecko/XPCOM and Compositor in separate threads (Android Firefox)]
* [https://wiki.mozilla.org/Electrolysis IPC - Multiprocess rendering - keeping Gecko/XPCOM and Compositor in separate processes (Firefox OS, Desktop Firefox Plugins)]
In order to build Simple browser with native UI (Gtk/Qt/UiKit) similar to Android Firefox with Java UI, some internal API need to be exposed.
Goal of this project to provide Generic embedding API which could be easily wrapped without bunch of extra code into Toolkit specific Widgets, and embed these widgets into platform UI applications.
</p><p>Benefits of new architecture comparing to default [https://wiki.mozilla.org/Embedding/NewApi mozilla embedding]:</p>
</p><p>Benefits of new architecture comparing to default [https://wiki.mozilla.org/Embedding/NewApi mozilla embedding]:</p>
<pre class="_fck_mw_lspace">* No XUL/XPCOM heavy content in UI process (Best memory footprint, fast startup)
<pre class="_fck_mw_lspace">* No XUL/XPCOM heavy content in UI process/Thread (Best memory footprint (process case), fast startup)
* IPC use current Mozilla Chromium/IPDL implementation (mostly tested with current FF/Mobile)
* Process/Thread communication between UI and Content(Gecko/XPCOM) Thread/Process - built on current Mozilla Chromium/IPDL implementation ([https://developer.mozilla.org/en-US/docs/IPDL/Tutorial IPDL Tutorial])
* Rendering based on Shadow/Shadowable layers tree (same as content rendering in XUL Mobile FF)
* Rendering based on Shadow/Shadowable layers tree which provide MultiThread/Process responsive rendering (same as content rendering in XUL Mobile FF/OMTC Android Firefox/Firefox OS)
* IPC provide ability to load memory greedy content in child process and kill it without increasing memory usage for Main UI application process
* Multi process model provide ability to load memory greedy content in child process and kill it without increasing memory usage for Main UI application process
</pre>
</pre>


== New embedding expected initialization sequence ==
== New embedding expected initialization sequence ==
  1. Native UI startup (android UI thread view or Gtk/Qt event loop)
  1. Native UI startup (android UI thread view or Gtk/Qt/UIKit event loop)
  2. Create XRE_InitIPCEmbedding
  2. Load EmbedLite library and start point API (can be called in parallel thread)
  3. Chromium message loop started
    LoadEmbedLite(argc, argv);
  a) Child process created and IPDL channel established
 
  4. Create embed view using embedding API
  3. Get EmbedLite application object, which allow to start Gecko in Child-Thread/Process, and create rendering views
  a) TabParent/Child, DOMWindow initialization
    EmbedLiteApp* embedapp = XRE_GetEmbedLite();
  b) PLayers protocol created
  4. Setup EmbedLite application listener (Some notifications about main app lifecycle)
  6. Render data from UI process LayerManager (SW/HW rendering) to Native UI GC.
    EmbedLiteAppListener[Derived]* listener = new EmbedLiteAppListener[Derived]();
  7. Load content in remote view (HTML, XUL)
    embedapp->SetListener(listener);
  5. Startup Embedding, blocking call, better to start UI first, and call this in some main loop event
    EmbedLite will not freeze application here, and continue native events processing along with Chromium Messages.
    embedapp->Start(EmbedLiteApp::EMBED_THREAD/PROCESS);
 
  [https://github.com/tmeshkova/gecko-dev/blob/embedlite/embedding/embedlite/tests/embedLiteCoreInitTest.cpp Core Embed Initialization example]
 
  6. After EmbedLiteAppListener[Derived] received "Initialized" notification, new EmbedLiteView can be created with:
    EmbedLiteView* view = embedapp->CreateView();
    And setup related EmbedLiteViewListener in order to receive notifications from View actions (title,progress,invalidate et.c.)
    view->SetListener(MyEmbedLiteViewListener);
  [https://github.com/tmeshkova/gecko-dev/blob/embedlite/embedding/embedlite/tests/embedLiteViewInitTest.cpp View Embed Initialization example]
 


=== Communication between UI and content process ===
=== Communication between UI and content process ===
  * Load remote content JS (privileged script with full access) and interact using MessageMessenger (send/receive JSON messages)
  * Load remote content JS (privileged script with full access) and interact using MessageMessenger (send/receive JSON messages)
  * Use embed messaging API (observe broadcast messages)
  * Use embed messaging API (observe broadcast messages)
  * Build Application specific IPC serialization library, load XPCOM component in child process linked with this IPC library.
  * Build Custom Gecko C++/JS extensions


== [http://hg.mozilla.org/users/romaxa_gmail.com/embedipc_queue Source code path queue] ==
== [http://hg.mozilla.org/users/romaxa_gmail.com/embedipc_queue Source code path queue] ==
== [https://bugzilla.mozilla.org/show_bug.cgi?id=713681 BMO Bug 713681] ==
== [https://bugzilla.mozilla.org/show_bug.cgi?id=713681 BMO Bug 713681(IPC)] ==
== [https://bugzilla.mozilla.org/show_bug.cgi?id=746800 BMO Bug 746800(OMTC)]  ==
 
== Development Build instructions: ==
 
Clone repository with submodules structure


== Build instructions: ==
<pre>
<pre>
hg clone http://hg.mozilla.org/users/romaxa_gmail.com/mozilla_ipcembed
git clone git://github.com/tmeshkova/xulrunner-package.git
cd mozilla_ipcembed
</pre>
cd .hg
 
hg clone http://hg.mozilla.org/users/romaxa_gmail.com/embedipc_queue patches
Build it with simple script run:
cd ..
<pre>
hg update -C embedipc
cd xulrunner-package
hg qpush -a
 
cp .hg/patches/mozconfig.gtkdesktop  mozconfig
# Pull all latest changes
# or
./pull.all.sh
# cp .hg/patches/mozconfig.qtdesktop mozconfig
 
MOZCONFIG=$(pwd)/mozconfig make -f client.mk build_all
# Not after each pull.all especially after major engine updates it make sense to do rm -f objdir-xxxx-xxx/full_*
 
# Check list of available options
./build.sh -h
 
# Haramttan
./build.sh -t harmattan
 
# Fremantle
./build.sh -t fremantle
 
# Desktop - Qt is default
./build.sh -t desktop
 
# For debug builds add -d argument
 
After build is finished you fill see inline message about how to run test UI
You can also export SWRENDER=1 - if your desktop or device does not support HW acceleration
 
 
</pre>
</pre>
== Packaged Build instructions (debian desktop, N9): ==
Clone repositories
<pre>
mkdir ~/mozembed
cd ~/mozembed
# Gecko Engine with EmbedLite API
git clone git://github.com/tmeshkova/gecko-dev.git
# Embedlite toolkit independent components (History,prompt et.c.)
git clone git@github.com:tmeshkova/embedlite-components.git
# qtmozembed - Qt wrapper library around EmbedLite API for quick integration into QML.
git clone git@github.com:tmeshkova/qtmozembed.git
# qmlmozbrowser UI application which works as simple reference UI
git clone git@github.com:tmeshkova/qmlmozbrowser.git
</pre>
Build with dpkg-buildpackage
<pre>
cd ~/mozembed/mozilla-central
dpkg-buildpackage -us -uc -b -nc
dpkg -i ../xulrunner*.deb
cd ~/mozembed/embedlite-components
dpkg-buildpackage -us -uc -b -nc
dpkg -i ../embedlite-components*.deb
cd ~/mozembed/qtmozembed
dpkg-buildpackage -us -uc -b -nc
dpkg -i ../libqtmozembed*.deb
cd ~/mozembed/qmlmozbrowser
dpkg-buildpackage -us -uc -b -nc
dpkg -i ../qmlmozbrowser*.deb
</pre>
# Simple test to check
<pre>
/usr/lib/xulrunner-21.0a1/embedLiteCoreInitTest
</pre>
# For development it is better to link xulrunner objdir to system (on desktop or on device with ~/mozembed mounted via NFS)
<pre>
cd ~/mozembed/mozilla-central
./link_to_system.sh obj-xr-qt /usr
# for embedlite-components, qtmozembed, qmlmozbrowser just run
./link_to_system.sh
</pre>
Finally run test example:
<pre>
qmlMozEmbedTest -url about:license
</pre>
== Try, Test, Report, Contribute: ==
Fell free to test it, submit issues to github issue tracker, or even better fork it, fix it, contribute changes via Pull Requests.
== Pre compiled binaries: ==
=== Mer (ARMv7 hardfp) ===
http://romaxa.info/mer/mozbuild.tar.gz
=== Harmattan ===
http://romaxa.info/mer/mozbuildhm.tar.gz
=== Qt-Desktop and BeagleBoard Ubuntu ===
http://romaxa.info/fennec/IPCEmbedding/


== Run test examples: ==
== Run test examples: ==
<pre>
<pre>
cd obj-build/dist/bin
cd obj-*/dist/bin
./testCoreEmbedXul
 
or GRE_HOME=xul-distpath ./testCoreEmbedXul
Basic core test
this works for GTK and Qt port.
./embedLiteCoreInitTest


EmbedLiteAPI wrapped into qt/qml widget
Qt only, qml based UI with scrolling and zooming (no clicks):
Qt only, qml based UI with scrolling and zooming (no clicks):
MOZ_QT_GRAPHICSSYSTEM=raster ./qtMozQmlEmbed -graphicssystem raster -url slashdot.org
./qmlMozEmbedTest -url linux.org.ru


raster is needed in order to optimize rendering and avoid X usage.
</pre>
on egl system like Maemo/Beagleboard export MOZ_ACCELERATED=1 and (also require setViewport(new QGLWidget()))
 
On desktop GLX accelerated rendering does not work, need to teach GLX backend use Qt GL context


=== Troubleshooting: ===
* If embedding does not start and libxul.so location differs from what has been used while building QtMozEmbed, then export path to GRE_HOME (path to folder with libxul.so)
<pre>
ex: export GRE_HOME=/opt/mozilla/xulrunner
</pre>
</pre>



Latest revision as of 16:05, 23 November 2014

Lite-weight Embedding API for Mozilla based applications with native UI.

Basic architecture is very close to WebKit2, and MicroB

Story

Mozilla has bunch of technologies integrated recently:

In order to build Simple browser with native UI (Gtk/Qt/UiKit) similar to Android Firefox with Java UI, some internal API need to be exposed. Goal of this project to provide Generic embedding API which could be easily wrapped without bunch of extra code into Toolkit specific Widgets, and embed these widgets into platform UI applications.

Benefits of new architecture comparing to default mozilla embedding:

* No XUL/XPCOM heavy content in UI process/Thread (Best memory footprint (process case), fast startup)
* Process/Thread communication between UI and Content(Gecko/XPCOM) Thread/Process - built on current Mozilla Chromium/IPDL implementation ([https://developer.mozilla.org/en-US/docs/IPDL/Tutorial IPDL Tutorial])
* Rendering based on Shadow/Shadowable layers tree which provide MultiThread/Process responsive rendering (same as content rendering in XUL Mobile FF/OMTC Android Firefox/Firefox OS)
* Multi process model provide ability to load memory greedy content in child process and kill it without increasing memory usage for Main UI application process

New embedding expected initialization sequence

1. Native UI startup (android UI thread view or Gtk/Qt/UIKit event loop)
2. Load EmbedLite library and start point API (can be called in parallel thread)
   LoadEmbedLite(argc, argv);
3. Get EmbedLite application object, which allow to start Gecko in Child-Thread/Process, and create rendering views
   EmbedLiteApp* embedapp = XRE_GetEmbedLite();
4. Setup EmbedLite application listener (Some notifications about main app lifecycle)
   EmbedLiteAppListener[Derived]* listener = new EmbedLiteAppListener[Derived]();
   embedapp->SetListener(listener);
5. Startup Embedding, blocking call, better to start UI first, and call this in some main loop event
   EmbedLite will not freeze application here, and continue native events processing along with Chromium Messages.
   embedapp->Start(EmbedLiteApp::EMBED_THREAD/PROCESS);
 Core Embed Initialization example
6. After EmbedLiteAppListener[Derived] received "Initialized" notification, new EmbedLiteView can be created with:
   EmbedLiteView* view = embedapp->CreateView();
   And setup related EmbedLiteViewListener in order to receive notifications from View actions (title,progress,invalidate et.c.)
   view->SetListener(MyEmbedLiteViewListener);

 View Embed Initialization example


Communication between UI and content process

* Load remote content JS (privileged script with full access) and interact using MessageMessenger (send/receive JSON messages)
* Use embed messaging API (observe broadcast messages)
* Build Custom Gecko C++/JS extensions

Source code path queue

BMO Bug 713681(IPC)

BMO Bug 746800(OMTC)

Development Build instructions:

Clone repository with submodules structure

git clone git://github.com/tmeshkova/xulrunner-package.git

Build it with simple script run:

cd xulrunner-package

# Pull all latest changes
./pull.all.sh

# Not after each pull.all especially after major engine updates it make sense to do rm -f objdir-xxxx-xxx/full_*

# Check list of available options
./build.sh -h 

# Haramttan
./build.sh -t harmattan

# Fremantle
./build.sh -t fremantle

# Desktop - Qt is default
./build.sh -t desktop

# For debug builds add -d argument

After build is finished you fill see inline message about how to run test UI
You can also export SWRENDER=1 - if your desktop or device does not support HW acceleration


Packaged Build instructions (debian desktop, N9):

Clone repositories

mkdir ~/mozembed
cd ~/mozembed
# Gecko Engine with EmbedLite API
git clone git://github.com/tmeshkova/gecko-dev.git

# Embedlite toolkit independent components (History,prompt et.c.)
git clone git@github.com:tmeshkova/embedlite-components.git

# qtmozembed - Qt wrapper library around EmbedLite API for quick integration into QML.
git clone git@github.com:tmeshkova/qtmozembed.git

# qmlmozbrowser UI application which works as simple reference UI
git clone git@github.com:tmeshkova/qmlmozbrowser.git

Build with dpkg-buildpackage

cd ~/mozembed/mozilla-central
dpkg-buildpackage -us -uc -b -nc
dpkg -i ../xulrunner*.deb

cd ~/mozembed/embedlite-components
dpkg-buildpackage -us -uc -b -nc
dpkg -i ../embedlite-components*.deb

cd ~/mozembed/qtmozembed
dpkg-buildpackage -us -uc -b -nc
dpkg -i ../libqtmozembed*.deb

cd ~/mozembed/qmlmozbrowser
dpkg-buildpackage -us -uc -b -nc
dpkg -i ../qmlmozbrowser*.deb
  1. Simple test to check
/usr/lib/xulrunner-21.0a1/embedLiteCoreInitTest
  1. For development it is better to link xulrunner objdir to system (on desktop or on device with ~/mozembed mounted via NFS)
cd ~/mozembed/mozilla-central
./link_to_system.sh obj-xr-qt /usr
# for embedlite-components, qtmozembed, qmlmozbrowser just run 
./link_to_system.sh

Finally run test example:

qmlMozEmbedTest -url about:license

Try, Test, Report, Contribute:

Fell free to test it, submit issues to github issue tracker, or even better fork it, fix it, contribute changes via Pull Requests.

Pre compiled binaries:

Mer (ARMv7 hardfp)

http://romaxa.info/mer/mozbuild.tar.gz

Harmattan

http://romaxa.info/mer/mozbuildhm.tar.gz

Qt-Desktop and BeagleBoard Ubuntu

http://romaxa.info/fennec/IPCEmbedding/

Run test examples:

cd obj-*/dist/bin

Basic core test
./embedLiteCoreInitTest

EmbedLiteAPI wrapped into qt/qml widget
Qt only, qml based UI with scrolling and zooming (no clicks):
./qmlMozEmbedTest -url linux.org.ru

Troubleshooting:

  • If embedding does not start and libxul.so location differs from what has been used while building QtMozEmbed, then export path to GRE_HOME (path to folder with libxul.so)
ex: export GRE_HOME=/opt/mozilla/xulrunner

Basic view of new architecture:

EmbedIPC-basic.png