WebDriver/RemoteProtocol/GutenbergCDPUsage: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
m (Note that top Network methods are irrelevant)
m (Whimboo moved page Remote/GutenbergCDPUsage to WebDriver/RemoteProtocol/GutenbergCDPUsage: Reorganization for WebDriver BiDi project documenation)
 
(14 intermediate revisions by 3 users not shown)
Line 2: Line 2:


__FORCETOC__
__FORCETOC__
== Running the gutenberg tests against Firefox ==
=== Prerequisites ===
The following is a quickstart based on Gutenberg's [https://github.com/WordPress/gutenberg/blob/master/docs/contributors/testing-overview.md#end-to-end-testing docs].
# Make sure Docker is running
# <code>nvm install</code> in gutenberg dir to ensure correct node version is installed
# <code>npm ci</code> -- you might see some warnings, this is fine.
# <code>npm run build</code>
# Build local env: <code>npm run wp-env install</code>. Avoid running <code>npm run dev</code>; takes a long time, not useful.
# Start the environment: <code>npm run wp-env start</code>
To stop the local WordPress instance: <code>npm run env stop</code>
=== Firefox prerequisites ===
Update the config in <code>packages/scripts/config/puppeteer.config.js</code>:
<pre>
module.exports = {
    launch: {
        headless: false,
        slowMo: 0,
        executablePath: 'path/to/firefox',
    },
};
</pre>
<small>Note: Puppeteer can download and install Firefox Nightly and you can use the path it installs into for the executablePath. For example, on macOS: <code>path/to/gutenberg/node_modules/puppeteer/.local-firefox/mac-80/Firefox\ Nightly.app/Contents/MacOS/firefox</code></small>
# Copy Firefox's modifications of Puppeteer:
<pre>
cp -r <mozilla-central>/remote/test/puppeteer/ node_modules/puppeteer/
npm run build
</pre>
=== Running tests ===
Select Firefox by setting <code>PUPPETEER_PRODUCT</code>, otherwise Chromium will be used. `DEBUG=*` will show all debug logs, or you
can filter it as in the example below:
<pre>
PUPPETEER_PRODUCT=firefox DEBUG=puppeteer:* npm run test-e2e
</pre>
To run a specific test:
<pre>
npm run test-e2e links.test.js
</pre>
The <code>--puppeteer-interactive</code> option can be useful for debugging:
<pre>
npm run test-e2e links.test.js -- --puppeteer-interactive
</pre>
Equivalent to <code>PUPPETEER_HEADLESS=false PUPPETEER_SLOWMO=80 npm run test-e2e:watch</code>
=== Simple tests to consider ===
* mentions.tests.js
* post-visibility.test.js
* manage-reusable-blocks.test.js
* style-variation-test.js


== Methods ==
== Methods ==
Line 72: Line 140:
* Target.setAutoAttach
* Target.setAutoAttach


== Methods & Events logged during test setup ==
=== Methods ===
<pre>
cat visit-admin-page.log create-new-post.log | grep '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
  52 Runtime.callFunctionOn
  52 Input.dispatchKeyEvent
  16 Runtime.releaseObject
  11 Runtime.evaluate
  9 Input.dispatchMouseEvent
  8 Page.navigate
  6 Emulation.setDeviceMetricsOverride
  4 Target.setDiscoverTargets
  3 Page.getLayoutMetrics
  3 DOM.getContentQuads
  2 Target.setAutoAttach
  2 Target.getBrowserContexts
  2 Target.createTarget
  2 Target.closeTarget
  2 Target.attachToTarget
  2 Runtime.enable
  2 Performance.enable
  2 Page.setInterceptFileChooserDialog
  2 Page.getFrameTree
  2 Page.enable
  2 Page.createIsolatedWorld
  2 Page.addScriptToEvaluateOnNewDocument
  2 Network.enable
  2 Log.enable
  1 DOM.resolveNode
  1 DOM.describeNode
</pre>
=== Events ===
<pre>
cat visit-admin-page.log create-new-post.log | grep -v '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
820 Network.dataReceived
576 Network.requestWillBeSent
571 Network.loadingFinished
560 Network.responseReceived
298 Network.responseReceivedExtraInfo
275 Network.requestServedFromCache
273 Network.requestWillBeSentExtraInfo
110 Page.lifecycleEvent
  53 Target.targetInfoChanged
  24 Runtime.executionContextCreated
  20 Runtime.executionContextDestroyed
  16 Target.targetCreated
  14 Runtime.consoleAPICalled
  10 Runtime.executionContextsCleared
  10 Page.loadEventFired
  10 Page.frameNavigated
  10 Page.frameStoppedLoading
  10 Page.frameStartedLoading
  10 Page.domContentEventFired
  8 Page.navigatedWithinDocument
  6 Emulation.setTouchEmulationEnabled
  6 Page.frameResized
  5 Target.targetDestroyed
  3 Network.resourceChangedPriority
  3 Log.entryAdded
  2 Target.detachedFromTarget
  2 Target.attachedToTarget
  2 Page.setLifecycleEventsEnabled
  2 Page.frameScheduledNavigation
  2 Page.frameRequestedNavigation
  2 Page.frameClearedScheduledNavigation
  2 Inspector.detached
</pre>


== Methods ranked by frequency ==
== Methods & Events ranked by frequency ==


Note that Puppeteer doesn't necessarily depend on many of the Network, Runtime, Fetch and Page items.
Note that Puppeteer doesn't necessarily depend on many of the Network, Runtime, Fetch and Page items.
=== Methods ===


<pre>
<pre>
cat gutenberg-logs  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
cat gutenberg-logs | grep '"id":' | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r


65119 Network.dataReceived
15594 Input.dispatchKeyEvent
56977 Network.requestWillBeSent
12738 Runtime.callFunctionOn
56906 Network.loadingFinished
7324 Input.dispatchMouseEvent
55864 Network.responseReceived
3088 Runtime.releaseObject
37106 Network.requestServedFromCache
1893 Page.getLayoutMetrics
14924 Input.dispatchKeyEvent
1893 DOM.getContentQuads
12462 Runtime.callFunctionOn
1101 Runtime.evaluate
7709 Input.dispatchMouseEvent
1091 Fetch.continueRequest
6232 Page.lifecycleEvent
  977 DOM.resolveNode
3367 Target.targetInfoChanged
  977 DOM.describeNode
3107 Runtime.releaseObject
  636 Emulation.setTouchEmulationEnabled
1891 Page.getLayoutMetrics
  636 Emulation.setDeviceMetricsOverride
1891 DOM.getContentQuads
  595 Page.navigate
1514 Runtime.executionContextCreated
  496 Runtime.getProperties
1510 Runtime.consoleAPICalled
  267 Page.handleJavaScriptDialog
1346 Runtime.executionContextDestroyed
   83 Target.setAutoAttach
1019 Runtime.evaluate
   83 Target.attachToTarget
  947 DOM.resolveNode
   83 Runtime.enable
  947 DOM.describeNode
   83 Performance.enable
  893 Fetch.requestPaused
   83 Page.setLifecycleEventsEnabled
881 Fetch.continueRequest
   83 Page.setInterceptFileChooserDialog
699 Page.navigatedWithinDocument
   83 Page.getFrameTree
681 Page.frameStoppedLoading
   83 Page.enable
678 Page.frameStartedLoading
   83 Page.createIsolatedWorld
676 Page.frameNavigated
   83 Page.addScriptToEvaluateOnNewDocument
674 Page.domContentEventFired
   83 Network.enable
666 Runtime.executionContextsCleared
   83 Log.enable
664 Page.loadEventFired
   81 Target.closeTarget
621 Target.targetCreated
   80 Target.setDiscoverTargets
598 Emulation.setTouchEmulationEnabled
   79 Target.getBrowserContexts
  598 Emulation.setDeviceMetricsOverride
   79 Target.createTarget
  560 Page.navigate
   35 Page.reload
  529 Runtime.getProperties
   23 Input.insertText
  238 Page.javascriptDialogOpening
238 Page.javascriptDialogClosed
238 Page.handleJavaScriptDialog
235 Target.targetDestroyed
  89 Page.frameResized
   81 Target.setAutoAttach
   81 Target.attachedToTarget
  81 Target.attachToTarget
   81 Runtime.enable
   81 Performance.enable
   81 Page.setLifecycleEventsEnabled
   81 Page.setInterceptFileChooserDialog
   81 Page.getFrameTree
   81 Page.enable
   81 Page.createIsolatedWorld
   81 Page.addScriptToEvaluateOnNewDocument
   81 Network.enable
   81 Log.enable
   79 Target.detachedFromTarget
  79 Target.closeTarget
   78 Target.setDiscoverTargets
   77 Target.getBrowserContexts
   77 Target.createTarget
   30 Page.frameScheduledNavigation
  30 Page.frameRequestedNavigation
  30 Page.frameClearedScheduledNavigation
  28 Page.reload
   22 Input.insertText
   14 Network.setCacheDisabled
   14 Network.setCacheDisabled
   12 Fetch.fulfillRequest
   12 Fetch.fulfillRequest
  11 Network.resourceChangedPriority
  10 Page.frameAttached
   8 Page.bringToFront
   8 Page.bringToFront
   8 Fetch.enable
   8 Fetch.enable
  7 Page.frameDetached
   6 Fetch.disable
   6 Fetch.disable
   4 Log.entryAdded
   4 Network.emulateNetworkConditions
   4 DOM.getBoxModel
   4 DOM.getBoxModel
  3 DOM.setFileInputFiles
</pre>
=== Events ===
These events are emitted during a gutenberg test run. Gutenberg tests don't explicitly listen to any CDP events.
<pre>
cat gutenberg-logs | grep -v '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
92569 Network.dataReceived
92363 Network.requestWillBeSent
92274 Network.loadingFinished
83137 Network.responseReceived
68170 Network.requestServedFromCache
24018 Network.responseReceivedExtraInfo
8669 Page.lifecycleEvent
6573 Network.requestWillBeSentExtraInfo
3597 Target.targetInfoChanged
2113 Runtime.consoleAPICalled
1626 Runtime.executionContextCreated
1456 Runtime.executionContextDestroyed
1104 Fetch.requestPaused
755 Page.navigatedWithinDocument
733 Page.frameStoppedLoading
731 Page.frameStartedLoading
730 Page.frameNavigated
725 Page.loadEventFired
724 Page.domContentEventFired
717 Runtime.executionContextsCleared
555 Target.targetCreated
267 Page.javascriptDialogOpening
267 Page.javascriptDialogClosed
241 Target.targetDestroyed
177 Page.frameResized
  83 Target.attachedToTarget
  81 Target.detachedFromTarget
  78 Inspector.detached
  56 Network.resourceChangedPriority
  35 Page.frameScheduledNavigation
  35 Page.frameRequestedNavigation
  35 Page.frameClearedScheduledNavigation
  13 Page.frameAttached
  11 Page.frameDetached
  8 Log.entryAdded
   3 Page.windowOpen
   3 Page.windowOpen
   3 Network.loadingFailed
   3 Network.loadingFailed
  3 DOM.setFileInputFiles
  2 Network.emulateNetworkConditions
</pre>
</pre>

Latest revision as of 20:09, 26 February 2021

Usage of CDP methods in https://github.com/WordPress/gutenberg end-to-end tests.


Running the gutenberg tests against Firefox

Prerequisites

The following is a quickstart based on Gutenberg's docs.

  1. Make sure Docker is running
  2. nvm install in gutenberg dir to ensure correct node version is installed
  3. npm ci -- you might see some warnings, this is fine.
  4. npm run build
  5. Build local env: npm run wp-env install. Avoid running npm run dev; takes a long time, not useful.
  6. Start the environment: npm run wp-env start

To stop the local WordPress instance: npm run env stop

Firefox prerequisites

Update the config in packages/scripts/config/puppeteer.config.js: 

module.exports = {
    launch: {
        headless: false,
        slowMo: 0,
        executablePath: 'path/to/firefox',
    },
};

Note: Puppeteer can download and install Firefox Nightly and you can use the path it installs into for the executablePath. For example, on macOS: path/to/gutenberg/node_modules/puppeteer/.local-firefox/mac-80/Firefox\ Nightly.app/Contents/MacOS/firefox

  1. Copy Firefox's modifications of Puppeteer:
cp -r <mozilla-central>/remote/test/puppeteer/ node_modules/puppeteer/
npm run build

Running tests

Select Firefox by setting PUPPETEER_PRODUCT, otherwise Chromium will be used. `DEBUG=*` will show all debug logs, or you can filter it as in the example below: 

PUPPETEER_PRODUCT=firefox DEBUG=puppeteer:* npm run test-e2e

To run a specific test: 

npm run test-e2e links.test.js

The --puppeteer-interactive option can be useful for debugging: 

npm run test-e2e links.test.js -- --puppeteer-interactive

Equivalent to PUPPETEER_HEADLESS=false PUPPETEER_SLOWMO=80 npm run test-e2e:watch

Simple tests to consider

  • mentions.tests.js
  • post-visibility.test.js
  • manage-reusable-blocks.test.js
  • style-variation-test.js

Methods

From https://bugzilla.mozilla.org/show_bug.cgi?id=1539202#c9: 

$ DEBUG=puppeteer:* npm run test-e2e packages/e2e-tests/specs/ > gutenberg-logs 2>&1
$ cat gutenberg-logs  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq


  • DOM.getBoxModel
  • DOM.getContentQuads
  • DOM.setFileInputFiles
  • Emulation.setDeviceMetricsOverride
  • Emulation.setTouchEmulationEnabled
  • Input.dispatchKeyEvent
  • Input.dispatchMouseEvent
  • Inspector.detached
  • Log.enable
  • Log.entryAdded
  • Network.continueInterceptedRequest
  • Network.dataReceived
  • Network.emulateNetworkConditions
  • Network.enable
  • Network.loadingFailed
  • Network.loadingFinished
  • Network.requestIntercepted
  • Network.requestServedFromCache
  • Network.requestWillBeSent
  • Network.resourceChangedPriority
  • Network.responseReceived
  • Network.setCacheDisabled
  • Network.setRequestInterception
  • Page.bringToFront
  • Page.domContentEventFired
  • Page.enable
  • Page.frameAttached
  • Page.frameClearedScheduledNavigation
  • Page.frameDetached
  • Page.frameNavigated
  • Page.frameResized
  • Page.frameScheduledNavigation
  • Page.frameStartedLoading
  • Page.frameStoppedLoading
  • Page.getFrameTree
  • Page.handleJavaScriptDialog
  • Page.javascriptDialogClosed
  • Page.javascriptDialogOpening
  • Page.lifecycleEvent
  • Page.loadEventFired
  • Page.navigate
  • Page.navigatedWithinDocument
  • Page.reload
  • Page.setLifecycleEventsEnabled
  • Page.windowOpen
  • Performance.enable
  • Runtime.callFunctionOn
  • Runtime.consoleAPICalled
  • Runtime.enable
  • Runtime.evaluate
  • Runtime.executionContextCreated
  • Runtime.executionContextDestroyed
  • Runtime.executionContextsCleared
  • Runtime.getProperties
  • Runtime.releaseObject
  • Security.enable
  • Security.securityStateChanged
  • Target.setAutoAttach

Methods & Events logged during test setup

Methods

 cat visit-admin-page.log create-new-post.log | grep '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
  52 Runtime.callFunctionOn
  52 Input.dispatchKeyEvent
  16 Runtime.releaseObject
  11 Runtime.evaluate
   9 Input.dispatchMouseEvent
   8 Page.navigate
   6 Emulation.setDeviceMetricsOverride
   4 Target.setDiscoverTargets
   3 Page.getLayoutMetrics
   3 DOM.getContentQuads
   2 Target.setAutoAttach
   2 Target.getBrowserContexts
   2 Target.createTarget
   2 Target.closeTarget
   2 Target.attachToTarget
   2 Runtime.enable
   2 Performance.enable
   2 Page.setInterceptFileChooserDialog
   2 Page.getFrameTree
   2 Page.enable
   2 Page.createIsolatedWorld
   2 Page.addScriptToEvaluateOnNewDocument
   2 Network.enable
   2 Log.enable
   1 DOM.resolveNode
   1 DOM.describeNode

Events

cat visit-admin-page.log create-new-post.log | grep -v '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
 820 Network.dataReceived
 576 Network.requestWillBeSent
 571 Network.loadingFinished
 560 Network.responseReceived
 298 Network.responseReceivedExtraInfo
 275 Network.requestServedFromCache
 273 Network.requestWillBeSentExtraInfo
 110 Page.lifecycleEvent
  53 Target.targetInfoChanged
  24 Runtime.executionContextCreated
  20 Runtime.executionContextDestroyed
  16 Target.targetCreated
  14 Runtime.consoleAPICalled
  10 Runtime.executionContextsCleared
  10 Page.loadEventFired
  10 Page.frameNavigated
  10 Page.frameStoppedLoading
  10 Page.frameStartedLoading
  10 Page.domContentEventFired
   8 Page.navigatedWithinDocument
   6 Emulation.setTouchEmulationEnabled
   6 Page.frameResized
   5 Target.targetDestroyed
   3 Network.resourceChangedPriority
   3 Log.entryAdded
   2 Target.detachedFromTarget
   2 Target.attachedToTarget
   2 Page.setLifecycleEventsEnabled
   2 Page.frameScheduledNavigation
   2 Page.frameRequestedNavigation
   2 Page.frameClearedScheduledNavigation
   2 Inspector.detached

Methods & Events ranked by frequency

Note that Puppeteer doesn't necessarily depend on many of the Network, Runtime, Fetch and Page items.

Methods

cat gutenberg-logs | grep '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r

15594 Input.dispatchKeyEvent
12738 Runtime.callFunctionOn
7324 Input.dispatchMouseEvent
3088 Runtime.releaseObject
1893 Page.getLayoutMetrics
1893 DOM.getContentQuads
1101 Runtime.evaluate
1091 Fetch.continueRequest
 977 DOM.resolveNode
 977 DOM.describeNode
 636 Emulation.setTouchEmulationEnabled
 636 Emulation.setDeviceMetricsOverride
 595 Page.navigate
 496 Runtime.getProperties
 267 Page.handleJavaScriptDialog
  83 Target.setAutoAttach
  83 Target.attachToTarget
  83 Runtime.enable
  83 Performance.enable
  83 Page.setLifecycleEventsEnabled
  83 Page.setInterceptFileChooserDialog
  83 Page.getFrameTree
  83 Page.enable
  83 Page.createIsolatedWorld
  83 Page.addScriptToEvaluateOnNewDocument
  83 Network.enable
  83 Log.enable
  81 Target.closeTarget
  80 Target.setDiscoverTargets
  79 Target.getBrowserContexts
  79 Target.createTarget
  35 Page.reload
  23 Input.insertText
  14 Network.setCacheDisabled
  12 Fetch.fulfillRequest
   8 Page.bringToFront
   8 Fetch.enable
   6 Fetch.disable
   4 Network.emulateNetworkConditions
   4 DOM.getBoxModel
   3 DOM.setFileInputFiles

Events

These events are emitted during a gutenberg test run. Gutenberg tests don't explicitly listen to any CDP events. 

cat gutenberg-logs | grep -v '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r

92569 Network.dataReceived
92363 Network.requestWillBeSent
92274 Network.loadingFinished
83137 Network.responseReceived
68170 Network.requestServedFromCache
24018 Network.responseReceivedExtraInfo
8669 Page.lifecycleEvent
6573 Network.requestWillBeSentExtraInfo
3597 Target.targetInfoChanged
2113 Runtime.consoleAPICalled
1626 Runtime.executionContextCreated
1456 Runtime.executionContextDestroyed
1104 Fetch.requestPaused
 755 Page.navigatedWithinDocument
 733 Page.frameStoppedLoading
 731 Page.frameStartedLoading
 730 Page.frameNavigated
 725 Page.loadEventFired
 724 Page.domContentEventFired
 717 Runtime.executionContextsCleared
 555 Target.targetCreated
 267 Page.javascriptDialogOpening
 267 Page.javascriptDialogClosed
 241 Target.targetDestroyed
 177 Page.frameResized
  83 Target.attachedToTarget
  81 Target.detachedFromTarget
  78 Inspector.detached
  56 Network.resourceChangedPriority
  35 Page.frameScheduledNavigation
  35 Page.frameRequestedNavigation
  35 Page.frameClearedScheduledNavigation
  13 Page.frameAttached
  11 Page.frameDetached
   8 Log.entryAdded
   3 Page.windowOpen
   3 Network.loadingFailed
Retrieved from "https://wiki.mozilla.org/index.php?title=WebDriver/RemoteProtocol/GutenbergCDPUsage&oldid=1234085"