WebDriver/RemoteProtocol/GutenbergCDPUsage: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
m (→‎Events: Clarify)
m (Whimboo moved page Remote/GutenbergCDPUsage to WebDriver/RemoteProtocol/GutenbergCDPUsage: Reorganization for WebDriver BiDi project documenation)
 
(10 intermediate revisions by 3 users not shown)
Line 2: Line 2:


__FORCETOC__
__FORCETOC__
== Running the gutenberg tests against Firefox ==
=== Prerequisites ===
The following is a quickstart based on Gutenberg's [https://github.com/WordPress/gutenberg/blob/master/docs/contributors/testing-overview.md#end-to-end-testing docs].
# Make sure Docker is running
# <code>nvm install</code> in gutenberg dir to ensure correct node version is installed
# <code>npm ci</code> -- you might see some warnings, this is fine.
# <code>npm run build</code>
# Build local env: <code>npm run wp-env install</code>. Avoid running <code>npm run dev</code>; takes a long time, not useful.
# Start the environment: <code>npm run wp-env start</code>
To stop the local WordPress instance: <code>npm run env stop</code>
=== Firefox prerequisites ===
Update the config in <code>packages/scripts/config/puppeteer.config.js</code>:
<pre>
module.exports = {
    launch: {
        headless: false,
        slowMo: 0,
        executablePath: 'path/to/firefox',
    },
};
</pre>
<small>Note: Puppeteer can download and install Firefox Nightly and you can use the path it installs into for the executablePath. For example, on macOS: <code>path/to/gutenberg/node_modules/puppeteer/.local-firefox/mac-80/Firefox\ Nightly.app/Contents/MacOS/firefox</code></small>
# Copy Firefox's modifications of Puppeteer:
<pre>
cp -r <mozilla-central>/remote/test/puppeteer/ node_modules/puppeteer/
npm run build
</pre>
=== Running tests ===
Select Firefox by setting <code>PUPPETEER_PRODUCT</code>, otherwise Chromium will be used. `DEBUG=*` will show all debug logs, or you
can filter it as in the example below:
<pre>
PUPPETEER_PRODUCT=firefox DEBUG=puppeteer:* npm run test-e2e
</pre>
To run a specific test:
<pre>
npm run test-e2e links.test.js
</pre>
The <code>--puppeteer-interactive</code> option can be useful for debugging:
<pre>
npm run test-e2e links.test.js -- --puppeteer-interactive
</pre>
Equivalent to <code>PUPPETEER_HEADLESS=false PUPPETEER_SLOWMO=80 npm run test-e2e:watch</code>
=== Simple tests to consider ===
* mentions.tests.js
* post-visibility.test.js
* manage-reusable-blocks.test.js
* style-variation-test.js


== Methods ==
== Methods ==
Line 72: Line 140:
* Target.setAutoAttach
* Target.setAutoAttach


== Methods & Events logged during test setup ==
=== Methods ===
<pre>
cat visit-admin-page.log create-new-post.log | grep '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
  52 Runtime.callFunctionOn
  52 Input.dispatchKeyEvent
  16 Runtime.releaseObject
  11 Runtime.evaluate
  9 Input.dispatchMouseEvent
  8 Page.navigate
  6 Emulation.setDeviceMetricsOverride
  4 Target.setDiscoverTargets
  3 Page.getLayoutMetrics
  3 DOM.getContentQuads
  2 Target.setAutoAttach
  2 Target.getBrowserContexts
  2 Target.createTarget
  2 Target.closeTarget
  2 Target.attachToTarget
  2 Runtime.enable
  2 Performance.enable
  2 Page.setInterceptFileChooserDialog
  2 Page.getFrameTree
  2 Page.enable
  2 Page.createIsolatedWorld
  2 Page.addScriptToEvaluateOnNewDocument
  2 Network.enable
  2 Log.enable
  1 DOM.resolveNode
  1 DOM.describeNode
</pre>
=== Events ===
<pre>
cat visit-admin-page.log create-new-post.log | grep -v '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
820 Network.dataReceived
576 Network.requestWillBeSent
571 Network.loadingFinished
560 Network.responseReceived
298 Network.responseReceivedExtraInfo
275 Network.requestServedFromCache
273 Network.requestWillBeSentExtraInfo
110 Page.lifecycleEvent
  53 Target.targetInfoChanged
  24 Runtime.executionContextCreated
  20 Runtime.executionContextDestroyed
  16 Target.targetCreated
  14 Runtime.consoleAPICalled
  10 Runtime.executionContextsCleared
  10 Page.loadEventFired
  10 Page.frameNavigated
  10 Page.frameStoppedLoading
  10 Page.frameStartedLoading
  10 Page.domContentEventFired
  8 Page.navigatedWithinDocument
  6 Emulation.setTouchEmulationEnabled
  6 Page.frameResized
  5 Target.targetDestroyed
  3 Network.resourceChangedPriority
  3 Log.entryAdded
  2 Target.detachedFromTarget
  2 Target.attachedToTarget
  2 Page.setLifecycleEventsEnabled
  2 Page.frameScheduledNavigation
  2 Page.frameRequestedNavigation
  2 Page.frameClearedScheduledNavigation
  2 Inspector.detached
</pre>


== Methods & Events ranked by frequency ==
== Methods & Events ranked by frequency ==
Line 82: Line 221:
cat gutenberg-logs | grep '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
cat gutenberg-logs | grep '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r


14924 Input.dispatchKeyEvent
15594 Input.dispatchKeyEvent
12462 Runtime.callFunctionOn
12738 Runtime.callFunctionOn
7709 Input.dispatchMouseEvent
7324 Input.dispatchMouseEvent
3107 Runtime.releaseObject
3088 Runtime.releaseObject
1891 Page.getLayoutMetrics
1893 Page.getLayoutMetrics
1891 DOM.getContentQuads
1893 DOM.getContentQuads
1514 Runtime.executionContextCreated
1101 Runtime.evaluate
1019 Runtime.evaluate
1091 Fetch.continueRequest
  947 DOM.resolveNode
  977 DOM.resolveNode
  947 DOM.describeNode
  977 DOM.describeNode
  881 Fetch.continueRequest
  636 Emulation.setTouchEmulationEnabled
676 Page.frameNavigated
  636 Emulation.setDeviceMetricsOverride
598 Emulation.setTouchEmulationEnabled
  595 Page.navigate
  598 Emulation.setDeviceMetricsOverride
  496 Runtime.getProperties
  560 Page.navigate
  267 Page.handleJavaScriptDialog
  529 Runtime.getProperties
   83 Target.setAutoAttach
  238 Page.handleJavaScriptDialog
   83 Target.attachToTarget
   81 Target.setAutoAttach
   83 Runtime.enable
   81 Target.attachToTarget
   83 Performance.enable
   81 Runtime.enable
   83 Page.setLifecycleEventsEnabled
   81 Performance.enable
   83 Page.setInterceptFileChooserDialog
   81 Page.setLifecycleEventsEnabled
   83 Page.getFrameTree
   81 Page.setInterceptFileChooserDialog
   83 Page.enable
   81 Page.getFrameTree
   83 Page.createIsolatedWorld
   81 Page.enable
   83 Page.addScriptToEvaluateOnNewDocument
   81 Page.createIsolatedWorld
   83 Network.enable
   81 Page.addScriptToEvaluateOnNewDocument
   83 Log.enable
   81 Network.enable
   81 Target.closeTarget
   81 Log.enable
   80 Target.setDiscoverTargets
   79 Target.closeTarget
   79 Target.getBrowserContexts
   78 Target.setDiscoverTargets
   79 Target.createTarget
   77 Target.getBrowserContexts
   35 Page.reload
   77 Target.createTarget
   23 Input.insertText
   28 Page.reload
   22 Input.insertText
   14 Network.setCacheDisabled
   14 Network.setCacheDisabled
   12 Fetch.fulfillRequest
   12 Fetch.fulfillRequest
Line 122: Line 259:
   8 Fetch.enable
   8 Fetch.enable
   6 Fetch.disable
   6 Fetch.disable
  4 Network.emulateNetworkConditions
   4 DOM.getBoxModel
   4 DOM.getBoxModel
   3 DOM.setFileInputFiles
   3 DOM.setFileInputFiles
  2 Network.emulateNetworkConditions
</pre>
</pre>


Line 134: Line 271:
cat gutenberg-logs | grep -v '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
cat gutenberg-logs | grep -v '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r


65119 Network.dataReceived
92569 Network.dataReceived
56977 Network.requestWillBeSent
92363 Network.requestWillBeSent
56906 Network.loadingFinished
92274 Network.loadingFinished
55864 Network.responseReceived
83137 Network.responseReceived
37106 Network.requestServedFromCache
68170 Network.requestServedFromCache
6232 Page.lifecycleEvent
24018 Network.responseReceivedExtraInfo
3367 Target.targetInfoChanged
8669 Page.lifecycleEvent
1510 Runtime.consoleAPICalled
6573 Network.requestWillBeSentExtraInfo
1346 Runtime.executionContextDestroyed
3597 Target.targetInfoChanged
893 Fetch.requestPaused
2113 Runtime.consoleAPICalled
  699 Page.navigatedWithinDocument
1626 Runtime.executionContextCreated
  681 Page.frameStoppedLoading
1456 Runtime.executionContextDestroyed
  678 Page.frameStartedLoading
1104 Fetch.requestPaused
  674 Page.domContentEventFired
  755 Page.navigatedWithinDocument
  666 Runtime.executionContextsCleared
  733 Page.frameStoppedLoading
  664 Page.loadEventFired
  731 Page.frameStartedLoading
  621 Target.targetCreated
  730 Page.frameNavigated
  238 Page.javascriptDialogOpening
  725 Page.loadEventFired
  238 Page.javascriptDialogClosed
  724 Page.domContentEventFired
  235 Target.targetDestroyed
717 Runtime.executionContextsCleared
  89 Page.frameResized
  555 Target.targetCreated
   81 Target.attachedToTarget
  267 Page.javascriptDialogOpening
   79 Target.detachedFromTarget
  267 Page.javascriptDialogClosed
   30 Page.frameScheduledNavigation
  241 Target.targetDestroyed
   30 Page.frameRequestedNavigation
177 Page.frameResized
   30 Page.frameClearedScheduledNavigation
   83 Target.attachedToTarget
   11 Network.resourceChangedPriority
   81 Target.detachedFromTarget
  10 Page.frameAttached
   78 Inspector.detached
  7 Page.frameDetached
  56 Network.resourceChangedPriority
   4 Log.entryAdded
  35 Page.frameScheduledNavigation
   35 Page.frameRequestedNavigation
   35 Page.frameClearedScheduledNavigation
   13 Page.frameAttached
  11 Page.frameDetached
   8 Log.entryAdded
   3 Page.windowOpen
   3 Page.windowOpen
   3 Network.loadingFailed
   3 Network.loadingFailed
</pre>
</pre>

Latest revision as of 20:09, 26 February 2021

Usage of CDP methods in https://github.com/WordPress/gutenberg end-to-end tests.


Running the gutenberg tests against Firefox

Prerequisites

The following is a quickstart based on Gutenberg's docs.

  1. Make sure Docker is running
  2. nvm install in gutenberg dir to ensure correct node version is installed
  3. npm ci -- you might see some warnings, this is fine.
  4. npm run build
  5. Build local env: npm run wp-env install. Avoid running npm run dev; takes a long time, not useful.
  6. Start the environment: npm run wp-env start

To stop the local WordPress instance: npm run env stop

Firefox prerequisites

Update the config in packages/scripts/config/puppeteer.config.js: 

module.exports = {
    launch: {
        headless: false,
        slowMo: 0,
        executablePath: 'path/to/firefox',
    },
};

Note: Puppeteer can download and install Firefox Nightly and you can use the path it installs into for the executablePath. For example, on macOS: path/to/gutenberg/node_modules/puppeteer/.local-firefox/mac-80/Firefox\ Nightly.app/Contents/MacOS/firefox

  1. Copy Firefox's modifications of Puppeteer:
cp -r <mozilla-central>/remote/test/puppeteer/ node_modules/puppeteer/
npm run build

Running tests

Select Firefox by setting PUPPETEER_PRODUCT, otherwise Chromium will be used. `DEBUG=*` will show all debug logs, or you can filter it as in the example below: 

PUPPETEER_PRODUCT=firefox DEBUG=puppeteer:* npm run test-e2e

To run a specific test: 

npm run test-e2e links.test.js

The --puppeteer-interactive option can be useful for debugging: 

npm run test-e2e links.test.js -- --puppeteer-interactive

Equivalent to PUPPETEER_HEADLESS=false PUPPETEER_SLOWMO=80 npm run test-e2e:watch

Simple tests to consider

  • mentions.tests.js
  • post-visibility.test.js
  • manage-reusable-blocks.test.js
  • style-variation-test.js

Methods

From https://bugzilla.mozilla.org/show_bug.cgi?id=1539202#c9: 

$ DEBUG=puppeteer:* npm run test-e2e packages/e2e-tests/specs/ > gutenberg-logs 2>&1
$ cat gutenberg-logs  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq


  • DOM.getBoxModel
  • DOM.getContentQuads
  • DOM.setFileInputFiles
  • Emulation.setDeviceMetricsOverride
  • Emulation.setTouchEmulationEnabled
  • Input.dispatchKeyEvent
  • Input.dispatchMouseEvent
  • Inspector.detached
  • Log.enable
  • Log.entryAdded
  • Network.continueInterceptedRequest
  • Network.dataReceived
  • Network.emulateNetworkConditions
  • Network.enable
  • Network.loadingFailed
  • Network.loadingFinished
  • Network.requestIntercepted
  • Network.requestServedFromCache
  • Network.requestWillBeSent
  • Network.resourceChangedPriority
  • Network.responseReceived
  • Network.setCacheDisabled
  • Network.setRequestInterception
  • Page.bringToFront
  • Page.domContentEventFired
  • Page.enable
  • Page.frameAttached
  • Page.frameClearedScheduledNavigation
  • Page.frameDetached
  • Page.frameNavigated
  • Page.frameResized
  • Page.frameScheduledNavigation
  • Page.frameStartedLoading
  • Page.frameStoppedLoading
  • Page.getFrameTree
  • Page.handleJavaScriptDialog
  • Page.javascriptDialogClosed
  • Page.javascriptDialogOpening
  • Page.lifecycleEvent
  • Page.loadEventFired
  • Page.navigate
  • Page.navigatedWithinDocument
  • Page.reload
  • Page.setLifecycleEventsEnabled
  • Page.windowOpen
  • Performance.enable
  • Runtime.callFunctionOn
  • Runtime.consoleAPICalled
  • Runtime.enable
  • Runtime.evaluate
  • Runtime.executionContextCreated
  • Runtime.executionContextDestroyed
  • Runtime.executionContextsCleared
  • Runtime.getProperties
  • Runtime.releaseObject
  • Security.enable
  • Security.securityStateChanged
  • Target.setAutoAttach

Methods & Events logged during test setup

Methods

 cat visit-admin-page.log create-new-post.log | grep '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
  52 Runtime.callFunctionOn
  52 Input.dispatchKeyEvent
  16 Runtime.releaseObject
  11 Runtime.evaluate
   9 Input.dispatchMouseEvent
   8 Page.navigate
   6 Emulation.setDeviceMetricsOverride
   4 Target.setDiscoverTargets
   3 Page.getLayoutMetrics
   3 DOM.getContentQuads
   2 Target.setAutoAttach
   2 Target.getBrowserContexts
   2 Target.createTarget
   2 Target.closeTarget
   2 Target.attachToTarget
   2 Runtime.enable
   2 Performance.enable
   2 Page.setInterceptFileChooserDialog
   2 Page.getFrameTree
   2 Page.enable
   2 Page.createIsolatedWorld
   2 Page.addScriptToEvaluateOnNewDocument
   2 Network.enable
   2 Log.enable
   1 DOM.resolveNode
   1 DOM.describeNode

Events

cat visit-admin-page.log create-new-post.log | grep -v '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r
 820 Network.dataReceived
 576 Network.requestWillBeSent
 571 Network.loadingFinished
 560 Network.responseReceived
 298 Network.responseReceivedExtraInfo
 275 Network.requestServedFromCache
 273 Network.requestWillBeSentExtraInfo
 110 Page.lifecycleEvent
  53 Target.targetInfoChanged
  24 Runtime.executionContextCreated
  20 Runtime.executionContextDestroyed
  16 Target.targetCreated
  14 Runtime.consoleAPICalled
  10 Runtime.executionContextsCleared
  10 Page.loadEventFired
  10 Page.frameNavigated
  10 Page.frameStoppedLoading
  10 Page.frameStartedLoading
  10 Page.domContentEventFired
   8 Page.navigatedWithinDocument
   6 Emulation.setTouchEmulationEnabled
   6 Page.frameResized
   5 Target.targetDestroyed
   3 Network.resourceChangedPriority
   3 Log.entryAdded
   2 Target.detachedFromTarget
   2 Target.attachedToTarget
   2 Page.setLifecycleEventsEnabled
   2 Page.frameScheduledNavigation
   2 Page.frameRequestedNavigation
   2 Page.frameClearedScheduledNavigation
   2 Inspector.detached

Methods & Events ranked by frequency

Note that Puppeteer doesn't necessarily depend on many of the Network, Runtime, Fetch and Page items.

Methods

cat gutenberg-logs | grep '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r

15594 Input.dispatchKeyEvent
12738 Runtime.callFunctionOn
7324 Input.dispatchMouseEvent
3088 Runtime.releaseObject
1893 Page.getLayoutMetrics
1893 DOM.getContentQuads
1101 Runtime.evaluate
1091 Fetch.continueRequest
 977 DOM.resolveNode
 977 DOM.describeNode
 636 Emulation.setTouchEmulationEnabled
 636 Emulation.setDeviceMetricsOverride
 595 Page.navigate
 496 Runtime.getProperties
 267 Page.handleJavaScriptDialog
  83 Target.setAutoAttach
  83 Target.attachToTarget
  83 Runtime.enable
  83 Performance.enable
  83 Page.setLifecycleEventsEnabled
  83 Page.setInterceptFileChooserDialog
  83 Page.getFrameTree
  83 Page.enable
  83 Page.createIsolatedWorld
  83 Page.addScriptToEvaluateOnNewDocument
  83 Network.enable
  83 Log.enable
  81 Target.closeTarget
  80 Target.setDiscoverTargets
  79 Target.getBrowserContexts
  79 Target.createTarget
  35 Page.reload
  23 Input.insertText
  14 Network.setCacheDisabled
  12 Fetch.fulfillRequest
   8 Page.bringToFront
   8 Fetch.enable
   6 Fetch.disable
   4 Network.emulateNetworkConditions
   4 DOM.getBoxModel
   3 DOM.setFileInputFiles

Events

These events are emitted during a gutenberg test run. Gutenberg tests don't explicitly listen to any CDP events. 

cat gutenberg-logs | grep -v '"id":'  | grep -oE '"method":"[^"]+\.[^"]+"' | sed 's/"method":"//' | sed 's/"//' | sort | uniq -c | sort -r

92569 Network.dataReceived
92363 Network.requestWillBeSent
92274 Network.loadingFinished
83137 Network.responseReceived
68170 Network.requestServedFromCache
24018 Network.responseReceivedExtraInfo
8669 Page.lifecycleEvent
6573 Network.requestWillBeSentExtraInfo
3597 Target.targetInfoChanged
2113 Runtime.consoleAPICalled
1626 Runtime.executionContextCreated
1456 Runtime.executionContextDestroyed
1104 Fetch.requestPaused
 755 Page.navigatedWithinDocument
 733 Page.frameStoppedLoading
 731 Page.frameStartedLoading
 730 Page.frameNavigated
 725 Page.loadEventFired
 724 Page.domContentEventFired
 717 Runtime.executionContextsCleared
 555 Target.targetCreated
 267 Page.javascriptDialogOpening
 267 Page.javascriptDialogClosed
 241 Target.targetDestroyed
 177 Page.frameResized
  83 Target.attachedToTarget
  81 Target.detachedFromTarget
  78 Inspector.detached
  56 Network.resourceChangedPriority
  35 Page.frameScheduledNavigation
  35 Page.frameRequestedNavigation
  35 Page.frameClearedScheduledNavigation
  13 Page.frameAttached
  11 Page.frameDetached
   8 Log.entryAdded
   3 Page.windowOpen
   3 Network.loadingFailed
Retrieved from "https://wiki.mozilla.org/index.php?title=WebDriver/RemoteProtocol/GutenbergCDPUsage&oldid=1234085"