AMO:Editors/EditorGuide/ThemeReviews

Firefox Theme Tests

These review guidelines are in addition to standard add-on review guidelines found at: https://wiki.mozilla.org/AMO:Editors/EditorGuide/AddonReviews

• Testing of themes should be done on a new profile or a profile dedicated to testing themes. With a dedicated theme testing profile you can speed up testing by setting the following general startup options:
  1. When Firefox starts: Show my windows and tabs from the last time
  2. Check: Don't load tabs until selected
  Also, set the about:config settings instructed in "Test Pages" below
• When testing themes make sure to disable any extensions that may be installed in the profile (other than DOM Inspector, which you may need).
• Themes have limited capabilities to run code, but this is generally not allowed. Make sure to run the validator to see if there is any JS in the theme. If there is, it should be flagged for Admin Reviewed.
• Make sure the theme looks good. Text should be easy to read and buttons should be clear and concise.
• As well as just looking stylish, make sure it all looks stylish in a consistent way.
• Another important aspect of a theme is to make sure it "fits." This is relevant for menus and dialogs - check to make sure that all the UI elements fit within the default boundaries and do not appear "clipped". If anything's clipped and/or if resizing is required to see it completely, make a note of it and request the author to change it.
• While testing a theme, don't forget to check that the author has included a suitable preview image.

OS Specific Issues

Linux

• Unstyled Firefox Button: A very common issue themes have on Linux is improperly or totally unstyled Firefox button, and the Firefox button not changing styling when in private browsing mode. The primary cause of this issue is that the Firefox button has a different ID in Linux from Windows. As a result, in the browser.css file any reference to #appmenu-button also needs a comparable reference to #appmenu-toolbar-button.
• Strong Black borders Around URL Bar Door Hangers (drop down panels): Many Windows based themes make use of the box-shadow style rule, however, it is not supported on Linux Firefox and resulting in the strong black border. If the strong black border is noticed, it should be noted to the developer and they should be encouraged to make sure any box-shadow style rules are targeted only to Windows.

OS-X

If you test any theme that is supposed to support OS-X please make sure that if the following files exist in "chrome://browser/skin/" they were also copied to "chrome://browser/skin/lion/":

• keyhole-circle.png
• Toolbar.png
• toolbarbutton-dropmarker.png
• tabbrowser/alltabs-box-bkgnd-icon.png
• tabview/tabview.png
• places/toolbar.png

The other solution theme developers can do is rename the files to something non-default and then make the proper file mappings in the theme's CSS files. NOTE: This issue doesn't require OS-X to test, you only need to look in the theme contents (via the "contents" or "compare" links) to see if the files in question exist. For more information see Bug 679708 and Bug 702558

Special Considerations

Windows and OS-X use different CSS rules to generate OS native scrollbars. As a result any cross OS theme either needs to target different scrollbar rules for each OS or use custom XUL scrollbars.

Test Dialogs

As soon as Firefox has started with theme to be tested installed, open the error console and make notes about any "skin" related CSS errors. Use the add-on Console%C2%B2 to replace the standard error console as it will allow you copy all errors at once making it easier to past those errors into review notes. It also allows you to disable content errors thus eliminating a lot of webpage related noise. Instructions assume the menubar is enabled.

Dialog/Feature	Policy on failure	What to test
View > Toolbars > Customize	Preliminary Review Preliminary Review Add Note	Make sure customize toolbar dialog is formatted reasonably. With the "show" options set to "icons", check and uncheck "use small icons", toolbar icons needs to display properly in both states (ideally they will be different sizes). Repeat test using show "icons and text". With "use small icons" checked, change "show" to "text". Make sure toolbar button height is reasonable for text only buttons and that the text is vertically aligned in middle of buttons. If it is not provide this note or something similar in the review: When text only toolbar buttons are selected in customize toolbars, text labels in toolbar buttons are not aligning properly. The common cause of this is style rules that reference [iconsize="small"] without excluding [mode="text"]. The solution usually is to append :not([mode="text"]) to those style rules. For instance use 'toolbar[iconsize="small"]:not([mode="text"])'. A little experimenting may be required to fully address this issue with your theme.
Check 'View > Toolbars > Bookmarks Toolbar'	Preliminary Review	Make sure toolbar is formatted reasonably.
Check 'View > Toolbars > Add-on Toolbar'	Preliminary Review	Using customize toolbars, add a toolbar button to this toolbar (at the bottom of the browser window). Make sure toolbar is formatted reasonably.
Check 'View > Toolbars > Tabs on Top' and uncheck 'View > Toolbars > Tabs on Top'	Preliminary Review Preliminary Review	Make sure Firefox button is formatted reasonably. On Windows, with browser window maximized, make sure tabs display properly between Firefox button and right hand window control buttons. Go to private browsing mode (Firefox > Start Private Browsing), and verify that Firefox button changes appearance when in private browsing mode on Windows. Exit private browsing mode, enable menu bar (check Firefox > Options/Preferences > Menu Bar) and uncheck "tabs on top".
'Edit > Find'	Preliminary Review	Make sure find toolbar is formatted reasonably.

Test Pages

All of the following URLs should be tested. If you are testing a series of themes, you might want to open each of the links/items below into their own tab and set Firefox startup options to "show my windows and tabs from last time". You might also want to check the option "don't load tabs until selected" as this will help with checking by CSS errors by allowing you to only load "about:addons" after restarting from installing the theme, thus the only CSS errors in the error console should be related to the theme. In regards to policies on failure, just add notes for minor formatting issues rather than rejecting or granting only a preliminary review (unless developer hasn't fixed issue after fair warning).

URL	Policy on failure	What to test
about:addons	Preliminary Review Preliminary Review Add Note	Make sure formatting is reasonable. Addon type tabs need distinct icons. Make sure there is a significant difference between enabled and disabled addons. It is strongly encouraged but not required that the icons of disabled addons be greyed out. Pin this tab to make sure pinned tabs work correctly. Assuming that this was the only tab that loaded per the instructions above, open the error console and see if there are any CSS errors/warnings.
about:home (<FF8)	Preliminary Review	Make sure page is formatted reasonably.
about:config	Add Note	Make sure it is formatted correctly. While in about config, set the following keys to aid later testing: geo.enabled=true (bool) browser.syncPromoViewsLeft=1000 (integer) security.warn_viewing_mixed=true (bool) security.warn_viewing_mixed.show_once=false (bool)
about:permissions (FF6+)	Add Note	Make sure it is formatted correctly. While in about:permissions, make sure that if there are entries for them, the following domains have the necessary permissions: addons.mozilla.org: Store password = Allow www.mozilla.org: Share location = Always ask
AMO Addon	Reject Preliminary Review Preliminary Review Preliminary Review Add Note Add Note	Test Identity box with corporate name and distinctive color. Make sure identity drop down displays correctly including required security icons. The default theme uses green for the button, but this is not required. Test RSS feed icon. Place RSS button on toolbar and make sure its appearance is different for pages that have RSS feeds from those that don't. If you don't have a live bookmark on your bookmark toolbar, add this page's feed to it. Test print preview. Make sure it is formatted reasonably. Test Fullscreen mode (F11). Make sure it is formatted reasonably and functions correctly. Log in to AMO (using a fake username/password is okay), make sure the save password panel is formatted reasonably and includes an appropriate key icon. On the save password dropdown from the identity box there is a sync "door hanger" it needs a proper border. See Bug 708797 Double click on the bookmark star in the URL bar. The edit bookmark box will appear there is a sync "door hanger" make sure the edit bookmark panel and sync door hanger are formatted reasonably
Bugzilla	Reject Preliminary Review	Identity box with just domain name and different distinctive color. Make sure identity drop down displays correctly. The default theme uses blue for the button, but this is not required. Popup window. Click on "click me" link. Make sure URL bar in popup window is formated as read only.
About Mozilla	Reject Preliminary Review	Non-encrypted objects on encrypted page warning (in about:config, the key security.warn_viewing_mixed must be true), there should be a popup warning about non-encrypted data and the identity box must NOT look like it is secure (checking the checkbox on the alert will ensure it triggers in the future without having to set about:config setting). If page looks secure double check page using default theme to make sure webpage wasn't fixed. Use video controls in embedded HTML5 video to make sure they look reasonable and function correctly.
Mozilla geolocation Test	Preliminary Review	Click on link labeled "Give it a try" and then the "where am I" button to activate geolocation alert. Make sure it displays correctly.
Google PageSpeed extension download	Preliminary Review	On page click on "install page speed link and make sure a prompt appears in identity box verifying whether or not to install extension.
Mozilla phishing test page	Preliminary Review	Phishing warning test page. Check notification strip after ignoring phishing warning.
Mozilla malware test page	Preliminary Review	Malware warning page. Check notification strip after ignoring malware warning
https://63.245.209.11/	Preliminary Review	Untrusted connection alert. Click on identity box to test drop down details.
jar:https://bugzilla.mozilla.org/attachment.cgi?id=288634!/	Preliminary Review	Unsafe Remote File Type. Access denied. Make sure it is formatted reasonably.
Mozilla FTP Archive	Preliminary Review	FTP. Make sure it is formatted reasonably.
For Linux use: file:/// For Windows use:file:///c:/	Preliminary Review	Local File System. Make sure it displays correctly. (Mac users may have to adapt file path)
chrome://global/content/bindings/textbox.xml	Add Note	Unformatted XML Preview. Make sure it is formatted reasonably.
about:memory	Add Note	Make sure it is formatted reasonably.

Widgets / Dialogs

• Add a livebookmark to your bookmarks toolbar if you don't have one.
• Check all the icons are there under View -> Toolbars -> Customize.
• Tab strip scrolling (open enough tabs to force the tab script to scroll) - make sure it scrolls properly and has scroll arrows at each end of tab strip.
• Pinned Tabs (verify pinned tabs function correctly)
• Tab Groups / Panorama
• Side bar (ctrl+B)
• Customize toolbar (verify small and large icons function properly)
• Customize toolbar (verify show: icons, icons and text, & text options function properly for both large and small icons)
• Places (All History / All Bookmarks) - some themes break the toolbar buttons in these when set to small / large
• Turn on all toolbars
• Place tabs on top and disable menu bar, maximize and restore browser window. Ensure tabs display in title bar correctly when browser window maximized (Windows/Linux)
• When menu bar is turned off, make sure window controls on right side of title bar (min/max/restore/close) display and function correctly
• Web Developer tools (Web console, Inspect, Scratchpad, etc.)
• Page Info
• Private Browsing
• Option. Make sure Options / Advanced / Network / Limit Cache > spin buttons are styled as well!
• About Firefox - the logo should not be replaced (but a custom made logo can be added)

Usability Tests

The Themes should be usable without Mouse - widgets have to change their appearance when tabbed through them with the keyboard. (e.g. checkboxes, radio buttons, tabs, should indicate key focus). Many of these tests can be conducted in conjunction with other tests above.

• Tab Strip: Tab through tab strip using Ctrl+Tab. Active tab should be visually different from other tabs.
• Group Tabs: Activate group tabs using Ctrl+Shift+E the highlighted tab should be visually different from other tabs use the up/down/left/right arrows to navigate from tab to tab and then press enter to open the desired tab.
• Menu Bar: Activate the file menu using Alt+F and then use keyboard to navigate through menus and sub-menus. The selected menu item must be visually different and the highlighted text must be legible.
• Places: In the places (History/Bookmarks) panel tab around and use keyboard arrows to navigate. Make sure it is visible what the selected item is.
• Options: Open the options panel and make sure you can navigate around the options panel and that you know what has focus. You must be able to change tabs, and settings from the keyboard.

Special Screen States

• Full screen mode - there must be restore and minimize buttons!
• Tabs on Top - Tabs may be above the Address bar or on the Caption of the Window, this is a matter of choice for the Theme author
• Try to move the search box to a different toolbar (e.g. menu bar); this is what people with narrow screens would do
• Sync Now

Next: The Mailing List