User:Bhashem/AddonAuditTools
From MozillaWiki
Here is my braindump of what some add-on audit tools might do. It's based on the AMO add-on review process.
To Do
- Add some priorities and which ones will give the best bang for the effort.
- Encode some best practices
- This list can probably be enhanced by examining the AMO Submit Manager as well as Firefox's Add-ons Manager code for semantic/syntactic checks that can be externalized as tools.
Summary
Overall Goal: Help authors publish high-quality, secure add-ons
- Provide tools to add-on authors so that they "lint before they submit"
- Provide AMO Editors tools to help them evaluate add-on quality
- Smooth the AMO review process
- Potentially use these tools to help with JS refactoring (for add-ons or JS components of Firefox)
General Techniques
- A: Use structural analysis to uncover errors and characterize add-ons
- B: Use language static analysis for syntactic and semantic checking of an add-on
- C: Use run-time analysis to characterize what an add-on is doing?
A: Structural Analysis
- Review the File/Directory Structure
- Is it a valid XPI or JAR? (for themes)
- Is jar packaging OK? (missing basic files, etc...)
- Are there duplicate files/directories (common mistake is to include chrome/myadd.jar + all the files at the same level - they are never used)
- Do there appear to be binary components or executables?
- Does it appear to have OS platform specific files?
- Is there orphaned content/skins/chrome/etc?
B: Static Analysis
Extension Type-Specific rules
- Is only allowed content included?
- For Dictionaries - e.g., no JS files!
- For Themes
- For Locale Packs
- Look for particular regex's or specific values for prefs and in js that might be used incorrectly.
File type: Install.rdf
- Syntactically correct RDF
- Semantic errors - e.g. invalid version ranges, invalid RDF references, unknown app GUIDs, invalid add-on GUID formats, etc...
File type: Chrome.manifest
- Syntax errors
- Are there dangling references in chrome.manifest?
File type: .js
- Passes Jslint with selected options
- Namespacing concerns
- Does it make use of XHR?
- Does it load content in chrome?
- Does it generate JavaScript errors on invocation/functional use?
- Does it use eval for evil?
- Does it load remote JS?
- Does it load remote XUL?
File type: .xul
File type: .css
- TBD
C: Run-time Analysis
- Use some sort of specially instrumented Firebug add-on or a JS Shell
- Does it leak memory?
- Does it load remote JS?
- Does it load remote XUL?
- Does it load non-https content as chrome?
- Automated testing frameworks (a la JSUnit but for chrome)