User:Bhashem/AddonAuditTools

From MozillaWiki
Jump to: navigation, search

Here is my braindump of what some add-on audit tools might do. It's based on the AMO add-on review process.

To Do

  • Add some priorities and which ones will give the best bang for the effort.
  • Encode some best practices
  • This list can probably be enhanced by examining the AMO Submit Manager as well as Firefox's Add-ons Manager code for semantic/syntactic checks that can be externalized as tools.

Summary

Overall Goal: Help authors publish high-quality, secure add-ons

  • Provide tools to add-on authors so that they "lint before they submit"
  • Provide AMO Editors tools to help them evaluate add-on quality
  • Smooth the AMO review process
  • Potentially use these tools to help with JS refactoring (for add-ons or JS components of Firefox)

General Techniques

  • A: Use structural analysis to uncover errors and characterize add-ons
  • B: Use language static analysis for syntactic and semantic checking of an add-on
  • C: Use run-time analysis to characterize what an add-on is doing?

A: Structural Analysis

  • Review the File/Directory Structure
    • Is it a valid XPI or JAR? (for themes)
    • Is jar packaging OK? (missing basic files, etc...)
    • Are there duplicate files/directories (common mistake is to include chrome/myadd.jar + all the files at the same level - they are never used)
    • Do there appear to be binary components or executables?
    • Does it appear to have OS platform specific files?
    • Is there orphaned content/skins/chrome/etc?

B: Static Analysis

Extension Type-Specific rules

  • Is only allowed content included?
    • For Dictionaries - e.g., no JS files!
    • For Themes
    • For Locale Packs
  • Look for particular regex's or specific values for prefs and in js that might be used incorrectly.

File type: Install.rdf

  • Syntactically correct RDF
  • Semantic errors - e.g. invalid version ranges, invalid RDF references, unknown app GUIDs, invalid add-on GUID formats, etc...

File type: Chrome.manifest

  • Syntax errors
  • Are there dangling references in chrome.manifest?

File type: .js

  • Passes Jslint with selected options
  • Namespacing concerns
  • Does it make use of XHR?
  • Does it load content in chrome?
  • Does it generate JavaScript errors on invocation/functional use?
  • Does it use eval for evil?
  • Does it load remote JS?
  • Does it load remote XUL?

File type: .xul

File type: .css

  • TBD

C: Run-time Analysis

  • Use some sort of specially instrumented Firebug add-on or a JS Shell
  • Does it leak memory?
  • Does it load remote JS?
  • Does it load remote XUL?
  • Does it load non-https content as chrome?
  • Automated testing frameworks (a la JSUnit but for chrome)