User:Clouserw/AMO/Validator:v2

From MozillaWiki
Jump to navigation Jump to search

Version 1 of our add-on validator has been a great success and really proven the value of an automated validation system. This year we need to take it to the next level with some new features, finishing some rough edges, and making the code rock solid.

Version 1 stuff:

Version 2

Idea List

  • Written in Python
  • Written in such a way as it's portable, e.g. ./validate.py addon.xpi
  • Supports offline processing via gearman so we don't hold up server threads waiting on processing and we don't hit memory/time limits.
    • Progress should be checked via ajax
    • We should be able to see how many add-ons are in the queue at any time. If it's ever a large number we'll get more processing power.
  • Jetpack verification support
  • Better L10n support (there are a couple old bugs about this, and we need to update the external)
  • Add-ons for unit tests can be built on the fly. Jbalogh has a short script that this can be based on.
  • Levels of flagging? Like, a minimum level and a super picky level. Or would this just be too confusing and everyone would always only use one?
  • Real time virus scanning

Jorge suggests:

  • Be smart about caching: only clear validation results cache when the file is modified or the validator is modified. Maybe clearing the validation cache could be added as a step for production pushes.
  • Give priority to add-on uploads (by authors) in the validation queue.
  • Give severity weight to warnings and sort them by severity.
  • Show help text next to the warnings, instead of a separate page.
  • Recognize common validation patterns and make it easier to add them. For example:
    • [ file mask (*.js), what to look for (/eval/), severity (high), grouping ].
    • (the grouping would be for grouping similar results together and showing the link to the validation page)
  • Easy to download and distribute (to developers).
  • Smarter JS library recognition. Use common internal patterns instead of just file names.
  • Nils' validator, written in Python. We should definitely coordinate with him and see how we can integrate his ideas.
Retrieved from "https://wiki.mozilla.org/index.php?title=User:Clouserw/AMO/Validator:v2&oldid=220161"