Security/Projects/Minion: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Remove link to dead vm.)
 
(35 intermediate revisions by 6 users not shown)
Line 1: Line 1:
<cite>
=Minion Overview=
Minion is a security testing framework built by Mozilla to bridge the gap between developers and security testers. To do so, it enables developers to scan their projects using a friendly interface.
__NOTOC__
</cite>


NOTE - this project is at a ''very'' early stage in its development.
Minion is an open source Security Automation platform.  The 0.3 release of Minion allows Development, QA, and Security team members to perform automated web security scans with a set of tools, and re-execute those scans as needed.


* Source code: https://github.com/ygjb/minion - public
The 0.3 release incorporates significant changes, including a migration away from Django and Bootstrap to a Flask and Angular.js based application. It also involves significant improvements in back-end performance and scaling, and an updated plugin architecture.
* Task management: https://trello.com/b/DlVPzGaS - currently private, contact one of the Minion developers to get access
Developers:
* [[User:Psiinon|Psiinon]]
* TBA


==Initial Diagram==
This is the first Minion release that is ready for large scale adoption with access management features to constrain which users can access scan results, and an invitation system to actively engage new users.
[[File:Minon_diagram.png]]


==Components==
= Minion Principles =
===Web Interface===
* Log in using Persona (BrowserID) (can be restricted by domain for use on central server by organizations)
* Menu -> New Scan, Running Scans, Completed Scans
** Future: Group Scans (member of groups, permissions, see other scans by group members/project)
* New Scan
** Basic: URL, Port
** Advanced: Login information, technologies used (customize scan such as SQLmap for SQL)
** Future: Scan type based on plugin (web app, client code, etc)


===Task Engine===
===Minion should be easy to use.===
* Instance started when user clicks start scan
A user should be able to log into Minion and immediately start to launch scans against sites available to them.  Results should be easy to discover and contain detailed explanations.
* Collects provided information
* Starts scan based on provided information
* Launches tools (Minions) and awaits responses


===Minions (Scanners)===
===The target audience for Minion should be developers===
* Receive kickoff from task engine
Developers should be able to use Minion the moment they have code written that can be tested.  The initial focus will be on supporting web applications and services, but this should extend to other applications with the cohort feature set.  The initial guidance surfaced by the default reporting engine should be useful, accurate, and actionable by developers, and meaningful to other audiences (QA, Security, Management).
* Scan target
* Send results back to task engine in necessary format (JSON)


===Target===
===Minion is a platform, not a security tool===
* Site hosted by developer
Minion is an extensible platform that allows automation of security tasks.  As such the focus should be on providing strong abstractions and a reliable, extensible platform without binding the platform to a specific suite of tools.  All security testing functionality should be external to Minion and implemented via plugins.
* Can be hosted or running locally


==Configuration Abstraction==
===Minion is scalable and extensible===
To pass data between components, a standard configuration and data scheme must be used. For Minion, we have decided to use JSON as the format for passing data. For example, the user interface will collect the necessary options, then call the task engine and pass it a JSON string of those options. The task engine will read the options, split them into new JSON strings as necessary to send to each installed tool (some tools may need different options than others). The tool will pass its results back to the task engine as a JSON string and the task engine will compile all returned results into a single JSON string which will be returned to the interface. The interface will deconstruct the string into human-readable results and display them on the results page.
It should be simple to add support for unsupported tools. In addition to the ease with which plugins can be authored, the plugin architecture must isolate plugin related issues from the core platform and allow delegation out to separate infrastructure, and must support scaling from small teams to enterprises that may deliver services to third parties.


===Basic Tool Requirements===
===Minion is a secure tool===
Each tool takes a number of base options as well as optional options that can improve the scan's accuracy or increase its functionality.
Ongoing security testing and security assessment should be performed on Minion.  Major releases and milestones should be accompanied with a detailed explanation of data collected, and risks or threats posed by the tools and plugins that are shipped by the core development team.


====Zed Attack Proxy====
===Minion is open===
Base options: URL
Minion is an open source platform, and the core team actively encourages developers, security professionals, and interested parties to provide feedback and feature requests.


Optional: spider depth-level, CSRF tokens, authentication information, parameters to fuzz
= Architecture=
[[File:minion-03-diagram-draft.png]]


====Garmr====
The Minion Front-End is an Angular.js based application that invokes a set of API on the backend.  The backend is comprised of a set of services that are collectively referred to as the Task Engine, and a set of plugins.  The architecture is designed to be distributed so the plugins can run on any platform that can expose the correct API, ensuring that tools can be incorporated into Minion regardless of operating system dependencies.
Base options: URL


Optional: parameters to test
== Roadmap ==
=== Q3 2013 ===
* Site Ownership Verification
* Results Reporting Improvements


Note: most options with Garmr involve output, which should be handled without interaction from the user.
=== Q4 2013 ===
* Reporting Plugins
* Landing Pages
* Deferred Execution Plugins
* Scan Intensity Level


====Skipfish====
=== Q1 2014 ===
Base options: URL
* Cohort PoC
* Historical Issue Tracking
* Interpolation Engine
* Common Configuration Schema (Extending DEX-JSON)


Optional: authentication credentials, cookie values, non-standard header information, scan time limit (see http://code.google.com/p/skipfish/wiki/SkipfishDoc for a full list)
=== Wish List ===
* Site and User Data Privacy
* Minion Event Model Extensions (simple extensibility)
* Scramble - interactive script for generating plugins


Note: Skipfish has a lot of additional options that include domains to exclude in crawling, domains to ignore in testing, wordlist generation, folder output, etc. To make it as easy to use as possible, the Skipfish plugin should include defaults for all of these options so that the user will never need to worry about the options.
==Details==
''This information is preserved from an early iteration of the project.  The information is '''obsolete''' and in most cases invalid, but the core concepts remain the same.''
* [[Security/Projects/Minion/Principles | Principles]]
* [[Security/Projects/Minion/Architecture | Architecture]]
* Components
** [[Security/Projects/Minion/WebUI | Web UI]]
** [[Security/Projects/Minion/TaskEngine | Task Engine]]
** [[Security/Projects/Minion/PluginService | Plugin Service]]
** [[Security/Projects/Minion/Plugins | Plugins]]
* [[Security/Projects/Minion/ImplDetails | Implementation Details]]
 
==Links==
All of the following are publicly accessible:
* Source code: https://github.com/mozilla/minion
* Email list: http://groups.google.com/group/mozilla-minion-dev
* [[Security/Projects/Minion/Roadmap]]
* We also use the #websectools channel on irc.mozilla.org
 
= Minion Virtual Machine =
 
You can find instructions and tools to build a Minion virtual machine [https://github.com/mozilla/minion-vm/ on Github].
 
= Developers =
 
You can find most of us on #websectools or on Github.
 
* April King (april)
* Stefan Arentz (st3fan)
* Simon Bennetts (psiinon)
* Yeuk Hon Wong (yeukhon)
* Matthew Fuller
* Mark Goodwin (mgoodwin)
* Yvan Boily (yvan)
 
 
== List of plugins ==
 
The following projects are optional plugins for minion that add more functionality or wrap existing tools:
 
    https://github.com/mozilla/minion-zap-plugin
    https://github.com/mozilla/minion-ssl-plugin
    https://github.com/mozilla/minion-skipfish-plugin
    https://github.com/mozilla/minion-nmap-plugin
    https://github.com/mozilla/minion-breach-plugin
    https://github.com/mozilla/minion-garmr-plugin
    https://github.com/mozilla/minion-zest-plugin

Latest revision as of 18:35, 27 August 2015

Minion Overview

Minion is an open source Security Automation platform. The 0.3 release of Minion allows Development, QA, and Security team members to perform automated web security scans with a set of tools, and re-execute those scans as needed.

The 0.3 release incorporates significant changes, including a migration away from Django and Bootstrap to a Flask and Angular.js based application. It also involves significant improvements in back-end performance and scaling, and an updated plugin architecture.

This is the first Minion release that is ready for large scale adoption with access management features to constrain which users can access scan results, and an invitation system to actively engage new users.

Minion Principles

Minion should be easy to use.

A user should be able to log into Minion and immediately start to launch scans against sites available to them. Results should be easy to discover and contain detailed explanations.

The target audience for Minion should be developers

Developers should be able to use Minion the moment they have code written that can be tested. The initial focus will be on supporting web applications and services, but this should extend to other applications with the cohort feature set. The initial guidance surfaced by the default reporting engine should be useful, accurate, and actionable by developers, and meaningful to other audiences (QA, Security, Management).

Minion is a platform, not a security tool

Minion is an extensible platform that allows automation of security tasks. As such the focus should be on providing strong abstractions and a reliable, extensible platform without binding the platform to a specific suite of tools. All security testing functionality should be external to Minion and implemented via plugins.

Minion is scalable and extensible

It should be simple to add support for unsupported tools. In addition to the ease with which plugins can be authored, the plugin architecture must isolate plugin related issues from the core platform and allow delegation out to separate infrastructure, and must support scaling from small teams to enterprises that may deliver services to third parties.

Minion is a secure tool

Ongoing security testing and security assessment should be performed on Minion. Major releases and milestones should be accompanied with a detailed explanation of data collected, and risks or threats posed by the tools and plugins that are shipped by the core development team.

Minion is open

Minion is an open source platform, and the core team actively encourages developers, security professionals, and interested parties to provide feedback and feature requests.

Architecture

Minion-03-diagram-draft.png

The Minion Front-End is an Angular.js based application that invokes a set of API on the backend. The backend is comprised of a set of services that are collectively referred to as the Task Engine, and a set of plugins. The architecture is designed to be distributed so the plugins can run on any platform that can expose the correct API, ensuring that tools can be incorporated into Minion regardless of operating system dependencies.

Roadmap

Q3 2013

  • Site Ownership Verification
  • Results Reporting Improvements

Q4 2013

  • Reporting Plugins
  • Landing Pages
  • Deferred Execution Plugins
  • Scan Intensity Level

Q1 2014

  • Cohort PoC
  • Historical Issue Tracking
  • Interpolation Engine
  • Common Configuration Schema (Extending DEX-JSON)

Wish List

  • Site and User Data Privacy
  • Minion Event Model Extensions (simple extensibility)
  • Scramble - interactive script for generating plugins

Details

This information is preserved from an early iteration of the project. The information is obsolete and in most cases invalid, but the core concepts remain the same.

Links

All of the following are publicly accessible:

Minion Virtual Machine

You can find instructions and tools to build a Minion virtual machine on Github.

Developers

You can find most of us on #websectools or on Github.

  • April King (april)
  • Stefan Arentz (st3fan)
  • Simon Bennetts (psiinon)
  • Yeuk Hon Wong (yeukhon)
  • Matthew Fuller
  • Mark Goodwin (mgoodwin)
  • Yvan Boily (yvan)


List of plugins

The following projects are optional plugins for minion that add more functionality or wrap existing tools: 

   https://github.com/mozilla/minion-zap-plugin
   https://github.com/mozilla/minion-ssl-plugin
   https://github.com/mozilla/minion-skipfish-plugin
   https://github.com/mozilla/minion-nmap-plugin
   https://github.com/mozilla/minion-breach-plugin
   https://github.com/mozilla/minion-garmr-plugin
   https://github.com/mozilla/minion-zest-plugin
Retrieved from "https://wiki.mozilla.org/index.php?title=Security/Projects/Minion&oldid=1092359"