View source for CrypoTestingSummit2006

== Session Title ==

Crypto related testing issues with Bob Lord and Co

== Session Leader ==

Marcia Knous, Tim Riley, Bob Lord

== Summary ==

Overview of crypto preferences for FFx and Tbird and how to test them better

== Agenda ==

We are thinking about a out-of-band meeting at 2:15 or 3:30pm Thursday in Building K.  Contact Tim Riley (IRC timr, IM tim_riley@yahoo, email timr@mozilla.com) or Marcia Knous. for details.

*  Crypto overview, understanding the intricacies of certificates in Firefox and Tbird.  
** Ways we can improve the automated testing of certs in Thunderbird and other ways automated testing may help?
* Improving our manual test of Security in Litmus.  
** Currently many tests are just visual confirmation that the UI is there.  
** Which areas are the most important for us to test?
* These security experts plan to join us: Bob Lord, Bob Relyea, Kai Engert, and Chandra Kannan

== Interested Attendees ==

* Tim Riley
* Marcia Knous
* Alice Nodelman
* [[User:Lord | Bob Lord]]
* Bob Relyea
* Kai Engert
* Chandra Kannan
* Robert Sayre
* Chris Cooper
* Nelson Bolyard
* [[User:rcampbell|robcee]]
* Wan-Teh Chang
* Juan Becerra

== Meeting Notes ==
Crypo testing
11/15/06

=== Attendees ===
Bob Lord, Bob Relyea, Bob Clary, Nelson Bolyard, Wan-Teh Chang, Chandra Kannan, Rob Campbel, Juan Becerra, Tracy Walker, Chris Cooper, Alice Nodelman, Tim Riley, 

=== Review Agenda ===
* Crypto problems found in Firefox 1.5.0.7
** RSA signature issue
*** found by very creative cryptographers - may or may not be a real vulnerability 
**  7 bad certificates
* Desire to do better manual testing of Security/Crypo IU's
* Interest in using existing automated test suites

=== Some recent work ===
* NSS run nightly
** what branches? Trunk and NSS 3.11 branch
** doesn't crash and doesn't leak (anymore!)
** 2Million tests and still problems found
*** test certs Generated by university in Finland
*** 4-5 CDs - ton of certs
** Can these get out of date?  [robcee]
*** [Nelson] then might
*** RSA public keys
*** Now elliptic curve certs
*** certs don't get out of date, but new technologies come along
* Tests run from tinderbox 
** SSL
** SMIME

* Agreement by Mozilla to take updates
** AI: Rob's team to check who made this commitment 

=== Testing the UI ===
* SMIME
* TSL/ECC - This is main area.  need to normalize this one
* Open SSL
* Test matrix between platforms and browsers
* Have seen regressions in: 
** Client Auth
** RSA Keygen

=== MoCo QA doesn't know how to test encryption UI ===
* Tracy:  don't understand how test UI
** Seems well tested before we (MoCo QA) gets it
** Chandra is a guru on PKI 

=== Smartcard testing in Firefox ===
* Bob R
** Get MoCo some USB smartcards
** There is smartcard support in FF1.5
** Bob R added hooks for registering when smartcard is inserted
** Could have a special page where if smartcard is inserted it will take you to a special page
** Need to setup public servers for testing (MoCo, Sec Test)
** Setup automation to capture info about TLS session (see Bob L's demo)
*** '''See Chandra for ideas'''
** '''MoCo, Sec Team to collaboration'''

=== PSM Testing ===
* Better testing [Nelson]
** PSM - core Firefox component 
*** Configuring PKI
*** QA on PSM??
**Great opportunity for MoCO to create unit tests
*** What is the use case?? [juan]
*** '''Talk to Kai E and Chandra << AI Who??'''

=== SMIME ===
* SMIME
** Automated tests?
** Nelson seen many regressions in mail
*** signed mail  get reported as invalid signatures
*** More trouble with IMAP
**** Setup messages on IMAP server
***** check for valid messages and attachments
***** try different IMAP servers
***** Setup canned set of messages (on CD, public server)
***** Lots of energy here!!

=== Setup Test Servers ===
* Need follow-up with Kai
** Has a server with a ton of tests
** '''Setup meeting with him <<== timr'''

=== Misc ===
* Shopping/SSL testing is a good starting point 
* IE trashing
** Warning:  SSL is about to be used
** requiring certs - sign by default even if you don't have a cert!
*** Then the message is rejected because no cert