View source for Connected Devices/ Projects/ Project SmartKitchen/ Experiment

== '''Status:	DONE''' ==
	
	
===Leap Of Faith Assumption (LOFA) We Are Testing:===	
(I) We assume that we can identify some percentage of food via image recognition	
	
===Hypothesis:===	
We believe that if we give a user a device for identifying their inventory that we will be able to recognize 20% of items with our existing training to date.	
	
===Experiment Steps:===
* Give 3 internal Mozilla users (not on the team) our prototype device	
* Have them sign up for a Kik account	
* Instruct them on how to provision their wifi	
* Have them try to identify items in their kitchen with the device when they are done with their shopping	
* Collect metrics on the accuracy of the item identification	
	
=== Experiment Timeline: ===
{| class="wikitable"
| 7/18
| Experiment Details Created
|-
| 07/19
| Email Sent out to SF/MTV Staff for 3 testers
|-
| 07/22
| Trial Instructions and Wifi Set Up Instructions completed
|-
| 07/25
| Software completed for prototype (server, username, etc.)
|-
| 07/27
| Hardware completed from Kevin and delivered to SF Office
|-
| 07/27
| Study Begins (each participants have 1 week and are starting on different days)
|-
| 07/27
| Metric Collection Begin
|-
| 7/28-8/5
| First week of metrics collected with no training
|-
| 08/06
| Upload additional training and update classifiers.json
|-
| 8/6-8/12
| Second week of metrics collected with additional training on common ingredients of trial participants
|}
	
===Experiment Details:===
* [https://docs.google.com/document/d/18i675-cmS4lEmnOv3O7hKu_-XdW7ScM0R97JWJh0fhk/edit Trial Instructions]
* [https://docs.google.com/document/d/1A07eKzNa4_JsUhW3udqLaakVIuyawTQRWIs6lnt2qFA/edit Wifi Instructions]
	
===Key Findings===
* Recognized 25% of the images but were very loose recognition. (i.e. Food)	
* Need to look at another recognition survey	
* Need to incentivize survey participants	
* Team needs to look at actual images to see what the problem is.	
===Next Steps===
* Possible secondary experiment: comparing two IR services