Raindrop/Extensions/Data Miners

	THIS PAGE IS A WORKING DRAFT
	The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly. If you have any questions or ideas, please add them as a new topic on the discussion page.

Data Mining messages is a how we provide more value to the user than just displaying the pure text of a message. By looking for patterns in our conversations we can help people get more information and do more faster.

Almost every data mining extension works in this way:

1. Find a pattern in a message
2. Possibly call out to an external service for more meta-data about the pattern
3. Create (emit) a document about the information

Here are some examples of different services:

Photo Services

When people share links to their online photographs they are really sharing the photos themselves. Our goal is to get a thumbnail of the image, a title, location, and any other data that might be interesting.

Flickr

The Flickr data mining extension scans messages for flickr URLs it recognizes. Currently this handles both flickr.com URLs and the shortened flic.kr URLs.

Here you can see the code for when the Flickr Data Miner was first added and here when the flick.kr urls where added as well.

Here are the latest versions of the Flickr data mining extension.

• JSON Registration File
• Data Mining Extension

TweetPhoto

Minimal TweetPhoto support has been added, [1]

• API: http://tweetphoto.com/developer
  • GET PHOTO: http://groups.google.com/group/tweetphoto/web/get-photo-details
• Example
  • http://tweetphoto.com/9298206
  • XML: http://tweetphotoapi.com/api/tpapi.svc/photos/9298206
  • JSON: http://tweetphotoapi.com/api/tpapi.svc/json/photos/9298206

Twitpic

Minimal Twitpic support has been added [2]

• API: http://twitpic.com/api.do#thumbnails
• Example
  • There isn't a method for getting a title other than grabbing the HTML and getting the actual title from the page
  • http://twitpic.com/y8imi
    • THUMB: http://twitpic.com/show/thumb/y8imi
    • MINI: http://twitpic.com/show/mini/y8imi

Link Shorteners

With the rise of Twitter there are a growing number of link shortening services. These services are excellent for saving on the number of characters in your messages however they hide the real location of links and can make messages harder to understand.

In Raindrop we expand these links to get their full URL, title, and any other information which might be interesting. This allows us to give the person more information about the link before clicking on it.

Currently Raindrop cannot data mine the full URL of shortened links which are expanded. There a limitation where a shortened link which points to youtube, flickr, or other another service will only show up as the full URL and will not be data mined further so a thumbnail or title can be retrieved.

Bit.ly

The bit.ly URL expander uses the API for expanding shortened links found in messages.

Here's the initial commit of the bit.ly extension

The extension does the following:

1. Search for bit.ly links in messages
2. Call out to the bit.ly API for the link title and other meta-data
3. Create (emit) a document describing the expanded link

Here are the latest versions of the extension:

• JSON Registration File
• Data Mining Extension

retwt.me

retwt.me has a nice and easy API for expanding a link. Without an API key you can quickly and easily get the expanded link version. The expanded link does not include a title or any extra meta-data.

e.g.

 curl "http://api.retwt.me/expand.json?shortUrl=http://retwt.me/1LVxs"

is.gd

An is.gd extension has not yet been implemented

is.gd offers an API via the tech page however the "expand" API method is to follow the links 301 redirect.

BudURL

A BudURL extension has not yet been implemented, do you want to take it on?

budurl.com offers an API for expanding shortened links found in messages.

tr.im

A tr.im extension has not yet been implemented

tr.im offers an API for expanding shortened links found in messages. An API key is required to make most calls.

su.pr

An su.pr extension has not yet been implemented

su.pr or StumbleUpon offers an API for expanding shortened links.

 curl "http://su.pr/api/expand?shortUrl=http://su.pr/$HASH"

 {
  "errorCode":0,
  "errorMessage":"",
  "results": {
             "$HASH": { 
                        "longUrl":"$HTTP"
                        }
             },
  "statusCode":"OK"
 }

Video Services

The intention of the Video service is to grab a thumbnail, title, location, and inline video player of a video link used in a message. With this information we can create a much better video experience that allows people to view the video inline.

YouTube

Here are the latest versions of the YouTube extension:

• JSON Registration File
• Data Mining Extension

Vimeo

Here's the initial commit of Vimeo link support into Raindrop.

Here are the latest extension files:

• JSON Registration File
• Data Mining Extension

Links

Certain web sites offer simple APIs for retrieving more information about the products they offer. We can optimize for many of these sites by creating extensions that find those links and grab interesting information from the sites to expand on the link given.

Mozilla Add-ons

This extension is a suggestion that has not been developed. Try it out!

Using the AMO API Raindrop can retrieve additional information about an Add-on link being sent via email.

If someone sends a link like: https://addons.mozilla.org/en-US/firefox/addon/748

Raindrop can find that link and retrieve the extra meta data via the API by constructing a URL like this: https://services.addons.mozilla.org/en-US/firefox/api/1.3/addon/748

Now you'd just need to create a quick AMO UI extension helper to display the real title, image, and author; you could even offer an install button for a link sent.

Four Square

This has been initially implemented in Raindrop

Four Square has an API which allows for simple lookups of venues

Gowalla

Gowalla support has not been implemented in Raindrop

Gowalla has announced an API which could be used to retrieve information about gowalla links.