Touch investigation

Some articles to look:

• http://msdn.microsoft.com/en-us/library/cc872774.aspx
• http://msdn.microsoft.com/en-us/library/dd371585%28VS.85%29.aspx

General touch behaviors

These are the current possibilities for touch in the Windows API, along with the common action associated with them.

• Flick: a quick and short movement on the screen. Triggers some Windows specified action ( [back, forward, page up, page down] and on the diagonals [delete, copy, paste, undo])

• Panning: slower movement on the screen. Can start a pan (scroll) both horinzontally or vertically, or start a drag (as "click and hold mouse button, move") which makes possible dragging and dropping elements or selecting text. The logic is controled by the Windows API, and the application can register which axis (horizontal, vertical or both) trigger which behavior.

• Moving windows: touch and hold a window's titlebar, then move around. If the window is moved to the upper part of the screen, an animation appears indicating that the window is gonna be maximized. If the window is moved to either leftmost or rightmost of the screen, the window can be half-screen maximized. These behaviors are also seen when moving windows using the mouse.

• Resizing windows: touch and hold on any edge of the window (even the titlebar). Left, Right and Bottom resizing doesn't work in Firefox main window because it looks like all that area is capturing the panning movements

• Moving vs. Resizing is kinda tricky. I hardly can get what I want in the first try.

• Double click: double tapping

• Right click:
  • tap and hold for a few seconds until a circle appears and completes; then release the finger. If you hold for a few more seconds the circle will be dismissed and the action will trigger a normal left click again.
  • tap and hold at one place, then tap with a different finger. This will isntantly trigger the right click at the location of the holding finger

• Windows task bar: hold for a bit one icon and the drag up to get the jump list (same as right clicking)

• 2 fingers tap: triggers auto-zoom

2 fingers:

• Panning: two fingers moving keeping the same distance between them.

• Zooming In/Out: two fingers moving apart, keeping a reasonable center.

• Rotating: One finger stopped, the other moving around. Probably detected by: keeping same distance, changing center.

Current Firefox support

(broken in revision 254c5cd4978b (bug 485101 - Implement panning feedback for touch enabled displays with win7))

• Scrolling: sometimes focus must be under cursor (XXX TODO verify). Bespin is problematic

• Tab bar: we only scroll, IE only drags. From experience it looks like scrolling is better. Can we invert the registration for drag/panning scroll here, so on the tab bar horizontal triggers panning and vertical triggers dragging? that would be perfect

• can't do mouseovers nor get tooltips and the like (alt="" and title="" from elements, or JS generated tooltips)

• scrolling: widgets like listviews, etc seems to scroll ok (but not pixel scroll though, only line by line). Richlistview does pixel scrolling and inertia.

• page with panning problem: http://social.msdn.microsoft.com/Forums/en-US/tabletandtouch/thread/f1367fe1-0127-490c-b761-97ea61c348a0 (try starting the pan inside the text block of a user response)

• iframe problem: when focus is inside it, it's scrolled corretly; when focus is outside, parent tries t oscrolls but it's buggy

• cannot drag objects using mousedrag events, or even implicit drags of links (to drag to the tab bar or the favorites bar for example). (webmail for example has these UI resizers, as well as Bespin).

• with the up/down panning there's no way to do the up/down flick. Same for IE.

• big resistance for panning hor/vert simultaneously. try at http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox

• Touch text input
  • not triggering manual input in text fields in webpages.
  • not triggering in options window
  • triggering on Search field on Bookmarks Organizer. Not triggering on ther fields there.
  • On toolbar we trigger (URL and search bar), but we can do better by offering pre-filed fields (http://www. ____ .com)

Differences

• no way to select text, always try to pan (reported as bug 501440)

• what IE does: only vertical movement starts panning, horizontal starts selection

• this page has a large horizontal scroll http://www.peter-pearson.com/ but no vertical scroll, so IE cant handle the scroll

• inertia horizontal performance: http://www.richardarran.com/ (this page is slow in Firefox)

• zoom:
  • works on time, not traveled distance
  • doesnt keep scroll position

• tab bar: we always scroll

• scrollbar: you can actually drag the scrollbar in IE; we keep the panning working over the scrollbar

• two-finger tap: We always zoom to 100%. Should be a smart zooming.

• scrolling inside frames: sometimes it works, sometimes it goes weird. eg. in gmail chat, we have to first be on focus ie scrolls without that needed, and also bump feedback it

• need to figure a way to how to use mouseover. Relevant links: ** http://www.quirksmode.org/blog/archives/2008/08/iphone_events.html
  • https://wiki.mozilla.org/Mobile/TouchScreen/MouseEvents

• opening links in new tabs: we cant; drag & drop the link in IE

• dragging things http://script.aculo.us , http://bespin.mozilla.com

UI interactions

• Menu: holding a finger and navigating through the menus, and release where you want. Currently already works if single-finger panning is not triggered.

• Drop down boxes: needs work. should be able to hold and drag it downwards, instead of having to click the tiny arrow on the right side

• Icons target areas: working on that

UI Guidelines

https://help.ubuntu.com/community/UMEGuide/DesigningForFingerUIs
http://www.ixda.org/discuss.php?post=42270
http://www.touchusability.com/2009/05/comparing-circular-and-twofinger-scrolling-on-touchpads.html
http://www.sapdesignguild.org/resources/TSDesignGL/Index.htm