Tag: Gestural User Interface

Posted on

Scrolling in the Apple TV UI

I’ve written before (here, here, and here) on the differences between scrolling on an Apple Macintosh and scrolling on an iOS device like an iPhone, and how those differences are going away.

It turns out that there remains at least one remote corner of the Apple universe where the reconciliation of gestural meaning is still a little awkward.

The Apple TV is a content delivery device that provides an elegant user experience for delivering content from a variety of sources to a television screen. It includes a wonderfully simple remote control that is generally a delight to use. One less than delightful aspect of the remote control, though, is the cumbersome method for entering text, such as when searching content. Happily, Apple provides an iOS app called Remote, which can be used to control the Apple TV and which enables easier text input using a keyboard.

Of course, the Remote app can control all aspects of the Apple TV, using a gestural UI that one would expect from iOS. It’s here that things get a little awkward.

Note the following models:

  • On a Mac (in OS X Lion), dragging two fingers on on the track pad moves the contents of a window (e.g., scrolling through a list)
  • On iOS, a swipe gesture moves the screen (e.g., scrolling through a list).
  • On Apple TV, clicking the arrows on the remote control moves the on-TV-screen selection indicator (e.g., selecting an item in a list).

Using a swipe gesture in the Apple TV Remote app, which in effect turns the iOS device into a trackpad when used this way, also repositions the on-TV-screen selection indicator. This is quite similar to the behaviour of the gesture on a Mac trackpad prior to Lion, where the gesture controlled a UI widget (scrollbar) rather than the content itself; it is the selection indicator that is being controlled by the gesture not the screen content. This makes sense for a point-and-click remote control, but not for a gestural one.

For me the awkwardness arises when scrolling through a long list, such as many rows of movies.

That is, when scrolling vertically the selection indicator stops moving in the middle of the list view port, and the list moves through the selection indicator. The experience for me feels strongly like the swipe gesture is moving the list in the opposite direction to the swipe. As a result, I use the regular remote control for navigating screens on Apple TV, one click at a time, and I use an iOS device for entering text when needed. This isn’t really optimal and I’m curious to see how the Apple evolves and improves the experience.

Posted on

The lion and the trackpad

Apple has released a version of the next version of Mac OS X, code-named Lion, to developers. That’s a fairly standard step on the release road for the company’s OS updates. In an article at AppleInsider, though, I noticed this interesting tidbit:

The new multi-touch gestures are designed to take advantage of the larger click TrackPads on more recent MacBook models, which could make them more difficult with older notebooks. Another strange quirk, people familiar with the developer preview said, is two-finger scrolling is reversed: to scroll down on a webpage in Safari, users must push up with their fingers, which is the opposite of how it works in Snow Leopard, but the same directly as scrolling on the iPad.

I’m pretty sure that nobody at Apple, or AppleInsider for that matter, reads this blog. Anyone who has read my previous post on scrolling from March of last year, though, will know that the change doesn’t feel like a strange quirk to me. It feels like the right direction to go, and Apple is clearly addressing the collision between old and new interaction paradigms.

Posted on

Perils of a gestural UI, part 2

Back in March I wrote a post about how I had discovered the source my discomfort with two-fingered scrolling on my MacBook. Basically, two-fingered scrolling goes in the opposite direction to what I experience when scrolling on my iPhone. In the end, I figured I could get used to it and make it work for me. I finished that post, though, by asking “how many more of these collisions will appear as Apple continues to build on its gestural UI?”

A new collision appeared this week, as Apple released a software update that adds three-finger dragging to the trackpad. That is, moving three fingers across the track pad will drag whatever is under the cursor (assuming that it’s draggable). The behaviour feels great when moving windows around, though moving icons around on the desktop feels slightly odd to me still. The collision, though, relates to my previous post.

With two-fingered scrolling, moving my fingers towards me on the trackpad moves the scrolling page upwards within the screen — my fingers are, in effect, interacting with the scrolling control rather than the page content itself.

With three-fingered scrolling, moving my fingers towards me on the trackpad moves the window (if it’s a window I’m dragging) downwards within the screen — the opposite direction to what happens with two-fingered scrolling. My fingers are interacting directly with the object that I’m moving.

I haven’t yet spent enough time with the change to know how this behaviour will feel for me in the longer term. My guess though is that I’ll find it more disconcerting than the contrast between trackpad and iPhone that I previously wrote about. And I do suspect that there are more collisions to come.

Posted on

Perils of a gestural UI

The iPhone was Apple’s first product that leapt completely into the world of gestural interface; it was later followed by the similar iPod Touch. The recent iPad looks to build upon the success of those products. While the iPhone isn’t perfect, as I’ve written previously, it’s a wonderful product for me.

The company’s gestural endeavours aren’t confined to new product categories. Apple has also built multi-touch gestural trackpads into various models of MacBook. I’ve never made a lot use of the extended capabilities in the trackpads — I found two-fingered scrolling to be pretty awkward (though rotation is fine for me).

While I hadn’t previously tried to analyze my response, I recently took a closer look and figured out what has thrown me about scrolling using the trackpad. To paraphrase Inigo Montoya in The Princess Bride, “You keep using that gesture. I do not think it means what you think it means.” Let me explain.

On the iPhone, dragging my finger on the display causes what’s visible on the screen to move in the direction that my finger is moving. A good example of this is seen in Safari, the iPhone’s web browser. If a web page doesn’t fit on the screen I can put my finger on the screen, drag it across the screen, and the web page moves with my finger. It’s as if the page were sitting on a table and I put a finger on the page to move it across the table in a particular direction. If I move my finger towards me, the page moves towards me — scrolling “up” on the screen. If I move my finger away from me, the page moves away from me — scrolling “down” on the screen.

The trackpad on my MacBook is different. Using Safari as an example again, when viewing a web page the entire page may not appear within the browser window. I can scroll the page in a few ways. I can use the cursor to move the scroll bar, or I can use the arrow keys on the keyboard to move the scroll bar. The key in both these cases is that I’m controlling the scroll bar, which in turn scrolls the page. A third way to scroll the page is via two-finger scrolling on the trackpad. Here is where things get interesting. If I move my fingers towards me, the page moves away from me — scrolling “down” on the screen. If I move my fingers away from me, the page moves towards me — scrolling “up” on the screen. These behaviours are the opposite of what’s happening on iPhone. The reason is that two-fingered trackpad scrolling is linked to moving the scroll bars rather than moving the page directly.

Moving back and forth between iPhone and Mac made it easier for me to finally identify the source of my trackpad scrolling discomfort.

This really feels like a collision between the historically dominant interaction paradigm as found in Mac OS X and Windows, and a new gestural paradigm as seen on iPhone. For me, when I’m gesturing to scroll I’m moving the page, not the UI control. iPhone supports that model. The MacBook trackpad doesn’t. The question I have is, how many more of these collisions will appear as Apple continues to build on its gestural UI (and, of course, as other companies add their own twists).