Tag Archive: gesture


This week is an official holiday week in the Netherlands. The school is still open, but we had to subscribe to get in and the school is only opened for limited hours. It’s really deserted here. Luckily my office partner is here to have some talks once in a while. Like every holiday period, the process is a bit slow.

Last week I mostly worked on the context research. I finished researching itself in the end of the week. I’m really happy I have around 15 filled in questionnaires. I wrote down all off the interesting results from the answers I got. Most of the assumptions I had where right, but a couple of them where a bit different then expected.

Besides the questionnaires, I also went to experience the television environment myself. Hereby I documented the pictures of the television environment in a moodboard mock-up, categorizing all the details in a visual way (I may not say moodboard because Irene hates it). Irene also pointed me a while ago to a relevant course going on somewhere else on the planet, about field research for design (link). It is mostly about documenting and using results from field research in an efficient way. I used (and still use) the blog as a guideline for my research. A picture what caught my attention was this one:

User Experience Treasure Map

User Experience Treasure Map

It really goes in detail of a step by step plan for the process of defining the user experience, so I won’t use every element because of the limited time.

When I was finished with documenting the results, I started writing down keywords of my ideas, findings and inspirations. I have some ideas in my head how the system is going to work. Through brainstorming and writing everything down I want to structure my thoughts and come up with solid ideas. The next step is prototyping and testing my ideas, hopefully with some user testing.

start of writing down keywords

start of writing down keywords

Besides the usual project stuff, I went to the university library to get some additional papers for my supportive narrative. I found some relatively recent papers about embodied interaction and gestural interfaces. Reading is on hold for a moment, but they can prove useful in the future.

What drives me?

Last Tuesday our class had the first meeting with our tutor. Irene van Peer is now going to guide me to the end of my project. We all had to do a small presentation what our status was of each project. It was very interesting to see what everyone is doing. Unfortunately I was the last person who had to do the presentation and I had to rush a bit because we were out of time.

The main comment was that I have to figure out what area interests me the most. Embodied interaction within the living environment is a broad subject, of course. The goal I had set for myself was that I want to look at the possibilities of the embodied interaction and to pick a specific subject through prototyping and research. The plan was to do this the next few weeks. The problem with this is that I can get lost in the process, and don’t come up with a clear subject in the end. It is better to define myself as soon as possible.

During my presentation I gave some examples of projects that use physical interaction as input. With every video, I had some (mostly negative) comments about the interaction. Irene gave me the task to look at every video, and figure out which underlying subject drives me. So here it goes.

Canesta TV Gesture User Interface Demo


the video in short:
A television remote is an annoying necessarily to control your television, the video explains. Especially the on-screen interface is badly designed. Control through gestures is more intuitive. Simple movements with your hands makes you happy, jay! You can even use it while cooking! No need to put your dirty hands on the remote to change the channel.

My comments
Is moving your hands really better in this situation? Do you want to wave, swipe, push your hand when you are sitting on your lazy couch after you come home from work? It are simple physical movements for sure, but doesn’t it become tiring after an extensive period of time? I think buttons on a remote are better then doing physical movements, because it’s more responsive. Even if the remote is unintuitive. The problem lays in the design of the remote, not the interaction. Reminds me of an example of Nicolas Zurcher, whereby he looked at the buttons of the remote we really use:

Nicolas Zurcher's remotes

Which parts of the remote do we actually use?

And how do you go through a large quantity of television stations with only gestures? How do you control other functions, like volume, source selection, the television guide and use of other complex things? How to achieve this without learning complex gestures or using another type of input?

Gestures are not the best way to interact with the system in this situation, I believe.

Embodied Playlist Access


The video in short
The concept is about using personal objects to control your music. Every object stands for a different type of music playlist. The point is to get access through different selections of data by moving personal objects.

My comments:
What interests me is quite simple; by using physical objects you get access to personalized data. It avoids fiddling with computers (i.e. keyboard and mice). It makes selecting you music preference personal and physical. The technical barrier (where tech savvy people are so used to) is gone. The question remains if personal objects are a good representation to get access to different types of music. I see some possibilities in other forms of data, however.

MusicShapes


The video
from the youtube description: “The objective of this project was to produce a concept and a prototype of a physical computing system that would leverage the principles of embodied interaction. We developed a system that combined a webcam and a series of geometric shapes, which would play a sound or music clip depending on which of the faces is facing up. The sounds played by an object were influenced by the surrounding objects and their positions.”

My comments
Not only the movements of the shapes are input, but also the distance between objects matter. The objects can ‘communicate’ with each other, manipulating the output by moving them. It is an very early prototype, but imagine the possibilities implementing this in everyday objects.

Scratch Mat

An very simple example in real life. What makes it awesome that it changes behavior. How many times do you wipe your feet when you enter a building? This mat makes wiping your feet a playful interaction. The output of the system is very simple, but still is effective in this situation.

Siftables

Irene gave this project as a suggestion to look at. Siftables are small interactive computers with a display, wireless communication and motion sensing. What is relevant for my project is that it uses real life gestures as input. For example, pouring paint from one siftable to another, and thus changing it’s color. I would murder someone to get some of these.

I will put my conclusions in the next post.

Linkdump

In this post some examples of gestural interfaces in the living environment.

Gesture Cube

An interesting concept of a multimedia device for the living room. The concept is al right, but I think a lot is wrong with it’s interaction. Why a cube? The visible area’s are not ergonomic / optimal for one. And why use gestures for this object, why not a form of multitouch? I think using gestures is not the best responsive input in this situation. And most importantly, why do you want to carry around a large object through the house? The sharing feature (1:48) is the most interesting aspect. But as said before, why not a form of multitouch (Microsoft Surface) or projection (Pico projectors)? The communication with external devices is awesome, using physical gestures to ‘throw’ them one others interface. But I’ve seen this before in other experimental interfaces.

CRISTAL: Control the living environment through multitouch

An interesting way to have complete control over the living room. An multitouch screen is a nice platform, but is it the most comfortable and best way to interact with your environment? You can see they didn’t think hard about the interface itself, only how to control your complete environment. it would be interesting to focus only one aspect instead of all of them.

Some other examples:

The Subject

In my previous post I wrote down keywords and some inspirational things I like to do for my graduation project. Writing down things on paper really helps to get to the core of what you really want.

I noticed that I was interested in physical interaction with systems, using the human body itself as input.

For instance, the Nintendo Wii was revolutionary at the time. It uses a remote that detects gestures as input, instead of the button-bashing that was, and is, common on other gaming systems. Instead of learning what each button does on the controller (which has a long learning curve) the input on the Wii is natural. Everybody knows how to use her body. The developers of the Wii saw what other console developers didn’t see; It made gaming available for everyone, instead of appealing the relatively small hardcore gamer market. Nintendo’s point of view was so simple. The Wii turned out to be the most popular gaming system, beating the Xbox and Playstation in console sales by far. It’s ironic that the Wii technically and graphically lays far behind on the other consoles, what was the other consoles main selling point. Of course, the Wii is a lot cheaper then the other consoles, what is an important factor for the popularity as well. But I personally think it’s success lays in the simplicity of it’s controller, that is usable by practically everyone. It’s fun, simple and easy to learn.

It’s interesting to see that both Sony and Microsoft want to duplicate this type of input. Sony announced Playstation Move, while Microsoft came with Project Natal. For me, these are very interesting developments in this field. Especially Project Natal makes a giant leap, getting rid of the controller entirely. This system uses gestures and the human body itself as direct input. Of course, what Microsoft showed the world were promotional videos and proof of concept demos. It is still the question if it will work as advertised.

The above video shows what I mean. This video went sort of viral after the announcement of Project Natal. Kudo Tsunoda, the presenter, does a physical action, but the system screws up the translation from physical input to digital output. I know this is a demo, but I already see the frustration of gamers, and limitation of interaction in games. The motion capture technology is still in it’s early stage, but you have to understand that these kind of situations are unacceptable in interfaces. It’s like that your television remote only works half the time, or your steering wheel of your car doesn’t recognize your steering motion.

To a certain level, the Wii has the same problem. It’s physical interaction is kind of limited. For instance, the golf game requires you to do a swinging motion, of course imitating swinging with a golf club. First time users will imitate this gesture entirely, but after a while the gesture of the user will evolve to a simpler gesture, what the system recognizes as proper input. A small “tap” is enough for the system to understand.

Another point of critique is that the in-game interaction is limited. I remembered playing Metroid Prime, a first person shooter. At one point in the game I had to open a door. I had to twist the Wii Remote 90 degrees, imitating opening a lock. I had to try a few times to get it open. At first, I thought it was an innovative interaction, I had to do something I had never done before in a game. But what did I had to do? I had to open a lock by twisting my wrist. One of the simplest interactions in our daily life, but what was a game element in a console game. Isn’t this a bit silly? Is this an achievement for a gamer to be proud of?

The distance between reality and virtual, what the developers meant to abridge, is still a to much. Long time interaction with the system will constrain the immersion. The Wii becomes a gimmick, what is it’s main problem. Will the upcoming systems from Sony and Microsoft have the same problem? Or is this even a problem? Is just using you body as input for a system enough, regardless what the actions are?

You noticed that these kind of ‘new’ interactions only apply for gaming consoles. For my project, I don’t want to focus myself on gaming controller input. This is not something where my interest lies. Just everyday interaction with the environment interests me more. I want to apply the use of gestures in this area and I want to research the coming weeks what will be a good platform to apply this subject.

My point of this long, long story is that using the body, and thereby using an input everybody knows how to use, interaction with systems can be available for everyone. I want to use this point of view as a core for my graduation project.

Powered by WordPress. Theme: Motion by 85ideas. Adapted by N1kname.