Controlling our
electronic devices has always been something of a challenge. There is
always some sort of mythical way to interact and control our devices
that seems just out of reach. We started out with physical switches and
buttons, then moved onto touch-sensitive interaction with trackpads and
now touchscreens. Throughout it all, there has also been work done to
create voice-interaction with our electronics, which has come a long
way, but is still not perfect, and can't cover all of the needs we have.
Now, we have a number of different companies trying to push forward
with touch-free gesture controls.
At Mobile
World Congress this past week, there was a company, Noalia, showing off
another iteration of touchless gesture controls. This time, the controls
were not only touchless, but didn't require the use of a camera either,
as many options do. The Aramis technology shown off by Noalia detects
the electrostatic energy around your hand, and translates that into
on-screen gestures. This means that you can control a phone or tablet
from a distance of about 10cm without touching the device.
Back in December, we saw an Israeli startup, XTR3D, that was showing off Kinect-like gesture-based controls for
electronics and mobile devices, and every once in a while we've seen
companies coming along and trying to make it seem as though the future
is in touchless gesture control, but frankly we just can't see why.
The case against touchless control
We
can certainly understand the idea behind Kinect, which is Microsoft's
gesture control system first introduced as part of the XBox 360 gaming
system, and has now officially made its way to PCs as well. The logic is
understandable, and it can be seen at the beginning of the video
showing off XTR3D's technology. On larger screens, like TVs or desktops,
where there is no inherent touch-control system and the device itself
is sometimes out of reach, a gesture-based system makes some sense,
although it doesn't really add any benefits over traditional controls
with remotes, mice or keyboards. The trouble is that companies seem to
think it is a good idea to convert this technology to mobile devices as
well. Of course, once you apply touchless gesture technology to a device
designed for touch interaction, you suddenly run into a number of
troubles.
First, what is the need for
touchless interaction on a device that's designed to be in your hand at
all times? The use case for touchless gestures on a mobile device are
incredibly few and far between. Essentially, the only reasons you would
ever have to use touchless gestures on your smartphone or tablets fall
into just one category: touch-free means smudge-free. If your hands are
dirty or if you really hate smudges, touch-free controls are a benefit,
but beyond that, there is little reason to need touchless gesture
controls. Touch-free screens would also make using touchscreen devices
in the winter much easier when you're wearing gloves, but again,
touchscreens have already evolved to the point where many can be used
even through gloves (depending on thickness) without the need for
special tips on the fingers. There is a possibility for using touchless
gestures as a way to easily "fling" information from one screen to
another, but that seems more like a gimmick used in "futuristic"
technology concepts. The same thing can be done right now with either
NFC, or wireless connections and traditional buttons. There is no
benefit to moving that function to touchless control.
If
you're in a car, gesture controls don't help, because you're still
taking your hands off the wheel, so voice-control is still the far
better option. Some may say that touchless gestures will be good for
touchscreen gaming, because your thumbs will finally get out of the way
and allow you to see more of the screen. But, most gamers will also tell
you that gaming without direct feedback isn't the same experience. It
may not change what happens in the game, but being able to hit the
screen a bit harder for an important movement is just flat out more
gratifying than waving your hand in the air.
Design for humans
And, that's the real issue behind this that no one bothers thinking about: hands are designed to touch and manipulate things.
A wave doesn't covey the same thing as a handshake, just like flipping
the bird doesn't convey the same thing as a slap in the face. We are
designed to feel, touch, and interact physically, so why would we want
to design products that go against that natural design? The logical
extension of a touchpad is a touchscreen, but the logical extension of a
touchscreen is not a touchless screen. Hands are designed to feel
things and understand kinetic feedback, but touchless gestures feature
none of that. A far better idea is one that we saw early in December
last year, where a company called Senseg
has designed a screen that can generate kinetic feedback in order to
make it feel as though the screen has various different textures on it.
This sort of technology has far more uses than any touch-free gesture
system.
With kinetic feedback, you can create touchscreen devices that can be used by blind people.
For more casual users, games could be made where users can feel
different textures. Imagine pulling back an Angry Bird in its slingshot
and being able to feel the feathers of the bird, and because the
technology works by creating electrostatic friction, you may even feel
the tension of the slingshot band as you pull back. Similarly, if you
have an app of a zen sand garden, you could actually feel the sand and
stones as you move things around. Doctors could use screens with kinetic
feedback to train, or musicians could practice the piano on the go and
actually feel the keys on their tablet. And, there is no telling what
innovations (not to mention absolutely disgusting websites) that could
come to the Internet if suddenly websites could have textures and
kinetic feedback.
It all comes back to the fact
that we design products to be extensions of ourselves, because
essentially they are just tools for us to better interact with each
other, the world and possibly most importantly to interact with our own
ideas. Last year, Bret Victor wrote one of the best posts on the subject
that we've ever seen, and it fits into what we're driving at here,
although his post focuses mostly on the fact that we only use our
fingertips to interact with devices rather than our whole hands, arms,
etc. The idea is still the same though: we design products in part to
fit a need, but also to maximize our own natural abilities. Right now,
we have designed touchscreen devices to the point that we can very
easily interact with computers in a traditional way, but it is time to
push forward, and we don't see that next step of evolution being to take
away the touch interaction.
Other options
More
likely, the devices that are the next step in evolution for mobile are
concepts like what we've seen from Nokia. Nokia has shown off two
different concepts that we would love to see as a reality far more than
any touch-free gesture system. The Nokia Gem
was an idea for a smartphone that is 100% touch-capable, meaning we
could use our entire hand for interactions. Even more impressive was the
HumanForm concept which not only included full touch capability, but a
flexible design for real physical interaction by bending the device. The
teardrop shape may not be the best for viewing pictures or video, but
obviously that's the least important thing here.
We've also seen flexible displays multiple times, and Samsung swears that they are coming "within a year",
but this only gets us part of the way there. The first couple
generations of flexible screens may not see much use beyond being
unbreakable touchscreens for devices that are no different from what we
already have. We still need flexible components like batteries in order
to be able to really build fully flexible devices.
Either
way, we need more ideas and we need to remember the core philosophy of
design, which is to build things that make sense, not just things that
look cool. A Kinect may make for some cool commercials, but so far it
hasn't added much as far as useful interactions. Sure, it looks cool to
swipe the air in order to navigate a menu, but it is in no way more
efficient or faster than using a remote control. The same idea goes for
touch-free gesture control. Sure, it looks cool and futuristic, but it
is also completely pointless unless you are in a situation where
touching the device is not an option at all. Voice control currently
fits into the same category. It has come quite a long way, and can be
extremely useful, but only if you're in a relatively quiet place and
have an acceptable accent.
Looking one step beyond
The
best options are still out there, but are being overshadowed by these
technologies which have niche uses and make for cool demos, but really
don't push forward devices all that much. The transition from physical
buttons to touchscreens lead to a huge leap forward in functionality
because of the options available with the new method of interaction. The
next step for our mobile devices will create a similar leap in
functionality, because it should make just as much design sense.
Touchscreens allowed for more than just pressing buttons, they opened
the door to gestures and multitouch interactions. Moving to touch-free
doesn't add anything, it only takes away. What we really need is to add
more senses like our sense of touch with kinetic feedback.
And
that is where touch-free controls can really find a revolutionary
application. Once we can combine touch-free controls with
tactile/kinetic feedback, that's where we have something really out of
the future. We never think about it, but all of the "futuristic"
interfaces that we see in movies like Minority Report, Iron Man, or
District 9, where people are interacting with holograms would be
extremely difficult to use without some sort of tactile feedback.
Without that feedback, you would have to be looking at what your hands
are doing at all times in order to be sure you're doing what you
intend.
Remember, we saw versions of laser
keyboards which would project a standard QWERTY keyboard onto a tabletop
and would detect when you hit a "key". The problem was that if you
can't feel the keys under your fingers, you can't touch type, which
drops your efficiency a huge amount. Smartphones have been able to make
touch typing without tactile feedback work because you
are essentially looking at the keyboard all the time anyway. If you are
interacting with a holographic object and you can't feel it, it requires
too much of your attention. However, when we can combine the technology
shown off by Senseg and that shown off by Noalia to create touch-free
gestures with tactile feedback, that's when we'll really be seeing the
future technology we all want.
0 Comment:
Post a Comment