What's the problem we try to solve?

Since Weiser’s vision, more and more interactive artifacts have been created: smartphones, tablets, laptops, TVs, smart watches, voice assistants…. We’ve become users of networks of artifacts rather than of individual ones and furthermore, we share and connect to each other’s digital artifacts. At the same time, the line between each artifact becomes muddled. Each capable of accessing various services and serving several and sometimes overlapping purposes. Accessing a digital service and content has become fragmented and the jarring cognitive switch between artifacts can disrupt thinking, productivity, and be a source of frustration and confusion. Therefore, the need for ubiquity across multiple artifacts and unlocking the full potential of artifacts around us has been enormous.

So our goal was to pursue a continuous cohesive experience, where we ease the transition between devices.

“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are distinguishable from it.” ---- Mark Weiser

1st semi structured interview.
Understand when and why do people use multiple devices? How do people currently do this? What are their frustrations and delights?

2nd semi structured interview.
More diverse age and professionals. More exploratory research to identify opportunities for transition instead of just current situations.

Survey.
Verify why people switch with a larger audience.

Inductive thematic analysis.
Went through all interview transcripts. Coded and generated key themes.

Pain points and design principles.

Users try to finish the task on one device and only consider transition as option B or C.

Concept testing.
We came up with four concepts - user initiated switch on the original device, user initiated switch on the targeted device, system suggested switch on the original device, system suggested switch on the targeted device. We created a concept video (as shown below) and aimed to identify which alternative makes more sense to users in different contexts.

Product viability testing.
Assess practical viability and general usability of the proposed interactions and gather subjective feedback.

Design scope.

In-app level (e.g. Spotify)
Pros: Broader target market - app builder instead of system builder. Apps have more control for customization.
Cons: Users need to locate the application to continue first which could be troublesome. Hard to access system level data.

Third-party level (e.g. Dropbox)
Pros: Highest flexibility - not constrained by apps / systems.
Cons: Will users willing to use one more app to do that? Hard to access system level data.

System level (e.g. Apple Handoff)
Pros: Support integrated system level decision making and reduce bias by separate apps.
Cons: Need to tailor for different ecosystems.

System level decision making is critical to support transition for the right app, at the right time, on the right device.

App switcher.
Similar with Apple Handoff

Pros: Make sense for receiving the app from another device as it’s also to switch to the app to continue.
Cons: For sending apps to other devices it’s not consistent as it’s device switcher instead of app switcher. Discoverability problem.

Control center.


Integrated system level control like Airplay / bluetooth.

In-app share.


Useful for sending the current app to other devices.

Siri suggestion.


Useful for receiving the app from another device.

ML-empowered suggestion.

Why is ML needed?
“I don’t know that I can switch” is the top funnel user pain points. We came up with two alternatives and verified needs through concept testing - user initiated switch and system predictive switch. Users highly praised the discoverability by predictive switch and felt it much more intelligent and convenient. But at the same time, it should be as less intrusive as possible. With sending prediction notifications, we need to be very intentional to make sure it only shows up at the right time, on the right device.


What if ML goes wrong?
False positive: False positives are where the system incorrectly predicts and pushes a suggesiton. Sometimes users pick up another device to work on a different task rather than continue the task. Sometimes users are just in the middle of something and do not have the intention to switch to another device even more appropriate. Notifications can be really distracting.

> False negative: False negative means the suggestion should show up but not which will not affect users too much as users are still able to accomplish a switch through conventional non-predictive methods.


Deal with false positive: Co-learning and adaptation
Whenever users dismiss or ignore a notification, we put it as the top priority that this scenario will never show up any more.

Wouldn’t it be great if contextual clues are presented and users can switch with a single click?

False positive case 1.
User picks up the phone. He wants to work on a different task rather than continue playing music.

False positive case 2.
User is in the middle of browsing. Even a laptop is near him and more suitable for deeper actions, he doesn't have the intention to switch.

Authentication.

What if...?

Before

After.
For non-personal devices, as multiple users may have the access to the device, approval is needed on the targeted device to avoid overwriting.

Before.

After.
Once added, as all devices can access the device’s information and the device can access all devices’ information as well, users need to be aware of the potential outcome of their actions.

Before.
Users only have the option to send, the control will hand over to the targeted device once switched.

After.
Even if users send the current app to another device, users may still want to retain their control on the original device due to the privacy concern.

Before

After.
With a list of devices, it happens when users mistouch the device. Users are always able to cancel the action while switching.