IA Summit 2015 Main Conference Talk
Topic(s): techniques and user research
One of the shortcomings of user interviews is the vast gulf between what people think they do versus what they actually do. Fortunately, a new generation of digital research tools are filling that gap by allowing users to provide quick feedback via their smartphone right after using a product. This allows rich details to be captured that might otherwise have been forgotten. These tools are breathing new life into a traditional research tool, the diary study.
This talk shares best practices for creating a digital diary study that collects relevant and insightful data. It will be framed by examples from a recent digital diary study I conducted that explored how people use their fitness trackers (Fitbit, etc).
About the speaker(s)
Sara Cambridge is a User Experience Researcher currently at Google. She has worked in diverse fields such as retail, telecom, fitness trackers, and consumer electronics.
Before getting her master’s degree from UC Berkeley School of Information, she worked as a graphic designer for many years and has significant experience in branding, marketing and interaction design.
Sara Cambridge: What is the best way to learn about the experience that people are having with your product? A lot of people would guess interviews, and you can certainly learn a lot that way. You can learn about what matters to people, how they function, and you can learn generally how they use your product, but you might be missing out on a lot of useful details.
That’s because people forget, really quickly, exactly what they do, especially if they’re multitasking, or they’re doing something that’s a habit, so by the time the interview rolls around, they may have forgotten some critical details that would actually be pretty relevant for you to know.
That’s where diary studies come in. Diary studies are a traditional research tool, usually done on paper, that ask people to self-report right after they do an action. The problem with diary studies has always been getting people to carry them around, pull them out, and then fill them out every time they do that action. But in this day and age, we’re all carrying around Smartphone that we’re pulling out and using all day anyway, so Smartphone are a great tool for running diary studies on.
There’s a lot of ways you can do diary studies on a Smartphone. It could be as simple as a text or email prompt that people reply to, or it could be a platform like the one I’m going to talk about, which is Dscout, and it is an iOS and Android app. This is a platform that we have had a lot of success with, collecting some great data.
I’m going to share with you some of the data we’ve collected in a recent project, but first I have a question. Is anybody here wearing a Fitbit, or a Nike FuelBand, or any kind of fitness tracker? OK, OK. Yeah, quite a few of you. That was the subject of our study, so this might be of particular interest to you.
We recruited 40 people from around the country who are wearing these devices, and asked them to report on five things they do with their tracker over a week, and they submitted a video and answered a few questions. I’m going to tell you a couple of stories that we got.
One is from Erica, and she lives in Georgia, and she is always trying to get to Fitbit’s recommended 10,000-step count every day. She showed in her video clip, she was talking about how she was about 2,500 steps shy, and she was getting ready to take her dog out for a walk, but it was 10:53 PM. That was something that we saw as a pattern, is some people take that recommended step count really seriously.
Another interesting story is from Susan, who’s a grad student in Illinois, and Susan—I’ll just read her quote, because she says it so well—”I got drunk with my friend and convinced her to download Snapchat, on the contingency that I send her a Snapchat every day. I use my Jawbone UP to remind me to send her a picture,” so it vibrates on her wrist at seven and she knows to send her friend a Snapchat.
She has hacked this device to do something not related to fitness, but about helping her build a relationship, which I think says a lot about how deeply this device has been integrated into her life.
There’s a little bit of some of the data we collected, but we all know that collecting data is only part of the story. The rest of it is analyzing it. These platforms offer some really helpful analysis tools as well, so I’m going to give you an example.
One of the questions we asked was, “How do you feel about this aspect of your tracker?” and everyone rated it from great to terrible. I wanted to see if I could find a pattern in the people who weren’t terribly satisfied of what they were talking about. I filtered out the high responses, and then manually went through the rest of them.
In about ten minutes, was able to figure out the three big buckets where people being unhappy with some aspect of the physical band, issues with charging or the battery, and issues with food tracking. It’s great to be able to get a little bit of analysis done really quickly.
That was a little bit about the project that we did with Dscout, some of the rich data that we were able to collect. The whole point is that I just wanted to communicate that there’s a lot of potential in collecting this in-the-moment feedback from users, beyond just what you can collect in an interview after the fact. You might not need it for every project, but on some projects, it can really add a lot of useful complexity.
I’ve got a video, and there’s some flyers on the sheet with the link as well. As you can tell, I’m a big advocate of diary studies. I’ve got a video with some more practical tips, and I’m also more than happy to answer questions if anybody is thinking about doing their diary study and they have some questions. You can contact me at email@example.com.
Thank you guys very much for listening, I really appreciate your time and your attention.