What we need is a good standards war

[reposted from QS]
I’ve been meaning to link to this post for a couple of weeks. Nathan Yau over at Flowing Data has been writing personal data collection projects quite a bit. In this post, A Perfect Personal Data Collection Application, he talks about what is missing from current tools and about his dream system for personal data collection.

The number of Web applications to collect data and information about yourself continues to grow; if you want to track something, most likely there’s an online tool to do it. This is great – especially since a lot of the applications seem to have a lot of users, which means an interest in data… However, as users, developers, and designers, we shouldn’t be satisfied too quickly with what we have. Want more. Demand more. It’s interesting and oftentimes fun to log data about your life – whether it be when you go the bathroom, your sugar levels, or your mood. You get some nice graphs and charts, it looks cool, and maybe you learn something about yourself.

But all the self-surveillance tools so far are mostly about a single dataset or two at most. You track your weight and what you eat, but it’s more complex than that. Life is complicated and data is an abstraction of life after all. Do you eat when you’re depressed or are you depressed when you eat? Do you feel better if you exercise? What about sleep? How much sleep and exercise is best for you? What days should you exericse and how many days in a row and for how long? What truly makes you happy? I want my self-surveillance application to not only give me the ability to find these answers but to give them to me with very little effort on my part.

Nathan argues that any good solution for part of the problem ought to at least aspire to solve all of it. He wants the tools to include some data processing, and to be ubiquitous, so that you can post from anywhere.

In the end, I want all of my data in one place with some machine learning in the background and the ability to analyze and visualize easily and thoroughly. We’re not quite there yet, but I’m looking forward to when we do. Information overload? No. Better-educated decisions and a completely different view of ourselves and our surroundings? Definitely.

Nathan is building his own multi-tracker at your.flowingdata.com. Right now it is by invitation only, but you can follow yfd on Twitter to connect to the next wave of invites.

At the second QS Show&Tell, Joe Betts-Lacroix gave a short talk about his dream system: a website that could receive data and put it into a database, with data would be gathered by little devices that could beam it to the web using simple protocols. (The picture below is of Dan Brown, not Joe Betts-Lacroix; Dan happened to be in the first frame of this segment of video, which is automatically used for reference. Joe shows up a few seconds in.)

In this version of the dream the ideal Web site would have some
simple graphic tools, the ability to export data, good security, and some sharing and privacy options. Since then, there have been quite a few demonstrations of various ideas at the QS Show&Tell meetings, as well as a steady stream of products and, naturally, announcements of products (cf. Fitbit) that aim to achieve some parts of what Nathan, Joe, and other self-trackers have called for.

My own vision is slightly different. I think we are inevitably going to see a bunch of competing solutions, most of which will seem pretty good to some people and deeply flawed to others. People come at self-tracking with different goals and values. Daytum, which is mainly about self-expression, will be nifty for the person who uses data mainly as a feature of personal identity. Daytum’s origin is in the Feltron Annual Report by Nicholas Felton; an annual report serves many purposes, but data analysis is not one of them. Meanwhile, Zume Life is designed for people tracking serious health conditions, who are often trying to manage complex prescription drug regimens. Zume Life has made the data entry process almost as easy as imaginable until the day arrives when we can beam our data from monitoring devices without intervening at all. Along with an iPhone app, there is a voice transcription service. Just press a button, say your number (or food eaten, or exercise accomplished) and a person will transcribe it into a database. This is going forward by going backward, and there is a kind of genius to it. The type of person who likes Daytum is not going to bother with Zume Life, and for the user of Zume Life, Daytum is pointless.

We are headed into a messy, confusing, and interesting period in self-tracking, when lots of new solutions emerge, each claiming a piece of territory and pushing up against neighbors. There will be no ideal system, but a bunch of different sytesms, and then a bunch of different solutions for gluing different parts of different systems together. Some of the people who manage to aggregate lots of users will find excuses for holding on to them. (This may not always be a bad thing: see Dead ends and walled gardens.) But others will see that making the data collected on their system available in standard form will speed adoption.

But what is the standard? You can insert your own favorite standards horror story here. But after you’ve given yourself the shivers, you can recover with the realization that this conflict over standards occurs because everybody can finally see the prize they are wrestling for, which means that a substantial amount of agreement has been won. Once important people start making highly emotional arguments for how quotidian personal data (QPD – now it’s official) should be represented, you can start celebrating.

Note that I said “quotidian personal data” and not “healthcare information.” QPD is the type of thing you are willing transmit in a text message, and SMS is an easy bet for the ubiquitous medium for QPD. But if you call it healthcare information, I’m out. They fight dirty.