Archive for the ‘data’ Category
-
Mapping Twitter Part 2: The Tweet-o-Meter
10th March 10
Came across this today. Tweet-o-Meter (link) is the beta version of a platform created by University College London’s Centre for Advanced Spatial Analysis. The Tweet-o-Meter supposedly updates every ten seconds (not sure it does quite do that right now), showing the number of tweets in each city per minute. The ambition is to log and analyze all geo-located tweets in these major cities. Once logged, they will be used to show Twitter activity over time and space. Various kinds of maps will be the main output. I imagine a variety of delicious visualizations will be forthcoming.
We are possibly attracted partly by the simple analogue-feel, dial-based interface. But we’re also struck by yet another work-in-progress attempt to bring life to the data spawned by Twitter (see also Getting to Know Your Twitter Followers & Why that Matters from earlier this week).
Tweet-o-Meter is part of a broader project called NeISS (National e-Infrastructure for Social Simulation), another UK Government-funded project. Read more about it here.
And of course it also reminds us of of the work by Google’s Aaron Koblin on visualizing SMS messages sent on New Year’s Eve in Amsterdam in 2007 (see below). We imagine as Tweet-o-Meter moves forward through beta they’ll need to figure out how to marry Koblin-esque visualizations to their gushing pipe of data. Bringing magic to the mayhem.
Amsterdam SMS messages on New Years Eve from Aaron on Vimeo.
-
Getting to know your Twitter followers & why that matters
8th March 10
Posted in awesomeness, data
Last week Aaron Richard (@ralphthemagi) contacted us at BBH Labs with something pretty cool, and we wanted to share it.
Aaron was most recently a digital strategist at Big Spaceship in Brooklyn. A while back Aaron created a map showing where @bigspaceship’s many thousands of followers lived (or claimed to live). I contacted Michael Lebowitz at BS and asked how they’d done it . . . a few days later Aaron wrote to us with our very own version of the data, mapped and analyzed. Brilliant.
Aaron goes into great detail on his site about how he did this, the problems he encountered, the choices he made in filtering, and so on. In short, he used the publicly accessible Twitter API combined with cURL software to play around with the data shared by our c.12,600 followers on Twitter.
After some fairly smart sounding parsing of the follower base to weed out spammers (or at least people who looked most like spammers) and non-actives (see his post for the detail) Aaron pulled down the following public data on each of the remaining followers.
- ID
- Name
- Username
- Location
- Profile Bio
- Profile Picture
- Web URL
- Privacy Settings
- # of Followers
- # of Friends (“following”)
- Account Creation Date
- # of Favorites
- UTC Offest
- Time Zone
- Per-tweet Geolocation Status
- Verified User Status
- # of Tweets
He then used one of Google’s Lab projects, Fusion Tables, to geo-code the massive amount of information he had in CSV form.
The result was two forms of map. First, a fully interactive Google map (launch it and take a look, click on the dots for detail), and second a heatmap showing concentration of followers by major cities. With the interactive map it’s possible to click on a follower and see the data that Twitter holds for them (which is a little scary, but I guess comes with the territory).
Aaron also looked at our follower data and pulled put out some insight about our followers, which we found fascinating.
- Average # of followers: 1,746 | Median: 163
- Average # of friends: 982 | Median: 206
- Average # of tweets: 987 | Median: 247
- 6% of followers keep their tweets private
- 9% have per-tweet geolocation enabled
- 12 followers are “verified”
As Aaron notes, one can see by the deltas between means and medians, all followers are not created equal.
So all this is fascinating to us (for example, to learn that @bigspaceship and @BBHLabs share the same two followers in Iceland . . . hi Islenka and Finnur). But I wanted to see what additional uses might be made of this kind of data and insight. For example, for brands, or for non-profits, or just for individuals. I pinged Aaron a few questions on this theme:
BBH LABS: So Aaron, thanks for this - this is fantastic. But thinking more broadly of potential uses of this kind of insight for marketers, brands and individuals, how do you think this might be used in a more applied way?
AARON: I think this kind of information can be used for setting better goals. Asking better questions and finding better answers. I think a lot of brand teams have this preconceived notion that they are using social media effectively if they have a lot of fans, followers, etc … I just don’t think that’s true.
BBH LABS: Give us some examples of what you mean.
AARON: The particular data set I pulled for BBH could be used in a number of ways. For example, say you wanted to give away something to a few Twitter followers with the goal of growing your network. Send them an iPod Shuffle, get them to tweet about it, drive a little positive PR. But how would you decide who to give stuff to if you wanted to maximize every give away? Well, with data like this you could easily find the top 20 people with the most followers and target them. Or look at the top 50 people with the most followers, then look at those with who have the least number of tweets (there’s something interesting about people with a lot of followers and few tweets, because when they do tweet their message tends to get retweeted a lot and cuts through the clutter).
BBH LABS: And for brands, can you give us an example of how they might make use of this? Maybe to make their stream more relevant? Maybe to get closer to their most valuable customers?
AARON: Sure. You can start to see how you might use this kind of information to challenge large incumbent brands. Imagine you wanted to take on Comcast as a small regional ISP. You could pull the data for everyone who follows Comcast Cares [on Twitter] then look at all the people in your region and start following them or sending them public messages. You could even target the people who are pissed off at Comcast and give them a special offer. Dell Outlet [on Twitter] has +1.5m followers. That’s 1.5m potential new customers for HP, if they provide the right incentive to get a customer to switch.
BBH LABS: This is only one particular series of API calls, as you point out. What else can you envisage coming out of the Twitter API?
AARON: Absolutely, this is really just one tiny piece of the data that’s available. I did this more for fun and to get a better idea of how to manage large API pulled data sets than I did to answer a specific question. Twitter has calls for search, tweets, retweets, lists, etc.. If, for example, you wanted to track something like brand mentions you could do that—and not just by using the regular old search.twitter.com or paying for something like radian6 (who’d never give you the raw data). You could look at all tweets by keyword, replies, retweets, etc., and then figure out who’s saying these things, where they live, and what (or who) they have in common.
I’m going to do a followup to this that talks about how to use API data in a more tactical way, using Facebook (and probably Coke) as an example to find the answer to things like, “What day of the week should I post something in order to maximize likes, comments, etc.?”
BBH LABS: Thanks again Aaron. Keep us in the loop. We’re keen to learn more as we go.
—
If you have any questions for Aaron feel free to post them under this post, or on Aaron’s own blog.
-
“Information Wants To Be Free” - The Razorfish FEED report
17th November 09
Last week saw the publication of FEED 2009 and an accompanying Ad Age article by its primary author, Garrick Schmitt.
The third year of this annual, US-based report, the 2009 edition makes a bit of a departure, with the emphasis squarely on brands and the degree to which digital brand experiences shape & drive purchase. It’s received a mix of high praise and some criticism. We’ve found the report itself and the reactions to it thought-provoking stuff, so caught up with Garrick who kindly agreed to mull over a few questions with us. Here’s a run-through of what particularly caught our attention.
-
Music Retail: The Rise of Digital
13th November 09
This is a good summary of some of the key shifts in music retail (although US-only data).
But what’s also really interesting is that it’s coming from a financial services company: mint.com
mint.com’s service - already brilliant on the web, and on a very strong iPhone app, now seems to be extending into data visualization and cultural commentary.
-
Mary Meeker’s Economy & Internet Trends Presentation 2009
21st October 09
Posted in data, interactive
I first came across this last year, and found it to be one of the best written and most insightful papers of the year.
At first glance this year’s presentation, posted yesterday (20th Oct) looks equally essential reading. See what you think.
-
“Do not glorify aesthetics”: a manifesto for Data Visualisation?
2nd September 09
We’re moderately obsessed with the world of data visualisaton here at Labs for a number of reasons: the ability to generate fresh insight from extraordinarily complex data sets, the ability to trigger radical reappraisal of familiar problems, the ability to put consumers in control of the vast quantities of personal data they generate every day. Not to mention the extraordinary fusion of technology and creativity it represents.
We firmly believe that data visualisation has a wealth of exciting commercial applications, from communicating in new ways to developing new tools, apps and utilities for clients and consumers alike. So we’ve grown slightly frustrated by the rise of visualisations that are moderately pretty but add little in terms of real insight, utility or illumination.
We’re also, as we may have mentioned, big fans of Manuel Lima here at Labs. So we were intrigued to see that he has authored an “Information Visualisation Manifesto”, a provocative (but characteristically generous and nuanced) take on the future of data visualisation which tackles head on the thorny questions at the heart of this ever-expanding field:
- Art versus Science
- Intrigue versus Immediacy
- Aesthetics versus apprehension.
Manuel comes down firmly on the side of clarity of communication versus visualisation for visualisation’s sake, citing the discipline’s roots in the desire “to facilitate understanding and aid cognition” and a growing frustration with the “eye candy” approach to the craft. Many of his principles are rooted in this utilitarian approach, reading almost like a Bauhaus manifesto (and none the worse for that):
- Form follows Function
- Do not glorify Aesthetics
- Look for relevancy
- Aspire for Knowledge
It’s a bold, purist and punchy vision yet also acknowledges the power of narrative and the role of intrigue. Indeed the question of narrative seems to lie at the heart of this Manifesto; the need to pose a specific question of the data and to weave coherent themes and stories from it. These themes then drive the aesthetic approach. As Manuel puts it:
“Form doesn’t follow data. Data is incongruent by nature. Form follows a purpose, and in the case of Information Visualisation, Form follows Revelation”
This is perhaps the key distinction between Information Visualisation as defined here and what Manuel suggests we start thinking of as “Information Art”. Within this approach, artists will freely allow form to follow data, using the random-ness this creates to add texture and interest. Take, for example, Aaron Koblin’s desire to embrace the random-ness of a data set and indeed the richness and texture added to his famous Radiohead video by “interrupting the data”:
“I think it really gives character, because I think it’s really that kind of intricacy and detail that builds character and in a sense it’s the errors and flaws that make art”.
Incongruity making art: Aaron Koblin's "House of Cards" promo for Radiohead
Both approaches are undoubtedly valid. Within any medium there will be times when we seek immediacy and times when we are prepared to be intrigued and to explore. There will be times when we want to understand our world better and times when we want to turn perceptions of it on its head. I can think of few practical applications of, say, the “Synchronous Objects” visualisation series but it mashes up art forms and messes with my mind in a truly delightful way.As ever, then, we need to return to objectives, to ask what we are trying to achieve:
- Do we want to educate around an issue, making complex questions simple?
- To shift perceptions and provoke a response?
- To offer a fresh perspective on an infrastructure question for our clients?
- To offer our consumers better comprehension and control of their behaviours?
Simply put, are we going to offer something that is either very, very useful or very, very beautiful? Either way, greater clarity of intent and greater discipline throughout the industry can only be an advantage in building credibility and engagement. Building that credibiltiy is vital if data viz is going to become not just an entertaining diversion but a vital tool for navigating a world generating more and richer data by the second.
If what we are building is neither very beautiful nor very useful, to Manuel’s final point “Avoid Gratuitous visualisations”: “Simply conveying data in a visual form, without shedding light on the portrayed subject, or even making it more complex, can only be considered a failure”.
Or as William Morris put it: “Have nothing in your house that you do not know to be useful, or believe to be beautiful”.
-
From Art to Apps: Data Visualisation finds a purpose
27th August 09
Posted in creativity, data, design, guest
Author: Jim Carroll, Chairman, BBH London
I recently attended an excellent Made by Many event hosted at BBH which featured a re-presentation by Manuel Lima of his 2009 TED talk on data visualisation. Manuel is the curator of visualcomplexity.com and is an eloquent, modest, charming pioneer in this fascinating field.
As a novice myself, I could not help wondering why we are all so immediately and instinctively attracted to the best of data visualisation.To start with, I’m sure there is some fundamental truth that for most of us data become meaningful only when we can see scale, change, patterns and relationships. Seeing is understanding.
It’s also very reassuring to discover that complex, seemingly chaotic data sets and networks can be expressed as elegant, colourful, ordered maps and models. Perhaps there’s something akin to what the Enlightenment scientists felt as every new discovery revealed the endless beauty of nature.
Indeed the best examples of data visualisation have their own aesthetic beauty. (I felt a nostalgic pang as I recalled time spent with spirograph in my bedroom as a child.)
Like spirograph, but better: Email map by Christopher Baker
-
“An Epochal Debate over the Value of Content”
26th May 09
That’s how Rupert Murdoch recently summed up the current relationship between online publishers and aggregators during a call with his shareholders.
He ended with a shot across the bow: “The current days of the Internet will soon be over.”
It’s about to get interesting.
Back when we were floating high inside the web 1.0 bubble, it became indisputably accepted that online content was going to be free and advertising was going to pay for it. And until recently this worked since there was still enough media money in circulation to fuel experimentation and allow digital to continue as a loss leader with an eye toward the future.
But things have changed. Quickly.
Long tail economics are working swimmingly for the aggregates – the blog networks, ad networks, search engines, etc – prosper through triangulation while those that actually create the content that gives these engines their value die a little more each day. Watch in the coming months as the providers, who are now quite literally in a fight for survival, begin to circle the wagons and shoot back.
But within this climate there is also real promise. Necessity being the mother of invention, we may now (finally) begin to see the growth of micropayments in our near future.
Numerous companies have already tried and failed to introduce these systems, but please keep in mind that only a few years ago, it was predicted that consumers wouldn’t trust online security in large enough numbers to sustain retail on a mass level. Consumers are increasingly willing to pay for great online content, it is the high, one time price tag and the hassle of inputting credit card info that is the barrier keeping publishers from our money.
When the barriers are removed, we are generally more than willing to pay 25 cents for a text, 99 cents for a song, so why not 1 cent for an article?
With the introduction of internet ez-pass type payments, users will be able to pass through web pages fractions of a cent at a time. From video games to recipes, from pornography to journalism, this will allow the actual creators to be properly compensated for their work.
Individuals like former Time editor Walter Isaacson and start-ups like Kachingle are pushing just such sytems. But leading this charge will likely require new habit-changing products like the Kindle, which is already beginning to do for print what iPhone did for music. Or more immediately, the new iPhone itself which will change the whole game again this summer by allowing for third party micropayments within its upcoming software update.
In our new data-driven world, micropayments might begin to apply to how creative agencies are compensated as well. Creative and media will likely increasingly begin merging services, molding to a more performance-based system. This doesn’t need to adversely effect creativity though, since appealing to more sophisticated eyeballs might pay better than the blunderbuss approach.
Watch for Labs to be dabbling in exactly these kinds of methods in the months ahead. We welcome further conversation by potentially interested partners and clients here.
-
“I’ve always been interested in microscopes”: an interview with Aaron Koblin
12th May 09
Posted in creativity, crowdsourcing, data
"House of Cards" promo www.aaronkoblin.com
As you may just have heard (we’ve been a tad over-excited…) data visualisation maestro Aaron Koblin came into to talk to us yesterday. He kicked off with a showcase of his work, from his exquisite grad school visualisations of flight paths (see post below) to his latest embryonic projects for Google labs. Along the way he showcased extraordinary visualisations of the ebb and flow of information in cities and around the globe, experiments in crowdsourced sound design and perhaps his most famous project, the Radiohead “House of Cards” promo.
Visualisation of SMS messages in Amsterdam www.aaronkoblin.com
In showcasing his extraordinary portfolio he touched on a number of powerful and provocative themes which we followed up on in our interview. Themes around the power of social context to make data compelling, the power of data visualisation to embrace the complexity of our lives today and the tension between the human and the machine present in crowd-sourcing engines. He also shared his key learnings from life at the front line of data visualisation:
Looking at everyday things in new ways completely changes your perspective: there is no ”mundane” data when you set it in context.
Use multiple visualisation techniques: there’s more than one way of seeing things
Stay true to the data, not the “real world” : There is a random-ness to data-it will make patterns you never anticipated. Respect the random-ness.
You don’t have to use all the data : sometimes seeing patterns is about what you leave out
Set the data free: open-source and let other people play with your data
Following his talk, very graciously agreed to be interviewed by Labs about our (and your) burning questions around data visualisation. It was a fascinating conversation for us and we hope for you. So over to Aaron….and many thanks to those who submitted questions for him.
Why do you think the world has suddenly gone crazy for data visualisation? 18 months ago it was a struggle to get anyone interested in data and now it’s the new rock and roll…
I guess it’s really the times that we live in, now you have tools like Twitter and Facebook and things that are widely used not just by the nerds but by everybody. Popular culture has also just all of a sudden embraced the power of storytelling through data and the relevance of all the data to their lives. All kinds of things have happened that simply weren’t possible before - the author you look up to, the musician, etc. they’re sharing all kinds of things - you can be intimately living their lives along with them and you see all different types of applications.
Do you think it’s partly about the explosion in the amount of data currently available, the data trail we leave behind us now or the fact that companies have more data than they can process so they end up giving it away?
I think ultimately the biggest change is that the data is now relevant to people’s lives. Before most of the data was about infrastructure at best and a lot of it was locked away or presented in aggregate form. When you’re presented with a huge lump sum number that has no context it’s just not interesting, but now when you get these granular stories, things that are saying at this specific point in time here’s the way that things changed, just by giving it that context and social relevance it becomes interesting.
-
“Data tells stories about our lives”
11th May 09
Posted in creativity, data, design
- Mind blowing: Flight patterns by Aaron Koblin http://www.aaronkoblin.com/
If data visualisation is the new rock and roll, Elvis has (just) left the building. Aaron Koblin played to an enthralled audience of BBH-ers this afternoon, blew our minds and incredibly kindly agreed to be interviewed by Labs afterwards.Our interview to follow soon, but to whet your appetite, a quick download of our (and your) key questions for the rock star of the data visualisation world.
Balancing immediacy and intrigue: A frequent criticism of data visualisation is that while often extremely beautiful, sometimes it doesn’t make the information contained any clearer-it can sometimes even seem to obfuscate in the name of art. Should great data visualisation simplify or should it embrace complexity and reward exploration? Should it be reductive or expansive in intent?Where left brain meets right brain: When embarking on a project, which comes first, the data or the technique? How critical a role does software play? Do the themes and memes recurring in data visualisation reflect the artists’ preoccupations or the data sets available?Proliferation versus privacy: One of the key enablers of data visualisation is the phenomenal explosion in the amounts of data we now generate everywhere we go. We live in a golden age of open-ness around personal data but will we reach a tipping point where we reclaim our personal privacies? Or will we opt in to share anonymised data for the common good?The power of synesthesia: Some of the most compelling data visualisation projects are those which express one medium-almost one sense- by means of another. Visually representing dance or music, aurally representing data sets-what is it we find so compelling about this “synesthetic” effect?Crowd-sourcing versus the wisdom of the crowd: Koblin’s recent work experiments with crowd-sourcing but suggests an ambivalence about the process. While a central theme of data visualisation is the wisdom of the crowd, how does it skew the data if the crowd knows it’s being watched? Is the unconscious wisdom of the crowd purer and more compelling or is conscious collaboration of the masses the future? How important is the role of the curator in that process?
Answers - or at least compelling and considered answers - on a blogpost near you shortly….
Ben Malbon
Mel Exon
Recent comments: