Waisda? Video Labeling Game: Evaluation Report

Monday, January 18th, 2010

The Waisda? (which translates to What’s that?) video labeling game was launched in May 2009. It invites users to tag what they see and hear and receive points for a tag if it matches a tag that their opponent has entered. Waisda? is the world’s first operational video labelling game. The underlying assumption is that tags are most probably valid if there’s mutual agreement. Over 2,000 people played the project and within six months, over 340k tags have been added to over 600 items from the archive. Initial findings have been published earlier, when the pilot period was still running. This evaluation report (PDF download, in Dutch), includes a quantitative and qualitative analysis of the tags, as well as a usability study of the game environment and a study into the incentives that apply to people playing the game. The evaluation report is written by Lotte Belice Baltussen, in collaboration with Maarten Brinkerink and Johan Oomen of the Netherlands Institute for Sound and Vision R&D Department. Researchers at the VU University Amsterdam, Business Web & Media Section, also provided crucial input. The VU University Amsterdam carries out this research in light of their involvement in the PrestoPRIME European research project.

The evaluation report provides evidence that crowdsourcing video annotation in a serious, social game setting can indeed enhance retrieval of video in archives. It features success factors organizations need to take into account in setting up services that aim to actively engage their audiences online. The main conclusions are listed below:



Uncommon Businessmodels

Thursday, February 28th, 2008

FreeWith the introduction of internet the traditional business model for spreading information has been challenged. Whereas before the largest part of the efforts and the investments where spent on the distribution side (printing, storing, selling and fulfillment) the internet (aka The Large Copying Machine) has facilitated easy and cheap distribution. Scientific publishers, who traditionally operated in a closed environment where they sold packages of journals and books through an annual license to libraries, are now (often forced by the community) turning their business model upside down. In this model authors are paying for the publication service in exchange for posting in so-called ‘open access‘ journals, where access is free at the point of use (also read Jan Velterop’s blog The Parachute). In this particular case it looks like a suitable business model has been found, as this model takes advantage of the power of the internet and leads to a greater return on investment for authors (visibility) while securing revenues for the service providers (publishers).

The music and film industry are facing similar issues but have yet to find a grip on the situation; the content is more often than not available for free through peer to peer networks therefore a large part of the incentive to go to a shop and buy a cd or film has vanished. As we are digitizing vast amounts of audio-visual cultural heritage we are facing the same questions: what models can be developed that fulfill the need for broad accessibility for the public while securing a solid return on investment for owners of the material (authors, producers, directors, etc).

Some, like Chris Anderson in his soon to be released new book’ Free’ build an entire economic theory based on the notion that free’ will be the leading model for media due to the vanishing marginal costs of distribution via the internet. The new model that rises from the ashes will be a model where the content or service is free, at least for the user. Google of course is a great example of a company that has turned ‘free’ to it’s advantage; the service is free to users while advertisers are the paying customers. At the core a beautiful system as the more you use the service the more revenue it generates for the service provider. Keeping the attention of the viewer is key in the ‘economy of abundance’, so you better make sure the service you develop is so appealing that users get hooked on it. In fact, if this becomes the case, there may be an opportunity to upsell them from freeloaders to paying customers by adding a an additional layer of services or privileges. This freemium model (term coined by venture capitalist Fred Wilson) has quickly become the leading model for web 2.0 companies like Flicrk and Linkedin. Interesting fact is that the rule of thumb is that the 1% of paying users supports the rest.

The crux of developing business models in this economy of abundance, where content is free, seems to be to tap into values that people are willing to pay for. And those values may not be the same as in the old days where content was king. Kevin Kelly calls them ‘generatives’:

‘’A generative value is a quality or attribute that must be generated, grown, cultivated, nurtured. A generative thing can not be copied, cloned, faked, replicated, counterfeited, or reproduced. It is generated uniquely, in place, over time. In the digital arena, generative qualities add value to free copies, and therefore are something that can be sold.” Think ‘trust’ or ‘personalisation’. In his blog ‘Better than free’ Kevin distiguishes eight of them.

So how does this translate to our audiovisual digitization adventure? Will the specific characteristics of cultural heritage lend itself to open content models like advertisements (google just released a beta service of video advertisements: Adsense for Video ), Freemium services or even community supported businessmodels?

We are hosting a workshop on this topic during the Economies of the Commons conference on saturday April 12 2008 in Amsterdam to investigate the options.


Freebase: the semantic web application

Monday, November 26th, 2007

Another Wikidia-style online encyclopedia has seen the light. But Freebase is something new. Its creator, the company Metaweb is setting out to create a vast public database intended to read by computers rather than people. Users still play an important rule in Freebase. They set the types of relations between pieces of information. People add metadata instead of data. In this way, information will be structured to make it possible for software to define relationships and even meaning. In the words of TechCrunch’ Micheal Arrington: This is cool unless its get consciousness and kills us all.

How does it work?
Freebase logo When logged in (registration is open for the public since november), you can add information on companies, movies, places, restaurants etc, just as in Wikipedia. But you not only enter the data, but also add the types of the information. For example, we choose to add a company to the database. When I entered Knowledgeland and told Freebase it’s a company, a new template with a lot of predefined structure came up, because Metaweb has defined a whole set of additional data that is typically associated with a company. I can choose to enter the empty fields such as employees. When I then click on the name of the employee, it’s relation with the company and it’s type is automatically established. Employees become persons, places become locations etc. And all these new topics come with their own predefined fields. Searching has become a lot more intuitive because you can use the same fields for narrowing down the results. A search string such as ‘show me all the companies in Amsterdam’ is done with two clicks.

Open for everyone
Freebase has already sucked in data from Wikipedia and other sources, and user can fill in their data too. Currently Freebase counts almost 3 million topics. More than 1200 relationships in the form of types have been established between these topics within 68 domains. Just as with Google, developers can extract information from Freebase and add it to their web applications. The information users add is licenced under the Creative Commons Attribution License or Public Domain. Because the information is structured, other web applications can use Freebase to display its information in new ways.

Freebase is interesting not only for its collective intelligence. The workflow of entering metadata is highly intuitive and can function as a blueprint for crowdsourcing purposes. Archives don’t need to worry about the types of relations, users create them on the fly.

Perhaps Freebase marks the start of a new era in gathering information. Perhaps not. But one thing is sure: Freebase in potential the Google killer for harvesting collective intelligence.

Introduction to Freebase (screencast)

Related posts
- Freebase @ Techcrunch
- Tim O’Reilly about Freebase


Education in bits and pieces

Monday, November 5th, 2007

ODE stands for Online Distribution Engine. It aims to be a store where educators can buy little bits of digital content and put them back together any way they like, a proces dubbed ‘Mash up teaching’. This idea has it’s roots in the music business where sampling has become a complete new industry. Slice up the content until you get to the core ingrediënts, an acapella vocal line for example. Now look on your shelf for other ingrediënts that will spice up your recipe, a nice fast drum beat for example. Put them together and by ‘mashing them up’ you have created a new piece of content. Not very different in fact from the way science has been functioning since the age of the Enlightenment. We are all standing on the shoulders of giants as Sir Isaac Newton already said centuries ago.

The question is how this rather refreshing principle will work out in the world of education. This is an environment where changes have been notoriously slow to take root. Where the current generation of young children that have in fact been ‘born digital’ spend their free time between msm, gameconsoles and their pc but receive their education primarily through old fashioned books and whiteboards. So are teachers ready to put their destiny in their own hands and create their own teaching materiaal? The people at ODE world, a wholy owned subsidiary of Hartcourt/Pearson certainly believe they will. Their Beta platform will be ready for launch in the spring of 2008, ready to conquer the UK education market. All they need now is high quality content and a targetted audience willing to make micro-payments for the material of their choice. It is the quality of the content and their ease of use for teaching purposes that will make the difference. That much we did learn from successful ventures in the music industry such as iTunes. For education, the proof will be in the pudding.