In the context of Images for the Future, The Netherlands Institute for Sound and Vision R&D department has designed and developed a complete technical architecture to support (mobile) location-based services for cultural heritage. This October, Sound and Vision will launch the first implementation based on this architecture in the form of an iPhone application that offers mobile access to relevant audiovisual heritage related to war monuments.

The architecture developed by Sound and Vision is based on a Drupal installation that connects to relevant open content repositories (for now: Open Images, Flickr: The Commons and Wikipedia) using their respective API’s, and enables the creation of (mobile) location-based services for cultural heritage. Locations in the real world are linked to digital heritage available online. By building a location-aware application that utilizes the location information and related audiovisual heritage stored in the Drupal installation, users can access relevant information – based on their GPS location to enrich their on-site experience. The architecture also supports applications to enable users to contribute comments and images to the locations themselves.

To promote reuse and further development of the architecture and to ensure that other institutions can easily participate, the complete architecture is based on open source components. Because all information and relations to the online content are centrally stored, it is relatively easy to develop applications on top of the Drupal based architecture. Because the content is separated from the application, this also makes the applications very lightweight. Applications can be an iOS based application (like our first iPhone application), but with a similar effort this can also be an Android of Symbian based application.

Within the context of the Images for the Future project the Netherlands Institute for Sound and Vision is – as mentioned before on this weblog – exploring potential of mobile and location-based access to digital heritage. The goal is to explore how digital heritage can be combined with mobile technology and location awareness to offer an augmented reality (AR) to different types of end users. In this context AR functions by augmenting the physical reality of end users with contextual information using digital means.

As part of this exploration Sound and Vision partnered with the V2_ Institute for the Unstable Media to initiate a multidisciplinary research project (later named ClosAR). The goal of the collaboration was explore new approaches to bring the audiovisual (AV) archive closer to its audience using innovative methods of presentation and interaction, such as augmented reality. The ClosAR group is comprised of five members with a background in new media studies, art science, industrial design engineering, and virtual games.

In their final report (you can download a PDF version here) Bas Bergervoet, Kate Cunningham, Aestha van Dam, Shauna Jin and Connie Yeh propose seven concept directions as possible starting points for Sound and Vision and V2_, with references to the background research and intermediate process:

The classic archive is static: a tomb where “official” documents, media, and information collect dust. Digital technology opens opportunities for information access and presentation, and most notably, participation. As the archive only exists to be accessed, the relationship between itself and its audience is quite important. Augmented reality (AR) brings dynamic information into the physical world. While Beeld en Geluid seeks to improve its visibility outside of its “Experience” and on- site archive search through mobile access, we explored the off-site extreme. How does moving the context of the archive open up possibilities for new interactions? What is the minimum amount of technology needed to achieve this?

This year’s theme “web.sharing.tv.content” created a space for a very broad range of topics, not only focussing on the television itself, but also on television content on the Internet, the mobile phone and crossovers between television and various other (social) media.

The first day of the conference was dedicated to tutorials, mainly about designing interactive television applications, and workshops about user experiences, the future of television and integrated, ambient television applications. The second and third day were filled with poster presentations, demonstrations and various sessions with a broad range of topics like personalization and recommendation systems, interactive applications and convergence and cross-media. These two days also included two keynotes by Marcos Gonzalez-Flower, Global Head of Media Consulting at Siemens UK and Jyri Huopaniemi, Director of the Nokia Research Center.

The Netherlands Institute for Sound and Vision has made a substantial contribution to the conference with a short paper presentation about expert search by broadcast professionals, two demonstrations about television and the semantic web, and browsing archival video with advanced video retrieval and last but not least, the Golden Award (first prize) for Waisda? in the Grand Challenge Competition.

Some conference highlights:

Methods for User Studies of Interactive (TV) Technologies

This full day workshop was chaired by Dr. Paul Marrow from BT Innovative & Design, Dr. Lydia Meesters from Eindhoven University of Technology and Prof. Marianna Obrist from the ICT&S Center of the University Salzburg. As the title of the workshop already suggests, this workshop wasn’t just about user studies methodologies for television technologies, but also for interactive technologies in general. Measuring user experiences for any interactive application is a very complex form of research, which requires a very pragmatic and ‘open’ research attitude. This workshop focussed on the many approaches that are used for measuring different levels of user experience. Approximately 20 people joined this workshop, which are all researchers working for various companies and universities, and doing research on interactive applications.

The morning programme contained presentations from the workshop participants about the various approaches and practices in researching user experience. In the afternoon session, participants were asked to pose a number of research questions and problems. After clustering these into main themes, the participants split up in three groups, where one theme was discussed in more detail. The results of the discussions were documented in a poster and presented to the other groups. The main insight of this day was that user experience has to be measured by combining various research methods, both quantitative and qualitative, and that the various stages within the product development require different forms of testing with different user groups.

Session ITV Applications for Communities

This session was scheduled to contain four presentations: two on mobile video sharing and two on digital archiving. Unfortunately, the speaker of the presentation about user inspired digital archiving of cultural heritage wasn’t able to attend, so the focus was mainly drawn towards video on mobile devices.

Mobile Video Sharing: Documentation Tools for Working Communities – Antti Koivisto

Antti Koivisto from Tampere University of Technology presented the development and the infrastructure of MoViE, a mobile application that allows users to create short clips of their environment and upload these to a website where they can share, edit and remix the content Together with his colleagues from Tampere university, he did a first test with the prototype during a festival and although users liked the application a lot, there were some technical issues like a failing Internet connection that made the use of the prototype difficult at times. The test results were used to improve the MoViE application and they are currently working on a new version.

Troubleshooting and Creating Joined Experience: Festival Personnel Engaging in Mobile Video – Anna Haverinen

This talk by Anna Haverinen from the University of Turku reported the results of the first MoViE tests from a user perspective. As a virtual anthropologist, she analysed the kind of content the test subjects created during the test. Eight festival employees acted as test subjects and were asked to shoot and upload videos during the festival. She discovered a great difference between the remarks that people made about the application and the actual use of it. For instance, some people for instance indicated that a commenting tool was very helpful and easy, without actually using it in practice.

Expert Search for Radio and Television: a Case Study amongst Dutch Broadcast Professionals – Wietske van den Heuvel

This talk was given by Wietske van den Heuvel from Sound and Vision. The presentation highlighted the results from a study in which eight broadcast professionals from various broadcasting companies were observed while performing searches with the online search system of Sound and Vision. The research focussed on the question if the task affects the professionals’ search strategy. Three kinds of factors were researched: difference in searching for radio or television content, difference in length (shots versus entire programs) and the influence of time restraints. Results from the study show there is no difference between the search strategies that are used during the performance of the tasks. People always use the same strategy, despite their information need.

The EuroITV 2010 Grand Challenge Competition

The Grand Challenge Competition honours the creation of interactive video content, applications, and services that enhance the television and video viewing experience. The entries are judged by an international jury of interactive media experts and the best entry is awarded with a prize of € 3,000. Three entries were nominated before the start of the conference:

1. Smart Video Buddy: an application that analyses video streams and automatically detects semantic concepts, which are used to create links to related content.
2. Smeet: one of the fastest growing online 3D world and chat community for entertainment and social viewing on the Internet.
3. Waisda?: a video labeling game by Sound and Vision and KRO Broadcasting. The labels or ‘tags’added by Waisda? players can be used as time-related metadata, which improves the findability of the content.

The awards ceremony took place during the first keynote session on June 10, where Waisda? was named as the Grand Challenge winner. The nominated applications were also demonstrated during the rest of the conference day.

Demonstrations

The programme included various interesting demonstrations, two of which were given by Johan Oomen from Sound and Vision. The first demonstration concerned Video Active and EUscreen . Both projects provide free online access to Europe’s television heritage. Video Active ended in August 2009 and EUscreen started as its follow-up project in October 2009. Both projects are funded within the eContentplus programme of the European Commision. EUscreen will contain over 30,000 items and is one of the main audiovisual aggregators for Europeana. The EUscreen portal will be online in January 2011. The second demonstration showed the project Hollands Glorie op Pinkpop, which ended in February 2010. The project is a cooperation between Sound and Vision, Videodock, the University of Amsterdam and the University of Twente. By using automatic concept detection and speech recognition, videos of pop concerts and interviews with the musicians are searchable on individual shot level. Videos from the pop festival Pinkpop were used for this public demo website. Users could also give feedback on the level of retrieval and indicate if the concept detection was right or wrong. This feedback will be used to improve the concept detection technology.

Further reading

Read the full conference programme here
Conference report by CTOi Consulting

1. Digitisation makes piracy irrelevant
2. The Economics of Copyright and Digitisation
3. Chicago Tribune begins digitizing and selling archive photographs
4. Pleidooi voor archivering van digitale kunst en cultuur
5. Google’s new search index: Caffeine
6. Opta: Nederland heeft hoogste breedbandpenetratie ter wereld
7. Nederland zet cultuur op de kaart
8. Wat is de waarde van kunst en cultuur? En voor wie heeft het waarde?
9. When the copy’s no exception – Interview with Paul Keller

Invited speakers and panelists:

Carmen Iannacone, Chief Technology Officer for the Smithsonian Institution. He is responsible for evaluating and implementing new technologies  for use in the Smithsonian IT infrastructure, and for optimizing  performance of its IT operations. The Institution’s nineteen museums and  research stations provide a diverse technology climate, and his role is  an integral liaison between centralized IT and the public. Prior to  joining the Smithsonian, Carmen Iannacone served as director of  worldwide IT operations for the Federal Acquisition Service of the  General Services Administration (GSA), and was partner a software  development consultancy in Alexandria, VA. He holds several patents for  software and lives in Sterling, VA. More information about Smithsonian  Institution: http://www.si.edu/

Anthony Jameson, Principal  Researcher in the Intelligent User Interfaces department at DFKI , the German  Research Center for Artificial Intelligence since 2001.    Through November 2009, he is visiting FBK-irst to direct the targeted  research unit Prevolution (funded by the Autonomous Province of Trento)  which studies the formation and evolution of users’ preferences, in  particular toward intelligent interactive systems. Information about a  workshop on this topic that he recently conducted at IUI 2009 with  Silvia Gabrielli and Antti Oulasvirta can be found at http://prevolution.fbk.eu .  Anthony Jameson has a B.A. in Philosophy from Harvard College, a  Diploma in Psychology from the University of Hamburg, a PhD in  Psychology from the University of Amsterdam, and a Habilitation in  Cognitive Science from Saarland University. Before moving to DFKI, he  conducted research in various areas of human-computer interaction and  artificial intelligence at the following institutions: University  of Hamburg, University of Nijmegen and Saarland University. Anthony  Jameson has also been an adjunct professor of  human-computer interaction at the International University in Germany since  2001. Much more detailed information will be found on his DFKI homepage .

Richard van Alphen, database  administrator at the Tropenmuseum .  He is responsible for managing The Museum System  (TMS), a collection management system that functions at the core of the  organization. He is responsible for quality control and user support,  and for sustainable media storage. Besides, Richard works as a  consultant/trainer in capacity building with partner-institutions in  developing countries.

Frank Meijer, project manager digitisation at the Tropenmuseum in  Amsterdam as well as project manager of the digital archive and website  of the Ridderlijke Duitsche Orde .

Rik Hoekstra, head of digital source publications at the institute of Netherlands History in The Hague and also responsible for the technical development of Biography Portal of the Netherlands (www.biografischporaal.nl <http://www.biografischportaal.nl> ), where accessibility, contextualization and knowledge discovery have long been the central issues in online publishing.

The meeting starts at 12:00 at the Amsterdam Historisch Museum in Amsterdam and is open to all.

There is no registration fee. Please register yourself at the NWO website.

The programme:

12:00 – 13:00 Registration and lunch
13:00 – 13:00 Welcome by CATCH programme committee (Paul  Doorenbosch, National Library of The Netherlands)
13:00 – 13:45 Presentation by Carmen Iannacone (Smithsonian  Institution, Washington, US)
13:45 – 14:30 Presentation by Anthony Jameson (FBK – Fondazione  Bruno Kessler, Trento, Italy)
14:30 – 15:00 Coffee and tea break
15:00 – 15:20 Agora demo presentation
15:20 – 15:50 Presentation by Frank Meijer and Richard van Alphen  (Royal Tropical Institute Amsterdam)
15:50 – 16:40 Panel chaired by Guus Schreiber (Free University  Amsterdam)
16:40 – 17:00 Update CATCHPlus by project team (www.catchplus.nl <http://www.catchplus.nl/> )
17:00 – 18:00 Reception

1. Spotify: Wapen tegen illegaal downloaden
2. Archief Beeld en Geluid ook voor Suriname
3. Layar announces Layar Stream: search and discovery for Augmented Reality
4. Staatssecretaris OC&W noemt Open Beelden als praktijkvoorbeeld van open video
5. European Parliament backs digital plans
6. Grote beeldcollecties visualiseren met Pivot
7. XS4ALL wil ‘Muziek en film 4ALL’.
8. Video search still a tough nut to crack
9. Public Domain Calculators at Europeana