Overview
VISNET II includes a wide range of jointly executed research activities that span the spectrum of networked audiovisual system and home platforms. These activities are clustered in three themes: video coding, audiovisual processing and security, and are further grouped into workpackages. Each activity deals with a specific area and bringing together a number of partners that work on this topic.
Coding Theme
The first theme is content coding, a critical element of networked AV systems. Advance video coding will be treated in WP1.1, with the aim to advance the state of the art through a number of integrated research activities. These activities are closely related in the sense that they share some basic tools and techniques. These synergies will be exploited during the research work. Furthermore, in-theme integration of the video coding theme will be addressed in WP1.2.
More specifically, the following topics will be addressed in WP1.1, aiming at efficient coding of visual content:
- Scalable Video Coding (SVC): The main goal of scalable video coding is to produce a compressed bitstream from which it is possible to extract multiple representations of the video. The objective of this activity is to define scalable video coding schemes and to assess their level of objective and subjective performance as well as their functional benefits and limitations.
- Distributed Video Coding (DVC): This activity will focus on distributed video coding (DVC) in order to explore the potential offered by this new coding paradigm to provide flexible distribution of complexity between the encoder and the decoder as well as robustness to channel losses.
- Non-normative tools: This task addresses the development of efficient tools for the modules that are non-normative in video coding standards such as rate control and error resilience.
Related to these research activities, in-theme integration activities will be addressed in WP1.2. These activities include:
- Development of integrated distributed video coding solutions: This activity consists of developing integrated solutions for distributed video coding (DVC) from the contributions made in WP1.1.
- Development of integrated video codecs: This activity consists of developing integrated video coding solutions including the non-normative tools developed in WP1.1.
- Common evaluation methodologies: This activity consists of getting the partners involved in WP1.1 to use common evaluation methodologies for their proposed video coding solutions. This will enable definition of application/user scenarios, common use of test material and testing conditions, definition of a set of criteria for the performance evaluation and comparison of results.
Processing Theme
The second theme considers audiovisual media processing. Integrated research activities on content adaptation are explored in WP2.1, with the aim to develop new ways of adapting the content so that the subjective quality of the user experience is improved. Audiovisual analysis is tackled in WP2.2. The goal of integrated research activities in this workpackage is to develop efficient methods for analysis and understanding of audiovisual material, for the detection and recognition of humans, and for automatic extraction of semantic and non-semantic metadata for content indexing and retrieval. Activities in WP2.1 and WP2.2 will focus on the integration aspects, e.g. by sharing common tools and defining joint methodologies and application scenarios. Finally, WP2.3 will deal with in-theme integration within the audiovisual processing theme.
WP2.1 on content adaptation will undertake more specifically the following activities:
- Context-based adaptation: This activity is aimed at the development of adaptation decision techniques based on context descriptors and their integration with content adaptation tools in order to support adaptive, context-aware, distributed applications that react to the characteristics and conditions of the usage environment.
- Coded video quality optimisation over networks: This activity addresses the problems related with the transfer of multimedia content across networks, and especially those arising from the use of wireless access links.
Audiovisual analysis activities, in WP2.2, include:
- Audio/Speaker Analysis: Work will focus on the development of tools for speaker segmentation and recognition, speech modelling, recognition and audio scene classification.
- 2D/3D Face detection, tracking, recognition and animation: This activity includes the detection and tracking of faces and facial features in 2D video scenes together with the implementation of efficient tools for the recognition of persons in non-controlled and non-cooperative conditions.
- Video segmentation and tracking: Robust segmentation and tracking will be considered under some serious circumstances, such as varying illumination conditions, complex and dynamic background, real-time applications, etc.
- Body detection, tracking, description and recognition: Methods for body detection, gait estimation, articulated object tracking, human motion pattern estimation, etc will be investigated and combined with the results obtained in the activity face detection and recognition.
- 3D reconstruction: The main aim of this activity is to investigate methods and algorithms for the reconstruction of 3D models from multiple images.
- Search and retrieval: This activity is related to the on-going standardization JPSearch initiative and to the MPEG-7 standard. The goal of this work is to define a complete end-to-end search and retrieval architecture which will allow efficient implementations in different environments.
Related to the audiovisual media processing theme, in-theme integration activities will be addressed in WP2.3. These activities include:
- Integration of technologies in the AV processing theme: Joint research work conducted in WP2.2 will be integrated for building a "cash machine simulator" with the following feature:
- User authorization by a fusion of audio/speaker recognition and 2D/3D
- Face detection, tracking and recognition
- Online 3D face reconstruction (model building)
- 3D face recognition by fast search in large databases
- Unusual human behaviour alarming by body localization, tracking and recognition
- Temporal segmentation of video recordings e.g. based on person identity.
- Common evaluation methodologies: This activity consists of getting the partners involved in the audiovisual processing activities of WP2.1 and WP2.2 to use common evaluation methodologies for their proposed media processing solutions and define application and user scenarios. This will enable evaluation of performance and comparison of results between different solutions proposed and techniques developed for the same topical area of media processing. Also, this includes the definition of shared databases and common performance evaluation criteria for search and retrieval.
Security Theme
The third theme addresses another very important topic in networked AV systems, namely security. Audiovisual media security and content protection integrated research activities will be carried out in WP3.1. Again, the emphasis will be on integration aspects, such as sharing common tools and software platforms, and defining joint methodologies. Similarly to the other themes, in-theme integration will be performed in a separate workpackage, WP3.2.
WP3.1 will address more specifically the following activities:
- Access security: Two topics will be investigated:
- Access control framework: Research and integration of an access control framework for multiparty handling of multimedia content.
- Security mechanisms associated with content adaptation: Research and integration of security mechanisms associated with content adaptation of multimedia flows during transmission over a network.
- DRM-based content protection: The following aspects will be looked into:
- Architecture to manage digital rights for audiovisual content: Create and deploy an architecture to manage digital rights for audiovisual content during its whole life cycle.
- Security in DRM and distribution: Diverse security problems related to the management and distribution of multimedia contents will be addressed, taking into account the corresponding digital rights.
- Licenses management support: Security technologies to support the management of licenses will be investigated.
- Keys management support: Security technologies oriented towards the management of coding keys in multimedia content will be designed and implemented for the protection of the associated rights.
Related to these activities, security theme integration will be addressed in WP3.2 with the following activities:
- Joint software development on DRM-based content protection: This activity aims at producing integrated research tools or solutions that could be used in both of the main WP3 activity areas and even in WP2. The intention is to provide a set of components that can be used in the construction of both a specific DRM architecture as well as a more general content protection (access control) architecture for use in collaborative working scenarios.
- Exchange of software tools: This activity aims at promoting and performing the exchange of research tools between partners active in the security thematic area of the JPA. Each partner has tools, and experience of their use, in complementary areas, including MPEG-21 Rights Expression Language (REL) and rules-engine based expert systems, and this activity will enable the sharing of this knowledge.
- Cross validation of architectures: This activity aims at providing independent analysis of the CW (Collaborative Working) access control architecture and the DRM-based content protection architecture which will be developed in the main activities of WP 3. This will be achieved by partners whose main expertise lies in one area, validating the work from the other area. By analysing the CW access control architecture from a DRM/MPEG-21 point of view and vice versa, not only will this improve the quality and performance of the architectures, but this will also enable synergies to be identified and built upon.
- Validation of developed solutions: This activity aims at joint validation of solutions and methods developed in the proposed security research activities. This will enable the NoE partners in this thematic area to benefit from a common approach to validation, as well as providing independent validation through the validation of each other's solutions.