Combined video and audio watermarking: Embedding content information in multimedia data

Audio and video watermarking enable the copyright protection with owner or customer authentication and the detection of media manipulations. The available watermarking technology concentrates on single media like audio or video. But the typical multimedia stream consists of both video and audio data. Our goal is to provide a solution with robust and fragile aspects to guarantee authentication and integrity by using watermarks in combination with content information. We show two solutions for the protection of audio and video data with a combined robust and fragile watermarking approach. The first solution is to insert a time code into the data: We embed a signal as a watermark to detect gaps or changes in the flow of time. The basic idea uses numbers increasing by one. If in the verification process the next number is smaller than the last one or the step is greater than one, the time flow has been changed. This is realized without the combination of video and audio data. But we can synchronize the two data streams. A time signal is only valid if the combination of audio and video signals satisfy a certain attribute. To keep the basic example: If we embed an increasing number in the audio and a decreasing number in the video, we could test if the combination of both always equals zero. The second solution is more complex: We use watermarks to embed information in each media about the content of the other media. With the help of speech recognition technology it is possible to embed the spoken text, the content, of an audio file in the video. With an algorithm previously developed in [1] we extract video content representation which is embedded in the audio stream. In our paper we present the problem of copyright protection and integrity checks for combined video and audio data. We show our two solutions and discuss our results.