Hierarchical segmentation of video sequences for content manipulation and adaptive coding

The problem of unsupervised hierarchical segmentation of video content is discussed. A segmentation approach combining local spatial homogeneity with motion homogeneity is proposed. Motion is the main characteristic which allows the decomposition of video sequences into a nested hierarchy of representation levels, each of them corresponding to a specified quality of motion compensation in terms of a proposed criterion. Two strategies of construction of such a hierarchy are studied. The first, a purely bottom-up approach, favours local motion homogeneity and constructs a hierarchical representation starting from the most detailed (gray-level based) segmentation then progressively merging the region primitives. The second, a cooperative strategy, uses a global motion analysis to guide the bottom-up approach. Results on test sequences MPEG ISO SC29/WG11 show interesting performances of such techniques for an automatic construction of hierarchical representations of time-varying images, allowing for flexible access and manipulation of video content. (C) 1998 Elsevier Science B.V. All rights reserved.