This paper presents a fast semi-automatic semantic object-tracking algorithm for digital video. The proposed method consists of two steps, namely, intra-frame object extraction and the inter-frame object tracking. For the intra-frame segmentation, we try to minimize the human intervention by exploiting the automatic homogenous region segmentation. Then, to achieve a fast inter-frame object tracking, we propose to use I-D projected motion estimation and an improved flooding method for the watershed algorithm.