Document (#35449)

Author

Kim, H.H.
Kim, Y.H.

Title

Toward a conceptual framework of key-frame extraction and storyboard display for video summarization

Source

Journal of the American Society for Information Science and Technology. 61(2010) no.5, S.927-939

Year

2010

Abstract

Two key problems in developing a storyboard are (a) the extraction of video key frames and (b) the display of the storyboard. On the basis of our findings from a preliminary study as well as the results of previous studies on the computerized extraction of key frames and human recognition of images and videos, we propose an algorithm for the extraction of key frames and the structural display of a storyboard. In order to evaluate the proposed algorithm, we conducted an experiment, the results of which suggest that participants produce better summaries of the given videos when they view storyboards that are composed of key frames extracted using the proposed algorithmic method. This finding held, regardless of whether the display pattern used was sequential or structural. In contrast, the experimental results suggest that in the case of employing a mechanical method, the use of a structural display pattern yields greater performance in terms of participants' ability to summarize the given videos. Elaborating on our results, we discuss the practical implications of our findings for video summarization and retrieval.

Form

Videos

Similar documents (content)

1. Tse, T.; Vegh, S.; Marchionini, G.; Shneiderman, B.: ¬An exploratory study of video browsing user interface designs and research methodologies : effectiveness in information seeking tasks (1999) 0.24

   0.23638743 = sum of:
     0.23638743 = product of:
       0.9849477 = sum of:
         0.03203577 = weight(abstract_txt:participants in 6677) [ClassicSimilarity], result of:
           0.03203577 = score(doc=6677,freq=1.0), product of:
             0.09556486 = queryWeight, product of:
               1.5168757 = boost
               5.3636074 = idf(docFreq=562, maxDocs=44218)
               0.011746033 = queryNorm
             0.33522546 = fieldWeight in 6677, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3636074 = idf(docFreq=562, maxDocs=44218)
               0.0625 = fieldNorm(doc=6677)
         0.01753632 = weight(abstract_txt:results in 6677) [ClassicSimilarity], result of:
           0.01753632 = score(doc=6677,freq=1.0), product of:
             0.08057068 = queryWeight, product of:
               1.9697196 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.011746033 = queryNorm
             0.21765138 = fieldWeight in 6677, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=6677)
         0.14363816 = weight(abstract_txt:video in 6677) [ClassicSimilarity], result of:
           0.14363816 = score(doc=6677,freq=4.0), product of:
             0.18738367 = queryWeight, product of:
               2.6014316 = boost
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.011746033 = queryNorm
             0.76654583 = fieldWeight in 6677, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.0625 = fieldNorm(doc=6677)
         0.22308287 = weight(abstract_txt:display in 6677) [ClassicSimilarity], result of:
           0.22308287 = score(doc=6677,freq=4.0), product of:
             0.2979515 = queryWeight, product of:
               4.2349052 = boost
               5.989777 = idf(docFreq=300, maxDocs=44218)
               0.011746033 = queryNorm
             0.74872214 = fieldWeight in 6677, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.989777 = idf(docFreq=300, maxDocs=44218)
               0.0625 = fieldNorm(doc=6677)
         0.18367413 = weight(abstract_txt:frames in 6677) [ClassicSimilarity], result of:
           0.18367413 = score(doc=6677,freq=1.0), product of:
             0.38570037 = queryWeight, product of:
               4.3096395 = boost
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.011746033 = queryNorm
             0.47620937 = fieldWeight in 6677, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.0625 = fieldNorm(doc=6677)
         0.38498047 = weight(abstract_txt:storyboard in 6677) [ClassicSimilarity], result of:
           0.38498047 = score(doc=6677,freq=1.0), product of:
             0.6316995 = queryWeight, product of:
               5.5153255 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.011746033 = queryNorm
             0.6094361 = fieldWeight in 6677, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.0625 = fieldNorm(doc=6677)
       0.24 = coord(6/25)
   

2. Kim, H.H.: Toward video semantic search based on a structured folksonomy (2011) 0.19

   0.19116701 = sum of:
     0.19116701 = product of:
       0.6827393 = sum of:
         0.008285943 = weight(abstract_txt:that in 4350) [ClassicSimilarity], result of:
           0.008285943 = score(doc=4350,freq=4.0), product of:
             0.027975626 = queryWeight, product of:
               1.0051628 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.011746033 = queryNorm
             0.2961844 = fieldWeight in 4350, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=4350)
         0.028753318 = weight(abstract_txt:proposed in 4350) [ClassicSimilarity], result of:
           0.028753318 = score(doc=4350,freq=2.0), product of:
             0.070575945 = queryWeight, product of:
               1.3035547 = boost
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.011746033 = queryNorm
             0.4074096 = fieldWeight in 4350, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.0625 = fieldNorm(doc=4350)
         0.029322274 = weight(abstract_txt:suggest in 4350) [ClassicSimilarity], result of:
           0.029322274 = score(doc=4350,freq=1.0), product of:
             0.09008929 = queryWeight, product of:
               1.4727787 = boost
               5.207682 = idf(docFreq=657, maxDocs=44218)
               0.011746033 = queryNorm
             0.32548013 = fieldWeight in 4350, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.207682 = idf(docFreq=657, maxDocs=44218)
               0.0625 = fieldNorm(doc=4350)
         0.024800103 = weight(abstract_txt:results in 4350) [ClassicSimilarity], result of:
           0.024800103 = score(doc=4350,freq=2.0), product of:
             0.08057068 = queryWeight, product of:
               1.9697196 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.011746033 = queryNorm
             0.30780554 = fieldWeight in 4350, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=4350)
         0.10156752 = weight(abstract_txt:video in 4350) [ClassicSimilarity], result of:
           0.10156752 = score(doc=4350,freq=2.0), product of:
             0.18738367 = queryWeight, product of:
               2.6014316 = boost
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.011746033 = queryNorm
             0.54202974 = fieldWeight in 4350, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.0625 = fieldNorm(doc=4350)
         0.105029695 = weight(abstract_txt:videos in 4350) [ClassicSimilarity], result of:
           0.105029695 = score(doc=4350,freq=1.0), product of:
             0.2414237 = queryWeight, product of:
               2.9528165 = boost
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.011746033 = queryNorm
             0.43504304 = fieldWeight in 4350, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.0625 = fieldNorm(doc=4350)
         0.38498047 = weight(abstract_txt:storyboard in 4350) [ClassicSimilarity], result of:
           0.38498047 = score(doc=4350,freq=1.0), product of:
             0.6316995 = queryWeight, product of:
               5.5153255 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.011746033 = queryNorm
             0.6094361 = fieldWeight in 4350, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.0625 = fieldNorm(doc=4350)
       0.28 = coord(7/25)
   

3. Kim, H.H.; Kim, Y.H.: Generic speech summarization of transcribed lecture videos : using tags and their semantic relations (2016) 0.18

   0.17702435 = sum of:
     0.17702435 = product of:
       0.49173427 = sum of:
         0.009263967 = weight(abstract_txt:that in 2640) [ClassicSimilarity], result of:
           0.009263967 = score(doc=2640,freq=5.0), product of:
             0.027975626 = queryWeight, product of:
               1.0051628 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.011746033 = queryNorm
             0.3311442 = fieldWeight in 2640, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=2640)
         0.014838203 = weight(abstract_txt:findings in 2640) [ClassicSimilarity], result of:
           0.014838203 = score(doc=2640,freq=1.0), product of:
             0.057208724 = queryWeight, product of:
               1.1736323 = boost
               4.1499133 = idf(docFreq=1894, maxDocs=44218)
               0.011746033 = queryNorm
             0.25936958 = fieldWeight in 2640, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1499133 = idf(docFreq=1894, maxDocs=44218)
               0.0625 = fieldNorm(doc=2640)
         0.053545345 = weight(abstract_txt:method in 2640) [ClassicSimilarity], result of:
           0.053545345 = score(doc=2640,freq=8.0), product of:
             0.06729651 = queryWeight, product of:
               1.2729084 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.011746033 = queryNorm
             0.795663 = fieldWeight in 2640, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=2640)
         0.07714481 = weight(abstract_txt:elaborating in 2640) [ClassicSimilarity], result of:
           0.07714481 = score(doc=2640,freq=1.0), product of:
             0.13627072 = queryWeight, product of:
               1.2808173 = boost
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.011746033 = queryNorm
             0.56611437 = fieldWeight in 2640, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.0625 = fieldNorm(doc=2640)
         0.028753318 = weight(abstract_txt:proposed in 2640) [ClassicSimilarity], result of:
           0.028753318 = score(doc=2640,freq=2.0), product of:
             0.070575945 = queryWeight, product of:
               1.3035547 = boost
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.011746033 = queryNorm
             0.4074096 = fieldWeight in 2640, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.0625 = fieldNorm(doc=2640)
         0.024800103 = weight(abstract_txt:results in 2640) [ClassicSimilarity], result of:
           0.024800103 = score(doc=2640,freq=2.0), product of:
             0.08057068 = queryWeight, product of:
               1.9697196 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.011746033 = queryNorm
             0.30780554 = fieldWeight in 2640, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=2640)
         0.106539756 = weight(abstract_txt:summarization in 2640) [ClassicSimilarity], result of:
           0.106539756 = score(doc=2640,freq=2.0), product of:
             0.16899449 = queryWeight, product of:
               2.0171452 = boost
               7.132539 = idf(docFreq=95, maxDocs=44218)
               0.011746033 = queryNorm
             0.6304333 = fieldWeight in 2640, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.132539 = idf(docFreq=95, maxDocs=44218)
               0.0625 = fieldNorm(doc=2640)
         0.07181908 = weight(abstract_txt:video in 2640) [ClassicSimilarity], result of:
           0.07181908 = score(doc=2640,freq=1.0), product of:
             0.18738367 = queryWeight, product of:
               2.6014316 = boost
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.011746033 = queryNorm
             0.38327292 = fieldWeight in 2640, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.0625 = fieldNorm(doc=2640)
         0.105029695 = weight(abstract_txt:videos in 2640) [ClassicSimilarity], result of:
           0.105029695 = score(doc=2640,freq=1.0), product of:
             0.2414237 = queryWeight, product of:
               2.9528165 = boost
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.011746033 = queryNorm
             0.43504304 = fieldWeight in 2640, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.0625 = fieldNorm(doc=2640)
       0.36 = coord(9/25)
   

4. lyer, H.; Lewis, C.D.: Prioritization strategies for video storyboard keyframes (2007) 0.16

   0.16302542 = sum of:
     0.16302542 = product of:
       0.81512713 = sum of:
         0.05056709 = weight(abstract_txt:summarize in 206) [ClassicSimilarity], result of:
           0.05056709 = score(doc=206,freq=1.0), product of:
             0.10282768 = queryWeight, product of:
               1.1126047 = boost
               7.8682456 = idf(docFreq=45, maxDocs=44218)
               0.011746033 = queryNorm
             0.49176535 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.8682456 = idf(docFreq=45, maxDocs=44218)
               0.0625 = fieldNorm(doc=206)
         0.020331666 = weight(abstract_txt:proposed in 206) [ClassicSimilarity], result of:
           0.020331666 = score(doc=206,freq=1.0), product of:
             0.070575945 = queryWeight, product of:
               1.3035547 = boost
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.011746033 = queryNorm
             0.2880821 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.0625 = fieldNorm(doc=206)
         0.124394305 = weight(abstract_txt:video in 206) [ClassicSimilarity], result of:
           0.124394305 = score(doc=206,freq=3.0), product of:
             0.18738367 = queryWeight, product of:
               2.6014316 = boost
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.011746033 = queryNorm
             0.66384816 = fieldWeight in 206, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.0625 = fieldNorm(doc=206)
         0.23485355 = weight(abstract_txt:videos in 206) [ClassicSimilarity], result of:
           0.23485355 = score(doc=206,freq=5.0), product of:
             0.2414237 = queryWeight, product of:
               2.9528165 = boost
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.011746033 = queryNorm
             0.97278583 = fieldWeight in 206, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.0625 = fieldNorm(doc=206)
         0.38498047 = weight(abstract_txt:storyboard in 206) [ClassicSimilarity], result of:
           0.38498047 = score(doc=206,freq=1.0), product of:
             0.6316995 = queryWeight, product of:
               5.5153255 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.011746033 = queryNorm
             0.6094361 = fieldWeight in 206, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.0625 = fieldNorm(doc=206)
       0.2 = coord(5/25)
   

5. Lee, Y.-S.; Wu, Y.-C.; Yang, J.-C.: BVideoQA : Online English/Chinese bilingual video question answering (2009) 0.13

   0.12587866 = sum of:
     0.12587866 = product of:
       0.3933708 = sum of:
         0.0041429717 = weight(abstract_txt:that in 2739) [ClassicSimilarity], result of:
           0.0041429717 = score(doc=2739,freq=1.0), product of:
             0.027975626 = queryWeight, product of:
               1.0051628 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.011746033 = queryNorm
             0.1480922 = fieldWeight in 2739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=2739)
         0.032789696 = weight(abstract_txt:method in 2739) [ClassicSimilarity], result of:
           0.032789696 = score(doc=2739,freq=3.0), product of:
             0.06729651 = queryWeight, product of:
               1.2729084 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.011746033 = queryNorm
             0.4872421 = fieldWeight in 2739, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=2739)
         0.020331666 = weight(abstract_txt:proposed in 2739) [ClassicSimilarity], result of:
           0.020331666 = score(doc=2739,freq=1.0), product of:
             0.070575945 = queryWeight, product of:
               1.3035547 = boost
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.011746033 = queryNorm
             0.2880821 = fieldWeight in 2739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.0625 = fieldNorm(doc=2739)
         0.03855918 = weight(abstract_txt:algorithm in 2739) [ClassicSimilarity], result of:
           0.03855918 = score(doc=2739,freq=1.0), product of:
             0.10813341 = queryWeight, product of:
               1.6135441 = boost
               5.705423 = idf(docFreq=399, maxDocs=44218)
               0.011746033 = queryNorm
             0.35658893 = fieldWeight in 2739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.705423 = idf(docFreq=399, maxDocs=44218)
               0.0625 = fieldNorm(doc=2739)
         0.050586965 = weight(abstract_txt:pattern in 2739) [ClassicSimilarity], result of:
           0.050586965 = score(doc=2739,freq=1.0), product of:
             0.12958871 = queryWeight, product of:
               1.7663814 = boost
               6.2458487 = idf(docFreq=232, maxDocs=44218)
               0.011746033 = queryNorm
             0.39036554 = fieldWeight in 2739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2458487 = idf(docFreq=232, maxDocs=44218)
               0.0625 = fieldNorm(doc=2739)
         0.01753632 = weight(abstract_txt:results in 2739) [ClassicSimilarity], result of:
           0.01753632 = score(doc=2739,freq=1.0), product of:
             0.08057068 = queryWeight, product of:
               1.9697196 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.011746033 = queryNorm
             0.21765138 = fieldWeight in 2739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=2739)
         0.124394305 = weight(abstract_txt:video in 2739) [ClassicSimilarity], result of:
           0.124394305 = score(doc=2739,freq=3.0), product of:
             0.18738367 = queryWeight, product of:
               2.6014316 = boost
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.011746033 = queryNorm
             0.66384816 = fieldWeight in 2739, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.1323667 = idf(docFreq=260, maxDocs=44218)
               0.0625 = fieldNorm(doc=2739)
         0.105029695 = weight(abstract_txt:videos in 2739) [ClassicSimilarity], result of:
           0.105029695 = score(doc=2739,freq=1.0), product of:
             0.2414237 = queryWeight, product of:
               2.9528165 = boost
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.011746033 = queryNorm
             0.43504304 = fieldWeight in 2739, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.0625 = fieldNorm(doc=2739)
       0.32 = coord(8/25)