Document (#35450)

Author
Kim, H.H.
Kim, Y.H.
Title
Toward a conceptual framework of key-frame extraction and storyboard display for video summarization
Source
Journal of the American Society for Information Science and Technology. 61(2010) no.5, S.927-939
Year
2010
Abstract
Two key problems in developing a storyboard are (a) the extraction of video key frames and (b) the display of the storyboard. On the basis of our findings from a preliminary study as well as the results of previous studies on the computerized extraction of key frames and human recognition of images and videos, we propose an algorithm for the extraction of key frames and the structural display of a storyboard. In order to evaluate the proposed algorithm, we conducted an experiment, the results of which suggest that participants produce better summaries of the given videos when they view storyboards that are composed of key frames extracted using the proposed algorithmic method. This finding held, regardless of whether the display pattern used was sequential or structural. In contrast, the experimental results suggest that in the case of employing a mechanical method, the use of a structural display pattern yields greater performance in terms of participants' ability to summarize the given videos. Elaborating on our results, we discuss the practical implications of our findings for video summarization and retrieval.
Form
Videos

Similar documents (content)

  1. Tse, T.; Vegh, S.; Marchionini, G.; Shneiderman, B.: ¬An exploratory study of video browsing user interface designs and research methodologies : effectiveness in information seeking tasks (1999) 0.24
    0.23606047 = sum of:
      0.23606047 = product of:
        0.9835853 = sum of:
          0.03364776 = weight(abstract_txt:participants in 678) [ClassicSimilarity], result of:
            0.03364776 = score(doc=678,freq=1.0), product of:
              0.09866944 = queryWeight, product of:
                1.527968 = boost
                5.45624 = idf(docFreq=490, maxDocs=42306)
                0.011835181 = queryNorm
              0.341015 = fieldWeight in 678, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.45624 = idf(docFreq=490, maxDocs=42306)
                0.0625 = fieldNorm(doc=678)
          0.017969616 = weight(abstract_txt:results in 678) [ClassicSimilarity], result of:
            0.017969616 = score(doc=678,freq=1.0), product of:
              0.08183035 = queryWeight, product of:
                1.9678643 = boost
                3.5135355 = idf(docFreq=3425, maxDocs=42306)
                0.011835181 = queryNorm
              0.21959597 = fieldWeight in 678, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5135355 = idf(docFreq=3425, maxDocs=42306)
                0.0625 = fieldNorm(doc=678)
          0.14494072 = weight(abstract_txt:video in 678) [ClassicSimilarity], result of:
            0.14494072 = score(doc=678,freq=4.0), product of:
              0.18837182 = queryWeight, product of:
                2.5856905 = boost
                6.155516 = idf(docFreq=243, maxDocs=42306)
                0.011835181 = queryNorm
              0.7694395 = fieldWeight in 678, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.155516 = idf(docFreq=243, maxDocs=42306)
                0.0625 = fieldNorm(doc=678)
          0.22027907 = weight(abstract_txt:display in 678) [ClassicSimilarity], result of:
            0.22027907 = score(doc=678,freq=4.0), product of:
              0.29522562 = queryWeight, product of:
                4.178977 = boost
                5.9691043 = idf(docFreq=293, maxDocs=42306)
                0.011835181 = queryNorm
              0.74613804 = fieldWeight in 678, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.9691043 = idf(docFreq=293, maxDocs=42306)
                0.0625 = fieldNorm(doc=678)
          0.18784305 = weight(abstract_txt:frames in 678) [ClassicSimilarity], result of:
            0.18784305 = score(doc=678,freq=1.0), product of:
              0.39121783 = queryWeight, product of:
                4.3027616 = boost
                7.682392 = idf(docFreq=52, maxDocs=42306)
                0.011835181 = queryNorm
              0.4801495 = fieldWeight in 678, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.682392 = idf(docFreq=52, maxDocs=42306)
                0.0625 = fieldNorm(doc=678)
          0.37890512 = weight(abstract_txt:storyboard in 678) [ClassicSimilarity], result of:
            0.37890512 = score(doc=678,freq=1.0), product of:
              0.62456197 = queryWeight, product of:
                5.436579 = boost
                9.706774 = idf(docFreq=6, maxDocs=42306)
                0.011835181 = queryNorm
              0.60667336 = fieldWeight in 678, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.706774 = idf(docFreq=6, maxDocs=42306)
                0.0625 = fieldNorm(doc=678)
        0.24 = coord(6/25)
    
  2. Kim, H.H.: Toward video semantic search based on a structured folksonomy (2011) 0.19
    0.19069926 = sum of:
      0.19069926 = product of:
        0.6810688 = sum of:
          0.008642987 = weight(abstract_txt:that in 1351) [ClassicSimilarity], result of:
            0.008642987 = score(doc=1351,freq=4.0), product of:
              0.02875179 = queryWeight, product of:
                1.0101852 = boost
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.011835181 = queryNorm
              0.30060694 = fieldWeight in 1351, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.0625 = fieldNorm(doc=1351)
          0.029656015 = weight(abstract_txt:proposed in 1351) [ClassicSimilarity], result of:
            0.029656015 = score(doc=1351,freq=2.0), product of:
              0.07199082 = queryWeight, product of:
                1.3051533 = boost
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.011835181 = queryNorm
              0.41194165 = fieldWeight in 1351, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.0625 = fieldNorm(doc=1351)
          0.030304298 = weight(abstract_txt:suggest in 1351) [ClassicSimilarity], result of:
            0.030304298 = score(doc=1351,freq=1.0), product of:
              0.09201982 = queryWeight, product of:
                1.475583 = boost
                5.2691774 = idf(docFreq=591, maxDocs=42306)
                0.011835181 = queryNorm
              0.3293236 = fieldWeight in 1351, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.2691774 = idf(docFreq=591, maxDocs=42306)
                0.0625 = fieldNorm(doc=1351)
          0.025412872 = weight(abstract_txt:results in 1351) [ClassicSimilarity], result of:
            0.025412872 = score(doc=1351,freq=2.0), product of:
              0.08183035 = queryWeight, product of:
                1.9678643 = boost
                3.5135355 = idf(docFreq=3425, maxDocs=42306)
                0.011835181 = queryNorm
              0.31055558 = fieldWeight in 1351, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.5135355 = idf(docFreq=3425, maxDocs=42306)
                0.0625 = fieldNorm(doc=1351)
          0.10248857 = weight(abstract_txt:video in 1351) [ClassicSimilarity], result of:
            0.10248857 = score(doc=1351,freq=2.0), product of:
              0.18837182 = queryWeight, product of:
                2.5856905 = boost
                6.155516 = idf(docFreq=243, maxDocs=42306)
                0.011835181 = queryNorm
              0.5440759 = fieldWeight in 1351, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.155516 = idf(docFreq=243, maxDocs=42306)
                0.0625 = fieldNorm(doc=1351)
          0.105658956 = weight(abstract_txt:videos in 1351) [ClassicSimilarity], result of:
            0.105658956 = score(doc=1351,freq=1.0), product of:
              0.24220319 = queryWeight, product of:
                2.9319632 = boost
                6.9798555 = idf(docFreq=106, maxDocs=42306)
                0.011835181 = queryNorm
              0.43624097 = fieldWeight in 1351, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.9798555 = idf(docFreq=106, maxDocs=42306)
                0.0625 = fieldNorm(doc=1351)
          0.37890512 = weight(abstract_txt:storyboard in 1351) [ClassicSimilarity], result of:
            0.37890512 = score(doc=1351,freq=1.0), product of:
              0.62456197 = queryWeight, product of:
                5.436579 = boost
                9.706774 = idf(docFreq=6, maxDocs=42306)
                0.011835181 = queryNorm
              0.60667336 = fieldWeight in 1351, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.706774 = idf(docFreq=6, maxDocs=42306)
                0.0625 = fieldNorm(doc=1351)
        0.28 = coord(7/25)
    
  3. Kim, H.H.; Kim, Y.H.: Generic speech summarization of transcribed lecture videos : using tags and their semantic relations (2016) 0.18
    0.17859039 = sum of:
      0.17859039 = product of:
        0.4960844 = sum of:
          0.009663154 = weight(abstract_txt:that in 4641) [ClassicSimilarity], result of:
            0.009663154 = score(doc=4641,freq=5.0), product of:
              0.02875179 = queryWeight, product of:
                1.0101852 = boost
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.011835181 = queryNorm
              0.33608878 = fieldWeight in 4641, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.0625 = fieldNorm(doc=4641)
          0.01602849 = weight(abstract_txt:findings in 4641) [ClassicSimilarity], result of:
            0.01602849 = score(doc=4641,freq=1.0), product of:
              0.06018297 = queryWeight, product of:
                1.1933278 = boost
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.011835181 = queryNorm
              0.26632932 = fieldWeight in 4641, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.0625 = fieldNorm(doc=4641)
          0.075848065 = weight(abstract_txt:elaborating in 4641) [ClassicSimilarity], result of:
            0.075848065 = score(doc=4641,freq=1.0), product of:
              0.13463715 = queryWeight, product of:
                1.2620902 = boost
                9.013627 = idf(docFreq=13, maxDocs=42306)
                0.011835181 = queryNorm
              0.5633517 = fieldWeight in 4641, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.013627 = idf(docFreq=13, maxDocs=42306)
                0.0625 = fieldNorm(doc=4641)
          0.054633267 = weight(abstract_txt:method in 4641) [ClassicSimilarity], result of:
            0.054633267 = score(doc=4641,freq=8.0), product of:
              0.06815327 = queryWeight, product of:
                1.2698905 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.011835181 = queryNorm
              0.8016236 = fieldWeight in 4641, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.0625 = fieldNorm(doc=4641)
          0.029656015 = weight(abstract_txt:proposed in 4641) [ClassicSimilarity], result of:
            0.029656015 = score(doc=4641,freq=2.0), product of:
              0.07199082 = queryWeight, product of:
                1.3051533 = boost
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.011835181 = queryNorm
              0.41194165 = fieldWeight in 4641, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.0625 = fieldNorm(doc=4641)
          0.025412872 = weight(abstract_txt:results in 4641) [ClassicSimilarity], result of:
            0.025412872 = score(doc=4641,freq=2.0), product of:
              0.08183035 = queryWeight, product of:
                1.9678643 = boost
                3.5135355 = idf(docFreq=3425, maxDocs=42306)
                0.011835181 = queryNorm
              0.31055558 = fieldWeight in 4641, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.5135355 = idf(docFreq=3425, maxDocs=42306)
                0.0625 = fieldNorm(doc=4641)
          0.106713235 = weight(abstract_txt:summarization in 4641) [ClassicSimilarity], result of:
            0.106713235 = score(doc=4641,freq=2.0), product of:
              0.16904958 = queryWeight, product of:
                2.0 = boost
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.011835181 = queryNorm
              0.6312541 = fieldWeight in 4641, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.0625 = fieldNorm(doc=4641)
          0.07247036 = weight(abstract_txt:video in 4641) [ClassicSimilarity], result of:
            0.07247036 = score(doc=4641,freq=1.0), product of:
              0.18837182 = queryWeight, product of:
                2.5856905 = boost
                6.155516 = idf(docFreq=243, maxDocs=42306)
                0.011835181 = queryNorm
              0.38471976 = fieldWeight in 4641, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.155516 = idf(docFreq=243, maxDocs=42306)
                0.0625 = fieldNorm(doc=4641)
          0.105658956 = weight(abstract_txt:videos in 4641) [ClassicSimilarity], result of:
            0.105658956 = score(doc=4641,freq=1.0), product of:
              0.24220319 = queryWeight, product of:
                2.9319632 = boost
                6.9798555 = idf(docFreq=106, maxDocs=42306)
                0.011835181 = queryNorm
              0.43624097 = fieldWeight in 4641, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.9798555 = idf(docFreq=106, maxDocs=42306)
                0.0625 = fieldNorm(doc=4641)
        0.36 = coord(9/25)
    
  4. lyer, H.; Lewis, C.D.: Prioritization strategies for video storyboard keyframes (2007) 0.16
    0.16269714 = sum of:
      0.16269714 = product of:
        0.8134857 = sum of:
          0.051827654 = weight(abstract_txt:summarize in 2207) [ClassicSimilarity], result of:
            0.051827654 = score(doc=2207,freq=1.0), product of:
              0.10445028 = queryWeight, product of:
                1.1116364 = boost
                7.939112 = idf(docFreq=40, maxDocs=42306)
                0.011835181 = queryNorm
              0.4961945 = fieldWeight in 2207, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.939112 = idf(docFreq=40, maxDocs=42306)
                0.0625 = fieldNorm(doc=2207)
          0.02096997 = weight(abstract_txt:proposed in 2207) [ClassicSimilarity], result of:
            0.02096997 = score(doc=2207,freq=1.0), product of:
              0.07199082 = queryWeight, product of:
                1.3051533 = boost
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.011835181 = queryNorm
              0.29128674 = fieldWeight in 2207, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.0625 = fieldNorm(doc=2207)
          0.12552235 = weight(abstract_txt:video in 2207) [ClassicSimilarity], result of:
            0.12552235 = score(doc=2207,freq=3.0), product of:
              0.18837182 = queryWeight, product of:
                2.5856905 = boost
                6.155516 = idf(docFreq=243, maxDocs=42306)
                0.011835181 = queryNorm
              0.6663542 = fieldWeight in 2207, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.155516 = idf(docFreq=243, maxDocs=42306)
                0.0625 = fieldNorm(doc=2207)
          0.23626061 = weight(abstract_txt:videos in 2207) [ClassicSimilarity], result of:
            0.23626061 = score(doc=2207,freq=5.0), product of:
              0.24220319 = queryWeight, product of:
                2.9319632 = boost
                6.9798555 = idf(docFreq=106, maxDocs=42306)
                0.011835181 = queryNorm
              0.97546446 = fieldWeight in 2207, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                6.9798555 = idf(docFreq=106, maxDocs=42306)
                0.0625 = fieldNorm(doc=2207)
          0.37890512 = weight(abstract_txt:storyboard in 2207) [ClassicSimilarity], result of:
            0.37890512 = score(doc=2207,freq=1.0), product of:
              0.62456197 = queryWeight, product of:
                5.436579 = boost
                9.706774 = idf(docFreq=6, maxDocs=42306)
                0.011835181 = queryNorm
              0.60667336 = fieldWeight in 2207, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.706774 = idf(docFreq=6, maxDocs=42306)
                0.0625 = fieldNorm(doc=2207)
        0.2 = coord(5/25)
    
  5. Lee, Y.-S.; Wu, Y.-C.; Yang, J.-C.: BVideoQA : Online English/Chinese bilingual video question answering (2009) 0.13
    0.12720625 = sum of:
      0.12720625 = product of:
        0.39751956 = sum of:
          0.0043214937 = weight(abstract_txt:that in 559) [ClassicSimilarity], result of:
            0.0043214937 = score(doc=559,freq=1.0), product of:
              0.02875179 = queryWeight, product of:
                1.0101852 = boost
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.011835181 = queryNorm
              0.15030347 = fieldWeight in 559, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.0625 = fieldNorm(doc=559)
          0.03345591 = weight(abstract_txt:method in 559) [ClassicSimilarity], result of:
            0.03345591 = score(doc=559,freq=3.0), product of:
              0.06815327 = queryWeight, product of:
                1.2698905 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.011835181 = queryNorm
              0.4908922 = fieldWeight in 559, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.0625 = fieldNorm(doc=559)
          0.02096997 = weight(abstract_txt:proposed in 559) [ClassicSimilarity], result of:
            0.02096997 = score(doc=559,freq=1.0), product of:
              0.07199082 = queryWeight, product of:
                1.3051533 = boost
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.011835181 = queryNorm
              0.29128674 = fieldWeight in 559, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.0625 = fieldNorm(doc=559)
          0.03893877 = weight(abstract_txt:algorithm in 559) [ClassicSimilarity], result of:
            0.03893877 = score(doc=559,freq=1.0), product of:
              0.10875937 = queryWeight, product of:
                1.6041918 = boost
                5.7284284 = idf(docFreq=373, maxDocs=42306)
                0.011835181 = queryNorm
              0.35802677 = fieldWeight in 559, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.7284284 = idf(docFreq=373, maxDocs=42306)
                0.0625 = fieldNorm(doc=559)
          0.050682496 = weight(abstract_txt:pattern in 559) [ClassicSimilarity], result of:
            0.050682496 = score(doc=559,freq=1.0), product of:
              0.1296534 = queryWeight, product of:
                1.7515192 = boost
                6.2545214 = idf(docFreq=220, maxDocs=42306)
                0.011835181 = queryNorm
              0.3909076 = fieldWeight in 559, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.2545214 = idf(docFreq=220, maxDocs=42306)
                0.0625 = fieldNorm(doc=559)
          0.017969616 = weight(abstract_txt:results in 559) [ClassicSimilarity], result of:
            0.017969616 = score(doc=559,freq=1.0), product of:
              0.08183035 = queryWeight, product of:
                1.9678643 = boost
                3.5135355 = idf(docFreq=3425, maxDocs=42306)
                0.011835181 = queryNorm
              0.21959597 = fieldWeight in 559, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5135355 = idf(docFreq=3425, maxDocs=42306)
                0.0625 = fieldNorm(doc=559)
          0.12552235 = weight(abstract_txt:video in 559) [ClassicSimilarity], result of:
            0.12552235 = score(doc=559,freq=3.0), product of:
              0.18837182 = queryWeight, product of:
                2.5856905 = boost
                6.155516 = idf(docFreq=243, maxDocs=42306)
                0.011835181 = queryNorm
              0.6663542 = fieldWeight in 559, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.155516 = idf(docFreq=243, maxDocs=42306)
                0.0625 = fieldNorm(doc=559)
          0.105658956 = weight(abstract_txt:videos in 559) [ClassicSimilarity], result of:
            0.105658956 = score(doc=559,freq=1.0), product of:
              0.24220319 = queryWeight, product of:
                2.9319632 = boost
                6.9798555 = idf(docFreq=106, maxDocs=42306)
                0.011835181 = queryNorm
              0.43624097 = fieldWeight in 559, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.9798555 = idf(docFreq=106, maxDocs=42306)
                0.0625 = fieldNorm(doc=559)
        0.32 = coord(8/25)