Document (#37664)

Author

Ostermann, F.O.
Tomko, M.
Purves, R.

Title

User evaluation of automatically generated keywords and toponyms for geo-referenced images

Source

Journal of the American Society for Information Science and Technology. 64(2013) no.3, S.480-499

Year

2013

Abstract

This article presents the results of a user evaluation of automatically generated concept keywords and place names (toponyms) for geo-referenced images. Automatically annotating images is becoming indispensable for effective information retrieval, since the number of geo-referenced images available online is growing, yet many images are insufficiently tagged or captioned to be efficiently searchable by standard information retrieval procedures. The Tripod project developed original methods for automatically annotating geo-referenced images by generating representations of the likely visible footprint of a geo-referenced image, and using this footprint to query spatial databases and web resources. These queries return raw lists of potential keywords and toponyms, which are subsequently filtered and ranked. This article reports on user experiments designed to evaluate the quality of the generated annotations. The experiments combined quantitative and qualitative approaches: To retrieve a large number of responses, participants rated the annotations in standardized online questionnaires that showed an image and its corresponding keywords. In addition, several focus groups provided rich qualitative information in open discussions. The results of the evaluation show that currently the annotation method performs better on rural images than on urban ones. Further, for each image at least one suitable keyword could be generated. The integration of heterogeneous data sources resulted in some images having a high level of noise in the form of obviously wrong or spurious keywords. The article discusses the evaluation itself and methods to improve the automatic generation of annotations.

Field

Geowissenschaften

Form

Bilder

Similar documents (author)

1. Purves, R.S.; Sanderson, M.: ¬A methodology to allow avalanche forecasting on an information retrieval system (1998) 4.95

   4.952564 = sum of:
     4.952564 = weight(author_txt:purves in 1073) [ClassicSimilarity], result of:
       4.952564 = fieldWeight in 1073, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.905128 = idf(docFreq=5, maxDocs=44218)
         0.5 = fieldNorm(doc=1073)
   

2. Purves, D.; Lotto, R.B.; Nundy, S.: Richtiges Sehen - eine optische Täuschung? (2003) 3.71

   3.7144227 = sum of:
     3.7144227 = weight(author_txt:purves in 1891) [ClassicSimilarity], result of:
       3.7144227 = fieldWeight in 1891, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.905128 = idf(docFreq=5, maxDocs=44218)
         0.375 = fieldNorm(doc=1891)
   

3. Purves, R.S.; Winter, S.; Kuhn, W.: Places in information science (2019) 3.71

   3.7144227 = sum of:
     3.7144227 = weight(author_txt:purves in 5433) [ClassicSimilarity], result of:
       3.7144227 = fieldWeight in 5433, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.905128 = idf(docFreq=5, maxDocs=44218)
         0.375 = fieldNorm(doc=5433)
   

4. Reichenbacher, T.; Sabbata, S. de; Purves, R.S.; Fabrikant, S.I.: Assessing geographic relevance for mobile search : a computational model and its validation via crowdsourcing (2016) 3.10

   3.0953524 = sum of:
     3.0953524 = weight(author_txt:purves in 3145) [ClassicSimilarity], result of:
       3.0953524 = fieldWeight in 3145, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.905128 = idf(docFreq=5, maxDocs=44218)
         0.3125 = fieldNorm(doc=3145)
   

Similar documents (content)

1. Tsai, C.-F.; McGarry, K.; Tait, J.: Qualitative evaluation of automatic assignment of keywords to images (2006) 0.28

   0.2821424 = sum of:
     0.2821424 = product of:
       0.78372884 = sum of:
         0.028525952 = weight(abstract_txt:methods in 963) [ClassicSimilarity], result of:
           0.028525952 = score(doc=963,freq=4.0), product of:
             0.05503286 = queryWeight, product of:
               1.048126 = boost
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.012661949 = queryNorm
             0.518344 = fieldWeight in 963, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.0625 = fieldNorm(doc=963)
         0.033738583 = weight(abstract_txt:qualitative in 963) [ClassicSimilarity], result of:
           0.033738583 = score(doc=963,freq=1.0), product of:
             0.09770119 = queryWeight, product of:
               1.3965367 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.012661949 = queryNorm
             0.34532416 = fieldWeight in 963, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0625 = fieldNorm(doc=963)
         0.02120752 = weight(abstract_txt:user in 963) [ClassicSimilarity], result of:
           0.02120752 = score(doc=963,freq=2.0), product of:
             0.065137185 = queryWeight, product of:
               1.3965695 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.012661949 = queryNorm
             0.32558239 = fieldWeight in 963, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.0625 = fieldNorm(doc=963)
         0.08066941 = weight(abstract_txt:image in 963) [ClassicSimilarity], result of:
           0.08066941 = score(doc=963,freq=3.0), product of:
             0.13865788 = queryWeight, product of:
               2.0376067 = boost
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.012661949 = queryNorm
             0.5817874 = fieldWeight in 963, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.0625 = fieldNorm(doc=963)
         0.1021548 = weight(abstract_txt:evaluation in 963) [ClassicSimilarity], result of:
           0.1021548 = score(doc=963,freq=8.0), product of:
             0.12881541 = queryWeight, product of:
               2.2677822 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.012661949 = queryNorm
             0.7930324 = fieldWeight in 963, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=963)
         0.06717294 = weight(abstract_txt:automatically in 963) [ClassicSimilarity], result of:
           0.06717294 = score(doc=963,freq=1.0), product of:
             0.19481462 = queryWeight, product of:
               2.7888696 = boost
               5.5168705 = idf(docFreq=482, maxDocs=44218)
               0.012661949 = queryNorm
             0.3448044 = fieldWeight in 963, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5168705 = idf(docFreq=482, maxDocs=44218)
               0.0625 = fieldNorm(doc=963)
         0.1199222 = weight(abstract_txt:annotations in 963) [ClassicSimilarity], result of:
           0.1199222 = score(doc=963,freq=1.0), product of:
             0.26048264 = queryWeight, product of:
               2.7927847 = boost
               7.3661537 = idf(docFreq=75, maxDocs=44218)
               0.012661949 = queryNorm
             0.4603846 = fieldWeight in 963, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3661537 = idf(docFreq=75, maxDocs=44218)
               0.0625 = fieldNorm(doc=963)
         0.10873422 = weight(abstract_txt:keywords in 963) [ClassicSimilarity], result of:
           0.10873422 = score(doc=963,freq=1.0), product of:
             0.28931624 = queryWeight, product of:
               3.7997835 = boost
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.012661949 = queryNorm
             0.37583172 = fieldWeight in 963, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.0625 = fieldNorm(doc=963)
         0.22160318 = weight(abstract_txt:images in 963) [ClassicSimilarity], result of:
           0.22160318 = score(doc=963,freq=3.0), product of:
             0.37714836 = queryWeight, product of:
               5.487677 = boost
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.012661949 = queryNorm
             0.5875756 = fieldWeight in 963, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.0625 = fieldNorm(doc=963)
       0.36 = coord(9/25)
   

2. Jörgensen, C.; Jaimes, A.; Benitez, A.B.; Chang, S.-F.: ¬A conceptual framework and empirical research for classifying visual descriptors (2001) 0.12

   0.123450994 = sum of:
     0.123450994 = product of:
       0.51437914 = sum of:
         0.052472305 = weight(abstract_txt:experiments in 6532) [ClassicSimilarity], result of:
           0.052472305 = score(doc=6532,freq=3.0), product of:
             0.090934426 = queryWeight, product of:
               1.3473071 = boost
               5.3304167 = idf(docFreq=581, maxDocs=44218)
               0.012661949 = queryNorm
             0.57703453 = fieldWeight in 6532, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.3304167 = idf(docFreq=581, maxDocs=44218)
               0.0625 = fieldNorm(doc=6532)
         0.01651657 = weight(abstract_txt:article in 6532) [ClassicSimilarity], result of:
           0.01651657 = score(doc=6532,freq=1.0), product of:
             0.0694692 = queryWeight, product of:
               1.4422622 = boost
               3.8040617 = idf(docFreq=2677, maxDocs=44218)
               0.012661949 = queryNorm
             0.23775385 = fieldWeight in 6532, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.8040617 = idf(docFreq=2677, maxDocs=44218)
               0.0625 = fieldNorm(doc=6532)
         0.08066941 = weight(abstract_txt:image in 6532) [ClassicSimilarity], result of:
           0.08066941 = score(doc=6532,freq=3.0), product of:
             0.13865788 = queryWeight, product of:
               2.0376067 = boost
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.012661949 = queryNorm
             0.5817874 = fieldWeight in 6532, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.0625 = fieldNorm(doc=6532)
         0.06717294 = weight(abstract_txt:automatically in 6532) [ClassicSimilarity], result of:
           0.06717294 = score(doc=6532,freq=1.0), product of:
             0.19481462 = queryWeight, product of:
               2.7888696 = boost
               5.5168705 = idf(docFreq=482, maxDocs=44218)
               0.012661949 = queryNorm
             0.3448044 = fieldWeight in 6532, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5168705 = idf(docFreq=482, maxDocs=44218)
               0.0625 = fieldNorm(doc=6532)
         0.11660967 = weight(abstract_txt:generated in 6532) [ClassicSimilarity], result of:
           0.11660967 = score(doc=6532,freq=3.0), product of:
             0.19510779 = queryWeight, product of:
               2.7909672 = boost
               5.52102 = idf(docFreq=480, maxDocs=44218)
               0.012661949 = queryNorm
             0.59766793 = fieldWeight in 6532, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.52102 = idf(docFreq=480, maxDocs=44218)
               0.0625 = fieldNorm(doc=6532)
         0.18093824 = weight(abstract_txt:images in 6532) [ClassicSimilarity], result of:
           0.18093824 = score(doc=6532,freq=2.0), product of:
             0.37714836 = queryWeight, product of:
               5.487677 = boost
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.012661949 = queryNorm
             0.4797535 = fieldWeight in 6532, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.0625 = fieldNorm(doc=6532)
       0.24 = coord(6/25)
   

3. Rasmussen, E.M.: Indexing images (1997) 0.12

   0.11939555 = sum of:
     0.11939555 = product of:
       0.5969777 = sum of:
         0.01782872 = weight(abstract_txt:methods in 2215) [ClassicSimilarity], result of:
           0.01782872 = score(doc=2215,freq=1.0), product of:
             0.05503286 = queryWeight, product of:
               1.048126 = boost
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.012661949 = queryNorm
             0.32396498 = fieldWeight in 2215, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
         0.1301797 = weight(abstract_txt:image in 2215) [ClassicSimilarity], result of:
           0.1301797 = score(doc=2215,freq=5.0), product of:
             0.13865788 = queryWeight, product of:
               2.0376067 = boost
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.012661949 = queryNorm
             0.9388554 = fieldWeight in 2215, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
         0.04514647 = weight(abstract_txt:evaluation in 2215) [ClassicSimilarity], result of:
           0.04514647 = score(doc=2215,freq=1.0), product of:
             0.12881541 = queryWeight, product of:
               2.2677822 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.012661949 = queryNorm
             0.35047412 = fieldWeight in 2215, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
         0.08396617 = weight(abstract_txt:automatically in 2215) [ClassicSimilarity], result of:
           0.08396617 = score(doc=2215,freq=1.0), product of:
             0.19481462 = queryWeight, product of:
               2.7888696 = boost
               5.5168705 = idf(docFreq=482, maxDocs=44218)
               0.012661949 = queryNorm
             0.4310055 = fieldWeight in 2215, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5168705 = idf(docFreq=482, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
         0.31985664 = weight(abstract_txt:images in 2215) [ClassicSimilarity], result of:
           0.31985664 = score(doc=2215,freq=4.0), product of:
             0.37714836 = queryWeight, product of:
               5.487677 = boost
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.012661949 = queryNorm
             0.8480924 = fieldWeight in 2215, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.078125 = fieldNorm(doc=2215)
       0.2 = coord(5/25)
   

4. Rowe, N.C.: Precise and efficient retrieval of captioned images : the MARIE project (1999) 0.12

   0.11735268 = sum of:
     0.11735268 = product of:
       0.5867634 = sum of:
         0.025213618 = weight(abstract_txt:methods in 847) [ClassicSimilarity], result of:
           0.025213618 = score(doc=847,freq=2.0), product of:
             0.05503286 = queryWeight, product of:
               1.048126 = boost
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.012661949 = queryNorm
             0.4581557 = fieldWeight in 847, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.078125 = fieldNorm(doc=847)
         0.121491596 = weight(abstract_txt:captioned in 847) [ClassicSimilarity], result of:
           0.121491596 = score(doc=847,freq=1.0), product of:
             0.15699872 = queryWeight, product of:
               1.2518016 = boost
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.012661949 = queryNorm
             0.7738381 = fieldWeight in 847, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.078125 = fieldNorm(doc=847)
         0.037868626 = weight(abstract_txt:experiments in 847) [ClassicSimilarity], result of:
           0.037868626 = score(doc=847,freq=1.0), product of:
             0.090934426 = queryWeight, product of:
               1.3473071 = boost
               5.3304167 = idf(docFreq=581, maxDocs=44218)
               0.012661949 = queryNorm
             0.41643882 = fieldWeight in 847, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3304167 = idf(docFreq=581, maxDocs=44218)
               0.078125 = fieldNorm(doc=847)
         0.08233288 = weight(abstract_txt:image in 847) [ClassicSimilarity], result of:
           0.08233288 = score(doc=847,freq=2.0), product of:
             0.13865788 = queryWeight, product of:
               2.0376067 = boost
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.012661949 = queryNorm
             0.59378433 = fieldWeight in 847, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.078125 = fieldNorm(doc=847)
         0.31985664 = weight(abstract_txt:images in 847) [ClassicSimilarity], result of:
           0.31985664 = score(doc=847,freq=4.0), product of:
             0.37714836 = queryWeight, product of:
               5.487677 = boost
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.012661949 = queryNorm
             0.8480924 = fieldWeight in 847, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.078125 = fieldNorm(doc=847)
       0.2 = coord(5/25)
   

5. Dolin, R.; Agrawal, D.; El Abbadi, A.; Pearlman, J.: Using automated classification for summarizing and selecting heterogeneous information sources (1998) 0.12

   0.116632745 = sum of:
     0.116632745 = product of:
       0.36447734 = sum of:
         0.0070589725 = weight(abstract_txt:number in 1253) [ClassicSimilarity], result of:
           0.0070589725 = score(doc=1253,freq=1.0), product of:
             0.05465916 = queryWeight, product of:
               1.0445613 = boost
               4.132649 = idf(docFreq=1927, maxDocs=44218)
               0.012661949 = queryNorm
             0.12914528 = fieldWeight in 1253, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.132649 = idf(docFreq=1927, maxDocs=44218)
               0.03125 = fieldNorm(doc=1253)
         0.007131488 = weight(abstract_txt:methods in 1253) [ClassicSimilarity], result of:
           0.007131488 = score(doc=1253,freq=1.0), product of:
             0.05503286 = queryWeight, product of:
               1.048126 = boost
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.012661949 = queryNorm
             0.129586 = fieldWeight in 1253, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.03125 = fieldNorm(doc=1253)
         0.01060376 = weight(abstract_txt:user in 1253) [ClassicSimilarity], result of:
           0.01060376 = score(doc=1253,freq=2.0), product of:
             0.065137185 = queryWeight, product of:
               1.3965695 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.012661949 = queryNorm
             0.16279119 = fieldWeight in 1253, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.03125 = fieldNorm(doc=1253)
         0.023287253 = weight(abstract_txt:image in 1253) [ClassicSimilarity], result of:
           0.023287253 = score(doc=1253,freq=1.0), product of:
             0.13865788 = queryWeight, product of:
               2.0376067 = boost
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.012661949 = queryNorm
             0.16794756 = fieldWeight in 1253, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.374322 = idf(docFreq=556, maxDocs=44218)
               0.03125 = fieldNorm(doc=1253)
         0.06717294 = weight(abstract_txt:automatically in 1253) [ClassicSimilarity], result of:
           0.06717294 = score(doc=1253,freq=4.0), product of:
             0.19481462 = queryWeight, product of:
               2.7888696 = boost
               5.5168705 = idf(docFreq=482, maxDocs=44218)
               0.012661949 = queryNorm
             0.3448044 = fieldWeight in 1253, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.5168705 = idf(docFreq=482, maxDocs=44218)
               0.03125 = fieldNorm(doc=1253)
         0.05436711 = weight(abstract_txt:keywords in 1253) [ClassicSimilarity], result of:
           0.05436711 = score(doc=1253,freq=1.0), product of:
             0.28931624 = queryWeight, product of:
               3.7997835 = boost
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.012661949 = queryNorm
             0.18791586 = fieldWeight in 1253, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.03125 = fieldNorm(doc=1253)
         0.1308845 = weight(abstract_txt:referenced in 1253) [ClassicSimilarity], result of:
           0.1308845 = score(doc=1253,freq=1.0), product of:
             0.51968575 = queryWeight, product of:
               5.0926385 = boost
               8.059301 = idf(docFreq=37, maxDocs=44218)
               0.012661949 = queryNorm
             0.25185317 = fieldWeight in 1253, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.059301 = idf(docFreq=37, maxDocs=44218)
               0.03125 = fieldNorm(doc=1253)
         0.063971326 = weight(abstract_txt:images in 1253) [ClassicSimilarity], result of:
           0.063971326 = score(doc=1253,freq=1.0), product of:
             0.37714836 = queryWeight, product of:
               5.487677 = boost
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.012661949 = queryNorm
             0.16961847 = fieldWeight in 1253, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.427791 = idf(docFreq=527, maxDocs=44218)
               0.03125 = fieldNorm(doc=1253)
       0.32 = coord(8/25)