Document (#37665)

Author
Ostermann, F.O.
Tomko, M.
Purves, R.
Title
User evaluation of automatically generated keywords and toponyms for geo-referenced images
Source
Journal of the American Society for Information Science and Technology. 64(2013) no.3, S.480-499
Year
2013
Abstract
This article presents the results of a user evaluation of automatically generated concept keywords and place names (toponyms) for geo-referenced images. Automatically annotating images is becoming indispensable for effective information retrieval, since the number of geo-referenced images available online is growing, yet many images are insufficiently tagged or captioned to be efficiently searchable by standard information retrieval procedures. The Tripod project developed original methods for automatically annotating geo-referenced images by generating representations of the likely visible footprint of a geo-referenced image, and using this footprint to query spatial databases and web resources. These queries return raw lists of potential keywords and toponyms, which are subsequently filtered and ranked. This article reports on user experiments designed to evaluate the quality of the generated annotations. The experiments combined quantitative and qualitative approaches: To retrieve a large number of responses, participants rated the annotations in standardized online questionnaires that showed an image and its corresponding keywords. In addition, several focus groups provided rich qualitative information in open discussions. The results of the evaluation show that currently the annotation method performs better on rural images than on urban ones. Further, for each image at least one suitable keyword could be generated. The integration of heterogeneous data sources resulted in some images having a high level of noise in the form of obviously wrong or spurious keywords. The article discusses the evaluation itself and methods to improve the automatic generation of annotations.
Field
Geowissenschaften
Form
Bilder

Similar documents (author)

  1. Purves, R.S.; Sanderson, M.: ¬A methodology to allow avalanche forecasting on an information retrieval system (1998) 4.94
    4.9355655 = sum of:
      4.9355655 = weight(author_txt:purves in 2074) [ClassicSimilarity], result of:
        4.9355655 = fieldWeight in 2074, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.871131 = idf(docFreq=5, maxDocs=42740)
          0.5 = fieldNorm(doc=2074)
    
  2. Purves, D.; Lotto, R.B.; Nundy, S.: Richtiges Sehen - eine optische Täuschung? (2003) 3.70
    3.701674 = sum of:
      3.701674 = weight(author_txt:purves in 2892) [ClassicSimilarity], result of:
        3.701674 = fieldWeight in 2892, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.871131 = idf(docFreq=5, maxDocs=42740)
          0.375 = fieldNorm(doc=2892)
    
  3. Purves, R.S.; Winter, S.; Kuhn, W.: Places in information science (2019) 3.70
    3.701674 = sum of:
      3.701674 = weight(author_txt:purves in 1434) [ClassicSimilarity], result of:
        3.701674 = fieldWeight in 1434, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.871131 = idf(docFreq=5, maxDocs=42740)
          0.375 = fieldNorm(doc=1434)
    
  4. Reichenbacher, T.; Sabbata, S. de; Purves, R.S.; Fabrikant, S.I.: Assessing geographic relevance for mobile search : a computational model and its validation via crowdsourcing (2016) 3.08
    3.0847285 = sum of:
      3.0847285 = weight(author_txt:purves in 5146) [ClassicSimilarity], result of:
        3.0847285 = fieldWeight in 5146, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.871131 = idf(docFreq=5, maxDocs=42740)
          0.3125 = fieldNorm(doc=5146)
    

Similar documents (content)

  1. Tsai, C.-F.; McGarry, K.; Tait, J.: Qualitative evaluation of automatic assignment of keywords to images (2006) 0.28
    0.28207263 = sum of:
      0.28207263 = product of:
        0.78353506 = sum of:
          0.02888462 = weight(abstract_txt:methods in 2964) [ClassicSimilarity], result of:
            0.02888462 = score(doc=2964,freq=4.0), product of:
              0.05538278 = queryWeight, product of:
                1.056068 = boost
                4.172361 = idf(docFreq=1790, maxDocs=42740)
                0.012569008 = queryNorm
              0.5215451 = fieldWeight in 2964, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.172361 = idf(docFreq=1790, maxDocs=42740)
                0.0625 = fieldNorm(doc=2964)
          0.02101682 = weight(abstract_txt:user in 2964) [ClassicSimilarity], result of:
            0.02101682 = score(doc=2964,freq=2.0), product of:
              0.06461729 = queryWeight, product of:
                1.3970903 = boost
                3.6797917 = idf(docFreq=2930, maxDocs=42740)
                0.012569008 = queryNorm
              0.32525072 = fieldWeight in 2964, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6797917 = idf(docFreq=2930, maxDocs=42740)
                0.0625 = fieldNorm(doc=2964)
          0.03520956 = weight(abstract_txt:qualitative in 2964) [ClassicSimilarity], result of:
            0.03520956 = score(doc=2964,freq=1.0), product of:
              0.100320734 = queryWeight, product of:
                1.4213461 = boost
                5.6155186 = idf(docFreq=422, maxDocs=42740)
                0.012569008 = queryNorm
              0.3509699 = fieldWeight in 2964, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.6155186 = idf(docFreq=422, maxDocs=42740)
                0.0625 = fieldNorm(doc=2964)
          0.07997191 = weight(abstract_txt:image in 2964) [ClassicSimilarity], result of:
            0.07997191 = score(doc=2964,freq=3.0), product of:
              0.13758317 = queryWeight, product of:
                2.0386035 = boost
                5.369471 = idf(docFreq=540, maxDocs=42740)
                0.012569008 = queryNorm
              0.5812623 = fieldWeight in 2964, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.369471 = idf(docFreq=540, maxDocs=42740)
                0.0625 = fieldNorm(doc=2964)
          0.10200205 = weight(abstract_txt:evaluation in 2964) [ClassicSimilarity], result of:
            0.10200205 = score(doc=2964,freq=8.0), product of:
              0.12843092 = queryWeight, product of:
                2.2743342 = boost
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.012569008 = queryNorm
              0.7942172 = fieldWeight in 2964, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.0625 = fieldNorm(doc=2964)
          0.0666065 = weight(abstract_txt:automatically in 2964) [ClassicSimilarity], result of:
            0.0666065 = score(doc=2964,freq=1.0), product of:
              0.19333243 = queryWeight, product of:
                2.7904346 = boost
                5.5122876 = idf(docFreq=468, maxDocs=42740)
                0.012569008 = queryNorm
              0.34451798 = fieldWeight in 2964, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5122876 = idf(docFreq=468, maxDocs=42740)
                0.0625 = fieldNorm(doc=2964)
          0.12086889 = weight(abstract_txt:annotations in 2964) [ClassicSimilarity], result of:
            0.12086889 = score(doc=2964,freq=1.0), product of:
              0.2613307 = queryWeight, product of:
                2.8096054 = boost
                7.400211 = idf(docFreq=70, maxDocs=42740)
                0.012569008 = queryNorm
              0.46251318 = fieldWeight in 2964, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.400211 = idf(docFreq=70, maxDocs=42740)
                0.0625 = fieldNorm(doc=2964)
          0.11026385 = weight(abstract_txt:keywords in 2964) [ClassicSimilarity], result of:
            0.11026385 = score(doc=2964,freq=1.0), product of:
              0.2914422 = queryWeight, product of:
                3.830458 = boost
                6.0534186 = idf(docFreq=272, maxDocs=42740)
                0.012569008 = queryNorm
              0.37833866 = fieldWeight in 2964, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.0534186 = idf(docFreq=272, maxDocs=42740)
                0.0625 = fieldNorm(doc=2964)
          0.21871091 = weight(abstract_txt:images in 2964) [ClassicSimilarity], result of:
            0.21871091 = score(doc=2964,freq=3.0), product of:
              0.37311572 = queryWeight, product of:
                5.4822173 = boost
                5.414848 = idf(docFreq=516, maxDocs=42740)
                0.012569008 = queryNorm
              0.5861745 = fieldWeight in 2964, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.414848 = idf(docFreq=516, maxDocs=42740)
                0.0625 = fieldNorm(doc=2964)
        0.36 = coord(9/25)
    
  2. Jörgensen, C.; Jaimes, A.; Benitez, A.B.; Chang, S.-F.: ¬A conceptual framework and empirical research for classifying visual descriptors (2001) 0.12
    0.122978725 = sum of:
      0.122978725 = product of:
        0.51241136 = sum of:
          0.05325962 = weight(abstract_txt:experiments in 533) [ClassicSimilarity], result of:
            0.05325962 = score(doc=533,freq=3.0), product of:
              0.09165903 = queryWeight, product of:
                1.3586016 = boost
                5.3676248 = idf(docFreq=541, maxDocs=42740)
                0.012569008 = queryNorm
              0.58106244 = fieldWeight in 533, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.3676248 = idf(docFreq=541, maxDocs=42740)
                0.0625 = fieldNorm(doc=533)
          0.016856864 = weight(abstract_txt:article in 533) [ClassicSimilarity], result of:
            0.016856864 = score(doc=533,freq=1.0), product of:
              0.07028003 = queryWeight, product of:
                1.4570218 = boost
                3.8376453 = idf(docFreq=2502, maxDocs=42740)
                0.012569008 = queryNorm
              0.23985283 = fieldWeight in 533, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.8376453 = idf(docFreq=2502, maxDocs=42740)
                0.0625 = fieldNorm(doc=533)
          0.07997191 = weight(abstract_txt:image in 533) [ClassicSimilarity], result of:
            0.07997191 = score(doc=533,freq=3.0), product of:
              0.13758317 = queryWeight, product of:
                2.0386035 = boost
                5.369471 = idf(docFreq=540, maxDocs=42740)
                0.012569008 = queryNorm
              0.5812623 = fieldWeight in 533, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.369471 = idf(docFreq=540, maxDocs=42740)
                0.0625 = fieldNorm(doc=533)
          0.0666065 = weight(abstract_txt:automatically in 533) [ClassicSimilarity], result of:
            0.0666065 = score(doc=533,freq=1.0), product of:
              0.19333243 = queryWeight, product of:
                2.7904346 = boost
                5.5122876 = idf(docFreq=468, maxDocs=42740)
                0.012569008 = queryNorm
              0.34451798 = fieldWeight in 533, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5122876 = idf(docFreq=468, maxDocs=42740)
                0.0625 = fieldNorm(doc=533)
          0.11713978 = weight(abstract_txt:generated in 533) [ClassicSimilarity], result of:
            0.11713978 = score(doc=533,freq=3.0), product of:
              0.19530927 = queryWeight, product of:
                2.8046644 = boost
                5.5403976 = idf(docFreq=455, maxDocs=42740)
                0.012569008 = queryNorm
              0.5997656 = fieldWeight in 533, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.5403976 = idf(docFreq=455, maxDocs=42740)
                0.0625 = fieldNorm(doc=533)
          0.17857671 = weight(abstract_txt:images in 533) [ClassicSimilarity], result of:
            0.17857671 = score(doc=533,freq=2.0), product of:
              0.37311572 = queryWeight, product of:
                5.4822173 = boost
                5.414848 = idf(docFreq=516, maxDocs=42740)
                0.012569008 = queryNorm
              0.47860944 = fieldWeight in 533, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.414848 = idf(docFreq=516, maxDocs=42740)
                0.0625 = fieldNorm(doc=533)
        0.24 = coord(6/25)
    
  3. Rasmussen, E.M.: Indexing images (1997) 0.12
    0.11822522 = sum of:
      0.11822522 = product of:
        0.5911261 = sum of:
          0.018052887 = weight(abstract_txt:methods in 3216) [ClassicSimilarity], result of:
            0.018052887 = score(doc=3216,freq=1.0), product of:
              0.05538278 = queryWeight, product of:
                1.056068 = boost
                4.172361 = idf(docFreq=1790, maxDocs=42740)
                0.012569008 = queryNorm
              0.3259657 = fieldWeight in 3216, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.172361 = idf(docFreq=1790, maxDocs=42740)
                0.078125 = fieldNorm(doc=3216)
          0.1290541 = weight(abstract_txt:image in 3216) [ClassicSimilarity], result of:
            0.1290541 = score(doc=3216,freq=5.0), product of:
              0.13758317 = queryWeight, product of:
                2.0386035 = boost
                5.369471 = idf(docFreq=540, maxDocs=42740)
                0.012569008 = queryNorm
              0.93800795 = fieldWeight in 3216, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                5.369471 = idf(docFreq=540, maxDocs=42740)
                0.078125 = fieldNorm(doc=3216)
          0.045078963 = weight(abstract_txt:evaluation in 3216) [ClassicSimilarity], result of:
            0.045078963 = score(doc=3216,freq=1.0), product of:
              0.12843092 = queryWeight, product of:
                2.2743342 = boost
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.012569008 = queryNorm
              0.35099775 = fieldWeight in 3216, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.078125 = fieldNorm(doc=3216)
          0.08325812 = weight(abstract_txt:automatically in 3216) [ClassicSimilarity], result of:
            0.08325812 = score(doc=3216,freq=1.0), product of:
              0.19333243 = queryWeight, product of:
                2.7904346 = boost
                5.5122876 = idf(docFreq=468, maxDocs=42740)
                0.012569008 = queryNorm
              0.43064746 = fieldWeight in 3216, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5122876 = idf(docFreq=468, maxDocs=42740)
                0.078125 = fieldNorm(doc=3216)
          0.31568202 = weight(abstract_txt:images in 3216) [ClassicSimilarity], result of:
            0.31568202 = score(doc=3216,freq=4.0), product of:
              0.37311572 = queryWeight, product of:
                5.4822173 = boost
                5.414848 = idf(docFreq=516, maxDocs=42740)
                0.012569008 = queryNorm
              0.84607 = fieldWeight in 3216, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.414848 = idf(docFreq=516, maxDocs=42740)
                0.078125 = fieldNorm(doc=3216)
        0.2 = coord(5/25)
    
  4. Dolin, R.; Agrawal, D.; El Abbadi, A.; Pearlman, J.: Using automated classification for summarizing and selecting heterogeneous information sources (1998) 0.12
    0.11729736 = sum of:
      0.11729736 = product of:
        0.36655426 = sum of:
          0.006983073 = weight(abstract_txt:number in 3254) [ClassicSimilarity], result of:
            0.006983073 = score(doc=3254,freq=1.0), product of:
              0.054158676 = queryWeight, product of:
                1.0443318 = boost
                4.1259933 = idf(docFreq=1875, maxDocs=42740)
                0.012569008 = queryNorm
              0.12893729 = fieldWeight in 3254, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1259933 = idf(docFreq=1875, maxDocs=42740)
                0.03125 = fieldNorm(doc=3254)
          0.007221155 = weight(abstract_txt:methods in 3254) [ClassicSimilarity], result of:
            0.007221155 = score(doc=3254,freq=1.0), product of:
              0.05538278 = queryWeight, product of:
                1.056068 = boost
                4.172361 = idf(docFreq=1790, maxDocs=42740)
                0.012569008 = queryNorm
              0.13038628 = fieldWeight in 3254, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.172361 = idf(docFreq=1790, maxDocs=42740)
                0.03125 = fieldNorm(doc=3254)
          0.01050841 = weight(abstract_txt:user in 3254) [ClassicSimilarity], result of:
            0.01050841 = score(doc=3254,freq=2.0), product of:
              0.06461729 = queryWeight, product of:
                1.3970903 = boost
                3.6797917 = idf(docFreq=2930, maxDocs=42740)
                0.012569008 = queryNorm
              0.16262536 = fieldWeight in 3254, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6797917 = idf(docFreq=2930, maxDocs=42740)
                0.03125 = fieldNorm(doc=3254)
          0.023085902 = weight(abstract_txt:image in 3254) [ClassicSimilarity], result of:
            0.023085902 = score(doc=3254,freq=1.0), product of:
              0.13758317 = queryWeight, product of:
                2.0386035 = boost
                5.369471 = idf(docFreq=540, maxDocs=42740)
                0.012569008 = queryNorm
              0.16779597 = fieldWeight in 3254, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.369471 = idf(docFreq=540, maxDocs=42740)
                0.03125 = fieldNorm(doc=3254)
          0.0666065 = weight(abstract_txt:automatically in 3254) [ClassicSimilarity], result of:
            0.0666065 = score(doc=3254,freq=4.0), product of:
              0.19333243 = queryWeight, product of:
                2.7904346 = boost
                5.5122876 = idf(docFreq=468, maxDocs=42740)
                0.012569008 = queryNorm
              0.34451798 = fieldWeight in 3254, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.5122876 = idf(docFreq=468, maxDocs=42740)
                0.03125 = fieldNorm(doc=3254)
          0.055131923 = weight(abstract_txt:keywords in 3254) [ClassicSimilarity], result of:
            0.055131923 = score(doc=3254,freq=1.0), product of:
              0.2914422 = queryWeight, product of:
                3.830458 = boost
                6.0534186 = idf(docFreq=272, maxDocs=42740)
                0.012569008 = queryNorm
              0.18916933 = fieldWeight in 3254, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.0534186 = idf(docFreq=272, maxDocs=42740)
                0.03125 = fieldNorm(doc=3254)
          0.13388091 = weight(abstract_txt:referenced in 3254) [ClassicSimilarity], result of:
            0.13388091 = score(doc=3254,freq=1.0), product of:
              0.52653766 = queryWeight, product of:
                5.1486006 = boost
                8.13653 = idf(docFreq=33, maxDocs=42740)
                0.012569008 = queryNorm
              0.25426656 = fieldWeight in 3254, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.13653 = idf(docFreq=33, maxDocs=42740)
                0.03125 = fieldNorm(doc=3254)
          0.0631364 = weight(abstract_txt:images in 3254) [ClassicSimilarity], result of:
            0.0631364 = score(doc=3254,freq=1.0), product of:
              0.37311572 = queryWeight, product of:
                5.4822173 = boost
                5.414848 = idf(docFreq=516, maxDocs=42740)
                0.012569008 = queryNorm
              0.169214 = fieldWeight in 3254, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.414848 = idf(docFreq=516, maxDocs=42740)
                0.03125 = fieldNorm(doc=3254)
        0.32 = coord(8/25)
    
  5. Rowe, N.C.: Precise and efficient retrieval of captioned images : the MARIE project (1999) 0.12
    0.11615981 = sum of:
      0.11615981 = product of:
        0.58079904 = sum of:
          0.025530638 = weight(abstract_txt:methods in 1973) [ClassicSimilarity], result of:
            0.025530638 = score(doc=1973,freq=2.0), product of:
              0.05538278 = queryWeight, product of:
                1.056068 = boost
                4.172361 = idf(docFreq=1790, maxDocs=42740)
                0.012569008 = queryNorm
              0.46098512 = fieldWeight in 1973, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.172361 = idf(docFreq=1790, maxDocs=42740)
                0.078125 = fieldNorm(doc=1973)
          0.11952858 = weight(abstract_txt:captioned in 1973) [ClassicSimilarity], result of:
            0.11952858 = score(doc=1973,freq=1.0), product of:
              0.15499397 = queryWeight, product of:
                1.2492429 = boost
                9.871131 = idf(docFreq=5, maxDocs=42740)
                0.012569008 = queryNorm
              0.7711821 = fieldWeight in 1973, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.871131 = idf(docFreq=5, maxDocs=42740)
                0.078125 = fieldNorm(doc=1973)
          0.03843682 = weight(abstract_txt:experiments in 1973) [ClassicSimilarity], result of:
            0.03843682 = score(doc=1973,freq=1.0), product of:
              0.09165903 = queryWeight, product of:
                1.3586016 = boost
                5.3676248 = idf(docFreq=541, maxDocs=42740)
                0.012569008 = queryNorm
              0.41934568 = fieldWeight in 1973, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.3676248 = idf(docFreq=541, maxDocs=42740)
                0.078125 = fieldNorm(doc=1973)
          0.08162099 = weight(abstract_txt:image in 1973) [ClassicSimilarity], result of:
            0.08162099 = score(doc=1973,freq=2.0), product of:
              0.13758317 = queryWeight, product of:
                2.0386035 = boost
                5.369471 = idf(docFreq=540, maxDocs=42740)
                0.012569008 = queryNorm
              0.59324837 = fieldWeight in 1973, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.369471 = idf(docFreq=540, maxDocs=42740)
                0.078125 = fieldNorm(doc=1973)
          0.31568202 = weight(abstract_txt:images in 1973) [ClassicSimilarity], result of:
            0.31568202 = score(doc=1973,freq=4.0), product of:
              0.37311572 = queryWeight, product of:
                5.4822173 = boost
                5.414848 = idf(docFreq=516, maxDocs=42740)
                0.012569008 = queryNorm
              0.84607 = fieldWeight in 1973, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.414848 = idf(docFreq=516, maxDocs=42740)
                0.078125 = fieldNorm(doc=1973)
        0.2 = coord(5/25)