Document (#43560)

Author
Hasanain, M.
Elsayed, T.
Title
Studying effectiveness of Web search for fact checking
Source
Journal of the Association for Information Science and Technology. 73(2022) no.5, S.738-751
Year
2022
Abstract
Web search is commonly used by fact checking systems as a source of evidence for claim verification. In this work, we demonstrate that the task of retrieving pages useful for fact checking, called evidential pages, is indeed different from the task of retrieving topically relevant pages that are typically optimized by search engines; thus, it should be handled differently. We conduct a comprehensive study on the performance of retrieving evidential pages over a test collection we developed for the task of re-ranking Web pages by usefulness for fact-checking. Results show that pages (retrieved by a commercial search engine) that are topically relevant to a claim are not always useful for verifying it, and that the engine's performance in retrieving evidential pages is weakly correlated with retrieval of topically relevant pages. Additionally, we identify types of evidence in evidential pages and some linguistic cues that can help predict page usefulness. Moreover, preliminary experiments show that a retrieval model leveraging those cues has a higher performance compared to the search engine. Finally, we show that existing systems have a long way to go to support effective fact checking. To that end, our work provides insights to guide design of better future systems for the task.
Content
Vgl.: https://asistdl.onlinelibrary.wiley.com/doi/10.1002/asi.24577. https://doi.org/10.1002/asi.24577.
Theme
Internet
Field
Kommunikationswissenschaften

Similar documents (content)

  1. Liu, Y.; Zhang, M.; Cen, R.; Ru, L.; Ma, S.: Data cleansing for Web information retrieval using query independent features (2007) 0.17 
    0.16936839 = sum of:
  0.16936839 = product of:
    0.60488707 = sum of:
      0.027611556 = weight(abstract_txt:useful in 607) [ClassicSimilarity], result of:
        0.027611556 = score(doc=607,freq=2.0), product of:
          0.06455156 = queryWeight, product of:
            1.2196481 = boost
            4.839373 = idf(docFreq=950, maxDocs=44218)
            0.01093662 = queryNorm
          0.42774418 = fieldWeight in 607, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.839373 = idf(docFreq=950, maxDocs=44218)
            0.0625 = fieldNorm(doc=607)
      0.041139323 = weight(abstract_txt:engine in 607) [ClassicSimilarity], result of:
        0.041139323 = score(doc=607,freq=2.0), product of:
          0.084207535 = queryWeight, product of:
            1.3930178 = boost
            5.5272765 = idf(docFreq=477, maxDocs=44218)
            0.01093662 = queryNorm
          0.48854682 = fieldWeight in 607, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.5272765 = idf(docFreq=477, maxDocs=44218)
            0.0625 = fieldNorm(doc=607)
      0.025654353 = weight(abstract_txt:performance in 607) [ClassicSimilarity], result of:
        0.025654353 = score(doc=607,freq=1.0), product of:
          0.08864631 = queryWeight, product of:
            1.7504798 = boost
            4.63042 = idf(docFreq=1171, maxDocs=44218)
            0.01093662 = queryNorm
          0.28940126 = fieldWeight in 607, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.63042 = idf(docFreq=1171, maxDocs=44218)
            0.0625 = fieldNorm(doc=607)
      0.042162694 = weight(abstract_txt:search in 607) [ClassicSimilarity], result of:
        0.042162694 = score(doc=607,freq=4.0), product of:
          0.09220796 = queryWeight, product of:
            2.3048115 = boost
            3.6580524 = idf(docFreq=3098, maxDocs=44218)
            0.01093662 = queryNorm
          0.45725656 = fieldWeight in 607, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            3.6580524 = idf(docFreq=3098, maxDocs=44218)
            0.0625 = fieldNorm(doc=607)
      0.040815763 = weight(abstract_txt:task in 607) [ClassicSimilarity], result of:
        0.040815763 = score(doc=607,freq=1.0), product of:
          0.13296932 = queryWeight, product of:
            2.4755511 = boost
            4.9112997 = idf(docFreq=884, maxDocs=44218)
            0.01093662 = queryNorm
          0.30695623 = fieldWeight in 607, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.9112997 = idf(docFreq=884, maxDocs=44218)
            0.0625 = fieldNorm(doc=607)
      0.017862331 = weight(abstract_txt:that in 607) [ClassicSimilarity], result of:
        0.017862331 = score(doc=607,freq=3.0), product of:
          0.06963785 = queryWeight, product of:
            2.6872625 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.01093662 = queryNorm
          0.2565032 = fieldWeight in 607, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0625 = fieldNorm(doc=607)
      0.40964103 = weight(abstract_txt:pages in 607) [ClassicSimilarity], result of:
        0.40964103 = score(doc=607,freq=9.0), product of:
          0.38974613 = queryWeight, product of:
            6.3573823 = boost
            5.6055775 = idf(docFreq=441, maxDocs=44218)
            0.01093662 = queryNorm
          1.0510458 = fieldWeight in 607, product of:
            3.0 = tf(freq=9.0), with freq of:
              9.0 = termFreq=9.0
            5.6055775 = idf(docFreq=441, maxDocs=44218)
            0.0625 = fieldNorm(doc=607)
    0.28 = coord(7/25)
  2. Choi, B.; Peng, X.: Dynamic and hierarchical classification of Web pages (2004) 0.17 
    0.16545853 = sum of:
  0.16545853 = product of:
    0.5909233 = sum of:
      0.022220358 = weight(abstract_txt:systems in 2555) [ClassicSimilarity], result of:
        0.022220358 = score(doc=2555,freq=3.0), product of:
          0.048129007 = queryWeight, product of:
            1.2898234 = boost
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.01093662 = queryNorm
          0.4616833 = fieldWeight in 2555, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.078125 = fieldNorm(doc=2555)
      0.027625965 = weight(abstract_txt:show in 2555) [ClassicSimilarity], result of:
        0.027625965 = score(doc=2555,freq=1.0), product of:
          0.0802586 = queryWeight, product of:
            1.6656072 = boost
            4.4059124 = idf(docFreq=1466, maxDocs=44218)
            0.01093662 = queryNorm
          0.3442119 = fieldWeight in 2555, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.4059124 = idf(docFreq=1466, maxDocs=44218)
            0.078125 = fieldNorm(doc=2555)
      0.032174695 = weight(abstract_txt:relevant in 2555) [ClassicSimilarity], result of:
        0.032174695 = score(doc=2555,freq=1.0), product of:
          0.08884293 = queryWeight, product of:
            1.7524202 = boost
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.01093662 = queryNorm
          0.36215258 = fieldWeight in 2555, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.078125 = fieldNorm(doc=2555)
      0.03726691 = weight(abstract_txt:search in 2555) [ClassicSimilarity], result of:
        0.03726691 = score(doc=2555,freq=2.0), product of:
          0.09220796 = queryWeight, product of:
            2.3048115 = boost
            3.6580524 = idf(docFreq=3098, maxDocs=44218)
            0.01093662 = queryNorm
          0.4041615 = fieldWeight in 2555, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.6580524 = idf(docFreq=3098, maxDocs=44218)
            0.078125 = fieldNorm(doc=2555)
      0.018230665 = weight(abstract_txt:that in 2555) [ClassicSimilarity], result of:
        0.018230665 = score(doc=2555,freq=2.0), product of:
          0.06963785 = queryWeight, product of:
            2.6872625 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.01093662 = queryNorm
          0.26179248 = fieldWeight in 2555, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.078125 = fieldNorm(doc=2555)
      0.11203718 = weight(abstract_txt:retrieving in 2555) [ClassicSimilarity], result of:
        0.11203718 = score(doc=2555,freq=1.0), product of:
          0.22464716 = queryWeight, product of:
            3.217708 = boost
            6.3836813 = idf(docFreq=202, maxDocs=44218)
            0.01093662 = queryNorm
          0.49872512 = fieldWeight in 2555, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.3836813 = idf(docFreq=202, maxDocs=44218)
            0.078125 = fieldNorm(doc=2555)
      0.3413675 = weight(abstract_txt:pages in 2555) [ClassicSimilarity], result of:
        0.3413675 = score(doc=2555,freq=4.0), product of:
          0.38974613 = queryWeight, product of:
            6.3573823 = boost
            5.6055775 = idf(docFreq=441, maxDocs=44218)
            0.01093662 = queryNorm
          0.8758715 = fieldWeight in 2555, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            5.6055775 = idf(docFreq=441, maxDocs=44218)
            0.078125 = fieldNorm(doc=2555)
    0.28 = coord(7/25)
  3. Juneström, A.: Discourses of fact-checking in Swedish news media (2022) 0.16 
    0.15514599 = sum of:
  0.15514599 = product of:
    0.9696625 = sum of:
      0.025739755 = weight(abstract_txt:relevant in 686) [ClassicSimilarity], result of:
        0.025739755 = score(doc=686,freq=1.0), product of:
          0.08884293 = queryWeight, product of:
            1.7524202 = boost
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.01093662 = queryNorm
          0.28972206 = fieldWeight in 686, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.0625 = fieldNorm(doc=686)
      0.020625643 = weight(abstract_txt:that in 686) [ClassicSimilarity], result of:
        0.020625643 = score(doc=686,freq=4.0), product of:
          0.06963785 = queryWeight, product of:
            2.6872625 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.01093662 = queryNorm
          0.2961844 = fieldWeight in 686, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0625 = fieldNorm(doc=686)
      0.26531157 = weight(abstract_txt:fact in 686) [ClassicSimilarity], result of:
        0.26531157 = score(doc=686,freq=10.0), product of:
          0.23156539 = queryWeight, product of:
            3.6524813 = boost
            5.79699 = idf(docFreq=364, maxDocs=44218)
            0.01093662 = queryNorm
          1.1457307 = fieldWeight in 686, product of:
            3.1622777 = tf(freq=10.0), with freq of:
              10.0 = termFreq=10.0
            5.79699 = idf(docFreq=364, maxDocs=44218)
            0.0625 = fieldNorm(doc=686)
      0.6579855 = weight(abstract_txt:checking in 686) [ClassicSimilarity], result of:
        0.6579855 = score(doc=686,freq=10.0), product of:
          0.42427462 = queryWeight, product of:
            4.9439573 = boost
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.01093662 = queryNorm
          1.5508481 = fieldWeight in 686, product of:
            3.1622777 = tf(freq=10.0), with freq of:
              10.0 = termFreq=10.0
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.0625 = fieldNorm(doc=686)
    0.16 = coord(4/25)
  4. Choi, Y.: Effects of contextual factors on image searching on the Web (2010) 0.14 
    0.1441417 = sum of:
  0.1441417 = product of:
    0.60059047 = sum of:
      0.032067943 = weight(abstract_txt:performance in 3995) [ClassicSimilarity], result of:
        0.032067943 = score(doc=3995,freq=1.0), product of:
          0.08864631 = queryWeight, product of:
            1.7504798 = boost
            4.63042 = idf(docFreq=1171, maxDocs=44218)
            0.01093662 = queryNorm
          0.3617516 = fieldWeight in 3995, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.63042 = idf(docFreq=1171, maxDocs=44218)
            0.078125 = fieldNorm(doc=3995)
      0.05270337 = weight(abstract_txt:search in 3995) [ClassicSimilarity], result of:
        0.05270337 = score(doc=3995,freq=4.0), product of:
          0.09220796 = queryWeight, product of:
            2.3048115 = boost
            3.6580524 = idf(docFreq=3098, maxDocs=44218)
            0.01093662 = queryNorm
          0.5715707 = fieldWeight in 3995, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            3.6580524 = idf(docFreq=3098, maxDocs=44218)
            0.078125 = fieldNorm(doc=3995)
      0.072152756 = weight(abstract_txt:task in 3995) [ClassicSimilarity], result of:
        0.072152756 = score(doc=3995,freq=2.0), product of:
          0.13296932 = queryWeight, product of:
            2.4755511 = boost
            4.9112997 = idf(docFreq=884, maxDocs=44218)
            0.01093662 = queryNorm
          0.5426271 = fieldWeight in 3995, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.9112997 = idf(docFreq=884, maxDocs=44218)
            0.078125 = fieldNorm(doc=3995)
      0.012891028 = weight(abstract_txt:that in 3995) [ClassicSimilarity], result of:
        0.012891028 = score(doc=3995,freq=1.0), product of:
          0.06963785 = queryWeight, product of:
            2.6872625 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.01093662 = queryNorm
          0.18511525 = fieldWeight in 3995, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.078125 = fieldNorm(doc=3995)
      0.2600916 = weight(abstract_txt:checking in 3995) [ClassicSimilarity], result of:
        0.2600916 = score(doc=3995,freq=1.0), product of:
          0.42427462 = queryWeight, product of:
            4.9439573 = boost
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.01093662 = queryNorm
          0.61302656 = fieldWeight in 3995, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.078125 = fieldNorm(doc=3995)
      0.17068376 = weight(abstract_txt:pages in 3995) [ClassicSimilarity], result of:
        0.17068376 = score(doc=3995,freq=1.0), product of:
          0.38974613 = queryWeight, product of:
            6.3573823 = boost
            5.6055775 = idf(docFreq=441, maxDocs=44218)
            0.01093662 = queryNorm
          0.43793574 = fieldWeight in 3995, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.6055775 = idf(docFreq=441, maxDocs=44218)
            0.078125 = fieldNorm(doc=3995)
    0.24 = coord(6/25)
  5. Vaughan, L.: New measurements for search engine evaluation proposed and tested (2004) 0.14 
    0.14284313 = sum of:
  0.14284313 = product of:
    0.510154 = sum of:
      0.021771418 = weight(abstract_txt:systems in 2535) [ClassicSimilarity], result of:
        0.021771418 = score(doc=2535,freq=2.0), product of:
          0.048129007 = queryWeight, product of:
            1.2898234 = boost
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.01093662 = queryNorm
          0.45235544 = fieldWeight in 2535, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.09375 = fieldNorm(doc=2535)
      0.087269686 = weight(abstract_txt:engine in 2535) [ClassicSimilarity], result of:
        0.087269686 = score(doc=2535,freq=4.0), product of:
          0.084207535 = queryWeight, product of:
            1.3930178 = boost
            5.5272765 = idf(docFreq=477, maxDocs=44218)
            0.01093662 = queryNorm
          1.0363643 = fieldWeight in 2535, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            5.5272765 = idf(docFreq=477, maxDocs=44218)
            0.09375 = fieldNorm(doc=2535)
      0.033151157 = weight(abstract_txt:show in 2535) [ClassicSimilarity], result of:
        0.033151157 = score(doc=2535,freq=1.0), product of:
          0.0802586 = queryWeight, product of:
            1.6656072 = boost
            4.4059124 = idf(docFreq=1466, maxDocs=44218)
            0.01093662 = queryNorm
          0.4130543 = fieldWeight in 2535, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.4059124 = idf(docFreq=1466, maxDocs=44218)
            0.09375 = fieldNorm(doc=2535)
      0.07696306 = weight(abstract_txt:performance in 2535) [ClassicSimilarity], result of:
        0.07696306 = score(doc=2535,freq=4.0), product of:
          0.08864631 = queryWeight, product of:
            1.7504798 = boost
            4.63042 = idf(docFreq=1171, maxDocs=44218)
            0.01093662 = queryNorm
          0.86820376 = fieldWeight in 2535, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            4.63042 = idf(docFreq=1171, maxDocs=44218)
            0.09375 = fieldNorm(doc=2535)
      0.07070899 = weight(abstract_txt:search in 2535) [ClassicSimilarity], result of:
        0.07070899 = score(doc=2535,freq=5.0), product of:
          0.09220796 = queryWeight, product of:
            2.3048115 = boost
            3.6580524 = idf(docFreq=3098, maxDocs=44218)
            0.01093662 = queryNorm
          0.7668426 = fieldWeight in 2535, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            3.6580524 = idf(docFreq=3098, maxDocs=44218)
            0.09375 = fieldNorm(doc=2535)
      0.015469233 = weight(abstract_txt:that in 2535) [ClassicSimilarity], result of:
        0.015469233 = score(doc=2535,freq=1.0), product of:
          0.06963785 = queryWeight, product of:
            2.6872625 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.01093662 = queryNorm
          0.22213829 = fieldWeight in 2535, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.09375 = fieldNorm(doc=2535)
      0.20482051 = weight(abstract_txt:pages in 2535) [ClassicSimilarity], result of:
        0.20482051 = score(doc=2535,freq=1.0), product of:
          0.38974613 = queryWeight, product of:
            6.3573823 = boost
            5.6055775 = idf(docFreq=441, maxDocs=44218)
            0.01093662 = queryNorm
          0.5255229 = fieldWeight in 2535, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.6055775 = idf(docFreq=441, maxDocs=44218)
            0.09375 = fieldNorm(doc=2535)
    0.28 = coord(7/25)