Document (#38531)

Author

Zuccala, A.
Someren, M. van
Bellen, M. van

Title

¬A machine-learning approach to coding book reviews as quality indicators : toward a theory of megacitation

Source

Journal of the Association for Information Science and Technology. 65(2014) no.11, S.2248-2260

Year

2014

Abstract

A theory of "megacitation" is introduced and used in an experiment to demonstrate how a qualitative scholarly book review can be converted into a weighted bibliometric indicator. We employ a manual human-coding approach to classify book reviews in the field of history based on reviewers' assessments of a book author's scholarly credibility (SC) and writing style (WS). In total, 100 book reviews were selected from the American Historical Review and coded for their positive/negative valence on these two dimensions. Most were coded as positive (68% for SC and 47% for WS), and there was also a small positive correlation between SC and WS (r = 0.2). We then constructed a classifier, combining both manual design and machine learning, to categorize sentiment-based sentences in history book reviews. The machine classifier produced a matched accuracy (matched to the human coding) of approximately 75% for SC and 64% for WS. WS was found to be more difficult to classify by machine than SC because of the reviewers' use of more subtle language. With further training data, a machine-learning approach could be useful for automatically classifying a large number of history book reviews at once. Weighted megacitations can be especially valuable if they are used in conjunction with regular book/journal citations, and "libcitations" (i.e., library holding counts) for a comprehensive assessment of a book/monograph's scholarly impact.

Theme

Informetrie

Similar documents (author)

1. Zuccala, A.: Modeling the invisible college (2006) 5.76

   5.7574883 = sum of:
     5.7574883 = weight(author_txt:zuccala in 3350) [ClassicSimilarity], result of:
       5.7574883 = fieldWeight in 3350, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.211981 = idf(docFreq=11, maxDocs=44218)
         0.625 = fieldNorm(doc=3350)
   

2. Zuccala, A.: Author cocitation analysis is to intellectual structure as Web colink analysis is to ... ? (2006) 5.76

   5.7574883 = sum of:
     5.7574883 = weight(author_txt:zuccala in 6008) [ClassicSimilarity], result of:
       5.7574883 = fieldWeight in 6008, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.211981 = idf(docFreq=11, maxDocs=44218)
         0.625 = fieldNorm(doc=6008)
   

3. Rousseau, R.; Zuccala, A.: ¬A classification of author co-citations : definitions and search strategies (2004) 4.61

   4.6059904 = sum of:
     4.6059904 = weight(author_txt:zuccala in 2266) [ClassicSimilarity], result of:
       4.6059904 = fieldWeight in 2266, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.211981 = idf(docFreq=11, maxDocs=44218)
         0.5 = fieldNorm(doc=2266)
   

4. Zuccala, A.; Leeuwen, T.van: Book reviews in humanities research evaluations (2011) 4.61

   4.6059904 = sum of:
     4.6059904 = weight(author_txt:zuccala in 4771) [ClassicSimilarity], result of:
       4.6059904 = fieldWeight in 4771, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.211981 = idf(docFreq=11, maxDocs=44218)
         0.5 = fieldNorm(doc=4771)
   

5. White, H.D.; Zuccala, A.A.: Libcitations, worldcat, cultural impact, and fame (2018) 4.61

   4.6059904 = sum of:
     4.6059904 = weight(author_txt:zuccala in 4578) [ClassicSimilarity], result of:
       4.6059904 = fieldWeight in 4578, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.211981 = idf(docFreq=11, maxDocs=44218)
         0.5 = fieldNorm(doc=4578)
   

Similar documents (content)

1. Zuccala, A.; Leeuwen, T.van: Book reviews in humanities research evaluations (2011) 0.20

   0.20334828 = sum of:
     0.20334828 = product of:
       0.8472845 = sum of:
         0.041502617 = weight(abstract_txt:review in 4771) [ClassicSimilarity], result of:
           0.041502617 = score(doc=4771,freq=2.0), product of:
             0.09664512 = queryWeight, product of:
               1.2890632 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.015431392 = queryNorm
             0.42943317 = fieldWeight in 4771, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0625 = fieldNorm(doc=4771)
         0.047380324 = weight(abstract_txt:history in 4771) [ClassicSimilarity], result of:
           0.047380324 = score(doc=4771,freq=1.0), product of:
             0.15225399 = queryWeight, product of:
               1.9815919 = boost
               4.9790826 = idf(docFreq=826, maxDocs=44218)
               0.015431392 = queryNorm
             0.31119266 = fieldWeight in 4771, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9790826 = idf(docFreq=826, maxDocs=44218)
               0.0625 = fieldNorm(doc=4771)
         0.124590226 = weight(abstract_txt:scholarly in 4771) [ClassicSimilarity], result of:
           0.124590226 = score(doc=4771,freq=4.0), product of:
             0.18272837 = queryWeight, product of:
               2.1708653 = boost
               5.4546638 = idf(docFreq=513, maxDocs=44218)
               0.015431392 = queryNorm
             0.68183297 = fieldWeight in 4771, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.4546638 = idf(docFreq=513, maxDocs=44218)
               0.0625 = fieldNorm(doc=4771)
         0.07923375 = weight(abstract_txt:positive in 4771) [ClassicSimilarity], result of:
           0.07923375 = score(doc=4771,freq=1.0), product of:
             0.21450798 = queryWeight, product of:
               2.3520775 = boost
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.015431392 = queryNorm
             0.36937436 = fieldWeight in 4771, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.0625 = fieldNorm(doc=4771)
         0.20563994 = weight(abstract_txt:reviews in 4771) [ClassicSimilarity], result of:
           0.20563994 = score(doc=4771,freq=7.0), product of:
             0.25108758 = queryWeight, product of:
               3.2852383 = boost
               4.952828 = idf(docFreq=848, maxDocs=44218)
               0.015431392 = queryNorm
             0.8189969 = fieldWeight in 4771, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.952828 = idf(docFreq=848, maxDocs=44218)
               0.0625 = fieldNorm(doc=4771)
         0.34893763 = weight(abstract_txt:book in 4771) [ClassicSimilarity], result of:
           0.34893763 = score(doc=4771,freq=7.0), product of:
             0.43451983 = queryWeight, product of:
               5.7982287 = boost
               4.856341 = idf(docFreq=934, maxDocs=44218)
               0.015431392 = queryNorm
             0.8030419 = fieldWeight in 4771, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.856341 = idf(docFreq=934, maxDocs=44218)
               0.0625 = fieldNorm(doc=4771)
       0.24 = coord(6/25)
   

2. Na, J.-C.; Sui, H.; Khoo, C.; Chan, S.; Zhou, Y.: Effectiveness of simple linguistic processing in automatic sentiment classification of product reviews (2004) 0.16

   0.162521 = sum of:
     0.162521 = product of:
       0.5804322 = sum of:
         0.11029276 = weight(abstract_txt:sentiment in 2624) [ClassicSimilarity], result of:
           0.11029276 = score(doc=2624,freq=4.0), product of:
             0.11680845 = queryWeight, product of:
               1.0020893 = boost
               7.5537524 = idf(docFreq=62, maxDocs=44218)
               0.015431392 = queryNorm
             0.94421905 = fieldWeight in 2624, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               7.5537524 = idf(docFreq=62, maxDocs=44218)
               0.0625 = fieldNorm(doc=2624)
         0.029346785 = weight(abstract_txt:review in 2624) [ClassicSimilarity], result of:
           0.029346785 = score(doc=2624,freq=1.0), product of:
             0.09664512 = queryWeight, product of:
               1.2890632 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.015431392 = queryNorm
             0.30365512 = fieldWeight in 2624, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0625 = fieldNorm(doc=2624)
         0.028519018 = weight(abstract_txt:approach in 2624) [ClassicSimilarity], result of:
           0.028519018 = score(doc=2624,freq=2.0), product of:
             0.086148895 = queryWeight, product of:
               1.4905782 = boost
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.015431392 = queryNorm
             0.33104333 = fieldWeight in 2624, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.0625 = fieldNorm(doc=2624)
         0.071508884 = weight(abstract_txt:classify in 2624) [ClassicSimilarity], result of:
           0.071508884 = score(doc=2624,freq=1.0), product of:
             0.1750033 = queryWeight, product of:
               1.7346321 = boost
               6.537832 = idf(docFreq=173, maxDocs=44218)
               0.015431392 = queryNorm
             0.4086145 = fieldWeight in 2624, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.537832 = idf(docFreq=173, maxDocs=44218)
               0.0625 = fieldNorm(doc=2624)
         0.11205344 = weight(abstract_txt:positive in 2624) [ClassicSimilarity], result of:
           0.11205344 = score(doc=2624,freq=2.0), product of:
             0.21450798 = queryWeight, product of:
               2.3520775 = boost
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.015431392 = queryNorm
             0.5223742 = fieldWeight in 2624, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.0625 = fieldNorm(doc=2624)
         0.13462295 = weight(abstract_txt:reviews in 2624) [ClassicSimilarity], result of:
           0.13462295 = score(doc=2624,freq=3.0), product of:
             0.25108758 = queryWeight, product of:
               3.2852383 = boost
               4.952828 = idf(docFreq=848, maxDocs=44218)
               0.015431392 = queryNorm
             0.53615934 = fieldWeight in 2624, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.952828 = idf(docFreq=848, maxDocs=44218)
               0.0625 = fieldNorm(doc=2624)
         0.09408835 = weight(abstract_txt:machine in 2624) [ClassicSimilarity], result of:
           0.09408835 = score(doc=2624,freq=1.0), product of:
             0.28519604 = queryWeight, product of:
               3.5012734 = boost
               5.2785225 = idf(docFreq=612, maxDocs=44218)
               0.015431392 = queryNorm
             0.32990766 = fieldWeight in 2624, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2785225 = idf(docFreq=612, maxDocs=44218)
               0.0625 = fieldNorm(doc=2624)
       0.28 = coord(7/25)
   

3. Jia, Y.; Liu, I.L.B.: Do consumers always follow "useful" reviews? : The interaction effect of review valence and review usefulness on consumers' purchase decisions (2018) 0.16

   0.16203226 = sum of:
     0.16203226 = product of:
       0.6751344 = sum of:
         0.029853811 = weight(abstract_txt:theory in 4541) [ClassicSimilarity], result of:
           0.029853811 = score(doc=4541,freq=1.0), product of:
             0.08424279 = queryWeight, product of:
               1.2035125 = boost
               4.5360413 = idf(docFreq=1287, maxDocs=44218)
               0.015431392 = queryNorm
             0.35437822 = fieldWeight in 4541, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5360413 = idf(docFreq=1287, maxDocs=44218)
               0.078125 = fieldNorm(doc=4541)
         0.22966404 = weight(abstract_txt:valence in 4541) [ClassicSimilarity], result of:
           0.22966404 = score(doc=4541,freq=3.0), product of:
             0.18066658 = queryWeight, product of:
               1.2462586 = boost
               9.394302 = idf(docFreq=9, maxDocs=44218)
               0.015431392 = queryNorm
             1.2712038 = fieldWeight in 4541, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               9.394302 = idf(docFreq=9, maxDocs=44218)
               0.078125 = fieldNorm(doc=4541)
         0.09705537 = weight(abstract_txt:review in 4541) [ClassicSimilarity], result of:
           0.09705537 = score(doc=4541,freq=7.0), product of:
             0.09664512 = queryWeight, product of:
               1.2890632 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.015431392 = queryNorm
             1.0042449 = fieldWeight in 4541, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.078125 = fieldNorm(doc=4541)
         0.025207488 = weight(abstract_txt:approach in 4541) [ClassicSimilarity], result of:
           0.025207488 = score(doc=4541,freq=1.0), product of:
             0.086148895 = queryWeight, product of:
               1.4905782 = boost
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.015431392 = queryNorm
             0.29260373 = fieldWeight in 4541, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.078125 = fieldNorm(doc=4541)
         0.09904219 = weight(abstract_txt:positive in 4541) [ClassicSimilarity], result of:
           0.09904219 = score(doc=4541,freq=1.0), product of:
             0.21450798 = queryWeight, product of:
               2.3520775 = boost
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.015431392 = queryNorm
             0.46171796 = fieldWeight in 4541, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.078125 = fieldNorm(doc=4541)
         0.19431148 = weight(abstract_txt:reviews in 4541) [ClassicSimilarity], result of:
           0.19431148 = score(doc=4541,freq=4.0), product of:
             0.25108758 = queryWeight, product of:
               3.2852383 = boost
               4.952828 = idf(docFreq=848, maxDocs=44218)
               0.015431392 = queryNorm
             0.77387935 = fieldWeight in 4541, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.952828 = idf(docFreq=848, maxDocs=44218)
               0.078125 = fieldNorm(doc=4541)
       0.24 = coord(6/25)
   

4. Nobarany, S.; Booth, K.S.: Use of politeness strategies in signed open peer review (2015) 0.14

   0.14119777 = sum of:
     0.14119777 = product of:
       0.58832407 = sum of:
         0.023883048 = weight(abstract_txt:theory in 1825) [ClassicSimilarity], result of:
           0.023883048 = score(doc=1825,freq=1.0), product of:
             0.08424279 = queryWeight, product of:
               1.2035125 = boost
               4.5360413 = idf(docFreq=1287, maxDocs=44218)
               0.015431392 = queryNorm
             0.28350258 = fieldWeight in 1825, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5360413 = idf(docFreq=1287, maxDocs=44218)
               0.0625 = fieldNorm(doc=1825)
         0.026432255 = weight(abstract_txt:human in 1825) [ClassicSimilarity], result of:
           0.026432255 = score(doc=1825,freq=1.0), product of:
             0.09013547 = queryWeight, product of:
               1.2448933 = boost
               4.692005 = idf(docFreq=1101, maxDocs=44218)
               0.015431392 = queryNorm
             0.29325032 = fieldWeight in 1825, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.692005 = idf(docFreq=1101, maxDocs=44218)
               0.0625 = fieldNorm(doc=1825)
         0.050830122 = weight(abstract_txt:review in 1825) [ClassicSimilarity], result of:
           0.050830122 = score(doc=1825,freq=3.0), product of:
             0.09664512 = queryWeight, product of:
               1.2890632 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.015431392 = queryNorm
             0.5259461 = fieldWeight in 1825, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0625 = fieldNorm(doc=1825)
         0.062295113 = weight(abstract_txt:scholarly in 1825) [ClassicSimilarity], result of:
           0.062295113 = score(doc=1825,freq=1.0), product of:
             0.18272837 = queryWeight, product of:
               2.1708653 = boost
               5.4546638 = idf(docFreq=513, maxDocs=44218)
               0.015431392 = queryNorm
             0.34091648 = fieldWeight in 1825, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4546638 = idf(docFreq=513, maxDocs=44218)
               0.0625 = fieldNorm(doc=1825)
         0.34564975 = weight(abstract_txt:reviewers in 1825) [ClassicSimilarity], result of:
           0.34564975 = score(doc=1825,freq=6.0), product of:
             0.27532563 = queryWeight, product of:
               2.1757429 = boost
               8.200379 = idf(docFreq=32, maxDocs=44218)
               0.015431392 = queryNorm
             1.2554216 = fieldWeight in 1825, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               8.200379 = idf(docFreq=32, maxDocs=44218)
               0.0625 = fieldNorm(doc=1825)
         0.07923375 = weight(abstract_txt:positive in 1825) [ClassicSimilarity], result of:
           0.07923375 = score(doc=1825,freq=1.0), product of:
             0.21450798 = queryWeight, product of:
               2.3520775 = boost
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.015431392 = queryNorm
             0.36937436 = fieldWeight in 1825, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.0625 = fieldNorm(doc=1825)
       0.24 = coord(6/25)
   

5. Chua, A.Y.K.; Banerjee, S.: Understanding review helpfulness as a function of reviewer reputation, review rating, and review depth (2015) 0.13

   0.12953736 = sum of:
     0.12953736 = product of:
       0.6476868 = sum of:
         0.035824575 = weight(abstract_txt:theory in 1641) [ClassicSimilarity], result of:
           0.035824575 = score(doc=1641,freq=1.0), product of:
             0.08424279 = queryWeight, product of:
               1.2035125 = boost
               4.5360413 = idf(docFreq=1287, maxDocs=44218)
               0.015431392 = queryNorm
             0.42525387 = fieldWeight in 1641, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5360413 = idf(docFreq=1287, maxDocs=44218)
               0.09375 = fieldNorm(doc=1641)
         0.11646644 = weight(abstract_txt:review in 1641) [ClassicSimilarity], result of:
           0.11646644 = score(doc=1641,freq=7.0), product of:
             0.09664512 = queryWeight, product of:
               1.2890632 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.015431392 = queryNorm
             1.2050939 = fieldWeight in 1641, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.09375 = fieldNorm(doc=1641)
         0.21166638 = weight(abstract_txt:reviewers in 1641) [ClassicSimilarity], result of:
           0.21166638 = score(doc=1641,freq=1.0), product of:
             0.27532563 = queryWeight, product of:
               2.1757429 = boost
               8.200379 = idf(docFreq=32, maxDocs=44218)
               0.015431392 = queryNorm
             0.7687856 = fieldWeight in 1641, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.200379 = idf(docFreq=32, maxDocs=44218)
               0.09375 = fieldNorm(doc=1641)
         0.11885062 = weight(abstract_txt:positive in 1641) [ClassicSimilarity], result of:
           0.11885062 = score(doc=1641,freq=1.0), product of:
             0.21450798 = queryWeight, product of:
               2.3520775 = boost
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.015431392 = queryNorm
             0.55406153 = fieldWeight in 1641, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.09375 = fieldNorm(doc=1641)
         0.16487877 = weight(abstract_txt:reviews in 1641) [ClassicSimilarity], result of:
           0.16487877 = score(doc=1641,freq=2.0), product of:
             0.25108758 = queryWeight, product of:
               3.2852383 = boost
               4.952828 = idf(docFreq=848, maxDocs=44218)
               0.015431392 = queryNorm
             0.6566584 = fieldWeight in 1641, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.952828 = idf(docFreq=848, maxDocs=44218)
               0.09375 = fieldNorm(doc=1641)
       0.2 = coord(5/25)