Search (7 results, page 1 of 1)

  • × author_ss:"Kantor, P."
  1. Wacholder, N.; Kelly, D.; Kantor, P.; Rittman, R.; Sun, Y.; Bai, B.; Small, S.; Yamrom, B.; Strzalkowski, T.: ¬A model for quantitative evaluation of an end-to-end question-answering system (2007) 0.00 
    0.0041973717 = product of:
  0.0125921145 = sum of:
    0.0125921145 = weight(_text_:a in 435) [ClassicSimilarity], result of:
      0.0125921145 = score(doc=435,freq=20.0), product of:
        0.05209492 = queryWeight, product of:
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.045180224 = queryNorm
        0.24171482 = fieldWeight in 435, product of:
          4.472136 = tf(freq=20.0), with freq of:
            20.0 = termFreq=20.0
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.046875 = fieldNorm(doc=435)
  0.33333334 = coord(1/3)
    Abstract
    We describe a procedure for quantitative evaluation of interactive question-answering systems and illustrate it with application to the High-Quality Interactive QuestionAnswering (HITIQA) system. Our objectives were (a) to design a method to realistically and reliably assess interactive question-answering systems by comparing the quality of reports produced using different systems, (b) to conduct a pilot test of this method, and (c) to perform a formative evaluation of the HITIQA system. Far more important than the specific information gathered from this pilot evaluation is the development of (a) a protocol for evaluating an emerging technology, (b) reusable assessment instruments, and (c) the knowledge gained in conducting the evaluation. We conclude that this method, which uses a surprisingly small number of subjects and does not rely on predetermined relevance judgments, measures the impact of system change on work produced by users. Therefore this method can be used to compare the product of interactive systems that use different underlying technologies.
    Type
    a
  2. Saracevic, T.; Kantor, P.: ¬A study of information seeking and retrieving : pt.2: users, questions, and effectiveness (1988) 0.00 
    0.003754243 = product of:
  0.011262729 = sum of:
    0.011262729 = weight(_text_:a in 3258) [ClassicSimilarity], result of:
      0.011262729 = score(doc=3258,freq=4.0), product of:
        0.05209492 = queryWeight, product of:
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.045180224 = queryNorm
        0.2161963 = fieldWeight in 3258, product of:
          2.0 = tf(freq=4.0), with freq of:
            4.0 = termFreq=4.0
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.09375 = fieldNorm(doc=3258)
  0.33333334 = coord(1/3)
    Type
    a
  3. Saracevic, T.; Kantor, P.: ¬A study of information seeking and retrieving : pt.3: searchers, searches, and overlap (1988) 0.00 
    0.003754243 = product of:
  0.011262729 = sum of:
    0.011262729 = weight(_text_:a in 3259) [ClassicSimilarity], result of:
      0.011262729 = score(doc=3259,freq=4.0), product of:
        0.05209492 = queryWeight, product of:
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.045180224 = queryNorm
        0.2161963 = fieldWeight in 3259, product of:
          2.0 = tf(freq=4.0), with freq of:
            4.0 = termFreq=4.0
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.09375 = fieldNorm(doc=3259)
  0.33333334 = coord(1/3)
    Type
    a
  4. Saracevic, T.; Kantor, P.: Online searching : still an imprecise art (1991) 0.00 
    0.003128536 = product of:
  0.009385608 = sum of:
    0.009385608 = weight(_text_:a in 4843) [ClassicSimilarity], result of:
      0.009385608 = score(doc=4843,freq=4.0), product of:
        0.05209492 = queryWeight, product of:
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.045180224 = queryNorm
        0.18016359 = fieldWeight in 4843, product of:
          2.0 = tf(freq=4.0), with freq of:
            4.0 = termFreq=4.0
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.078125 = fieldNorm(doc=4843)
  0.33333334 = coord(1/3)
    Abstract
    Summarises the results of a research project which investigated the human decision making and user-system interactions which take place during online searching. Searches were carried out by professional searchers using DIALOG databases, and rated for relevance, precision and recall
    Type
    a
  5. Saracevic, T.; Kantor, P.; Chamis, A.Y.: ¬A study of information seeking and retrieving : pt.1: Background and methodology (1988) 0.00 
    0.00296799 = product of:
  0.00890397 = sum of:
    0.00890397 = weight(_text_:a in 1963) [ClassicSimilarity], result of:
      0.00890397 = score(doc=1963,freq=10.0), product of:
        0.05209492 = queryWeight, product of:
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.045180224 = queryNorm
        0.1709182 = fieldWeight in 1963, product of:
          3.1622777 = tf(freq=10.0), with freq of:
            10.0 = termFreq=10.0
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.046875 = fieldNorm(doc=1963)
  0.33333334 = coord(1/3)
    Abstract
    The objectives of the study were to conduct a series of observations and experiments under as real-life a situation as possible related to: (i) user context of questions in information retrieval; (ii) the structure and classification of questions; (iii) cognitive traits and decision making of searchers; and (iv) different searches of the same question. The study is presented in three parts: part 1 presents the background of the study and describes the models, measures, methods, procedures and statistical analyses used. Pt.2 is devoted to results related to users, questions, and effectiveness measures, and pt.3 to results related to searchers, searches, and overlap studies. A concluding summary of all results is presented in pt.3
    Type
    a
  6. Kantor, P.; Kim, M.H.; Ibraev, U.; Atasoy, K.: Estimating the number of relevant documents in enormous collections (1999) 0.00 
    0.002473325 = product of:
  0.0074199745 = sum of:
    0.0074199745 = weight(_text_:a in 6690) [ClassicSimilarity], result of:
      0.0074199745 = score(doc=6690,freq=10.0), product of:
        0.05209492 = queryWeight, product of:
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.045180224 = queryNorm
        0.14243183 = fieldWeight in 6690, product of:
          3.1622777 = tf(freq=10.0), with freq of:
            10.0 = termFreq=10.0
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.0390625 = fieldNorm(doc=6690)
  0.33333334 = coord(1/3)
    Abstract
    In assessing information retrieval systems, it is important to know not only the precision of the retrieved set, but also to compare the number of retrieved relevant items to the total number of relevant items. For large collections, such as the TREC test collections, or the World Wide Web, it is not possible to enumerate the entire set of relevant documents. If the retrieved documents are evaluated, a variant of the statistical "capture-recapture" method can be used to estimate the total number of relevant documents, providing the several retrieval systems used are sufficiently independent. We show that the underlying signal detection model supporting such an analysis can be extended in two ways. First, assuming that there are two distinct performance characteristics (corresponding to the chance of retrieving a relevant, and retrieving a given non-relevant document), we show that if there are three or more independent systems available it is possible to estimate the number of relevant documents without actually having to decide whether each individual document is relevant. We report applications of this 3-system method to the TREC data, leading to the conclusion that the independence assumptions are not satisfied. We then extend the model to a multi-system, multi-problem model, and show that it is possible to include statistical dependencies of all orders in the model, and determine the number of relevant documents for each of the problems in the set. Application to the TREC setting will be presented
    Type
    a
  7. Kelly, D.; Wacholder, N.; Rittman, R.; Sun, Y.; Kantor, P.; Small, S.; Strzalkowski, T.: Using interview data to identify evaluation criteria for interactive, analytical question-answering systems (2007) 0.00 
    0.0018771215 = product of:
  0.0056313644 = sum of:
    0.0056313644 = weight(_text_:a in 332) [ClassicSimilarity], result of:
      0.0056313644 = score(doc=332,freq=4.0), product of:
        0.05209492 = queryWeight, product of:
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.045180224 = queryNorm
        0.10809815 = fieldWeight in 332, product of:
          2.0 = tf(freq=4.0), with freq of:
            4.0 = termFreq=4.0
          1.153047 = idf(docFreq=37942, maxDocs=44218)
          0.046875 = fieldNorm(doc=332)
  0.33333334 = coord(1/3)
    Abstract
    The purpose of this work is to identify potential evaluation criteria for interactive, analytical question-answering (QA) systems by analyzing evaluative comments made by users of such a system. Qualitative data collected from intelligence analysts during interviews and focus groups were analyzed to identify common themes related to performance, use, and usability. These data were collected as part of an intensive, three-day evaluation workshop of the High-Quality Interactive Question Answering (HITIQA) system. Inductive coding and memoing were used to identify and categorize these data. Results suggest potential evaluation criteria for interactive, analytical QA systems, which can be used to guide the development and design of future systems and evaluations. This work contributes to studies of QA systems, information seeking and use behaviors, and interactive searching.
    Type
    a

Authors

Years

Themes