Search (80 results, page 4 of 4)

0.001153389 = product of:
  0.010380501 = sum of:
    0.010380501 = product of:
      0.020761002 = sum of:
        0.020761002 = weight(_text_:web in 716) [ClassicSimilarity], result of:
          0.020761002 = score(doc=716,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.21634221 = fieldWeight in 716, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.046875 = fieldNorm(doc=716)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Abstract

The objective of this paper is to highlight the importance of a scientifically sounded approach to search engine evaluation. Nowadays there is a flourishing literature which describes various attempts at conducting such evaluation by following all sort of approaches, but very often only the final results are published with little, if any, information about the methodology and the procedures adopted. These various experiments have been critically investigated and catalogued according to their scientific foundation by Bell [1] in the attempt to provide a valuable framework for future studies in this area. This paper reconsiders some of Bell's ideas in the light of the crisis of classic evaluation techniques for information retrieval and tries to envisage some form of collaboration between the IR and web communities in order to design a better and more consistent platform for the evaluation of tools for interactive information retrieval.

0.001153389 = product of:
  0.010380501 = sum of:
    0.010380501 = product of:
      0.020761002 = sum of:
        0.020761002 = weight(_text_:web in 1034) [ClassicSimilarity], result of:
          0.020761002 = score(doc=1034,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.21634221 = fieldWeight in 1034, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.046875 = fieldNorm(doc=1034)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Abstract

Search engines play an essential role in the usability of Internet-based information systems and without them the Web would be much less accessible, and at the very least would develop at a much slower rate. Given that non-English users now tend to make up the majority in this environment, our main objective is to analyze and evaluate the retrieval effectiveness of various indexing and search strategies based on test-collections written in four different languages: English, French, German, and Italian. Our second objective is to describe and evaluate various approaches that might be implemented in order to effectively access document collections written in another language. As a third objective, we will explore the underlying problems involved in searching document collections written in the four different languages, and we will suggest and evaluate different database merging strategies capable of providing the user with a single unique result list.

0.001153389 = product of:
  0.010380501 = sum of:
    0.010380501 = product of:
      0.020761002 = sum of:
        0.020761002 = weight(_text_:web in 570) [ClassicSimilarity], result of:
          0.020761002 = score(doc=570,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.21634221 = fieldWeight in 570, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.046875 = fieldNorm(doc=570)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

0.001153389 = product of:
  0.010380501 = sum of:
    0.010380501 = product of:
      0.020761002 = sum of:
        0.020761002 = weight(_text_:web in 640) [ClassicSimilarity], result of:
          0.020761002 = score(doc=640,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.21634221 = fieldWeight in 640, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.046875 = fieldNorm(doc=640)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Abstract

Involving users in early phases of software development has become a common strategy as it enables developers to consider user needs from the beginning. Once a system is in production, new opportunities to observe, evaluate and learn from users emerge as more information becomes available. Gathering information from users to continuously evaluate their behavior is a common practice for commercial software, while the Cranfield paradigm remains the preferred option for Information Retrieval (IR) and recommendation systems in the academic world. Here we introduce the Infrastructures for Living Labs STELLA project which aims to create an evaluation infrastructure allowing experimental systems to run along production web-based academic search systems with real users. STELLA combines user interactions and log files analyses to enable large-scale A/B experiments for academic search.

0.001106661 = product of:
  0.009959949 = sum of:
    0.009959949 = product of:
      0.019919898 = sum of:
        0.019919898 = weight(_text_:22 in 2339) [ClassicSimilarity], result of:
          0.019919898 = score(doc=2339,freq=2.0), product of:
            0.10297151 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.02940506 = queryNorm
            0.19345059 = fieldWeight in 2339, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.0390625 = fieldNorm(doc=2339)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Date

22. 9.1997 19:16:05

0.001106661 = product of:
  0.009959949 = sum of:
    0.009959949 = product of:
      0.019919898 = sum of:
        0.019919898 = weight(_text_:22 in 1184) [ClassicSimilarity], result of:
          0.019919898 = score(doc=1184,freq=2.0), product of:
            0.10297151 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.02940506 = queryNorm
            0.19345059 = fieldWeight in 1184, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.0390625 = fieldNorm(doc=1184)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Date

22. 9.1997 19:16:05

0.001106661 = product of:
  0.009959949 = sum of:
    0.009959949 = product of:
      0.019919898 = sum of:
        0.019919898 = weight(_text_:22 in 2026) [ClassicSimilarity], result of:
          0.019919898 = score(doc=2026,freq=2.0), product of:
            0.10297151 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.02940506 = queryNorm
            0.19345059 = fieldWeight in 2026, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.0390625 = fieldNorm(doc=2026)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Source

Information processing and management. 44(2008) no.1, S.22-38

0.001106661 = product of:
  0.009959949 = sum of:
    0.009959949 = product of:
      0.019919898 = sum of:
        0.019919898 = weight(_text_:22 in 4197) [ClassicSimilarity], result of:
          0.019919898 = score(doc=4197,freq=2.0), product of:
            0.10297151 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.02940506 = queryNorm
            0.19345059 = fieldWeight in 4197, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.0390625 = fieldNorm(doc=4197)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Date

22. 1.2011 14:20:56

0.001106661 = product of:
  0.009959949 = sum of:
    0.009959949 = product of:
      0.019919898 = sum of:
        0.019919898 = weight(_text_:22 in 4540) [ClassicSimilarity], result of:
          0.019919898 = score(doc=4540,freq=2.0), product of:
            0.10297151 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.02940506 = queryNorm
            0.19345059 = fieldWeight in 4540, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.0390625 = fieldNorm(doc=4540)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Date

12. 7.2011 18:29:22

0.001106661 = product of:
  0.009959949 = sum of:
    0.009959949 = product of:
      0.019919898 = sum of:
        0.019919898 = weight(_text_:22 in 1786) [ClassicSimilarity], result of:
          0.019919898 = score(doc=1786,freq=2.0), product of:
            0.10297151 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.02940506 = queryNorm
            0.19345059 = fieldWeight in 1786, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.0390625 = fieldNorm(doc=1786)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Date

6. 4.2015 19:31:22

0.001106661 = product of:
  0.009959949 = sum of:
    0.009959949 = product of:
      0.019919898 = sum of:
        0.019919898 = weight(_text_:22 in 5287) [ClassicSimilarity], result of:
          0.019919898 = score(doc=5287,freq=2.0), product of:
            0.10297151 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.02940506 = queryNorm
            0.19345059 = fieldWeight in 5287, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.0390625 = fieldNorm(doc=5287)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Date

20. 1.2015 18:30:22

9.611576E-4 = product of:
  0.008650418 = sum of:
    0.008650418 = product of:
      0.017300837 = sum of:
        0.017300837 = weight(_text_:web in 6690) [ClassicSimilarity], result of:
          0.017300837 = score(doc=6690,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.18028519 = fieldWeight in 6690, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.0390625 = fieldNorm(doc=6690)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Abstract

In assessing information retrieval systems, it is important to know not only the precision of the retrieved set, but also to compare the number of retrieved relevant items to the total number of relevant items. For large collections, such as the TREC test collections, or the World Wide Web, it is not possible to enumerate the entire set of relevant documents. If the retrieved documents are evaluated, a variant of the statistical "capture-recapture" method can be used to estimate the total number of relevant documents, providing the several retrieval systems used are sufficiently independent. We show that the underlying signal detection model supporting such an analysis can be extended in two ways. First, assuming that there are two distinct performance characteristics (corresponding to the chance of retrieving a relevant, and retrieving a given non-relevant document), we show that if there are three or more independent systems available it is possible to estimate the number of relevant documents without actually having to decide whether each individual document is relevant. We report applications of this 3-system method to the TREC data, leading to the conclusion that the independence assumptions are not satisfied. We then extend the model to a multi-system, multi-problem model, and show that it is possible to include statistical dependencies of all orders in the model, and determine the number of relevant documents for each of the problems in the set. Application to the TREC setting will be presented

9.611576E-4 = product of:
  0.008650418 = sum of:
    0.008650418 = product of:
      0.017300837 = sum of:
        0.017300837 = weight(_text_:web in 5231) [ClassicSimilarity], result of:
          0.017300837 = score(doc=5231,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.18028519 = fieldWeight in 5231, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.0390625 = fieldNorm(doc=5231)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Abstract

In this issue we begin with a paper where Eastman points out that conventional narrower queries (the use of conjunctions and phrases) in a web engine search will reduce returned number of hits but not necessarily increase precision in the top ranked documents in the return. Thus by precision anomalies Eastman means that search narrowing activity results in no precision change or a decrease in precision. Multiple queries with multiple engines were run by students for a three-year period and the formulation/engine combination was recorded as was the number of hits. Relevance was also recorded for the top ten and top twenty ranked retrievals. While narrower searches reduced total hits they did not usually improve precision. Initial high precision and poor query reformulation account for some of the results, as did Alta Vista's failure to use the ranking algorithm incorporated in its regular search in its advanced search feature. However, since the top listed returns often reoccurred in all formulations, it would seem that the ranking algorithms are doing a consistent job of practical precision ranking that is not improved by reformulation.

9.611576E-4 = product of:
  0.008650418 = sum of:
    0.008650418 = product of:
      0.017300837 = sum of:
        0.017300837 = weight(_text_:web in 2094) [ClassicSimilarity], result of:
          0.017300837 = score(doc=2094,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.18028519 = fieldWeight in 2094, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.0390625 = fieldNorm(doc=2094)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Abstract

Some of the most popular measures to evaluate information filtering systems are usually independent of the users because they are based in relevance judgments obtained from experts. On the other hand, the user-centred evaluation allows showing the different impressions that the users have perceived about the system running. This work is focused on discussing the problem of user-centred versus system-centred evaluation of a Web content personalization system where the personalization is based on a user model that stores long term (section, categories and keywords) and short term interests (adapted from user provided feedback). The user-centred evaluation is based on questionnaires filled in by the users before and after using the system and the system-centred evaluation is based on the comparison between ranking of documents, obtained from the application of a multi-tier selection process, and binary relevance judgments collected previously from real users. The user-centred and system-centred evaluations performed with 106 users during 14 working days have provided valuable data concerning the behaviour of the users with respect to issues such as document relevance or the relative importance attributed to different ways of personalization. The results obtained shows general satisfaction on both the personalization processes (selection, adaptation and presentation) and the system as a whole.

8.853288E-4 = product of:
  0.007967959 = sum of:
    0.007967959 = product of:
      0.015935918 = sum of:
        0.015935918 = weight(_text_:22 in 2752) [ClassicSimilarity], result of:
          0.015935918 = score(doc=2752,freq=2.0), product of:
            0.10297151 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.02940506 = queryNorm
            0.15476047 = fieldWeight in 2752, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.03125 = fieldNorm(doc=2752)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Date

22. 3.2009 18:48:28

8.323869E-4 = product of:
  0.007491482 = sum of:
    0.007491482 = product of:
      0.014982964 = sum of:
        0.014982964 = weight(_text_:web in 636) [ClassicSimilarity], result of:
          0.014982964 = score(doc=636,freq=6.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.15613155 = fieldWeight in 636, product of:
              2.4494898 = tf(freq=6.0), with freq of:
                6.0 = termFreq=6.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.01953125 = fieldNorm(doc=636)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Abstract

The Text REtrieval Conference (TREC), a yearly workshop hosted by the US government's National Institute of Standards and Technology, provides the infrastructure necessary for large-scale evaluation of text retrieval methodologies. With the goal of accelerating research in this area, TREC created the first large test collections of full-text documents and standardized retrieval evaluation. The impact has been significant; since TREC's beginning in 1992, retrieval effectiveness has approximately doubled. TREC has built a variety of large test collections, including collections for such specialized retrieval tasks as cross-language retrieval and retrieval of speech. Moreover, TREC has accelerated the transfer of research ideas into commercial systems, as demonstrated in the number of retrieval techniques developed in TREC that are now used in Web search engines. This book provides a comprehensive review of TREC research, summarizing the variety of TREC results, documenting the best practices in experimental information retrieval, and suggesting areas for further research. The first part of the book describes TREC's history, test collections, and retrieval methodology. Next, the book provides "track" reports -- describing the evaluations of specific tasks, including routing and filtering, interactive retrieval, and retrieving noisy text. The final part of the book offers perspectives on TREC from such participants as Microsoft Research, University of Massachusetts, Cornell University, University of Waterloo, City University of New York, and IBM. The book will be of interest to researchers in information retrieval and related technologies, including natural language processing.

Content

Enthält die Beiträge: 1. The Text REtrieval Conference - Ellen M. Voorhees and Donna K. Harman 2. The TREC Test Collections - Donna K. Harman 3. Retrieval System Evaluation - Chris Buckley and Ellen M. Voorhees 4. The TREC Ad Hoc Experiments - Donna K. Harman 5. Routing and Filtering - Stephen Robertson and Jamie Callan 6. The TREC Interactive Tracks: Putting the User into Search - Susan T. Dumais and Nicholas J. Belkin 7. Beyond English - Donna K. Harman 8. Retrieving Noisy Text - Ellen M. Voorhees and John S. Garofolo 9.The Very Large Collection and Web Tracks - David Hawking and Nick Craswell 10. Question Answering in TREC - Ellen M. Voorhees 11. The University of Massachusetts and a Dozen TRECs - James Allan, W. Bruce Croft and Jamie Callan 12. How Okapi Came to TREC - Stephen Robertson 13. The SMART Project at TREC - Chris Buckley 14. Ten Years of Ad Hoc Retrieval at TREC Using PIRCS - Kui-Lam Kwok 15. MultiText Experiments for TREC - Gordon V. Cormack, Charles L. A. Clarke, Christopher R. Palmer and Thomas R. Lynam 16. A Language-Modeling Approach to TREC - Djoerd Hiemstra and Wessel Kraaij 17. BM Research Activities at TREC - Eric W. Brown, David Carmel, Martin Franz, Abraham Ittycheriah, Tapas Kanungo, Yoelle Maarek, J. Scott McCarley, Robert L. Mack, John M. Prager, John R. Smith, Aya Soffer, Jason Y. Zien and Alan D. Marwick Epilogue: Metareflections on TREC - Karen Sparck Jones

Footnote

Rez. in: JASIST 58(2007) no.6, S.910-911 (J.L. Vicedo u. J. Gomez): "The Text REtrieval Conference (TREC) is a yearly workshop hosted by the U.S. government's National Institute of Standards and Technology (NIST) that fosters and supports research in information retrieval as well as speeding the transfer of technology between research labs and industry. Since 1992, TREC has provided the infrastructure necessary for large-scale evaluations of different text retrieval methodologies. TREC impact has been very important and its success has been mainly supported by its continuous adaptation to the emerging information retrieval needs. Not in vain, TREC has built evaluation benchmarks for more than 20 different retrieval problems such as Web retrieval, speech retrieval, or question-answering. The large and intense trajectory of annual TREC conferences has resulted in an immense bulk of documents reflecting the different eval uation and research efforts developed. This situation makes it difficult sometimes to observe clearly how research in information retrieval (IR) has evolved over the course of TREC. TREC: Experiment and Evaluation in Information Retrieval succeeds in organizing and condensing all this research into a manageable volume that describes TREC history and summarizes the main lessons learned. The book is organized into three parts. The first part is devoted to the description of TREC's origin and history, the test collections, and the evaluation methodology developed. The second part describes a selection of the major evaluation exercises (tracks), and the third part contains contributions from research groups that had a large and remarkable participation in TREC. Finally, Karen Spark Jones, one of the main promoters of research in IR, closes the book with an epilogue that analyzes the impact of TREC on this research field.

7.6892605E-4 = product of:
  0.0069203344 = sum of:
    0.0069203344 = product of:
      0.013840669 = sum of:
        0.013840669 = weight(_text_:web in 6519) [ClassicSimilarity], result of:
          0.013840669 = score(doc=6519,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.14422815 = fieldWeight in 6519, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.03125 = fieldNorm(doc=6519)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Abstract

Relevance has been a difficult concept to define, let alone measure. In this paper, a simple operational definition of relevance is proposed for a Web-based library catalog: whether or not during a search session the user saves, prints, mails, or downloads a citation. If one of those actions is performed, the session is considered relevant to the user. An analysis is presented illustrating the advantages and disadvantages of this definition. With this definition and good transaction logging, it is possible to ascertain the relevance of a session. This was done for 905,970 sessions conducted with the University of California's Melvyl online catalog. Next, a methodology was developed to try to predict the relevance of a session. A number of variables were defined that characterize a session, none of which used any demographic information about the user. The values of the variables were computed for the sessions. Principal components analysis was used to extract a new set of variables out of the original set. A stratified random sampling technique was used to form ten strata such that each new strata of 90,570 sessions contained the same proportion of relevant to nonrelevant sessions. Logistic regression was used to ascertain the regression coefficients for nine of the ten strata. Then, the coefficients were used to predict the relevance of the sessions in the missing strata. Overall, 17.85% of the sessions were determined to be relevant. The predicted number of relevant sessions for all ten strata was 11 %, a 6.85% difference. The authors believe that the methodology can be further refined and the prediction improved. This methodology could also have significant application in improving user searching and also in predicting electronic commerce buying decisions without the use of personal demographic data

7.6892605E-4 = product of:
  0.0069203344 = sum of:
    0.0069203344 = product of:
      0.013840669 = sum of:
        0.013840669 = weight(_text_:web in 2880) [ClassicSimilarity], result of:
          0.013840669 = score(doc=2880,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.14422815 = fieldWeight in 2880, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.03125 = fieldNorm(doc=2880)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Abstract

Purpose - The purpose of this paper is to report a study of how the test instrument of a simulated work task situation is used in empirical evaluations of interactive information retrieval (IIR) and reported in the research literature. In particular, the author is interested to learn whether the requirements of how to employ simulated work task situations are followed, and whether these requirements call for further highlighting and refinement. Design/methodology/approach - In order to study how simulated work task situations are used, the research literature in question is identified. This is done partly via citation analysis by use of Web of Science®, and partly by systematic search of online repositories. On this basis, 67 individual publications were identified and they constitute the sample of analysis. Findings - The analysis reveals a need for clarifications of how to use simulated work task situations in IIR evaluations. In particular, with respect to the design and creation of realistic simulated work task situations. There is a lack of tailoring of the simulated work task situations to the test participants. Likewise, the requirement to include the test participants' personal information needs is neglected. Further, there is a need to add and emphasise a requirement to depict the used simulated work task situations when reporting the IIR studies. Research limitations/implications - Insight about the use of simulated work task situations has implications for test design of IIR studies and hence the knowledge base generated on the basis of such studies. Originality/value - Simulated work task situations are widely used in IIR studies, and the present study is the first comprehensive study of the intended and unintended use of this test instrument since its introduction in the late 1990's. The paper addresses the need to carefully design and tailor simulated work task situations to suit the test participants in order to obtain the intended authentic and realistic IIR under study.

6.728103E-4 = product of:
  0.0060552927 = sum of:
    0.0060552927 = product of:
      0.012110585 = sum of:
        0.012110585 = weight(_text_:web in 5082) [ClassicSimilarity], result of:
          0.012110585 = score(doc=5082,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.12619963 = fieldWeight in 5082, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02734375 = fieldNorm(doc=5082)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Abstract

Text retrieval technology targets a problem that is all too familiar: finding relevant information in large stores of electronic documents. The problem is an old one, with the first research conference devoted to the subject held in 1958 [11]. Since then the problem has continued to grow as more information is created in electronic form and more people gain electronic access. The advent of the World Wide Web, where anyone can publish so everyone must search, is a graphic illustration of the need for effective retrieval technology. The Text REtrieval Conference (TREC) is a workshop series designed to build the infrastructure necessary for the large-scale evaluation of text retrieval technology, thereby accelerating its transfer into the commercial sector. The series is sponsored by the U.S. National Institute of Standards and Technology (NIST) and the U.S. Department of Defense. At the time of this writing, there have been twelve TREC workshops and preparations for the thirteenth workshop are under way. Participants in the workshops have been drawn from the academic, commercial, and government sectors, and have included representatives from more than twenty different countries. These collective efforts have accomplished a great deal: a variety of large test collections have been built for both traditional ad hoc retrieval and related tasks such as cross-language retrieval, speech retrieval, and question answering; retrieval effectiveness has approximately doubled; and many commercial retrieval systems now contain technology first developed in TREC.

4.805788E-4 = product of:
  0.004325209 = sum of:
    0.004325209 = product of:
      0.008650418 = sum of:
        0.008650418 = weight(_text_:web in 6299) [ClassicSimilarity], result of:
          0.008650418 = score(doc=6299,freq=2.0), product of:
            0.09596372 = queryWeight, product of:
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.02940506 = queryNorm
            0.09014259 = fieldWeight in 6299, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.2635105 = idf(docFreq=4597, maxDocs=44218)
              0.01953125 = fieldNorm(doc=6299)
      0.5 = coord(1/2)
  0.11111111 = coord(1/9)

Footnote

Rez. in: Machine translation review: 1999, no.10, S.26-27 (D. Lewis): "Cross Language Information Retrieval (CLIR) addresses the growing need to access large volumes of data across language boundaries. The typical requirement is for the user to input a free form query, usually a brief description of a topic, into a search or retrieval engine which returns a list, in ranked order, of documents or web pages that are relevant to the topic. The search engine matches the terms in the query to indexed terms, usually keywords previously derived from the target documents. Unlike monolingual information retrieval, CLIR requires query terms in one language to be matched to indexed terms in another. Matching can be done by bilingual dictionary lookup, full machine translation, or by applying statistical methods. A query's success is measured in terms of recall (how many potentially relevant target documents are found) and precision (what proportion of documents found are relevant). Issues in CLIR are how to translate query terms into index terms, how to eliminate alternative translations (e.g. to decide that French 'traitement' in a query means 'treatment' and not 'salary'), and how to rank or weight translation alternatives that are retained (e.g. how to order the French terms 'aventure', 'business', 'affaire', and 'liaison' as relevant translations of English 'affair'). Grefenstette provides a lucid and useful overview of the field and the problems. The volume brings together a number of experiments and projects in CLIR. Mark Davies (New Mexico State University) describes Recuerdo, a Spanish retrieval engine which reduces translation ambiguities by scanning indexes for parallel texts; it also uses either a bilingual dictionary or direct equivalents from a parallel corpus in order to compare results for queries on parallel texts. Lisa Ballesteros and Bruce Croft (University of Massachusetts) use a 'local feedback' technique which automatically enhances a query by adding extra terms to it both before and after translation; such terms can be derived from documents known to be relevant to the query.