Search (2 results, page 1 of 1)

0.040641725 = product of:
  0.08128345 = sum of:
    0.08128345 = sum of:
      0.050501406 = weight(_text_:i in 2117) [ClassicSimilarity], result of:
        0.050501406 = score(doc=2117,freq=4.0), product of:
          0.17138503 = queryWeight, product of:
            3.7717297 = idf(docFreq=2765, maxDocs=44218)
            0.045439374 = queryNorm
          0.29466638 = fieldWeight in 2117, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            3.7717297 = idf(docFreq=2765, maxDocs=44218)
            0.0390625 = fieldNorm(doc=2117)
      0.03078204 = weight(_text_:22 in 2117) [ClassicSimilarity], result of:
        0.03078204 = score(doc=2117,freq=2.0), product of:
          0.15912095 = queryWeight, product of:
            3.5018296 = idf(docFreq=3622, maxDocs=44218)
            0.045439374 = queryNorm
          0.19345059 = fieldWeight in 2117, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.5018296 = idf(docFreq=3622, maxDocs=44218)
            0.0390625 = fieldNorm(doc=2117)
  0.5 = coord(1/2)

Abstract

This paper presents an application of the model described in Part I to the evaluation of Web search engines by undergraduates. The study observed how 36 undergraduate used four major search engines to find information for their own individual problems and how they evaluated these engines based an actual interaction with the search engines. User evaluation was based an 16 performance measures representing five evaluation criteria: relevance, efficiency, utility, user satisfaction, and connectivity. Non-performance (user-related) measures were also applied. Each participant searched his/ her own topic an all four engines and provided satisfaction ratings for system features and interaction and reasons for satisfaction. Each also made relevance judgements of retrieved items in relation to his/her own information need and participated in post-search Interviews to provide reactions to the search results and overall performance. The study found significant differences in precision PR1 relative recall, user satisfaction with output display, time saving, value of search results, and overall performance among the four engines and also significant engine by discipline interactions an all these measures. In addition, the study found significant differences in user satisfaction with response time among four engines, and significant engine by discipline interaction in user satisfaction with search interface. None of the four search engines dominated in every aspect of the multidimensional evaluation. Content analysis of verbal data identified a number of user criteria and users evaluative comments based an these criteria. Results from both quantitative analysis and content analysis provide insight for system design and development, and useful feedback an strengths and weaknesses of search engines for system improvement

Date

24. 1.2004 18:27:22

Footnote

Teil I im selben Heft

0.012625352 = product of:
  0.025250703 = sum of:
    0.025250703 = product of:
      0.050501406 = sum of:
        0.050501406 = weight(_text_:i in 5164) [ClassicSimilarity], result of:
          0.050501406 = score(doc=5164,freq=4.0), product of:
            0.17138503 = queryWeight, product of:
              3.7717297 = idf(docFreq=2765, maxDocs=44218)
              0.045439374 = queryNorm
            0.29466638 = fieldWeight in 5164, product of:
              2.0 = tf(freq=4.0), with freq of:
                4.0 = termFreq=4.0
              3.7717297 = idf(docFreq=2765, maxDocs=44218)
              0.0390625 = fieldNorm(doc=5164)
      0.5 = coord(1/2)
  0.5 = coord(1/2)

Abstract

The project proposes and tests a comprehensive and systematic model of user evaluation of Web search engines. The project contains two parts. Part I describes the background and the model including a set of criteria and measures, and a method for implementation. It includes a literature review for two periods. The early period (1995-1996) portrays the settings for developing the model and the later period (1997-2000) places two applications of the model among contemporary evaluation work. Part II presents one of the applications that investigated the evaluation of four major search engines by 36 undergraduates from three academic disciplines. It reports results from statistical analyses of quantitative data for the entire sample and among disciplines, and content analysis of verbal data containing users' reasons for satisfaction. The proposed model aims to provide systematic feedback to engine developers or service providers for system improvement and to generate useful insight for system design and tool choice. The model can be applied to evaluating other compatible information retrieval systems or information retrieval (IR) techniques. It intends to contribute to developing a theory of relevance that goes beyond topicality to include value and usefulness for designing user-oriented information retrieval systems.