Search (6 results, page 1 of 1)

0.10061574 = product of:
  0.13415432 = sum of:
    0.029088326 = weight(_text_:web in 3593) [ClassicSimilarity], result of:
      0.029088326 = score(doc=3593,freq=2.0), product of:
        0.16134618 = queryWeight, product of:
          3.2635105 = idf(docFreq=4597, maxDocs=44218)
          0.049439456 = queryNorm
        0.18028519 = fieldWeight in 3593, product of:
          1.4142135 = tf(freq=2.0), with freq of:
            2.0 = termFreq=2.0
          3.2635105 = idf(docFreq=4597, maxDocs=44218)
          0.0390625 = fieldNorm(doc=3593)
    0.06598687 = weight(_text_:search in 3593) [ClassicSimilarity], result of:
      0.06598687 = score(doc=3593,freq=8.0), product of:
        0.17183559 = queryWeight, product of:
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.049439456 = queryNorm
        0.3840117 = fieldWeight in 3593, product of:
          2.828427 = tf(freq=8.0), with freq of:
            8.0 = termFreq=8.0
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.0390625 = fieldNorm(doc=3593)
    0.03907912 = product of:
      0.07815824 = sum of:
        0.07815824 = weight(_text_:engine in 3593) [ClassicSimilarity], result of:
          0.07815824 = score(doc=3593,freq=2.0), product of:
            0.26447627 = queryWeight, product of:
              5.349498 = idf(docFreq=570, maxDocs=44218)
              0.049439456 = queryNorm
            0.29552078 = fieldWeight in 3593, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              5.349498 = idf(docFreq=570, maxDocs=44218)
              0.0390625 = fieldNorm(doc=3593)
      0.5 = coord(1/2)
  0.75 = coord(3/4)

Abstract

We present the first systematic study of the influence of time on user judgements for rankings and relevance grades of web search engine results. The goal of this study is to evaluate the change in user assessment of search results and explore how users' judgements change. To this end, we conducted a large-scale user study with 86 participants who evaluated 2 different queries and 4 diverse result sets twice with an interval of 2 months. To analyze the results we investigate whether 2 types of patterns of user behavior from the theory of categorical thinking hold for the case of evaluation of search results: (a) coarseness and (b) locality. To quantify these patterns we devised 2 new measures of change in user judgements and distinguish between local (when users swap between close ranks and relevance values) and nonlocal changes. Two types of judgements were considered in this study: (a) relevance on a 4-point scale, and (b) ranking on a 10-point scale without ties. We found that users tend to change their judgements of the results over time in about 50% of cases for relevance and in 85% of cases for ranking. However, the majority of these changes were local.

0.097201824 = product of:
  0.19440365 = sum of:
    0.07918424 = weight(_text_:search in 3071) [ClassicSimilarity], result of:
      0.07918424 = score(doc=3071,freq=18.0), product of:
        0.17183559 = queryWeight, product of:
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.049439456 = queryNorm
        0.460814 = fieldWeight in 3071, product of:
          4.2426405 = tf(freq=18.0), with freq of:
            18.0 = termFreq=18.0
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.03125 = fieldNorm(doc=3071)
    0.1152194 = sum of:
      0.08842595 = weight(_text_:engine in 3071) [ClassicSimilarity], result of:
        0.08842595 = score(doc=3071,freq=4.0), product of:
          0.26447627 = queryWeight, product of:
            5.349498 = idf(docFreq=570, maxDocs=44218)
            0.049439456 = queryNorm
          0.3343436 = fieldWeight in 3071, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            5.349498 = idf(docFreq=570, maxDocs=44218)
            0.03125 = fieldNorm(doc=3071)
      0.026793454 = weight(_text_:22 in 3071) [ClassicSimilarity], result of:
        0.026793454 = score(doc=3071,freq=2.0), product of:
          0.17312855 = queryWeight, product of:
            3.5018296 = idf(docFreq=3622, maxDocs=44218)
            0.049439456 = queryNorm
          0.15476047 = fieldWeight in 3071, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.5018296 = idf(docFreq=3622, maxDocs=44218)
            0.03125 = fieldNorm(doc=3071)
  0.5 = coord(2/4)

Abstract

Purpose - One of the under-explored aspects in the process of user information seeking behaviour is influence of time on relevance evaluation. It has been shown in previous studies that individual users might change their assessment of search results over time. It is also known that aggregated judgements of multiple individual users can lead to correct and reliable decisions; this phenomenon is known as the "wisdom of crowds". The purpose of this paper is to examine whether aggregated judgements will be more stable and thus more reliable over time than individual user judgements. Design/methodology/approach - In this study two simple measures are proposed to calculate the aggregated judgements of search results and compare their reliability and stability to individual user judgements. In addition, the aggregated "wisdom of crowds" judgements were used as a means to compare the differences between human assessments of search results and search engine's rankings. A large-scale user study was conducted with 87 participants who evaluated two different queries and four diverse result sets twice, with an interval of two months. Two types of judgements were considered in this study: relevance on a four-point scale, and ranking on a ten-point scale without ties. Findings - It was found that aggregated judgements are much more stable than individual user judgements, yet they are quite different from search engine rankings. Practical implications - The proposed "wisdom of crowds"-based approach provides a reliable reference point for the evaluation of search engines. This is also important for exploring the need of personalisation and adapting search engine's ranking over time to changes in users preferences. Originality/value - This is a first study that applies the notion of "wisdom of crowds" to examine an under-explored in the literature phenomenon of "change in time" in user evaluation of relevance.

Date

20. 1.2015 18:30:22

0.07432193 = product of:
  0.09909591 = sum of:
    0.032909684 = weight(_text_:web in 1634) [ClassicSimilarity], result of:
      0.032909684 = score(doc=1634,freq=4.0), product of:
        0.16134618 = queryWeight, product of:
          3.2635105 = idf(docFreq=4597, maxDocs=44218)
          0.049439456 = queryNorm
        0.2039694 = fieldWeight in 1634, product of:
          2.0 = tf(freq=4.0), with freq of:
            4.0 = termFreq=4.0
          3.2635105 = idf(docFreq=4597, maxDocs=44218)
          0.03125 = fieldNorm(doc=1634)
    0.052789498 = weight(_text_:search in 1634) [ClassicSimilarity], result of:
      0.052789498 = score(doc=1634,freq=8.0), product of:
        0.17183559 = queryWeight, product of:
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.049439456 = queryNorm
        0.30720934 = fieldWeight in 1634, product of:
          2.828427 = tf(freq=8.0), with freq of:
            8.0 = termFreq=8.0
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.03125 = fieldNorm(doc=1634)
    0.013396727 = product of:
      0.026793454 = sum of:
        0.026793454 = weight(_text_:22 in 1634) [ClassicSimilarity], result of:
          0.026793454 = score(doc=1634,freq=2.0), product of:
            0.17312855 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.049439456 = queryNorm
            0.15476047 = fieldWeight in 1634, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.03125 = fieldNorm(doc=1634)
      0.5 = coord(1/2)
  0.75 = coord(3/4)

Abstract

Purpose - Ontologies are prone to wide semantic variability due to subjective points of view of their composers. The purpose of this paper is to propose a new approach for maximal unification of diverse ontologies for controversial domains by their relations. Design/methodology/approach - Effective matching or unification of multiple ontologies for a specific domain is crucial for the success of many semantic web applications, such as semantic information retrieval and organization, document tagging, summarization and search. To this end, numerous automatic and semi-automatic techniques were proposed in the past decade that attempt to identify similar entities, mostly classes, in diverse ontologies for similar domains. Apparently, matching individual entities cannot result in full integration of ontologies' semantics without matching their inter-relations with all other-related classes (and instances). However, semantic matching of ontological relations still constitutes a major research challenge. Therefore, in this paper the authors propose a new paradigm for assessment of maximal possible matching and unification of ontological relations. To this end, several unification rules for ontological relations were devised based on ontological reference rules, and lexical and textual entailment. These rules were semi-automatically implemented to extend a given ontology with semantically matching relations from another ontology for a similar domain. Then, the ontologies were unified through these similar pairs of relations. The authors observe that these rules can be also facilitated to reveal the contradictory relations in different ontologies. Findings - To assess the feasibility of the approach two experiments were conducted with different sets of multiple personal ontologies on controversial domains constructed by trained subjects. The results for about 50 distinct ontology pairs demonstrate a good potential of the methodology for increasing inter-ontology agreement. Furthermore, the authors show that the presented methodology can lead to a complete unification of multiple semantically heterogeneous ontologies. Research limitations/implications - This is a conceptual study that presents a new approach for semantic unification of ontologies by a devised set of rules along with the initial experimental evidence of its feasibility and effectiveness. However, this methodology has to be fully automatically implemented and tested on a larger dataset in future research. Practical implications - This result has implication for semantic search, since a richer ontology, comprised of multiple aspects and viewpoints of the domain of knowledge, enhances discoverability and improves search results. Originality/value - To the best of the knowledge, this is the first study to examine and assess the maximal level of semantic relation-based ontology unification.

Date

20. 1.2015 18:30:22

Series

Special issue: Semantic search

Theme

Semantic Web

0.06982846 = product of:
  0.13965692 = sum of:
    0.03490599 = weight(_text_:web in 3320) [ClassicSimilarity], result of:
      0.03490599 = score(doc=3320,freq=2.0), product of:
        0.16134618 = queryWeight, product of:
          3.2635105 = idf(docFreq=4597, maxDocs=44218)
          0.049439456 = queryNorm
        0.21634221 = fieldWeight in 3320, product of:
          1.4142135 = tf(freq=2.0), with freq of:
            2.0 = termFreq=2.0
          3.2635105 = idf(docFreq=4597, maxDocs=44218)
          0.046875 = fieldNorm(doc=3320)
    0.104750924 = weight(_text_:search in 3320) [ClassicSimilarity], result of:
      0.104750924 = score(doc=3320,freq=14.0), product of:
        0.17183559 = queryWeight, product of:
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.049439456 = queryNorm
        0.6095997 = fieldWeight in 3320, product of:
          3.7416575 = tf(freq=14.0), with freq of:
            14.0 = termFreq=14.0
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.046875 = fieldNorm(doc=3320)
  0.5 = coord(2/4)

Abstract

The goal of this research is to develop a generic ontological model for proverbs that unifies potential classification criteria and various characteristics of proverbs to enable their effective retrieval and large-scale analysis. Because proverbs can be described and indexed by multiple characteristics and criteria, we built a multidimensional ontology suitable for proverb classification. To evaluate the effectiveness of the constructed ontology for improving search and retrieval of proverbs, a large-scale user experiment was arranged with 70 users who were asked to search a proverb repository using ontology-based and free-text search interfaces. The comparative analysis of the results shows that the use of this ontology helped to substantially improve the search recall, precision, user satisfaction, and efficiency and to minimize user effort during the search process. A practical contribution of this work is an automated web-based proverb search and retrieval system which incorporates the proposed ontological scheme and an initial corpus of ontology-based annotated proverbs.

0.052532993 = product of:
  0.10506599 = sum of:
    0.06598687 = weight(_text_:search in 4457) [ClassicSimilarity], result of:
      0.06598687 = score(doc=4457,freq=8.0), product of:
        0.17183559 = queryWeight, product of:
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.049439456 = queryNorm
        0.3840117 = fieldWeight in 4457, product of:
          2.828427 = tf(freq=8.0), with freq of:
            8.0 = termFreq=8.0
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.0390625 = fieldNorm(doc=4457)
    0.03907912 = product of:
      0.07815824 = sum of:
        0.07815824 = weight(_text_:engine in 4457) [ClassicSimilarity], result of:
          0.07815824 = score(doc=4457,freq=2.0), product of:
            0.26447627 = queryWeight, product of:
              5.349498 = idf(docFreq=570, maxDocs=44218)
              0.049439456 = queryNorm
            0.29552078 = fieldWeight in 4457, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              5.349498 = idf(docFreq=570, maxDocs=44218)
              0.0390625 = fieldNorm(doc=4457)
      0.5 = coord(1/2)
  0.5 = coord(2/4)

Abstract

In this study we aim to explore users' behavior when assessing search results relevance based on the hypothesis of categorical thinking. To investigate how users categories search engine results, we perform several experiments where users are asked to group a list of 20 search results into several categories, while attaching a relevance judgment to each formed category. Moreover, to determine how users change their minds over time, each experiment was repeated three times under the same conditions, with a gap of one month between rounds. The results show that on average users form 4-5 categories. Within each round the size of a category decreases with the relevance of a category. To measure the agreement between the search engine's ranking and the users' relevance judgments, we defined two novel similarity measures, the average concordance and the MinMax swap ratio. Similarity is shown to be the highest for the third round as the users' opinion stabilizes. Qualitative analysis uncovered some interesting points that users tended to categories results by type and reliability of their source, and particularly, found commercial sites less trustworthy, and attached high relevance to Wikipedia when their prior domain knowledge was limited.

0.008248359 = product of:
  0.032993436 = sum of:
    0.032993436 = weight(_text_:search in 5459) [ClassicSimilarity], result of:
      0.032993436 = score(doc=5459,freq=2.0), product of:
        0.17183559 = queryWeight, product of:
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.049439456 = queryNorm
        0.19200584 = fieldWeight in 5459, product of:
          1.4142135 = tf(freq=2.0), with freq of:
            2.0 = termFreq=2.0
          3.475677 = idf(docFreq=3718, maxDocs=44218)
          0.0390625 = fieldNorm(doc=5459)
  0.25 = coord(1/4)

Abstract

Purpose The need for inclusive and logically consistent representation of diverse and even confronting viewpoints on the domain knowledge has been widely discussed in the literature in the past decade. The purpose of this paper is to propose a generic model for building an open coherent diversified knowledge organization system (KOS). Design/methodology/approach The proposed model incorporates a generic epistemological component, the validity scope type, assigned to each statement in the constructed KOS. Statements are clustered by their association with various validity scope types into internally coherent subsystems. These subsystems form a knowledge organization network connected through the universal (consensual) subsystems with more than one validity scope type. The model extends the Galili's Cultural Content Representation paradigm, which divides the knowledge content of a scientific theory into two confronting parts: body and periphery. Findings The knowledge organization network model makes it possible to comparatively examine similarities and differences among various viewpoints and theories on the domain knowledge. The presented approach conforms with the principle of Open Knowledge Network initiative for creation of open accessible knowledge. Practical implications The proposed model can be used for ontological reasoning by a variety of information services, such as ontology-based decision-support and learning systems, diversified search and customer management applications. Social implications The model enables explicit representation of social and cultural minority voices and historical knowledge in the KOS. Originality/value The main contribution of the proposed model is that it generalizes and enhances various previously proposed representations of epistemological aspects of KOS and allows for multiple inter-linked subsystems to coherently co-exist as part of the extensible network.