Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
3,759,375
result(s) for
"Ranking"
Sort by:
The Power of Rankings: Quantifying the Effect of Rankings on Online Consumer Search and Purchase Decisions
Online search intermediaries, such as Amazon or Expedia, use rankings (ordered lists) to present third-party sellers’ products to consumers. These rankings decrease consumer search costs and increase the probability of a match with a seller, ultimately increasing consumer welfare. Constructing relevant rankings requires understanding their causal effect on consumer choices. However, this is challenging because rankings are endogenous: consumers pay more attention to highly ranked products, and intermediaries rank the most relevant products at the top. In this paper, I use the first data set with experimental variation in the ranking from a field experiment at Expedia to make three contributions. First, I identify the causal effect of rankings and show that they affect what consumers search, but conditional on search, do not affect purchases. Second, I quantify the effect of rankings using a sequential search model and find an average position effect of $1.92, which is lower than literature estimates obtained without experimental variation. I also use model predictions, data patterns, and a feature of the data set (opaque offers) to show rankings lower search costs, instead of affecting consumer expectations or utility. Finally, I show a utility-based ranking built on this model’s estimates benefits consumers and the search intermediary.
Data and the online appendix are available at
https://doi.org/10.1287/mksc.2017.1072
.
Journal Article
Learning and decision-making from rank data
The ubiquitous challenge of learning and decision-making from rank data arises in situations where intelligent systems collect preference and behavior data from humans, learn from the data, and then use the data to help humans make efficient, effective, and timely decisions. Often, such data are represented by rankings. This book surveys some recent progress toward addressing the challenge from the considerations of statistics, computation, and socio-economics. We will cover classical statistical models for rank data, including random utility models, distance-based models, and mixture models. We will discuss and compare classical and state-of-the-art algorithms, such as algorithms based on Minorize-Majorization (MM), Expectation-Maximization (EM), Generalized Method-of-Moments (GMM), rank breaking, and tensor decomposition. We will also introduce principled Bayesian preference elicitation frameworks for collecting rank data. Finally, we will examine socio-economic aspects of statistically desirable decision-making mechanisms, such as Bayesian estimators. This book can be useful in three ways: (1) for theoreticians in statistics and machine learning to better understand the considerations and caveats of learning from rank data, compared to learning from other types of data, especially cardinal data; (2) for practitioners to apply algorithms covered by the book for sampling, learning, and aggregation; and (3) as a textbook for graduate students or advanced undergraduate students to learn about the field. This book requires that the reader has basic knowledge in probability, statistics, and algorithms. Knowledge in social choice would also help but is not required.
Book
SPECTRAL METHOD AND REGULARIZED MLE ARE BOTH OPTIMAL FOR TOP-K RANKING
This paper is concerned with the problem of top-K ranking from pairwise comparisons. Given a collection of n items and a few pairwise comparisons across them, one wishes to identify the set of K items that receive the highest ranks. To tackle this problem, we adopt the logistic parametric model—the Bradley–Terry–Luce model, where each item is assigned a latent preference score, and where the outcome of each pairwise comparison depends solely on the relative scores of the two items involved. Recent works have made significant progress toward characterizing the performance (e.g., the mean square error for estimating the scores) of several classical methods, including the spectral method and the maximum likelihood estimator (MLE). However, where they stand regarding top-K ranking remains unsettled.
We demonstrate that under a natural random sampling model, the spectral method alone, or the regularized MLE alone, is minimax optimal in terms of the sample complexity—the number of paired comparisons needed to ensure exact top-K identification, for the fixed dynamic range regime. This is accomplished via optimal control of the entrywise error of the score estimates. We complement our theoretical studies by numerical experiments, confirming that both methods yield low entrywise errors for estimating the underlying scores. Our theory is established via a novel leave-one-out trick, which proves effective for analyzing both iterative and noniterative procedures. Along the way, we derive an elementary eigenvector perturbation bound for probability transition matrices, which parallels the Davis–Kahan sin Θ theorem for symmetric matrices. This also allows us to close the gap between the ℓ2 error upper bound for the spectral method and the minimax lower limit.
Journal Article
University rankings: Time to reconsider
University rankings offer some benefits but also come with significant drawbacks. While they can encourage healthy competition, they often lead to unethical practices and prioritize short- term gains over long-term educational purposes. Relying on biased metrics like citations and journal impact factors is a major flaw, potentially misrepresenting the true value of scholarly work. The foremost focus of universities should be on educating proficient students, advancing dependable knowledge, and addressing societal needs. Annual rankings based on one year's criteria and output prove impractical, as research outcomes and educational impact require more time to materialize. It is crucial to consider abandoning or reevaluating ranking systems to prevent biased, financially-driven approaches from causing harm. An internal assessment, gauging satisfaction levels within the university community and the quality of education provided, could offer a more effective approach to ranking universities. Acknowledging the negative impact of journal rankings took decades. It is imperative to avoid subjecting educational systems to similarly detrimental effects from university rankings. The most effective method for ranking universities is through an internal system that takes into account the satisfaction levels of university community members regarding their work conditions and overall institution, as well as whether students are acquiring the education and skills they seek.
Journal Article
A multifaceted graphical display, including treatment ranking, was developed to aid interpretation of network meta-analysis
Network meta-analysis (NMA) is becoming a popular statistical tool for analyzing a network of evidence comparing more than two interventions. A particular advantage of NMA over pairwise meta-analysis is its ability to simultaneously compare multiple interventions including comparisons not previously trialed together, permitting intervention hierarchies to be created. Our aim was to develop a novel graphical display to aid interpretation of NMA to clinicians and decision-makers that incorporates ranking of interventions.
Current literature was searched, scrutinized, and provided direction for developing the novel graphical display. Ranking results were often found to be misinterpreted when presented alone and, to aid interpretation and effective communication to inform optimal decision-making, need to be displayed alongside other important aspects of the analysis including the evidence networks and relative intervention effect estimates.
Two new ranking visualizations were developed—the ‘Litmus Rank-O-Gram’ and the ‘Radial SUCRA’ plot—and embedded within a novel multipanel graphical display programmed within the MetaInsight application, with user feedback gained.
This display was designed to improve the reporting, and facilitate a holistic understanding, of NMA results. We believe uptake of the display would lead to better understanding of complex results and improve future decision-making.
•The ability to conduct a network meta-analysis (NMA) and thus rank included interventions is a powerful tool; however, the presentation of rankings in particular can easily be misinterpreted.•Two new ranking visualizations have been developed—the ‘Litmus Rank-O-Gram’ and the ‘Radial SUCRA’ curve plot—to improve the presentation of ranking results.•The new visualizations have been embedded within a novel multipanel display, alongside presentation of evidence networks and relative effect estimates, programmed within MetaInsight, an interactive web-based application for NMA, with user feedback gained.•The multipanel display provides an all-encompassing graphic to encourage a holistic understanding of the NMA, in turn, improving the interpretation and dissemination of NMAs for clinicians and decision-makers.
Journal Article
A proposal for an operational methodology to assist the ranking-aggregation problem in manufacturing
Ranking aggregation is an ancient problem with some characteristic elements: a number of
experts
, who individually rank a set of
objects
according to a certain (subjective)
attribute
, and the need to aggregate the resulting
expert rankings
into a
collective judgment
. Although this problem is traditionally very popular in fields such as
social choice
,
psychometrics
, and
economics
, it can also have several interesting applications in
manufacturing
, e.g., for customer-oriented design, reliability engineering, production management, etc. Through a case study related to a cobot-assisted manual (dis)assembly, the paper illustrates an operational methodology and various useful tools that assist in tackling the problem practically, effectively, and with a critical mind. Some of the proposed tools allow to estimate the degree of
concordance
among experts, and the collective judgment’s
consistency
and
robustness
. The paper is aimed at scientists and practitioners in manufacturing.
Journal Article
Designing Ranking Systems for Hotels on Travel Search Engines by Mining User-Generated and Crowdsourced Content
User-generated content on social media platforms and product search engines is changing the way consumers shop for goods online. However, current product search engines fail to effectively leverage information created across diverse social media platforms. Moreover, current ranking algorithms in these product search engines tend to induce consumers to focus on one single product characteristic dimension (e.g., price, star rating). This approach largely ignores consumers' multidimensional preferences for products. In this paper, we propose to generate a ranking system that recommends products that provide, on average, the best value for the consumer's money. The key idea is that products that provide a higher surplus should be ranked higher on the screen in response to consumer queries. We use a unique data set of U.S. hotel reservations made over a three-month period through Travelocity, which we supplement with data from various social media sources using techniques from text mining, image classification, social geotagging, human annotations, and geomapping. We propose a random coefficient hybrid structural model, taking into consideration the two sources of consumer heterogeneity the different travel occasions and different hotel characteristics introduce. Based on the estimates from the model, we infer the economic impact of various location and service characteristics of hotels. We then propose a new hotel ranking system based on the average utility gain a consumer receives from staying in a particular hotel. By doing so, we can provide customers with the \"best-value\" hotels early on. Our user studies, using ranking comparisons from several thousand users, validate the superiority of our ranking system relative to existing systems on several travel search engines. On a broader note, this paper illustrates how social media can be mined and incorporated into a demand estimation model in order to generate a new ranking system in product search engines. We thus highlight the tight linkages between user behavior on social media and search engines. Our interdisciplinary approach provides several insights for using machine learning techniques in economics and marketing research.
Journal Article
Evaluating Journal Quality and the Association for Information Systems Senior Scholars' Journal Basket Via Bibliometric Measures: Do Expert Journal Assessments Add Value?
Information systems journal rankings and ratings help scholars focus their publishing efforts and are widely used surrogates for judging the quality of research. Over the years, numerous approaches have been used to rank IS journals, approaches such as citation metrics, school lists, acceptance rates, and expert assessments. However, the results of these approaches often conflict due to a host of validity concerns. In the current scientometric study, we make significant strides toward correcting for these limitations in the ranking of mainstream IS journals. We compare expert rankings to bibliometric measures such as the ISI Impact Factor™, the h-index, and social network analysis metrics. Among other findings, we conclude that bibliometric measures provide very similar results to expert-based methods in determining a tiered structure of IS journals, thereby suggesting that bibliometrics can be a complete, less expensive, and more efficient substitute for expert assessment. We also find strong support for seven of the eight journals in the Association for Information Systems Senior Scholars' \"basket\" of journals. A cluster analysis of our results indicates a twotiered separation in the quality of the highest quality IS journals—with MIS Quarterly, Information Systems Research, and Journal of Management Information Systems belonging, in that order, to the highest A+ tier. Journal quality metrics fit nicely into the sociology of science literature and can be useful in models that attempt to explain how knowledge disseminates through scientific communities.
Journal Article