Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
3,806,031
result(s) for
"Ranking"
Sort by:
A critical comparative analysis of five world university rankings
To provide users insight into the value and limits of world university rankings, a comparative analysis is conducted of five ranking systems: ARWU, Leiden, THE, QS and U-Multirank. It links these systems with one another at the level of individual institutions, and analyses the overlap in institutional coverage, geographical coverage, how indicators are calculated from raw data, the skewness of indicator distributions, and statistical correlations between indicators. Four secondary analyses are presented investigating national academic systems and selected pairs of indicators. It is argued that current systems are still one-dimensional in the sense that they provide finalized, seemingly unrelated indicator values rather than offering a dataset and tools to observe patterns in multi-faceted data. By systematically comparing different systems, more insight is provided into how their institutional coverage, rating methods, the selection of indicators and their normalizations influence the ranking positions of given institutions.
Journal Article
Learning and decision-making from rank data
The ubiquitous challenge of learning and decision-making from rank data arises in situations where intelligent systems collect preference and behavior data from humans, learn from the data, and then use the data to help humans make efficient, effective, and timely decisions. Often, such data are represented by rankings. This book surveys some recent progress toward addressing the challenge from the considerations of statistics, computation, and socio-economics. We will cover classical statistical models for rank data, including random utility models, distance-based models, and mixture models. We will discuss and compare classical and state-of-the-art algorithms, such as algorithms based on Minorize-Majorization (MM), Expectation-Maximization (EM), Generalized Method-of-Moments (GMM), rank breaking, and tensor decomposition. We will also introduce principled Bayesian preference elicitation frameworks for collecting rank data. Finally, we will examine socio-economic aspects of statistically desirable decision-making mechanisms, such as Bayesian estimators. This book can be useful in three ways: (1) for theoreticians in statistics and machine learning to better understand the considerations and caveats of learning from rank data, compared to learning from other types of data, especially cardinal data; (2) for practitioners to apply algorithms covered by the book for sampling, learning, and aggregation; and (3) as a textbook for graduate students or advanced undergraduate students to learn about the field. This book requires that the reader has basic knowledge in probability, statistics, and algorithms. Knowledge in social choice would also help but is not required.
Book
Comparing university rankings
Recently there is increasing interest in university rankings. Annual rankings of world universities are published by QS for the Times Higher Education Supplement, the Shanghai Jiao Tong University, the Higher Education and Accreditation Council of Taiwan and rankings based on Web visibility by the Cybermetrics Lab at CSIC. In this paper we compare the rankings using a set of similarity measures. For the rankings that are being published for a number of years we also examine longitudinal patterns. The rankings limited to European universities are compared to the ranking of the Centre for Science and Technology Studies at Leiden University. The findings show that there are reasonable similarities between the rankings, even though each applies a different methodology. The biggest differences are between the rankings provided by the QS-Times Higher Education Supplement and the Ranking Web of the CSIC Cybermetrics Lab. The highest similarities were observed between the Taiwanese and the Leiden rankings from European universities. Overall the similarities are increased when the comparison is limited to the European universities.
Journal Article
The Power of Rankings: Quantifying the Effect of Rankings on Online Consumer Search and Purchase Decisions
Online search intermediaries, such as Amazon or Expedia, use rankings (ordered lists) to present third-party sellers’ products to consumers. These rankings decrease consumer search costs and increase the probability of a match with a seller, ultimately increasing consumer welfare. Constructing relevant rankings requires understanding their causal effect on consumer choices. However, this is challenging because rankings are endogenous: consumers pay more attention to highly ranked products, and intermediaries rank the most relevant products at the top. In this paper, I use the first data set with experimental variation in the ranking from a field experiment at Expedia to make three contributions. First, I identify the causal effect of rankings and show that they affect what consumers search, but conditional on search, do not affect purchases. Second, I quantify the effect of rankings using a sequential search model and find an average position effect of $1.92, which is lower than literature estimates obtained without experimental variation. I also use model predictions, data patterns, and a feature of the data set (opaque offers) to show rankings lower search costs, instead of affecting consumer expectations or utility. Finally, I show a utility-based ranking built on this model’s estimates benefits consumers and the search intermediary.
Data and the online appendix are available at
https://doi.org/10.1287/mksc.2017.1072
.
Journal Article
SPECTRAL METHOD AND REGULARIZED MLE ARE BOTH OPTIMAL FOR TOP-K RANKING
This paper is concerned with the problem of top-K ranking from pairwise comparisons. Given a collection of n items and a few pairwise comparisons across them, one wishes to identify the set of K items that receive the highest ranks. To tackle this problem, we adopt the logistic parametric model—the Bradley–Terry–Luce model, where each item is assigned a latent preference score, and where the outcome of each pairwise comparison depends solely on the relative scores of the two items involved. Recent works have made significant progress toward characterizing the performance (e.g., the mean square error for estimating the scores) of several classical methods, including the spectral method and the maximum likelihood estimator (MLE). However, where they stand regarding top-K ranking remains unsettled.
We demonstrate that under a natural random sampling model, the spectral method alone, or the regularized MLE alone, is minimax optimal in terms of the sample complexity—the number of paired comparisons needed to ensure exact top-K identification, for the fixed dynamic range regime. This is accomplished via optimal control of the entrywise error of the score estimates. We complement our theoretical studies by numerical experiments, confirming that both methods yield low entrywise errors for estimating the underlying scores. Our theory is established via a novel leave-one-out trick, which proves effective for analyzing both iterative and noniterative procedures. Along the way, we derive an elementary eigenvector perturbation bound for probability transition matrices, which parallels the Davis–Kahan sin Θ theorem for symmetric matrices. This also allows us to close the gap between the ℓ2 error upper bound for the spectral method and the minimax lower limit.
Journal Article
On the credibility of QS and THE ranking by subject area: misalignment of subject mapping to academic disciplines
In this study, we point the attention to some inaccuracies in the mapping between the journal subject classification by Elsevier and the narrow subject field used by QS (Quacquarelli Symonds) and THE (Times Higher Education) in their World University Rankings by subject. We noticed that inaccuracies in this mapping will result in classifying some publications under far disciplines, rendering the announced subject ranking inaccurate. To give an example of these inaccuracies, publications on fuel technology, nuclear engineering, and all energy-related studies are classified under Civil Engineering in THE ranking and under Electrical and Electronics Engineering under the QS ranking. This is completely unfair as many of these studies are conducted by researchers in Mechanical or Chemical engineering disciplines. To demonstrate the effect of this erroneous mapping on the final ranking, we obtained the publications data for 13 institutions from the top 20 institutions in the Arab World from 2017 to 021 and their citations until mid-2022 as indexed in Scopus. Following QS and THE subject ranking methodology, we then re-ranked these institutions based on citations per paper and h-index indicators based on a modified subject mapping suggested by a sample of 12 faculty members from 6 different engineering departments at the authors’ institution. We found that the new ranking differed considerably from the one calculated by QS and THE based on their controversial subject mapping. Many institutions (sometimes 10 out of 13) had their rank change in some subject areas with the rank of some institutions dropping 6 ranks out of 13 in some cases! We believe this study sheds light on the inaccuracies in subject rankings and the importance of coming up with a unified subject mapping to be used by the different ranking bodies.
Journal Article
A multifaceted graphical display, including treatment ranking, was developed to aid interpretation of network meta-analysis
Network meta-analysis (NMA) is becoming a popular statistical tool for analyzing a network of evidence comparing more than two interventions. A particular advantage of NMA over pairwise meta-analysis is its ability to simultaneously compare multiple interventions including comparisons not previously trialed together, permitting intervention hierarchies to be created. Our aim was to develop a novel graphical display to aid interpretation of NMA to clinicians and decision-makers that incorporates ranking of interventions.
Current literature was searched, scrutinized, and provided direction for developing the novel graphical display. Ranking results were often found to be misinterpreted when presented alone and, to aid interpretation and effective communication to inform optimal decision-making, need to be displayed alongside other important aspects of the analysis including the evidence networks and relative intervention effect estimates.
Two new ranking visualizations were developed—the ‘Litmus Rank-O-Gram’ and the ‘Radial SUCRA’ plot—and embedded within a novel multipanel graphical display programmed within the MetaInsight application, with user feedback gained.
This display was designed to improve the reporting, and facilitate a holistic understanding, of NMA results. We believe uptake of the display would lead to better understanding of complex results and improve future decision-making.
•The ability to conduct a network meta-analysis (NMA) and thus rank included interventions is a powerful tool; however, the presentation of rankings in particular can easily be misinterpreted.•Two new ranking visualizations have been developed—the ‘Litmus Rank-O-Gram’ and the ‘Radial SUCRA’ curve plot—to improve the presentation of ranking results.•The new visualizations have been embedded within a novel multipanel display, alongside presentation of evidence networks and relative effect estimates, programmed within MetaInsight, an interactive web-based application for NMA, with user feedback gained.•The multipanel display provides an all-encompassing graphic to encourage a holistic understanding of the NMA, in turn, improving the interpretation and dissemination of NMAs for clinicians and decision-makers.
Journal Article
A proposal for an operational methodology to assist the ranking-aggregation problem in manufacturing
Ranking aggregation is an ancient problem with some characteristic elements: a number of
experts
, who individually rank a set of
objects
according to a certain (subjective)
attribute
, and the need to aggregate the resulting
expert rankings
into a
collective judgment
. Although this problem is traditionally very popular in fields such as
social choice
,
psychometrics
, and
economics
, it can also have several interesting applications in
manufacturing
, e.g., for customer-oriented design, reliability engineering, production management, etc. Through a case study related to a cobot-assisted manual (dis)assembly, the paper illustrates an operational methodology and various useful tools that assist in tackling the problem practically, effectively, and with a critical mind. Some of the proposed tools allow to estimate the degree of
concordance
among experts, and the collective judgment’s
consistency
and
robustness
. The paper is aimed at scientists and practitioners in manufacturing.
Journal Article
University rankings: Time to reconsider
University rankings offer some benefits but also come with significant drawbacks. While they can encourage healthy competition, they often lead to unethical practices and prioritize short- term gains over long-term educational purposes. Relying on biased metrics like citations and journal impact factors is a major flaw, potentially misrepresenting the true value of scholarly work. The foremost focus of universities should be on educating proficient students, advancing dependable knowledge, and addressing societal needs. Annual rankings based on one year's criteria and output prove impractical, as research outcomes and educational impact require more time to materialize. It is crucial to consider abandoning or reevaluating ranking systems to prevent biased, financially-driven approaches from causing harm. An internal assessment, gauging satisfaction levels within the university community and the quality of education provided, could offer a more effective approach to ranking universities. Acknowledging the negative impact of journal rankings took decades. It is imperative to avoid subjecting educational systems to similarly detrimental effects from university rankings. The most effective method for ranking universities is through an internal system that takes into account the satisfaction levels of university community members regarding their work conditions and overall institution, as well as whether students are acquiring the education and skills they seek.
Journal Article