Explore the vast range of titles available.

Peer Review is at the heart of scholarly communications and the cornerstone of scientific publishing. However, academia often criticizes the peer review system as non-transparent, biased, arbitrary, a flawed process at the heart of science , leading to researchers arguing with its reliability and quality. These problems could also be due to the lack of studies with the peer-review texts for various proprietary and confidentiality clauses. Peer review texts could serve as a rich source of Natural Language Processing (NLP) research on understanding the scholarly communication landscape, and thereby build systems towards mitigating those pertinent problems. In this work, we present a first of its kind multi-layered dataset of 1199 open peer review texts manually annotated at the sentence level (∼ 17k sentences) across the four layers, viz. Paper Section Correspondence, Paper Aspect Category, Review Functionality, and Review Significance. Given a text written by the reviewer, we annotate: to which sections (e.g., Methodology, Experiments, etc.), what aspects (e.g., Originality/Novelty, Empirical/Theoretical Soundness, etc.) of the paper does the review text correspond to, what is the role played by the review text (e.g., appreciation, criticism, summary, etc.), and the importance of the review statement (major, minor, general) within the review. We also annotate the sentiment of the reviewer (positive, negative, neutral) for the first two layers to judge the reviewer’s perspective on the different sections and aspects of the paper. We further introduce four novel tasks with this dataset, which could serve as an indicator of the exhaustiveness of a peer review and can be a step towards the automatic judgment of review quality . We also present baseline experiments and results for the different tasks for further investigations. We believe our dataset would provide a benchmark experimental testbed for automated systems to leverage on current NLP state-of-the-art techniques to address different issues with peer review quality , thereby ushering increased transparency and trust on the holy grail of scientific research validation . Our dataset and associated codes are available at https://www.iitp.ac.in/~ai-nlp-ml/resources.html#Peer-Review-Analyze .

AstroSage-Llama-3.1-8B is a domain-specialized natural-language AI assistant tailored for research in astronomy, astrophysics, cosmology, and astronomical instrumentation. Trained on the complete collection of astronomy-related arXiv papers from 2007 to 2024 along with millions of synthetically-generated question-answer pairs and other astronomical literature, AstroSage-Llama-3.1-8B demonstrates remarkable proficiency on a wide range of questions. AstroSage-Llama-3.1-8B scores 80.9% on the AstroMLab-1 benchmark, greatly outperforming all models—proprietary and open-weight—in the 8-billion parameter class, and performing on par with GPT-4o. This achievement demonstrates the potential of domain specialization in AI, suggesting that focused training can yield capabilities exceeding those of much larger, general-purpose models. AstroSage-Llama-3.1-8B is freely available, enabling widespread access to advanced AI capabilities for astronomical education and research.

Finding the lineage of a research topic is crucial for understanding the prior state of the art and advancing scientific displacement. The deluge of scholarly articles makes it difficult to locate the most relevant previous work. It causes researchers to spend a considerable amount of time building up their literature list. Citations play a crucial role in discovering relevant literature. However, not all citations are created equal. The majority of the citations that a paper receives provide contextual and background information to the citing papers. In those cases, the cited paper is not central to the theme of citing papers. However, some papers build upon a given paper and further the research frontier. In those cases, the concerned cited paper plays a pivotal role in the citing paper. Hence, the nature of the citation that the former receives from the latter is . In this work, we discuss our investigations towards discovering of a given paper. We further show how we can leverage significant citations to build a research lineage via a . We demonstrate the efficacy of our idea with two real-life case studies. Our experiments yield promising results with respect to the current state of the art in classifying , outperforming the earlier ones by a relative margin of 20 points in terms of precision. We hypothesize that such an automated system can facilitate relevant literature discovery and help identify knowledge flow for a particular category of papers.

Machine Reading Comprehension (MRC) of a document is a challenging problem that requires discourse-level understanding. Information extraction from scholarly articles nowadays is a critical use case for researchers to understand the underlying research quickly and move forward, especially in this age of infodemic. MRC on research articles can also provide helpful information to the reviewers and editors. However, the main bottleneck in building such models is the availability of human-annotated data. In this paper, firstly, we introduce a dataset to facilitate question answering (QA) on scientific articles. We prepare the dataset in a semi-automated fashion having more than 100k human-annotated context–question–answer triples. Secondly, we implement one baseline QA model based on Bidirectional Encoder Representations from Transformers (BERT). Additionally, we implement two models: the first one is based on Science BERT (SciBERT), and the second is the combination of SciBERT and Bi-Directional Attention Flow (Bi-DAF). The best model (i.e., SciBERT) obtains an F1 score of 75.46%. Our dataset is novel, and our work opens up a new avenue for scholarly document processing research by providing a benchmark QA dataset and standard baseline. We make our dataset and codes available here at https://github.com/TanikSaikh/Scientific-Question-Answering.

Citations are crucial artifacts to provide additional information to the reader to comprehend the research under concern. There are different roles that citations play in scientific discourse. Correctly identifying the intent of the citations finds applications ranging from predicting scholarly impact, finding idea propagation, to text summarization. With the rapid growth in scientific literature, the need for automated methods to classify citations is now growing intense. However, we can only fully understand the intent of a citation if we look at the citation context in the citing paper and also the primary purpose of the cited article. In this work, we propose a neural multi-task learning framework that harnesses the structural information of the research papers and the cited paper’s information for the effective classification of citation intents. We analyze the impact of three auxiliary tasks on the performance of our approach for citation classification. Our experiments on three benchmark citation classification datasets show that incorporating cited paper information (title) shows that our deep neural model achieves a new state-of-the-art on the ACL-ARC dataset with an absolute increase of 5.3% in the F1 score over the previous best model. We also achieve comparable performance with respect to the best-performing systems in the SDP 2021 3C Shared task on Citation Context Classification. We make our codes available at https://github.com/Tirthankar-Ghosal/citationclassification-SCIM

Online news consumption via social media platforms has accelerated the growth of digital journalism. Adverse to traditional media, digital media has lower entry barriers and allows everyone as a content creator, resulting in numerous fake news productions to attract public attention. As multimedia content is more convenient for users than expressing their feelings through text, images and video-embedded fake news is being circulated rapidly on social media nowadays. Emotional appeal in fake news is also a driving factor in its rapid dissemination. Although prior studies have made a remarkable effort toward fake news detection, they give less emphasis on exploring video modality and emotional appeal in fake news. To bridge this gap, this paper presents the following two contributions: i) It first develops a video-based multimodal fake news detection dataset named FakeClips and ii) It introduces a deep multitask framework dedicated to video-embedded multimodal fake news detection in which fake news detection is the main task and emotion recognition is the auxiliary task. The results reveal that investigating emotion and fake news together in a multitasking framework achieves 9.04% and 5.27% gains in terms of accuracy and f-score, respectively over the state-of-the-art model i.e. Fake Video Detection Model.

Peer reviews form an essential part of scientific communication. Scholarly peer review is probably the most accepted way to evaluate research papers by involving multiple experts to review the concerned research independently. Usually, the area chair, the program chair, or the editor takes a call weighing the reviewer’s judgments. It communicates the decision to the author via writing a meta-review by summarizing the review comments. With the exponential rise in research paper submissions and the corresponding rise in the reviewer pool, it becomes stressful for the chairs/editors to manage conflicts, arrive at a consensus, and also write an informative meta-review. Here in this work, we propose a novel deep neural network-based approach for generating meta-reviews in an unsupervised fashion. To generate consistent meta-reviews, we use a generic template where the task is like to slot-fill the template with the generated meta-review text. We consider the setting where only peer reviews with no summaries or meta-reviews are provided and propose an end-to-end neural network model to perform unsupervised opinion-based abstractive summarization. We first use an aspect-based sentiment analysis model, which classifies the review sentences with the corresponding aspects (e.g., novelty, substance, soundness, etc.) and sentiment. We then extract opinion phrases from reviews for the corresponding aspect and sentiment labels. Next, we train a transformer model to reconstruct the original reviews from these extraction. Finally, we filter the selected opinions according to their aspect and/or sentiment at the time of summarization. The selected opinions of each aspect are used as input to the trained Transformer model, which uses them to construct an opinion summary. The idea is to give a concise meta-review that maximizes information coverage by focusing on aspects and sentiment present in the review, coherence, readability, and redundancy. We evaluate our model on the human written template-based meta-reviews to show that our framework outperforms competitive baselines. We believe that the template-based meta-review generation focusing on aspect and sentiment will help the editor/chair in decision-making and assist the meta-reviewer in writing better and more informative meta-reviews. We make our codes available at https://github.com/sandeep82945/Unsupervised-meta-review-generation.

With the growing presence of multimodal content on the web, a specific category of fake news is rampant on popular social media outlets. In this category of fake online information, real multimedia contents (images, videos) are used in different but related contexts with manipulated texts to mislead the readers. The presence of seemingly non-manipulated multimedia content reinforces the belief in the associated fabricated textual content. Detecting this category of misleading multimedia fake news is almost impossible without relevance to any prior knowledge. In addition to this, the presence of highly novel and emotion-invoking contents can fuel the rapid dissemination of such fake news. To counter this problem, in this paper, we first introduce a novel multimodal fake news dataset that includes background knowledge (from authenticate sources) of the misleading articles. Second, we design a multimodal framework using Supervised Contrastive Learning (SCL) based novelty detection and Emotion Prediction tasks for fake news detection. We perform extensive experiments to reveal that our proposed model outperforms the state-of-the-art (SOTA) models.

One key frontier of artificial intelligence (AI) is the ability to comprehend research articles and validate their findings, posing a magnanimous problem for AI systems to compete with human intelligence and intuition. As a benchmark of research validation, the existing peer-review system still stands strong despite being criticized at times by many. However, the paper vetting system has been severely strained due to an influx of research paper submissions and increased conferences/journals. As a result, problems, including having insufficient reviewers, finding the right experts, and maintaining review quality, are steadily and strongly surfacing. To ease the workload of the stakeholders associated with the peer-review process, we probed into what an AI-powered review system would look like. In this work, we leverage the interaction between the paper’s full text and the corresponding peer-review text to predict the overall recommendation score and final decision. We do not envisage AI reviewing papers in the near future. Still, we intend to explore the possibility of a human–AI collaboration in the decision-making process to make the current system FAIR. The idea is to have an assistive decision-making tool for the chairs/editors to help them with an additional layer of confidence, especially with borderline and contrastive reviews. We use a deep attention network between the review text and paper to learn the interactions and predict the overall recommendation score and final decision. We also use sentiment information encoded within peer-review texts to guide the outcome further. Our proposed model outperforms the recent state-of-the-art competitive baselines. We release the code of our implementation here: https://github.com/PrabhatkrBharti/PEERRec.git.

The ever-growing number of pre-trained large language models (LLMs) across scientific domains presents a challenge for application developers. While these models offer vast potential, fine-tuning them with custom data, aligning them for specific tasks, and evaluating their performance remain crucial steps for effective utilization. However, applying these techniques to models with tens of billions of parameters can take days or even weeks on modern workstations, making the cumulative cost of model comparison and evaluation a significant barrier to LLM-based application development. To address this challenge, we introduce an end-to-end pipeline specifically designed for building conversational and programmable AI agents on high performance computing (HPC) platforms. Our comprehensive pipeline encompasses: model pre-training, fine-tuning, web and API service deployment, along with crucial evaluations for lexical coherence, semantic accuracy, hallucination detection, and privacy considerations. We demonstrate our pipeline through the development of chatHPC, a chatbot for HPC question answering and script generation. Leveraging our scalable pipeline, we achieve end-to-end LLM alignment in under an hour on the Frontier supercomputer. We propose a novel self-improved, self-instruction method for instruction set generation, investigate scaling and fine-tuning strategies, and conduct a systematic evaluation of model performance. The established practices within chatHPC will serve as a valuable guidance for future LLM-based application development on HPC platforms.