Explore the vast range of titles available.

Background: In adult spinal deformity (ASD) surgery, appropriate rod length determination is crucial, as excessive cranial rod length can lead to skin problems, especially in thin elderly patients if proximal junctional kyphosis (PJK) develops. In adolescent idiopathic scoliosis (AIS), correction is primarily performed in the coronal plane, and rod length changes are relatively predictable. Moreover, PJK is uncommon in AIS, making excess rod length rarely a clinical concern. In contrast, ASD correction involves more complex three-dimensional realignment, including restoration of lumbar lordosis (LL), which makes it challenging to predict postoperative changes in rod trace length (RTL). Furthermore, because PJK occurs more frequently in ASD surgery, appropriate rod length selection becomes clinically important. This study aimed to quantitatively evaluate changes in RTL before and after posterior correction. Method: Thirty patients with ASD who underwent staged lateral lumbar interbody fusion (LLIF) followed by posterior corrective fusion from T9 to the pelvis were retrospectively analyzed. RTL before posterior correction (Pre-RTL) was estimated from the planned screw insertional point on axial CT after LLIF, and postoperative RTL (Post-RTL) was measured from screw head centers on post-operative CT. LL and Cobb angle were assessed before and after posterior correction. Correlations between RTL change and alignment change were evaluated. Results: Postoperative RTL was shortened in all patients, with an average reduction of approximately 16–17 mm. RTL shortening demonstrated significant correlations with LL correction (R = 0.51, p = 0.003) and Cobb angle correction (R = 0.70, p = 0.00001). Greater shortening of RTL was observed on the convex side in patients with preoperative Cobb angle ≥ 10° (p = 0.04). Conclusions: Greater coronal deformity, particularly on the convex side, was associated with increased RTL shortening. These findings suggest that routine preparation of excessively long rods may be unnecessary. Consideration of anticipated RTL shortening may help avoid excessive cranial rod length and potentially reduce the risk of skin complications associated with PJK, particularly in thin elderly patients.

Purpose We have developed and reported on the usefulness of single and double endplates penetrating screw trajectories (SEPST/DEPST), which penetrate the endplates of the vertebrae to increase the fixation strength in patients with diffuse idiopathic skeletal hyperostosis (DISH). The purpose of this study is to compare the peak insertion torque (PIT) and pullout force (POF) of traditional trajectory (TT) and SEPST/DEPST using cadaveric spine. Methods The eighth thoracic vertebra to the fifth lumbar vertebra from six formalin-fixed adult human cadavers (two males, four females; age at death 85.7 ± 8.8 years) were used and PIT and POF were evaluated in the following three studies. Firstly, a comparison among TT/SEPST/DEPST. Secondly, a comparison of reinsertion of screws with DEPST (re-DEPST) in vertebrae inserted with TT and removed, assuming reoperation. Lastly, a comparison between the endplate thread screw (ETS) which has a smaller pitch and the conventional cancellous thread screw (CTS) was made. Results SEPST/DEPST had a 32–70% higher PIT and a 45–82% higher POF than TT, and DEPST was about 1.4 times higher than SEPST. Assuming reoperation, the results show that fixation with re-DEPST provided almost the same fixation strength as initial TT insertion. ETS, which was considered useful for DEPS, showed a relatively higher fixation strength than CTS, however, the difference was not significant. Conclusion The present study using cadaver demonstrated the strong fixation of SEPST/DEPST and its usefulness for revision surgery, however, there were no significant differences in the mechanical properties between ETS and CTS.

Minimum Bayes-Risk (MBR) decoding is shown to be a powerful alternative to beam search decoding for a wide range of text generation tasks. However, MBR requires a huge amount of time for inference to compute the MBR objective, which makes the method infeasible in many situations where response time is critical. Confidence-based pruning (CBP) (Cheng and Vlachos, 2023) has recently been proposed to reduce the inference time in machine translation tasks. Although it is shown to significantly reduce the amount of computation, it requires hyperparameter tuning using a development set to be effective. To this end, we propose Approximate Minimum Bayes-Risk (AMBR) decoding, a hyperparameter-free method to run MBR decoding approximately. AMBR is derived from the observation that the problem of computing the sample-based MBR objective is the medoid identification problem. AMBR uses the Correlated Sequential Halving (CSH) algorithm (Baharav and Tse, 2019), the best approximation algorithm to date for the medoid identification problem, to compute the sample-based MBR objective. We evaluate AMBR on machine translation, text summarization, and image captioning tasks. The results show that AMBR achieves on par with CBP, with CBP selecting hyperparameters through an Oracle for each given computation budget.

Many strong decoding methods for text generation follow a sample-and-rerank paradigm: they draw multiple candidates, score each under a utility (reward) function using consensus across samples, and return the best one. Although effective, these methods incur high computational costs during inference due to repeated sampling and scoring. Prior attempts to amortize inference-time computation typically rely on gold references, teacher labels, or curated preference data, increasing dataset construction effort and the demand for high-fidelity reward models. We propose Consensus Group Relative Policy Optimization (C-GRPO), which distills Minimum Bayes Risk (MBR) decoding into training by formulating the consensus utility as a group-relative objective within GRPO. C-GRPO requires only a utility function and policy samples, without gold references or explicit preference labels. Under ideal conditions, we show that the objective function of C-GRPO is directionally aligned with the gradient of the expected-utility objective underlying MBR decoding, leading to a convergence guarantee. Experiments on machine translation (WMT 2024) and text summarization (XSum) demonstrate that C-GRPO successfully achieves performance comparable to MBR decoding without the associated inference-time overhead, while outperforming reference-free baseline methods.

Minimum Bayes Risk (MBR) decoding optimizes output selection by maximizing the expected utility value of an underlying human distribution. While prior work has shown the effectiveness of MBR decoding through empirical evaluation, few studies have analytically investigated why the method is effective. As a result of our analysis, we show that, given the size \\(n\\) of the reference hypothesis set used in computation, MBR decoding approaches the optimal solution with high probability at a rate of \\(O(n^-12)\\), under certain assumptions, even though the language space \\(Y\\) is significantly larger \\(|Y| n\\). This result helps to theoretically explain the strong performance observed in several prior empirical studies on MBR decoding. In addition, we provide the performance gap for maximum-a-posteriori (MAP) decoding and compare it to MBR decoding. The result of this paper indicates that MBR decoding tends to converge to the optimal solution faster than MAP decoding in several cases.

Best-of-N (BoN) sampling with a reward model has been shown to be an effective strategy for aligning Large Language Models (LLMs) to human preferences at the time of decoding. BoN sampling is susceptible to a problem known as reward hacking when the accuracy of the reward model is not high enough due to the quality or the quantity of the preference dataset. Because the reward model is an imperfect proxy for the true objective, over-optimizing its value can compromise its performance on the true objective. In this research, we propose MBR-BoN, a variant of BoN that aims to mitigate reward hacking at inference time by incorporating the Minimum Bayes Risk (MBR) objective as a proximity regularization term. We show empirically and analytically that the MBR objective quantifies the proximity of the response to the reference policy, serving as a proximity regularizer. We evaluate MBR-BoN on the AlpacaFarm and Anthropic's hh-rlhf datasets and show that it outperforms both BoN sampling and MBR decoding. We also evaluate MBR-BoN to generate a pairwise preference learning dataset for Direct Preference Optimization (DPO). Empirical results show that models trained on a dataset generated with MBR-BoN outperform those with vanilla BoN. Our code is available at https://github.com/CyberAgentAILab/regularized-bon

Reinforcement learning from human feedback (RLHF) plays a crucial role in aligning language models with human preferences. While the significance of dataset quality is generally recognized, explicit investigations into its impact within the RLHF framework, to our knowledge, have been limited. This paper addresses the issue of text quality within the preference dataset by focusing on direct preference optimization (DPO), an increasingly adopted reward-model-free RLHF method. We confirm that text quality significantly influences the performance of models optimized with DPO more than those optimized with reward-model-based RLHF. Building on this new insight, we propose an extension of DPO, termed filtered direct preference optimization (fDPO). fDPO uses a trained reward model to monitor the quality of texts within the preference dataset during DPO training. Samples of lower quality are discarded based on comparisons with texts generated by the model being optimized, resulting in a more accurate dataset. Experimental results demonstrate that fDPO enhances the final model performance. Our code is available at https://github.com/CyberAgentAILab/filtered-dpo.

Best-of-N (BoN) sampling with a reward model has been shown to be an effective strategy for aligning Large Language Models (LLMs) to human preferences at the time of decoding. BoN sampling is susceptible to a problem known as reward hacking. Because the reward model is an imperfect proxy for the true objective, over-optimizing its value can compromise its performance on the true objective. A common solution to prevent reward hacking in preference learning techniques is to optimize a reward using proximity regularization (e.g., KL regularization), which ensures that the language model remains close to the reference model. In this research, we propose Regularized Best-of-N (RBoN), a variant of BoN that aims to mitigate reward hacking by incorporating a proximity term in response selection, similar to preference learning techniques. We evaluate RBoN on the AlpacaFarm and Anthropic's hh-rlhf datasets and find that it outperforms BoN. As an application of RBoN, we use RBoN to generate a pairwise preference learning dataset. Experimental results show that a DPO model trained on a dataset generated with RBoN outperforms a DPO model generated with vanilla BoN. Our code is available at https://github.com/CyberAgentAILab/regularized-bon

Minimum Bayes Risk (MBR) decoding has been shown to be a powerful alternative to beam search decoding in a variety of text generation tasks. MBR decoding selects a hypothesis from a pool of hypotheses that has the least expected risk under a probability model according to a given utility function. Since it is impractical to compute the expected risk exactly over all possible hypotheses, two approximations are commonly used in MBR. First, it integrates over a sampled set of hypotheses rather than over all possible hypotheses. Second, it estimates the probability of each hypothesis using a Monte Carlo estimator. While the first approximation is necessary to make it computationally feasible, the second is not essential since we typically have access to the model probability at inference time. We propose Model-Based MBR (MBMBR), a variant of MBR that uses the model probability itself as the estimate of the probability distribution instead of the Monte Carlo estimate. We show analytically and empirically that the model-based estimate is more promising than the Monte Carlo estimate in text generation tasks. Our experiments show that MBMBR outperforms MBR in several text generation tasks, both with encoder-decoder models and with large language models.

Best-of-N (BoN) sampling with a reward model has been shown to be an effective strategy for aligning Large Language Models (LLMs) with human preferences at the time of decoding. BoN sampling is susceptible to a problem known as reward hacking. Since the reward model is an imperfect proxy for the true objective, an excessive focus on optimizing its value can lead to a compromise of its performance on the true objective. Previous work proposes Regularized BoN sampling (RBoN), a BoN sampling with regularization to the objective, and shows that it outperforms BoN sampling so that it mitigates reward hacking and empirically (Jinnai et al., 2024). However, Jinnai et al. (2024) introduce RBoN based on a heuristic and they lack the analysis of why such regularization strategy improves the performance of BoN sampling. The aim of this study is to analyze the effect of BoN sampling on regularization strategies. Using the regularization strategies corresponds to robust optimization, which maximizes the worst case over a set of possible perturbations in the proxy reward. Although the theoretical guarantees are not directly applicable to RBoN, RBoN corresponds to a practical implementation. This paper proposes an extension of the RBoN framework, called Stochastic RBoN sampling (SRBoN), which is a theoretically guaranteed approach to worst-case RBoN in proxy reward. We then perform an empirical evaluation using the AlpacaFarm and Anthropic's hh-rlhf datasets to evaluate which factors of the regularization strategies contribute to the improvement of the true proxy reward. In addition, we also propose another simple RBoN method, the Sentence Length Regularized BoN, which has a better performance in the experiment as compared to the previous methods.