Journal Article

Word embeddings as autonomous predictors in materials design—the effect of inherent variability on information transfer

Batalović, Katarina, Radaković, Jana, Novaković, Nikola

2026

Overview

We propose that word embeddings of atoms derived from scientific literature are revisited as autonomous machine learning predictors in materials design. If static word embeddings encode comprehensive physicochemical information, joined embeddings of chemical elements constituting a chemical compound represent a viable source of physicochemical knowledge. Nevertheless, static word embeddings are susceptible to variability due to information heterogeneity within training material. We analysed whether variability occurs in embeddings affiliated with physicochemical entities, including explicit atoms, and whether it affects therein-encoded domain-specialized information or inhibits the information transfer. Results demonstrate the substantial variability in individual atomic embeddings, which is highly dependent on vocabulary terms selected for language modelling. Regardless, variability does not obstruct the mapping of materials' composite predictors into physicochemical properties when joined atomic embeddings are implemented within a regression model estimating the compound stability by predicting its formation energy. Moreover, the encoded information and the model's predictive performance maintained stability following compound vector calibration via dimensional reduction. Scientific contribution The magnitude of variability in word embeddings of physicochemical entities, including chemical elements, occurring due to information heterogeneity in complementary training material of materials science, chemistry, and physics scientific literature was observed and quantified. The research shows that notable variability of vectorial representations of chemical elements does not obstruct the underlying statistical properties, nor does it inhibit the information transfer. Accordingly, regardless of their origin, conjoined atomic embeddings representing chemical compounds facilitate stable predictive performance when implemented within a regression model.

• Publisher: Springer International Publishing,BioMed Central Ltd,Springer Nature B.V,BMC
• Subject: Algorithms / Autonomous machinery / Chemical compounds / Chemical elements / Chemistry / Chemistry and Materials Science / Coding / Comment / Composite materials / Computational Biology/Bioinformatics / Computational linguistics / Computer Applications in Chemistry / Digital materials / Documentation and Information in Chemistry / Equipment and supplies / Free energy / Heat of formation / Heterogeneity / In-silico materials design / Information transfer / Language / Language processing / Machine learning / Materials science / Materials stability / Natural language interfaces / Natural language processing / NLP-based autonomous predictors / Physicochemical properties / Physics / Regression models / Stability / Statistical analysis / Teaching / Theoretical and Computational Chemistry / Variability / Word embeddings in materials design / Word2vec / Words (language)