Language Models Encode Functional-Group Knowledge Before They Can Name It - Artificial Intelligence

An analysis of five open-source models finds that their internal representations capture chemical relationships among functional groups, even when their generated answers are incorrect or incomplete.

Language models may encode information about molecular functional groups even when they fail to identify them in their answers, according to a study of five open-source models. The researchers examined internal representations for prompts about 92 molecules spanning 20 functional groups and found that the models organized these groups in chemically meaningful ways.

The study highlights a gap between what a model represents internally and what it says. In one example, a model asked to identify the primary functional group in 1-fluorobutane might answer “the alkyne group” rather than “alkyl fluoride.” An answer alone cannot show whether the model failed to recognize the fluorine or recognized it but failed to express that knowledge correctly.

The researchers assessed five 8-billion-parameter models: a general-purpose base model, three chemistry-trained models, and a model distilled for reasoning. They compared the models’ generated answers with hidden-state representations extracted at each layer. The base model named the correct group in 58.5% of prompts, the lowest accuracy among the five, but its internal representations still reflected functional-group families and consistent chemical differences.

Those representations included relationships between related structures. For example, the change from a thioether to a thiol, applied to an ether, pointed toward an alcohol in the models’ representation space. The researchers also report that functional-group information emerged sharply in middle layers, where representations shifted from reflecting prompt wording to reflecting the group itself.

The findings suggest that chemistry fine-tuning may reorganize existing chemical information so that functional groups become more prominent in a model’s representations, rather than simply adding knowledge that was absent. The authors argue that evaluating chemical language models only by their written answers can therefore underestimate the information they encode, and that representation analysis can help distinguish missing knowledge from difficulty expressing it.

Sources: arXiv cs.AI.