Emergent behaviours as phase transitions
Contact person: Arnoldo Frigessi
Status: Pre-project
When deploying and aligning modern Large Language Models, two controls can be adjusted when you prompt the LLM and ask to answer a question or solve a problem: sampling temperature and alignment steering.
Temperature controls diversity of the answer or solution that you get. A low temperature will make the answer deterministic, and you would observe a lack of creativity and it will soon get boring. A too high temperature, then words come at random, and you get just garbage out. Mathematically, the temperature is a rescaling the probability distribution of the answer.
Alignment steering can be used to steer the answer or solution in a wished direction, so that it aligns to what you want: have a polite tone, align with a law, behave safely. It works by adding a vector to your prompt, that directs the model's internal state to shift its behaviour, all without retraining. However if you steer too hard, again, the model sticks to your alignment vector too much and does not solve your problem well. A too low alignment steering will not help to align.
Both controls share a major flaw: they have a failure threshold past which the generated text completely collapses. Until now, the only way to find these usable limits was trial and error.
In this pre-project we work on this problem. We try to control such negative emerging behaviors a priori, by deciding the temperature and the strength of the alignment steering before we use the LLM. We do this by using the lens of statistical mechanics: the probabilities of the tokens needed to write the output act as energy levels and the steering vector acts as an external field.
What does this mean in practice? It means we can compute the exact usable window in advance of temperature and alignment steering. So, our results should give developers zero-cost diagnostic tools computed straight from the weights, eliminating brute-force parameter optimisation and allowing deployed LLM systems to operate safely within their true limits. Please take contact with us if you wish to test our tools in your LLMs.