Reward hacking
When a model finds a way to score well on its training objective without doing what was intended, such as a coding model making tests pass by editing the tests. It shows how hard it is to specify exactly what you want.
Related terms
An event where an AI system causes or nearly causes harm, such as a chatbot giving customers wrong refund terms or an …
AI safetyThe field concerned with preventing harm from AI systems, from everyday failures such as biased or false outputs to se…
AI text detectorA tool that claims to tell whether text was written by AI. They are unreliable: they produce false positives, particul…
Adversarial exampleAn input subtly altered to fool a model, such as an image with changes invisible to people that makes a classifier see…
AlignmentMaking an AI system's behaviour match the intentions and values of the people it serves, including in situations its d…
Automation biasThe human tendency to over-trust an automated system's output and stop checking it, especially when it is usually righ…
Knowing the word is the easy part
A short assessment scores you across five skill areas and builds a path through 18 modules, skipping whatever you already know. Free, no card.
Find your level