On-line Novelty Detection through Self-Organisation, with Applications to Inspection Robotics
Presentation based on my PhD Thesis (pdf)The GWR Algorithm
The thesis itself
My PhD thesis, which I was awarded in 2002, looked at on-line novelty detection. Novelty detection is related to the topic of statistical outlier detection and is concerned with recognising when an input differs in some important way from those that have been seen before. This has many benefits for machine learning systems, as it can focus attention onto new and therefore potentially interesting stimuli, reduce the amount of learning that is required (since there is no point learning about things that you already know) -- which reduces the danger of over-fitting -- and suggest when the predictions of the network should be treated with caution.
Novelty detection is often used in places where there is a shortage of data about an important class. For example, in areas such as medical diagnosis, many more tests show the absence of a disease than the cases of it. Therefore, training a classifier to recognise the disease would be problematic.
Another application is that a novelty detector can be used as an inspection agent, after training with data that is known to be `normal' (that is, data that is seen during usual operation), any inputs that are highlighted by the novelty detector are inputs that were not seen during training, and are therefore potential faults. An example of this type of application of the work was described in the New Scientist (see here),
The approach that I used in my thesis was to train a clustering network to categorise inputs, and used a model of the biological process of habituation (a decrement in response when a stimulus is seen frequently without ill effect) to effectively learn to ignore inputs that are seen often. Anything that is highlighted by the network is therefore novel.
The thesis was a runner-up for the `Distinguished Dissertation' award of the British Computer Society.
My supervisors were Dr Jonathan Shapiro and Dr Ulrich Nehmzow.
More details are given in any of the relevant publications or in my thesis, which is available via link below.
Stephen Marsland
"On-line Novelty Detection Through Self-Organisation, With Application to Inspection Robotics"
PhD thesis, Department of Computer Science, The University of Manchester, 2002
gzipped PDF