Publications
(See also:
Google Scholar).
2017
The Shattered Gradients Problem: If resnets are the answer, then what is the question?
D Balduzzi, M Frean, L Leary, JP Lewis, KWD Ma, B McWilliams 
ICML 2017, arXiv preprint arXiv:1702.08591 (Winner, Best Paper award in the workshop "Principled approaches to Deep Learning") 
Abstract: A longstanding obstacle to progress in deep learning is the problem of vanishing and exploding gradients. The problem has largely been overcome through the introduction of carefully constructed initializations and batch normalization. Nevertheless, architectures incorporating skipconnections such as resnets perform much better than standard feedforward architectures despite wellchosen initialization and batch normalization. In this paper, we identify the shattered gradients problem. Specifically, we show that the correlation between gradients in standard feedforward networks decays exponentially with depth resulting in gradients that resemble white noise. In contrast, the gradients in architectures with skipconnections are far more resistant to shattering decaying sublinearly. Detailed empirical evidence is presented in support of the analysis, on both fullyconnected networks and convnets. Finally, we present a new "looks linear" (LL) initialization that prevents shattering. Preliminary experiments show the new initialization allows to train very deep networks without the addition of skipconnections. 
Back to RGB: Deep Articulated Hand Pose Estimation From A Single Camera Image
WanDuo Kurt Ma, J.P. Lewis, Marcus Frean and David Balduzzi 
IVCNZ 2017 (Image And Vision Computing New Zealand), paper 581 
Abstract: Abstract—In this work, we demonstrate a method called Deep Hand Pose Machine(DHPM) that effectively detects the anatomical joints in the human hand based on single RGB images. Current stateoftheart methods are able to robustly infer hand poses from RGBD images. However, the depth map from an infrared camera does not operate well under direct sunlight. Performing hand tracking outdoors using depth sensors results in unreliable depth information and inaccurate poses. For this reason we were motivated to create this method which solely utilizes ordinary RGB image without additional depth information. Our approach adapts the pose machine algorithm, which has been used in the past to detect human body joints. We perform pose machine training on synthetic data to accurately predict the position of the joints in a real hand image. 
2016
An Overview of the SKA Science Analysis Pipeline
Detecting Diffuse Sources in Astronomical Images
T ButlerYeoman, M Frean, CP Hollitt, DW Hogg, M JohnstonHollitt 
arXiv preprint arXiv:1601.00266 
Abstract: We present an algorithm capable of detecting diffuse, dim sources of any size in an astronomical image. These sources often defeat traditional methods for source finding, which expand regions around points of high intensity. Extended sources often have no bright points and are only detectable when viewed as a whole, so a more sophisticated approach is required. Our algorithm operates at all scales simultaneously by considering a tree of nested candidate bounding boxes, and inverts a hierarchical Bayesian generative model to obtain the probability of sources existing at given locations and sizes. This model naturally accommodates the detection of nested sources, and no prior knowledge of the distribution of a source, or even the background, is required. The algorithm scales nearly linear with the number of pixels making it feasible to run on large images, and requires minimal parameter tweaking to be effective. We demonstrate the algorithm on several types of astronomical and artificial images. 
2015
Magnetron: Fitting bursts from magnetars
D Huppenkothen, BJ Brewer, DW Hogg, I Murray, M Frean 
Astrophysics Source Code Library 
Dissecting magnetar variability with Bayesian hierarchical models
Daniela Huppenkothen, Brendon J Brewer, David W Hogg, Iain Murray, Marcus Frean, Chris Elenbaas, Al Watts, Y Levin, AJ van der Horst, C Kouveliotou 
The Astrophysical Journal 810 (1), 66. See also arXiv preprint arXiv:1501.05251 
Abstract: Neutron stars are a prime laboratory for testing physical processes under conditions of strong gravity, high density, and extreme magnetic fields. Among the zoo of neutron star phenomena, magnetars stand out for their bursting behaviour, ranging from extremely bright, rare giant flares to numerous, less energetic recurrent bursts. The exact trigger and emission mechanisms for these bursts are not known; favoured models involve either a crust fracture and subsequent energy release into the magnetosphere, or ... 
PDF on arXiv 
2014
Source detection in astronomical images by Bayesian model comparison
Marcus Frean, Anna Friedlander, Melanie JohnstonHollitt and Chris Hollitt 
AIP Conference Proceedings 1636 (1), 5561 
Particle filter parallelisation using random network based resampling
PB Choppala, PD Teal, MR Frean 
Information Fusion (FUSION), 2014 17th International Conference on. July, 2014 (Salamanca, Spain) 
Abstract: The particle filter approximation to the posterior density converges to the true posterior as the number of particles used increases. The greater the number of particles, the higher the computational load, which can be implemented by operating the particle filter in parallel architectures. However, the resampling stage in the particle filter requires synchronisation, extensive interchange and routing of particle information, and thus impedes the use of parallel hardware systems. This paper presents a novel resampling technique ... 
PDF 
Adapting the multiBernoulli filter to phased array observations using MUSIC as pseudolikelihood
PB Choppala, PD Teal, MR Frean 
Information Fusion (FUSION), 2014 17th International Conference on. July, 2014 (Salamanca, Spain) 
Abstract: In this paper, we consider Bayesian multitarget tracking using phased array of sensors. Although joint Bayesian filtering is theoretically the optimal approach to multitarget tracking, the method suffers from high computational complexity for large numbers of targets. The PHD and multiBernoulli filters avoid this complexity by operating in the dimensionality of a single target space. However, these filters do not possess a mathematical framework to operate directly on signals from the phased sensor array. Therefore, it is necessary for the ... 
Soft systematic resampling for accurate posterior approximation and increased information retention in particle filtering
Praveen B. Choppala, Marcus R. Frean, Paul D. Teal 
In Proc. IEEE Int'l. Workshop Statistical Signal Processing (SSP). June, 2014 (Gold Coast, Australia). Pages 260263 
Abstract: The resampling step in the particle filter results in the loss of information contained in the lower weight particles. While the stochastic resamplers cause further loss by reweighing all particles to the same number, the deterministic resamplers impede the accumulation of information contained in the weights that are increasing over time. The soft resampler overcomes this information loss problem by redistributing the discarded weight among the lower weights. However, the technique does not accurately approximate the ... 
On the Local Popularity Impact in Object Replica Placement over Wireless Mesh Networks
Zakwan AlArnaout , Qiang Fu, and Marcus Frean. 
IEEE International WoWMoM 2014 Conference. (under review) 
On the Placement of Web Content Replicas in Wireless Mesh Networks
Zakwan AlArnaout , Qiang Fu, and Marcus Frean. 
IEEE International Conference on Communications,ICC 2014 (accepted) 
2013
The effect of population structure on the rate of evolution
Marcus Frean, Paul Rainey and Arne Traulsen 
Proceedings B (Proceedings of the Royal Society, Biological) 
Ecological factors exert a range of effects on the dynamics of the
evolutionary process. A particularly marked effect comes from
population structure, which can affect the probability that new mutations
reach fixation. Our interest is in population structures, such as
those depicted by ``star graphs'', that amplify the effects of
selection by further increasing the fixation probability of
advantageous mutants and decreasing the fixation probability of
disadvantageous mutants. The fact that star graphs increase the
fixation probability of beneficial mutations has lead to the
conclusion that evolution proceeds more rapidly in starstructured
populations, compared to, for example, mixed (unstructured)
populations. Here we show that the effects of population structure on
the rate of evolution are more complex and subtle that previously
recognized and draw attention to the importance of fixation time. By
comparing population structures that amplify selection with other
population structures, both analytically and numerically, we show that
evolution can slow down substantially even in populations where
selection is amplified. 
Source detection in astronomical images by Bayesian model comparison
Marcus Frean, Anna Friedlander, Melanie JohnstonHollitt and Chris Hollitt 
MaxEnt 2013  the 33rd International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, ANU Canberra, Dec 2013 
The next generation of radio telescopes will generate exabytes of data on hundreds of millions of objects, making automated methods for the detection of astronomical objects (``sources") essential. Of particular importance are faint diffuse objects embedded in noise, which are not well found by current automated methods. There is a pressing need for source finding software that identifies these sources, involves little manual tuning, yet is tractable to calculate. This paper describes how to build Dirichlet and multinomial models for pixel intensity distributions in radio astronomy images, and to use these to find sources. 
FREAN_MAXENT2013.pdf 
Soft resampling for improved information retention in particle filtering
Praveen B. Choppala, Paul D. Teal, Marcus R. Frean 
Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. Held May, 2013 (Vancouver, Canada). Pages 40364040 
ABSTRACT: The reweighting process of particle filtering can result in very large variance of particle weights; a problem known as degeneracy. The usual solution to this is an intermediary resampling step in which particles with lower weights are replaced by copies of those with large weights. This resampling inevitably results in loss of the information contained in those particles of low weights. Most of the existing stochastic and deterministic resampling schemes cause further loss of information because all the resampled particles ... 
Exploiting Graph Partitioning for Hierarchical Replica Placement in Wireless Mess Networks
Zakwan Arnaout, Qiang Fu and Marcus Frean 
MSWiM '13: Proceedings of the 16th ACM International Conference on Modeling, Analysis & Simulation of Wireless and Mobile Systems, held 3  8 Nov. 2013, Barcelona, Spain. 
PDF 
A Divideandconquer Approach for Content Replication in Wireless Mesh Networks
Zakwan AlArnaout , Qiang Fu, and Marcus Frean. 
The International Journal of Computer and Telecommunication Networks ( COMNET ), Elsevier , vol. 57, no. 18, pp. 3914–3928, 2013 
2012
Latent Dirichlet Allocation for Image Segmentation and Source Finding in Radio Astronomy Images
Anna Friedlander, Marcus Frean, Melanie JohnstonHollitt and Christopher Hollitt 
IVCNZ 2012, 27th Image and Vision Computing New Zealand (IVCNZ conference) 
We present exploratory work into the application of the topic modelling algorithm latent Dirichlet allocation (LDA) to image segmentation in greyscale images, and in particular, source detection in radio astronomy images. LDA performed similarly to the standard sourcedetection software on a representative sample of radio astronomy images. Our use of LDA underperforms on fainter and diffuse sources, but yields superior results on a representative image polluted with artefacts — the type of image in which the standard sourcedetection software requires manual intervention by an astronomer for adequate results. 
Linked at ACM, and PDF 
Gaussian Process Dynamical Models for Nonparametric Speech Representation and Synthesis
Gustav Eje Henter, Marcus R. Frean, W. Bastiaan Kleijn (2012). 
ICASSP 2012, _37th International Conference on Acoustics, Speech, and Signal Processing, Kyoto, Japan_ 
We propose Gaussian process dynamical models (GPDMs) as a new, nonparametric paradigm in acoustic models of speech. These use multidimensional, continuous statespaces to overcome familiar issues with discretestate, HMMbased speech models. The added dimensions allow the state to represent and describe more than just temporal structure as systematic differences in mean, rather than as mere correlations in a residual (which dynamic features or ARHMMs do). Being based on Gaussian processes, the models avoid restrictive parametric or linearity assumptions on signal structure. We outline GPDM theory, and describe model setup and initialization schemes relevant to speech applications. Experiments demonstrate subjectively better quality of synthesized speech than from comparable HMMs. In addition, there is evidence for unsupervised discovery of salient speech structure. 
PDF 
A Content Replication Scheme for Wireless Mesh Networks
Zakwan AlArnaout , Qiang Fu, and Marcus Frean. 
Proceedings of the 22nd ACM international workshop on Network and Operating System Support for Digital Audio and Video, NOSSDAV’12 , Toronto, ON, Canada , June 78, 2012. 
PDF 
MPDNA: A Novel Distributed Replica Placement Heuristic for Wreless Mesh Networks
Zakwan AlArnaout , Jonathan Hart, Qiang Fu, and Marcus Frean 
37th Annual IEEE Conference on Local Computer Networks, LCN’12 , 22  25 Oct. 2012, Clearwater Beach, FL, USA. 
Content delivery is an area that has been well explored in the context of the wired Internet, but has received comparatively less attention when it comes to Wireless Mesh Networks (WMNs). A number of replica placement algorithms exist that are specifically designed for the Internet, but these do not consider the special features of wireless networks. In this paper, we study the problem of optimal content replication in WMNs. (etc). 
PDF 
Ecological Signalling
Joseph Bulbulia, Paul Reddish and Marcus Frean 
Chapter in A New Science of Religion , Routledge, September 2012 link 
On the issue of separability for problem decomposition in cooperative neuroevolution
Rohitash Chandra, Marcus Frean, Mengjie Zhang 
Neurocomputing, March 2012 
Cooperative coevolution divides an optimisation problem into subcomponents and employs evolutionary algorithms for evolving them. (etc). 
2011
Adapting Modularity During Learning in Cooperative Coevolutionary Recurrent Neural Networks
A Novel Distributed Content Replication and Placement Scheme for Wireless Mesh Networks
Zakwan AlArnaout , Qiang Fu, and Marcus Frean. 
Australasian Telecommunication Networks and Applications Conference,ATNAC’11, 9  11 Nov. 2011, Melbourne, Australia 
Emergence of rockpaperscissors ecologies from a twospecies system via intraspecific competition
Marcus Frean and Richard Mansfield 
International Conference on Complex Systems (ICCS) 2011, hosted by NECSI (New England Complex Systems Institute), July 16, 2011 
We show how cyclic 'rockpaperscissors' ecologies can emerge from simpler systems of just two species. We also highlight a counterintuitive effect in such systems: they behave in such a way as to hide the underlying fitness of their constituents. 
ICCS261FreanMansfield.pdf (abstract) 
Ongoing evolution on networks
M. Frean, G. Baxter and P. Rainey 
International Conference on Complex Systems (ICCS) 2011, hosted by NECSI (New England Complex Systems Institute), July 16, 2011 
In an evolving population, network structure can have striking effects on the survival probability of a mutant allele and on the rate at which it spreads. In networks with `hubs' (representing geographic or other constraints), the heightened probability of an initially rare mutant has led to the prediction that such networks act to amplify the effects of selection over drift. But selection and mutation interplay in a subtle way in such populations: hubs also slow the mutant's rate of invasion, so that if multiple mutants are allowed to spread at the same time, more of them may be present. (etc)... 
ICCS260FreanBaxterRainey.pdf 
Neutral Evolution as a Route to Largescale Cooperation in the Stag Hunt Game.
M. Frean and J. Bulbulia 
International Conference on Complex Systems (ICCS) 2011, hosted by NECSI (New England Complex Systems Institute), July 16, 2011 
A Stag Hunt is a game with two pure equilibria: alldefect and allcooperation. The cooperative equilibrium pays better than the defective equilibrium, and there can be no benefit from defection when all cooperate. Nevertheless, formal and empirical models agree that the defective equilibrium becomes easily entrenched. Why? The cooperative benefit ('Stag') relies on the participation of many partners, whereas the defective benefit is independent of what others do. Thus (etc...) 
ICCS259FreanBulbulia.pdf (abstract) 
Affording cooperative populations
Joseph Bulbulia and Marcus Frean 
Religion, Brain & Behaviour, 1 : 1, pp 6670, 2011 
Begins: `Much of the kerfuffle about cooperation's evolution has centred on the problem of freeriding, according to which "The prospect of defection without loss of reward provides powerful incentives ... to free ride on the efforts of others"....' 
BulbuliaFrean2011a.pdf (jump to page 66, ie. 22nd page of this PDF) 
Bayesian multiple person tracking using probability hypothesis density smoothing
S. Hernandez and M. Frean 
International Journal on Smart Sensing and Intelligent Systems , submitted OUTCOME?? 
We present a PHD filtering approach to estimate the state of an unknown number of persons in a video sequence. Persons are represented by moving blobs, which are tracked across different frames using a firstorder moment approximation to the posterior density. The PHD filter is a good alternative to standard multitarget tracking algorithms, since overrides making explicit associations between measurements and persons locations. The recursive method has linear complexity in the number of targets, so it also has the potential benefit of scaling well with a large number of persons being tracked. The PHD filter achieves interesting results for the multiple persons tracking problem, albeit discarding useful information from higher order interactions. Nevertheless, a backward statespace representation using PHD smoothing can be used to refine the filtered estimates. In this paper, we present two smoothing strategies for improving PHD filter estimates in multiple persons tracking. Results from using PHD smoothing techniques in a video sequence shows a slight gain in the cardinality estimates (meaning the number of persons in a particular video frame), but good performance in the individual location estimates. 
2011_HernandezFrean.pdf 
Tutorial on Deep belief nets
Encoding subcomponents in cooperative coevolutionary recurrent neural networks
R. Chandra, M. Frean, M. Zhang and C. W. Omlin 
Cooperative coevolution employs evolutionary algorithms to solve a highdimensional search problem by decomposing it into lowdimensional subcomponents.(etc). 
Neurocomputing, Elsevier, Chandra et al 2011 
Modularity Adaptation in Cooperative Coevolutionary Feedforward Neural Networks
R. Chandra, M. Frean, M. Zhang 
IEEE International Joint Conference on Neural Networks, Accepted, 2011, San Jose, USA, In Press. 
Memetic Cooperative Coevolutionary Recurrent Neural Networks
R. Chandra, M. Frean, M. Zhang 
IEEE International Joint Conference on Neural Networks, Accepted, 2011, San Jose, USA, In Press. 
Software Graphs and Programmer Awareness
Gareth Baxter and Marcus Frean, In preparation 
Dependencies between types in objectoriented software can be viewed as directed graphs, with types as nodes and dependencies as edges. A programmer working on a node is almost certainly more aware of its outdegree, which is immediately evident, than its indegree, which is not. This fundamental asymmetry of information is reflected in the software graph in the large, with the indegree and outdegree distributions of such graphs having quite different forms; the former resembles a powerlaw distribution and the latter an exponential distribution. We give a simple generative model for software graphs that essentially ignores all aspects of programmer behaviour and software functionality other than this asymmetry of awareness, and show that it reproduces the indegree and outdegree distributions observed across 14 different type relationships spanning 12 large and varied Java applications. 
BaxterFrean2008_progattn.pdf 
2010
The Evolution of Charismatic Cultures
Joseph Bulbulia and Marcus Frean 
Method and Theory in the Study of Religion, 22 : 4, pp 254271, 2010. 
The following essay explains how religion may evolve to support cooperation among anonymous partners. It first reviews honest signalling theory, and reveals a limitation in the model's capacity to explain largescale cooperation. It then suggests that much cooperation is threatened by uncertainty, rather than by cheating. Finally, it explains how signalling theory can be extended to address the problem of cooperation threatened by uncertainty, 'fragile cooperation'. The resulting extension of signalling theory  called 'charismatic signalling'  directs attention to potential cooperative benefits from religion's fascinating and diverse effects on the body. The charismatic signalling model is presented as a 'howpossibly model', not as a 'justso story. The model's interest comes from its ability to organise seemingly unrelated puzzles under a common solution, and to motivate the study of cooperative strategies harboured in shared ecologies. 
BulbuliaFreanMSTR2010.pdf 
An Encoding Scheme for Cooperative Coevolutionary Neural Networks
R. Chandra, M. Frean, M. Zhang 
Proceedings of the 23rd Australasian Joint Conference on Artificial Intelligence. AI 2010: Advances in Artificial Intelligence. Lecture Notes in Artificial Intelligence. Vol. 6464. Springer. Adelaide, Australia, 2010. pp. 253262. 
2009
Twofilter probability hypothesis density smoothing
Sergio Hernandez, Paul Teal and Marcus Frean 
First Chilean Workshop on Pattern Recognition (CWPR) , pp 1219, 2009. 
Mutation and Selection on Graphs
Marcus Frean and Gareth Baxter 
NETSCI 09 International Workshop and Conference on Complex Networks and their Applications, Istituto Veneto Scienze Lettere Ed Arti, June 29  July 3, 2009. 
R. Chandra, M. Frean, L. Rolland 
Proceedings of 8th IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA 2009), Daejeon, Korea, December 2009, pp. 171176. (doi: 10.1109/CIRA.2009.5423212) 
2008
Religion as Superorganism: On David Sloan Wilson's Darwin's Cathedral
Joseph Bulbulia and Marcus Frean 
Chapter in: M. Stausberg (Ed.) Contemporary Theories of Religion: A Critical Companion. New York: Routledge. , 2008. 
One of the most important biological theories of religion is also the most controversial. Here we describe and partially defend David Sloan Wilson's group selectionist model. According to Wilson, religions are best explained as "superorganisms" adapted to succeed in competition against others. The evolutionary history of religion is a battle of these titans. 
BulbuliaFreanDarwinsCathedral2008.pdf 
Using Gaussian Processes to Optimize Expensive Functions
Marcus Frean and Phillip Boyle 
Lecture Notes in Computer Science (LNCS 5360). Proceedings Springer 2008, ISBN: 9783540893776., pp 258267. (_AI08, 21st Australasian Joint Conference on Artificial Intelligence, 35 Dec, Auckland, New Zealand, 2008)_ 
The task of finding the optimum of some function f(x) is commonly accomplished by generating and testing sample solutions iteratively, choosing each new sample x heuristically on the basis of results to date. We use Gaussian processes to represent predictions and uncertainty about the true function, and describe how to use these predictions to choose where to take each new sample in an optimal way. By doing this we were able to solve a difficult optimization problem  finding weights in a neural network controller to simultaneously balance two vertical poles  using an order of magnitude fewer samples than reported elsewhere. 
FreanBoyleGPO2008.pdf 
2007
Evolutionary dynamics on networks: selection versus drift.
Marcus Frean (2007) 
BIOWIRE 2007: A workshop on Bioinspired design of networks, in particular wireless networks and selforganizing properties of biological networks. Cambridge University, 25 April 2007. 
FreanBIOWIREslides2007.pdf 
Sense and Scale: Simulation of movement patterns and the response of egg distributions to resource density for Pieris rapae (Lepidoptera) at multiple spatial scales.
Jim Barritt, Stephen Hartley, and Marcus Frean 
Annual Meeting of the British Ecological Society , 2007 
2006
Understanding the Shape of Java Software
Gareth Baxter, Marcus Frean, James Noble, Mark Rickerby, Hayden Smith, Matt Visser, Hayden Melton and Ewan Tempero 
OOPSLA , 2006 
Large amounts of Java software have been written since the language's escape into unsuspecting software ecology more than ten years ago. Surprisingly little is known about the structure of Java programs in the wild: about the way methods are grouped into classes and then into packages, the way packages related to each other, or the way inheritance and composition are used to put these programs together. We present the results of the first indepth study of the structure of Java programs. We have collected a number of Java programs and measured their key structural attributes. We have found evidence that some relationships follow powerlaws, while others do not. We have also observed variations that seem related to some characteristic of the application itself. (etc...) 
OOPSLA_2006_BaxterEtAl.pdf 
Implementing Gaussian Process inference with neural networks
Marcus Frean, Matt Lilley and Phillip Boyle (2006) 
International Journal for Neural Systems, Special Issue on Adaptive Neural Methods for Intelligent Data Analysis, Editors H. Yin, M. Gallagher and M. MagdonIsmail 
Gaussian processes compare favourably with backpropagation neural networks as a tool for regression, and Bayesian neural networks have Gaussian process behaviour when the number of hidden neurons tends to infinity. We describe a simple recurrent neural network with connection weights trained by oneshot Hebbian learning. This network amounts to a dynamical system which relaxes to a stable state in which it generates predictions identical to those of Gaussian process regression. In effect an infinite number of hidden units in a feedforward architecture can be replaced by a merely finite number, together with recurrent connections. 
FreanLilleyBoyle_IJNS_2006.pdf 
Emergence of cyclic competitions in spatial ecosystems
Frean, M.R. 
SIRC 2006: Interactions and Spatial Process (Eighteenth Colloquium hosted by the Spatial Information Research Centre, November 67 2006, held in Dunedin, New Zealand) 
This paper considers ways in which cyclic competitions between species might emerge from other systems. An approach is suggested that begins with a single species. In some regimes it leads to networks of competing species, while in others it generates evolutionary dynamics that are akin to the ``red queen'' effect, but within a single species. I speculate on whether these behaviours might arise from a more realistic model, and sketch the form this model might take. 
Frean_sirc_2006.ps.gz 
A tutorial on the sumproduct algorithm (belief propagation)
A model for adjustment of the retinotectal mapping, based on Ephdependent regulation of ephrin levels.
Frean, M.R. 
CNS 2006 (Fifteenth Annual Computational Neuroscience Meeting, Edinburgh, July 1620, 2006) 
The formation of a topographically ordered map in the retinotectal system, independent of neural activity, has long been thought to rely on the matching of molecular cues between the innervating retinal ganglion cell axons and their targets in the tectum. In the last few years Ephephrin signalling has emerged as the likely substrate for this matching process. For example, EphA receptors are expressed in a decreasing gradient along the nasotemporal axis of the retina while their ephrin A ligands increase along the caudalrostral axis of the tectum. In principle this allows a retinal axon to be targetted to a particular termination zone within the tectum. There are several plausible mechanisms for how this targetting might occur. These models are able to account to varying degrees for recent key findings, but do not address a number of experiments carried out even before the discovery of Ephephrin signalling. The experiments involved recovery of the topographic projection following ablation of portions of the retina or tectum, in which the resulting mapping is seem to expand or contract appropriately, apparently making optimal use of the remaining areas. This paper describes a model for the formation of topographic mappings that incorporates the recent discoveries about Ephephrin signalling and is able to account for the expansion and contraction experiments. The model features (a) regulation of ephrin expression in cells that are innervated from the retina, with changes acting to match the current ephrin value to a target level, and (b) smoothing of ephrin levels in the tectum via a local diffusion process. It also incorporates a continuous tectum and `soft' competition between RGC axons for tectal space, as well as a tendency  rather than a hard constraint  for those axons to terminate in the tectum. An appealing feature is that since the ephrin levels are being reset, an axon growing from the retina at a later time should still find the correct position in the tectum. 
FreanCNS2006.pdf 
Combining random search and deterministic attraction in simulations of animal foraging.
Jim Barritt, Marcus Frean, and Stephen Hartley 
Ecology across the Tasman 2006 (Joint conference of the NZ Ecological Society and the Ecological Society of Australia) , 2006 
2005
Scalefree geometry in objectoriented programs
Potanin, A., Noble, J., Frean, M.R. and R. Biddle 
Communications of the ACM , 48, (5), 99103. May 2005 
Though conventional OO design suggests programs should be built from many small objects, like Lego bricks, they are instead built from objects that are scalefree, like fractals, and unlike Lego bricks. 
CACM.pdf 
A comparison of spiking neuron models for the control of unstable systems
Tod, R. and Frean, M.R. (2005) 
Motor Control and Cognitive Neuroscience Conference , Dunedin, 79 Dec 2005. 
Although there is much current interest in the neurodynamics of biologicaly plausible spiking neurons, there has been very little work investigating the role these properties might play in solving lowlevel physical control problems of the kind that living systems (and robots) face. We investigated four computationally inexpensive models of spiking neurons from the literature. Simple controllers with a feed forward architecture of spiking neurons were arrived at through an evolutionary process, and tested on tasks involving the control of unstable physical systems. Two models were readily able to solve a challenging version of the benchmark polebalancing problem, in which two poles are balanced simultaneously, and agents have no direct access to the cart or pole velocities. These velocities are required to solve the task in principle, and must therefore be inferred by the controller itself. Spike frequency adaptation, a distinctive feature of biological neurons, was found to be a crucial neurocomputational property. Spiking neurons models without spike frequency adaptation were unable to solve this task. Neuron models that exhibit adaptation have negative feedback to membrane potential, which dampens pole oscillations and leads to stable control. In effect, velocities are included in the computation implicitly via this adaptation, rather than explicitly as in most other standard controllers. Moreover, in successful agents the networks of spiking neurons were simpler that those arrived at by evolving conventional recurrent neural networks. 
Neural Networks: a replacement for Gaussian Processes?
Lilley, M. and Frean, M.R. (2005) 
Lecture Notes in Computer Science, SpringerVerlag (ISSN: 03029743), Volume 3578, p 95103, 2005 (Proc. Sixth Int. Conf. Intelligent Data Engineering and Automated Learning IDEAL'05, 68 July, Brisbane, Australia, July 68, 2005. Proceedings Editors: Marcus Gallagher, James Hogan, Frederic Maire) 
Gaussian processes have been favourably compared to backpropagation neural networks as a tool for regression. We show that a recurrent neural network can implement exact Gaussian process inference using only linear neurons that integrate their inputs over time, inhibitory recurrent connections, and oneshot Hebbian learning. The network amounts to a dynamical system which relaxes to the correct solution. We prove conditions for convergence, show how the system can act as its own teacher in order to produce rapid predictions, and comment on the biological plausibility of such a network. 
lilleyfrean.pdf 
Multiple Output Gaussian Process Regression
Boyle, P.K. and Frean, M.R. (2005) 
Technical Report, CSTR052 
Abstract: as for our Dependent Gaussian Processes paper, of which this is an expanded account. 
CSTR052.pdf 
Spatially explicit simulation of individual foraging behaviour across patchy resources.
James Barritt, Steve Hartley, Marcus Frean, and Marc Hasenbank. 
SIRC 2005  the 17th annual colloquium of the Spatial Information Research Centre, 2005 
Populationbased Continuous Optimization, Probabilistic Modelling and Mean Shift.
Gallagher, M.G. and Frean, M.R. (2005) 
Evolutionary Computation , 13: 1, 2942. Spring 2005 
Evolutionary algorithms perform optimization using a population of sample solution points. An interesting development has been to view populationbased optimization as the process of evolving an explicit, probabilistic model of the search space. This paper investigates a formal basis for continuous, populationbased optimization in terms of a stochastic gradient descent on the KullbackLeibler divergence between the model probability density and the objective function, represented as an unknown density of assumed form. This leads to an update rule that is related and compared with previous theoretical work, a continuous version of the populationbased incremental learning algorithm, and the generalized mean shift clustering framework. Experimental results are presented that demonstrate the dynamics of the new algorithm on a set of simple test problems. 
GallagherFrean2005.pdf 
2004
Dependent Gaussian Processes
Boyle, P.K. and Frean, M.R. (2004) 
Neural Information Processing Systems (NIPS) 
Gaussian processes are usually parameterised in terms of their covariance functions. However, this makes it difficult to deal with multiple outputs, because ensuring that the covariance matrix is positive definite is problematic. An alternative formulation is to treat Gaussian processes as white noise sources convolved with smoothing kernels, and to parameterise the kernel instead. Using this, we extend Gaussian processes to handle multiple, coupled outputs. 
paper: BoyleFrean2005.pdf, and the poster: dependentGPs_poster.pdf 
Adaptation and enslavement in hostendosymbiont associations
Frean, M.R. and Abraham,E. (2004) 
Physical Review E , 69, (051913) (6 pages). See also the article in the "Physics Update" section (page 9) of Physics Today, June 2004 
The evolutionary persistence of symbiotic associations is a puzzle. Adaptation should eliminate cooperative traits if it is possible to enjoy the advantages of cooperation without reciprocating?a facet of cooperation known in game theory as the Prisoner's Dilemma. Despite this barrier, symbioses are widespread and may have been necessary for the evolution of complex life. The discovery of strategies such as titfortat has been presented as a general solution to the problem of cooperation. However, this only holds for withinspecies cooperation, where a single strategy will come to dominate the population. In a symbiotic association each species may have a different strategy, and the theoretical analysis of the singlespecies problem is no guide to the outcome. We present basic analysis of twospecies cooperation and show that a species with a fast adaptation rate is enslaved by a slowly evolving one. Paradoxically, the rapidly evolving species becomes highly cooperative, whereas the slowly evolving one gives little in return. This helps understand the occurrence of endosymbioses where the host benefits, but the symbionts appear to gain little from the association. 
FreanAbraham2004.pdf 
2003
Optimizing connectionist architectures.
Frean, M.R. (2003) 
Encyclopedia of Cognitive Science, Nature MacMillan 
A key issue in using connectionist models is the choice of which network architecture to use. There are a number of ways this choice can be made automatically, driven by the problem at hand. 
ECS_69.pdf 
2001
Rockscissorspaper and the survival of the weakest.
Frean, M.R. and Abraham,E. (2001) 
Proceedings of the Royal Society (London) , Series B. 268, (1474), 13231328. This paper has been fairly widely cited, e.g. in Nature , P.N.A.S., Proceedings B. , Am. Nat. , Ecology, J. Theoretical Biology 
In the children's game of rockscissorspaper, players each choose one of three strategies. A rock beats a pair of scissors, scissors beat a sheet of paper and paper beats a rock, so the strategies form a competitive cycle. Although cycles in competitive ability appear to be reasonably rare among terrestrial plants, they are common among marine sessile organisms and have been reported in other contexts. Here we consider a system with three species in a competitive loop and show that this simple ecology exhibits two counterintuitive phenomena. First, the species that is least competitive is expected to have the largest population and, where there are oscillations in a finite population, to be the least likely to die out. As a consequence an apparent weakening of a species leads to an increase in its population. Second, evolution favours the most competitive individuals within a species, which leads to a decline in its population. This is analogous to the tragedy of the commons, but here, rather than leading to a collapse, the 'tragedy' acts to maintain diversity. 
RSP.pdf 
A voter model of the spatial prisoner's dilemma.
Frean, M.R. and E. Abraham (2001) 
IEEE Transactions on Evolutionary Computation 5 , (2), 117121. 
The prisoner's dilemma (PD) involves contests between two players, and may naturally be played on a spatial grid using voter model rules. In the model of spatial PD discussed here, the sites of a 2 dimensional lattice are occupied by strategies. At each timestep a site is chosen to play a PD game with one of its neighbors. The strategy of the chosen site then invades its neighbor with a probability which is proportional to the payoff from the game. Using results from the analysis of voter models, it is shown that with simple linear strategies this scenario results in the longterm survival of only one strategy. If three nonlinear strategies have a cyclic dominance relation between oneanother, then it is possible for relatively cooperative strategies to persist indefinitely. With the voter model dynamics, however, the average level of cooperation decreases with time if mutation of the strategies is included. Spatial effects are not in themselves sufficient to lead to the maintenance of cooperation. 
FreanAbraham2001voter.pdf 
Populationbased Continuous Optimization and Probabilistic Modelling.
Gallagher, M. and Frean, M.R. (2001) 
Tech Report No. MG12001, Centre for Intelligent Systems, School of Information Technology and Electrical Engineering, University of Queensland, QLD 4072, Australia. 
Abstract: see our later paper in Evolutionary Computation (above), which develops similar ideas and is better written. 
2000
Boosting algorithms as gradient descent in function space.
Mason,L., Baxter,J., Bartlett,P. and Frean, M.R. (2000) 
Neural Information Processing Systems, 1999. 12, 512518. MIT Press, 2000 
Much recent attention, both experimental and theoretical, has been focussed on classification algorithms which produce voted combinations of classifiers. Recent theoretical work has shown that the impressive generalization performance of algorithms like AdaBoost can be attributed to the classifier having large margins on the training data. We present abstract algorithms for finding linear and convex combinations of functions that minimize arbitrary cost functionals (i.e functionals that do not necessarily depend on the margin). Many existing voting methods can be shown to be special cases of these algorithms. Then, (etc)... 
Mason_etal99A.ps.gz 
How a schoolyard game echoes nature.
Frean, M.R. (2000) Update: Marsden Fund Newsletter , 13, p11.
Survival of the unfit.
Frean, M.R. and Abraham,E. (2000) 
Workshop on Evolutionary Computation and Cognitive Science, Melbourne, Feb 2000. 
In genetic algorithms it is often taken for granted that selection of the most suc Physics UpdateNeural Information Processing Systems, 1999. 12, 512518. MIT Press, 2000cessful members of a population will result in individuals whose fitness is higher than their ancestors. On the contrary, there exist circumstances in which "survival of the fittest" is catastrophically bad, and survival of the least fit leads to the highest population fitness over time. Such situations are succinctly described in terms of the Prisoner's Dilemma concept from game theory. We discuss the evolution of behaviour under selection pressures which amount to the prisoner's dilemma, with particular attention to the case of evolution in spatially structured environments. The Voter model provides a particularly exacting scenario for the evolution of cooperation, in which most existing results about the evolution of "titfortat" and related simple strategies fail to hold. Despite this, the model can exhibit complex cooperative behaviour including cycles of invasion. We 2006show that these cycles give rise to selfsimilar (fractal) spatially structured populations. Surprisingly, it pays each member of such a cycle to be minimally aggressive towards the species it can invade. 
1999
Functional Gradient Techniques for Combining Hypotheses.
Mason,L., Baxter,J., Bartlett,P. and Frean, M.R. (1999) 
Chapter 12 in Advances in Large Margin Classifiers , Smola, Bartlett, Scholkopf and Schuurmans (eds.), MIT Press 
Abstract: see "Boosting algorithms as gradient descent in function space" above. 
Mason_etal99B.ps.gz 
Realvalued evolutionary optimization using a flexible probability estimator
Gallagher,M., Frean, M.R. and Downs,T. (1999). 
Proceedings of the Genetic and Evolutionary Computation Conference, Orlando, Florida, July 1999. Volume 1, 840846 
Populationbased Incremental Learning (PBIL) is an abstraction of a genetic algorithm, which solves optimization problems by explicitly constructing a probabilistic model of the promising regions of the search space. At each iteration the model is used to generate a population of candidate solutions and is itself modified in response to these solutions. Through the extension of PBIL to realvalued search spaces, a more powerful and general algorithmic framework arises which enables the use of arbitrary probability density estimation techniques in evolutionary optimization. To illustrate the usefulness of this framework, we propose and implement an evolutionary algorithm which uses a finite Adaptive Gaussian mixture model density estimator (etc)... 
Catastrophic forgetting in simple networks: an analysis of the pseudorehearsal solution.
Frean, M.R. and Robins,A.V. (1999) 
Network: Computation in Neural Systems , 10, 227236 
Catastrophic forgetting is a major problem for sequential learning in neural networks. One very general solution to this problem, known as `pseudorehearsal', works well in practice for nonlinear networks but has not been analysed before. This paper formalises pseudorehearsal in linear networks. We show the method can fail in low dimensions but is guaranteed to succeed in high dimensions under fairly general conditions. In this case an optimal version of the method is equivalent to a simple modification of the `delta rule'. 
FreanRobins99.pdf 
1998
A simple cost function for boosting.
Frean, M.R. and Downs,T. (1998) 
Technical report, Dept. of Computer Science and Electrical Engineering, University of Queensland. 
For twoclass classification problems the boosting algorithm "AdaBoost" is equivalent to minimizing the costfunction: (..see the postscript...). Using this we show that the D variables are unnecessary for AdaBoost to work, and that `reboosting' of previous classifiers is straightforward. 
FreanDowns98.pdf 
Local learning algorithms for sequential learning tasks in neural networks.
Robins,A.V. and Frean, M.R. (1998) 
Journal of Advanced Computational Intelligence, 2, (6), 107  111 
In this paper we explore the concept of sequential learning and the efficacy of global and local neural network learning algorithms on a sequential learning task. Pseudorehearsal, a method developed by Robins (1995) to solve the catastrophic forgetting problem which arises from the excessive plasticity of neural networks, is significantly more effective than other local learning algorithms for the sequential task. We further consider the concept of local learning and suggest that pseudorehearsal is so effective because it works directly at the level of the learned function, and not indirectly on the representation of the function within the network. We also briefly explore the effect of local learning on generalisation within the task. 
RobinsFrean98A.ps.gz 
Proceedings of the 8th Australian Conference on Neural Networks (ACNN'98)
Editors: Downs,T., Frean,M.R. and Gallagher,M. (1998).
Catastrophic forgetting and `pseudorehearsal' in linear networks.
Frean, M.R. and Robins,A.V. (1998) 
Proceedings of the 8th Australian Conference on Neural Networks (ACNN'98), University of Queensland, Brisbane, 1113 Feb 1998 
Abstract: see our journal paper in Network: Computation in Neural Systems (above), which is better. 
Learning and generalization in a stable network.
Robins,A.V. and Frean, M.R. (1998) 
In Progress in ConnectionistBased Information Systems: Proc. 1997 Conference on Neural Information Processing and Intelligent Information Systems, Kasabov et. al. (Eds), Singapore: SpringerVerlag, pp. 314317 
Most neural networks suffer from excessive plasticity: the learning of new information interferes with information already stored in the network. In this paper we review the pseudorehearsal solution to this problem, proposed by Robins (1995). By localising the changes to the function learned by the network pseudorehearsal allows networks to be stable in the face of new learning, successfully integrating both new and previously learned information. In this paper we explore the impact that this mechanism has on the ability of the network to generalise. 
RobinsFrean98B.ps.gz 
before 1998
The evolution of degrees of cooperation.
Frean, M.R. (1996) 
Journal of Theoretical Biology , 182, pages 549559 
The Prisoner's Dilemma has been widely studied as a model for the evolution of cooperation, and most of this work has dealt with agents who either cooperate or not. In this paper we look at the consequences of allowing agents to have intermediate levels of cooperation, and to update these levels over time. The familiar strategy of ``tit for tat'' emerges as a robust mode of behaviour, yet there are important differences between this case and that of ``all or nothing'' cooperation. 
Frean96.pdf 
The Prisoner's Dilemma without synchrony
Frean, M.R. (1994) 
Proceedings of the Royal Society of London (B) , 257, pages 75  79 
There are many situations in which biological organisms cooperate despite obvious incentives to do otherwise. Such situations are commonly modelled using a paradigm known as the Prisoner's Dilemma. In this way cooperative behaviour has previously been shown to emerge in a model population of strategies. If players can make probabilistic choices taking into account their coplayer's previous action, a strategy known as `Generous Tit For Tat' dominates the longterm behaviour of such a population. If they can also take into account their own previous action, a strategy of `win stay, lose shift' dominates instead. These models assumed that participants make their decisions in synchrony, which seems improbable in many biological situations. Here we show that the timing of decisions is critical in determining which strategy emerges in the long run. If individuals make their decisions at different times, neither of the above strategies survives given the usual payoffs. In the former case Generous Tit For Tat succumbs to inveterate defectors, and in the latter a new strategy takes over. This `firm but fair' strategy is retaliatory yet highly cooperative. In particular, continued exploitation of a sucker is no longer a successful behaviour. 
Frean94.pdf 
Fantasy engines and brain theory
Frean, M.R. (1996)
Technical Report, A.I.Memo 34961, Department of Computer Science, Otago University. 
A 'thermal' perceptron learning rule.
Frean, M.R. (1992)
Neural Computation, 4, (6), 946  957
The thermal perceptron is a simple extension to Rosenblatt's perceptron learning rule for training individual linear threshold units. It finds stable weights for nonseparable problems as well as separable ones. Experiments indicate that if a good initial setting for a temperature parameter, $T_{0}$, has been found, then the thermal perceptron outperforms the Pocket algorithm and methods based on gradient descent. The learning rule stabilises the weights (learns) over a fixed training period. For separable problems it finds separating weights much more quickly than the usual rules. 
Frean92thermalperceptron.pdf 
The Upstart Algorithm: a method for constructing and training feedforward neural networks.
Frean, M.R. (1990)
Neural Computation, 2 (2), 189  209.
A general method for building and training multilayer perceptrons composed of linear threshold units is proposed. A simple recursive rule is used to build the net's structure by adding units as they are needed, while a modified Perceptron algorithm is used to learn the connection strengths. Convergence to zero errors is guaranteed for any Boolean classification on patterns of binary variables. Simulations suggest that this method is efficient in terms of the numbers of units constructed, and the networks it builds can generalise over patterns not in the training set. 

Frean90UpstartAlgorithm.pdf (digitized from a scan).
Small Nets and Short Paths: Optimising Neural Computation.
Frean, M.R. (1990)
PhD Thesis, University of Edinburgh, Scotland. 1990
This thesis explores two aspects of optimisation in neural network research. 1. The question of how to find the optimal feedforward neural network architecture for learning a given binary classification is addressed. The socalled constructive approach is reviewed whereby intermediate, hidden units are built as required for the particular problem. Current constructive algorithms are compared, and three new methods are introduced. One of these, the Upstart algorithm, in shown to outperform all other constructive algorithms of this type. This work led on to the ancillary problem of finding a satisfactory procedure for changing the weights values of an individual unit in a network. The new thermal perceptron rule is described and shown to compare favorably with its competitors. Finally, the spectrum of possible learning rules is surveyed. 2. Neurobiologically inspired algorithms for mapping between spaces of different dimensionality are applied to a classic optimisation problem, the Travelling Salesman Problem. Two new methods are described that can tackle the general symmetric form of the TSP, thus overcoming the restriction on other neural networks to the geometric case. 
Frean_thesis.ps.gz 
Removal of observer variability from the determination of volume of isoflow.
Lambert,R.K., Lau,T., Asher,M.I., Frean,M.R., Quin,J. and Hill,P. (1987)
Lung, 165 (2), 353  369.