Associate Professor Marcus Frean

School of Engineering and Computer Science
Victoria University of Wellington

Publications

(See also: Google Scholar).

2019
- - A Hybrid Online Offline System for Network Anomaly Detection
  - Utility function generated saccade strategies for robot active vision: a probabilistic approach
2018
2017
2016
- - An Overview of the SKA Science Analysis Pipeline
  - Detecting Diffuse Sources in Astronomical Images
2015
- - Magnetron: Fitting bursts from magnetars
  - Dissecting magnetar variability with Bayesian hierarchical models
2014
2013
2012
2011
2010
- - The Evolution of Charismatic Cultures
  - An Encoding Scheme for Cooperative Coevolutionary Neural Networks
2009
2008
- - Religion as Superorganism: On David Sloan Wilson's Darwin's Cathedral
  - Using Gaussian Processes to Optimize Expensive Functions
2007
- - Evolutionary dynamics on networks: selection versus drift.
  - Sense and Scale: Simulation of movement patterns and the response of egg distributions to resource density for Pieris rapae (Lepidoptera) at multiple spatial scales.
2006
2005
2004
- - Dependent Gaussian Processes
  - Adaptation and enslavement in host-endosymbiont associations
2003
- - Optimizing connectionist architectures.
2001
2000
1999
1998
before 1998

2019

A Hybrid Online Offline System for Network Anomaly Detection

Odiathevar, Murugaraj and Seah, Winston KG and Frean, Marcus
28th International Conference on Computer Communication and Networks (ICCCN), pdf here

Odiathevar, Murugaraj and Seah, Winston KG and Frean, Marcus

28th International Conference on Computer Communication and Networks (ICCCN), pdf here

Utility function generated saccade strategies for robot active vision: a probabilistic approach

Bhakta, Arindam and Hollitt, Christopher and Browne, Will N and Frean, Marcus
Autonomous Robots, 43 (4), pp 947--966

Bhakta, Arindam and Hollitt, Christopher and Browne, Will N and Frean, Marcus

Autonomous Robots, 43 (4), pp 947--966

2018

Sample-Efficient Optimization Using Bayesian Neural Networks

Aryan, Mashall and Frean, Marcus and Lewis, John
2018 International Conference on Image and Vision Computing New Zealand (IVCNZ), pdf here

Aryan, Mashall and Frean, Marcus and Lewis, John

2018 International Conference on Image and Vision Computing New Zealand (IVCNZ), pdf here

A Comparison of RGB and HSV Colour Spaces for Visual Attention Models

Ajmal, Aisha and Hollitt, Christopher and Frean, Marcus and Al-Sahaf, Harith
2018 International Conference on Image and Vision Computing New Zealand (IVCNZ)

Ajmal, Aisha and Hollitt, Christopher and Frean, Marcus and Al-Sahaf, Harith

2018 International Conference on Image and Vision Computing New Zealand (IVCNZ)

Predictive Modelling of The Dynamic Patterns of Thinking in Attention-Deficit/Hyperactivity Disorder: Diagnostic Accuracy of Spatiotemporal Fractal Measures

Fabian Labra-Sprohnle, Garth Smith, Helmut Ahammer, Claire Postlethwaite, Ivy Liu, Paul Teesdale-Spittle, Marcus Frean
2018, bioRxiv
Abstract: Attention-Deficit/Hyperactivity Disorder (ADHD) is a neurodevelopmental condition characterized by executive function (EF) dynamics disturbances. Notwithstanding, current advances in translational neuroscience, no ADHD objective, clinically useful, diagnostic marker is available to date. Objectives: Using a customized definition of EF and a new clinical paradigm, we performed a prospective diagnostic accuracy trial to assess the diagnostic value of several fractal measures from the thinking processes or inferences in a cohort of ADHD children and typically developing controls. Method: We included children from age five to twelve diagnosed with a reference standard based on case history, physical and neurological examination, Conners 3rd Edition, and DSM-VTM. The index test consisted of a computer-based inference task with a set of eight different instances of the “Battleships” game to be solved. A consecutive series of 18 cases and 18 controls (n = 36) recruited at the primary paediatrics service from the Nelson Marlborough Health in New Zealand underwent the reference standard and the index test. Several fractal measures were obtained from the inference task to produce supervised classification models. Results: Notably, the summarized logistic regression’s predicted probabilities from the eight games played by each children yielded a 100% classification accuracy, sensitivity and specificity in both a training and an independent testing/validating cohort. Conclusions: From a translational vantage point the expeditious method and the robust results make this technique a promising candidate to develop a screening, diagnostic and monitoring system for ADHD, and may serve to assess other EF disturbances

Fabian Labra-Sprohnle, Garth Smith, Helmut Ahammer, Claire Postlethwaite, Ivy Liu, Paul Teesdale-Spittle, Marcus Frean

2018, bioRxiv

Abstract: Attention-Deficit/Hyperactivity Disorder (ADHD) is a neurodevelopmental condition characterized by executive function (EF) dynamics disturbances. Notwithstanding, current advances in translational neuroscience, no ADHD objective, clinically useful, diagnostic marker is available to date. Objectives: Using a customized definition of EF and a new clinical paradigm, we performed a prospective diagnostic accuracy trial to assess the diagnostic value of several fractal measures from the thinking processes or inferences in a cohort of ADHD children and typically developing controls. Method: We included children from age five to twelve diagnosed with a reference standard based on case history, physical and neurological examination, Conners 3rd Edition, and DSM-VTM. The index test consisted of a computer-based inference task with a set of eight different instances of the “Battleships” game to be solved. A consecutive series of 18 cases and 18 controls (n = 36) recruited at the primary paediatrics service from the Nelson Marlborough Health in New Zealand underwent the reference standard and the index test. Several fractal measures were obtained from the inference task to produce supervised classification models. Results: Notably, the summarized logistic regression’s predicted probabilities from the eight games played by each children yielded a 100% classification accuracy, sensitivity and specificity in both a training and an independent testing/validating cohort. Conclusions: From a translational vantage point the expeditious method and the robust results make this technique a promising candidate to develop a screening, diagnostic and monitoring system for ADHD, and may serve to assess other EF disturbances

2017

The Shattered Gradients Problem: If resnets are the answer, then what is the question?

D Balduzzi, M Frean, L Leary, JP Lewis, KWD Ma, B McWilliams
ICML 2017, arXiv preprint arXiv:1702.08591 (Winner, Best Paper award in the workshop "Principled approaches to Deep Learning")
Abstract: A long-standing obstacle to progress in deep learning is the problem of vanishing and exploding gradients. The problem has largely been overcome through the introduction of carefully constructed initializations and batch normalization. Nevertheless, architectures incorporating skip-connections such as resnets perform much better than standard feedforward architectures despite well-chosen initialization and batch normalization. In this paper, we identify the shattered gradients problem. Specifically, we show that the correlation between gradients in standard feedforward networks decays exponentially with depth resulting in gradients that resemble white noise. In contrast, the gradients in architectures with skip-connections are far more resistant to shattering decaying sublinearly. Detailed empirical evidence is presented in support of the analysis, on both fully-connected networks and convnets. Finally, we present a new "looks linear" (LL) initialization that prevents shattering. Preliminary experiments show the new initialization allows to train very deep networks without the addition of skip-connections.

D Balduzzi, M Frean, L Leary, JP Lewis, KWD Ma, B McWilliams

ICML 2017, arXiv preprint arXiv:1702.08591 (Winner, Best Paper award in the workshop "Principled approaches to Deep Learning")

Abstract: A long-standing obstacle to progress in deep learning is the problem of vanishing and exploding gradients. The problem has largely been overcome through the introduction of carefully constructed initializations and batch normalization. Nevertheless, architectures incorporating skip-connections such as resnets perform much better than standard feedforward architectures despite well-chosen initialization and batch normalization. In this paper, we identify the shattered gradients problem. Specifically, we show that the correlation between gradients in standard feedforward networks decays exponentially with depth resulting in gradients that resemble white noise. In contrast, the gradients in architectures with skip-connections are far more resistant to shattering decaying sublinearly. Detailed empirical evidence is presented in support of the analysis, on both fully-connected networks and convnets. Finally, we present a new "looks linear" (LL) initialization that prevents shattering. Preliminary experiments show the new initialization allows to train very deep networks without the addition of skip-connections.

Back to RGB: Deep Articulated Hand Pose Estimation From A Single Camera Image

Wan-Duo Kurt Ma, J.P. Lewis, Marcus Frean and David Balduzzi
IVCNZ 2017 or PDF here (Image And Vision Computing New Zealand), paper 58-1
Abstract: Abstract—In this work, we demonstrate a method called Deep Hand Pose Machine(DHPM) that effectively detects the anatomical joints in the human hand based on single RGB images. Current state-of-the-art methods are able to robustly infer hand poses from RGB-D images. However, the depth map from an infrared camera does not operate well under direct sunlight. Performing hand tracking outdoors using depth sensors results in unreliable depth information and inaccurate poses. For this reason we were motivated to create this method which solely utilizes ordinary RGB image without additional depth information. Our approach adapts the pose machine algorithm, which has been used in the past to detect human body joints. We perform pose machine training on synthetic data to accurately predict the position of the joints in a real hand image.

Wan-Duo Kurt Ma, J.P. Lewis, Marcus Frean and David Balduzzi

IVCNZ 2017 or PDF here (Image And Vision Computing New Zealand), paper 58-1

Abstract: Abstract—In this work, we demonstrate a method called Deep Hand Pose Machine(DHPM) that effectively detects the anatomical joints in the human hand based on single RGB images. Current state-of-the-art methods are able to robustly infer hand poses from RGB-D images. However, the depth map from an infrared camera does not operate well under direct sunlight. Performing hand tracking outdoors using depth sensors results in unreliable depth information and inaccurate poses. For this reason we were motivated to create this method which solely utilizes ordinary RGB image without additional depth information. Our approach adapts the pose machine algorithm, which has been used in the past to detect human body joints. We perform pose machine training on synthetic data to accurately predict the position of the joints in a real hand image.

Active shift attention based object tracking system

Ajmal, Aisha and Hollitt, Christopher and Frean, Marcus
2017 International Conference on Image and Vision Computing New Zealand (IVCNZ)

Ajmal, Aisha and Hollitt, Christopher and Frean, Marcus

2017 International Conference on Image and Vision Computing New Zealand (IVCNZ)

2016

An Overview of the SKA Science Analysis Pipeline

C Hollitt, M Johnston-Hollitt, S Dehghan, M Frean, T Bulter-Yeoman
arXiv preprint arXiv:1601.04113

C Hollitt, M Johnston-Hollitt, S Dehghan, M Frean, T Bulter-Yeoman

arXiv preprint arXiv:1601.04113

Detecting Diffuse Sources in Astronomical Images

T Butler-Yeoman, M Frean, CP Hollitt, DW Hogg, M Johnston-Hollitt
arXiv preprint arXiv:1601.00266
Abstract: We present an algorithm capable of detecting diffuse, dim sources of any size in an astronomical image. These sources often defeat traditional methods for source finding, which expand regions around points of high intensity. Extended sources often have no bright points and are only detectable when viewed as a whole, so a more sophisticated approach is required. Our algorithm operates at all scales simultaneously by considering a tree of nested candidate bounding boxes, and inverts a hierarchical Bayesian generative model to obtain the probability of sources existing at given locations and sizes. This model naturally accommodates the detection of nested sources, and no prior knowledge of the distribution of a source, or even the background, is required. The algorithm scales nearly linear with the number of pixels making it feasible to run on large images, and requires minimal parameter tweaking to be effective. We demonstrate the algorithm on several types of astronomical and artificial images.

T Butler-Yeoman, M Frean, CP Hollitt, DW Hogg, M Johnston-Hollitt

arXiv preprint arXiv:1601.00266

Abstract: We present an algorithm capable of detecting diffuse, dim sources of any size in an astronomical image. These sources often defeat traditional methods for source finding, which expand regions around points of high intensity. Extended sources often have no bright points and are only detectable when viewed as a whole, so a more sophisticated approach is required. Our algorithm operates at all scales simultaneously by considering a tree of nested candidate bounding boxes, and inverts a hierarchical Bayesian generative model to obtain the probability of sources existing at given locations and sizes. This model naturally accommodates the detection of nested sources, and no prior knowledge of the distribution of a source, or even the background, is required. The algorithm scales nearly linear with the number of pixels making it feasible to run on large images, and requires minimal parameter tweaking to be effective. We demonstrate the algorithm on several types of astronomical and artificial images.

2015

Magnetron: Fitting bursts from magnetars

D Huppenkothen, BJ Brewer, DW Hogg, I Murray, M Frean
Astrophysics Source Code Library

D Huppenkothen, BJ Brewer, DW Hogg, I Murray, M Frean

Astrophysics Source Code Library

Dissecting magnetar variability with Bayesian hierarchical models

Daniela Huppenkothen, Brendon J Brewer, David W Hogg, Iain Murray, Marcus Frean, Chris Elenbaas, Al Watts, Y Levin, AJ van der Horst, C Kouveliotou
The Astrophysical Journal 810 (1), 66. See also arXiv preprint arXiv:1501.05251
Abstract: Neutron stars are a prime laboratory for testing physical processes under conditions of strong gravity, high density, and extreme magnetic fields. Among the zoo of neutron star phenomena, magnetars stand out for their bursting behaviour, ranging from extremely bright, rare giant flares to numerous, less energetic recurrent bursts. The exact trigger and emission mechanisms for these bursts are not known; favoured models involve either a crust fracture and subsequent energy release into the magnetosphere, or ...
PDF on arXiv

Daniela Huppenkothen, Brendon J Brewer, David W Hogg, Iain Murray, Marcus Frean, Chris Elenbaas, Al Watts, Y Levin, AJ van der Horst, C Kouveliotou

The Astrophysical Journal 810 (1), 66. See also arXiv preprint arXiv:1501.05251

Abstract: Neutron stars are a prime laboratory for testing physical processes under conditions of strong gravity, high density, and extreme magnetic fields. Among the zoo of neutron star phenomena, magnetars stand out for their bursting behaviour, ranging from extremely bright, rare giant flares to numerous, less energetic recurrent bursts. The exact trigger and emission mechanisms for these bursts are not known; favoured models involve either a crust fracture and subsequent energy release into the magnetosphere, or ...

PDF on arXiv

(patent, with GreenButton Ltd) A METHOD FOR ESTIMATING JOB RUN TIME	U.S. Pat. No. 9,058,216

(patent, with GreenButton Ltd) A METHOD FOR ESTIMATING JOB RUN TIME

U.S. Pat. No. 9,058,216

2014

Source detection in astronomical images by Bayesian model comparison

Marcus Frean, Anna Friedlander, Melanie Johnston-Hollitt and Chris Hollitt
AIP Conference Proceedings 1636 (1), 55-61

Marcus Frean, Anna Friedlander, Melanie Johnston-Hollitt and Chris Hollitt

AIP Conference Proceedings 1636 (1), 55-61

Particle filter parallelisation using random network based resampling

PB Choppala, PD Teal, MR Frean
Information Fusion (FUSION), 2014 17th International Conference on. July, 2014 (Salamanca, Spain)
Abstract: The particle filter approximation to the posterior density converges to the true posterior as the number of particles used increases. The greater the number of particles, the higher the computational load, which can be implemented by operating the particle filter in parallel architectures. However, the resampling stage in the particle filter requires synchronisation, extensive interchange and routing of particle information, and thus impedes the use of parallel hardware systems. This paper presents a novel resampling technique ...
PDF

PB Choppala, PD Teal, MR Frean

Information Fusion (FUSION), 2014 17th International Conference on. July, 2014 (Salamanca, Spain)

Abstract: The particle filter approximation to the posterior density converges to the true posterior as the number of particles used increases. The greater the number of particles, the higher the computational load, which can be implemented by operating the particle filter in parallel architectures. However, the resampling stage in the particle filter requires synchronisation, extensive interchange and routing of particle information, and thus impedes the use of parallel hardware systems. This paper presents a novel resampling technique ...

PDF

Adapting the multi-Bernoulli filter to phased array observations using MUSIC as pseudo-likelihood

PB Choppala, PD Teal, MR Frean
Information Fusion (FUSION), 2014 17th International Conference on. July, 2014 (Salamanca, Spain)
Abstract: In this paper, we consider Bayesian multi-target tracking using phased array of sensors. Although joint Bayesian filtering is theoretically the optimal approach to multi-target tracking, the method suffers from high computational complexity for large numbers of targets. The PHD and multi-Bernoulli filters avoid this complexity by operating in the dimensionality of a single target space. However, these filters do not possess a mathematical framework to operate directly on signals from the phased sensor array. Therefore, it is necessary for the ...

PB Choppala, PD Teal, MR Frean

Information Fusion (FUSION), 2014 17th International Conference on. July, 2014 (Salamanca, Spain)

Abstract: In this paper, we consider Bayesian multi-target tracking using phased array of sensors. Although joint Bayesian filtering is theoretically the optimal approach to multi-target tracking, the method suffers from high computational complexity for large numbers of targets. The PHD and multi-Bernoulli filters avoid this complexity by operating in the dimensionality of a single target space. However, these filters do not possess a mathematical framework to operate directly on signals from the phased sensor array. Therefore, it is necessary for the ...

Soft systematic resampling for accurate posterior approximation and increased information retention in particle filtering

Praveen B. Choppala, Marcus R. Frean, Paul D. Teal
In Proc. IEEE Int'l. Workshop Statistical Signal Processing (SSP). June, 2014 (Gold Coast, Australia). Pages 260-263
Abstract: The resampling step in the particle filter results in the loss of information contained in the lower weight particles. While the stochastic resamplers cause further loss by reweighing all particles to the same number, the deterministic resamplers impede the accumulation of information contained in the weights that are increasing over time. The soft resampler overcomes this information loss problem by redistributing the discarded weight among the lower weights. However, the technique does not accurately approximate the ...

Praveen B. Choppala, Marcus R. Frean, Paul D. Teal

In Proc. IEEE Int'l. Workshop Statistical Signal Processing (SSP). June, 2014 (Gold Coast, Australia). Pages 260-263

Abstract: The resampling step in the particle filter results in the loss of information contained in the lower weight particles. While the stochastic resamplers cause further loss by reweighing all particles to the same number, the deterministic resamplers impede the accumulation of information contained in the weights that are increasing over time. The soft resampler overcomes this information loss problem by redistributing the discarded weight among the lower weights. However, the technique does not accurately approximate the ...

On the Local Popularity Impact in Object Replica Placement over Wireless Mesh Networks

Zakwan Al-Arnaout , Qiang Fu, and Marcus Frean.
IEEE International WoWMoM 2014 Conference. (under review)

Zakwan Al-Arnaout , Qiang Fu, and Marcus Frean.

IEEE International WoWMoM 2014 Conference. (under review)

On the Placement of Web Content Replicas in Wireless Mesh Networks

Zakwan Al-Arnaout , Qiang Fu, and Marcus Frean.
IEEE International Conference on Communications,ICC 2014 (accepted)

Zakwan Al-Arnaout , Qiang Fu, and Marcus Frean.

IEEE International Conference on Communications,ICC 2014 (accepted)

2013

The effect of population structure on the rate of evolution

Marcus Frean, Paul Rainey and Arne Traulsen
Proceedings B (Proceedings of the Royal Society, Biological)
Ecological factors exert a range of effects on the dynamics of the evolutionary process. A particularly marked effect comes from  population structure, which can affect the probability that new mutations  reach fixation. Our interest is in population structures, such as  those depicted by ``star graphs'', that amplify the effects of  selection by further increasing the fixation probability of  advantageous mutants and decreasing the fixation probability of disadvantageous mutants. The fact that star graphs increase the  fixation probability of beneficial mutations has lead to the  conclusion that evolution proceeds more rapidly in star-structured  populations, compared to, for example, mixed (unstructured)  populations. Here we show that the effects of population structure on  the rate of evolution are more complex and subtle that previously  recognized and draw attention to the importance of fixation time. By  comparing population structures that amplify selection with other  population structures, both analytically and numerically, we show that  evolution can slow down substantially even in populations where  selection is amplified.

Marcus Frean, Paul Rainey and Arne Traulsen

Proceedings B (Proceedings of the Royal Society, Biological)

Ecological factors exert a range of effects on the dynamics of the evolutionary process. A particularly marked effect comes from  population structure, which can affect the probability that new mutations  reach fixation. Our interest is in population structures, such as  those depicted by ``star graphs'', that amplify the effects of  selection by further increasing the fixation probability of  advantageous mutants and decreasing the fixation probability of disadvantageous mutants. The fact that star graphs increase the  fixation probability of beneficial mutations has lead to the  conclusion that evolution proceeds more rapidly in star-structured  populations, compared to, for example, mixed (unstructured)  populations. Here we show that the effects of population structure on  the rate of evolution are more complex and subtle that previously  recognized and draw attention to the importance of fixation time. By  comparing population structures that amplify selection with other  population structures, both analytically and numerically, we show that  evolution can slow down substantially even in populations where  selection is amplified.

Source detection in astronomical images by Bayesian model comparison

Marcus Frean, Anna Friedlander, Melanie Johnston-Hollitt and Chris Hollitt
MaxEnt 2013 - the 33rd International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, ANU Canberra, Dec 2013
The next generation of radio telescopes will generate exabytes of data on hundreds of millions of objects, making automated methods for the detection of astronomical objects (``sources") essential. Of particular importance are faint diffuse objects embedded in noise, which are not well found by current automated methods. There is a pressing need for source finding software that identifies these sources, involves little manual tuning, yet is tractable to calculate. This paper describes how to build Dirichlet and multinomial models for pixel intensity distributions in radio astronomy images, and to use these to find sources.
FREAN_MAXENT2013.pdf

Marcus Frean, Anna Friedlander, Melanie Johnston-Hollitt and Chris Hollitt

MaxEnt 2013 - the 33rd International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, ANU Canberra, Dec 2013

The next generation of radio telescopes will generate exabytes of data on hundreds of millions of objects, making automated methods for the detection of astronomical objects (``sources") essential. Of particular importance are faint diffuse objects embedded in noise, which are not well found by current automated methods. There is a pressing need for source finding software that identifies these sources, involves little manual tuning, yet is tractable to calculate. This paper describes how to build Dirichlet and multinomial models for pixel intensity distributions in radio astronomy images, and to use these to find sources.

FREAN_MAXENT2013.pdf

Soft resampling for improved information retention in particle filtering

Praveen B. Choppala, Paul D. Teal, Marcus R. Frean
Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. Held May, 2013 (Vancouver, Canada). Pages 4036-4040
ABSTRACT: The reweighting process of particle filtering can result in very large variance of particle weights; a problem known as degeneracy. The usual solution to this is an intermediary resampling step in which particles with lower weights are replaced by copies of those with large weights. This resampling inevitably results in loss of the information contained in those particles of low weights. Most of the existing stochastic and deterministic resampling schemes cause further loss of information because all the resampled particles ...

Praveen B. Choppala, Paul D. Teal, Marcus R. Frean

Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. Held May, 2013 (Vancouver, Canada). Pages 4036-4040

ABSTRACT: The reweighting process of particle filtering can result in very large variance of particle weights; a problem known as degeneracy. The usual solution to this is an intermediary resampling step in which particles with lower weights are replaced by copies of those with large weights. This resampling inevitably results in loss of the information contained in those particles of low weights. Most of the existing stochastic and deterministic resampling schemes cause further loss of information because all the resampled particles ...

Exploiting Graph Partitioning for Hierarchical Replica Placement in Wireless Mess Networks

Zakwan Arnaout, Qiang Fu and Marcus Frean
MSWiM '13: Proceedings of the 16th ACM International Conference on Modeling, Analysis & Simulation of Wireless and Mobile Systems, held 3 - 8 Nov. 2013, Barcelona, Spain.
PDF

Zakwan Arnaout, Qiang Fu and Marcus Frean

MSWiM '13: Proceedings of the 16th ACM International Conference on Modeling, Analysis & Simulation of Wireless and Mobile Systems, held 3 - 8 Nov. 2013, Barcelona, Spain.

PDF

A Divide-and-conquer Approach for Content Replication in Wireless Mesh Networks

Zakwan Al-Arnaout , Qiang Fu, and Marcus Frean.
The International Journal of Computer and Telecommunication Networks ( COMNET ), Elsevier , vol. 57, no. 18, pp. 3914–3928, 2013

Zakwan Al-Arnaout , Qiang Fu, and Marcus Frean.

The International Journal of Computer and Telecommunication Networks ( COMNET ), Elsevier , vol. 57, no. 18, pp. 3914–3928, 2013

2012

Latent Dirichlet Allocation for Image Segmentation and Source Finding in Radio Astronomy Images

Anna Friedlander, Marcus Frean, Melanie Johnston-Hollitt and Christopher Hollitt
IVCNZ 2012, 27th Image and Vision Computing New Zealand (IVCNZ conference)
We present exploratory work into the application of the topic modelling algorithm latent Dirichlet allocation (LDA) to image segmentation in greyscale images, and in particular, source detection in radio astronomy images. LDA performed similarly to the standard source-detection software on a representative sample of radio astronomy images. Our use of LDA underperforms on fainter and diffuse sources, but yields superior results on a representative image polluted with artefacts — the type of image in which the standard source-detection software requires manual intervention by an astronomer for adequate results.
Linked at ACM, and PDF

Anna Friedlander, Marcus Frean, Melanie Johnston-Hollitt and Christopher Hollitt

IVCNZ 2012, 27th Image and Vision Computing New Zealand (IVCNZ conference)

We present exploratory work into the application of the topic modelling algorithm latent Dirichlet allocation (LDA) to image segmentation in greyscale images, and in particular, source detection in radio astronomy images. LDA performed similarly to the standard source-detection software on a representative sample of radio astronomy images. Our use of LDA underperforms on fainter and diffuse sources, but yields superior results on a representative image polluted with artefacts — the type of image in which the standard source-detection software requires manual intervention by an astronomer for adequate results.

Linked at ACM, and PDF

Gaussian Process Dynamical Models for Nonparametric Speech Representation and Synthesis

Gustav Eje Henter, Marcus R. Frean, W. Bastiaan Kleijn (2012).
ICASSP 2012, _37th International Conference on Acoustics, Speech, and Signal Processing, Kyoto, Japan_
We propose Gaussian process dynamical models (GPDMs) as a new, nonparametric paradigm in acoustic models of speech. These use multidimensional, continuous state-spaces to overcome familiar issues with discrete-state, HMM-based speech models. The added dimensions allow the state to represent and describe more than just temporal structure as systematic differences in mean, rather than as mere correlations in a residual (which dynamic features or AR-HMMs do). Being based on Gaussian processes, the models avoid restrictive parametric or linearity assumptions on signal structure. We outline GPDM theory, and describe model setup and initialization schemes relevant to speech applications. Experiments demonstrate subjectively better quality of synthesized speech than from comparable HMMs. In addition, there is evidence for unsupervised discovery of salient speech structure.
PDF

Gustav Eje Henter, Marcus R. Frean, W. Bastiaan Kleijn (2012).

ICASSP 2012, _37th International Conference on Acoustics, Speech, and Signal Processing, Kyoto, Japan_

We propose Gaussian process dynamical models (GPDMs) as a new, nonparametric paradigm in acoustic models of speech. These use multidimensional, continuous state-spaces to overcome familiar issues with discrete-state, HMM-based speech models. The added dimensions allow the state to represent and describe more than just temporal structure as systematic differences in mean, rather than as mere correlations in a residual (which dynamic features or AR-HMMs do). Being based on Gaussian processes, the models avoid restrictive parametric or linearity assumptions on signal structure. We outline GPDM theory, and describe model setup and initialization schemes relevant to speech applications. Experiments demonstrate subjectively better quality of synthesized speech than from comparable HMMs. In addition, there is evidence for unsupervised discovery of salient speech structure.

PDF

A Content Replication Scheme for Wireless Mesh Networks

Zakwan Al-Arnaout , Qiang Fu, and Marcus Frean.
Proceedings of the 22nd ACM international workshop on Network and Operating System Support for Digital Audio and Video, NOSSDAV’12 , Toronto, ON, Canada , June 7-8, 2012.
PDF

Zakwan Al-Arnaout , Qiang Fu, and Marcus Frean.

Proceedings of the 22nd ACM international workshop on Network and Operating System Support for Digital Audio and Video, NOSSDAV’12 , Toronto, ON, Canada , June 7-8, 2012.

PDF

MP-DNA: A Novel Distributed Replica Placement Heuristic for Wreless Mesh Networks

Zakwan Al-Arnaout , Jonathan Hart, Qiang Fu, and Marcus Frean
37th Annual IEEE Conference on Local Computer Networks, LCN’12 , 22 - 25 Oct. 2012, Clearwater Beach, FL, USA.
Content delivery is an area that has been well explored in the context of the wired Internet, but has received comparatively less attention when it comes to Wireless Mesh Networks (WMNs). A number of replica placement algorithms exist that are specifically designed for the Internet, but these do not consider the special features of wireless networks. In this paper, we study the problem of optimal content replication in WMNs. (etc).
PDF

Zakwan Al-Arnaout , Jonathan Hart, Qiang Fu, and Marcus Frean

37th Annual IEEE Conference on Local Computer Networks, LCN’12 , 22 - 25 Oct. 2012, Clearwater Beach, FL, USA.

Content delivery is an area that has been well explored in the context of the wired Internet, but has received comparatively less attention when it comes to Wireless Mesh Networks (WMNs). A number of replica placement algorithms exist that are specifically designed for the Internet, but these do not consider the special features of wireless networks. In this paper, we study the problem of optimal content replication in WMNs. (etc).

PDF

Ecological Signalling

Joseph Bulbulia, Paul Reddish and Marcus Frean
Chapter in A New Science of Religion , Routledge, September 2012 link

Joseph Bulbulia, Paul Reddish and Marcus Frean

Chapter in A New Science of Religion , Routledge, September 2012 link

On the issue of separability for problem decomposition in cooperative neuro-evolution

Rohitash Chandra, Marcus Frean, Mengjie Zhang
Neurocomputing, March 2012
Cooperative coevolution divides an optimisation problem into subcomponents and employs evolutionary algorithms for evolving them. (etc).

Rohitash Chandra, Marcus Frean, Mengjie Zhang

Neurocomputing, March 2012

Cooperative coevolution divides an optimisation problem into subcomponents and employs evolutionary algorithms for evolving them. (etc).

2011

Adapting Modularity During Learning in Cooperative Co-evolutionary Recurrent Neural Networks

Rohitash Chandra, Marcus Frean and Mengjie Zhang
Soft Computing, Springer, November 2011
Adaptation during evolution has been an important focus of research in training neural networks. (etc).

Rohitash Chandra, Marcus Frean and Mengjie Zhang

Soft Computing, Springer, November 2011

Adaptation during evolution has been an important focus of research in training neural networks. (etc).

A Novel Distributed Content Replication and Placement Scheme for Wireless Mesh Networks

Zakwan Al-Arnaout , Qiang Fu, and Marcus Frean.
Australasian Telecommunication Networks and Applications Conference,ATNAC’11, 9 - 11 Nov. 2011, Melbourne, Australia

Zakwan Al-Arnaout , Qiang Fu, and Marcus Frean.

Australasian Telecommunication Networks and Applications Conference,ATNAC’11, 9 - 11 Nov. 2011, Melbourne, Australia

Emergence of rock-paper-scissors ecologies from a two-species system via intraspecific competition

Marcus Frean and Richard Mansfield
International Conference on Complex Systems (ICCS) 2011, hosted by NECSI (New England Complex Systems Institute), July 1-6, 2011
We show how cyclic 'rock-paper-scissors' ecologies can emerge from simpler systems of just two species. We also highlight a counter-intuitive effect in such systems: they behave in such a way as to hide the underlying fitness of their constituents.
ICCS-261-FreanMansfield.pdf (abstract)

Marcus Frean and Richard Mansfield

International Conference on Complex Systems (ICCS) 2011, hosted by NECSI (New England Complex Systems Institute), July 1-6, 2011

We show how cyclic 'rock-paper-scissors' ecologies can emerge from simpler systems of just two species. We also highlight a counter-intuitive effect in such systems: they behave in such a way as to hide the underlying fitness of their constituents.

ICCS-261-FreanMansfield.pdf (abstract)

Ongoing evolution on networks

M. Frean, G. Baxter and P. Rainey
International Conference on Complex Systems (ICCS) 2011, hosted by NECSI (New England Complex Systems Institute), July 1-6, 2011
In an evolving population, network structure can have striking effects on the survival probability of a mutant allele and on the rate at which it spreads. In networks with `hubs' (representing geographic or other constraints), the heightened probability of an initially rare mutant has led to the prediction that such networks act to amplify the effects of selection over drift. But selection and mutation interplay in a subtle way in such populations: hubs also slow the mutant's rate of invasion, so that if multiple mutants are allowed to spread at the same time, more of them may be present. (etc)...
ICCS-260-FreanBaxterRainey.pdf

M. Frean, G. Baxter and P. Rainey

International Conference on Complex Systems (ICCS) 2011, hosted by NECSI (New England Complex Systems Institute), July 1-6, 2011

In an evolving population, network structure can have striking effects on the survival probability of a mutant allele and on the rate at which it spreads. In networks with `hubs' (representing geographic or other constraints), the heightened probability of an initially rare mutant has led to the prediction that such networks act to amplify the effects of selection over drift. But selection and mutation interplay in a subtle way in such populations: hubs also slow the mutant's rate of invasion, so that if multiple mutants are allowed to spread at the same time, more of them may be present. (etc)...

ICCS-260-FreanBaxterRainey.pdf

Neutral Evolution as a Route to Large-scale Cooperation in the Stag Hunt Game.

M. Frean and J. Bulbulia
International Conference on Complex Systems (ICCS) 2011, hosted by NECSI (New England Complex Systems Institute), July 1-6, 2011
A Stag Hunt is a game with two pure equilibria: all-defect and all-cooperation. The cooperative equilibrium pays better than the defective equilibrium, and there can be no benefit from defection when all cooperate. Nevertheless, formal and empirical models agree that the defective equilibrium becomes easily entrenched. Why? The cooperative benefit ('Stag') relies on the participation of many partners, whereas the defective benefit is independent of what others do. Thus (etc...)
ICCS-259-FreanBulbulia.pdf (abstract)

M. Frean and J. Bulbulia

International Conference on Complex Systems (ICCS) 2011, hosted by NECSI (New England Complex Systems Institute), July 1-6, 2011

A Stag Hunt is a game with two pure equilibria: all-defect and all-cooperation. The cooperative equilibrium pays better than the defective equilibrium, and there can be no benefit from defection when all cooperate. Nevertheless, formal and empirical models agree that the defective equilibrium becomes easily entrenched. Why? The cooperative benefit ('Stag') relies on the participation of many partners, whereas the defective benefit is independent of what others do. Thus (etc...)

ICCS-259-FreanBulbulia.pdf (abstract)

Affording cooperative populations

Joseph Bulbulia and Marcus Frean
Religion, Brain & Behaviour, 1 : 1, pp 66-70, 2011
Begins: `Much of the kerfuffle about cooperation's evolution has centred on the problem of free-riding, according to which "The prospect of defection without loss of reward provides powerful incentives ... to free ride on the efforts of others"....'
BulbuliaFrean2011a.pdf (jump to page 66, ie. 22nd page of this PDF)

Joseph Bulbulia and Marcus Frean

Religion, Brain & Behaviour, 1 : 1, pp 66-70, 2011

Begins: `Much of the kerfuffle about cooperation's evolution has centred on the problem of free-riding, according to which "The prospect of defection without loss of reward provides powerful incentives ... to free ride on the efforts of others"....'

BulbuliaFrean2011a.pdf (jump to page 66, ie. 22nd page of this PDF)

Bayesian multiple person tracking using probability hypothesis density smoothing

S. Hernandez and M. Frean
International Journal on Smart Sensing and Intelligent Systems , submitted OUTCOME??
We present a PHD filtering approach to estimate the state of an unknown number of persons in a video sequence. Persons are represented by moving blobs, which are tracked across different frames using a first-order moment approximation to the posterior density. The PHD filter is a good alternative to standard multi-target tracking algorithms, since overrides making explicit associations between measurements and persons locations. The recursive method has linear complexity in the number of targets, so it also has the potential benefit of scaling well with a large number of persons being tracked. The PHD filter achieves interesting results for the multiple persons tracking problem, albeit discarding useful information from higher order interactions. Nevertheless, a backward state-space representation using PHD smoothing can be used to refine the filtered estimates. In this paper, we present two smoothing strategies for improving PHD filter estimates in multiple persons tracking. Results from using PHD smoothing techniques in a video sequence shows a slight gain in the cardinality estimates (meaning the number of persons in a particular video frame), but good performance in the individual location estimates.
2011_Hernandez-Frean.pdf

S. Hernandez and M. Frean

International Journal on Smart Sensing and Intelligent Systems , submitted OUTCOME??

We present a PHD filtering approach to estimate the state of an unknown number of persons in a video sequence. Persons are represented by moving blobs, which are tracked across different frames using a first-order moment approximation to the posterior density. The PHD filter is a good alternative to standard multi-target tracking algorithms, since overrides making explicit associations between measurements and persons locations. The recursive method has linear complexity in the number of targets, so it also has the potential benefit of scaling well with a large number of persons being tracked. The PHD filter achieves interesting results for the multiple persons tracking problem, albeit discarding useful information from higher order interactions. Nevertheless, a backward state-space representation using PHD smoothing can be used to refine the filtered estimates. In this paper, we present two smoothing strategies for improving PHD filter estimates in multiple persons tracking. Results from using PHD smoothing techniques in a video sequence shows a slight gain in the cardinality estimates (meaning the number of persons in a particular video frame), but good performance in the individual location estimates.

2011_Hernandez-Frean.pdf

Tutorial on Deep belief nets

Online video lecture of a (3 hour) tutorial I gave at the Machine Learning Summer School held at ANU, Canberra, in December 2010: http://videolectures.net/marcus_frean/

Online video lecture of a (3 hour) tutorial I gave at the Machine Learning Summer School held at ANU, Canberra, in December 2010: http://videolectures.net/marcus_frean/

Encoding subcomponents in cooperative co-evolutionary recurrent neural networks

R. Chandra, M. Frean, M. Zhang and C. W. Omlin
Cooperative coevolution employs evolutionary algorithms to solve a high-dimensional search problem by decomposing it into low-dimensional subcomponents.(etc).
Neurocomputing, Elsevier, Chandra et al 2011

R. Chandra, M. Frean, M. Zhang and C. W. Omlin

Cooperative coevolution employs evolutionary algorithms to solve a high-dimensional search problem by decomposing it into low-dimensional subcomponents.(etc).

Neurocomputing, Elsevier, Chandra et al 2011

Modularity Adaptation in Cooperative Coevolutionary Feedforward Neural Networks

R. Chandra, M. Frean, M. Zhang
IEEE International Joint Conference on Neural Networks, Accepted, 2011, San Jose, USA, In Press.

R. Chandra, M. Frean, M. Zhang

IEEE International Joint Conference on Neural Networks, Accepted, 2011, San Jose, USA, In Press.

Memetic Cooperative Coevolutionary Recurrent Neural Networks

R. Chandra, M. Frean, M. Zhang
IEEE International Joint Conference on Neural Networks, Accepted, 2011, San Jose, USA, In Press.

R. Chandra, M. Frean, M. Zhang

IEEE International Joint Conference on Neural Networks, Accepted, 2011, San Jose, USA, In Press.

Software Graphs and Programmer Awareness

Gareth Baxter and Marcus Frean, In preparation
Dependencies between types in object-oriented software can be viewed as directed graphs, with types as nodes and dependencies as edges. A programmer working on a node is almost certainly more aware of its out-degree, which is immediately evident, than its in-degree, which is not. This fundamental asymmetry of information is reflected in the software graph in the large, with the in-degree and out-degree distributions of such graphs having quite different forms; the former resembles a power-law distribution and the latter an exponential distribution. We give a simple generative model for software graphs that essentially ignores all aspects of programmer behaviour and software functionality other than this asymmetry of awareness, and show that it reproduces the in-degree and out-degree distributions observed across 14 different type relationships spanning 12 large and varied Java applications.
BaxterFrean2008_progattn.pdf

Gareth Baxter and Marcus Frean, In preparation

Dependencies between types in object-oriented software can be viewed as directed graphs, with types as nodes and dependencies as edges. A programmer working on a node is almost certainly more aware of its out-degree, which is immediately evident, than its in-degree, which is not. This fundamental asymmetry of information is reflected in the software graph in the large, with the in-degree and out-degree distributions of such graphs having quite different forms; the former resembles a power-law distribution and the latter an exponential distribution. We give a simple generative model for software graphs that essentially ignores all aspects of programmer behaviour and software functionality other than this asymmetry of awareness, and show that it reproduces the in-degree and out-degree distributions observed across 14 different type relationships spanning 12 large and varied Java applications.

BaxterFrean2008_progattn.pdf

2010

The Evolution of Charismatic Cultures

Joseph Bulbulia and Marcus Frean
Method and Theory in the Study of Religion, 22 : 4, pp 254-271, 2010.
The following essay explains how religion may evolve to support cooperation among anonymous partners. It first reviews honest signalling theory, and reveals a limitation in the model's capacity to explain large-scale cooperation. It then suggests that much cooperation is threatened by uncertainty, rather than by cheating. Finally, it explains how signalling theory can be extended to address the problem of cooperation threatened by uncertainty, 'fragile cooperation'. The resulting extension of signalling theory - called 'charismatic signalling' - directs attention to potential cooperative benefits from religion's fascinating and diverse effects on the body. The charismatic signalling model is presented as a 'how-possibly model', not as a 'just-so story. The model's interest comes from its ability to organise seemingly unrelated puzzles under a common solution, and to motivate the study of cooperative strategies harboured in shared ecologies.
BulbuliaFrean-MSTR-2010.pdf

Joseph Bulbulia and Marcus Frean

Method and Theory in the Study of Religion, 22 : 4, pp 254-271, 2010.

The following essay explains how religion may evolve to support cooperation among anonymous partners. It first reviews honest signalling theory, and reveals a limitation in the model's capacity to explain large-scale cooperation. It then suggests that much cooperation is threatened by uncertainty, rather than by cheating. Finally, it explains how signalling theory can be extended to address the problem of cooperation threatened by uncertainty, 'fragile cooperation'. The resulting extension of signalling theory - called 'charismatic signalling' - directs attention to potential cooperative benefits from religion's fascinating and diverse effects on the body. The charismatic signalling model is presented as a 'how-possibly model', not as a 'just-so story. The model's interest comes from its ability to organise seemingly unrelated puzzles under a common solution, and to motivate the study of cooperative strategies harboured in shared ecologies.

BulbuliaFrean-MSTR-2010.pdf

An Encoding Scheme for Cooperative Coevolutionary Neural Networks

R. Chandra, M. Frean, M. Zhang
Proceedings of the 23rd Australasian Joint Conference on Artificial Intelligence. AI 2010: Advances in Artificial Intelligence. Lecture Notes in Artificial Intelligence. Vol. 6464. Springer. Adelaide, Australia, 2010. pp. 253-262.

R. Chandra, M. Frean, M. Zhang

Proceedings of the 23rd Australasian Joint Conference on Artificial Intelligence. AI 2010: Advances in Artificial Intelligence. Lecture Notes in Artificial Intelligence. Vol. 6464. Springer. Adelaide, Australia, 2010. pp. 253-262.

2009

Two-filter probability hypothesis density smoothing

Sergio Hernandez, Paul Teal and Marcus Frean
First Chilean Workshop on Pattern Recognition (CWPR) , pp 12-19, 2009.

Sergio Hernandez, Paul Teal and Marcus Frean

First Chilean Workshop on Pattern Recognition (CWPR) , pp 12-19, 2009.

Mutation and Selection on Graphs

Marcus Frean and Gareth Baxter
NETSCI 09 International Workshop and Conference on Complex Networks and their Applications, Istituto Veneto Scienze Lettere Ed Arti, June 29 - July 3, 2009.

Marcus Frean and Gareth Baxter

NETSCI 09 International Workshop and Conference on Complex Networks and their Applications, Istituto Veneto Scienze Lettere Ed Arti, June 29 - July 3, 2009.

A Hybrid Meta-Heuristic Paradigm for Solving the Forward Kinematics of 6-6 General Parallel Manipulator

R. Chandra, M. Frean, L. Rolland
Proceedings of 8th IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA 2009), Daejeon, Korea, December 2009, pp. 171-176. (doi: 10.1109/CIRA.2009.5423212)

R. Chandra, M. Frean, L. Rolland

Proceedings of 8th IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA 2009), Daejeon, Korea, December 2009, pp. 171-176. (doi: 10.1109/CIRA.2009.5423212)

2008

Religion as Superorganism: On David Sloan Wilson's Darwin's Cathedral

Joseph Bulbulia and Marcus Frean
Chapter in: M. Stausberg (Ed.) Contemporary Theories of Religion: A Critical Companion. New York: Routledge. , 2008.
One of the most important biological theories of religion is also the most controversial. Here we describe and partially defend David Sloan Wilson's group selectionist model. According to Wilson, religions are best explained as "superorganisms" adapted to succeed in competition against others. The evolutionary history of religion is a battle of these titans.
BulbuliaFrean-DarwinsCathedral-2008.pdf

Joseph Bulbulia and Marcus Frean

Chapter in: M. Stausberg (Ed.) Contemporary Theories of Religion: A Critical Companion. New York: Routledge. , 2008.

One of the most important biological theories of religion is also the most controversial. Here we describe and partially defend David Sloan Wilson's group selectionist model. According to Wilson, religions are best explained as "superorganisms" adapted to succeed in competition against others. The evolutionary history of religion is a battle of these titans.

BulbuliaFrean-DarwinsCathedral-2008.pdf

Using Gaussian Processes to Optimize Expensive Functions

Marcus Frean and Phillip Boyle
Lecture Notes in Computer Science (LNCS 5360). Proceedings Springer 2008, ISBN: 978-3-540-89377-6., pp 258-267. (_AI-08, 21st Australasian Joint Conference on Artificial Intelligence, 3-5 Dec, Auckland, New Zealand, 2008)_
The task of finding the optimum of some function f(x) is commonly accomplished by generating and testing sample solutions iteratively, choosing each new sample x heuristically on the basis of results to date. We use Gaussian processes to represent predictions and uncertainty about the true function, and describe how to use these predictions to choose where to take each new sample in an optimal way. By doing this we were able to solve a difficult optimization problem - finding weights in a neural network controller to simultaneously balance two vertical poles - using an order of magnitude fewer samples than reported elsewhere.
FreanBoyle-GPO-2008.pdf

Marcus Frean and Phillip Boyle

Lecture Notes in Computer Science (LNCS 5360). Proceedings Springer 2008, ISBN: 978-3-540-89377-6., pp 258-267. (_AI-08, 21st Australasian Joint Conference on Artificial Intelligence, 3-5 Dec, Auckland, New Zealand, 2008)_

The task of finding the optimum of some function f(x) is commonly accomplished by generating and testing sample solutions iteratively, choosing each new sample x heuristically on the basis of results to date. We use Gaussian processes to represent predictions and uncertainty about the true function, and describe how to use these predictions to choose where to take each new sample in an optimal way. By doing this we were able to solve a difficult optimization problem - finding weights in a neural network controller to simultaneously balance two vertical poles - using an order of magnitude fewer samples than reported elsewhere.

FreanBoyle-GPO-2008.pdf

2007

Evolutionary dynamics on networks: selection versus drift.

Marcus Frean (2007)
BIOWIRE 2007: A workshop on Bioinspired design of networks, in particular wireless networks and self-organizing properties of biological networks. Cambridge University, 2-5 April 2007.
Frean-BIOWIRE-slides-2007.pdf

Marcus Frean (2007)

BIOWIRE 2007: A workshop on Bioinspired design of networks, in particular wireless networks and self-organizing properties of biological networks. Cambridge University, 2-5 April 2007.

Frean-BIOWIRE-slides-2007.pdf

Sense and Scale: Simulation of movement patterns and the response of egg distributions to resource density for Pieris rapae (Lepidoptera) at multiple spatial scales.

Jim Barritt, Stephen Hartley, and Marcus Frean

Annual Meeting of the British Ecological Society , 2007

2006

Understanding the Shape of Java Software

Gareth Baxter, Marcus Frean, James Noble, Mark Rickerby, Hayden Smith, Matt Visser, Hayden Melton and Ewan Tempero

OOPSLA , 2006

Large amounts of Java software have been written since the language's escape into unsuspecting software ecology more than ten years ago. Surprisingly little is known about the structure of Java programs in the wild: about the way methods are grouped into classes and then into packages, the way packages related to each other, or the way inheritance and composition are used to put these programs together. We present the results of the first in-depth study of the structure of Java programs. We have collected a number of Java programs and measured their key structural attributes. We have found evidence that some relationships follow power-laws, while others do not. We have also observed variations that seem related to some characteristic of the application itself. (etc...)

OOPSLA_2006_BaxterEtAl.pdf

Implementing Gaussian Process inference with neural networks

Marcus Frean, Matt Lilley and Phillip Boyle (2006)

International Journal for Neural Systems, Special Issue on Adaptive Neural Methods for Intelligent Data Analysis, Editors H. Yin, M. Gallagher and M. Magdon-Ismail

Gaussian processes compare favourably with backpropagation neural networks as a tool for regression, and Bayesian neural networks have Gaussian process behaviour when the number of hidden neurons tends to infinity. We describe a simple recurrent neural network with connection weights trained by one-shot Hebbian learning. This network amounts to a dynamical system which relaxes to a stable state in which it generates predictions identical to those of Gaussian process regression. In effect an infinite number of hidden units in a feed-forward architecture can be replaced by a merely finite number, together with recurrent connections.

FreanLilleyBoyle_IJNS_2006.pdf

Emergence of cyclic competitions in spatial ecosystems

Frean, M.R.

SIRC 2006: Interactions and Spatial Process (Eighteenth Colloquium hosted by the Spatial Information Research Centre, November 6-7 2006, held in Dunedin, New Zealand)

This paper considers ways in which cyclic competitions between species might emerge from other systems. An approach is suggested that begins with a single species. In some regimes it leads to networks of competing species, while in others it generates evolutionary dynamics that are akin to the ``red queen'' effect, but within a single species. I speculate on whether these behaviours might arise from a more realistic model, and sketch the form this model might take.

Frean_sirc_2006.ps.gz

A tutorial on the sum-product algorithm (belief propagation)

Marcus Frean

Presentation at 3rd workshop on Hidden Markov Models and Complex Systems, Wellington, 2006.

frean-sum-product-tutorial.pdf

A model for adjustment of the retinotectal mapping, based on Eph-dependent regulation of ephrin levels.

Frean, M.R.

CNS 2006 (Fifteenth Annual Computational Neuroscience Meeting, Edinburgh, July 16-20, 2006)

The formation of a topographically ordered map in the retinotectal system, independent of neural activity, has long been thought to rely on the matching of molecular cues between the innervating retinal ganglion cell axons and their targets in the tectum. In the last few years Eph-ephrin signalling has emerged as the likely substrate for this matching process. For example, Eph-A receptors are expressed in a decreasing gradient along the naso-temporal axis of the retina while their ephrin A ligands increase along the caudal-rostral axis of the tectum. In principle this allows a retinal axon to be targetted to a particular termination zone within the tectum. There are several plausible mechanisms for how this targetting might occur. These models are able to account to varying degrees for recent key findings, but do not address a number of experiments carried out even before the discovery of Eph-ephrin signalling. The experiments involved recovery of the topographic projection following ablation of portions of the retina or tectum, in which the resulting mapping is seem to expand or contract appropriately, apparently making optimal use of the remaining areas. This paper describes a model for the formation of topographic mappings that incorporates the recent discoveries about Eph-ephrin signalling and is able to account for the expansion and contraction experiments. The model features (a) regulation of ephrin expression in cells that are innervated from the retina, with changes acting to match the current ephrin value to a target level, and (b) smoothing of ephrin levels in the tectum via a local diffusion process. It also incorporates a continuous tectum and `soft' competition between RGC axons for tectal space, as well as a tendency - rather than a hard constraint - for those axons to terminate in the tectum. An appealing feature is that since the ephrin levels are being reset, an axon growing from the retina at a later time should still find the correct position in the tectum.

FreanCNS2006.pdf

Combining random search and deterministic attraction in simulations of anima|l foraging.

Jim Barritt, Marcus Frean, and Stephen Hartley

Ecology across the Tasman 2006 (Joint conference of the NZ Ecological Society and the Ecological Society of Australia) , 2006

2005

Scale-free geometry in object-oriented programs

Potanin, A., Noble, J., Frean, M.R. and R. Biddle

Communications of the ACM , 48, (5), 99-103. May 2005

Though conventional OO design suggests programs should be built from many small objects, like Lego bricks, they are instead built from objects that are scale-free, like fractals, and unlike Lego bricks.

CACM.pdf

A comparison of spiking neuron models for the control of unstable systems

Tod, R. and Frean, M.R. (2005)

Motor Control and Cognitive Neuroscience Conference , Dunedin, 7-9 Dec 2005.

Although there is much current interest in the neurodynamics of biologicaly plausible spiking neurons, there has been very little work investigating the role these properties might play in solving low-level physical control problems of the kind that living systems (and robots) face. We investigated four computationally inexpensive models of spiking neurons from the literature. Simple controllers with a feed forward architecture of spiking neurons were arrived at through an evolutionary process, and tested on tasks involving the control of unstable physical systems. Two models were readily able to solve a challenging version of the benchmark pole-balancing problem, in which two poles are balanced simultaneously, and agents have no direct access to the cart or pole velocities. These velocities are required to solve the task in principle, and must therefore be inferred by the controller itself. Spike frequency adaptation, a distinctive feature of biological neurons, was found to be a crucial neuro-computational property. Spiking neurons models without spike frequency adaptation were unable to solve this task. Neuron models that exhibit adaptation have negative feedback to membrane potential, which dampens pole oscillations and leads to stable control. In effect, velocities are included in the computation implicitly via this adaptation, rather than explicitly as in most other standard controllers. Moreover, in successful agents the networks of spiking neurons were simpler that those arrived at by evolving conventional recurrent neural networks.

Neural Networks: a replacement for Gaussian Processes?

Lilley, M. and Frean, M.R. (2005)

Lecture Notes in Computer Science, Springer-Verlag (ISSN: 0302-9743), Volume 3578, p 95-103, 2005 (Proc. Sixth Int. Conf. Intelligent Data Engineering and Automated Learning IDEAL'05, 6-8 July, Brisbane, Australia, July 6-8, 2005. Proceedings Editors: Marcus Gallagher, James Hogan, Frederic Maire)

Gaussian processes have been favourably compared to backpropagation neural networks as a tool for regression. We show that a recurrent neural network can implement exact Gaussian process inference using only linear neurons that integrate their inputs over time, inhibitory recurrent connections, and one-shot Hebbian learning. The network amounts to a dynamical system which relaxes to the correct solution. We prove conditions for convergence, show how the system can act as its own teacher in order to produce rapid predictions, and comment on the biological plausibility of such a network.

lilleyfrean.pdf

Multiple Output Gaussian Process Regression

Boyle, P.K. and Frean, M.R. (2005)

Technical Report, CS-TR-05-2

Abstract: as for our Dependent Gaussian Processes paper, of which this is an expanded account.

CS-TR-05-2.pdf

Spatially explicit simulation of individual foraging behaviour across patchy resources.

James Barritt, Steve Hartley, Marcus Frean, and Marc Hasenbank.

SIRC 2005 - the 17th annual colloquium of the Spatial Information Research Centre, 2005

Population-based Continuous Optimization, Probabilistic Modelling and Mean Shift.

Gallagher, M.G. and Frean, M.R. (2005)

Evolutionary Computation , 13: 1, 29-42. Spring 2005

Evolutionary algorithms perform optimization using a population of sample solution points. An interesting development has been to view population-based optimization as the process of evolving an explicit, probabilistic model of the search space. This paper investigates a formal basis for continuous, population-based optimization in terms of a stochastic gradient descent on the Kullback-Leibler divergence between the model probability density and the objective function, represented as an unknown density of assumed form. This leads to an update rule that is related and compared with previous theoretical work, a continuous version of the population-based incremental learning algorithm, and the generalized mean shift clustering framework. Experimental results are presented that demonstrate the dynamics of the new algorithm on a set of simple test problems.

GallagherFrean2005.pdf

2004

Dependent Gaussian Processes

Boyle, P.K. and Frean, M.R. (2004)

Neural Information Processing Systems (NIPS)

Gaussian processes are usually parameterised in terms of their covariance functions. However, this makes it difficult to deal with multiple outputs, because ensuring that the covariance matrix is positive definite is problematic. An alternative formulation is to treat Gaussian processes as white noise sources convolved with smoothing kernels, and to parameterise the kernel instead. Using this, we extend Gaussian processes to handle multiple, coupled outputs.

paper: BoyleFrean2005.pdf, and the poster: dependentGPs_poster.pdf

Adaptation and enslavement in host-endosymbiont associations

Frean, M.R. and Abraham,E. (2004)

Physical Review E , 69, (051913) (6 pages). See also the article in the "Physics Update" section (page 9) of Physics Today, June 2004

The evolutionary persistence of symbiotic associations is a puzzle. Adaptation should eliminate cooperative traits if it is possible to enjoy the advantages of cooperation without reciprocating?a facet of cooperation known in game theory as the Prisoner's Dilemma. Despite this barrier, symbioses are widespread and may have been necessary for the evolution of complex life. The discovery of strategies such as tit-for-tat has been presented as a general solution to the problem of cooperation. However, this only holds for within-species cooperation, where a single strategy will come to dominate the population. In a symbiotic association each species may have a different strategy, and the theoretical analysis of the single-species problem is no guide to the outcome. We present basic analysis of two-species cooperation and show that a species with a fast adaptation rate is enslaved by a slowly evolving one. Paradoxically, the rapidly evolving species becomes highly cooperative, whereas the slowly evolving one gives little in return. This helps understand the occurrence of endosymbioses where the host benefits, but the symbionts appear to gain little from the association.

FreanAbraham2004.pdf

2003

Optimizing connectionist architectures.

Frean, M.R. (2003)

Encyclopedia of Cognitive Science, Nature MacMillan

A key issue in using connectionist models is the choice of which network architecture to use. There are a number of ways this choice can be made automatically, driven by the problem at hand.

ECS_69.pdf

2001

Rock-scissors-paper and the survival of the weakest.

Frean, M.R. and Abraham,E. (2001)

Proceedings of the Royal Society (London) , Series B. 268, (1474), 1323-1328. This paper has been fairly widely cited, e.g. in Nature , P.N.A.S., Proceedings B. , Am. Nat. , Ecology, J. Theoretical Biology

In the children's game of rock-scissors-paper, players each choose one of three strategies. A rock beats a pair of scissors, scissors beat a sheet of paper and paper beats a rock, so the strategies form a competitive cycle. Although cycles in competitive ability appear to be reasonably rare among terrestrial plants, they are common among marine sessile organisms and have been reported in other contexts. Here we consider a system with three species in a competitive loop and show that this simple ecology exhibits two counter-intuitive phenomena. First, the species that is least competitive is expected to have the largest population and, where there are oscillations in a finite population, to be the least likely to die out. As a consequence an apparent weakening of a species leads to an increase in its population. Second, evolution favours the most competitive individuals within a species, which leads to a decline in its population. This is analogous to the tragedy of the commons, but here, rather than leading to a collapse, the 'tragedy' acts to maintain diversity.

RSP.pdf

A voter model of the spatial prisoner's dilemma.

Frean, M.R. and E. Abraham (2001)

IEEE Transactions on Evolutionary Computation 5 , (2), 117-121.

The prisoner's dilemma (PD) involves contests between two players, and may naturally be played on a spatial grid using voter model rules. In the model of spatial PD discussed here, the sites of a 2 dimensional lattice are occupied by strategies. At each time-step a site is chosen to play a PD game with one of its neighbors. The strategy of the chosen site then invades its neighbor with a probability which is proportional to the pay-off from the game. Using results from the analysis of voter models, it is shown that with simple linear strategies this scenario results in the long-term survival of only one strategy. If three non-linear strategies have a cyclic dominance relation between one-another, then it is possible for relatively cooperative strategies to persist indefinitely. With the voter model dynamics, however, the average level of cooperation decreases with time if mutation of the strategies is included. Spatial effects are not in themselves sufficient to lead to the maintenance of cooperation.

FreanAbraham2001-voter.pdf

Population-based Continuous Optimization and Probabilistic Modelling.

Gallagher, M. and Frean, M.R. (2001)

Tech Report No. MG-1-2001, Centre for Intelligent Systems, School of Information Technology and Electrical Engineering, University of Queensland, QLD 4072, Australia.

Abstract: see our later paper in Evolutionary Computation (above), which develops similar ideas and is better written.

2000

Boosting algorithms as gradient descent in function space.

Mason,L., Baxter,J., Bartlett,P. and Frean, M.R. (2000)

Neural Information Processing Systems, 1999. 12, 512-518. MIT Press, 2000

Much recent attention, both experimental and theoretical, has been focussed on classification algorithms which produce voted combinations of classifiers. Recent theoretical work has shown that the impressive generalization performance of algorithms like AdaBoost can be attributed to the classifier having large margins on the training data. We present abstract algorithms for finding linear and convex combinations of functions that minimize arbitrary cost functionals (i.e functionals that do not necessarily depend on the margin). Many existing voting methods can be shown to be special cases of these algorithms. Then, (etc)...

Mason_etal99A.ps.gz

How a schoolyard game echoes nature.

Frean, M.R. (2000) Update: Marsden Fund Newsletter , 13, p11.

Survival of the unfit.

Frean, M.R. and Abraham,E. (2000)

Workshop on Evolutionary Computation and Cognitive Science, Melbourne, Feb 2000.

In genetic algorithms it is often taken for granted that selection of the most suc Physics UpdateNeural Information Processing Systems, 1999. 12, 512-518. MIT Press, 2000cessful members of a population will result in individuals whose fitness is higher than their ancestors. On the contrary, there exist circumstances in which "survival of the fittest" is catastrophically bad, and survival of the least fit leads to the highest population fitness over time. Such situations are succinctly described in terms of the Prisoner's Dilemma concept from game theory. We discuss the evolution of behaviour under selection pressures which amount to the prisoner's dilemma, with particular attention to the case of evolution in spatially structured environments. The Voter model provides a particularly exacting scenario for the evolution of cooperation, in which most existing results about the evolution of "tit-for-tat" and related simple strategies fail to hold. Despite this, the model can exhibit complex cooperative behaviour including cycles of invasion. We 2006show that these cycles give rise to self-similar (fractal) spatially structured populations. Surprisingly, it pays each member of such a cycle to be minimally aggressive towards the species it can invade.

1999

Functional Gradient Techniques for Combining Hypotheses.

Mason,L., Baxter,J., Bartlett,P. and Frean, M.R. (1999)

Chapter 12 in Advances in Large Margin Classifiers , Smola, Bartlett, Scholkopf and Schuurmans (eds.), MIT Press

Abstract: see "Boosting algorithms as gradient descent in function space" above.

Mason_etal99B.ps.gz

Real-valued evolutionary optimization using a flexible probability estimator

Gallagher,M., Frean, M.R. and Downs,T. (1999).

Proceedings of the Genetic and Evolutionary Computation Conference, Orlando, Florida, July 1999. Volume 1, 840-846

Population-based Incremental Learning (PBIL) is an abstraction of a genetic algorithm, which solves optimization problems by explicitly constructing a probabilistic model of the promising regions of the search space. At each iteration the model is used to generate a population of candidate solutions and is itself modified in response to these solutions. Through the extension of PBIL to real-valued search spaces, a more powerful and general algorithmic framework arises which enables the use of arbitrary probability density estimation techniques in evolutionary optimization. To illustrate the usefulness of this framework, we propose and implement an evolutionary algorithm which uses a finite Adaptive Gaussian mixture model density estimator (etc)...

Catastrophic forgetting in simple networks: an analysis of the pseudorehearsal solution.

Frean, M.R. and Robins,A.V. (1999)

Network: Computation in Neural Systems , 10, 227-236

Catastrophic forgetting is a major problem for sequential learning in neural networks. One very general solution to this problem, known as `pseudorehearsal', works well in practice for non-linear networks but has not been analysed before. This paper formalises pseudorehearsal in linear networks. We show the method can fail in low dimensions but is guaranteed to succeed in high dimensions under fairly general conditions. In this case an optimal version of the method is equivalent to a simple modification of the `delta rule'.

FreanRobins99.pdf

1998

A simple cost function for boosting.

Frean, M.R. and Downs,T. (1998)

Technical report, Dept. of Computer Science and Electrical Engineering, University of Queensland.

For two-class classification problems the boosting algorithm "AdaBoost" is equivalent to minimizing the cost-function: (..see the postscript...). Using this we show that the D variables are unnecessary for AdaBoost to work, and that `re-boosting' of previous classifiers is straightforward.

FreanDowns98.pdf

Local learning algorithms for sequential learning tasks in neural networks.

Robins,A.V. and Frean, M.R. (1998)

Journal of Advanced Computational Intelligence, 2, (6), 107 - 111

In this paper we explore the concept of sequential learning and the efficacy of global and local neural network learning algorithms on a sequential learning task. Pseudorehearsal, a method developed by Robins (1995) to solve the catastrophic forgetting problem which arises from the excessive plasticity of neural networks, is significantly more effective than other local learning algorithms for the sequential task. We further consider the concept of local learning and suggest that pseudorehearsal is so effective because it works directly at the level of the learned function, and not indirectly on the representation of the function within the network. We also briefly explore the effect of local learning on generalisation within the task.

RobinsFrean98A.ps.gz

Proceedings of the 8th Australian Conference on Neural Networks (ACNN'98)

Editors: Downs,T., Frean,M.R. and Gallagher,M. (1998).

Catastrophic forgetting and `pseudorehearsal' in linear networks.

Frean, M.R. and Robins,A.V. (1998)

Proceedings of the 8th Australian Conference on Neural Networks (ACNN'98), University of Queensland, Brisbane, 11-13 Feb 1998

Abstract: see our journal paper in Network: Computation in Neural Systems (above), which is better.

Learning and generalization in a stable network.

Robins,A.V. and Frean, M.R. (1998)

In Progress in Connectionist-Based Information Systems: Proc. 1997 Conference on Neural Information Processing and Intelligent Information Systems, Kasabov et. al. (Eds), Singapore: Springer-Verlag, pp. 314-317

Most neural networks suffer from excessive plasticity: the learning of new information interferes with information already stored in the network. In this paper we review the pseudorehearsal solution to this problem, proposed by Robins (1995). By localising the changes to the function learned by the network pseudorehearsal allows networks to be stable in the face of new learning, successfully integrating both new and previously learned information. In this paper we explore the impact that this mechanism has on the ability of the network to generalise.

RobinsFrean98B.ps.gz

before 1998

The evolution of degrees of cooperation.

Frean, M.R. (1996)

Journal of Theoretical Biology , 182, pages 549-559

The Prisoner's Dilemma has been widely studied as a model for the evolution of cooperation, and most of this work has dealt with agents who either cooperate or not. In this paper we look at the consequences of allowing agents to have intermediate levels of cooperation, and to update these levels over time. The familiar strategy of ``tit for tat'' emerges as a robust mode of behaviour, yet there are important differences between this case and that of ``all or nothing'' cooperation.

Frean96.pdf

The Prisoner's Dilemma without synchrony

Frean, M.R. (1994)

Proceedings of the Royal Society of London (B) , 257, pages 75 - 79

There are many situations in which biological organisms cooperate despite obvious incentives to do otherwise. Such situations are commonly modelled using a paradigm known as the Prisoner's Dilemma. In this way cooperative behaviour has previously been shown to emerge in a model population of strategies. If players can make probabilistic choices taking into account their co-player's previous action, a strategy known as `Generous Tit For Tat' dominates the long-term behaviour of such a population. If they can also take into account their own previous action, a strategy of `win stay, lose shift' dominates instead. These models assumed that participants make their decisions in synchrony, which seems improbable in many biological situations. Here we show that the timing of decisions is critical in determining which strategy emerges in the long run. If individuals make their decisions at different times, neither of the above strategies survives given the usual payoffs. In the former case Generous Tit For Tat succumbs to inveterate defectors, and in the latter a new strategy takes over. This `firm but fair' strategy is retaliatory yet highly cooperative. In particular, continued exploitation of a sucker is no longer a successful behaviour.

Frean94.pdf

Fantasy engines and brain theory

Frean, M.R. (1996)

Technical Report, A.I.Memo 34-96-1, Department of Computer Science, Otago University.

A 'thermal' perceptron learning rule.

Frean, M.R. (1992) Neural Computation, 4, (6), 946 - 957

The thermal perceptron is a simple extension to Rosenblatt's perceptron learning rule for training individual linear threshold units. It finds stable weights for non-separable problems as well as separable ones. Experiments indicate that if a good initial setting for a temperature parameter, $T_{0}$, has been found, then the thermal perceptron outperforms the Pocket algorithm and methods based on gradient descent. The learning rule stabilises the weights (learns) over a fixed training period. For separable problems it finds separating weights much more quickly than the usual rules.

Frean92-thermal-perceptron.pdf

The Upstart Algorithm: a method for constructing and training feed-forward neural networks.

Frean, M.R. (1990) Neural Computation, 2 (2), 189 - 209.

A general method for building and training multi-layer perceptrons composed of linear threshold units is proposed. A simple recursive rule is used to build the net's structure by adding units as they are needed, while a modified Perceptron algorithm is used to learn the connection strengths. Convergence to zero errors is guaranteed for any Boolean classification on patterns of binary variables. Simulations suggest that this method is efficient in terms of the numbers of units constructed, and the networks it builds can generalise over patterns not in the training set.

Frean90-Upstart-Algorithm.pdf (digitized from a scan).

Small Nets and Short Paths: Optimising Neural Computation.

Frean, M.R. (1990) PhD Thesis, University of Edinburgh, Scotland. 1990

This thesis explores two aspects of optimisation in neural network research. 1. The question of how to find the optimal feed-forward neural network architecture for learning a given binary classification is addressed. The so-called constructive approach is reviewed whereby intermediate, hidden units are built as required for the particular problem. Current constructive algorithms are compared, and three new methods are introduced. One of these, the Upstart algorithm, in shown to outperform all other constructive algorithms of this type. This work led on to the ancillary problem of finding a satisfactory procedure for changing the weights values of an individual unit in a network. The new thermal perceptron rule is described and shown to compare favorably with its competitors. Finally, the spectrum of possible learning rules is surveyed. 2. Neurobiologically inspired algorithms for mapping between spaces of different dimensionality are applied to a classic optimisation problem, the Travelling Salesman Problem. Two new methods are described that can tackle the general symmetric form of the TSP, thus overcoming the restriction on other neural networks to the geometric case.

Frean_thesis.ps.gz