Sensitivity Analysis for Decision Boundaries
Engelbrecht, AP. 1999.
Neural Processing Letters, 10(3):253-266, Kluwer Academic Publishers

Download this publication.

Abstract:

A novel approach is presented to visualize and analyze decision boundaries for feedforward neural networks. First order sensitivity analysis of the neural network output function with respect to input perturbations is used to visualize the position of decision boundaries over input space. Similarly, sensitivity analysis of each hidden unit activation function reveals which boundary is implemented by which hidden unit. The paper shows how these sensitivity analysis models can be used to better understand the data being modelled, and to visually identify irrelevant input and hidden units.

Back to top

Training Product Unit Neural Networks
Engelbrecht, AP. Ismail, A. 1999.
Stability and Control: Theory and Applications, 2(1/2):59-74

Download this publication from the Neural Networks and Swarm Intelligence groups.

Abstract:

Product units enable a neural network to form higher-order combinations of inputs, having the advantages of increased information capacity and smaller network architectures. Training product unit networks using gradient descent, or any other local optimization algorithm, is difficult, because of an increased number of local minima and increased chances of network paralysis. This paper illustrates the shortcomings of gradient descent optimization when faced with product units, and presents a comparative investigation into global optimization algorithms for the training of product unit neural networks. A comparison of results obtained from particle swarm optimization, genetic algorithms, LeapFrog and random search show that these global optimization algorithms successfully train product unit neural networks. Results of product unit neural networks are also compared to results obtained from using gradient optimization with summation units.

Back to top

Using the Taylor Expansion of Multilayer Feedforward Neural Networks
Engelbrecht, AP. 2000.
South African Computer Journal, 26:181-189

Download this publication from the Neural Networks group.

Abstract:

The Taylor series expansion of continuous functions has shown - in many fields - to be an extremely powerful tool to study the characteristics of such functions. This paper illustrates the power of the Taylor series expansion of multilayer feedforward neural networks. The paper shows how these expansions can be used to investigate positions of decision boundaries, to develop active learning strategies and to perform architecture selection.

Back to top

Cooperative Learning in Neural Networks using Particle Swarm Optimizers
van den Bergh, F. Engelbrecht, AP. 2000.
South African Computer Journal, 26:84-90

Unavailable for download.

Abstract:

This paper presents a method to employ particle swarms optimizers in a cooperative configuration. This is achieved by splitting the input vector into several sub-vectors, each which is optimized cooperatively in its own swarm. The application of this technique to neural network training is investigated, with promising results.

Back to top

Sensitivity Analysis for Selective Learning by Feedforward Neural Networks
Engelbrecht, AP. 2001.
Fundamenta Informaticae, 45(1):295-328, IOS Press

Download this publication from the Neural Networks group.

Abstract:

Research on improving the performance of feedforward neural networks has concentrated mostly on the optimal setting of initial weights and learning parameters, sophisticated optimization techniques, architecture optimization, and adaptive activation functions. An alternative approach is presented in this paper where the neural network dynamically selects training patterns from a candidate training set during training, using the network's current attained knowledge about the target concept. Sensitivity analysis of the neural network output with respect to small input perturbations is used to quantify the informativeness of candidate patterns. Only the most informative patterns, which are those patterns closest to decision boundaries, are selected for training. Experimental results show a significant reduction in the training set size, without negatively influencing generalization performance and convergence characteristics. This approach to selective learning is then compared to an alternative where informativeness is measured as the magnitude in prediction error.

Back to top

A New Pruning Heuristic Based on Variance Analysis of Sensitivity Information
Engelbrecht, AP. 2001.
IEEE Transactions on Neural Networks, 12(6):1386-1399

Download this publication from the Neural Networks group.

Abstract:

Architecture selection is a very important aspect in the design of neural networks to optimally tune performance and computational complexity. Sensitivity analysis has been used successfully to prune irrelevant parameters from feedforward neural networks. This paper presents a new pruning algorithm that uses sensitivity analysis to quantify the relevance of input and hidden units. A new statistical pruning heuristic is proposed, based on variance analysis, to decide which units to prune. The basic idea is that a parameter with a variance in sensitivity not significantly different from zero, is irrelevant and can be removed. Experimental results show that the new pruning algorithm correctly prunes irrelevant input and hidden units. The new pruning algorithm is also compared with standard pruning algorithms.

Back to top

Supervised Training Using an Unsupervised Approach to Active Learning
Engelbrecht, AP. Brits, R. 2002.
Neural Processing Letters, 15:247-260, Kluwer Academic Publishers

Download this publication from the Neural Networks group.

Abstract:

Active learning algorithms allow neural networks to dynamically take part in the selection of the most informative training patterns. This paper introduces a new approach to active learning, which combines an unsupervised clustering of training data with a pattern selection approach based on sensitivity analysis. Training data is clustered into groups of similar patterns based on Euclidean distance, and the most informative pattern from each cluster is selected for training using the sensitivity analysis incremental learning algorithm in \cite{eng99d}. Experimental results show that the clustering approach improves on standard active learning as presented in \cite{eng99d}.

Back to top

Learning to Play Games using a PSO-based Competitive Learning Approach
Messerschmidt, L. Engelbrecht, AP. 2002.
4th Asia-Pacific Conference on Simulated Evolution and Learning

Download this publication from the Swarm Intelligence group.

Abstract:

A new competitive approach is developed for learning agents to play two-agent games. This approach uses particle swarm optimizers (PSO) to train neural networks to predict the desirability of states in the end nodes of a game tree. The new approach is applied to the TicTacToe game, and compared to the performance of the evolutionary approach developed by Fogel. The results show that the new PSO-based approach outperforms the evolutionary approach.

Back to top

A Cooperative Approach to Particle Swarm Optimisation
van den Bergh, F. Engelbrecht, AP. 2004.
IEEE Transactions on Evolutionary Computation, 8(3):225-239, IEEE

Unavailable for download.

Abstract:

The particle swarm optimizer (PSO) is a stochastic, population-based optimization technique that can be applied to a wide range of problems, including neural network training. This paper presents a variation on the traditional PSO algorithm, called the cooperative particle swarm optimizer, or CPSO, employing cooperative behavior to significantly improve the performance of the original algorithm. This is achieved by using multiple swarms to optimize different components of the solution vector cooperatively. Application of the new PSO algorithm on several benchmark optimization problems shows a marked improvement in performance over the traditional PSO.

Back to top

Evolving intelligent game-playing agents
Franken, N. Engelbrecht, AP. 2004.
South African Computer Journal, 32:44-52

Unavailable for download.

Back to top

Social Networks as a Task Allocation Tool for Multi-Robot Teams
Rodic, D. Engelbrecht, AP. 2004.
South African Computer Journal

Unavailable for download.

Abstract:

The last decade saw a renewed interest in the field of robotics research and a shift in research focus. In the eighties and early nineties, the focus of robotic research was on finding optimal robot architectures, often resulting in non-cognitive, insect-like entities.In recent years, the processing power available to embedded autonomous agents (robots) has improved and this development has allowed for more complex robot architectures. The focus has shifted from single robot to multi-robot teams. The key to the full utilisation of multi-robot teams lies in coordination. Although a robot is a special case of an agent, many existing multiagent coordination techniques could not be directly ported to multi-robot teams. In this paper, we overview mainstream multi-robot coordination techniques and propose a new approach to coordination, based on models of organisational sociology, namely social networks. The social network based approach relies on trust and kinship relationships,modified for use in heterogeneous multi-robot teams. The proposed task allocation mechanism is then tested using two approaches: the multi-robot team task allocation simulation and a more realistic coordination problem in simulated robot environments. For the purpose of these two tests, two robotic simulators were developed. The social networks based task allocation algorithm has performed according to expectations and the obtained results are very promising. Although it is applied to simulated multi-robot teams, the proposed coordination model is not robot specific, but can also be applied to any multi-agent system without major modifications.

Back to top

SIGT: Synthetic Image Generation Tool for Clustering Algorithms
Salman, A. Omran, MG. Engelbrecht, AP. 2005.
International Journal on Graphics, Vision and Image Processing , 2:33--44

Download this publication from the Swarm Intelligence group.

Abstract:

A new automatic image generation tool is proposed in this paper tailored specifically for verification and comparison of different image clustering algorithms. The tool can be used to produce different images (in raw format) with different criteria based on user specification. The user specifies the number of clusters to be included in the image along with the probability distribution that govern set of points that belong to different clusters. On the other hand, the tool can be used to verify the degree of approximation a new algorithm has been able to achieve compared to the original image. This allows for a scientific confident comparison between any new algorithm and existing algorithms. The features of the tool is demonstrated with reference to the well-known K-means clustering algorithm.

Back to top

Particle Swarm Optimization Method for Image Clustering
Omran, MG. Engelbrecht, AP. Salman, A. 2005.
International Journal on Pattern Recognition and Artificial Intelligence

Download this publication from the Swarm Intelligence group.

Abstract:

An image clustering method that is based on the particle swarm optimizer (PSO) is developed in this paper. The algorithm finds the centroids of a user specified number of clusters, where each cluster groups together similar image primitives. To illustrate its wide applicability, the proposed image classifier has been applied to synthetic, MRI and satellite images. Experimental results show that the PSO image classifier performs better than a conventional image classifier (namely, K-means) in all measured criteria. The influence of different values of PSO control parameters is also illustrated.

Back to top

A PSO-Based Color Image Quantizer
Omran, MG. Salman, A. Engelbrecht, AP. 2005.
Informatica Journal, 29(3):263-271

Not Available For Download.

Abstract:

Not Available

Back to top

A PSO-Based End-Member Selection Method for Spectral Unmixing of Multispectral Satellite Images
Omran, MG. Salman, A. Engelbrecht, AP. 2005.
International Journal of Computational Intelligence, 2(2):124-132

Not Available For Download.

Abstract:

Not Available

Back to top

Analysis of PSO Approaches to Co-evolve IPD Strategies
Franken, N. Engelbrecht, AP. 2005.
IEEE Transactions on Evolutionary Computation, (9)6:562-579

Not Available For Download.

Abstract:

Not Available

Back to top

Composing linear evaluation functions from observable features
Duminy, WH. Engelbrecht, AP. 2005.
SACJ, No. 35., 2005, 48-58, South African Computing Journal

Download this publication from the Games group.

Abstract:

Discovering useful knowledge is a problem that remains elusive, even for publicly solved games such as Checkers. This paper presents a formal language F that represents knowledge that can be used by two-player board game agents. The language assumes minimal domain knowledge and as basic symbols it employs information that can be obtained directly from the observation of a game position. The language operators can be used to construct and manipulate knowledge within the context of an evaluation function.

Back to top

Dynamic Clustering using Particle Swarm Optimization with Application in Image Segmentation
Omran, MG. Salman, A. Engelbrecht, AP. 2006.
Pattern Analysis and Applications Journal, 8(4):332-344

Not Available For Download.

Abstract:

Not Available

Back to top

A Study of Particle Swarm Optimization Particle Trajectories
van den Bergh, F. Engelbrecht, AP. 2005.
Information Sciences Journal

Download this publication from the Swarm Intelligence group.

Abstract:

Particle swarm optimization (PSO) has shown to be an efficient, robust and simple optimization algorithm. Most of the PSO studies are empirical, with only a few theoretical analyses that concentrate on understanding particle trajectories. These theoretical studies concentrate mainly on simplified PSO systems. This paper overviews current theoretical studies, and extend these studies to investigate particle trajectories for general swarms to include the influence of the inertia term. The paper also provides a formal proof that each particle converges to a stable point. An empirical analysis of multi-dimensional stochastic particles is also presented. Experimental results are provided to support the conclusions drawn from the theoretical findings.

Back to top

Optimised Coverage of Non-self with Evolved Lymphocytes in Artificial Immune Systems
Graaff, A. Engelbrecht, AP. 2006.
International Journal of Comutational Intelligence Research, 2(2):127-150.

Not Available For Download.

Abstract:

Not Available

Back to top

A Comparative Study of Sample Selection Methods for Data Mining
Lutu, PEN. Engelbrecht, AP.2006.
ARIMA/SACJ, 36:69-85

Not Available For Download.

Abstract:

Not Available

Back to top

Particle Swarms for Equality-Constrained Optimization
Paquet, U. Engelbrecht, AP. 2006.
IOS Press, 76:1-24

Not Available For Download.

Abstract:

Not Available

Back to top

Genetic Algorithms for the Structural Optimisation of Learned Polynomial Expressions
Potgieter, G. Engelbrecht, AP. 2007.
Applied Mathematics and Computation, 186:1441-1466

Not Available For Download.

Abstract:

Not Available

Back to top

Application of Fuzzy Logic in Topology Design of Distributed Local Area Networks
Khan, SA. Engelbrecht, AP. 2005.
Information Sciences

Unavailable for download.

Back to top

Empirical Analysis of Self-Adaptive Differential Evolution
Omran, M. Engelbrecht, AP. Ayed, S. 2007.
European Journal of Operational Research, 183:785-804

Not Available For Download.

Abstract:

Not Available

Back to top

An Overview of Clustering Methods
Omran, M. Engelbrecht, AP. Syed, S. 2007.
Intelligent Data Analysis, 11(7):584-605

Not Available For Download.

Abstract:

Not Available

Back to top

Empirical Analysis of Using Neighborhood Topologies with Differential Evolution
Omran, M. Engelbrecht, AP. Zraibi, M. Omran, E. 2008.
Advances in Computer Science and Engineering Journal, 1(3):189-222

Not Available For Download.

Abstract:

Not Available

Back to top

Social Networks in Simulated Multi-Robot Environment
Rodic, D. Engelbrecht, AP. 2008.
International Journal of Intelligent Computing and Cybernetics, 1(1):110-127

Not Available For Download.

Abstract:

Not Available

Back to top

Evolving Model Trees for Mining Data Sets with Cintinuous-Valued Classes
Potgieter, G. Engelbrecht, AP. 2008.
Expert Systems with Applications, 35:1513-1532

Not Available For Download.

Abstract:

Not Available

Back to top

Fuzzy Hybrid Simulated Annealing Algorithms for Topology Design of Switched Local Area Networks
Engelbrecht, AP. 2008.
Soft Computing, 13(1):45-61

Not Available For Download.

Abstract:

Not Available

Back to top

Particle Swarm Optimization
Dorigo, M. Montes de Oca, MA. Engelbrecht, AP. 2008.
Scholarpedia, 3(11):1486

Not Available For Download.

Abstract:

Not Available

Back to top

DNA Sequence Optimization Based Continuous Particle Swarm Optimization for Reliable DNA Computing and DNA Nanotechnology
Khalid, NK. Ibrahim, Z. Kurniawan, TB. Khalid, M. Engelbrecht, AP. 2008.
Journal of Computer Science, 4(11):942-950

Not Available For Download.

Abstract:

Not Available

Back to top

Bare Bones Differential Evolution
Omran, M. Engelbrecht, AP. Salman, A. 2009.
European Journal of Operational Research, 196(1):128-139

Not Available For Download.

Abstract:

Not Available

Back to top

Function Minimization in DNA Sequence Design based on Continuous Particle Swarm Optimization
Khalid, NK. Ibrahim, Z. Kurniawan, TB. Khalid, M. Sarmin, NH. Engelbrecht, AP. 2009.
Innovative Computing, Information and Control Express Letters, 3(1):27-32

Not Available For Download.

Abstract:

Not Available

Back to top

Metaheuristics for the multi-objective FJSP with sequence-dependent set-up times, auxiliary resources and machine down time
Grobler, J. Engelbrecht, AP. Kok, S. Yadavalli, S. 2009.
Annals of Operations Research

Not Available For Download.

Abstract:

Not Available

Back to top

A Novel Particle Swarm Niching Technique based on Extensive Vector Operations
Schoeman, IL. Engelbrecht, AP. 2009.
Natural Computing

Not Available For Download.

Abstract:

Not Available

Back to top

A Decision Rule-based Method for Feature Selection in Predictive Mining
Lutu, PEN. Engelbrecht, AP. 2010.
Expert Systems with Aplications, 37(1):602-609

Not Available For Download.

Abstract:

Not Available

Back to top

Robotic Architectures: A Review
Mtshali, M. Engelbrecht, AP. 2010.
Defence Science Journal

Not Available For Download.

Abstract:

Not Available

Back to top

A Polar Coordinate Particle Swarm Optimiser
Matthysen, W. Engelbrecht, AP. 2011.
Applied Soft Computing, 11(1):1322-1339

Not Available For Download.

Abstract:

Not Available

Back to top

Clustering Data in an Uncertain Environment using an Artificial Immune System
Graaff, AJ. Engelbrecht, AP. 2011.
Pattern Recognition Letters, 32(2):342-351

Not Available For Download.

Abstract:

Not Available

Back to top

Using Sequential Deviation to Dynamically Determine the Number of Clusters Found by a Local Network Neighbourhood Artificial Immune System
Graaff, AJ. Engelbrecht, AP. 2011.
Applied Soft Computing, 11:2698-2713

Not Available For Download.

Abstract:

Not Available

Back to top