Collaborative Adaptive Filters

Nature of signals — The majority of real world signals are represented by the areas marked '?'

Signal modality characterisation allows us to understand more about the nature of the signals we are investigating. While some signal characteristics are well defined many signals do not fall neatly into these categories.

By using adaptive filters in a prediction configuration we can compare the output of filters designed for different inputs. This allows us to determine which algorithm has the best prediction of the signal.

This solution is simple but the algorithms do not co-operate, to create an online signal modality characterisation solution with synergy between the filters the filters feed into a mixing algorithm which is also adaptive.

Using a convex combination of the filters guarantees the existence and uniqueness of a solution. The convex mixing parameter is adapted using a gradient descent update:

\[\begin{align*} \lambda(k+1) =& \lambda(k) - \mu_\lambda\nabla_\lambda E(k)_{|\lambda=\lambda(k)}\\ =& \lambda(k) - \frac{\mu_\lambda}{2}\frac{\partial e^2(k)}{\partial\lambda(k)}\\ =& \lambda(k) + \mu_\lambda e(k)\big(y_1(k) - y_2(k)\big) \end{align*}\]

This solution can provide improved performance over the individual constituent filters;
The mixing algorithm can give an indication of which filter is currently responding to the input signal most effectively;
By appropriate selection of the algorithms the mixing algorithm can adapt according to fundamental properties of the input signal.

Generic hybrid filter — General hybrid filter structure.

Performance of linear hybrid filter — Example of the performance of a hybrid filter combining the LMS and GNGD algorithms.

By tracking the mixing parameter rather than the performance of the filter we can see which of the subfilters is being favoured by the hybrid filter. We can also combine several hybrid filters to gain a more complete picture of the nature of the signal under investigation.

Comparison of linear and nonlinear signals — Comparison of mixing parameter behaviour for linear and nonlinear signals.

Alternating signals — Signal alternating from linear to nonlinear then linear to sparse, blue line - sparse hybrid filter, red line - nonlinear hybrid filter.

2D mixing parameters — 2D representation of both filters, linear sections represented in green, nonlinear in red and sparse in blue.

Example Applications

Speech

Unvoiced Sounds
- Most consonants are unvoiced
- Result in a noise like waveform
- Linear autoregressive models are adequate for unvoiced sounds
Voiced Sound
- A voiced sound is one where the vocal folds are vibrating
- Periodic in nature with periodic excitation source
- All vowels and certain consonants are voiced speech
- Voiced sounds require a nonlinear model
Nasals
- Nasals are sounds like \n\or \m\
- For nasals or certain vowels a chaotic model is appropriate

Comparing results for hybrid filters for nonlinear and sparse shows a high degree of correlation.

\s\ sounds are more linear in nature and voiced sounds are indicated by the regions where \(\lambda\)exhibits a spiky behaviour [1].

Example speech signal. Middle: Mixing parameter for nonlinear hybrid filter. Bottom: Mixing parameter for sparse hybrid filter.

EEG - Epilepsy

Epileptic signals fundamentally change the nature of brain activity. Using hybrid filters for both nonlinearity and sparsity we can build a feature map of the epileptic signal [2], [3].

Example epilepsy signals — Top: Two example EEG signals showing the onset of epileptic seizures. Middle: Mixing parameters for nonlinear and sparse hybrid filters. Bottom: Feature maps for the combination of the two hybrid filters.

EEG - Consciousness States

Identification of brain consciousness states is a important area of EEG analysis. There are obvious legal implications to declaring a patient brain dead. Normally this assessment requires several invasive medical procedures.

A trail test has been performed on EEG data for 34 patients, 17 in a coma and 17 considered to be brain dead. The recordings were taken in the intensive care unit of a hospital leading to high levels of noise generated by other monitoring machines.

Nonlinearity showed clear differences but sparsity was less obvious, combining the two to produce a feature map gave a clear difference [4].

EEG nonlinear hybrid filter — Average performance of the nonlinear hybrid filter for the coma and quasi-brain dead patients.

EEG sparse hybrid filter — Average performance of the sparse hybrid filter for the same patients.

EEG sparse nonlinear feature map — Feature maps for the combination of the two hybrid filters.

Wind

Wind vectors \(v(k)\) can be represented in the complex domain as

\[v(k) = |v(k)|e^{j\theta(k)} = v_E(k) + jv_N(k)\]

To compare the nature of the complex signals split- & fully-complex subfilters were used to predict the wind data.

Looking at the wind data from an urban area over the course of 24 hours, the wind can be considered more fully-complex in nature. The first & last samples are more unstable in nature, these areas indicate from 2pm until 6pm and from 8am until 2pm. During these periods the wind would fluctuate more than during the calm period late at night & early in the morning [2], [5]

Polar representation of wind — Polar wind Profile.

Complex wind representation — Top: Real and imaginary parts of a wind vector. Bottom: mixing parameter for split- and fully-complex hybrid filter.

Radar

Radar can be represented in the complex domain and has been shown to be fully complex when the target is in the beam.

Alternating high (turbulent) sea state and low (calm) sea state we can see that the high sea state was predominantly fully-complex and the low sea state split complex [3].

Example radar signals — Mixing parameter for split vs fully complex hybrid filter with radar signal alternating between turbulent and calm sea states.

Code

Hybrid filter code can be found here: hybrid-filters
This code contains several example filters and benchmark signals and can be customised to include any adaptive filter of your choosing.

References

Exploiting Nonlinearity in Adaptive Signal Processing

P. Vayanos, M. Chen, B. Jelfs, and D. P. Mandic

In Advances in Nonlinear Speech Processing, M. Chetouani et al. (eds), Springer Berlin Heidelberg, 2007, pp. 57–77.

abstract doi pdf code

Quantitative performance criteria for the analysis of machine learning architectures and algorithms have been long established. However, the qualitative performance criteria, e.g., nonlinearity assessment, are still emerging. To that end, we employ some recent developments in signal characterisation and derive criteria for the assessment of the changes in the nature of the processed signal. In addition, we also propose a novel online method for tracking the system nonlinearity. A comprehensive set of simulations in both the linear and nonlinear settings and their combination supports the analysis.
Characterisation of Signal Modality: Exploiting Signal Nonlinearity in Machine Learning and Signal Processing

B. Jelfs, S. Javidi, P. Vayanos, and D. Mandic

Journal of Signal Processing Systems, 2010, vol. 61, no. 1, pp. 105–115.

abstract doi pdf code

A novel method for online tracking of the changes in the nonlinearity within both real-domain and complex-valued signals is introduced. This is achieved by a collaborative adaptive signal processing approach based on a hybrid filter. By tracking the dynamics of the adaptive mixing parameter within the employed hybrid filtering architecture, we show that it is possible to quantify the degree of nonlinearity within both real- and complex-valued data. Implementations for tracking nonlinearity in general and then more specifically sparsity are illustrated on both benchmark and real world data. It is also shown that by combining the information obtained from hybrid filters of different natures it is possible to use this method to gain a more complete understanding of the nature of the nonlinearity within a signal. This also paves the way for building multidimensional feature spaces and their application in data/information fusion.
Collaborative Adaptive Filters for Online Knowledge Extraction and Information Fusion

B. Jelfs, P. Vayanos, S. Javidi, V. S. L. Goh, and D. Mandic

In Signal Processing Techniques for Knowledge Extraction and Information Fusion, D. Mandic et al. (eds), Springer, 2008, vol. 61, no. 1, pp. 3–21.

abstract doi pdf code

We present a method for extracting information (or knowledge) about the nature of a signal. This is achieved by employing recent developments in signal characterisation for online analysis of the changes in signal modality. We show that it is possible to use the fusion of the outputs of adaptive filters to produce a single collaborative hybrid filter and that by tracking the dynamics of the mixing parameter of this filter rather than the actual filter performance, a clear indication as to the nature of the signal is given. Implementations of the proposed hybrid filter in both the real R and the complex C domains are analysed and the potential of such a scheme for tracking signal nonlinearity in both domains is highlighted. Simulations on linear and nonlinear signals in a prediction configuration support the analysis; real world applications of the approach have been illustrated on electroencephalogram (EEG), radar and wind data.
Modelling of Brain Consciousness Based on Collaborative Adaptive Filters

L. Li, Y. Xia, B. Jelfs, J. Cao, and D. P. Mandic

Neurocomputing, 2012, vol. 76, no. 1, pp. 36–43.

abstract doi pdf code

A novel method for the discrimination between discrete states of brain consciousness is proposed, achieved through examination of nonlinear features within the electroencephalogram (EEG). To allow for real time modes of operation, a collaborative adaptive filtering architecture, using a convex combination of adaptive filters is implemented. The evolution of the mixing parameter within this structure is then used as an indication of the predominant nature of the EEG recordings. Simulations based upon a number of different filter combinations illustrate the suitability of this approach to differentiate between the coma and quasi-brain-death states based upon fundamental signal characteristics.
An Adaptive Approach for the Identification of Improper Complex Signals

B. Jelfs, D. P. Mandic, and S. C. Douglas

Signal Processing, 2012, vol. 92, no. 2, pp. 335–344.

abstract doi pdf code

A real-time approach for the identification of second-order noncircularity (improperness) of complex valued signals is introduced. This is achieved based on a convex combination of a standard and widely linear complex adaptive filter, trained by the corresponding complex least mean square (CLMS) and augmented CLMS (ACLMS) algorithms. By providing a rigorous account of widely linear autoregressive modelling the analysis shows that the monitoring of the evolution of the adaptive convex mixing parameter within this structure makes it possible to both detect and track the complex improperness in real time, unlike current methods which are block based and static. The existence and uniqueness of the solution are illustrated through the analysis of the convergence of the convex mixing parameter. The analysis is supported by simulations on representative datasets, for a range of both proper and improper inputs.