Actions involving the hand require positioning multiple fingers each with multiple degrees of freedom. However, we regularly perform complex actions without the need to consciously consider the position of each and every joint. Understanding the neural basis of finger coordination has many uses in rehabilitation and prosthetics.
Small Synergistic Movement in Pianists
The aim of this study was to study the differences in finger movement between experienced and novice pianists with the aim of understanding more about how the fingers move:
Individually - the joints
Together - simultaneous movement
Independently - sequential movement
Movement of the hands during piano playing was recorded using a data glove with 14 sensors measuring finger flexion and abduction. To study how the movement of the fingers differed between the experienced and novice players the concept of movement synergies was used.
Piano playing while wearing a data glove.
Muscle Synergies:
Fundamental patterns of movement
Serve as building blocks for more complicated movements
Reduce the complexity (degrees of freedom) to be controlled
Time-varying principal component analysis for movement synergy extraction.
Results showed that a small number of components accounted for the majority of the variance in the data but that these components are common across the groups. Classification results showed accurate classification was obtained from components accounting for only small amount of variance primarily related to the abduction of the fingers [1].
The variance accounted for by the first 10 components.
Finger
Accuracy (%)
No. Components
VAF
Thumb
93.68
15/253
1.38
Index
94.81
14/212
0.11
Middle
93.40
17/409
2.70
Ring
96.52
18/115
5.97
Little
98.29
17/175
0.96
Classification results showing the accuracy, number of components and variance accounted for, for each of the fingers.
Muscle Synergy Extraction
The muscles which control the fingers are primarily located in the forearm and for each gesture we perform multiple muscles are required. As with the movement we can also use synergies to represent the common patterns of muscle activity which serve as the building blocks to produce detailed movements associated with the control of the fingers.
Nonnegative matrix factorization.
Muscle synergy extraction can be performed using nonnegative matrix factorization. That is for a data matrix \(Y\in\mathbb{R}^{X\times N}\) find nonnegative matrices \(W\in\mathbb{R}^{X\times K}\) and \(H\in\mathbb{R}^{K\times N}\) via $$\min_{W,H} D(Y||WH)\quad\text{ subject to}\quad W\geq 0,\quad H\geq 0$$ where \(D(\cdot)\) is a measure of goodness of fit
To deal with the complexity of the neuromuscular system and extend beyond the variance accounted for as a measure we proposed a method based on using fuzzy entropy as a similarity measure [2]. Examples of the differences between the synergies extracted using 16 channels of EMG recorded from the arm show compared to an alternating least squares NMF our method places greater significance on a smaller number of muscles.
Example synergies for a value of k=3 for 2 different subjects.
Code
Fuzzy entropy NMF code can be found here: Entropy_NMF
References
Recruitment of Small Synergistic Movement Makes a Good Pianist
B. Jelfs, S. Zhou, B. K. Y. Wong, C. Tin, and R. H. M. Chan
In Proc. Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2015, pp. 242–245.
Time-varying synergies from kinematic data can be used to discern fundamental patterns of movement. We show through simultaneous extraction of synergies from both novice and experienced pianists that movement common to both groups can be identified. The extracted synergies successfully allow for the majority of the variability of the data to be accounted for by a limited number of components. Furthermore, classification of the weightings representing the recruitment of each of the synergies accurately distinguishes between the piano playing of the two groups of subjects. However, the major differences between the two groups lie not in the synergies representing the majority of the variance of the data but in the recruitment of smaller synergies.
Fuzzy Entropy Based Nonnegative Matrix Factorization for Muscle Synergy Extraction
B. Jelfs, L. Li, C. Tin, and R. H. M. Chan
In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, 2016, pp. 739–743.
The concept of muscle synergies has proven to be an effective method for representing patterns of muscle activation. The number of degrees of freedom to be controlled are reduced while also providing a flexible platform for producing detailed movements using synergies as building blocks. It has previously been shown that small components of movement are crucial to producing precise and coordinated movement. Methods which focus on the variance of the data make it possible to overlook these small components in the synergy extraction process. However, algorithms which address the inherent complexity in the neuromuscular system are lacking. To that end we propose a new nonnegative matrix factorization algorithm which employs a cross fuzzy entropy similarity measure, thus, extracting muscle synergies which preserve the complexity of the recorded muscular data. The performance of the proposed algorithm is illustrated on representative EMG data.
