Theis, Fabian J. and Bauer, Ch. and Lang, Elmar W. (2002) Comparison of maximum entropy and minimal mutual information in a nonlinear setting. SIGNAL PROCESSING, 82 (7): PII S0165-. pp. 971-980. ISSN 0165-1684
Full text not available from this repository. (Request a copy)Abstract
In blind source separation (BSS), two different separation techniques are mainly used: minimal mutual information (MMI), where minimization of the mutual output information yields an independent random vector, and maximum entropy (ME), where the output entropy is maximized. However, it is yet unclear why ME should solve the separation problem, i.e. result in an independent vector. Yang and Amari have given a partial confirmation for ME in the linear case in [18], where they prove that under the assumption of vanishing expectation of the sources ME does not change the solutions of MMI except for scaling and permutation. In this paper, we generalize Yang and Amari's approach to nonlinear BSS problems, where random vectors are mixed by output functions of layered neural networks. We show that certain solution points of MMI are kept fixed by ME if no scaling in all layers is allowed. In general, ME, however, might also change the scaling in the non-output network layers, hence, leaving the MMI solution points. Therefore, we conclude this paper by suggesting that in nonlinear ME algorithms, the norm of all weight matrix rows of each non-output layer should be kept fixed in later epochs during network training. (C) 2002 Elsevier Science B.V. All rights reserved.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | INDEPENDENT COMPONENT ANALYSIS; BLIND SEPARATION; ALGORITHMS; nonlinear blind source separation; nonlinear independent component analysis; minimal mutual information; maximum entropy; neural networks |
| Subjects: | 500 Science > 570 Life sciences |
| Divisions: | Biology, Preclinical Medicine > Institut für Biophysik und physikalische Biochemie > Prof. Dr. Elmar Lang Biology, Preclinical Medicine > Institut für Biophysik und physikalische Biochemie > Prof. Dr. Elmar Lang > Arbeitsgruppe Dr. Fabian Theis |
| Depositing User: | Dr. Gernot Deinzer |
| Date Deposited: | 19 Oct 2021 05:57 |
| Last Modified: | 19 Oct 2021 05:57 |
| URI: | https://pred.uni-regensburg.de/id/eprint/40140 |
Actions (login required)
 |
View Item |
