A method similar to Newton’s method is applied to the estimation of negentropy described above to find a mixed signal with minimum nongaussianity and maximum independence.
A Taylor series can be used to approximate a function as follows:
f(x0+e) = f(x0) + f’(x0)e
If we are trying to find the zero of f(x) then e can be described as the step needed from x0 to make f(x0+e)=0.
Applied iteratively, we find Newton’s method:
J(x) is the negentropy approximation. We want to find the minimum value of J(x) so we want to find a place where J’(x)=0. Hyvärinen has a complex analysis showing how Newton’s method can be used to derive the following algorithm.
A single column of W can be found as follows:
The second equation is confusing and it is worth the time to show how it actually works.
We can find multiple columns of W the same way, but we need to decorrelate the rows at each step to prevent multiple columns from converging to the same solution. This is similar to orthogonalization of eigenvectors. Gram-Schmidt can be used to orthogonalize each vector with the previously discovered vectors.
Alternatively, all of the vectors can be calculated at once. The "square root" method is used to decorrelate all of the vectors at once.
W = (WWT)-1/2W
This method requires an eigenvalue decomposition and is computationally expensive.
This can also be calculated iteratively:
The norm is step one can be the 1-norm or 2-norm, but not the Frobenius norm.