Finding the Mixing Matrix

A method similar to Newtons method is applied to the estimation of negentropy described above to find a mixed signal with minimum nongaussianity and maximum independence.

A Taylor series can be used to approximate a function as follows:

f(x0+e) = f(x0) + f(x0)e

If we are trying to find the zero of f(x) then e can be described as the step needed from x0 to make f(x0+e)=0.


Applied iteratively, we find Newtons method:

J(x) is the negentropy approximation. We want to find the minimum value of J(x) so we want to find a place where J(x)=0. Hyvrinen has a complex analysis showing how Newtons method can be used to derive the following algorithm.

A single column of W can be found as follows:

  1. select a random vector w with ||w||2=1
  2. Let w+ = mean(x*G(wTx)) - mean(G(wTx))w
  3. repeat until wTw is converges to 1

The second equation is confusing and it is worth the time to show how it actually works.

We can find multiple columns of W the same way, but we need to decorrelate the rows at each step to prevent multiple columns from converging to the same solution. This is similar to orthogonalization of eigenvectors. Gram-Schmidt can be used to orthogonalize each vector with the previously discovered vectors.

Alternatively, all of the vectors can be calculated at once. The "square root" method is used to decorrelate all of the vectors at once.

W = (WWT)-1/2W

This method requires an eigenvalue decomposition and is computationally expensive.

This can also be calculated iteratively:

  1. W = W / sqrt(||WWT||)
  2. W = (3/2)*W - (1/2)*WWTW
  3. Repeat step 2 until convergence

The norm is step one can be the 1-norm or 2-norm, but not the Frobenius norm.

Prev Next