Network improvement. Nelder-Mead

xni — Sat, 27 Dec 2008 15:49:09 +0000

As you could see in the previous post, neural solution is quite accurately approximates analytical. But there is a problem: it’s too slow. It takes about 5 hours to have all the calculations done.

My mentor gave me an advise to use Nelder – Mead method to calculate weights. In this post I will talk about implementation of this method using F#.

My main decision was using special class to implement this method. All of the operations are done by methods inside class and the end-user only passes a functional and gets the solution that minimizes it.

My class is not a classical implementation of Nelder-Mead method, it have some improvements, which guarantee that solution at the next iteration will be better than current.

You can download source code of this class here.

With this method I got a solution in a few seconds, but it is not so good as previous. I haven’t made any further researches for Nelder-Mead and gradient descent methods and their combinations, but I am going to do that and share my results.

Also I have an ideas:

to use classification (by Kohonen maps), to sort out the worst-studied neurons
to research combination of methods to adjust widths and centers
to use asynchronism and parallel operations with lists to improve performance

Complete source code of my project you can find here.

fs4learning » speed improvement

Network improvement. Nelder-Mead