Network improvement. Nelder-Mead
As you could see in the previous post, neural solution is quite accurately approximates analytical. But there is a problem: it’s too slow. It takes about 5 hours to have all the calculations done.
My mentor gave me an advise to use Nelder – Mead method to calculate weights. In this post I will talk about implementation of this method using F#.
My main decision was using special class to implement this method. All of the operations are done by methods inside class and the end-user only passes a functional and gets the solution that minimizes it.
My class is not a classical implementation of Nelder-Mead method, it have some improvements, which guarantee that solution at the next iteration will be better than current.
You can download source code of this class here.
With this method I got a solution in a few seconds, but it is not so good as previous. I haven’t made any further researches for Nelder-Mead and gradient descent methods and their combinations, but I am going to do that and share my results.
Also I have an ideas:
- to use classification (by Kohonen maps), to sort out the worst-studied neurons
- to research combination of methods to adjust widths and centers
- to use asynchronism and parallel operations with lists to improve performance
Complete source code of my project you can find here.
