Network improvement. Nelder-Mead

As you could see in the previous post, neural solution is quite accurately approximates analytical. But there is a problem: it’s too slow. It takes about 5 hours to have all the calculations done.

My mentor gave me an advise to use Nelder – Mead method to calculate weights. In this post I will talk about implementation of this method using F#.

My main decision was using special class to implement this method. All of the operations are done by methods inside class and the end-user only passes a functional and gets the solution that minimizes it.

My class is not a classical implementation of Nelder-Mead method, it have some improvements, which guarantee that solution at the next iteration will be better than current.

You can download source code of this class here.

With this method I got a solution in a few seconds, but it is not so good as previous.  I haven’t made any further researches for Nelder-Mead and gradient descent methods and their combinations, but I am going to do that and share my results.

Also I have an ideas:

1. to use classification (by Kohonen maps), to sort out the worst-studied neurons
2. to research combination of methods to adjust widths and centers
3. to use asynchronism and parallel operations with lists to improve performance

Complete source code of my project you can find here.

WPF & F#

In this post I would like to write a few words about creating user interfaces to F# programs.

I don’t know about constructors for GUI, and I’m not sure about their existance at all, what’s why in this article we will create UI using objects of .NET Framework 3.0;

At first, we need to add some references to our project to make WPF available. This references are: PresentationCore, PresentationFramework and WindowsBase (all of dll’s are located at C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.0\)

type ResultWindow = class
    inherit Window

Here we are creating a class called ResultWindow, that inherites defined in the framework class Window.

val mutable private minX : float

I use val keyword for declaration of the field. It’s much more simple to use mutable structures instead of immutable, but I need to notice that you shouldn’t use mutable stuctures in pure functional code. All of class members declared with val keyword must be defined in constructor.

new ( (* parameters here *) ) as this = {
        (* definition of fields *)
        minX = 0.0
                                     } then
        (* some other actions *)

Now it’s time to create a method (static methods are created without this. prefix).

member this.DrawGrid =
        let gridLine = new Line()
        (* and many-many lines of code *)

If you are not familiar to WPF my advise is to read some books about it. For example:

At the pictures presented below you can see analytic and neural solution graphics:

I don’t publish my code in this post, because it is quite large. You can download it.

F# and WPF

By the moment I have a solution of the problem: I have the simplest artificial neural network that solves elliptic equation. For further research process I need a visualization of my solution.

There are three way to have this done:

• Text output into file and writing application on other language to show the results. This is a fast and simple solution. But my F# program returns list of weights, widths and centers. So I need to write the neural network modeler again! Oh, no… I don’t like this way.
• Use the ability of Microsoft Visual Studio to call procedures, written on other languages inside one project. At first I wanted to use this ability, but yesterday I find the best way to do the modeler.
• Windows Presentation Foundation (WPF). It’s a part of .NET framework 3 and gives us an opportunity to have separate code and markup written.

I need to have my visualizer finished in 3 days, so results are coming soon!

There are a lot of materials about F# and WPF, but I liked this one.

Neural network overview

If you still don’t know what the artificial neural network is -  you should read some information about it (e.g. http://en.wikipedia.org/wiki/Artificial_neuron).

In this article I will use artificial neurons based on radial basis functions (RBF). The influence of a neuron located in point c upon x is calculated as: where a is also a parameter of current neuron. Let us consider that the network consists on N neurons. Then we are able to calculate total influence:

The first partial differential equation that I’ll solve is:

Read the rest of this entry »

Atmosphere module

As an example of F# usage for calculations and demonstration of module creation I decided to publish here my code for Standard Atmosphere (atmex 3.0). You can download code or library below.

(* using light syntax *)
#light
(* module name *)
module AtmosphereLib74
 
(* a structure for storing atmosphere parameters *)
(*
 *  Of course, I could use tuple float * float * float * float,
 *  but at http://blogs.msdn.com/jomo_fisher/archive/2008/09/16/f-performance-tweaking.aspx
 *  I read, what returning structures insted of tuples is more efficient
 *)
type AtmosphereParameters =
    struct
        val height : float
        val pressure : float
        val density : float
        val temperature : float
        val sonic : float
        new (h,p,d,t,s) = {height=h; pressure=p; density=d; temperature=t; sonic=s}
    end
 
let getAtmosphere h =
    (*
     calculations of pressure, density, temp, sonic
    *)
    (* and here I create and return a structure *)
    AtmosphereParameters(h,pressure,density,temp,sonic)

You can download this example as a code or as a compiled library.

To use any kind of libraries you should just add reference to it into your project and use it’s namespace.

Why F#?

When I started using F# I didn’t know what it would be like. But when I coded the Runge-Kutta method and a core of my ballistic calculation project in a few hours, without any errors and bugs, spending no time debugging, I realised that functional programming is a very handy idiom for various computations.

I don’t know all of advantages of F#, but during this month I found what F# is

• Easy to learn and understand. You may not have a wide knowledge to start.
• Usually it’s much more expressive.
• It’s really difficult to make a bug!
• You can use .NET libraries and classes.
• You can combine Functional abilities with object oriented and imperative idioms.

Don’t be afraid of using it! It takes about two weeks to get accustomed to F#. You just need to have F# compiler to start. If you want to have IDE, my advise is downloading Microsoft Visual Studio 2008 Shell.

I use following books as a source of information about F# and functional programming:

• Foundations of F#
• Expert F#
• A. Field, P. Harrison: Functional Programming, Addison-Wesley, 1988

What this blog will be about

Hello everyone!

My name is Konstantin Nikitin and I’m a 4^th year student in the Moscow Aviation Institute. I’m interested in studying and permanently searching for something new. Recently I heard about F# programming language and decided to start using it for solving my problems.

In this blog I’ll publish articles about my activities with F# and my current researches. At the moment I use F# for my term papers on the numerical methods and the ballistics. Here you will be able to see the results of my work.

Sincerely yours,
Konstantin