Computational Intelligence within Finance

Computational Intelligence has promised many things over the last three decades; automated stock picking, portfolio optimization, neural prosthesis, predictive models for complex systems and many more. To say the very least, these methods have come up short on all fronts. There are furious debates in academia why this is so, I’m only concerned with moving this field forward and delivery some of the computational scalability these methods show in theory. So instead of tackling all the different reasons why they do not deliver theoretical scale, I’m going to concentrate on some obvious whys how to improve these methods.

I have done a lot of work with the Self-organizing Kohonen Feature map (SOM); which is a neural network that resembles the way in which the visual cortex self-organizes the features it is presented by the visual system. These neural networks have proven to be very useful in simulation mode or low data frequency. I initially used the computational method to reverse engineer 500 KLOC of COBOL code into and object-oriented representation for the European Space Agency. The system performed extremely well and recognized 80% of the objects that human engineers agreed to. But it was very slow. And what I want to do now needs high performance: the analysis of Level II quotes in real time for the full NASDAQ Market.

One of the fundamental problems with computational intelligence, and neural networks in general, is that they gain almost all there scale from being massively parallel. The ironic bit about this is we have spent almost all of our research time on neural network topology, learning methods and applications. And relegated experimental platforms to what we call Artificial Neural Networks; which is where we transform highly parallel computations into linear computations so the instructions can be “corralled” through a single pipe on a processor (maybe two if you’re lucky). Transforming what 4 billion neurons do simultaneously into a purposely-built linear processor is counter intuitive. And the scale required by the SOM will never be reached to do anything meaningful in real-time in this manner. Attempts have been made to build purposely-build parallel machines with very simple processing units (e.g. Thinking Machines) but this approach failed seemingly due to the cost of the machine and the limited of use. Hybrid approaches are now being explored that use simpler micro-controllers as processing units but this approach is still expensive to get to scale. At the other end of the scale is grid computing where you use a grid of many fully functional, sophisticated processors. To simulate a sizable neural network using a grid, the grid would need to be unreasonable massive and very expensive.

I proposed to a colleague, Damien Morton a highly motivated and innovative consultant, that what I really need is an “out of the box” approach for simulating a neural network that would at least get me an order of magnitude performance over the main processing unit of any machine. This type of performance would get me close to analyzing real-time Level II Quotes.

Damien Morton, a highly motivated and innovative consultant living in Australia, has been the sole implementer of the SOM on the GPU and has been key in the demonstration and theory of this approach.

To Be Continued…