Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Experimental one step AForge-based NN predictor for OHLC data (Neural Network)
NN predictors may be a dime a dozen but this one is mine and therefore unique in all the world (apologies to "The Little Prince")
The program discussed here is being developed as part of a study of backpropagation as a possible way to include adaptation (learning) in a separate fuzzy logic based automated trading system, but is presented in this thread simply as a platform for learning about NN one-step prediction by testing a few simple architectures with various sized training and test data sets.
To make it interesting the program is designed to work with OHLC price data (any time frame, any instrument), hence will attempt to predict the close of the bar following the current bar based on OHLC values of current & previous bars.
The code is in a state of flux and is offered with no claims or guarantees except that it hasn't crashed my computer yet and (barring any undetected bugs that are affecting results) initially at least appears capable of generating a network that can be used to create a simple strategy with positive expectancy.
That is not saying much given a system that flips a coin to decide whether to go long or short can likely be profitable if we pay attention to recent price variance when setting stops and targets and manage money appropriately.
Example Application to Daily Stock OHLC Data
Since one question a trader might ask is therefore whether NN prediction gives any edge over coin tossing, a quick test was performed using daily OHLC data for ticker CS on the TSX exchange between February 2006 and May 2012 (1583 values).
The (not necessarily optimal) predictor topology for the test was as follows:
- 4 input nodes corresponding to OHLC "returns" (first difference in raw OHLC prices);
- 8 nodes in the first hidden layer;
- 20 nodes in a 2nd hidden layer; and,
- 1 output node corresponding to the return on the next day close (predicted difference between "today's close" and "tomorrow's close).
The network was trained on the first 1060 OHLC returns (difference between successive day-to-day values for each O, H, L & C) obtained from the CS data set (February 13, 2006 through April 30, 2010, see Figure 1, below) by iterating on the training set 100,000 times. The number 1060 was chosen because it's approximately 5 times the number of connections inside the network (a rule of thumb to minimize curve fitting). The trained network was then applied to the remaining 523 OHLC returns (May 3, 2010 through May 31, 2012, see Figure 2). [In both the training set and the test set network inputs and targets the final return was set equal to the previous value.]
The only statistical analysis performed on the data set was to compare the returns probability density (training and test vectors combined) with a normal distribution having the same standard deviation and mean (Figure 3).
Results of the training and test runs were saved to .CSV files and imported into Open Office Calc (any spreadsheet app will do) where the following simple "trading strategy" was applied to the data:
1. if tomorrow's predicted close is greater than today's close, buy a number of shares of CS stock near end of day when the close can be estimated (10,000 shares are used in the example below, at a cost of $20,000 - $40,000, but the amount is immaterial) and close the trade at the end of the following day.
2. if tomorrow's predicted close is less than today's close, sell a number of shares of CS stock and close the trade at market close on the next day.
The same strategy was also applied to the same test data using a pseudo random buy/sell sequence rather than price data to simulate entry based on a coin toss.
Note: One quibble with this strategy realized in hindsight is that CS traded under $5 for the entire period and I for one am unable to short stocks under $5, so in lieu of more testing we either take it on faith the results presented below might apply to pricier stocks, or divide the cumulative P&L by approximately 2
Results
As expected the strategy using predictions obtained for the test data set did not do as well as predictions for the training data. While a result of this sort is de rigeur for NN's and of no consequence (and we're interested in any advantage over a coin toss) for the record predictor performance on test data was 25% that of its performance on trained data.
The results of the strategy guided by the predictor vs results of the same strategy applied to a coin toss are shown as plots of respective cumulative P&L in Figure 4.
Conclusion
Results in Figure 4 speak for themselves--namely, more work is required to determine what they mean :-/
Changes to the program will be posted as they become available, as well as an interface between the trained network & NT .
Notes on attachments
.ZIP files containing the project (TDOHLC20120616.zip, Visual Studio Ultimate, .NET 4.0) including Zedgraph and AForge libraries, and the data used in the example (CS20120604Cleaned.zip) have been attached. See my journal for some instructions on use.
This project is intended for relatively experienced developers who might be unfamiliar with implementing an NN predictor.
Questions about the program and bug reports welcome.
Figures
Figure 1. Training set for ticker CS on the TSX (Open Office Calc plot)
Figure 2. Test set for ticker CS on the TSX (Open Office Calc plot)
Figure 3. Returns density vs Normal density for entire data set (Open Office Calc plot)
Figure 4. 2 Year Cumulative P&L for Predictor vs Coin Toss (Open Office Calc plot)
Can you help answer these questions from other members on NexusFi?
Roger and thanks Mike. I'm curious whether it's capable of identifying 3 and 5-bar candlestick patterns sometimes associated with turning points but more curious how it will perform with spot Forex so Forex result and the real time NT interface will likely be the next installments.
NNets need to see the past, not just a current OHLC.. Try also giving it OHLC of previous day and OHLC of 2 days ago plus some MA's of the prices.. also try using % chg in prices rather than raw data..it needs to see relative changes in price..
There are an infinite amount of ideas to be tested and just as many possible targets....
Think of what YOU would need to make an educated guess about the future...
Thanks sptrader--the capability for looking back more than 1 period is built in, and in fact I've done that in the past with high expectations (unfortunately nothing like actual testing to burst a bubble ) just not used explicitly in the example. The example does use OHLC values looking back 20 periods implicitly however to calculate the local mean and standard deviation used to scale the returns.
I might not have been clear that while the program can use raw price it defaults to returns (first differences in price) so that applying a % would merely scale the data linearly--moot as you know because it has to be rescaled anyway. Haven't tried log of the returns however, which would change the stats.
I agree the inclusion of more features alters the behaviour of a system (in fact the system under construction, which the predictor is being used to construct, can use 56 features), but IIRC network connectivity increases as 2 ^ (# inputs), which tends to put a damper on things given the number of training vectors required.
Agree about the infinite number of ideas-- I've tested a lot of them in my career but only rarely able to prove the potential ROI justifies the investment in effort required to develop an idea to production stage--which is why I stuck to one idea in this thread. After all, all it takes is one idea and a whole of toil to make money
Finally, as you may be suggesting prediction and trading are 2 different things, which is one reason the predictor doesn't incorporate a trading strategy. While prediction may be a component of a strategy the main difference IMO is that trading tends to involve probabilities rather than confidence levels (which say nothing about the probability the predicted value lies within a given interval, just that we are "confident" than in some reality it does) and in general the greater our confidence the less precise the estimation of the measured quantity, which makes it tough to set stops and targets
The beta of the NT interface for a 1-step predictor is in forward testing with an 1800 Tick EUR/USD chart, online so far for 6 hours (mid European session through start of N. American). It has been implemented as a strategy and a DLL (DLL in c# built with .Net 3.0 for compatibility with NT).
The DLL project (NNPredictorDLLVSProject.zip), the NT strategy (as a ZIP'd .CS file, NNPredictorNTStrategy.zip) and the trained neural net (NTPredictionNet.zip) used in this example are attached. I'd export the strategy from NT (as a ZIP file) but NT wants to include a large number of irrelevant objects in the export file; not sure why (only 2 cups of coffee in me so far today and not in the mood right now for another Battle Royal with NT).
As suggested above the network was trained on 1800 Tick EUR/USD so while it likely won't work (as well) with any other instrument/bar, that has not been confirmed.
Figure 1 shows real time performance so far trading 1 standard contract through IB (approximately 28x leverage, commission 0.00002 x value of the leveraged order amount with a minimum of $2.50)--nothing to write home about but to paraphrase Samuel Johnson, "Sir, a computer trading is like a dog's walking on his hind legs. It is not done well; but you are surprised to find it done at all."
Figure 1. Real time (forward) 6 hour performance
The DLL code is still being tested for bugs--improvements will have to wait. The intent is simply to give experienced developers an idea of how I'm going about the interface, not to provide a production strategy, and questions/comments/recommendations about the code are welcome.
ETA: The purpose of forward testing is to determine how actual results compare with theoretical (backtest) cumulative P&L for the last 13 days shown in Figure 2 (backtest based on recently updated network--not sure if the DLL is loading updates automatically as it's programmed to do or not :-/)
Figure 2. Theoretical predictor cumulative P&L over period of 13 days for 1800 Tick EUR/USD. Blue = Backtest, Red = actual obtained.
BTW at the moment I plan to update Figure 2 with actual results on my server here only, which means that updates will appear when the thread is opened or refreshed, but (I strongly suspect) no email thread update notification will be sent to anyone.
After making the grandiose claim "it hasn't crashed my computer yet", coincidentally the computer running the forward test (on NT's Sim account) has developed some kind of issue, the only symptom so far being mega instances of Windows Error Reporting (werFault.exe) being spawned and using up all available memory. It took a couple of reboots before I managed to restrict the number of times werFault.exe runs and in the process NT dropped performance data for the Sim account.
So far studying Event Manager hasn't turned up the cause. My guess is the strategy is not responsible since in hindsight there were probably indications all was not right with the computer prior to this and may end up replacing RAM. While the paper account is somewhat more robust in terms of hanging onto performance data (primarily because a record is kept at the broker) it is being used for other things. Bottom line it may be a few days before the test can be installed on another computer.
As an aside, in 2 weeks my significant other and I are planning to hop on our motorcycles and go walkabout for most of July so things are up against that deadline.
Thanks--these trips get me away from the computer once or twice a year whether I need it or not; neighbour recently confided to my wife he's amazed the sudden exposure to sunshine & fresh air doesn't make me catch fire
Just a reminder about the strategy (pointed out by a son, a budding trader who will be in charge in my absence). It's running without stops or targets in order to compare with theoretical performance, which makes it unsuitable as is for the real world.
ETA: And a comment about test performance: in the early stages and speaking qualitatively it looks like the strategy is able more or less to hold its own during chop but manages to get on the right side of breakouts while they're forming. This behaviour may be consistent with theory in that it seems during network training target matching precision falls off for data segments where price stats approach random but improves dramatically for spikes--almost the opposite of e.g. particle oscillator behaviour. Wonder if this has to do with the way backpropagation error minimization works, the squeaky wheel getting most of the grease.
Journal20120616/CSTrainingSet.png)
Journal20120616/CSTestSet.png)
Journal20120616/ReturnsDensityVSNormal.png)
Journal20120616/CS2YearReturn.png)
Journal20120619/Performance.png)
Journal20120619/Theory.png)