Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
NT8's Strategy Analyzer - I use commission, slippage, Tick Replay, COET - sometimes I get 12.5 profit factors and $30K per month in Total Net Profit. I only optimize for bar type and bar size. Then I run the strategy in Sim Trade on a chart - usually 15 minute - and steadily lose. What to do? The SA seems a quick way to evaluate a strategy when the market is running and I can't do Market Replay but what's the use if the results are so invalid? Is there a better way?
Can you help answer these questions from other members on NexusFi?
When there is a difference between the strategy analyzer and market replay, you need to analyze the differences trade by trade.
There are many potential sources for errors and discrepancies. You will only find out what is happening when you analyze it.
Typical problems:
- Use of exotic bar types which are not backtestable: What bar type do you use?
- Use of fast bars (high frequency trading) such that many bars have the same time stamp and orders can not be properly executed.
- Use of other than market orders without selecting the correct order fill resolution or adding a secondary bar series to the strategy.
As there are many potential sources of errors, you need to give more information on your strategy, such as
- bar types used
- bar period used
- order types used
In most cases, when a strategy backtest goes wrong, it is the result of someone having committed a few mathematical crimes. :-)
In case you do not want to share further information, just run a backtest over a short period of time and examine the trades one by one, comparing the backtest and the market replay.
Also you may want to strip down the strategy to a simpler version and then go step by step adding features and look whether it has an impact on the discrepancy between backtest and market replay.
Hello, thanks for your responses. I am generally using minute bars. I always use market orders as I want to be sure and get a fill. I should have had all my information lined up in advance so to catch up I ran a study today. It's a simple Donchian Strategy with no stops or profit targets: I think it was the stops & profit targets hurting the prior results. I use Low3Ago (Low[3]) to exit the longs and High3Ago (High[3] to exit the shorts. I usually run the backtest on 15 min, then optimize from there. Attached are today's results. The time periods are a little weird since I started with 5 minutes and advanced by 10s. I was surprised to see that longer time frames gave better results. I have no way to determine the drawdowns other than go through the trades with the chart's Strategy Performance/Historical. I assume longer time frames have larger drawdowns.
The Strategy Analyzer for the 2nd best result (115 min) give $66K for the 6/1-8/22 period with tick replay, commission and slippage of 1. Market Replay gave +$12.7. (Great though).
The SA for for the 5th best result (65 min) gives $71.3K - MR was +$4.1K.
The SA for the 10th best result (35 min) gives $91K - MR was +$13.9K.
The SA shows the best results for a 95 min period with +$75K. Unfortunately, the Market Replay stalled out after 6 days. I will have to run it again.
Well, this looks promising and gives me some hope. A little more tweaking and a lot more testing and I may be ready to try it.
Is there any reason why I can't run 4 ES charts at once with the same strategy? Why would one stall out on me?
Could the Strategy Analyzer software be skewed to be wildly optimistic?
Does running Market Relay at 1000x hurt the results?
Apparently my question is very common. In my case, I am getting fantastic results with COET and Tick Replay and an added 1-tick data series. It's when I go to Market Replay/Playback that the 12.5 profit factor drops to 1.1 and the drawdowns go from $650 to $6500.
Wish I knew how to rectify that. Is it slippage? Do I need to use a Stop-Market order? Why are entries & exits happening a bar late?
My tests suggest that Tick Replay must also be used with Market Replay to get closer to IRL results. Try it - you'll see huge differences between with and without Tick Replay. My tests also suggest you will need to have a 12.5 profit factor or better in the Strategy Analyzer to get a possible 1.5 profit factor in Market Replay/Tick Replay.
Here's a tip: for a Market Replay from 8/1 - 8/22, set Days to Load in the Data Series to 22. They you can access Strategy Performance >> RealTime and get the Trades, Max Drawdown, Profit Factor, etc.
I am learning all this by drips and drops. Does someone teach this? Kevin Davey works with Tradestation which is a much simpler system; Easy Language not C#.
I have also run into problems with NT8 backtesting vs real-life trading. If you run your backtest with Order fill Resolution = Standard (fastest), there is a strange calculation being used in the background which leads to the results being just wrong. Instead, set the Order fill Resolution = High, then 1 tick for highest accuracy. Definitely takes much longer to run the backtest, but your results will be as close to real trading as possible. Thats actually what they replied to me in an email when I asked them about it.
