NexusFi: Find Your Edge


Home Menu

 





Backtesting With Only OHLC Data


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one goodoboy with 8 posts (2 thanks)
    2. looks_two grausch with 5 posts (5 thanks)
    3. looks_3 rleplae with 5 posts (8 thanks)
    4. looks_4 DarthJewel with 1 posts (1 thanks)
      Best Posters
    1. looks_one rleplae with 1.6 thanks per post
    2. looks_two grausch with 1 thanks per post
    3. looks_3 DarthJewel with 1 thanks per post
    4. looks_4 goodoboy with 0.3 thanks per post
    1. trending_up 5,378 views
    2. thumb_up 16 thanks given
    3. group 4 followers
    1. forum 19 posts
    2. attach_file 0 attachments




 
Search this Thread

Backtesting With Only OHLC Data

  #11 (permalink)
 
rleplae's Avatar
 rleplae 
Gits (Hooglede) Belgium
Legendary Market Wizard
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
Posts: 3,003 since Sep 2013
Thanks Given: 2,442
Thanks Received: 5,863


goodoboy View Post
Thank you so much for clarifying and correcting me.

Appreciate the help.

Market Replay Comments:
This make good sense as well to just let the automated strategy run on market replay for 120 days and confirm the system is trading per design and the review the trading performance as time goes by. Also, i read the market replay can be done for multiple markets at the same time. I will have to research the difference between NT Historical Data used for market replay vs recording the live data for 120 days.

Then just for experiment sake it would be a decent idea to back test the strategy for these same 120 days using the Strategy Analyzer and compare the results.

Would you recommend this as an opinion?

Thanks,

Play with both and compare.

Keep in mind if you replay, you will have to do it for every front month, unless you do it with a continuous contract

Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
Thanked by:

Can you help answer these questions
from other members on NexusFi?
New Micros: Ultra 10-Year & Ultra T-Bond -- Live Now
Treasury Notes and Bonds
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
Exit Strategy
NinjaTrader
Futures True Range Report
The Elite Circle
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
39 thanks
NexusFi site changelog and issues/problem reporting
26 thanks
GFIs1 1 DAX trade per day journal
18 thanks
The Program
18 thanks
  #12 (permalink)
goodoboy
Houston
 
Posts: 380 since Dec 2016
Thanks Given: 344
Thanks Received: 246


grausch View Post

To tie in with your original question, I have always used OHLC when backtesting. When using 1 minute candles there is not much to be gained by going to tick data unless you wish to trade or hold through news events. .

Thank you grausch for the recommendation regarding market replay

In your opinion, is there much of a difference between using a 3 minute vs 1 minute candle when back tesing using the OHLC?

Logically, it take three 1 minute bars to create at one 3 minute bar.

I trade 3 min bar chart, and just trying to grasp an understanding for not needing tick data and understand how inaccurate the back testing can be per my trading method which involves limit orders on entry and calculated stop and profit targets.

Thanks,

Reply With Quote
  #13 (permalink)
goodoboy
Houston
 
Posts: 380 since Dec 2016
Thanks Given: 344
Thanks Received: 246



rleplae View Post
Play with both and compare.

Keep in mind if you replay, you will have to do it for every front month, unless you do it with a continuous contract

Thanks rleplae,

Yes, I agree, playing with both market replay and strategy analyzer is good learning and experimental, then I decide path forward.

I will use continuous contract to keep things simple.

Thanks,

Reply With Quote
  #14 (permalink)
 grausch 
Luxembourg, Luxembourg
 
Experience: Advanced
Platform: TWS
Broker: Interactive Brokers
Trading: Stocks
Posts: 494 since May 2012
Thanks Given: 1,731
Thanks Received: 1,159


goodoboy View Post
Thank you grausch for the recommendation regarding market replay

In your opinion, is there much of a difference between using a 3 minute vs 1 minute candle when back tesing using the OHLC?

Logically, it take three 1 minute bars to create at one 3 minute bar.

I trade 3 min bar chart, and just trying to grasp an understanding for not needing tick data and understand how inaccurate the back testing can be per my trading method which involves limit orders on entry and calculated stop and profit targets.

Thanks,

It depends entirely on how your systems trades. If you have stops that are pretty wide, then 3 minute data makes no difference vs tick data. If you are scalping using very tight stops, then anything other than tick data is not sufficient.

When you look at a OHLC bar, it gives you all of the detail you need to determine whether you entered a trade or whether it got stopped out. You know where it opened, you know how high it traded, you know how low it traded and you know where it closed. If you are basing your decisions on this bar, then you don't need tick data.

Where things get problematic is if you use close stops. Assuming you go long based on your signal, but the low is below your stop, how do you know which triggered first? In this case, the market could have dipped first, then your long triggered and the stop was never in danger. The opposite could also have happened, namely the market went up first, your long triggered, then it was stopped out and then the candle closed. You really have no way of knowing what happened first and you then need to make assumptions on whether or not you got stopped out or are still in the trade.

The only way to get reduce the above problem is to go to smaller intervals (1-minute bars vs 3-minute bars). However, if your stop is far enough away it won't matter - 3-minute bars will work just fine. It's really all about where you place your stop.

Reply With Quote
Thanked by:
  #15 (permalink)
goodoboy
Houston
 
Posts: 380 since Dec 2016
Thanks Given: 344
Thanks Received: 246


grausch View Post

The only way to get reduce the above problem is to go to smaller intervals (1-minute bars vs 3-minute bars). However, if your stop is far enough away it won't matter - 3-minute bars will work just fine. It's really all about where you place your stop.

Thank you very much for response.

I understand now.

Thanks,

Reply With Quote
Thanked by:
  #16 (permalink)
 grausch 
Luxembourg, Luxembourg
 
Experience: Advanced
Platform: TWS
Broker: Interactive Brokers
Trading: Stocks
Posts: 494 since May 2012
Thanks Given: 1,731
Thanks Received: 1,159


goodoboy View Post

1. How can I back test my strategy if my stops and targets prices are calculated/predefined? For instance, if bar close above resistance level 53.50, enter limit order 53.52. What happens if the OHLC of the next bar is 53.59, 53.61,53.48, and 53.56, respectively ?

Does this mean I will not be filled at 53.52?

Just a note when I mentioned close stops being a problem with longer duration candles (3 min vs 1 min) I was thinking of market or stop orders to enter.

In your case, and since you use limit orders, it would not matter how the price reached your level. In your example you have a limit order of 53.52. If the low of the candle goes to 53.52, there is a chance you may be filled. If the low of the candle goes to 53.51 you will be filled at 53.52. It is always good practice to only consider a fill on a limit order once it passes through your limit price. Merely touching your limit price may not lead to you being filled in real life. How would you implement this in Ninja? No clue, I never tried to do this.

Let's say, as an example that your stop is at 53.48. Since you are trading a limit order, it does not matter if price touched 53.61 first and then dropped to 53.51 for you to get filled. The fact of the matter is that you will get filled as price drops and therefore the sequence of events is irrelevant. Should the price however touch 53.48, your assumed stop will trigger due to the difference between limit orders and stop orders. Edit: If the price drops to 53.48, you will be in the trade due to it passing through the limit price, therefore you have no uncertainty on whether a stop executed. If you see a Low at or below your stop, you can assume that you entered and got stopped out on the same bar.

These type of different nuances are something to be aware of when designing backtests. I am not sure how you would implement this in Ninja - I just used market orders when backtesting (and added in some slippage). Depending on your trade frequency, your backtest results can be overly optimistic if you do not correctly account for these two items I show above.

With regards to profit targets, if you have pretty close profit targets, then my previous post regarding stops is relevant. If your profit target triggers in the same bar as your entry, then you will need to guess if a) the market opened at the Open, the market dropped to your limit price, the market then bounced to your target and you exit for a profit, or b) the market opened at the Open, ran up to the High, dropped to your limit price, and then continued on its way with you still in the trade. These type of items can really screw with your backtest results which is why I recommended running the backtest manually as well. If the above occurs frequently, then you need to use shorter time-frame candles. If it still occurs frequently, then your backtest results may not be worth anything.

On a final note, you can also run a strategy live in Ninja on a demo account. When done this way, Ninja will use actual prices for fills. I am not sure how a limit order would be filled, i.e. does it merely need to touch the limit order or pass through it, but you could run the strategy like this for a month, and then perform a backtest over the same period and compare the results. If your backtest results are close to the "actual" results, you have a reasonable amount of certainty that fills won't be an issue and it gives you more confidence for a longer term (1-year or longer) backtest.

If the results of the demo account over a month are significantly different from the backtest, then you have a problem with either how fills are handled in the backtest or the profit-target/stop problem I highlighted before. The profit-target/stoploss issue is not a problem when Ninja uses live data - it only affects you when using bars / candlestick data. In any case, should the two results not line up, then you need to figure out when went wrong and whether or not it is merely a backtesting issue or whether the system is actually viable.

Reply With Quote
Thanked by:
  #17 (permalink)
goodoboy
Houston
 
Posts: 380 since Dec 2016
Thanks Given: 344
Thanks Received: 246

Thank you grausch for the detailed and effort in responding to me.

I respond below.


grausch View Post
It is always good practice to only consider a fill on a limit order once it passes through your limit price. Merely touching your limit price may not lead to you being filled in real life. How would you implement this in Ninja? No clue, I never tried to do this.

Great advice. Ninja Trader has conservative fill algorithm as an option for its back testing function which means price has to pass through my limit or stop order to get filled.


grausch View Post
Let's say, as an example that your stop is at 53.48. Since you are trading a limit order, it does not matter if price touched 53.61 first and then dropped to 53.51 for you to get filled. The fact of the matter is that you will get filled as price drops and therefore the sequence of events is irrelevant. Should the price however touch 53.48, your assumed stop will trigger due to the difference between limit orders and stop orders. Edit: If the price drops to 53.48, you will be in the trade due to it passing through the limit price, therefore you have no uncertainty on whether a stop executed. If you see a Low at or below your stop, you can assume that you entered and got stopped out on the same bar.

I think I understand what you mean here.

Are you stating that if I entered a trade on the same bar at limit buy order 53.51, and my stop loss order is 53.48 which is happens to be the Low of the bar, did I get stop out or not?

The answer is No, I was not stopped out because Ninja Trader has conservative fill algorithm as an option for its back testing function which means price has to pass through my stop order at 53.48 to 53.47 to get filled. So if the next bar hits 53.47, I will be stopped out.

Please let me know if this what you meant in your statement above and if I answered it.


grausch View Post
With regards to profit targets, if you have pretty close profit targets, then my previous post regarding stops is relevant. If your profit target triggers in the same bar as your entry, then you will need to guess if a) the market opened at the Open, the market dropped to your limit price, the market then bounced to your target and you exit for a profit, or b) the market opened at the Open, ran up to the High, dropped to your limit price, and then continued on its way with you still in the trade. These type of items can really screw with your backtest results which is why I recommended running the backtest manually as well. If the above occurs frequently, then you need to use shorter time-frame candles. If it still occurs frequently, then your backtest results may not be worth anything.

I am not sure I follow you. Let’s set up an example so I make sure I observe the problem.

Previous Bar Executes limit buy order on bar Close

Limit buy order Entry = 53.52
Sell Limit order Profit target = 53.60
Stop loss market order = 53.45

Next Bar in process
Open = 53.57
High = 53.64
Close = 53.55
Low = 53.42

Is the problem or question related to if the back testing software functionality knows which direction price moves after Market Open similar to live price movement? For example 1) On market open 53.57, did price go to High of 53.64 first and then down to and fill limit buy order at 53.52 and then trigger stop loss market order of 53.45, which will cause a loss OR 2)On market open 53.57, did price go to fill limit buy order at 53.52, then reverse up to profit target of 53.60 sell limit order and returning a profit.

So you are asking is the back testing software smart enough provide the OHLC data and smart enough to know the real price movement (up or down) once the bar starts creating?

This is a good questions cause I use profit target of 8 ticks on 1st contract and that can happen in the same bar.

I have to think about this one, I need some sleep.

Thanks

Reply With Quote
Thanked by:
  #18 (permalink)
 grausch 
Luxembourg, Luxembourg
 
Experience: Advanced
Platform: TWS
Broker: Interactive Brokers
Trading: Stocks
Posts: 494 since May 2012
Thanks Given: 1,731
Thanks Received: 1,159


goodoboy View Post
Are you stating that if I entered a trade on the same bar at limit buy order 53.51, and my stop loss order is 53.48 which is happens to be the Low of the bar, did I get stop out or not?

The answer is No, I was not stopped out because Ninja Trader has conservative fill algorithm as an option for its back testing function which means price has to pass through my stop order at 53.48 to 53.47 to get filled. So if the next bar hits 53.47, I will be stopped out.

Please let me know if this what you meant in your statement above and if I answered it.

This is one of the items where you really need to consider how price can move within a specified bar. For instance, based on the prior bar you know you will be entering on the current bar. There are really only three things that can happen here:
  1. Price does not drop to your limit order and you do not enter into the trade. How do you know this happened? The LOW of the bar was above your limit price - 0.01 (conservative fill algorithm of NT)
  2. Price drops enough so that you can enter, but you do not get stopped out. This one is quite straightforward again - the LOW of the bar drops to your limit price -0.01 or lower, but does not drop below your stop.
  3. Price drops enough that you enter the trade and then get stopped out on the same bar. This is determined by just comparing the LOW of the bar to your stop price. If the LOW is equal to or lower than your stop price, then you would have taken a quick loss.

With the above, the only relevant info is the LOW of the bar (we are assuming no profit targets are hit). None of the other information is needed to determine what had happened with the potential trade. Please note however, that with stop orders the price only needs to touch your stop order for it to execute. You could put in a limit sell of 53.48 once price gets there, but what if you don't get filled? Price can drop 100 ticks or more and you will have a massive loss. When you have stop-losses, you should execute them either with stop or market orders to ensure you get filled.


goodoboy View Post
Next Bar in process
Open = 53.57
High = 53.64
Close = 53.55
Low = 53.42

Is the problem or question related to if the back testing software functionality knows which direction price moves after Market Open similar to live price movement? For example 1) On market open 53.57, did price go to High of 53.64 first and then down to and fill limit buy order at 53.52 and then trigger stop loss market order of 53.45, which will cause a loss OR 2)On market open 53.57, did price go to fill limit buy order at 53.52, then reverse up to profit target of 53.60 sell limit order and returning a profit.

So you are asking is the back testing software smart enough provide the OHLC data and smart enough to know the real price movement (up or down) once the bar starts creating?

The above is where the problem with OHLC data comes in. I will walk you through the scenarios and the just give a brief explanation. We are assuming you got filled on the prior bar and did not get stopped out. Current bar scenarios (as you correctly surmised) are then as follows:
  1. Market runs straight up to your profit target and you get a profit before it drops to the low. You exit with a profit in this case. Note that only HIGH and LOW are used in this case. OPEN and CLOSE don't matter.
  2. Market drops to your stop and you get stopped out before it rallies to the target. In this case you get stopped out for a loss. Note that only HIGH and LOW are used in this case. OPEN and CLOSE don't matter.

Since you only have OHLC data, no backtesting engine can determine which happened first. At this point you need to make an assumption. Would you assume that profit targets get hit or do you go for the safe route and assume all of these get stopped out? Neither solution is perfect. You could opt to look at where the market closed in relation to HIGH and LOW and use this to guess what happened, but this isn't perfect either.

If you have a system that has the above happening quite frequently in backtesting, then the backtesting result becomes unreliable. Which is why I suggested running the system live on a sim account for one month, and thereafter running a backtest on the same one month data. The backtest will then use the 3-minute bars, while the "live" test used actual data. If the results are wildly different, then you would need to reassess your longer term backtest results.

The reason someone would use 1-minute is to reduce the likelihood of the above occurring. If the likelihood of this is still too great, then you would need to use tick data. Going to 1-minute data can be done quite easily with some systems - for instance if you use the 1 hour low as support, then 3-minute data would use the LOW of the last 20 bars as support and 1-minute data would use the LOW of the last 60 bars as support. However, some things can't get converted as cleanly, i.e EMA crossing over price for instance as it would be subject to more whipsaws.

Reply With Quote
Thanked by:
  #19 (permalink)
 tr8er 
Europe
Market Wizard
 
Experience: Advanced
Platform: TradeNavigator, BookMap
Trading: ES, CL, 6E, 6B
Posts: 704 since Jan 2017
Thanks Given: 220
Thanks Received: 709

Since you only have OHLC data, no backtesting engine can determine which happened first. At this point you need to make an assumption. Would you assume that profit targets get hit or do you go for the safe route and assume all of these get stopped out? Neither solution is perfect. You could opt to look at where the market closed in relation to HIGH and LOW and use this to guess what happened, but this isn't perfect either.


Hi grausch,
I use TradeNavigator and TradeNavigator has the technology to know what happend first for backtesting. Most other platforms don't have this most important tool for backtesting. regards tr8er

Reply With Quote
  #20 (permalink)
 grausch 
Luxembourg, Luxembourg
 
Experience: Advanced
Platform: TWS
Broker: Interactive Brokers
Trading: Stocks
Posts: 494 since May 2012
Thanks Given: 1,731
Thanks Received: 1,159



tr8er View Post
Hi grausch,
I use TradeNavigator and TradeNavigator has the technology to know what happend first for backtesting. Most other platforms don't have this most important tool for backtesting. regards tr8er

Unless TradeNavigator drops down to smaller intervals when you have issues like this, i.e. it uses 1-minute or tick data (sometimes 1-minute bars are still too large) for bars with this type of problem, then it is not possible to determine what happened first. A system can make certain assumptions based on where the candle opened and where it closed relative to the high and the low, but without access to smaller data intervals, all you have is an approximation.

That is not to say the approximation can't be reasonably accurate, but if I was betting my own money on something, then I would want to work with data as accurate as possible. As an FYI - whenever I backtested, I used stops and targets larger than the bars I was working with to avoid the issue. Doesn't mean results would be perfect, but the chances of the above becoming an issue are much less.

Edit: Something to note - I refer to tick data quite frequently, but 5-second data would also work. The only time 5-second data would really be unreliable is when there are news events that cause large spikes - however, these large spikes can often be random, so it may very well be impossible to get backtesting results over something like the oil inventory reports that actually mean something. If using 5-second data, then I would just convert my system so that it runs equivalent to the 3-minute system (assuming it is possible of course).

Reply With Quote




Last Updated on February 27, 2017


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts