ES - Penalising backtesting results - Beginners and Introductions | futures io social trading
futures io futures trading


ES - Penalising backtesting results
Updated: Views / Replies:193 / 5
Created: by TradingTom Attachments:0

Welcome to futures io.

Welcome, Guest!

This forum was established to help traders (especially futures traders) by openly sharing indicators, strategies, methods, trading journals and discussing the psychology of trading.

We are fundamentally different than most other trading forums:
  • We work extremely hard to keep things positive on our forums.
  • We do not tolerate rude behavior, trolling, or vendor advertising in posts.
  • We firmly believe in openness and encourage sharing. The holy grail is within you, it is not something tangible you can download.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.


You'll need to register in order to view the content of the threads and start contributing to our community. It's free and simple, and we will never resell your private information.

-- Big Mike

Reply
 
Thread Tools Search this Thread
 

ES - Penalising backtesting results

  #1 (permalink)
Trading Apprentice
Munich Germany
 
Futures Experience: None
Platform: NinjaTrader
Favorite Futures: Emini ES
 
Posts: 23 since Jan 2017
Thanks: 32 given, 9 received

ES - Penalising backtesting results

Hello FIOs!
Once again I'm hoping some of the experienced traders can share some advice.

When trying to gauge the expectancy of a manually backtested system, is it enough to penalise the results of every trade by 2 ticks i.e. assuming limit orders on entry and exit wouldn't have been triggered and manual market orders had to be used?
Assuming small size (1-5 contracts) on ES during the European and US session (~0700GMT - 2100GMT) and at least trying to stay away from big news + not holding overnight, do I have to additionally include slippage?

Thanks for any advice & have a great day!
Tom

Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,256 since Jul 2012
Thanks: 1,059 given, 4,101 received



TradingTom View Post
Hello FIOs!
Once again I'm hoping some of the experienced traders can share some advice.

When trying to gauge the expectancy of a manually backtested system, is it enough to penalise the results of every trade by 2 ticks i.e. assuming limit orders on entry and exit wouldn't have been triggered and manual market orders had to be used?
Assuming small size (1-5 contracts) on ES during the European and US session (~0700GMT - 2100GMT) and at least trying to stay away from big news + not holding overnight, do I have to additionally include slippage?

Thanks for any advice & have a great day!
Tom

1 tick slippage per side is reasonable. BUT, the bigger issue is in the accuracy of your manual backtest. When I used to manually backtest, I had a tendency to cheat with the results. Sometimes it would be very subtle, but whether it is skipping trades, accidentally missing trades, or assuming fills that got touched but not filled, or something else, I personally never found manual backtest to be accurate.

It is kind of like how when you look at an indicator on a chart, usually it looks like it is great, and it is picking great trades for you. But on close inspection, the losers (which you barely saw at first glance) appear and usually take over...

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following 3 users say Thank You to kevinkdog for this post:
 
  #4 (permalink)
Elite Member
Luxembourg, Luxembourg
 
Futures Experience: Advanced
Platform: TWS
Broker/Data: Interactive Brokers
Favorite Futures: Stocks
 
Posts: 487 since May 2012
Thanks: 1,626 given, 1,107 received


TradingTom View Post
Hello FIOs!
Once again I'm hoping some of the experienced traders can share some advice.

When trying to gauge the expectancy of a manually backtested system, is it enough to penalise the results of every trade by 2 ticks i.e. assuming limit orders on entry and exit wouldn't have been triggered and manual market orders had to be used?
Assuming small size (1-5 contracts) on ES during the European and US session (~0700GMT - 2100GMT) and at least trying to stay away from big news + not holding overnight, do I have to additionally include slippage?

Thanks for any advice & have a great day!
Tom

As @kevinkdog mentioned, you need to think about how you are going to ensure accuracy of your manual backtests. I have never really had issues with this, but I would just start at a certain point and work my way through the data, only observing the results at the end. Same thing that Kevin mentioned with the indicators is very applicable here - our mind will tend to focus on the instances where they work and ignore instances where they don't. Automated backtests avoid the issue completely.

With regards to slippage - sure 1 tick per side sounds reasonable, but if you just use limit orders you do not need to use any slippage on your winning trades. However, that does not mean that all trades were executed, so a better method to test this would be to insist that the market passes 1 tick beyond your limit order before it executes at your price. For instance, if you want to buy CL at 52.95, you should only consider that the position executed if price drops to 52.94. You would use the same logic with your profit targets.

Regarding stops, start by using 1 tick slippage, then increase it to 2, then 3 to get an indication of how sensitive the system is to slippage. You can't control slippage, but knowing how sensitive a system is to slippage gives an indication of robustness.

You then also need to consider the impact of news events. I am now referring to the EIA reports for CL, but when those were issued, slippage of 50 (or more) ticks can happen. Whenever you backtest over such an event, it would be prudent to assume the absolute worst result, i.e. if the market moves 50 ticks in an instant, odds are you will be stopped out at the bottom therefore your backtest should make this assumption.

Reply With Quote
The following 3 users say Thank You to grausch for this post:
 
  #5 (permalink)
Trading Apprentice
Munich Germany
 
Futures Experience: None
Platform: NinjaTrader
Favorite Futures: Emini ES
 
Posts: 23 since Jan 2017
Thanks: 32 given, 9 received


kevinkdog View Post
BUT, the bigger issue is in the accuracy of your manual backtest. When I used to manually backtest, I had a tendency to cheat with the results. Sometimes it would be very subtle, but whether it is skipping trades, accidentally missing trades, or assuming fills that got touched but not filled, or something else, I personally never found manual backtest to be accurate.

It is kind of like how when you look at an indicator on a chart, usually it looks like it is great, and it is picking great trades for you. But on close inspection, the losers (which you barely saw at first glance) appear and usually take over...

I agree 100% and while I have been trying to be very strict when going through the charts, I will check the initial performance from that against a bar by bar replay using historic data and trading it manually "on the right edge".
This will then hopefully expose any bias I may have introduced during the testing on the charts.

Reply With Quote
The following user says Thank You to TradingTom for this post:
 
  #6 (permalink)
Trading Apprentice
Munich Germany
 
Futures Experience: None
Platform: NinjaTrader
Favorite Futures: Emini ES
 
Posts: 23 since Jan 2017
Thanks: 32 given, 9 received


grausch View Post
... but if you just use limit orders you do not need to use any slippage on your winning trades. However, that does not mean that all trades were executed, so a better method to test this would be to insist that the market passes 1 tick beyond your limit order before it executes at your price.

I failed to note that on when recoding the backtesting which is why I now asked about the 2 tick penalty i.e. simulating market orders vs. limit orders.
Still not the same, as you mentioned so for further tests I will have to include limit order vs market order result to see which performs better overall.
But the final plan will be anyway to execute on Orderflow data (Bookmap probably) which (in theory) should only improve the results of the purely chart based backtest...

As I mentioned, I'm currently looking at ES and will try to avoid news, so 50 tick slippage is hopefully out of the question

Reply With Quote
The following user says Thank You to TradingTom for this post:

Reply



futures io > > > > ES - Penalising backtesting results

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

FIO Journal Challenge featuring NinjaTrader ($2,000+ of prizes)

May

An overview of volumetric analytical tools w/Sergey Sokolov @ PTMC

Elite only

John Hoagland (TBA)

Elite only

FuturesTrader71 Extended Ask Me Anything (AMA)

Elite only

Pete @ Jigsaw Trading (TBA)

Elite only

John @ No BS Day Trading (TBA)

Elite only

An Afternoon with FIO member Softsoap (being rescheduled)

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Results of Backtesting>> Profitable system!! What other tests should I perform? AshleyQueenTrader Currency Futures 15 July 16th, 2016 08:40 AM
Overview recurring (economic) events to gauge backtesting results Koepisch Traders Hideout 2 September 5th, 2012 04:59 AM
Backtesting results from different platforms dryg Platforms and Indicators 6 July 30th, 2012 11:04 PM
Backtesting identical strategy tradestation and multicharts different results crbucks MultiCharts 7 April 26th, 2012 07:30 AM
Ninja Trader and backtesting results. sinisa NinjaTrader 2 November 9th, 2011 12:40 AM


All times are GMT -4. The time now is 10:23 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-05-28 in 0.12 seconds with 20 queries on phoenix via your IP 54.81.69.159