I just completed my first revision of the emini-S&P trading strategy, and I would like to post the results here for input & feedback.
Start Date/Time 2001-01-02 8:45:00 AM
End Date/Time 2015-06-16 3:00:00 PM
Initial Capital $20,000.00
Commission $2.50 per Share/Contract Slippage $2.50 per Share/Contract
No market order and no trailing stop loss
Only limit orders are used
Total Net profit = $177,757
#of trades = 12,178
DD = $7,830
For more details, see below picture. In your opinion, can this system be improved further, is it average, or is it pretty good? I am putting the results here to benchmark my system against other knowledgeable members. Please provide constructive feedback and share some of your own strategy performance report. Let's challenge each other to improve the development of our trading systems.
It isn't possible to tell that from the information you've provided.
It has an overall PF of only 1.17.
It's had 19 consecutive losers and can have more than that in future.
The position-sizing is presumably very small, with the biggest win being 0.06%, but increasing the position-size may make some of the drawdowns scary?
The report above may also have underestimated commissions and slippage, which may make a much bigger difference to the overall picture than you expect?
And - as mentioned above - MAE information is missing?
These are theoretical, back-tested results, I imagine? I'm wondering whether there are any concealed assumptions relating to granularity?
You'd need to test it reliably, with a small, real account, moving forward, before deciding whether it suits you, anyway?
1) what commissions and slippage would you use for your backtesting?
2) should I recode the software to run on the 1M chart to ensure the accuracy of the backtest? This should solve any granularity issue.
3) what is consider a reasonable pf? Does this value change with different timeframes and different trading styles i.e. scalp, daily vs swing?
I'd do a little bit of trading wherever/however you're actually going to trade, moving forward, and see what commissions and slippage actually come to, realistically, rather than guessing. That should give you some figures to test?
Finding out the MAE will resolve that issue for you, I think? (I have no specific reason to imagine that you're about to unearth a disaster, on that front - it's just something you need to know?).
People's answers to that will vary a lot, I think. For myself, I'm interested in principle in investigating anything with a PF above 1.5, provided it doesn't fall foul of other sensible parameters - but I suspect that many here may give a higher figure than that.
It doesn't for me (but I don't hold positions overnight, anyway, myself).
Don't take me the wrong way: I don't mean it rudely at all, but I suspect from this thread that you may possibly just be "trying to copy something that 'works'," without any experience at all of its underlying parameters or methodology? That's not a recipe for success, in trading. It's traders who "work", not systems.
The stoploss is 64 ticks while the profit target is 32 ticks.
I personally don't believe in trailing stoploss; hence, trades are exited are based on 1) end of day exit 2) stop out 3) take profit 4) exit based on reversing position i.e. exit buy trade when sell signal occurs or exit sell trade when buy signal occur.
What the chart says is that it does not pay off to have such a big stop loss.
once they are losers, they don't become profitable
i think you might be able to tune the algorithm by reducing the stop
you could simulate it with smaller stops
The following 2 users say Thank You to rleplae for this post:
Thank you very much for your feedback. I was hoping to get some general benchmark on setting the commission and slippage. My background is in currency trading; hence, trading S&P is new to me. As for PF >1.5, it is easily achieved by reducing the number of trades i.e. trending systems. In my experience, when PF is too high, you don't have a large enough sample of trades and your system will tend to curve fit. All of my trading systems must have over 500 trades else I would not use them.
The following user says Thank You to fiverr for this post:
It looks like it may have some potential. . . what I was initially impressed with is your data pool of trades, as most people that bring strategies to the forum seem to be thrilled with their holy grail strategy that has some 100 or less trades over the past 10 years. The larger the data pool, the more *potentially* trustworthy that your underlying logic is saying something real, though its absolutely no guarantee.
RP had an excellent point, the best thing you can do, by far, is to test the 'live walk' period of time, now. . i.e. from the date of creation, to the present date, and see if it falls flat.
You asked if there was room for improvement, and to that I would say 'absolutely'. . . The profit factor is certainly a bit low for anything you'd want to realistically trade live, *but* since your data pool of total trades is still quite large, you have room to make some tweaks and refinements.
Have you tested it yet over any similar instruments, such as NQ, YM, etc, to determine whether it performs decently there? Or any market wide testing in general? This would be another valuable step, its extremely useful 'confirmation' to see that a strategies underlying logic works across multiple instruments, especially instruments that tend to be highly correlative with the one you've created the strategy for.
Also, do you have any information as to the average 'time in market' of your average trade?
Lastly, I don't use tradestation, can you be sure that your tests aren't taking trades on top of one another, i.e. entering into another trade even if the first trade is still active? This can produce very large trade counts that aren't actually realistic, because what its actually doing, in effect, is entering several times, each time your conditions are triggered, over a short and staggered period of time. .
Lastly, just to confirm, the long/short directions use mirror image logic, right. . the same logic, flipped, for both directions? Oh, and lastly lastly, in regards to ballpark numbers you should be looking to obtain, I agree with Tymb, roughly, in that anything below 1.5 PF is generally uninteresting, but you can obtain that, at least, with just a little TLC here, I believe.
Let me know the answers to those q's, and perhaps I can be a bit of further help, if you'd like
To give you some idea of what I consider to be decent results, here is a strategy I've been running live for a few months now. . . however, its *vital* to note that no strategy should ever be trusted solely on the merit of its own backtest results in isolation. This strategy has been tested marketwide, and 'scored' according to not only its own backtest results, but the backtest results of every other instrument upon which the core logic was tested. . . I also look at its performance numbers relative to all other optimization results I've ever had with the same Instrument/HourOfDay/MarketDirection combination, as I think its vital that you compare apples to apples insofar as is possible. . and this is just the tip of the iceberg, unfortunately. Long story short, backtest results like this do mean something, but woe to anyone that trusts them implicitly, in isolation, without examining a plethora of other data points/factors. The healthy looking backtest is only a starting point, and a somewhat weak one, at that.
Also, its vital to note that one should never expect live results to perform equal or better to the backtest results, over a significant enough period of time, a large enough data pool of trades. . . it will virtually always, without exception, perform worse. This is to be expected, since attaining a strategy with *zero* curve fitting is simply an absolute impossibility. . the goal here is to create as robust and high-confidence of a strategy as is possible, with the expectation that it will earn more than it loses. . not that it will produce a 2.0/3.0+ PF across all market conditions.
Now that the disclaimers are out of the way:
It's my first time uploading graphics to this forum, fingers crossed that I did so properly. . .!
The following 2 users say Thank You to Dionysus for this post:
You need to forward test or your system will be curve fitted with data snooping bias. You should try do your analysis on a small set of data and then forward test for around the same amount of data.
If your forward test PF doesn't degrade significantly (it will almost certainly degrade), then it's probably workable. However your PF is a bit low, so any slight degradation will make this unviable.
Also the market has changed heaps since 2001, you should try test from 2008~ onwards as you would need to cater for all the algo activity present today.
Final thing is $12-15 net per trade is too low, any sort of additional slippage, or any degradation can turn this negative very fast.
Also not sure why you chose $2.50 for slippage? ES is $12.50 a tick so you're slippage is $10 too low.
Commission at IB, which is one of the cheapest is $4.04 per round turn, so your commission also too low.
If you fix those two, then your $12-15 net profit will now be less than 0 I believe.
You might want to try discretionary trading the ES for a few months first so you understand how the market behaves and then try to automate that understanding, at the moment it looks like you have just data mined this, which is pretty much guaranteed to not work.
The following 4 users say Thank You to PeakGrowth for this post:
Absoutely. The combination of a PF of only 1.17 and inadequate allowance for dealing costs is almost bound to turn the method described from overall profit into overall loss, I'm afraid. This was all pointed out in some detail in the other thread, a month or so ago, though.
Thank you for your comment. Can you be more specific on your criteria on how you judge a trading system to be curve fitted? As for me, I have a few simple rules for a system design to avoid curve fitting:
1) number of trades must be greater than 1,000 to avoid curve fitting
2) the test period must be over 10+ years or longer
3) single year profit can not account for more than 30% of the total profit
4) Ratio of total profit divided by max drawdown must be greater than 20
5) Use WFO whenever possible
6) Fixed lot trading during system design i.e. no compounding or manipulation of trading lot size
The following user says Thank You to fiverr for this post:
Currently, I am focused on currency trading and have not made the switch to ES. My 8-month old daughter is a handful and most of the time, I am not getting enough sleep. If anyone have a demo account, I can run this system on your account for free. As long as your broker can work with MC, we should be good. If the performance is good after 3-6 months, I also offer to run on your live accounts for free for another year or so. This way you have a stake in the project to ensure it is success.
Thanks for sharing. Are you using market orders or limit orders to enter the trades? What will happen if you extend your back testing from 2000 to 2015? BTW, are you holding orders overnight and during the weekends?
Also, how did you design this system? Did you come up with an idea and test it for the first time and these are the results or did you design, test, modify and these are the results?
You might be on to something or you might not, it really depends on the design and testing methodology you employed.
The following user says Thank You to TheTradeSlinger for this post:
Always market orders, in the initial phases of backtesting, as limit orders are slightly less accurate to real-world resuls, and there is a danger of being deluded. . I like to make the initial hurdles as difficult as possible, and only towards the end of strategy creation/refinement to I test things like limit orders.
It holds up well prior to 2009, though not quite as well, largely because I use a longer period in the initial phases, and towards the end I focus on only 2006 to present, then finally 2009 to present in the last stages, often with an element of time-weighting to boot. My theory is that if I can establish a wide and robust initial foundation of a very large number of trades with a sizeable 'edge' over normalized profit factor/sharpe/etc, then I can slowly test various changes, additions, detractions, to try to make it even stronger and more robust (a process which organically prunes trades).
One of the foremost aspects of this is making sure the changes I make not only improve the strategies results over the instrument I'm creating it for (GC, in the above case), but also instruments which highly correlate with the given instrument, within the given hours of trading. . in the above case, that might be Silver and Platinum, for example.
Seeing that any additional entry filters, or exit conditions, or anything else I may add to the strategy, also improves it for these other instruments, indicates that the underlying logic is slightly more likely to be something *real*, and not merely curve fitting brilliance-on-paper.
It would take me awhile to explain just how I calculate 'correlation' as its more than just correlative price movement, but the main point here is that solid underlying logic should perform well across many instruments and markets. . anything that doesn't should be viewed with extreme suspicion, and should be forced to jump much larger hurdles and pass more daunting litmus tests before you consider taking it live.
Lastly, every strategy I run currently is intraday, though some hold positions up to 400-500 minutes, and often I'll manually hold my positions overnight if I seem to see a reason to. . otherwise I rarely 'overrule' the algo's. . virtually never, unless I'm extremely convinced I've seen a recent and consistent pattern of price movement it may be missing.
Each of my trade is entered with market limit and not market order. I don't use trailing stop loss and each of the trades is entered with a fixed stoploss and takeprofit. I don't expect slippage each time. For example if my SL is $200, I expect to see losses of $200 say 90% of the time and not $212.5 each time. I am new with futures trading, are you guys observing one tick slippage each time?
Are the futures brokers also using asymmetrical slippage? If there is slippage on the losses side, there should be slippage toward our favorable trades. In the past, several FX brokers have been fined for slippage on the loss side but never on the gain side.
In @fiverr's defense the commission and slippage settings seem to be per side, so it would be $10 for a round turn. Which in my opinion would still be too low to reflect what the strategy would have to deal with real time.
I'd suggest to start more conservatively with your slippage assumptions and rather get the nice surprise that it's a bit better in reality, than underestimating this amount and spend your time with systems that will break real time due to that right away.
Regards,
ABCTG
The following user says Thank You to ABCTG for this post:
There is slippage in the fixed stop loss as those are not limits but market orders. If you use a limit order stop loss you are playing a dangerous game as the market may not trade back through to fill your stop. The majority of your slip will come from the stop and it can be 1-3 ticks depending on how fast the market is moving and how fast your connection is.
For example, you still have a:
1. Delay in your data feed getting data to you
2. Delay in your trading platform in processing the data and generate signal
3. Delay in your platform sending order to broker
4. Delay in your broker sending to the market
All the while, a hundred algo's and pro's just got in and out 10x before you've even got past step 1 - and they all could be selling before you, especially at inflection points like the LOD. A big hitter can also drop 5k contracts and break 4 levels before your platform even knows its happened, dropping straight through your stop.
This can easily cause 1-2 tick slippage, with 1 tick being almost certain to happen quite often at key areas that break. You always need to cater for worse case scenario with slippage all the time. Your back test needs to survive the worst assumptions to even have a chance of surviving real ones. This is to cater for those times when you hit 2 ticks or more slippage in a very fast market. If you can only just scrape by, by making a few more favorable assumptions, it won't work.
Futures is not FX, there is only 1 market and whatever fill you get just depends on whoever gets there first.
You are not playing the same game, you would know all this if you've tried to trade the market before on a discretionary basis. As it stands, you have 0 chance of getting a data mined algo to work because you don't understand the basics of futures trading or the nuances of the ES, which is very complex. You can't just assume the same anything between FX and Futures.
Also I don't know what data you are using, but if you are using tick data with a tick based back test, it should simulate slippage and the real market better - though I doubt your data is granular as this since you don't have even have live data yet - another road block to making something work.
You're better off just spending sometime to sit through some sessions of the ES and do some reading.
In case you're wondering, I personally would not be happy with a system unless it has expectancy of over $50 with slippage accounted for. So even if for whatever reason I lose anther 2 ticks cause the world hates me, I still have $25 of expectancy left for the system to be profitable. My one has more than $50.
I also have one which does $400, slippage doesn't even register on this one's radar.
The following 2 users say Thank You to PeakGrowth for this post:
I am a bit confuse about the ES market. Some brokers told me that there are no spread and only commission per trade. But I know that some brokers have both spread and commission. Does anyone have live ES trading results that they are willing to share? I would like to formulate the correct specifications for my next revision of the code. Here are the information that I am interested in. Maybe we can use this as the benchmark/design specifications for everyone:
1) Should we or should we not hold a trade over night?
2) On your last 30 live trades of the ES, what is the total slippage did you experience? For example, if your stoploss is $200, you should have experience a loss of $6,000 ($200 x 30 = $6,000). Was your total loss more? Please share with us your real numbers that we can reference. Was it $6,500, $7,000 or $10,00?
3) Has anyone come across any literature that states that PF must be greater 1.5 to be profitable?
4) Has anyone experienced positive slippage?
I don't mind over designing my system to build more margin into it. It is good engineering practice. Unfortunately, we don't have the resources of NASA whereby we we can build in triple redundancy.
I finally got LIVE trading data from a friend who trades Emini S&P with a well-known broker and the numbers are shocking. I am surprised that this broker has not been sued for asymmetrical slippage practice. As I told you guys previously, my background is in currency trading; hence, I know all about asymmetrical slippage, see below.
For those that are interested in the numbers, here are my findings based on 168 trades (79 buy & 90 sell trades).
1) 82 trades match with TS backtesting
2) Only 7 trades have positive slippage: 6 trades with $12.50 and 1 trade at $25
3) 79 trades have negative slippage ranging from $12.5 to $275 and they costed him $3,925.
This is insane. It is so convenience for the losing trades to go against him; whereas, he did not benefit much from the positive slippage. Anyhow, I am not going to mention the broker name but this is crazy. My friend made some profit with his trading system but he should have made more.
