Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
in my opinion, back testing, optimization and real time simulated testing are the best tools you have available to put together an automated strategy that does make more money than it loses.
the first part of my advice to you would be that you learned how to optimize the settings for the parameters in your strategies, as it is apparent you do not know how to do that. all the most popular trading platforms offer optimization engines, learn how to use them for the particular platform you happen to be using (there are even video tutorials availble on the internet). make sure to include realistic and even exaggerated values for commissions and slippage which all will be going against you all the time.
if your strategies lose more money than they make even when all the parameters have been optimized, you can be sure they will never work and it would be a waste of time to continue trying to develop them.
second, maybe the most advantageous thing you can do to determine if your strategies are winners or not is to optimize them always leaving a sizable period of time out of sample. optimize the parameters in your strategies over long enough periods of time (4 years or more), so that, ideally, your strategies will be tested on all kinds of market conditions; uptrends, downtrends and flat / ranging periods. leaving an extended period of time out of sample means that you would never optimize your strategies to the present day and consider those results to be overly credible, always optimize up to 6 - 12 months ago. once you have optimized your strategies, let's say, to 12 months ago and they made money up to that point, and then you included the latest 12 months of market data keeping the same values for your parameters and your strategies still turned a profit consistently over that period, then you can be certain you have a winner.
the next step would be to test your strategies for a number of weeks as they make trades in real time on a simulated account. if any of your strategies makes more than it loses over all these tests (remember to always use realistic / exaggerated values for total commissions and slippage when evaluating simulated results), you can be certain that as long as the market keeps moving in fairly similar patterns as it has done in the past you could make real money going forward.
thanks for the advice I will look into testing with this approach as I think the problem I have been having is only backtesting and not doing any out of sample testing / walk forward.
So here's a couple of the problems you're going to run into that people often don't consider. I use Ninjatrader to backtest, but these considerations are applicable to every platform.
Does it run the same way in in live vs backtesting? If your strategy is based completely on closing prices for signals then you're probably ok. That means you have to wait for the close of the bar though, which means the strategy is always late. One alternative is to use a shorter timeframe bars like seconds instead of minutes. Then the problem is it requires more data to backtest.
What is the max drawdown and longest losing streak? Many automated traders curve fit their targets and stops which inevitably leads to having a larger stop than your target. However, a larger stop could result in worse drawdowns. Here's where the losing streak is important. Is it significantly worse than what can expected based on chance alone? If it is then that could be a sign that the strategy was bullied in the past.
What are the risk adjusted returns? Generally speaking a strategy will do better the more it is in the market. Some strategies do this by really pushing winners. Others do this by constantly changing directions, and always being in the market. So for instance it's a lot more common to see swinger traders with a positive P&L. However, if you're in the market for days at a time then you have to consider what the performance would be if you just bought and hold. To determine if the signal really has edge outside of the long holding periods then you need to look at risk adjusted metrics like Sharpe ratio.
What is the chances of it working in the future? Past performance is not indicative of future results. The probability stat in Ninjatrader is one way to look at this as it tells you how consistent the strategy is. You can also run Monte-Carlo simulations. Ultimately though, you'll need to forward test it both on live and sim to really know.
TLDR: You need to look at more than just total net profit.
Little something to chime in here, as most other retailers in market I've played with so many indicators and setups that I've lost count, if it wasn't for backtesting I think I might not have enough money or time in this life.
So, its very useful. At least it is one of the very few tools retail traders actually have to do their fair work and expect some results. Its much better than me imagining a strategy with some indicators I thought traded the way I wanted, and then blowing my account to realise how wrong I was. At least with backtests that can be avoided. Granted you are not just looking at the % profit column. Its surprising how many people actually "only look at that".
It is a waste of time. Trading systems are not the way to trade profitably. They only work for a short amount of time, then they get out of sync with the market and you give all back.
Think of the market as a sinewave, and your system another sinewave that is out of phase with the market. At one moment they will sync and it will give you the illusion that the system "works".
Now, if you are using a simulator to replay the market and practice, oh yes that is a must and very necessary. But you have to master a method like VSA (volume spread analysis), which is mostly discretionary but still retains some "rigid" fundamental principles.
I tell you that in all these years of reading and watching all trading books/courses you can think of (and perhaps you never heard about), thousands of hours in this, studying, practicing, losing money, the only method that really taught me how to read a chart properly was VSA. When I read the "Master the Markets" book by Tom Williams, that was a game changer, I just ditched all the rest (never worked, only lost money with classic TA).
There is a reason the retail trading education industry pushes this kind of education (combination of different indicators, chart patterns and candle patterns): they make you lose money and provide liquidity for the smart money. They know exactly what are you doing.
Whilst I agree with some of your points above, as a very competent back tester I can tell you that back testing isn't a waste of time. However, for it not to be a waste of time it needs to be executed correctly.
This is the biggest reason why some people think back testing is useless, because it didn't work for them. However, their execution was likely flawed to begin with. They were attempting to design an 'all rounder' strategy that would work for 12 months of the year in all types of market conditions, ringing up the cash register every day.
This isn't how it works. You don't buy one coat to wear all year because it will not perform well in every season, which is exactly like the market and back tests. Your comment about Sine Waves was a great analogy to this.
Knowing when to deploy a strategy is the key to successful implementation. Many of the strategies available to the public are trying to balance good performance by being in the market continuously. IMO this isn't the correct way to do it, but who is going to pay for a strategy that only runs on gap up or gap down days? Probably not many, so they compromise.
In conclusion back testing is not a waste of time. If executed correctly and deployed (turned on and off basically) at the most opportune time established by the correctly executed back test, they can be very rewarding.
I could hand out a long and short strategy in GC tomorrow. It is doing quite well, but it only runs for a few hours per day so the return is slow to build but steady. I would never have known the most profitable time frame(s) in which to run these without a back test.
--------------------------------------------------------
- Trade what you see. Invest in what you believe -
--------------------------------------------------------
Now I think people are getting more into curve fitting issues. So the first approach would be find markets where the setups do well and don't do well. Find higher time-frame indicators to identify those conditions and change how you trade it. Or the more common approach would be just select those times when the strategy does well, and only trade those.
Both are curve fitting though. You're taking past data, and modifying to strategy to conform to that data as best as you can. Such methods are no more or less likely to be successful than any other approach based on purely on technical analysis because it the strategy is still entirely premised on inspecting past data.
There is nothing mystical about backtesting. It is just testing what you would have done if you were 100% disciplined with your plan.
It is true that a winning backtested strategy may fail or become caduc in real trading.... but you are 99,9% sure that a strategy that is failing in backtesting, will fail in real trading.
So the first step is to have a winning backtested strategy.
Then see how it does in real testing.
Also some strategies are more prone to backtesting and automated trading than others.
Backtesting a tick scalping strategy on illiquid instrument will fail, guaranteed.
Backtesting a strategy on longer time frame, where you would enter and exit with limit orders on real trading (only stop losses will be prone to slippage) on very liquid instrument will be much more realistic and replicable in real trading.
