Have you ever wondered why a trading method or approach that backtests well or visually appears to give good signals fails when you take it to the market ? This drove me crazy for years and cost me some real money until I backed off and regrouped . It still mystifies me but I think I have explainations for this phenomena .

Firstly , do most would be traders approach a market arbitrarily or do they choose a market that they can identify the personality of ? Secondly , do they apply a method to that market that compliments the personality of that market ? Judging by what I see here most days Id say they apply a method that stimulates the eye to a market that was grabbed out of a hat . I mean that with due respect of course but few can deny this is happenning every day . Its natural to want to jump into trading with a canned approach and join in where the pack is involved but this is not a game in which the followers last long enough to pick up scraps from the leaders .

What I found to be a better approach is to choose markets in which I can detect a personality and then trade around that personality . We have the great benefit of tons of historical data at our disposal and most of it is gratis so its a good idea to exploit this gift . The greats of trading like Victor sperandeo , Richard donchian and Toby crabel to name a few succeeded at this game without the aid of computers , at least at first . They thought outside the box and were keen masters at feeling the markets pulse . Take the initiative and watch for telltale behavioral nuances that each market or pair offers because they all have them . Pick one pair or market and see what happens around news releases or lunchtime or when a market opens or closes across the globe .

After you find its pulse then just add tools to divine the hints that precede those moves you found to be inherent in your market of choice . Thats a "tops down" approach and a systematic way to do things that all experts in their fields employ , just ask one . Behave like an expert and you'll become an expert . If you want to get results that others cant seem to attain then why would you want to do or approach things the same way the others are doing them ?

Back testing often results in curve-fitted systems. That is your system is perfectly adapted to the past, but not the future. Ideally you would want to test your system on tomorrow's data but I have yet to find a data vendor who sells these data..

That is hard to understand but markets are not static, they change. And this does not mean mere price changes but the way markets behave changes. In statistical language: the underlying distribution changes.

Let's make an overly simplified example: you test a system for a market which moves up from the start of your data till present with some corrections in between. Your system tells you to go long after a retraction (small down move).

You go live with your system. At this point of time the market starts a serious down trend. You will buy in a down move, maybe see a little profit when the market retraces a bit up but then lose because it turns down again viciously.

There are all kinds of things to try to prevent curve-fitting e.g. walk forward testing, out of sample testing. They may help but all of them cannot avoid curve-fitting.

Change of trend is just a very simple example. There are changes of volatility, of liquidity, how markets react to external events etc. Testing very long time-frames does not help either, because markets change over time.

Optimizing parameters, signals, whatever, makes it worse only. There is a proverb: if you torture the data long enough they'll confess to anything.

Is back testing totally useless? No, You need a really good and preferably simple idea. And then look when it worked well and when not and try to figure out why.

Something that is hard to accept when trading is missing a huge move after you exit with a profit or missing the whole thing altogether . Accepting whatever happens is what makes great traders .

Getting out when your target is hit is an achievement and stopping out when your stop is hit is equally an achievement . Both events say something about you , you carry out your plan . Regret has no place in your trading regime and if you let it in it'll erode your confidence like rust on steel .

Stopping out at your predetermined level says that you refuse to be stubborn and insist on protecting your capital . Exiting at your predetermined level says you trust you abilities and your analysis and insist on building your capital base .

Watching price fly way beyond your exit level and regretting not staying in is a fools game . Praising yourself for sticking to your guns and letting the odds guide you is the route you take when you understand that trading is a game that depends on probabilities . Finding for yourself market(s) , setups , average move size , average # of waves etc. and knowing the statistics to guide you in and out of that market is all you need to know about trading .

Then when you stop out a couple times you have your stats to lean on and keep you from losing faith in your approach and finally yourself . You also have those same stats to keep you from getting high on a long winning streak . What I do to keep me centered while playing the market is when I stop out I recall that a winner is likely on the next trade and when I get my profit targets I recall that a loser is likely on the next trade . I dont expect a winner or loser I just remind myself constantly that the results of the last trade have zero bearing on the next trade - zero .

develop a strong conviction about their trade decisions

An excellent article, by Brent Steenbarger, in reference to the impact of non-stationarity on indicators, that is a " must read".

Every trader is familiar with what Victor Niederhoffer calls the “ever-changing cycles” within the market. Just as a pattern makes itself evident in the market, the pattern shifts to a new configuration. For example, the market may trade for a while within a range, offering nice buy and sell signals with a 14 period RSI. Then, abruptly, the market will break out and an overbought RSI will stay overbought or oversold for a prolonged period as the market makes a trending move.

It is because of these ever-changing cycles that traditional tools of technical analysis cannot be successfully applied in a purely mechanical fashion. The website Barchart.com has a nice feature where they track trading signals from such standard tools as moving averages. Over time, it is clear from their tracking that the signals do not perform better than random chance. For a while the signals will prove profitable, only to degrade once the cycles change.

It is ironic that traders spend considerable time researching better indicators and models while giving little thought to the time frame over which these trading tools might be valid. If, indeed, the market consists of ever-changing cycles, then any system or indicator is apt to degrade in its performance over time. In fact, if one waits for an indicator or system to develop a fine historical track record, the odds are good that their useful life are limited.

What can a trader do in the face of such uncertainty?

Stationarity

The statistician’s term for ever-changing cycles is stationarity. A number series is stationary if the process that generated the series has been constant. Clifford Sherry, in his excellent text The Mathematics of Technical Analysis explains, “A stationary time series is one in which the underlying rules that generate the time series do not change over time.” (p. 9).

My favorite example is the Las Vegas casino. Let’s say that you are playing blackjack and think that you have a superior card-counting strategy that will help you make money. By counting the number of picture cards vs. other cards that have been dealt, you can assess the probability of drawing a picture card on subsequent hands, tilting odds in your favor.

Such a strategy will work as long as the number of decks employed by the dealer is constant. If, however, the dealer intermittently and secretly changes the number of decks in the shoe, the card counting strategy would be imperiled. If the gambler assumed that twelve cards worth 10 or higher were left in the deck because eight had been dealt, the assumption would be faulty if two decks instead of one were being used. By changing the rules for dealing cards, the dealer creates a distribution that is nonstationary. Clifford Sherry notes the importance of nonstationarity for traders: “If you use these methods and techniques and find that your time series is nonstationary, it is probably best to stop and think carefully about your investment strategy. Nonstationarity implies that the underlying rules that ‘generated’ your time series change from time to time without warning. Therefore, you are dealing with maximum uncertainty about the potential outcome of your investment.” (p. 6). Sherry’s phrase “from time to time” is important. If, say, the card dealer changed the number of decks in the shoe after each and every hand, no card counting strategy would be possible. What makes counting viable is that the cycles are changing, but not constantly changing. A regime—a period in which the market follows a stable set of rules—can last for a while, allowing an alert trader to
profit while it is in force.

What should be clear is that a skill essential to trading success is early identification of regime change: those occasions when the cycles are shifting and the distributions of price changes are significantly varying from their recent norms. If a card counter can quickly identify when the number of decks in the shoe have changed, he can avoid betting his old system and take the time to develop a new one. Similarly, once a trader notes that market behavior has shifted, he or she can stand back and identify the new rules that the market is following and position themselves for the new regime.

Amazingly, very few traders bother to look for stationarity and even fewer shift their trading strategies according to the characteristics of recent price change series. This includes technical analysts who employ the same indicators and indicator values across all markets and quantitative traders who fail to properly adjust their lookback periods when testing a relationship between predictors and price change. Would we expect the time series from 2000 to 2002 to provide an accurate database for gauging relationships in the 2003 market? Did the market from 1998 to 2000 provide useful guides over the subsequent two years?

Assuming stationarity when it is not there is one of the cardinal errors of trading.If you are trading a pattern that has been valid in the past and you don’t know if thecurrent distribution of price changes match those from the past, you are flying blind. Successful trading requires that you identify the rules the market is following and base your strategy on those.

Fundamental Uncertainty in Trading

Let’s go back to that last sentence. The trader knows that there are ever-changing cycles, but makes a fundamental assumption. That assumption is that the regime that is in place will not change over the next trading interval. The trader assumes that the market’s rules will continue to be in force at least one more time. Without that assumption, the trader is either assuming randomness or is assuming regime change in the absence of concrete evidence of such. The only way we know if a regime has changed is by seeing an actual shift in the distribution of price changes. That means that there is a fundamental uncertainty in trading. The next trade may be the one in which the cycles shift. We cannot know for sure. Any trading strategy needs sound money management for this reason. Betting the house on a single trade—or during a single time frame—is courting ruin.

This has some interesting implications. For example, a well-researched trade that loses money may be an important source of trading information. If I have tested a historical period and found stationarity and then test a relationship between predictors and prospective price change over that period, my trade should have a high probability of success—if the market is remaining stationary. A losing streak with well-researched trades is often a sign that the markets are changing. Standing aside, waiting for evidence of the new regime, and remodeling the market over the more recent time frame corresponding to the new regime may allow the trader to learn from losses—and recoup them as well!

The fundamental uncertainty of trading is highest in daytrading the stock market —particularly index futures such as the SP/ES and ND/NQ. This is because markets are nonstationary on an intraday basis—almost without fail. Markets are most volatile early in the day’s trading, retreat to lowest volatility in early afternoon, and then pick up volatility toward the close (only to plunge in volatility during Globex trading). It is rare indeed that the distribution of price changes from 09:30 – 11:30 AM ET will match those of 11:30 AM – 13:30 PM ET. Using the same indicators and indicator values in morning trading as in early afternoon and Globex sessions is a sure road to the poorhouse. Conversely, identifying regime change and valid relationships with each intraday regime shift requires a nimbleness—and an ability to control losses—that most traders lack.

Interestingly, markets exhibit greater stationarity from day to day and week to week than from hour to hour. That is one of the factors that has sped my transition from intraday trading to swing trading. But if stationarity is as important to trading as Sherry and I believe it to be, then it makes little sense to pigeonhole oneself as a short-term trader, a long-term trader, a daytrader, etc. One should trade the time frames that offer the greatest stationarity. If the market is stationary over a period of weeks and if you can clearly identify the rules the market is following over that period, it makes sense to trade those rules. Later, the market may exhibit stationarity over a shorter time frame, covering a series of days. The rules that capture that regime will provide the basis for trading.

Many times, we hear of the distinction between mechanical and discretionary trading. This is a false dichotomy, because both mechanical and discretionary trading often fail to take ever-changing cycles into account. The real alternative to mechanical trading is flexible trading that searches for regimes and the rules guiding regimes, exploiting these in a rule-governed manner.

From Theory to Practice

Analyzing the market for trades should begin with tests for stationarity. In my new swing trading system, I begin my analysis by identifying the longest swing period in which the markets are exhibiting a stationary series of price changes. (There may be more than one such stationary swing period, permitting diversification of trading by time frame, and—of course—there may be stationarity for certain instruments and not others, permitting diversification by trading vehicles.) My procedure for assessing stationarity is to divide the time series into halves and statistically test to see if the means and standard deviations for the halves are equivalent. For readers interested in the math involved, Sherry’s book outlines a practical procedure for testing stationarity. The math is simple; I employ a quick-and-dirty t-test to the data and conduct the test entirely within Excel. What takes time is the repetitive testing of various lookback periods to find the proper window of stationarity.

Once I have that window, I then analyze the market qualitatively. I look at my indicators and observe how they have behaved during the stationary lookback period. The indicators that have consistently traced swing highs and lows over that period are the ones I will use to plan my next trade. I test signals yielded by the indicators (individually and in concert) over the lookback period to examine their entries, exits, and drawdowns. When I have a cadre of indicators that have performed well over the lookback period, I
rely on them for my next trade.

But that’s curve-fitting, you might protest. Isn’t it dangerous to overfit the data with an optimized model?

My response is that optimization is only a problem when you fail to take stationarity into account. If you know you are trading within a stable regime, it makes sense to do your best to capture the rules the market is following over that period. My swing trading methodology might best be described as serial optimization: continually hunting for periods of stationary market behavior and trading optimized models derived from those periods.

Now here’s the rub. When markets shift regimes, the window for the new regime is small. In testing the indicators that best follow the new, emerging rules, there aren’t enough instances to properly conduct statistical tests. That is where historical tests become important. By identifying past periods of market history where the markets were following the same rules as today, we can see if the indicators and signals that work in the recent lookback period also worked back then. The crucial assumption is that markets that exhibit stationarity and equivalent means and standard deviations in price changes are following the same rules—regardless of whether those markets were taking place in 2003, 1993, or 1983. If the strategy that we’ve optimized in the recent, stationary market window also produces profitable trading signals during past, similar regimes, we increase our confidence in the strategy and, indeed, can even test its signals statistically to ensure their departure from randomness.

Perhaps this is why we see so few traders incorporating stationarity into their analyses: It is time-consuming to assess market windows, operative trading rules, and test strategies for exploiting those rules. It is easier—and far more beguiling—to assume that a single system or indicator will produce consistent profits. More than one person has encouraged me to make my writing, research, and trading strategies less complex so that they can be more readily understood and accepted by the bulk of traders who attend seminars, buy trading books, and hire gurus for advice. One seminar organizer even fretted that I might be a threat to the self-esteem of traders, because the majority of traders lack the data and/or statistical background to conduct my kind of trading. I took that, of course, as quite a compliment.

Afterword

If you read the Trading Psychology Weblog with any frequency, you’ll notice that many of the charts that I post have a common—and seemingly random—starting date. For instance, as I write this (12/28/03), many of my charts begin with 8/1/03. This is not an accident. The period from August through December represents one of those stationary windows from which we can extract useful swing trading strategies. By posting charts over stationary time frames in the market, the Weblog can assist you in identifying tradable market patterns.

Incorporating stationarity into your market thinking and trading opens the door to innovative trading approaches. For instance, within a longer stationary window (several months), you might identify a smaller window (the past several days) for a short-term trade. By nesting and aligning the short-term trade within a longer-term pattern, you can formulate some high probability trades. Beginning January, 2004, I will be posting real time swing trades to the Weblog that take advantage of rules derived over one or more stationary windows.

Yet another avenue for research is the use of very short-term nonstationarities to identify points of larger regime change. A while ago, when I was exclusively trading the SP on an intraday basis, I noticed how short-term shifts in the NYSE Composite TICK tended to occur at points of trend change in the market. The short-term nonstationarity was a marker for longer-term trend change. I believe the same occurs at all time frames. By monitoring shifts in short-term patterns and indicators, we may be able to hop aboard early phases of regime change.