Recap: what we've learned so far

Let's just re-cap what we've learned to date...

* Our pullback system as it currently stands gives us approximately 48 opportunities over a 61 trading day time span, or 62 opportunities if we permit re-entry (complex pullbacks). This includes both regular pullbacks and breakout pullbacks with decent range size.
* We've been able to establish that over our chosen sample size, a more sophisticated stop trailing strategy can produce significantly better results than a primitive 'trail the last candle' approach.
* We've seen some evidence that permitting second entries might enhance our PnL.
* We're experiencing serious underperformance in 2/3rds of the sample, showing a downward trend with both exit management systems, suggesting either poor trade selection or a bout of 'bad luck'.
* We haven't been able to make any statistically significant conclusions about the expectancy of our strategy due to its variance and small sample size.

All in all, my target is to have a system that offers a 0.2 %R expectancy net of frictions (my %R calculations, incidentally, do not include frictions while my point PnL does - but this does not affect our work to date).

If we take a look at the best performance we've gotten, it has yielded 29 CL points net of frictions, and 51 CL points if we include re-entries. The broker, meanwhile, made the equivalent of 24 and 31 CL points respectively, and 48 and 64 CL points went to slippage. That means our profit is merely 40% and 53% respectively of frictions. Now you see what they mean when they say how difficult it is for day traders to get over the 'vig'. This is what we're up against.

Clearly we don't have a satisfactory system yet, but at the very least these exercises have given us a helpful point of departure.

Next on our agenda we have to explore the area of trade selection. If we could just get rid of 2 1.0x %R losers we could increase our PnL by a factor of 2. Of course, this also means we may have to eliminate some winners as well, so good trade selection may not immediately prove its worth.

Naturally it will also mean we'll have to be disciplined enough to let go of those opportunities when it comes time to trade, especially if we're going through a draw down like we were in February and the market becons us with tempting signals that may have followed through once or twice.

A plus we can look forward to is we may even discover trades that work well in the opposite direction at times, including opportunities where we may want to reverse our position. It is important to remember, however, that not every failed trade should be a reverse, and this should be approached as if you are developing a separate strategy (note, for example, that we can likewise treat our 'second entries' as a different strategy and refine its criteria, too).

We can also work at improving our exit system further, possibly using the 512 tick chart to get a finer read on things.

Lastly, once we're ready we can increase our sample size and see what kind of real variation is out there. That may force us to change our rules again and it may also give us ideas how to change as the market changes.

Screening Trades: Applying a Filter

Let's see if we can get a picture of what is going on on the larger scale of things. I've created this interesting chart that features the trading signals provided by the system over our three month sample. The candles represent a trading session (9:00 - 14:30 EST), with the trades of the day on top of the candle, with re-entries on the lower end of the candle.

If you see what the key says, the red plusses are the successful short trades, and the green plusses are the successful long trades, while the red minuses are the unsuccessful short trades, and the green minuses are the unsuccessful long trades. I know, it's a bit confusing at first, but try to disassociate the color red and green from PnL and think of it in the bull and bear context only, with the + and - sign substituting for PnL.



Below is the Average True Range, an indicator that lags but gives us a hint as to volatility.

Keep in mind when you're looking at this, the trades are made without the benefit of knowing how the candle closes for the day. This is a very important aspect in our testing, we have to be on the look out for future information creeping into our analysis.

One thing struck me right away when I saw this: long term bias. Clearly crude was in a down trend, then in an up trend, then reversed down once more, then up again. There are also various degrees of trendiness and session overlap. Could success or failure be hidden in taking pullback trades in less than favorable trend conditions?

I decided to look at trend bias at first. I used the 21 period EMA as a filter. In Excel, I created a condition that if the PRIOR DAY closed above the 21 pd EMA on the higher time frame, then only long pullback trades were permitted going forward. The same condition held for the short side, only in reverse: if the market closed below the 21 pd EMA, then the market was a short for the day. If the market closed at or very close to the EMA, no trades were permitted.

The results I got were very interesting. Instead of having 48 trades on the no-reentry system, I got 18 a mere trades. Applying the filter to the re-entries, I got an extra 5 trades instead of an extra 14. In each case, the signals went down by approximately two thirds.

The performance of the system, using our "B" exit criteria (from now on this will be our default exit system), we were up 68 CL points with the no reentry system. Reentries added another 15 CL points to our PnL, giving us a total of 83 CL points (interestingly enough this is less than what we would have gotten had we taken unfiltered second entries, but with just five second entry samples in the EMA filtered system we can't draw any solid conclusions). All numbers here are net frictions.

Wow, that's awesome! Our signals went down in number but we experienced a dramatic improvement in performance at the same time.

In case you're wondering, adding long or short trades when the prior day closed AT the moving average slightly degraded performance for both subsystems.

Here is our performance on a trade by trade basis:



Looks interesting, but our only problem is that we now have a small sample size again, so that makes it difficult to draw any statistically significant conclusions.

Still, we have potentially honed in on an interesting concept - a filter that may have drastically reduced our trading signals (now we'll be getting as few as 1/3rd of the previous opportunities), but more than DOUBLED our PnL!

Sure, we got rid of some profitable signals for certain. But more important, we got rid of the stinkers which were eating away at our precious account, and now we're richer for it, even though we're much more bored since we have to sit and watch trades pass by.

Now does this mean we can idly slap on the 21 pd EMA, wait for a qualified pullback, and then feel alright? It's not that easy.

We may be able to still get rid of some more stinkers that fit under the EMA filter, and we may also find some nuggets from the trades we've filtered out.

If we take all the profitable trades that DON'T fit the EMA filter from our no re-entry system, we find that we have 12 out of 30 rejected trades (40%) which could have contributed to our PnL. Those profitable trades total to 162 CL points. If we add the profitable trades from trading the re-entries, we get an additional 47 CL points, totalling 209 points of good trades we can't enter simply because they fail the filter.

If we could grab just a QUARTER of that currently rejected PnL via better entry criteria, we'd be up an extra 40 points for our simple pullback system and 52 points when second entries are included. That is a 58% and 62% potential increase in PnL.

We could also, theoretically, go the other way - trade only EMA filtered trades and find other criteria to further pare them down to increase profit. We might get an even tighter performance out of our system. However, this runs the risk of seriously depleting our opportunities and also introduces the risk of becoming too curve fitted. Caution is required here.

Points of Departure



Now we're at a juncture where we need to make some decisions as to which direction to take our research in.

Our initial goal was to cast a fairly wide net of trade setups and not be too picky about it. There are several reasons for this.

First, when operating with the benefit of hindsight, it's very easy to say 'hey, this is a successful trade, of course there was a signal here", and the opposite when we see a failed signal "but of course that was a bad trade, should've been obvious". I tried very hard to look at the criteria I established and apply them as broadly as I could.

Second, I don't want to ascribe selection criteria preemptively without seeing evidence of what makes for a good versus a bad trade firsthand.

What we need to start doing now is becoming more specific on why something fails and why it succeeds, so we can begin to properly parse our A|B trades (the trades that conform to our system), and our C|D trades (the trades we exclude from our system).

We're separating the profitable trades into A's (system trades that work) and C's (non-system trades that work), and the unprofitable trades into B's (system trades that failed) and D's (non-system trades that failed).

We can begin by creating the following sub-systems:

1) Pullback trades.
2) Breakout pullback trades.

and in turn, each of these sub-systems can be further broken down into:

a) Simple entries.
b) Complex entries.

We can illustrate them like so:



This might or might not be helpful for us, but I think it makes it a bit easier to separately look at some specific things that we otherwise wouldn't pay attention to. For example, we would want to look at the strength of a breakout:



Notice how in the example on the left, which succeeded (albeit on the second try), the bullish breakout was fairly strong. In the example on the right, which failed quite drastically, the bearish breakout (A) was considerably weaker.

Also, we may want to see circumstances in which it would be wiser to wait for a complex pullback as opposed to a simple one, for example when the trend shows signs of weakening.

Once we look at enough samples, we may begin getting ideas for a counter-strategy - that is knowing when to stand by and wait for a failure:



A failed pullback can be thought of as a double top/bottom, or what Al Brooks calls a 'higher low/lower high major trend reversal'. It might not be necessary to analyze this for the sake of possibly trading it, but it would help in terms of being able to not enter or exit early if we notice the environment is conducive to a counter-trend move.

Now, in our last analysis we discovered what might be an effective trade filter, a moving average, that reduced the number of trades but increased the profit by a factor of 2.8. The rationale is simple: we're taking only those trades that are in alignment with the longer term trend, which is signaled to us by the moving average.

However, two caveats: 1) We did not test this filter over a very large sample size, and 2) We are leaving some potentially good trades on the table. If we can find a way to separate out those good trades that do not fit our 'filter' but merit being taken, we may add to our PnL.

To this end, our next step should be to better categorize our existing trades and see if we can do some weeding out of inferior signals. This is not an easy task for one reason: it requires a lot of discretion and as a result involves subjectivity. That makes it difficult to test, as we are going to be even more prone to hindsight bias as we're trying to optimize the criteria on observable trade samples. During back-testing we will be able to get a better idea of what we could reasonably expect, but even that will never give us the real picture until we enter the live market.

Does this make analysis fruitless? Absolutely not. It just means that we need to be aware of our limitations going forward so we're not disappointed later on.

Absolutely fascinating thread thank you. I am also heavily into pullbacks and think that you would find anything written by Lance Beggs extremely interesting.

Andy