BackTesting and Programming Planning | Traders Hideout - futures trading strategies, market news, trading charts and platforms

Traders Hideout

Discuss day trading practices and futures trading strategies on this forum for all markets. This forum is also for discussing and reviews for brokers, data feeds, and commercial or third party add-ons


BackTesting and Programming Planning

  #8 (permalink)

Posts: 94 since Dec 2016
Thanks: 31 given, 45 received

tpredictor View Post
I will try to address everything. If you have a method that is simple enough to be tested then you should certainly test it. However, most discretionary traders integrate information from multiple sources in such a way that either they can't be backtested or simulated in any sort of trivial way or can't fully be backtested.

I should add a couple notes on the above. First, I am not saying that methods can't be backtested or quantified but that some methods are more difficult. Certainly as one's quantitative fluency develops then the ability to backtest more difficult hypothesis will develop, as well. Also, I should note that most simple quasi-mechanical methods can be backtested at least to some degree.

I think you wanted to know what high level traders think and do. The hallmark is synthesizing and integrating multiple types of data in non trivial ways that includes or anticipates other traders positions, reactions, or market forces in multiple paradigms, time frames, and modalities. At the end, I will show by way of example of that of complex non linear thought processes. Beyond that, the best possess at least some superior task specific skills in tape reading, technical analysis, etc.

Backtesting is form of empiricism, i.e. reasoning from past or statistical experience, but has certain considerable limitations when it comes to the market. It is closely tied to the idea of the scientific method. The scientific method does not seek to prove but merely simply seeks to excludes things. In a similar way, one can argue that backtesting doesn't prove anything will work but merely excludes things that did not work in the past. Quantitative thought is typically required for backtesting and is often associated with superior results.

The testing of most trade ideas does not require countless hours. The countless hours is the process of trying to build and refine something that works because most things break down in backtesting. Yes, I think probably getting Tradestation and testing your ideas is a reasonable step forward.

Below hypothetical examples of the type of complex/multi-dimensional analysis a top level trader might use:

"Quantitative models price in oil below $60 over the 6 months. However, we speculate OPEC will work in concert to drive up prices. We believe that if oil gets above $60 then certain large traders who use the quantitative models may be forced to hedge to cover which could temporarily spike price higher. We will use our exceptional technical analysis and price action to weight the probability of our thesis playing out. But our idea would be for a spike above $60 to drive out the quants and then a return to the normal projected quantitative range."

Here is another hypothetical example of the type of structure/analysis that professional traders might use,

"We have a quantitative model suggesting that the optimal stop level for the S&P 500 is around 2193. Our long intermediate models are thus positions long with a stop below this point. However, our combination of sentiment analysis and shorter term models are bearish. Our quantitative short day trading model is a trend based system. However, we will set a target of 2193 because we know that this will be the stop level where liquidity is highest which makes a good target and also represents an area where new longs may enter."

Again quantitative information is integrated in these examples but integrated in non simple/linear fashion. The optimal stop loss used as a trading target in one example under specific conditions. But yes, developing greater quantitative fluency will probably be as valuable as backtesting and backtesting is one method to develop that. These are static examples most-likely used by swing traders. A day trader is going to be doing similar analysis but in real-time. I do not mean to convey that trading methods that work have to be complex or at adds with the quantitative data. In most cases, the trader will want to take advantage of the historical probabilities. But merely that the hallmark of higher level trading is the ability to think about the market in multiple modalities and frameworks simultaneously. It could be as simple as combining market sentiment with technical analysis and behavioral understanding of other traders. Another example..

"A tape reader gets long an overnight rally before an important market event, i.e. report based on real-time tape read and technical analysis. However, this trader knows that it is unlikely that large traders will position before the event. After the market has run up 8 points, the tape reader is paying close attention already anticipating a potential sell off. Some unusual selling starts to hit the market and the tape reader hits out of the market with 7 points of profit." The trader didn't wait for some trailing stop loss to be hit but instead read a shift in the market that was already anticipated and acted on it at the opportune time. This required understanding the sentiment of multiple types of traders. The trader used superior tape reading and technical analysis skills, i.e. developed over time but also was able to integrate anticipatory other information.

There is one other shtick that is somewhat grating which is the idea that if something can't be falsified then it doesn't exist. It is powerful but false narrative. It merely means it can't be measured and most jobs pay more in line with the difficulty in measuring job performance. For example, take 2 CEOs. One CEO is very technical, very up on the market and very creative. Always innovating new ideas. Another CEO is not as up-to-date on the market, not as creative, but is very good at improving existing processes. Which CEO is best? The answer is we don't know. Even the performance quarter to quarter in the stock doesn't tell us. Because imagine the CEO that is improving processes is improving the bottom line on the short-term basis but might be missing huge changes that will eventually cause the company to collapse. On the other hand, the visionary CEO might be losing a lot of money but positioning the company to become a powerhouse on down the line. i.e It true that if something doesn't exist then it can't be measured. It is not true that if something can't be measured that it does not exist. I.e. instruments may not be precise enough or of the right type for measuring

Hello tpredictor,

Happy New Year, sorry for late response.

I really appreciate your response. I will need to read your response about 3 times to understand it. But definitely good information to know and learn from. Everything I do pertaining must be simple and straight forward.

My plan is as follows.

1. Re-Write my trading ideas/stragies so its clear and robotic thinking. I like trading resistance and support breakout trading because I have an edge with defining support and resistance daily. My discretionary trading is simple.

2. Manually backtest of 30-50 trades for each strategy idea.

3. Collect the right data in spreadsheet I have built today for the 30 trades. This part is challenging for me cause I manually backtest and want to make sure I record the correct criteria for data analyzing. I believe I have the spreadsheet ready to go.

4. If and only if the backtesting reveals a profitable system from 30-50 trades, backtest and record data for +100 trades.

5. Collect the right data in spreadsheet I have built today for the 100 trades. Review and analyze the date for future profitability.

6. If and only if step 5 is profitability, I will proceed to programming (I will have to learn) the system in TradeStation or NinjaTrader (still undecided) and confirm profitability over historical years of data.

7. If step 6 is profitable, sim trade for a month.

8. If step 7 is profitable, go live with one contract. And proceed from there.

Step 1 will be the challenge, but I have to keep things simple.

I like the manual backtesting because:

1. It tells me in detail how the system will perform and all the "what if this", "oh what about this scenario" , "what will the computer program do for this situation" , "oh, look this would be a good strategy to test later, let me write it down", "oh, I better record this parameter in my spreadsheet, cause I will need it analyze it", "this is too complex and too many what if factors, scratch it and lets try again"

2. Price action chart experience. Lets me review many 130 different charts for more learning along with day to day forward testing.

3. Better system design on paper for when its time to program.

4. Enjoyable. I actually like sitting down recording data chart for chart. It helps builds confidence in what I am doing.

There is some negative to manual testing such as kids bothering me. Or distractions. Problem solved with going to library for 1-3 hours and staying focus and fast.

I believe this approach is simple and doable for me. Along with learning price action. Overall, I do prefer discretionary trading, so we see how step 1 thru 5 goes. I may just stop them and skip programming and get right to live and discretionary trading a system.

The bottom line for a newbie or non profitable trader is to have a plan and keep on learning. Read, understand, keep it simple by all means.

Reply With Quote