I have been chewing over a couple of questions for the past six months or so and I've decided I'm "stuck" because it's ground any system development/testing to a halt. So, I figured y'all might be able to give some input. Maybe I'm chasing meaningless answers but maybe there's value here? This post might get a little long...
I've been developing & testing strategies for several years. Like most, I've backtested, walk-forward tested, and over-optimized countless strategies in the last 5+ years with little end value to show for any of it and I think there's something fundamentally flawed with what I do (and probably most people?) as system developer.
Issue #1: System Development
Previously, when developing systems, I'd come up with an idea (or borrow an idea) for long/short entries, code it up, immediately throw on some sort of exit logic (reversal of positions, stops, profit targets, or maybe close at EOD or some time of day) and start running the system through the TradeStation optimizer with a "reasonable" parameters set. If the optimization looked OK I'd maybe do some walk forward testing or just run it in the background on a chart and watch it for a month or so to see if it worked or self-destructed on real-time data. Generally, they all self-destructed.
Late last year, I stumbled onto doing something which started me down this current path of questioning. That something was: time-based exits. I was testing a system and thought, "Maybe I should hold the position for X-bars before taking any exits other than a protective stop?" It was sort of a "sit on your hands for a while before exiting the trade" approach. After incorporating that into some systems my brain thought "I wonder how many orders my systems make if I were to close the trade instantly (same bar as the opening order)." At that point, when I would look at a system, I'd code up the entry and for exits, I'd have the system exit on the same bar as the entry. Once I did that on a few of my systems, I realized: No wonder all of these systems fail. They generate so many orders, in the same direction which you never see because the system is already in a trade so when the market "behavior" changes from the data which you've optimized on, you start "uncovering" entries your system was always flagging which are in "bad places" which then lead to more stop losses and drawdowns.
I've been trying to come up with an idea for a metric for how to quantify the "quality" of the entries but I haven't been able to...yet. I think the most promising method would be to compare each order to a local pivot point. The thinking there is: orders occuring closer to a pivot are "better" but, that is very much a hindsight calculation and would not be able to be used in real time but could maybe give a sense of how well the system is at picking turning points.
Issue #2: Charts / Data
In my backtesting efforts, I've tried just about every chart style: range, momentum, Kase, volume, tick, time, etc... I've tried different periods for each with different strategies and some combinations of strategies and chart intervals perform better than others but ultimately, each bar, regardless of chart type, consists of 4 single trades: First trade, Highest high, Lowest Low, and the Last trade (OHLC).
My problem with this is: if we look at an hourly chart on an ES, for example, there can be many thousands of contracts traded in that hour and our charts only care about 4 individual trades which happened in that hour. That could be a minimum of 4 individual contracts setting the "bounds" for each candle when 1000's of contracts were traded within those bounds. It seems like we our charts are throwing away mountains of data which is valuable to us. We can't really get away from using some sort of intervals for trading but maybe there is a better way to represent the data on the …