Hello kevindog,

I appreciate your advice on keeping the iterations no more than 100 or even 200.

I think the premise behind this is:

1. Less is better. Adding too many inputs in the strategy leads to too much decision making.
2. Adding too many iterations leads to second guessing after the test is complete. Then we go round and round again with let me just try this and that.
3. If we get to deep in the overoptimizing process of the strategy, we eventually for get what we are really doing, which is confirm if the trading idea/method makes money with reasonable drawdown with in sample data.

I guess its the whole mindset of less is better.

Hello kevindog,

I appreciate your advice on keeping the iterations no more than 100 or even 200.

I think the premise behind this is:

1. Less is better. Adding too many inputs in the strategy leads to too much decision making.
2. Adding too many iterations leads to second guessing after the test is complete. Then we go round and round again with let me just try this and that.
3. If we get to deep in the overoptimizing process of the strategy, we eventually for get what we are really doing, which is confirm if the trading idea/method makes money with reasonable drawdown with in sample data.

I guess its the whole mindset of less is better.

Hey @goodoboy -

Whatever happened to this?

Hello kevindog,

I appreciate your advice on keeping the iterations no more than 100 or even 200.

I think the premise behind this is:

1. Less is better. Adding too many inputs in the strategy leads to too much decision making.
2. Adding too many iterations leads to second guessing after the test is complete. Then we go round and round again with let me just try this and that.
3. If we get to deep in the overoptimizing process of the strategy, we eventually for get what we are really doing, which is confirm if the trading idea/method makes money with reasonable drawdown with in sample data.

I guess its the whole mindset of less is better.

I use Multicharts for my optimization...
Also, in Multicharts at least, optimization speed is directly related to cores. You can select how many cores you want to utilize during optimization (and change on the fly, which is important for me). So I see you have 2 cores available for optimization. I have 16 cores available. Which means I can optimize 8 times faster than you, in Multicharts at least.

Hey @goodoboy -

Whatever happened to this?

For me at least, the only part of your 3 points I really agree with is "less than better."

"Too much decision making," "second guessing after testing" and "reasonable drawdown with in sample data" are not reasons I limit # of iterations.

But I guess the reasons do not matter as much as the final desired outcome - limiting iterations.

If I only had a nickel for every backtester who told me they needed a faster computer...

Then again, last November bought a top of line CPU to do more backtesting. I've run about 50 million unique backtests during that time. But never more than 100 iterations per strategy...

kevin how do you ensure accuracy in the strategy analyzer? I was optimizing a strategy that didn't need tick data, only needed candle close info and got wildly different results as compared to market replay for the same time period. I manually checked the market replay and it was spot on, but I don't feel comfortable with only 3 months of data on a trade

I use Tradestation, not Ninja, for strategy testing, so I can't really tell you what the issue is in your case.

I can say Tradestation is accurate when the strategy is set up correctly, and I am surprised at your comment about Ninja testing - I have heard it is good, too.

Kevin

yeah it's actually quite awful. Here are the results for a super curve-fit overoptimized dog-turd strategy I was working on:
From NT:
+13,000, 244 trades, 54,000 max drawdown
Actual
+21,000, 15 trades, 2875 max drawdown