ES strategy - looking for feedback

August 26th, 2020, 05:49 PM

0ilTrader

How many parameters are in your strategy? ~20
Over what dates did you optimize them? I didn't run any optimization yet, I just did a backtest from 2006 to 2020 with my most basic buy and sell signals.

Ok. Probably not overfit ... yet.

0ilTrader

Which Ninja Trader optimization method did you use (standard, walk forward)? standard, when I do the walk forward, I get an error saying there are no variables to optimize (something like that)

Ok, I get what you mean, you actually used 'backtest' ('Backtest' is not optimizing, unless you manually repeat backtesting while tweaking parameters for many cycles).' Two other options are 'optimization' and 'walk forward optimization.'

That error is, imo, a long standing weird bug in NT. Hacking the Genetic Optimizer C# code can greatly help, if you're comfortable with using Visual Studio and modifying code NT didn't intend for you to modify.

You can also limit the ranges of your parameters until it works, or hard code (requires unlocking the strategy and being able to code ninjascript/c#) most of them while you optimize only a few at a time; then hard code those optimized values and optimize a different group.

Repeat until no more improvement is seen after completing a whole round through all parameters. Also, you can try using larger 'increment' values for the parameters.

Another option is to use the grouping method above with 'brute force' optimization. That bug only seems to show up with the genetic optimizer.

0ilTrader

About how long did the optimization last (10 min., 10 hours, 1 day, more)? n/a
And about how powerful is the computer that ran the optimization (slow, average, gaming, etc.)? gaming

I'm curious if a strat that runs on the 1D chart is sustainable. Since I went back to basics, I need to see if I should find a better fitted time frame (in mins) instead. with the 1D tf I only get ~200 trades over 14 years

You can easily try different time frames while backtesting (assuming you have 1 minute data, downloaded or a feed. maybe tick data would work too?). If you get the optimizer working, it can have NT optimize for the best time frame.

So I guess now you have to decide if you're going to try again to optimize, and/or research the best time frame.

August 26th, 2020, 06:19 PM

JonnyBoy

It is difficult to assess in depth by only looking at a summary screen as I don't know how the strategy was built and what the parameters are etc.

Some quick observations:

The back test is showing a probability of 1.07%. This is a statistic that determines how likely a trade is to occur that would return the same PnL as your Average trade. It is based on how many trade's PnL fall within a standard deviation of the Avg. trade.

So the probability of near zero means that it is not likely that a given trade would be equal to the determined average trade. Your average trade is the sum of all trades PnL by the number of trades, so therefore you should expect that each trade live would not be the same as your average trade indicated by the back test. The accuracy of this probability can be best determined by using Market Replay (on a section of data) which will be real world accurate and essentially give you a better sense of live type performance.

Slippage is added or subtracted from your entry and exit prices for MKT and STP orders. No slippage is added to LMT orders. 1 to 2 ticks slippage is reasonable for the ES. Essentially if you went MKT at any non volatile moment on the ES on any given day there is a high probability you will get a fill within 2 ticks.

Thanks for the information @JonnyBoy - do you mind attaching a complete summary of one of your strategies over a long period of time, just so I can visualize what a solid strategy looks like in terms of statistics?

August 26th, 2020, 07:04 PM

0ilTrader

Thanks for the information @JonnyBoy - do you mind attaching a complete summary of one of your strategies over a long period of time, just so I can visualize what a solid strategy looks like in terms of statistics?

I'll be honest and state that your best bet here is to purchase @kevinkdog book. The link is here.

I am certainly not an expert, just quite experienced. However, Kevin is an expert and I started myself by reading his book.

August 27th, 2020, 05:59 PM

Any tips as to how to properly 'forward-testing' a strat that runs on the 1D chart? Replay is limited in NT IIRC.
I'll try to hack the optimizer tomorrow to see if other tf's are a better fit.

August 27th, 2020, 06:16 PM

0ilTrader

Any tips as to how to properly 'forward-testing' a strat that runs on the 1D chart? Replay is limited in NT IIRC.
I'll try to hack the optimizer tomorrow to see if other tf's are a better fit.

1. You use Strategy Analyzer and historical data for backtesting, not Replay.

2. You don't need to hack the optimizer to try different time frames.

3. How were you going to hack the optimizer, I never told you how?

August 27th, 2020, 07:39 PM

userque

1. You use Strategy Analyzer and historical data for backtesting, not Replay.

2. You don't need to hack the optimizer to try different time frames.

3. How were you going to hack the optimizer, I never told you how?

First, I hack for a living.
Second, I saw a few posts about it in ninja already.

I know I don�t need to hack the optimizer to run the backtest on multiple time frames, however I�m curious as to which tf the optimization will find as the best one.

August 29th, 2020, 01:35 AM

userque

And about how powerful is the computer that ran the optimization (slow, average, gaming, etc.)?

I couldn't begin to confidently offer meaningful advice without knowing these basics. And after reading your answers, I still may have to ask another question or so.

How does the power of the computer affect the outcome of the test? May be the final excuse for me to get that new computer.

August 29th, 2020, 09:33 AM

Bionan

How does the power of the computer affect the outcome of the test? May be the final excuse for me to get that new computer.

I was trying to get some idea of the degrees of freedom. Shorter run-time, fewer degrees. But a powerful computer would require less time to run the same test, so I wanted to know about the computer. More degrees of freedom, greater chance of over-fitting.

But to actually answer your question:

If you have a lot of parameters, with wide ranges, and you want to test as many possibilities as possible, then you'll need a lot of computing power if you want the test to complete within this century. Some folks have multiple servers running searches.

August 29th, 2020, 11:11 AM

Hello,

Use diferent periods and you will see that nice green -> red.
The strategy analysed only is in Long. But when the market is trending down the behaviour is another.

You should use market replay taking the most close dates up today and see what happens. Then try some months before and check again.
Try to find an ALL terrain for almost all markets conditions. Up - down and Ranging. (In this condition Halt the strategy ...better dont trade)

Sent using the NexusFi mobile app

August 29th, 2020, 01:54 PM

bcomas

Hello,

Use diferent periods and you will see that nice green -> red.
The strategy analysed only is in Long. But when the market is trending down the behaviour is another.

You should use market replay taking the most close dates up today and see what happens. Then try some months before and check again.
Try to find an ALL terrain for almost all markets conditions. Up - down and Ranging. (In this condition Halt the strategy ...better dont trade)

Sent using the NexusFi mobile app

Does a more powerful computer provide more stability when running market replay at faster speeds? What is the fastest speed you would be comfortable running market replay and still trust the results?

ES strategy - looking for feedback

Discussion in NinjaTrader

ES strategy - looking for feedback