NexusFi: Find Your Edge


Home Menu

 





What should be the next step in my backtesting??


Discussion in Emini and Emicro Index

Updated
      Top Posters
    1. looks_one budfox with 40 posts (2 thanks)
    2. looks_two treydog999 with 21 posts (25 thanks)
    3. looks_3 tellytub with 6 posts (4 thanks)
    4. looks_4 Fat Tails with 2 posts (1 thanks)
      Best Posters
    1. looks_one Ming80 with 2.5 thanks per post
    2. looks_two treydog999 with 1.2 thanks per post
    3. looks_3 shodson with 1 thanks per post
    4. looks_4 tellytub with 0.7 thanks per post
    1. trending_up 16,301 views
    2. thumb_up 41 thanks given
    3. group 9 followers
    1. forum 76 posts
    2. attach_file 4 attachments




 
Search this Thread

What should be the next step in my backtesting??

  #11 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013


Ming80 View Post
Hi BudFox,

Glad you've built your first model. Just like you, when I started my first model from excel I had tons of questions and wanted to share any some info that might be of use to you.

1. On Backtesting Software - Treydog has given excellent points. Apart from the speed and scale which you can backtest many markets/asset classes, there is also the huge benefit of automated execution. I found my usage of time much more productive in research instead of monitoring trades. One can now trade up to 20 or more positions simulatneously (long term or short term). The caveat of course is that the model developed is accurate and robust and monitoring systems from time to time is still a prerequisite.

2. On Monte Carlo - It is an interesting measure but wouldn't put too much faith in it. Actual drawdowns are a function of a specific market events/environment in relation a series of bad trades produced. Trade scrambling removes the serial correlation which happens during this period and how the drawdown was actually produced. It would be useful to look into the periods which drawdowns occured and understand what caused them. This is crucial to deduce if risk management approaches are needed (stopping the trading model, develop non-correlated models etc.) ... All these decisions have 'costs' to the long term outcome of the eventual pnl. Alternatively, one could accept that this is a function of markets where drawdowns are inevitable from time to time and just continue to stick to the plan.

3. Number of trades - Perhaps instead of measures of statistical significance, one could take a qualitative approach and see what the system was designed to capture. Warren buffett as well as many long term trend followers although might fail statistical tests by their number of trades, but their performance is certainly more than random. One needs to take into account the holding frequency and frequency of conditions present.

4. Clean Data - Good models need to be built on clean and more importantly adjusted data. Even accredited data providers do not provide properly adjusted data for backtests and will lead to incorrect trading conclusions. Thus, the need for backtesting software to adjust data to generate a proper backtest.

Lastly, I personally found the most crucial parts in building good trading models was in thinking and capturing the stationarity of the market. I got this general idea from visiting the local library and reading books by clifford j sherry. The 2nd part was asking myself how long this 'stationarity' would tend to persist. The better which these 2 questions could be answered, the better the models have performed going forward. I hope some of my personal experiences would be of use to you going forward.

Thank You for your advice and understanding I am new backtester,

could you explain to me what you mean by stationarity of the market?

I still want to read your post more carefully so I can come back with more intelligent answers.

Started this thread Reply With Quote

Can you help answer these questions
from other members on NexusFi?
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
New Micros: Ultra 10-Year & Ultra T-Bond -- Live Now
Treasury Notes and Bonds
ZombieSqueeze
Platforms and Indicators
Deepmoney LLM
Elite Quantitative GenAI/LLM
Better Renko Gaps
The Elite Circle
 
  #12 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013

I have been able to replicate most of the conditions from my manual excel model into Strategy Analyzer, but since my rules are quite particular i think it will be extremely challenging, if not impossible, to replicate all.

I think I will inevitably have to learn to code in Ninjascript (how hard is it?) .

Using Strategy Analyzer is easier from the perspective that I can more quickly backtest more trades, but manually is easier from the perspective that I do not have to figure out how to "code" my conditions into the software.

My conditions are a lot more complicated that "Go long" when ma crosses over pivot etc.

Its more like go long if this bar closed higher than this bars open ONLY during this timeframe, BUT skip the trade if you have this divergence.

It's the skip the trade when you have this divergence condition that I am trying to "code".


I really appreciate the help and support from some of the members of this great forum.

Started this thread Reply With Quote
  #13 (permalink)
 
treydog999's Avatar
 treydog999 
seoul, Korea
 
Experience: Intermediate
Platform: Multicharts
Broker: CQG, DTN IQfeed
Trading: YM 6E
Posts: 897 since Jul 2012
Thanks Given: 291
Thanks Received: 1,039



budfox View Post
I have been able to replicate most of the conditions from my manual excel model into Strategy Analyzer, but since my rules are quite particular i think it will be extremely challenging, if not impossible, to replicate all.

I think I will inevitably have to learn to code in Ninjascript (how hard is it?) .

Using Strategy Analyzer is easier from the perspective that I can more quickly backtest more trades, but manually is easier from the perspective that I do not have to figure out how to "code" my conditions into the software.

My conditions are a lot more complicated that "Go long" when ma crosses over pivot etc.

Its more like go long if this bar closed higher than this bars open ONLY during this timeframe, BUT skip the trade if you have this divergence.

It's the skip the trade when you have this divergence condition that I am trying to "code".


I really appreciate the help and support from some of the members of this great forum.



If your going to be serious about backtesting strategies, your going to have to learn to code. Its part and parcel of the whole backtesting strategy development method. With out it your just going to probably not get the kind of system or results you were looking for originally. Like you mentioned the point and click rule set is not specific enough, and its your first system. you already reach the limit of what that type of analyzer can do. Its better to invest in yourself and learn to code then it is to struggle with sub par tools that you can easily grow out of anyway.

Reply With Quote
Thanked by:
  #14 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013


treydog999 View Post
If your going to be serious about backtesting strategies, your going to have to learn to code. Its part and parcel of the whole backtesting strategy development method. With out it your just going to probably not get the kind of system or results you were looking for originally. Like you mentioned the point and click rule set is not specific enough, and its your first system. you already reach the limit of what that type of analyzer can do. Its better to invest in yourself and learn to code then it is to struggle with sub par tools that you can easily grow out of anyway.

Ok I have absolutely zero experience in how to code. First, what language should I learn to code in? Java? Python? C+?

I am currently using NT Strategy Analyzer (since its free)

Started this thread Reply With Quote
  #15 (permalink)
 
treydog999's Avatar
 treydog999 
seoul, Korea
 
Experience: Intermediate
Platform: Multicharts
Broker: CQG, DTN IQfeed
Trading: YM 6E
Posts: 897 since Jul 2012
Thanks Given: 291
Thanks Received: 1,039


budfox View Post
Ok I have absolutely zero experience in how to code. First, what language should I learn to code in? Java? Python? C+?

I am currently using NT Strategy Analyzer (since its free)

easylanguage is the "easiest" although very specific to the tradestation or multicharts platform. Learning a more "legitimate" programming language Java, Python, R, or anything in the C family will give you more power and not be bound by a specific platform. It depends on your goals, how much you are willing to put in to study and learn a specific one. I would say that easylanguage is the best way to dip your toe in, and be able to test a very many different systems and get them to run and execute automatically. If I were to pick another language it would be from the C family, either C++ or C# this is because a lot of programs use it (including ninja trader).

R and Python I put into another category because they are very high level scripting languages. Powerful, useful and easy to develop for but are not very well integrated into other platforms.

Reply With Quote
  #16 (permalink)
NoiseTrader716
Buffalo, NY
 
Posts: 64 since Jul 2014
Thanks Given: 4
Thanks Received: 38


Ming80 View Post
H
3. Number of trades - Perhaps instead of measures of statistical significance, one could take a qualitative approach and see what the system was designed to capture. Warren buffett as well as many long term trend followers although might fail statistical tests by their number of trades, but their performance is certainly more than random. One needs to take into account the holding frequency and frequency of conditions present.


No doubt. I think there is some confusion here about repeated trials process vs a Bayesian statistical measure.
A 20% chance of rain tomorrow doesn't mean that tomorrow is going to happen a 100 times and 20 times it rains and 80 times it doesn't. Since you are only going to get one sample of tomorrow the 20% prediction is a reflection of the belief in our information at hand.
Of course running a simulation of the weather tomorrow 100 times can turn this into a repeated trials process but the prediction is only going to be as good as the model. You wouldn't put any faith if the weather model is based on flipping an unfair coin and summing the results but that is basically what that montecarlo spread sheet is doing.

You can also consider only using backtesting to find a model of interest but then only test in real time if the time frame is short enough that you can still put on real trades without things having completely changed. Things like max draw down in a backtest are beyond useless IMO. The only certainty we have is that the future will not look like the past.

For learning to code, learning C# in ninja would be the way to go because you can learn by doing and creating the very thing you are interested in this thread without having to bother with learning all the plumbing and boring stuff.

Reply With Quote
  #17 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013


treydog999 View Post
easylanguage is the "easiest" although very specific to the tradestation or multicharts platform. Learning a more "legitimate" programming language Java, Python, R, or anything in the C family will give you more power and not be bound by a specific platform. It depends on your goals, how much you are willing to put in to study and learn a specific one. I would say that easylanguage is the best way to dip your toe in, and be able to test a very many different systems and get them to run and execute automatically. If I were to pick another language it would be from the C family, either C++ or C# this is because a lot of programs use it (including ninja trader).

R and Python I put into another category because they are very high level scripting languages. Powerful, useful and easy to develop for but are not very well integrated into other platforms.


If I had to use easylanguage, I would only be able to use tradestation or multicharts?

Maybe I should learn C# since that is what ninja uses, and I am currently using their strategy analyzer?

If you were my boss and you told me to learn C#, how long would you expect me to have a working knowledge of it? what deadline from 8 September would you want me to be able to use it by?

Started this thread Reply With Quote
  #18 (permalink)
 
tturner86's Avatar
 tturner86 
Portland, Oregon
 
Experience: Intermediate
Platform: F-16CM-40
Trading: GBU-39
Posts: 6,191 since Sep 2013
Thanks Given: 10,459
Thanks Received: 12,695


budfox View Post
If I had to use easylanguage, I would only be able to use tradestation or multicharts?

Maybe I should learn C# since that is what ninja uses, and I am currently using their strategy analyzer?

If you were my boss and you told me to learn C#, how long would you expect me to have a working knowledge of it? what deadline from 8 September would you want me to be able to use it by?

OEC uses easy language. Futures Online, Futures Trading Platform, Futures Trading Demo, Electronic Trading System

Visit my NexusFi Trade Journal Reply With Quote
Thanked by:
  #19 (permalink)
 
treydog999's Avatar
 treydog999 
seoul, Korea
 
Experience: Intermediate
Platform: Multicharts
Broker: CQG, DTN IQfeed
Trading: YM 6E
Posts: 897 since Jul 2012
Thanks Given: 291
Thanks Received: 1,039


budfox View Post
If I had to use easylanguage, I would only be able to use tradestation or multicharts?

Maybe I should learn C# since that is what ninja uses, and I am currently using their strategy analyzer?

If you were my boss and you told me to learn C#, how long would you expect me to have a working knowledge of it? what deadline from 8 September would you want me to be able to use it by?

Yes, C# is a great language and is very versatile. Honestly as someone who knows C++ ,the C family its really very common and powerful.

I like that question, although I want to put another spin on it. As in your boss wants you to be able to fluently write code in ninja trader using C# to develop trading strategies. I would say learning on an everyday basis in 90 days or so you will have learned enough to start playing with simple strategies in ninja trader and start to be able to get results. After that your going to be probably pretty fluid in using their functions, shortcuts, and classes/methods that it will get faster and you will spend less time debugging and looking up arguments or syntax. After that its like a real language, the more you use it the more fluent you get.

Honestly trading systems developers want to make it as easy as possible to get started so they add a lot of very useful functions that are ready to use out side of the box. I have never worked with ninjatrader but once you learn the basic code structure of a C program and how it works you should be able to piece together whats going on. Use examples and pre build strategies, dissect them line by line and see how they work.

Reply With Quote
  #20 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013


I'm actually trying to figure out how to set my conditions in Strategy Wizard first so I can have a proper backtest of my current model, then later attempt to understand/learn C#.

the support at NT is great.

Just really hard to get data, trying to figure out how to import data from SC to NT?

If anyone would be so kind to lend me some ES data that would be brilliant.

Started this thread Reply With Quote




Last Updated on December 1, 2014


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts