What should be the next step in my backtesting?? - Emini Index Futures Trading | futures io social day trading
futures io futures trading


What should be the next step in my backtesting??
Updated: Views / Replies:6,536 / 76
Created: by budfox Attachments:4

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 4  
 
Thread Tools Search this Thread
 

What should be the next step in my backtesting??

  #11 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received


Ming80 View Post
Hi BudFox,

Glad you've built your first model. Just like you, when I started my first model from excel I had tons of questions and wanted to share any some info that might be of use to you.

1. On Backtesting Software - Treydog has given excellent points. Apart from the speed and scale which you can backtest many markets/asset classes, there is also the huge benefit of automated execution. I found my usage of time much more productive in research instead of monitoring trades. One can now trade up to 20 or more positions simulatneously (long term or short term). The caveat of course is that the model developed is accurate and robust and monitoring systems from time to time is still a prerequisite.

2. On Monte Carlo - It is an interesting measure but wouldn't put too much faith in it. Actual drawdowns are a function of a specific market events/environment in relation a series of bad trades produced. Trade scrambling removes the serial correlation which happens during this period and how the drawdown was actually produced. It would be useful to look into the periods which drawdowns occured and understand what caused them. This is crucial to deduce if risk management approaches are needed (stopping the trading model, develop non-correlated models etc.) ... All these decisions have 'costs' to the long term outcome of the eventual pnl. Alternatively, one could accept that this is a function of markets where drawdowns are inevitable from time to time and just continue to stick to the plan.

3. Number of trades - Perhaps instead of measures of statistical significance, one could take a qualitative approach and see what the system was designed to capture. Warren buffett as well as many long term trend followers although might fail statistical tests by their number of trades, but their performance is certainly more than random. One needs to take into account the holding frequency and frequency of conditions present.

4. Clean Data - Good models need to be built on clean and more importantly adjusted data. Even accredited data providers do not provide properly adjusted data for backtests and will lead to incorrect trading conclusions. Thus, the need for backtesting software to adjust data to generate a proper backtest.

Lastly, I personally found the most crucial parts in building good trading models was in thinking and capturing the stationarity of the market. I got this general idea from visiting the local library and reading books by clifford j sherry. The 2nd part was asking myself how long this 'stationarity' would tend to persist. The better which these 2 questions could be answered, the better the models have performed going forward. I hope some of my personal experiences would be of use to you going forward.

Thank You for your advice and understanding I am new backtester,

could you explain to me what you mean by stationarity of the market?

I still want to read your post more carefully so I can come back with more intelligent answers.

Reply With Quote
 
  #12 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received

I have been able to replicate most of the conditions from my manual excel model into Strategy Analyzer, but since my rules are quite particular i think it will be extremely challenging, if not impossible, to replicate all.

I think I will inevitably have to learn to code in Ninjascript (how hard is it?) .

Using Strategy Analyzer is easier from the perspective that I can more quickly backtest more trades, but manually is easier from the perspective that I do not have to figure out how to "code" my conditions into the software.

My conditions are a lot more complicated that "Go long" when ma crosses over pivot etc.

Its more like go long if this bar closed higher than this bars open ONLY during this timeframe, BUT skip the trade if you have this divergence.

It's the skip the trade when you have this divergence condition that I am trying to "code".


I really appreciate the help and support from some of the members of this great forum.

Reply With Quote
 
  #13 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received



budfox View Post
I have been able to replicate most of the conditions from my manual excel model into Strategy Analyzer, but since my rules are quite particular i think it will be extremely challenging, if not impossible, to replicate all.

I think I will inevitably have to learn to code in Ninjascript (how hard is it?) .

Using Strategy Analyzer is easier from the perspective that I can more quickly backtest more trades, but manually is easier from the perspective that I do not have to figure out how to "code" my conditions into the software.

My conditions are a lot more complicated that "Go long" when ma crosses over pivot etc.

Its more like go long if this bar closed higher than this bars open ONLY during this timeframe, BUT skip the trade if you have this divergence.

It's the skip the trade when you have this divergence condition that I am trying to "code".


I really appreciate the help and support from some of the members of this great forum.



If your going to be serious about backtesting strategies, your going to have to learn to code. Its part and parcel of the whole backtesting strategy development method. With out it your just going to probably not get the kind of system or results you were looking for originally. Like you mentioned the point and click rule set is not specific enough, and its your first system. you already reach the limit of what that type of analyzer can do. Its better to invest in yourself and learn to code then it is to struggle with sub par tools that you can easily grow out of anyway.

Reply With Quote
The following 3 users say Thank You to treydog999 for this post:
 
  #14 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received

What should be the next step in learning how to code


treydog999 View Post
If your going to be serious about backtesting strategies, your going to have to learn to code. Its part and parcel of the whole backtesting strategy development method. With out it your just going to probably not get the kind of system or results you were looking for originally. Like you mentioned the point and click rule set is not specific enough, and its your first system. you already reach the limit of what that type of analyzer can do. Its better to invest in yourself and learn to code then it is to struggle with sub par tools that you can easily grow out of anyway.

Ok I have absolutely zero experience in how to code. First, what language should I learn to code in? Java? Python? C+?

I am currently using NT Strategy Analyzer (since its free)

Reply With Quote
 
  #15 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


budfox View Post
Ok I have absolutely zero experience in how to code. First, what language should I learn to code in? Java? Python? C+?

I am currently using NT Strategy Analyzer (since its free)

easylanguage is the "easiest" although very specific to the tradestation or multicharts platform. Learning a more "legitimate" programming language Java, Python, R, or anything in the C family will give you more power and not be bound by a specific platform. It depends on your goals, how much you are willing to put in to study and learn a specific one. I would say that easylanguage is the best way to dip your toe in, and be able to test a very many different systems and get them to run and execute automatically. If I were to pick another language it would be from the C family, either C++ or C# this is because a lot of programs use it (including ninja trader).

R and Python I put into another category because they are very high level scripting languages. Powerful, useful and easy to develop for but are not very well integrated into other platforms.

Reply With Quote
 
  #16 (permalink)
Trading for Fun
Buffalo, NY
 
Futures Experience: None
Platform: Python, Excel, R, Ninjatrader
Broker/Data: IB quandl
Favorite Futures: Low frequency trading
 
Posts: 64 since Jul 2014
Thanks: 4 given, 37 received


Ming80 View Post
H
3. Number of trades - Perhaps instead of measures of statistical significance, one could take a qualitative approach and see what the system was designed to capture. Warren buffett as well as many long term trend followers although might fail statistical tests by their number of trades, but their performance is certainly more than random. One needs to take into account the holding frequency and frequency of conditions present.


No doubt. I think there is some confusion here about repeated trials process vs a Bayesian statistical measure.
A 20% chance of rain tomorrow doesn't mean that tomorrow is going to happen a 100 times and 20 times it rains and 80 times it doesn't. Since you are only going to get one sample of tomorrow the 20% prediction is a reflection of the belief in our information at hand.
Of course running a simulation of the weather tomorrow 100 times can turn this into a repeated trials process but the prediction is only going to be as good as the model. You wouldn't put any faith if the weather model is based on flipping an unfair coin and summing the results but that is basically what that montecarlo spread sheet is doing.

You can also consider only using backtesting to find a model of interest but then only test in real time if the time frame is short enough that you can still put on real trades without things having completely changed. Things like max draw down in a backtest are beyond useless IMO. The only certainty we have is that the future will not look like the past.

For learning to code, learning C# in ninja would be the way to go because you can learn by doing and creating the very thing you are interested in this thread without having to bother with learning all the plumbing and boring stuff.

Reply With Quote
 
  #17 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received


treydog999 View Post
easylanguage is the "easiest" although very specific to the tradestation or multicharts platform. Learning a more "legitimate" programming language Java, Python, R, or anything in the C family will give you more power and not be bound by a specific platform. It depends on your goals, how much you are willing to put in to study and learn a specific one. I would say that easylanguage is the best way to dip your toe in, and be able to test a very many different systems and get them to run and execute automatically. If I were to pick another language it would be from the C family, either C++ or C# this is because a lot of programs use it (including ninja trader).

R and Python I put into another category because they are very high level scripting languages. Powerful, useful and easy to develop for but are not very well integrated into other platforms.


If I had to use easylanguage, I would only be able to use tradestation or multicharts?

Maybe I should learn C# since that is what ninja uses, and I am currently using their strategy analyzer?

If you were my boss and you told me to learn C#, how long would you expect me to have a working knowledge of it? what deadline from 8 September would you want me to be able to use it by?

Reply With Quote
 
  #18 (permalink)
Elite Member
Austin, TX
 
Futures Experience: Intermediate
Platform: F-16CM-50
Favorite Futures: JDAM
 
tturner86's Avatar
 
Posts: 5,784 since Sep 2013
Thanks: 9,849 given, 11,159 received
Forum Reputation: Legendary


budfox View Post
If I had to use easylanguage, I would only be able to use tradestation or multicharts?

Maybe I should learn C# since that is what ninja uses, and I am currently using their strategy analyzer?

If you were my boss and you told me to learn C#, how long would you expect me to have a working knowledge of it? what deadline from 8 September would you want me to be able to use it by?

OEC uses easy language. Futures Online, Futures Trading Platform, Futures Trading Demo, Electronic Trading System

Reply With Quote
The following user says Thank You to tturner86 for this post:
 
  #19 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


budfox View Post
If I had to use easylanguage, I would only be able to use tradestation or multicharts?

Maybe I should learn C# since that is what ninja uses, and I am currently using their strategy analyzer?

If you were my boss and you told me to learn C#, how long would you expect me to have a working knowledge of it? what deadline from 8 September would you want me to be able to use it by?

Yes, C# is a great language and is very versatile. Honestly as someone who knows C++ ,the C family its really very common and powerful.

I like that question, although I want to put another spin on it. As in your boss wants you to be able to fluently write code in ninja trader using C# to develop trading strategies. I would say learning on an everyday basis in 90 days or so you will have learned enough to start playing with simple strategies in ninja trader and start to be able to get results. After that your going to be probably pretty fluid in using their functions, shortcuts, and classes/methods that it will get faster and you will spend less time debugging and looking up arguments or syntax. After that its like a real language, the more you use it the more fluent you get.

Honestly trading systems developers want to make it as easy as possible to get started so they add a lot of very useful functions that are ready to use out side of the box. I have never worked with ninjatrader but once you learn the basic code structure of a C program and how it works you should be able to piece together whats going on. Use examples and pre build strategies, dissect them line by line and see how they work.

Reply With Quote
 
  #20 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received


I'm actually trying to figure out how to set my conditions in Strategy Wizard first so I can have a proper backtest of my current model, then later attempt to understand/learn C#.

the support at NT is great.

Just really hard to get data, trying to figure out how to import data from SC to NT?

If anyone would be so kind to lend me some ES data that would be brilliant.

Reply With Quote

Reply



futures io > > > > What should be the next step in my backtesting??

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Step MA crazybears Sierra Chart 2 July 15th, 2014 06:25 AM
Top Step Trader website down? birdoggg Platforms and Indicators 2 November 7th, 2012 10:07 AM
The Next Step Towards The End Of The Euro Quick Summary News and Current Events 0 November 16th, 2011 12:50 AM


All times are GMT -4. The time now is 01:15 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-13 in 0.17 seconds with 20 queries on phoenix via your IP 54.82.79.109