Taking a Trading System Live - Trading Journals | futures io social day trading
futures io futures trading


Taking a Trading System Live
Updated: Views / Replies:47,433 / 529
Created: by kevinkdog Attachments:189

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 189  
 
Thread Tools Search this Thread
 

Taking a Trading System Live

  #301 (permalink)
Elite Member
Cincinnati Ohio
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: Futures Broker
Favorite Futures: AUD
 
Silver Dragon's Avatar
 
Posts: 1,597 since Feb 2011
Thanks: 3,860 given, 3,318 received


kevinkdog View Post
I am curious what people out there think. If this was your trading system, would you still be trading it?

Please answer a 1 question survey here: https://www.surveymonkey.com/s/XS6X7DN
(If this is against forum rules, I apologize and ask Big mike to delete the post).


Here are some performance curves to help you make a decision. For the sake of the survey, assume you started trading with real money on the green portion of the curve.

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).


Attachment 126958


Attachment 126959

@kevinkdog

Sorry if some of these thoughts and observations are repeats... I have not read your entire thread but thought it was important to give some insight from a data perspective.

First and foremost; There is no way you should quit or even think about quitting unless you have data to back it up your decision. You posted some guidelines for quitting here: https://futures.io/trading-journals/28404-taking-trading-system-live.html#post350780 Have these changed? Have you hit one of these conditions?

From a data perspective you need more data points before you start asking if it is working or not. A data set needs a minimum of 30 data points to be statically significant. For trading you need more and from what I can see you are not there yet. Think of your entire equity curve as your arm. The live testing is about size of your fingernail. Your not comparing apples to apples . Give it time to develop and dont let your emotions get involved when determining if it is a good system or bad system. Emotions cloud your judgement and cause irrational decisions. If the system is truly bad, the data will bare it out in the in the end.

If you gave this system to me to manage here is the one question I would ask you:

What is max draw down for the system. In other words- What dollar amount are you willing to risk to properly test the system? Think of it as a longer term investment.. You have looked at the prospectus now you have to determine how much you want to risk and for how long you want to invest. I believe you have defined this in the post above as 10%

After that I would just let it run and DONT MESS WITH IT!!! Doing so nullifies your testing and any future comparisons... IMO you need to have at least 6 months to year worth of data for a proper comparison. This will capture different market conditions and news cycles. After this time is up then I would compare it to the past. (This assumes the max draw down has not been hit.)

My other suggestion would be to run the same strategy in SIM alongside live account. Maybe you are already doing this?? This will give you invaluable information. If there is a major difference in the day to day trades between the live and sim then I would stop immediately to find out whats going on. If it is taking similar trades, taking into account slippage and market conditions then you know it is working as designed. Finally I would run another back test after the 6 months is up to compare.

At this point you will have 3 data sets. In theory the two SIM back tests should be nearly, if not identical. The live set should be within acceptable levels of the SIM sets taking into account slippage and unforeseen market conditions which cause bad entries etc. If you do not want to wait the full six months to run the 3rd data set then run it monthly. Again, this will tell you if there is a problem in the code or the data. If it is significantly off then run a second month to see if the results are the similar. Stop the system if it is. Otherwise let it continue to play out.

The thing to remember is proper analysis takes time. Its easy to back test because of the 5 minute instant gratification factor. It is harder to stay the course and let the system play out for a year for better or for worse.

At this point I would not worry about $$ performance. Thats not important from an analysis standpoint. All that is going to do is cloud your judgement. Instead focus on the nuts and bolts of your operation. Make sure everything is running as expected. Do a weekly and monthly comparison of trades between SIM and Live and define the differences in a journal. In fact if you have time do it daily. Define the market conditions and what caused the difference. Once you get done with all that then you will have a detailed document of what went right and what went wrong. Then you can make a determination on if it was successful or not.

Finally, something to put things in perspective; Its possible to do everything right and still lose the game. So, you can have a losing system $$ wise but it still be a successful system from a execution stand point. In this case the detailed notes of the SIM vs Live executions will tell you where the adjustment needs to be made. Fact is systems have been known to fail $$ wise on as little as 1 tick slippage per trade.

BTW- I have had the same thing happen to me when I went live with my automated system; Results varied from the back test. I determined the issue had to do with the fills and the exotic bar type I was using. So your not alone.

Robert

nosce te ipsum

You make your own opportunities in life.
Reply With Quote
The following 6 users say Thank You to Silver Dragon for this post:
 
  #302 (permalink)
Elite Member
detroit, mi
 
Futures Experience: Beginner
Platform: TradeStation
Favorite Futures: SPY
 
Posts: 62 since Nov 2011
Thanks: 1,125 given, 65 received

Accelerated Testing?

kevin,

thanks for the thread.

just looking at your hypothetical curve diagrams and assuming diagrams had significance then you are nowhere near quitting point.

1) would you consider the curve as representation (or sum total) of various statistics?
2) what about the curve during the backtesting and forward testing portion made you decided to use the system?
3) what about the curve during the backtesting and forward testing portion would make you decide to not pursue the system?

since you used period of about 4 years
5) you would have to wait perhaps at least that long to decide whether system is worth it?
6) or if you have drawdown much longer in time duration (1.5x or 2x) and/or in much greater dollar loss (1.5x or 2x)

in my dayjob we do accelerated testing to validate the life of the product. the test has duration, predefined stresses and reliability with confidence attribute in the event sample test parts successfully pass the test.

if the backtest period ( 4 years in this case) is the life of the product, do you have to wait for 4 years to find out whether your product survives the life or is there an accelerated way to stress your product in much shorter period of time ( this would not be a walkforward test and period)? i could further expand on the analogy. perhaps it could be a useful analogy to what you are trying to do or it is just misguided rambling....

regards,
tihfa

Reply With Quote
The following user says Thank You to tihfa for this post:
 
  #303 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,139 given, 4,387 received
Forum Reputation: Legendary



Silver Dragon View Post
@kevinkdog

Sorry if some of these thoughts and observations are repeats... I have not read your entire thread but thought it was important to give some insight from a data perspective.

First and foremost; There is no way you should quit or even think about quitting unless you have data to back it up your decision. You posted some guidelines for quitting here: https://futures.io/trading-journals/28404-taking-trading-system-live.html#post350780 Have these changed? Have you hit one of these conditions?

From a data perspective you need more data points before you start asking if it is working or not. A data set needs a minimum of 30 data points to be statically significant. For trading you need more and from what I can see you are not there yet. Think of your entire equity curve as your arm. The live testing is about size of your fingernail. Your not comparing apples to apples . Give it time to develop and dont let your emotions get involved when determining if it is a good system or bad system. Emotions cloud your judgement and cause irrational decisions. If the system is truly bad, the data will bare it out in the in the end.

If you gave this system to me to manage here is the one question I would ask you:

What is max draw down for the system. In other words- What dollar amount are you willing to risk to properly test the system? Think of it as a longer term investment.. You have looked at the prospectus now you have to determine how much you want to risk and for how long you want to invest. I believe you have defined this in the post above as 10%

After that I would just let it run and DONT MESS WITH IT!!! Doing so nullifies your testing and any future comparisons... IMO you need to have at least 6 months to year worth of data for a proper comparison. This will capture different market conditions and news cycles. After this time is up then I would compare it to the past. (This assumes the max draw down has not been hit.)

My other suggestion would be to run the same strategy in SIM alongside live account. Maybe you are already doing this?? This will give you invaluable information. If there is a major difference in the day to day trades between the live and sim then I would stop immediately to find out whats going on. If it is taking similar trades, taking into account slippage and market conditions then you know it is working as designed. Finally I would run another back test after the 6 months is up to compare.

At this point you will have 3 data sets. In theory the two SIM back tests should be nearly, if not identical. The live set should be within acceptable levels of the SIM sets taking into account slippage and unforeseen market conditions which cause bad entries etc. If you do not want to wait the full six months to run the 3rd data set then run it monthly. Again, this will tell you if there is a problem in the code or the data. If it is significantly off then run a second month to see if the results are the similar. Stop the system if it is. Otherwise let it continue to play out.

The thing to remember is proper analysis takes time. Its easy to back test because of the 5 minute instant gratification factor. It is harder to stay the course and let the system play out for a year for better or for worse.

At this point I would not worry about $$ performance. Thats not important from an analysis standpoint. All that is going to do is cloud your judgement. Instead focus on the nuts and bolts of your operation. Make sure everything is running as expected. Do a weekly and monthly comparison of trades between SIM and Live and define the differences in a journal. In fact if you have time do it daily. Define the market conditions and what caused the difference. Once you get done with all that then you will have a detailed document of what went right and what went wrong. Then you can make a determination on if it was successful or not.

Finally, something to put things in perspective; Its possible to do everything right and still lose the game. So, you can have a losing system $$ wise but it still be a successful system from a execution stand point. In this case the detailed notes of the SIM vs Live executions will tell you where the adjustment needs to be made. Fact is systems have been known to fail $$ wise on as little as 1 tick slippage per trade.

BTW- I have had the same thing happen to me when I went live with my automated system; Results varied from the back test. I determined the issue had to do with the fills and the exotic bar type I was using. So your not alone.

Robert


Thanks for the comments. I agree with pretty much everything you said.


One important part you mention is comparing SIM (or strategy results) vs. live. This is critical, especially for exotic bar types or strategies where slippage is a factor (and initial estimates of it were low).


I appreciate your excellent reply, even if you are a Bengals fan. But, they are doing much better than my Browns this year!

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following 2 users say Thank You to kevinkdog for this post:
 
  #304 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,139 given, 4,387 received
Forum Reputation: Legendary


tihfa View Post
kevin,



1) would you consider the curve as representation (or sum total) of various statistics?

Not sure I understand this question. Can you explain?


tihfa View Post
2) what about the curve during the backtesting and forward testing portion made you decided to use the system?

3) what about the curve during the backtesting and forward testing portion would make you decide to not pursue the system?

I set out my goals and objectives before developing the system. Since it met those goals, I then considered it for live trading. For me, the primary number I look at is Return/DD ratio.

There are a lot of reasons why I might not trade the system, even if the number look good. Let's say, for example, that the numbers are great, but the equity has been down the past 2 years. That might be a deal killer.



tihfa View Post
since you used period of about 4 years
5) you would have to wait perhaps at least that long to decide whether system is worth it?
6) or if you have drawdown much longer in time duration (1.5x or 2x) and/or in much greater dollar loss (1.5x or 2x)

I would use the 1.5-2.0 x drawdown before I'd wait 4 years. Your point is good though: the longer the real time out of sample period, the more confidence you'd have in the system.


tihfa View Post
in my dayjob we do accelerated testing to validate the life of the product. the test has duration, predefined stresses and reliability with confidence attribute in the event sample test parts successfully pass the test.

if the backtest period ( 4 years in this case) is the life of the product, do you have to wait for 4 years to find out whether your product survives the life or is there an accelerated way to stress your product in much shorter period of time ( this would not be a walkforward test and period)? i could further expand on the analogy. perhaps it could be a useful analogy to what you are trying to do or it is just misguided rambling....

regards,
tihfa

I understand what you are getting at, but I'm not sure how you could do this. A couple of thoughts come to mind:

1. Run the system on simulated data - data that isn't real, but has same general characteristics as real data. I think Tushar Chande's book describes this method.

2. You could arbitrarily reduce all trades by a certain percentage. Knock 25% off wins, and add 25% to losses. Does the system still work? This might simulates future performance.


Thanks for the comments. You've put forth some good ideas to think about.

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
  #305 (permalink)
Elite Member
detroit, mi
 
Futures Experience: Beginner
Platform: TradeStation
Favorite Futures: SPY
 
Posts: 62 since Nov 2011
Thanks: 1,125 given, 65 received

Equity Curve Use vs Stats Use


kevinkdog View Post
Not sure I understand this question. Can you explain?

kevin,

you have yourself pointed to the curves as basis for opinion whether to continue trading or not. yet you have statistics calculations which are calculated independently of the shape or form of the curve?

could you visually judge/compare Return/DD ratio of live trading period to that of backtested/walkforward period. If the answer is no, then you should not have offered the curves as the basis of opinion, but rather stats only. Anyhow a minor point...

regards,
tihfa

Reply With Quote
The following user says Thank You to tihfa for this post:
 
  #306 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,139 given, 4,387 received
Forum Reputation: Legendary


tihfa View Post
kevin,

you have yourself pointed to the curves as basis for opinion whether to continue trading or not. yet you have statistics calculations which are calculated independently of the shape or form of the curve?

could you visually judge/compare Return/DD ratio of live trading period to that of backtested/walkforward period. If the answer is no, then you should not have offered the curves as the basis of opinion, but rather stats only. Anyhow a minor point...

regards,
tihfa

Thanks for clarifying. I would not recommend looking at Return/DD for such a short period of time as I have so far in live testing. Maybe in a few years that will be appropriate. Hopefully I will still be trading it at that time!

Thanks for your comment!

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
  #307 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,240 since Jun 2009
Thanks: 29,352 given, 83,231 received


kevinkdog View Post
1. Run the system on simulated data - data that isn't real, but has same general characteristics as real data. I think Tushar Chande's book describes this method.

BTW, if you are interested in this, there is a data generation thread here that creates data based on samples you provide:

https://futures.io/elite-circle/19156-random-data-generator.html

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
The following 2 users say Thank You to Big Mike for this post:
 
  #308 (permalink)
Fiddler
Nashville, TN
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: IB
Favorite Futures: NQ ES
 
Posts: 468 since Feb 2011
Thanks: 323 given, 542 received

I took the equity curve chart (most of it) and windowed it (cutting it in half and then cutting the pieces in half) until I reached a point where there were sections that significantly underperformed the "average performance" of the total equity curve. Place smallest oval possible around each section that contains the start, finish, and all equity curve action in between. Make copies of oval or ovals and place in such a manner that would be inherently pessimistic to the "live trade section". Exits outside the bottom side of the risk ovals would indicate problems.

Going forward, move the ovals to new start point as new data points come in at intervals the size of the windowed region the ovals originated from. Looks like a few weeks ago there should a new start point for the small oval if you keep my windowing points.

The large oval appears to be about $7,500 profit top to bottom. Since the oval won't allow for placement that takes the full range, probably 2/3rds of that would be enough to get the equity curve outside the range...your $5k drawdown power-off button seems right.

Can't say I know of a proof or theory that legitimizes this as a means to test an equity curve going forward. I'll let someone else do the math. Fun with Ovals....yay!!!

If nothing else, it shows that you have plenty of time to consider performance.

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).

Reply With Quote
The following 5 users say Thank You to Luger for this post:
 
  #309 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,139 given, 4,387 received
Forum Reputation: Legendary

Very interesting analysis. Thanks for taking the time to do it, and to share it. Looks like I need to avoid the ovals!

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
 
  #310 (permalink)
Fiddler
Nashville, TN
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: IB
Favorite Futures: NQ ES
 
Posts: 468 since Feb 2011
Thanks: 323 given, 542 received


I should have given a bit more background about my analysis but it was late. Your survey about continuing or quitting the system got me thinking.

The statistics of how much data you need to say that your live sample is different from the in-sample period seemed to be the most important factor. That is what got me slicing up the data into windows to see how much time constituted a period of underperformance in the in-sample data.

Also I've known traders that have used ovals on price/time squared charts. The basic shape is interesting when you think about how positive and negative volatility works over time.

Sure the analysis is subjective, but it adds a visual component that is representative of underperforming periods in the in-sample data. In this case it shows that you need a good bit more time before trashing the system. It also frames the volatility of in-sample underperforming periods which should helpful in determining drawdown potential going forward.

You may laugh, but I am going to be adding this as a subjective component to my system tracking. I'll be doing it in excel instead of paint though...lol

Reply With Quote

Reply



futures io > > > Taking a Trading System Live

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Comparing accuracy of sim trading on different platforms vs live trading Titian Traders Hideout 4 February 11th, 2017 02:50 PM
Taking the leap towards self-employment (Trading CL) Indians00c Elite Trading Journals 462 February 6th, 2015 11:08 AM
Questions concerning risk taking and speculative trading for a Bsc dissertation Shichao Psychology and Money Management 24 March 14th, 2012 07:33 PM
Webinar Trading Video: Live Trading and Technical Analysis of EuroStoxx 50 max-td Trading Reviews and Vendors 1 March 13th, 2010 07:35 AM


All times are GMT -4. The time now is 04:18 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-14 in 0.20 seconds with 20 queries on phoenix via your IP 54.91.38.173