Taking a Trading System Live - Trading Journals | futures io social day trading
futures io futures trading


Taking a Trading System Live
Updated: Views / Replies:47,424 / 529
Created: by kevinkdog Attachments:189

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 189  
 
Thread Tools Search this Thread
 

Taking a Trading System Live

  #151 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,138 given, 4,387 received
Forum Reputation: Legendary


Big Mike View Post
Kevin,

One advantage I think NinjaTrader has over TradeStation is the custom fitness freedom.

Check this thread:

https://futures.io/elite-automated-ninjatrader-trading/25792-ninjatrader-optimizer-types-custom-fitness.html

Tons and tons and tons of different fitness and can all be tweaked to exactly what is important to you, and you can create your own formulas if the metric you wan't doesn't already exist.

Mike


I need sleep too (I drove home in middle of night last night after watching my Michigan Wolverines beat Notre Dame - biggest crowd in NCAA football history), but I'll try to keep up with you.


Tradestation I know is trying to work on better fitness functions. Most of the standard ones are weak. I usually use Net Profit, for simplicity, and typically, the good net profit cases have low drawdowns.

One other big issue is that using drawdown as part of a fitness function with walkforward can give very misleading results, if the walkforward window moves with time. I will try to explain when I answer an earlier post on walkforward.

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
 
  #152 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,138 given, 4,387 received
Forum Reputation: Legendary


Big Mike View Post
Kevin, out of curiosity have you watched @NJAMC's webinar "An Intro to Machine Learning"?

https://futures.io/elite-automated-trading/26007-webinar-intro-machine-learning-njamc.html

NJAMC has been working on his project for a while now, he and @Luger were also involved in the Artificial Bee Colony thread, the predecessor to Greg's ML thread:

https://futures.io/elite-automated-ninjatrader-trading/22389-artificial-bee-colony-abc-algorithm.html

I tried to help where I could using my Wave59 license, but in the end those guys were just too far ahead of me. That's what I get for being a high school drop out, it sucks to be a trader that is poor at math (if only I had known...)

I bring it up because I was thinking of maybe getting together a consortium of sorts, like maybe John Ehlers, Suri Duddella, Manesh Patel, and @NJAMC, @Luger, @Fat Tails and some other fine folks who I can't think of right now, and see if maybe some sort of collaborative project could be started to further the work that Greg (NJAMC) is doing on the ML front. I figured maybe a few 1-hour conference calls or skype sessions might be enough to get some really great brainstorming done to make some real headway in this area.

Perhaps we should take this off-line... I've cluttered your thread enough.

Mike


That's OK, this is good stuff. I have the webinar on my to do list.

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
 
  #153 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,221 received



kevinkdog View Post
Without originally intending to, I think I did just tear your system apart - not by analyzing your results, but by asking you how you got those results. Based on your answers (no walkforward, no out of sample, optimized over all data set), I would have no need to look further at your results. They might be spectacular results, but optimized results almost always are. Doesn;t mean jack going forward.

Where it would get tricky is if you lied and said it wasn't optimized, or that it had x% out of sample results at end. How would I be able to tell you were lying? One way I know is the "if it looks too good to be true, it probably is" test. Another way would be watching it in real time (incubation). Maybe someone else has a better approach they would like to share.


I think this is why it is so hard for people who buy strategies or systems. How do they know if the system results are "real," or if they are just some optimized BS? Most times, a vendor who sells optimized systems would probably lie to you about how he got the results anyway. I know, because sometimes for fun I converse with vendors of these super duper systems, and 9 times out of 10 it is clear to me that
a) the results are garbage
b) the vendor doesn't know what he is talking about
c) the vendor's personality is some kind of combination of unethical, immoral, irresponsible and feeble minded.


I propose you keep tracking the system, and let's see how it does a few months from now. That should be very revealing...

You make good points and I agree completely.

I see the same stuff vendors are selling, and the worse thing is -- people are buying it. The other real problem with NinjaTrader's summary page is there is no position sizing anywhere. So you can make a system look great by just adding a few zero's to the position size, and it shows more trades, more profit. People then think it is not curve fitted because it has so many more trades.

The other big one is when people use limit orders. That's an easy way to make NT reports inaccurate.

I generally prefer systems that trade 1,000 times a year or more on 1 contract. If I start dipping below 500 trades a year, I get worried about over fitting. Obviously you take a very different approach, and this is just one of many ways we make a market together

I prefer systems that have very few input parameters, like 1 or 2 ideally, where everything else is dynamic and based on volatility and such in the market. That way I can optimize for my best score, and that golden looking equity curve we are all after, by combining that system along with many more into a portfolio and looking at how it benefits or balances the portfolio as a whole.

As for keeping track of the system, I wish I had fixed the code I was using to automatically post charts and trade results into a futures.io (formerly BMT) thread via the futures.io (formerly BMT) API. I just never had time to fix it, and certainly don't want to start now (I am preparing for vacation mode...). So I can't do any kind of live test. Closest I could do would be to shelf it and come back in two months and just run a new report to see how that out of sample data looks, but it would still be historical sim not live.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #154 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,221 received


kevinkdog View Post
Tradestation I know is trying to work on better fitness functions.

What ever came of their acquisition of Grail Computer? I thought they would be killing it by now with all the work Grail was doing. Have they integrated it into a production product yet?

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #155 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,138 given, 4,387 received
Forum Reputation: Legendary


Big Mike View Post
What ever came of their acquisition of Grail Computer? I thought they would be killing it by now with all the work Grail was doing. Have they integrated it into a production product yet?

Mike

They incorporated at least some of Grail's capabilities into version 9.0. I think they are continuously more features.

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
 
  #156 (permalink)
Elite Member
Atkinson, NH USA
 
Futures Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker/Data: NinjaTrader Brokerage
Favorite Futures: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,925 since Dec 2010
Thanks: 2,962 given, 2,294 received


Big Mike View Post
You make good points and I agree completely.

I see the same stuff vendors are selling, and the worse thing is -- people are buying it. The other real problem with NinjaTrader's summary page is there is no position sizing anywhere. So you can make a system look great by just adding a few zero's to the position size, and it shows more trades, more profit. People then think it is not curve fitted because it has so many more trades.

The other big one is when people use limit orders. That's an easy way to make NT reports inaccurate.

I generally prefer systems that trade 1,000 times a year or more on 1 contract. If I start dipping below 500 trades a year, I get worried about over fitting. Obviously you take a very different approach, and this is just one of many ways we make a market together

Mike

@Big Mike,

Unfortunately, it is easy to over-fit even a large data-set. It is a matter of having enough degrees of freedom in your "solution" or what I am less familiar with, but have seen none the less is a simple MA crossing that gets lucky. (Likely large drawdown ,MAE, or ETDs)

From the "black magic" side, all you really need to do is give an algorithm enough time and degrees of freedom (input values, products/sums of terms, square's, square roots, etc.) and you can match to the max of degrees (ultimately more good trades) of freedom. Sometime it is difficult to see all the degrees of freedom, but there is likely more than you can see.

This is why Forward or Out of Sample testing is critical. The more "over-fit" a function is, the fast it will fall apart as you test values that were not used to train the system. What you are looking for is a "fit" function. One that does okay, but generalizes the solution into the future. This is the goal of my Genetic Programming investigation.

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Reply With Quote
The following 3 users say Thank You to NJAMC for this post:
 
  #157 (permalink)
Elite Member
Atkinson, NH USA
 
Futures Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker/Data: NinjaTrader Brokerage
Favorite Futures: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,925 since Dec 2010
Thanks: 2,962 given, 2,294 received


Big Mike View Post
I looked at his Adap Trade stuff. What turned me off is it's for TradeStation (EasyLanguage) only, and given my falling out with MultiCharts I don't want to go back down that path.

I also could not tell what type of machine learning he was really using. Maybe @NJAMC could view his youtube videos and make a guess. My best guess was it is not the type of ML that @NJAMC is working on, or that Wave59 has, but instead just a Genetic Optimization type formula.

I was also a bit turned off by the "brute force" method of the app, basically trying all kinds of crazy indicators to find a good fit. This clearly is a concern for overfitting....

Anyway, his MSA software is another matter, and I think is quite good for analysis.

Mike

@Big Mike,

I do believe the Adaptrade Software is using Genetic Programming based upon configuration and function. I certainly cannot attest to the ability it has to solve the problem at hand.

As discussed above, it is very easy to "over-fit" a solution using any Machine Learning approach or (manually this is trickier but possible). I can tell you that I have many systems that will over-fit a solution to the training set. This is quite easy. The difficult part is to figure out how to get a ML system to create a Generic solution without access to the Out of Sample data.

This is a bit of a paradox for me right now. I want to create a solution that fits my training set (in sample data), but the second requirement is that it needs to also fit data I have not seen yet (Out of sample data). I can test the OOS data, but then it is touched and now should be considered "In Sample" data. It is unclear in systems like the Adaptrade system when it shows the OOS data on its charts, has it reviewed this data. It should be okay to calculate the answer (Profit or EC), but that data cannot be brought back into the Engine to affect the genetics of the system without it now being considered In Sample data again.

My current approach is to simply store anything that looks profitable. I then use a 2nd analysis which is manual for me to accept the strategy as generic. What I do is run all the "looks profitable" through NT (I use the optimizer, but only to load the different strategies for analysis) and check for nice EC curves and profitability OOS.

In the future, I may add more logic to truly analyse the "looks profitable" before storing. What may not be overly obvious is the longer you run a ML algorithm on a training set, the more likely it will specialize (over-fit) the training dataset. So the "looks profitable" strategies need to be pealed off as soon as they start to look good before over-fitting starts.

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Reply With Quote
The following 2 users say Thank You to NJAMC for this post:
 
  #158 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


NJAMC View Post
@Big Mike,

I do believe the Adaptrade Software is using Genetic Programming based upon configuration and function. I certainly cannot attest to the ability it has to solve the problem at hand.

As discussed above, it is very easy to "over-fit" a solution using any Machine Learning approach or (manually this is trickier but possible). I can tell you that I have many systems that will over-fit a solution to the training set. This is quite easy. The difficult part is to figure out how to get a ML system to create a Generic solution without access to the Out of Sample data.

This is a bit of a paradox for me right now. I want to create a solution that fits my training set (in sample data), but the second requirement is that it needs to also fit data I have not seen yet (Out of sample data). I can test the OOS data, but then it is touched and now should be considered "In Sample" data. It is unclear in systems like the Adaptrade system when it shows the OOS data on its charts, has it reviewed this data. It should be okay to calculate the answer (Profit or EC), but that data cannot be brought back into the Engine to affect the genetics of the system without it now being considered In Sample data again.

My current approach is to simply store anything that looks profitable. I then use a 2nd analysis which is manual for me to accept the strategy as generic. What I do is run all the "looks profitable" through NT (I use the optimizer, but only to load the different strategies for analysis) and check for nice EC curves and profitability OOS.

In the future, I may add more logic to truly analyse the "looks profitable" before storing. What may not be overly obvious is the longer you run a ML algorithm on a training set, the more likely it will specialize (over-fit) the training dataset. So the "looks profitable" strategies need to be pealed off as soon as they start to look good before over-fitting starts.

I have played with the Adaptrade software for a while. It was interesting, but not my cup of tea. In regards to the contamination of data set of OOS since it plots that in the chart. There is a thread in the googlegroup for them. As far as I understand it, it remains untouched as the results are not recycled. Basically all results that are used for the GP/GA optimization and next generation are based on the in sample. Then that goes to seed the next generation. The OOS results are shown but are not used to generate the next generation. But as a user you can see whats going on. You can go check the google group to be sure or ask questions that are answered by the programmer/founder of adaptrade himself.

Reply With Quote
The following 2 users say Thank You to treydog999 for this post:
 
  #159 (permalink)
Elite Member
Austin, TX
 
Futures Experience: Master
Platform: ninjatrader, r-trader
Favorite Futures: CL, NG, TF, NQ, YM, GC, ES
 
liquidcci's Avatar
 
Posts: 862 since Jun 2011
Thanks: 609 given, 1,051 received


kevinkdog View Post
I have never used the product above, but Mike Bryant has some other good tools, for free (through his newsletters) and for pay. He has some very good ideas on trading system development.

@kevinkdog I find Adaptrades MSA is an excellent piece of software. If you have a solid system on 1 contract you can plug your test results in and really see how position size using different methods affects your outcome. Very powerful tool. Also has a nice portfolio feature and some other features that are good for analyzing a system. There a few things I wish it would do better but overall it is some of the best money I have ever spent. It is really inexpensive for what it does.

"The day I became a winning trader was the day it became boring. Daily losses no longer bother me and daily wins no longer excited me. Took years of pain and busting a few accounts before finally got my mind right. I survived the darkness within and now just chillax and let my black box do the work."

Last edited by liquidcci; September 9th, 2013 at 11:36 AM.
Reply With Quote
The following user says Thank You to liquidcci for this post:
 
  #160 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,138 given, 4,387 received
Forum Reputation: Legendary

Monitoring a Strategy


In an earlier post, I shared with you one way I track a trading strategy. Here is another tracking tool I use...

When I used to work in aerospace (or the "real world" as I sometimes refer to it), our small company ($250 million annual sales) would have a weekly meeting called "How We Doin." Incorrect grammar aside, it was an excellent way for the managers of the company to quickly see how sales were for the month and quarter, what quality and production problems were occurring, and just a general sense of where the company currently stood.

Now, fast forward a few years. I am trading full time, working alone. But, I still want to see at a glance "how I'm doin" with my strategies and trading. Obviously, my account statements and equity curve tell the overall story, but that is not enough detail for me. What strategies are doing good? Which are underperforming? Of strategies I am incubating, how do they look? Should I make some changes in what I am trading? This "How I'm Doin" report can help me answer all of these questions.

I developed a little spreadsheet (sorry, I am not sharing it, but it is easy enough to do yourself) to help me with this task. It tells me at a glance how my strategies are doing, and I can easily drill down and see detail if I need to.

First, there is a summary page. I include every strategy I am trading live. I also include in another section the strategies I currently am incubating. This summary sheet collects all the data I am interested in (of course, if you did this yourself, you'd likely pick different metrics than I did). This summary sheet gets the data from the individual sheets, which I will describe a bit later.

To keep things simple, I base everything on one contract being traded, even though that is usually not what I am actually trading. Why? My goal with this spreadsheet is to see how my strategies are doing compared to how I thought (calculated) they'd be doing. If I included position sizing, it would muddy up the view for me.

Of all the numbers on this sheet, I am primarily interested in two columns:

1. Return Efficiency - How am I doing, compared to my expectations? That is how I define return efficiency, and it is simply my actual return divided by my expected return. If my strategy is performing exactly as I had calculated, it will be 100%. Obviously, I want this to be close to or above 100%. Typically, when I take all the strategies together, I find my efficiency is somewhere between 70-100%. So, this says that if my historical testing says I should make $10 a year, I am actually making somewhere between $7-10.

2. Drawdown Efficiency - This is how I am doing with regards to drawdown. Just like with return efficiency, I calculate this as my actual drawdown divided by my expected drawdown. I then subtract the result from 1, to make the number 100% the ideal value. It is a bit backwards to do this, but I do it that way so that both efficiency numbers have 100% as their ideal value. Then, the closer to efficiencies get to zero, the worse off things are.


Once a month, I go through and update each of the individual system sheets with performance data, and that automatically updates the main sheet.

In the next post I'll show an individual sheet.

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following 8 users say Thank You to kevinkdog for this post:

Reply



futures io > > > Taking a Trading System Live

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Comparing accuracy of sim trading on different platforms vs live trading Titian Traders Hideout 4 February 11th, 2017 02:50 PM
Taking the leap towards self-employment (Trading CL) Indians00c Elite Trading Journals 462 February 6th, 2015 11:08 AM
Questions concerning risk taking and speculative trading for a Bsc dissertation Shichao Psychology and Money Management 24 March 14th, 2012 07:33 PM
Webinar Trading Video: Live Trading and Technical Analysis of EuroStoxx 50 max-td Trading Reviews and Vendors 1 March 13th, 2010 07:35 AM


All times are GMT -4. The time now is 09:15 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-13 in 0.21 seconds with 20 queries on phoenix via your IP 54.82.56.95