iSystems Journal - futures io
futures io futures trading



iSystems Journal


Discussion in Trading Journals

Updated by SMCJB
      Top Posters
    1. looks_one SMCJB with 88 posts (260 thanks)
    2. looks_two Mabi with 15 posts (15 thanks)
    3. looks_3 mattz with 7 posts (6 thanks)
    4. looks_4 tpredictor with 7 posts (11 thanks)
      Best Posters
    1. looks_one Big Mike with 13.0 thanks per post
    2. looks_two kevinkdog with 4.8 thanks per post
    3. looks_3 SMCJB with 3.0 thanks per post
    4. looks_4 Mabi with 1.0 thanks per post
    1. trending_up 16,439 views
    2. thumb_up 367 thanks given
    3. group 43 followers
    1. forum 164 replies
    2. attach_file 91 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 100,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

iSystems Journal

(login for full post details)
  #141 (permalink)
Golden Bay, New Zealand
 
Experience: Beginner
Platform: Sierra Chart
Trading: ES, NQ
 
Posts: 186 since May 2012
Thanks: 851 given, 335 received

Interesting.
Was that 1 system each for the highest/lowest/nearest or a certain % of the systems - ie. top 5%, bottom 5% etc?

Reply With Quote

Can you help answer these questions
from other members on futures io?
IB Bracker Order
Platforms and Indicators
DLL with webrequest function for MC
MultiCharts
Webinar: Linda Bradford Raschke Dealing with August burnout and Trader Camaraderie
The Elite Circle
Times and Sales more data to be displayed
TradeStation
MT4 Switch All Charts 1 click
Platforms and Indicators
 
 
(login for full post details)
  #142 (permalink)
Site Sponsor

Web: Jigsaw Trading
AMA: Ask Me Anything
Webinars: Jigsaw Trading Webinars
Elite offer: Click here
 
 
Jigsaw Trading's Avatar
 
Posts: 2,977 since Nov 2010
Thanks: 823 given, 10,285 received


SMCJB View Post
Interestingly there are 115 systems that are in my system list from last year, that are not in my list from this week. I have double checked and iSystems returns an error for these systems. This represents 10.4% of the systems from last year!

Offsetting that, there are 494 new systems, so the total number of systems actually has increased by 34.2%. Similarly when looking at my 15 month data set there are now 2092 samples that are no longer available, which is offset by 9191 new samples for a 45.8% increase. Interestingly the average sample in my old data lost $2416 over the 15 months, while the average sample in the new data set losses $2743. So things are getting worse not better! The 115 systems/2092 samples that have disappeared had an average loss of $4370 over the 15 months. The new 9191 samples have an average loss of $3689. Note some of these are new systems but some of these represent new data for old systems. Remember, each sample is made up of "tracked" data, and does not include any (over fitted make believe) backtest results, which I'm sure were all positive!

So first question is, do I add the now removed 2092 samples into my new/current data set?

Sounds like you could pick any one, fade it & make money

If you have any questions about the products or services provided, please send me a Private Message or use the futures.io "Ask Me Anything" thread
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to Jigsaw Trading for this post:
 
(login for full post details)
  #143 (permalink)
Legendary Market Wizard
Houston, TX
 
Experience: Advanced
Platform: Trading Technologies
Broker: Advantage Futures
Trading: Energy
 
Posts: 3,559 since Dec 2013
Thanks: 2,839 given, 6,661 received


Been Traveling

sixtyseven View Post
Was that 1 system each for the highest/lowest/nearest or a certain % of the systems - ie. top 5%, bottom 5% etc?

It was supposed to be the single highest but if there were multiple systems with the same score then it took all of them. I didn't check to see how prevalent that issue was. I did run a chart of the "Top 5" systems that had the highest/lowest/nearest performance in the previous 12 months when compared to the backtest, and the Top 5 was NOT as good as the Top 1.

Jigsaw Trading View Post
Sounds like you could pick any one, fade it & make money

Very True. One thing to remember is each system does have a fee and commissions. So if we say the average system charges $100 and trades 10 round turns a month of $12.50 per side, we would expect the performance of a system doing the opposite of what it actually does, to be about $450/month worse. So a system would need to be consistently losing at least $450/month before we could make money trading the opposite of it.

Reply With Quote
The following 3 users say Thank You to SMCJB for this post:
 
(login for full post details)
  #144 (permalink)
Legendary Market Wizard
Houston, TX
 
Experience: Advanced
Platform: Trading Technologies
Broker: Advantage Futures
Trading: Energy
 
Posts: 3,559 since Dec 2013
Thanks: 2,839 given, 6,661 received

Still here, and still working on this, although much of most recent work has been rewriting code as my R is much better than it was when I started this almost 2 years ago. The two major changes are moving everything into tibbles and using tidyverse and also using recursive functions like map from purrr. Another big change is to run the web scrape once per month and store the results in a (large) file. So now I just load that file and run data requests against it rather than have to rescrape the webpage for every request. Will save a lot of computation time and also has the added advantage that as systems get deleted I will still have the historical data!

A private message from somebody has given me another idea. Currently everything I do is around individual systems, with the idea that if we can find systems we expect to perform well we could then combine them into a portfolio. Would we be more successful if we reversed that? Is there a way to combine systems into portfolio's and then predict how the protfolio's would perform? Question is how would we create portfolio's? There are currently 987 systems that have 13 months or more of tracked history. So that's 973,182 possible portfolio's of 2 systems and 958 million possible portfolio's of 3 systems! Theoretically should be able to handle that quiet easily - well at least the portfolio's of two systems - should just take a lot longer to calculate! Something else for me to think about.

Reply With Quote
The following 2 users say Thank You to SMCJB for this post:
 
(login for full post details)
  #145 (permalink)
Legendary Market Wizard
Houston, TX
 
Experience: Advanced
Platform: Trading Technologies
Broker: Advantage Futures
Trading: Energy
 
Posts: 3,559 since Dec 2013
Thanks: 2,839 given, 6,661 received

For a private DM. Can't attach these to the message.


Reply With Quote
 
(login for full post details)
  #146 (permalink)
Legendary Market Wizard
Houston, TX
 
Experience: Advanced
Platform: Trading Technologies
Broker: Advantage Futures
Trading: Energy
 
Posts: 3,559 since Dec 2013
Thanks: 2,839 given, 6,661 received

As a (potentially final) update I closed the account that I had used for trading iSystems right before year end. While I still find this dataset very interesting I just don't have the time to do the analysis I would like. All in all this ended up being quiet an expensive experiment!

A final word of warning is that I should probably highlight that 32% of my losses where system fees and commissions. I think anybody actively trading iSystems (or anything similar) will probably find that system fees and commissions are 5% or even higher annually* on a properly funded account. Unless you have some edge that's quiet a heavy cost structure to overcome, especially if you compare it to any CTA index!

*While I did have the account open for 2 years 3 months, I only actively traded in 11 of the 27 months. Obviously how well your account is funded will have a big impact on what fees are as a percentage of equity. I would consider my account to be well funded and my systems fees and commissions in those 11 months was 8.2% which would be about 9% on an annualized basis!

Reply With Quote
The following 3 users say Thank You to SMCJB for this post:
 
(login for full post details)
  #147 (permalink)
Legendary Market Wizard
Houston, TX
 
Experience: Advanced
Platform: Trading Technologies
Broker: Advantage Futures
Trading: Energy
 
Posts: 3,559 since Dec 2013
Thanks: 2,839 given, 6,661 received

BLAST FROM THE PAST !

Guess who's back, back again
SMCJB's back, tell a friend
- Without Me, Eminem

So to recap, 3 years ago almost exactly....

SMCJB View Post
Saturday 5th August

In @kevinkdog’s Strategy Factory class, he teaches how to test and develop trading systems. Once you have developed a system, he teaches that the last thing you should do before going live is to ‘incubate’ the system. This basically means that you watch the system for a period of time to verify that it’s performance is what you expect to be. If the system performs significantly different during incubation than it did in backtest, that’s probably the best warning you could get that something is probably wrong, without risking and losing real money.

I’m not going to go into detail about what iSystems is because I assume that if you are reading this then you probably already know. Something that I do want to mention though is how the iSystems timeline works. Each system has a “Start Date”, a “Tracked Since Date” and a “Live Since Date”. The way I interpret this is that the “Start Date” is the date that the backtest starts. The “Tracked Since Date” is the date that isystems started tracking or following the system, and is such outside of the backtest. Finally the “Live Since Date” is the date that somebody first started trading the system.

While I am interested in how systems have performed in there backtest, what I am really interested in is how they have performed outside the backtest, aka since being tracked. The main “Explore Systems” table has lots of good information on systems including Drawdowns, Sharpe Ratio’s etc but nearly all of this information includes the backtest. While it is possible to see the statistics only since tracked on the individual system pages, it is not possible to see this data for all systems on the “Explore Systems” page.

At the time of writing, iSystems has 869 systems that have an average start date of June’06 (11.1 years) and an average tracked since date of May’15 (2.2 years). The systems have an average backtest PnL of just over $100K (almost $9k/year) but an average PnL since being tracked that is negative!!! Obviously it is important to separate the wheat from the chaff – assuming there is some wheat to be found.

In line with Kevin’s ‘incubation’ period the first thing that I did was eliminate every system that hadn’t been tracked for a minimum period of time and any system that didn’t have a positive PnL since being tracked. This reduced my system universe from 869 to 158. The systems have an average backtest PnL of over $105K and an average PnL since being tracked of almost $12k. This equates to over $11k/year and $3.5k/year respectively.



For these systems I calculated average yearly profit since their start date, and since the tracked date. The ratio of these two values, Since Tracked divided by Since Start, is one of my key indicators. Using this ratio along with Profit Factor and the three risk ratios, Sharpe Sortino & Sterling, I further reduced my system universe from 158 to 32. The 32 systems have an average start date of May’06 (11.2 years) and an average tracked since date of Mar’15 (2.4 years).

The systems have an average backtest PnL of over $120K and an average PnL since being tracked of almost $16k. This equates to almost $14k/year and $6.5k/year respectively. The 32 systems, are from 15 different developers, spanning 10 different symbols (although FDAX is 11 of the 32), with an approximate 2:1 intraday:swing ratio.

Now that I have a manageable amount of systems I download all the monthly data for each system and calculated average PnL, Profit Factor, Sharpe Ratio and Sortino Ratio* for each system, both since start and since tracked, and the ratio of the two. The average ratios where
Monthly Average PnL 69%
Sharpe Ratio 69%
Sortino Ratio 67%

My goal is to find the portfolio of systems that has the best risk to reward, while at the same time having a since tracked performance as close to possible to the backtest. As such we are not looking at outright PnL numbers, but are only interested in profit scaled by risk. There was actually one system, whose performance has been so much better since tracked than the backtest that I eliminated it (at least for now).

I have recently spent a lot of time looking at portfolio optimization and have written a Monte Carlo program in excel, to optimize portfolio building. Unfortunately with most of these systems, the backtests look so good, that when combined into portfolio’s the portfolio show’s unrealistic results, in many cases resulting in zero drawdowns. I could repeat the process only using ‘since tracked’ data but the data set would be so small I don’t think it would be statistically significant. I wish I could perform this type of analysis on a better granularity than monthly, but unfortunately that’s not possible. While I can drill down to the per trade data for individual systems, since trades don’t line up day to day, there’s no way to model portfolio performance without day by day, rather than trade by trade, PnL.

Looking predominantly at since tracked Profit Factor, Sharpe Ratio and Sortino Ratio, but also taking into consideration the ‘since tracked/since start’ ratio’s I selected 6 systems which ranked 1, 2, 3, 5, 6 & 9 in my rankings. The #4 ranked system was the already mentioned system whose since tracked results are significantly higher than the back test. The #7 system was a swing system but I wanted another day trading system. The #8 system was a day trading system but it trades the same symbol as my only other day trading system.

So I have 6 systems, from 5 different developers, trading 5 different symbols, 4 swing and 2 intra day. The average start date is Apr’09 (8.3 years), tracked date is Apr’16 (1.3 years), backtest profit of just over $100k ($14.5k/year) and since tracked profit of almost $14k ($10.4k/year).



*When calculating the Sortino Ratio I have found that dividing the square of the negative returns by the number of negative returns rather than the total number of returns, yields a ratio that has a lower correlation to the Sharpe Ratio, which I prefer.

So what happened to that Portfolio of 6 systems over the last 3 years.....






So what do I think?

To be honest the results are actually better than I expected, I really did wonder whether the portfolio would be in severe drawdown! I can confidently say though I'm glad I wasn't trading that protfolio for the last 3 years. If you had invested in Aug'17 like I did and held the position until now (which I didn't!) you would have been in a continual drawdown until March of this year. Even with the run up in the last 5 months performance is significantly below what would be expected and drawdowns have been 800% of what was modelled in the backrest. The Simplified* Monthly Sharpe Ratio of the Portfolio while the systems were in backtest was 1.18 but while the systems have been 'live' this has dropped to 0.17. Since Aug'17 it has been just 0.09. I literally picked the worse month possible to invest, the start of a 31 month drawdown!

NOTE: The drawdown values illustrated and discussed represent % drawdown of the portfolio INCLUDING backtest results. If we were to just look at the live results and assume the portfolio was backed with $100,000 then the drawdown would have maxed at 33%. Similar analysis since Aug'17 would have yielded a max drawdown of 49%! As you see from the 4th chart, for most of the 3 years, all 6 systems were in an individual drawdown.

* For the Quants out there, I say simplified because in this example I'm calculating the Sharpe Ratio as the 'Average Monthly $ Profit divided by the Standard Deviation of the Monthly $ Profit'. This ignores the risk free interest rate and also the complication of calculating what the theoretical** 'return' is of a highly leveraged futures portfolio.
** What is the denominator in the return of a futures contract? The Margin Requirement? The Notional Size of the Contract? For iSystems is it the 'Suggested Capital' or is it the "Minimum Capital". Do either of these really relate to the capital of a portfolio of non-correlated systems?

Reply With Quote
The following 2 users say Thank You to SMCJB for this post:
 
(login for full post details)
  #148 (permalink)
Legendary Market Wizard
Houston, TX
 
Experience: Advanced
Platform: Trading Technologies
Broker: Advantage Futures
Trading: Energy
 
Posts: 3,559 since Dec 2013
Thanks: 2,839 given, 6,661 received


dadarara View Post
algo by the name : CIRUS RVST SILVER ARTICROAK

results for live activity:
Jul17 - Feb18(curr)
+2700,-3913,+3910,+5460,+4881,+9575,-2727,+3660

looks amazing.

what am I missing? the Live activity is looking very good and in line with the historical run.

so why do I need anything else?
why this should not be the first choice ? and maybe the only one?


SMCJB View Post
Greetings @dadarara and thank you for your interest in this thread.

First let me say, when I did my initial work this system was only weeks old and as such was eliminated from the analysis. Since I decided the hypothesis that recent performance similar to that in the back test isn't necessarily a good indicator of future success, I have not analyzed any new systems. I know from reference tables though that isystems have added a lot (and deleted a few) systems recently.

Your question gives me an oppurtunity to show off my latest system charts...



Two things jump out at me looking at that chart.
  • First, the fact that the equity line is below the lighter of the two gray funnels means that the system is performing 2 standard deviations worse than the backtest results.
  • Second, in the eight months that it has been live, it has had two drawdowns that were 4 times larger than the largest drawdown seen in the backtest
Performing some quick calculations the in backtest performance was $7540/month with a standard deviation of $4716, while since live it is $2943/month with a standard deviation of $4387. So your getting less than 40% of the performance with almost the same risk. (Ave/SD's 1.60 vs 0.67). So from a "compared to it's backtest perspective" I don't think it's in line with it's historical results at all. Of course that doesn't mean it isn't a good system.

Here's an Update.... hopefully you didn't invest....


Reply With Quote
 
(login for full post details)
  #149 (permalink)
Legendary Market Wizard
Houston, TX
 
Experience: Advanced
Platform: Trading Technologies
Broker: Advantage Futures
Trading: Energy
 
Posts: 3,559 since Dec 2013
Thanks: 2,839 given, 6,661 received


SMCJB View Post
On a final note Ilan Levy-Major mentioned an iSystems system in his recent @CannonTrading webinar. The out of sample (non-backtest) performance of the system is charted below. While the equity curve is positive it's worth noting that in its 13 months live, it's only achieved it's AVERAGE monthly performance seen in the backtest once. Remember the black line shows backtest performance, and the grey funnels show one and two standard deviation bands around that performance. So it's currently approximately three standard deviations below backtest expectations. It's also already exceeded it's previous maximum drawdown once as well. Another case of much weaker returns but much higher risk than seen in the backtest! Again doesn't mean it's a bad system, but would you consider it to be passing or failing incubation?



Update... Went up for 1 more month, and then straight down ever since.



For clarification, not trying to call out @CannonTrading in any way, just highlighting specific systems that were discussed in this thread and how they have performed since. As you can see, in almost every case the answer is 'poorly'.

Reply With Quote
The following user says Thank You to SMCJB for this post:
 
(login for full post details)
  #150 (permalink)
Legendary Market Wizard
Houston, TX
 
Experience: Advanced
Platform: Trading Technologies
Broker: Advantage Futures
Trading: Energy
 
Posts: 3,559 since Dec 2013
Thanks: 2,839 given, 6,661 received



InvestorZ View Post
MANDALA GASOLINE
Swing 6 _ 30y T-Bond ZB
CIRUS ST STOP 1% NASDAQ ODOR


SMCJB View Post
MiniSp Sevilla StopTakeProfit Intraday is very new, and already arguably not performing like its backtest.


Mandala Gasoline is performing exactly in line with it's backtest. I find this interesting in that Gasoline/RBOB is one of the most volatile futures contract out there but this system has small wins and doesn't experience any wild swings.


Swing 6 _ 30y T-Bond ZB again is very new with only 5 months of real data. Does it concern you that in the 27 months of backtest the only losing months were <$385> and <$432> but in the first month of non backtest data it lost <$1269>? Anyway another system that was down for the 11 months following the original post but has finally turned positive in the last two months. (We found one!!)


CIRUS ST STOP 1% NASDAQ ODOR is another (rare) system that seems to be performing very much in line with its backtest.

System 1, down since.


For some reason getting an error in my code for the second system. Can not replicate this with any other system. I think its related to the "-" in Feb 2019. Since the system costs $75/month is this suggesting the system made exactly $75 resulting in a net zero? A quick review of the 3 trades for the month implies not profits were $273 before commissions and slippage!?!


System 3. Flat since. Significantly underperforming expected results.

Reply With Quote
The following user says Thank You to SMCJB for this post:


futures io Trading Community Trading Journals > iSystems Journal


August 8, 2020


Upcoming Webinars and Events
 

Dealing with burnout w/Linda Bradford Raschke

Elite only
     



Copyright © 2020 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts