NexusFi: Find Your Edge


Home Menu

 





iSystems Journal


Discussion in Trading Journals

Updated
      Top Posters
    1. looks_one SMCJB with 94 posts (273 thanks)
    2. looks_two Mabi with 15 posts (15 thanks)
    3. looks_3 mattz with 7 posts (6 thanks)
    4. looks_4 tpredictor with 7 posts (11 thanks)
      Best Posters
    1. looks_one Big Mike with 14 thanks per post
    2. looks_two kevinkdog with 4.8 thanks per post
    3. looks_3 SMCJB with 2.9 thanks per post
    4. looks_4 Mabi with 1 thanks per post
    1. trending_up 40,699 views
    2. thumb_up 394 thanks given
    3. group 53 followers
    1. forum 179 posts
    2. attach_file 91 attachments




 
Search this Thread

iSystems Journal

  #121 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,041 since Dec 2013
Thanks Given: 4,375
Thanks Received: 10,192

Interestingly there are 115 systems that are in my system list from last year, that are not in my list from this week. I have double checked and iSystems returns an error for these systems. This represents 10.4% of the systems from last year!

Offsetting that, there are 494 new systems, so the total number of systems actually has increased by 34.2%. Similarly when looking at my 15 month data set there are now 2092 samples that are no longer available, which is offset by 9191 new samples for a 45.8% increase. Interestingly the average sample in my old data lost $2416 over the 15 months, while the average sample in the new data set losses $2743. So things are getting worse not better! The 115 systems/2092 samples that have disappeared had an average loss of $4370 over the 15 months. The new 9191 samples have an average loss of $3689. Note some of these are new systems but some of these represent new data for old systems. Remember, each sample is made up of "tracked" data, and does not include any (over fitted make believe) backtest results, which I'm sure were all positive!

So first question is, do I add the now removed 2092 samples into my new/current data set?

Started this thread Reply With Quote
Thanked by:

Can you help answer these questions
from other members on NexusFi?
New Micros: Ultra 10-Year & Ultra T-Bond -- Live Now
Treasury Notes and Bonds
Deepmoney LLM
Elite Quantitative GenAI/LLM
ZombieSqueeze
Platforms and Indicators
Are there any eval firms that allow you to sink to your …
Traders Hideout
The space time continuum and the dynamics of a financial …
Emini and Emicro Index
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
39 thanks
NexusFi site changelog and issues/problem reporting
26 thanks
The Program
18 thanks
GFIs1 1 DAX trade per day journal
18 thanks
  #122 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,041 since Dec 2013
Thanks Given: 4,375
Thanks Received: 10,192

Pretty Charts

Meant mosty as memory refresh




Since more than 50% of the systems are equity index's and since we know they behave completely differently than most other commodity futures (positive return bias and non asymmetric returns) I think a new 'feature' I will be creating is "Is Equity Index". This could potentially divide our data set into two. We will see.

Looking at the systems that were dropped vs currently active systems...
(Not much to see here, dropped systems if anything appear to have a higher slope).



Now looking at the systems that are equity index systems versus non-equity index systems we can see that the equity systems are a lot more noisy!



So what happens if we look at each of the Equity Index products separately? FDAX appears to have a lot more noise than anything else, but we know that the FDAX contract is significantly larger than the other contracts. I really need to redo this chart, and the prior one, with everything proportional to contract size but I don't currently have an accurate contract size variable.

Started this thread Reply With Quote
Thanked by:
  #123 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,041 since Dec 2013
Thanks Given: 4,375
Thanks Received: 10,192


In case you were wondering what the non-equity charts look like. Remember these still haven't been scaled by contract size




Eyeballing these the two that appear to have a positive correlation are Euro FX (URO) and Soybeans (ZS). Euro looks likes it could be due to outliers, but here is Soybeans. Is there a reason why systems would be more effective for a single contract over others??


Started this thread Reply With Quote
Thanked by:
  #124 (permalink)
 sixtyseven 
Golden Bay, New Zealand
 
Experience: Beginner
Platform: Sierra Chart
Trading: ES, NQ
Posts: 186 since May 2012
Thanks Given: 851
Thanks Received: 337


SMCJB View Post
So first question is, do I add the now removed 2092 samples into my new/current data set?

If I recall correctly you are creating a 'system' to identify isystems to trade as a portfolio. If thats the case then I'd want to include all the removed samples. I assume they were removed because they were curve fitted and sucked Live, rather than being raging winners. I also assume your 'system' includes sufficiently large number of Live tracked trades with decent performance, which means your 'system' probably avoided selecting any of the removed samples. I'm guessing that 115 that was on your list looked good tracked live, but didn't have the required track record to be included in the portfolio.

If your 'system' for selecting isystems is a WIP it certainly makes sense to include the removed.

Good luck in sorting the wheat from the chaff. Hopefully there is some wheat in there!

Reply With Quote
  #125 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,041 since Dec 2013
Thanks Given: 4,375
Thanks Received: 10,192

It's interesting rereading this thread, that a lot of my losses I experienced with isystems were mostly due to equity systems getting destroyed in the December '18 equity market route. License fees, commissions and brokerage where also a significant drag on the portfolio as well. Outside of that my experiences really weren't that bad. When the hard drive on my laptop failed and I lost almost two months of work, when of the really frustrating things was the loss of my charting functions. I was relatively new to R and very new to ggplot and they took me a lot of time to get right. Now that I'm considerably better at both they were a lot easier to recreate. Although I did spend way to much time trying to get the legend correct!

Back in February of 2018 @dadarara asked

dadarara View Post
algo by the name : CIRUS RVST SILVER ARTICROAK

results for live activity:
Jul17 - Feb18(curr)
+2700,-3913,+3910,+5460,+4881,+9575,-2727,+3660

looks amazing.

what am I missing? the Live activity is looking very good and in line with the historical run.

so why do I need anything else?
why this should not be the first choice ? and maybe the only one?

to which I replied...

SMCJB View Post
Your question gives me an oppurtunity to show off my latest system charts...



Two things jump out at me looking at that chart.
  • First, the fact that the equity line is below the lighter of the two gray funnels means that the system is performing 2 standard deviations worse than the backtest results.
  • Second, in the eight months that it has been live, it has had two drawdowns that were 4 times larger than the largest drawdown seen in the backtest
Performing some quick calculations the in backtest performance was $7540/month with a standard deviation of $4716, while since live it is $2943/month with a standard deviation of $4387. So your getting less than 40% of the performance with almost the same risk. (Ave/SD's 1.60 vs 0.67). So from a "compared to it's backtest perspective" I don't think it's in line with it's historical results at all. Of course that doesn't mean it isn't a good system.

So since I've just rewritten my charting function it seems a good time to see both how this system has actually done in the 15 months since, and to show off my new charts.



Very pretty huh? No seriously, the thing I find very interesting about this, is that since @dadarara asked this system has gone due south and performed terribly. Wish I could give myself a 'pat on the back' for flashing a warning sign but my results don't deserve any 'pats on the back'! Still I found it to be an interesting real world comparison.

Moving on...


sixtyseven View Post
If I recall correctly you are creating a 'system' to identify isystems to trade as a portfolio. If thats the case then I'd want to include all the removed samples. I assume they were removed because they were curve fitted and sucked Live, rather than being raging winners. I also assume your 'system' includes sufficiently large number of Live tracked trades with decent performance, which means your 'system' probably avoided selecting any of the removed samples. I'm guessing that 115 that was on your list looked good tracked live, but didn't have the required track record to be included in the portfolio.

If your 'system' for selecting isystems is a WIP it certainly makes sense to include the removed.

Good luck in sorting the wheat from the chaff. Hopefully there is some wheat in there!

I agree the obvious conclusion is to include the old data, and that is what I have started doing for now, hopefully it valuable in predicting the chaff! With regards to my prediction system, saying it's a WIP would be a stretch at this point. Since losing my hard drive I've done nothing until recently, and even then most of my recent work is data processing. Right before it failed I had just implemented a very crude random forest model and posted the results. The chart was rather crude, but blue represents trading the highest rated system every month, while red is trading the lowest rated. Rated meaning percentage of the trees within the RF (500 trees total) that voted "Good". I did discuss it in a little more detail in the quoted post.

SMCJB View Post

Obviously that shows some potential, even if you probably wouldn't want to touch it, as is. Going forward I need to do some more data processing, or more specifically identifying 'features' of the systems that I will then feed to the ML models. Obvious features would be the typical metrics, Profit Factor, Sharpe Ratio, Return to Drawdown etc. But there are less obvious ones as well. As I mentioned earlier this week "Is this an Equity System" will probably be a Binary Feature. I've thought about making "System Developer" a feature, with the idea that maybe certain developers are better than others, but I'm not sure how to handle developers that have very few systems. I could put them in an "other" category, but I'm worried that would pollute the data. Then what to do. Still hope to develop a Random Forest, Support Vector Machine, and a Neural Network and then ensemble the results and see what we get.

On a final note Ilan Levy-Major mentioned an iSystems system in his recent @CannonTrading webinar. The out of sample (non-backtest) performance of the system is charted below. While the equity curve is positive it's worth noting that in its 13 months live, it's only achieved it's AVERAGE monthly performance seen in the backtest once. Remember the black line shows backtest performance, and the grey funnels show one and two standard deviation bands around that performance. So it's currently approximately three standard deviations below backtest expectations. It's also already exceeded it's previous maximum drawdown once as well. Another case of much weaker returns but much higher risk than seen in the backtest! Again doesn't mean it's a bad system, but would you consider it to be passing or failing incubation?



Started this thread Reply With Quote
  #126 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,041 since Dec 2013
Thanks Given: 4,375
Thanks Received: 10,192

Here's a second charting function I finished which gives a slightly wider perspective on "21265".

It's not so much the "since tracked" data doesn't match the backtest as much as it is the first 8 months of the backtest seem different to the next 27 months! If I could only ignore that first 8 months... ohhhh hang on a second I can see what that looked like, back in 30 seconds...

How that looks a lot more reasonable. I'm not sure that just ignoring the inconvenient data would be considered sound analysis though, especially since the backtest is designed on much of the data just thrown out! If that second data did represent real complete data though, I'd probably think that chart looked decent! Question is would I have stopped trading it 3 months ago when performance exceeded 2 standard deviations from expectations? Probably depend upon when you actually started trading it.

Anyway, onwards.
Here is a high level system view of the data I am dealing with.
  • 1490 Systems
  • 866 With 15 months of tracked data (Tracked before 3/1/18)
  • 268 With 15 months tracked and is profitable while tracked.
  • 362 With 15 months tracked and is profitable in the last 3 months.
  • 159 With 15 months tracked and is both profitable while tracked and in the last 3 months.
So the systems I'm trying to identify probably represent less than 20% of the data! When you have very unbalanced data sets like this it becomes difficult, because the best way to improve accuracy is improve prediction of the dominant data category but we're only interest in the less dominant data. The most obvious example of this is that if we predict every system to be poor, we are immediately 80% right!

Started this thread Reply With Quote
  #127 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,041 since Dec 2013
Thanks Given: 4,375
Thanks Received: 10,192

Replace charts in previous deleted charts with ones that show results of trading both best and worst system.



Remember Maximum PnL definitely does not equate to "Best". Not surprisingly over 70% of the systems traded after having the Maximum PnL are FDAX systems, which is by far the largest contract. I suppose it's reassuring that the systems with the Max 12 month PnL outperformed the ones with the lowest 12 month PnL, but it's not reassuring that they still lost money. Great illustration of past performance is not a guarantee of future performance.



Now we see a much more significant out performance. But still PnL is all in the first 3 years and has been flat/negative in the 10 years since.

edit: Date shows the date of the beginning of the 12 months, hence the actual traded month is 13 months later

Started this thread Reply With Quote
  #128 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,041 since Dec 2013
Thanks Given: 4,375
Thanks Received: 10,192

Couple more.
First (Profit Factor) looks a lot like the Sharpe chart. The second is interesting but overall ugly.



Started this thread Reply With Quote
Thanked by:
  #129 (permalink)
InvestorZ
Toronto, Ontario, Canada
 
Posts: 2 since Jun 2019
Thanks Given: 1
Thanks Received: 1

Well this has been a very exciting and informative read. I joined futures.io for more info on isystems but it is very scarce. Literally only two threads on the entirety of the internet about isystems and they are both on this forum which is cool I guess. Two things that are discouraging to me

1. To this day, I cannot find anywhere that somebody says clearly "Yes. This is profitable". These are the systems that are good. If robot trading for the retail trader worked wouldn't there be more consensus? Or is it because it does work, and nobody wants to give out their secret sauce. Are there just not enough people doing this? I wish there was a way to see how many subscribers each system currently actually has.

2. People are hesitant to state which systems they are live trading, why? I plan to put money into MiniSp Sevilla StopTakeProfit Intraday very soon as it has a low drawdown, high win rate, at least some live profits, and doesn't require too much capital. Even though it is a newer system I am skeptically hopeful. Others that interest me are

MANDALA GASOLINE
Swing 6 _ 30y T-Bond ZB
CIRUS ST STOP 1% NASDAQ ODOR

Reply With Quote
  #130 (permalink)
 
KillerJukeBox's Avatar
 KillerJukeBox 
Dublin
 
Experience: Intermediate
Platform: Ninjatrader, Tastytrade
Broker: Ninjatrader Brokerage, IBKR, Tastytrade,
Trading: MNQ,MES. Stocks, Stock options
Posts: 84 since Oct 2018
Thanks Given: 13
Thanks Received: 95


just wanted to chime in that I appreciate all the effort and sharing going on here. Sadly i'm not in a position to contribute on this topic, but its really interesting! thanks

Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote




Last Updated on October 18, 2022


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts