iSystems Journal

September 29th, 2017, 03:24 PM

Friday 29th September - Understanding what went wrong this Month ☹

This month has - to be blunt - been almost disastrous. After the nice $4k profit in August I somehow managed to lose about $10k in September, meaning I'm now down about $6k. Given that this portfolio of 8 systems largest drawdown ever, which was also a single month, was approximately $4.7 that's rather eye opening.

Looking at each system individually none of them are performing terrible, but 7 of the 8 eight had really bad months. Profit in terms of standard deviations away from expected were +0.51, -0.84, -0.97, -1.07, -1.2, -1.23, -1.36, -1.71 which isn't pretty. Since in average month we would expect about 5 systems to win, and 3 to lose, the summed up results are magnified and the portfolio performed 2.48 SD's below expectation!

So we had a bad month, and the killer was correlation. I think this chart sums up the problem in September. While the blue/89 months/all data does include a lot of backtest data, for the orange/12 months all 8 systems were live. As might be expected the live data distribution is shifted to the left, but still a result of 1 is a significant outlier.

For what it's worth I deactivated 1 system today because after two months it's Tracked/Backtest ratio has dropped to a level that it no longer would qualify for the short list.

Probably post more this weekend on exact month end results and some analysis on thoughts on when to quit.

September 29th, 2017, 03:43 PM

SMCJB

Profit in terms of standard deviations away from expected were +0.51, -0.84, -0.97, -1.07, -1.2, -1.23, -1.36, -1.71 which isn't pretty.

I should have probably mentioned that for the average system I can be 0.478 SD's below expectation and still be breakeven which probably makes those numbers look slightly less horrific. The portfolio can be 1.14 SD's below expectation and still be breakeven.

October 1st, 2017, 11:56 AM

Sorry quick side question: these systems do u know a) what the logic is b) can u tweak the logic c) give u entries and exit stats live I.e u can actually manually go look at the trades yourself for the last month etc and d) does it tell u what timeframe charts are being used in the calculation like 5 min, 60 min etc- point is can u take these and actually go to a chart and plot trades to visualize/analyze?

Sent from my iPhone using NexusFi mobile app

October 1st, 2017, 06:16 PM

Switching Systems Off

Mabi

I am working on another approach to actually switch on systems first when they are loosing .Since all systems will have loosing perdiods no matter how good they are. I am trying to use consecutive winners and loosers and standard deviation using the equity curve from the strategy to determain when to start trading it. I always launched strategies when they are doing good and then they go in to drawdown almost immediately. But since it is not the strategy that changes it is the market so it depends on the type of strategy and Kevin D got it nailed by using WFO do adapt them which You cant do if You rent a system.

I think that's called "trading the equity curve" something I have no experience with at all. I didn't have a formal plan on switching systems off. I expected that as new systems became available I would switch them on, and switch off the worst performer, which is very different than what to do when a system goes wrong.

Last weekend at @kevinkdog's "Beyond the Strategy Club" the subject of when to switch off systems came up a lot. Performance vs history and % of max drawdown where the most discussed. In his book and his class, Kevin teaches a method where you plot trades (or weeks or months) and compare it to both of these metrics. The chart below shows this for the system that I switched off at the end of August. In this case I'm plotting monthly PnL since the system has been tracked. The red drawdown line shows the level at which the max drawdown would be exceeded and the upper and lower bounds show expected (from backtest) plus/minus 1 standard deviation. This clearly shows how the system surpassed it's previous max drawdown all in one month. The accompanying equity curve shows backtest PnL, the matching since tracked PnL and the drawdown.

The second chart shows the same information for the system I swicthed off a few days ago. Again it clearly shows how the system has dropped below 50% expected performance, which is the same as a 50% tracked/backtest ratio I use in my initial system screen. Another down month and it could also exceed it's max drawdown.

Finally, just to prove everything isn't negative, here's an example of one that is actually over performing.

October 1st, 2017, 06:20 PM

jokertrader

Sorry quick side question: these systems do u know a) what the logic is

No

jokertrader

b) can u tweak the logic

No

jokertrader

c) give u entries and exit stats live I.e u can actually manually go look at the trades yourself for the last month etc

Yes. There's available spreadsheets that show this. Unfortunately there's no way to download all these sheets automatically, and even if you could, theres a lot of work to do to convert this info into usable trade by trade analysis.

jokertrader

and d) does it tell u what timeframe charts are being used in the calculation like 5 min, 60 min etc

No, although that info is available in some of the system names

jokertrader

- point is can u take these and actually go to a chart and plot trades to visualize/analyze?

You would have to do that manually. There are no charts that show this automatically. In fact there are no charts other than equity curves.

October 3rd, 2017, 12:37 AM

Scraping & Analysis
Worked out a way to scrape the monthly PnL table using R. This could be a huge improvement in my analysis techniques.
Next step is automate/loop so that I download data for every system.
Then the question is do I dump it out of R into Excel, or try and do further calcs in R.

Code

> XYZ
  Year   Jan   Feb   Mar   Apr   May   Jun   Jul   Aug   Sep   Oct  Nov   Dec  Total
1 2017  5804 -4728  1671   -60   -60   -60  -874  2756  6499   -60   NA    NA  10888
2 2016 -2891 10984 15152 -2854   694 23598  2204   -60  6100 -4375  -60  5625  54117
3 2015  8496 15394  2377  3724  3777 24911 -1047 15951 11478  7996 4267 14795 112120
4 2014   -60  7462  2712  9796   -60   -60   -60  7321  2010 23937 1942 -4800  50142
5 2013   -60  2128   -60  1181  7151  4150 -8871   -60   -60   -60 2031 -5688   1782
6 2012 -2557   -60   -60  2060 11437  2867  9747   -60   -60   -60 6944   -60  30138
7 2011    NA    NA  9722   -60   -60  5374 13720  5099 17429 21594 8038  -975  79882

October 4th, 2017, 01:04 AM

Ended up with this chart, not sure it's particularly insightful but thought since I have it I might as well post it.

Each cross represents a system, with its "Start Date" (aka back test start date) on the x-axis and its "Tracked Since Date" (aka out of sample - not backtest) on the y-axis.
The red lines are the border for systems that have been tracked for 1, 2 and 3 years.
The blue line is the border for systems whose "time since tracked" is equal to "back test time". Everything above the line has a smaller out of sample than in sample.
The faint diagonal black lines show the border for systems that have a backtest of 0, 1, 2 and 3 years.

I took a look at all the systems down there in the "low center", that have a very long "since tracked"/oos but <3 years backtest and over half have lost money. The ones that haven't, while profitable have some low return/DD or MAR ratio's, although at least one has clients!

EDIT: Just ran the correlation of "Years Tracked / Years in Backtest" vs "USD Year Tracked / USD Year Backtest" and over the almost 1000 systems it was 0.016 so that isn't a good indicator!

October 5th, 2017, 09:39 AM

SMCJB

Friday 29th September - Understanding what went wrong this Month ☹

This month has - to be blunt - been almost disastrous. After the nice $4k profit in August I somehow managed to lose about $10k in September, meaning I'm now down about $6k. Given that this portfolio of 8 systems largest drawdown ever, which was also a single month, was approximately $4.7 that's rather eye opening.

Thank you SMCJB for sharing all this, very useful info especially for starters like me. However, all parameters you use are somewhat historic, standard deviation etc. - all work until there are issues like you had in August. Would it be better to just select, say, 2 systems with the opposite strategies to minimize/offset losses and keep it that simple. Also, two months are not representative and you've done soo much analysis already based on 2 months only.

And one side quick question, they offer 5 clearing companies on their website, i'm trying to choose one, have heard a lot good and bad about AMP, but will likely use them. To be honest, I do not see fundamental differences between these 5 for isystem (yes, capital is different, but the higher the capital the higher the fees, plus i only invest a few thousand). So if I need to know something very important about selecting clearing please guide me... Thank you!

October 5th, 2017, 09:49 AM

tradeday

Thank you SMCJB for sharing all this, very useful info especially for starters like me. However, all parameters you use are somewhat historic, standard deviation etc. - all work until there are issues like you had in August. Would it be better to just select, say, 2 systems with the opposite strategies to minimize/offset losses and keep it that simple.

Thanks for the comments. I'm just curious how would you choose those two systems if most math calculations are 'somewhat historic'?

tradeday

Also, two months are not representative and you've done soo much analysis already based on 2 months only.

I've been trading the systems for 2 months, but the analysis is based upon the entire history of these systems which in some cases is over 15 years. I would agree though that 2 months of results isn't enough to draw a final conclusion.

tradeday

And one side quick question, they offer 5 clearing companies on their website, i'm trying to choose one, have heard a lot good and bad about AMP, but will likely use them. To be honest, I do not see fundamental differences between these 5 for isystem (yes, capital is different, but the higher the capital the higher the fees, plus i only invest a few thousand). So if I need to know something very important about selecting clearing please guide me... Thank you!

As far as I know everything (Capital, Commissions, License Fees) is the same no matter what FCM or Introducing Broker you use (at least in the US).

mattz · October 5th, 2017, 02:22 PM

SMCJB

As far as I know everything (Capital, Commissions, License Fees) is the same no matter what FCM or Introducing Broker you use (at least in the US).

That is correct. The FCM/Broker must adhere to the fees as advertised by iSystems.

Thanks,
Matt Z
Optimus Futures

There is a substantial risk of lss in futures trading. Past performance is not indicative of future results.

iSystems Journal

Discussion in Trading Journals

iSystems Journal