Hi guys,
I am also interested to know if anyone has any personal experience with the automated isystems platform?

The backtested results for some of the algo systems are certainly ...interesting, but if someone could give their experience on what it's actually like to have real money vested into any of the automated trading solutions (and not just a sim account) that would be great.

Thanks,
~J

We have implemented the iSystems services and the automated execution of trading.
I do not think you can talk in terms of "personal experience" because there are many systems with a variety of results.
Therefore, the experience will vary on the performance of the system that was chosen.
Prior to use of this automation engine, I must have spent 10 hours on this to develop a due diligence process in the system selection. These systems can not be chosen blindly based on performance only. There are many variables that you can consider prior to choosing one. I find that the execution and reporting of the systems is one of the highest calibers out there.

THERE IS S SUBSTANTIAL RISK OF LOSS IN FUTURES TRADING. PAST PERFORMANCE IS NOT INDICTIVE IF FUTURE RESULTS.

Matt
Optimus Futures

even if one of these systems work as more and more users come online and others leverage up is it not dooms day at some point.

Well, what you can do is follow the slippage that is available on the platform. You can directly see the difference between the hypothetical and real-time execution. If you find it growing over time you can make a decision as far as the impact it has on the performance. If you need help with seeing this, I would be more than happy to guide you.

Thank you,
Matt Z
Optimus Futures

There is a substantial risk of loss in future trading. Past performance is not indicative of future results.

@mattz
Is there a way to look at the performance data only for when the systems have been tracked or traded live? I've filtered the data to have been tracked for at a least year, but all the performance data is still from inception. So the few systems I have looked at all have great back tests - and good looking performance data - but most have performed significantly worse since being tracked. It would be good to be able to search without their (in many cases unrealistic) back tests.

Thanks @mattz, I'll try and call you this week, but it may slip to next week.

@mattz @Stage5 Anthony
Is there a way to look at the performance data only for when the systems have been tracked or traded live? I've filtered the data to have been tracked for at a least year, but all the performance data is still from inception. So the few systems I have looked at all have great back tests - and good looking performance data - but most have performed significantly worse since being tracked. It would be good to be able to search without their (in many cases unrealistic) back tests.

Here's some analysis I did do the other night. I filtered to only include systems that have a track record of over a year and are profitable. I think of this as every system that is through it's 'incubation period'. I then calculated their "Profit per Day since Start Date", and "Profit per Day since 'Tracked Since' Date". The chart shows what I got.

Obviously everything below the "y=x" line has performed worse since being tracked while everything below the "y = 0.5x" line has significantly performance worse (less than 50%). The line of best fit, y = 0.2989 says that the average system performance being tracked is approximately 30% of what it was for the full history, but a 21% R Squared isn't that high.

While I can think of several reasons why this may not an effective way to look at every system I would also point out that the "Full History" includes the "Tracked Since History". If I extracted that, the disparity would be even worse. This is why I wanted to have a performance table based purely upon tracked data, excluding the backtest data. The results would be very different. Three of the four systems with the highest Dollars/Day are in the bottom quartile of the ratio.



As an aside, 9 of the 13 (~69%) systems I screened to take a more detailed look at were FDAX systems.

I'm travelling for the next few days but will follow up next week.

This is some great work. Hopefully people will realize the implications of this - real time performance is usually not as good as backtest performance.

Thanks for sharing!

I do not think you can combine systems together via iSystems itself, but I have a few customers who downloaded it via CSV to Excel spreadsheets in an attempt to build some negative correlations.

Thanks,
Matt Z
Optimus Futures

There is a substantial risk of loss in futures trading. Past performance is not indicative of future results.

Is there a link to download as a CSV file? I didn't see one at your website. Thanks.

Go to https://optimusfutures.isystems.com/
1) choose the system you wish to evaluate
2) Scroll to the bottom of the systems where is shows performance
3) Click on the Tab that says "Trade Log" or "Session Log". Both will have a small X logo for Excel, and you can just click on it.

I hope this helps.

Regards,
Matt Z
Optimus Futures

There is a substantial risk of loss in futures trading. Past performance is not indicative of future results.

Here's some analysis ....

The line of best fit, y = 0.2989 says that the average system performance being tracked is approximately 30% of what it was for the full history, but a 21% R Squared isn't that high.

As an aside, 9 of the 13 (~69%) systems I screened to take a more detailed look at were FDAX systems.

Great ideas @SMCJB. Does this have the capability to do portfolio level testing, to look for risk opportunities?

I do not think you can combine systems together via iSystems itself, but I have a few customers who downloaded it via CSV to Excel spreadsheets in an attempt to build some negative correlations.

Thanks,
Matt Z

This is some great work. Hopefully people will realize the implications of this - real time performance is usually not as good as backtest performance.

Thanks for sharing!

As far as I know, and based upon @mattz comment it does not, but my goal is to develop a methodology to do exactly that myself. Portfolio of uncorrelated systems to maximize Return/Drawdown. As they say, Diversification is the only free lunch!

Go to https://optimusfutures.isystems.com/
1) choose the system you wish to evaluate
2) Scroll to the bottom of the systems where is shows performance
3) Click on the Tab that says "Trade Log" or "Session Log". Both will have a small X logo for Excel, and you can just click on it.

I hope this helps.

Regards,
Matt Z

Not the best of formats, but usable.

So this is the 10 Systems I quickly picked. Haven't done any portfolio optimization at all at this point, just constructed a portfolio that trades one lot of each system, which is obviously very unweighted.



Slope ...
during pure backtest (blue) was $195/calendar day
during mixed backtest/live (red) was $334/calendar day
during pre live-tracked (blue+red) was $268/calendar day
during live/tracked (green) was $143/calendar day

Obviously live is quite a bit less than backtest. Max drawdown was about 13K but since all 10 systems have been live, just over 13 months, there's been two different 10k drawdowns.

Diversification is the only holy grail

82% of the systems that have been tracked live for at least a year and have a positive PnL are equity systems and you know what they say about people who make money in a bull market and think they are genius's! (Dax 1 year return is +27% and S&P is +16%)
1 Gold, 1 Wheat, 5 Euro, 8 Bund & 10 Crude. Those are the only non-equity systems (that meet the above criteria). Still looking for that Feeder Cattle System :-)

For anybody interested, the first phase of my isystems experiment is finished after a $5k loss over the first 2.67 months of live trading.

It appears that how these systems perform in any (non-back tested) 12 month period is not in anyway indicative of how they will perform in the next 12 months. ie Using that analysis technique in isolation is non-robust.

The analysis of how I reached that conclusion, and details of my actual experiences can be found in the journal linked above.

I still believe that this data set can be very valuable as it contains almost a 1000 systems that in some cases have several years of real, non backtested, performance history. My next steps will be to take a different path and use some machine learning techniques to try and identify which systems to trade.

I have recently discovered that some systems do get deleted, and don't show in the performance reports anymore. Unfortunately things does bring survivorship-bias into the equation!

What did everyone think of the webinar?

Anybody put real money in any of these systems? And what was the result?

I seem not to be able to find anybody with real life experience with real $$ in any of those systems.

I'm a bit skeptical as I had a bad experience few years ago with a professional managed account that was auto traded. There was a lot of things that weren't disclosed before investing.

The profit factors seem to be optimistic to say the least and not count all the fees/commissions,etc... If you ever developed your own system you know how hard it's to get a PF > 1.5 after expenses in real world.

Anyone??
Please, no 'depends' type answers... I just want to know which system you invested in and what is the results so far.

Being blunt did you actually read the entire thread and my related journal? If you do, and have more questions feel free to ask, here, in my journal or by PM, I'm more than willing to help others who are seriously interested, assuming they are willing to share as well.

So to sum up having the decision making process cloaked/hidden when you buy a black box makes the whole system trading process much harder. I think you are much better off to program your own ideas and that way you know why you made or lost money.

Anybody put real money in any of these systems? And what was the result?

I seem not to be able to find anybody with real life experience with real $$ in any of those systems.

I'm a bit skeptical as I had a bad experience few years ago with a professional managed account that was auto traded. There was a lot of things that weren't disclosed before investing.

The profit factors seem to be optimistic to say the least and not count all the fees/commissions,etc... If you ever developed your own system you know how hard it's to get a PF > 1.5 after expenses in real world.

Anyone??

Please, no 'depends' type answers... I just want to know which system you invested in and what is the results so far.

I've put real money on several systems and lost money on almost all of them. I've been trying to make a profit on iSystems for over two years with no luck. I choose a system with what appears to be an outstanding track record, but shortly after I activate the system it starts losing and continues losing for months. I'm not talking about a couple systems. I've watched this happen over and over again on almost every system I've traded. I've traded almost two dozen different systems. I wish I could tell you why this keeps happening but I can't figure it out.

I've put real money on several systems and lost money on almost all of them. I've been trying to make a profit on iSystems for over two years with no luck. I choose a system with what appears to be an outstanding track record, but shortly after I activate the system it starts losing and continues losing for months. I'm not talking about a couple systems. I've watched this happen over and over again on almost every system I've traded. I've traded almost two dozen different systems. I wish I could tell you why this keeps happening but I can't figure it out.

I've put real money on several systems and lost money on almost all of them. I've been trying to make a profit on iSystems for over two years with no luck. I choose a system with what appears to be an outstanding track record, but shortly after I activate the system it starts losing and continues losing for months. I'm not talking about a couple systems. I've watched this happen over and over again on almost every system I've traded. I've traded almost two dozen different systems. I wish I could tell you why this keeps happening but I can't figure it out.

We looked at the TradingMotion / iSystems platform. It's well done technically, but in our opinion it has one important design flaw:

There was no way to retire a system.
However, the management of systems is essential!

Only the vendor knows when a system is not viable anymore and therefore it should be the vendor's responsibility to take care of this.

Retiring could mean that a system data is still there for later review but the system is stopped, no more data is collected and nobody can be subscribed to it.

We requested such a feature but it never happened.

The result now is that there is a huge number of systems which are not taken care of. The customer has no idea if a system is still valid.

This makes it almost impossible to be profitable for the customer. How can he know when to stop a system?

We should not forget, markets change and no system will work forever. At least fine adjustments may be required over time.

Only if they were all managed together and turned on and off by some OTHER switching software that might be something.

Short of that. I'm out.

A developer uploads 10 systems. Let's assume they are curvefitted and have no real edge. What will happen that a few may actually look good for a few months, just by pure luck. It's pure statistics.

I have 15498 samples of data, with 15 consecutive months of since tracked, not backtest, data. For now I am keeping back the most recent data and using 13924 samples. If a system has a 50% chance of being profitable each month, then the probability of being profitable 12 months in a row is 0.5^12 which would equate to 3.4 of my 13924 samples. In fact I have 4! Maybe surprisingly all 4 of these lost money over the next 3 months! The distribution of winning months is as follows. With a mean of 5.31 and sd of 1.89 the distribution is skewed to the lowerside

In this case the responsibility of system selection is with the developer, who knows best, and NOT with the customer.

Not sure if you saw it but I tried to develop a framework for identifying and trade profitable isystems several years ago. This was a real money experiment which can be put down as a failure. I even revisted the thread 18 months after I finished and did an update.

An additional issue I have is that there is no way to have a system developed use a rolling walkforward optimization. That will always lead to either a) over optimization or b) an out of date system.

While I like your portfolio idea, system creation and portfolio optimization are two equally complex and equally difficult tasks.

Thanks @Tirutrade for all the comments and suggestions. Not sure if you saw it but I tried to develop a framework for identifying and trade profitable isystems several years ago. This was a real money experiment which can be put down as a failure. I even revisted the thread 18 months after I finished and did an update. Not a pretty picture. The thread is

iSystems Journal Greetings and welcome to my iSystems Journal. While most people seem to create journals to help them with the psychological aspects of trading, the intent of this journal is to document my experiences with isystems.com in the hope that it may be helpful …

I think this is what happened, Maybe wasn't intentional at the beginning but that's what we ended up with. People don't realize that if you have a 1000 systems, and each system has a 50/50 chance of profitability, that in 10 years time you would expect one system to have made money in all 10 years. Great system or just pure luck? From my thread....


An additional issue I have is that there is no way to have a system developed use a rolling walkforward optimization. That will always lead to either a) over optimization or b) an out of date system.

With regard to stopping or pausing systems they definitely do deactivate systems as there are systems no longer available that used to be. Not sure if this is due to developer request or another issue. There are also systems on the platform that no longer trade, and haven't for years so I assume the developer has a date condition in the code.


While I like your portfolio idea, system creation and portfolio optimization are two equally complex and equally difficult tasks. In my experience of developers trying to build portfolios, systems are not being selected based upon how they work together, but how their combined equity curve looks. They are using overfitting further by using risk/reward optimization.

Regardless there is an inherent problem with picking systems. Its akin to picking stocks.
You buy the stocks that are rocketing up only to find you buy at the top.
Sell (with everyone else as it falters) and jump on the next red hot system.

That would be true if you think all systems are just curvefitted and provide no real edge.

But if a system is a mathematical description of a high probability setup and is able to execute it reliably, then the system will outperform the market as long as the edge is there.

Now the question is, do you believe such edges exist?
If yes, do you believe they can be described with mathematical formulas?

Great discussion.

I do think that *most* systems are curvefitted and provide no real edge. In fact I would say most provide a NEGATIVE edge.
A "high probably setup" is subjective and a moving target.
If you can "mathematically" define a setup which is a prerequisite for a mechanical system anyway then it is STATIC.
The MARKET will then be the deciding factor IF the system is profitable.
The system will do well as long as its aligned with the market.

Yes great conversation.

I think all the things you describe above are typical characteristics of a curvefitted system, not of a system with an edge. I agree however that many systems are curvefitted.

For me an edge is a particular behavior of a specific market, which this market keeps doing over and over again. The system exploits this behavior. Therefore as long as the edge = behavior is there, the system will be profitable.

It's not really a moving target and it's not about the system being aligned with the market. That would apply to a curvefitted system only.

But a system with an edge does not care what the market it trades does in general, but relies only on a very specific behavior which occurs here and there. Only on that. And if it's there, the system will work. Of course if the edge is gone, then it's game over too here.
But usually it's an edge because it's a behavior which been around for a long time. Else it would not be an edge.

I hope all this is not too confusing. It's also a bit too about how your understand the term edge here. This is basically how I understand it.

I do know of one fully mechanical system I've seen work first hand and work over time.

It's edge? The system is built around IF the market is in THIS environment trade THIS WAY and take these set of signals. If on the other hand the market is signaling it's in THAT environment then trade THAT WAY and take a different set of signals.

It looks to exploit certain signals IN certain environments only. The signals rely on the environment. And environment is described by the system.

Al Brooks says a 2/1 risk/reward within the proper context IS an edge.
Support and resistance used a certain way can be an edge.
To me these are ALL "edges" as are... "patience"... "perseverance"... "ability to learn from ones mistakes" ect...

Yes, I think this is a very good approach.



I see. Well you are really using the term edge in regards to discretionary trading. I was never talking about that.

I am a system trader. In this context an edge is a repeating pattern in the market which can exploited by a mechanical trading system. Whatever I said was related to that.

It's really about how you use the word edge here.

I agree with what you said about discretionary trading and edge. Anything which gives you and advantage vs. random is an edge then.

But in system trading, since it's rule based, the definition of the pattern has to be much more precise.

Since you have the data already in spreadsheet could you calculate Tracked/Total and sort based upon that percentage. Looks like Rombo Capital and Quantum Financial may have semi decent results.

If I remember correctly 'developer' was not Statistically Significant when it came to identifying good systems, but if you turned the model around, and tried to predict bad systems 'developer' was Statistically Significant!

Not sure what you want to do, but I attached the data in XLS.

Reducing the eligible universe to systems of overall profitable developers would probably improve chances quite a lot.

Of the 6 largest developers, 5 have a negative sum of live PnL and the 6th is barely positive (<$1000). Eliminating developers with negative Live results eliminates 52% of the developers and 73% of the systems!