Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,049 since Dec 2013
Thanks Given: 4,388
Thanks Received: 10,207
The old T-Stat. Been a long time since I did my Statistics class.
So this was my portfolio.
Assuming I'm doing the Math right
The backtest/red line has Mean Monthly PnL 7749, StDev 6585 and Month Count of 68. Hence the Std Err is 799.
To test whether this is significantly different than zero (7749-0)/799 = 9.73 which has a p value of 0. Given that it actually did make money I guess that is accurate.
Now to test whether this is significantly different than 50% of the mean (7749-3875)/799 = 4.85 which still has a p value of 0. Actual performance for the 54 blue and green values had an average of 2577. So even though the test says we have an approximate 100% change the mean of population is above 3875, our next 54 observations had an average of just 2577.
I have to test whether this is significantly different than 6417 (7749-6417)/799 = 1.67 until I get a p value of 0.05. 6417 is 249% of the actual realized 2577
Finally If I test whether this is significantly different than 7749 (7749-7749)/799 = 0 which has a p value of 0.5 as we would expect.
Can you help answer these questions from other members on NexusFi?
Okey . I am actually not using T stat. yet I have plugin for it for a program with Java math libraries so i will test start testing diffrent periods for forcasting in automated workflows making strategies. The key i understand from people that are using it is to test diffrent periods because forcast validity is limited and time based. If you use 100 weeks of data you can forcast only 5 weeks. So with 68 months your forcast is only valid for 3.4 months. My buddy who is using it found it best to only use last 100 weeks data and forcast 5 weeks and then rebuild his portfolios.
Also
We need calculate using each strategy because they are independent. So, the standard error generally is greater because we take in account covariance between strategies and there Variances. I use modern portfolio theory in this case, that is the same summing samples approuch in statistics. for example, a portfolio with two strategies, portfolio mean would be Str1mean + Str2mean. And the Portfolio Variance would be str1Var + str2Var + 2correlation12str1stdErr*str2stdErr (Covariance). Portfolio total std error would be square from Total Variance. Doing this, the risk will be more realistic.
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,049 since Dec 2013
Thanks Given: 4,388
Thanks Received: 10,207
That makes a lot of sense. Which leads me to the following charts which hopefully are self explanatory. (Again the underlying is my original portfolio of 6 systems).
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,049 since Dec 2013
Thanks Given: 4,388
Thanks Received: 10,207
Sharpe Ratio, T-Stat and Van Tharp's System Quality Number
Sharpe Ratio = Average (Return - Risk Free Return) / Standard Deviation (Return - Risk Free Return)
t-Stat = (Average - μ)/ Standard Deviation / SQRT(count) = SQRT(count) * (Average - μ) / Standard Deviation
Van Tharp's System Quality Number = SQRT(minimum(100, count)) * Average(Trade Pnl) / Standard Deviation(Trade PnL)
So in the case when Risk Free Return = 0 and μ = 0
Sharpe Ratio = Average (Return) / Standard Deviation (Return)
t-Stat = Average/ Standard Deviation / SQRT(count) = SQRT(count) * Average / Standard Deviation
Van Tharp's System Quality Number = SQRT(minimum(100, count)) * Average(Trade Pnl) / Standard Deviation(Trade PnL)
So basically they are the same thing but multiplied by a scalar.
Obviously a t-Test then compares the t-Stat to the T-Distribution, which isn't a linear relationship, actually looks like a sigmoid function.
If I go back to the data I last saved, which was June'19 and pick the three systems with the highest calculated Sharpe Ratio (defined as Average Monthly PnL / Standard Deviation of Monthly PnL) these are the three systems...
That's three pretty ugly systems, especially that last one!
My conclusion, which I had already reached a long time ago, is that the Sharpe Ratio of a backtest is as unreliable as any other statistic from an overfit backtest. By extension a T-test is also.
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,049 since Dec 2013
Thanks Given: 4,388
Thanks Received: 10,207
Y Axis should be T-Stat not Probablility
Key words in that statement are overfit backtest. By definition an overfitted back test isn't an independent distribution, it's already been cherry picked
Just wanted to thank SMCJB for your awesome thread.
As people have mentioned, there is very little material on Isystems review that is not an affiliate link or some sponsor. So glad to have found this thread.
Just wanted to share my experience with Isystems as well. I wish I would have read this thread before I signed up but better late then never I guess.
I signed up through AMP futures and started with one bot. Looking at the backtesting results looks pretty tempting to try.
First trade the bot made I was up a decent amount so I was like great start. Signed up for a few more bots and made a few more positive trades. Then probably the next few days, all the bot end up hitting their biggest drawdowns. Coincidence, who knows.
I think finding a profitable bot is probably harder then picking the right mutual fund. Its basically just pray and see what happens. I was really thinking I need a longer sample size but it seems like over the long run, just the fees, commissions is the biggest hurdle to overcome because then you already starting in the negative.
Can you be profitable using these bots? I think so but a lot of what determines is pure luck. Picking right bot, and running the bot at the right time. Someone mentioned before they were activating the bot when it was its in worst draw down or when it was losing, I tried that, led to more losses. Thats like going to Vegas and playing roulette using the past history board to determine the future results. its a random event with no memory of the past.
I think the seduction of the past performance chart, hooks you in, thinking it will keep repeating those results but there is no guarantee it will do that. I think if you have a large bankroll, meaning your using Suggested Capital and your okay with losing it all, I think it could work out.
I started with this experiment knowing full well I was okay losing money but I think if I keep it going, Ill eventually lose it all. Yes you will win here and there but overall the odds are against you in being profitable in the long term.
