My journaled journey to success - futures io
futures io



My journaled journey to success


Discussion in Trading Journals

Updated
      Top Posters
    1. looks_one gordo with 83 posts (271 thanks)
    2. looks_two Big Mike with 29 posts (74 thanks)
    3. looks_3 kareem40 with 19 posts (30 thanks)
    4. looks_4 Mickey Caine with 6 posts (16 thanks)
      Best Posters
    1. looks_one gordo with 3.3 thanks per post
    2. looks_two Mickey Caine with 2.7 thanks per post
    3. looks_3 Big Mike with 2.6 thanks per post
    4. looks_4 kareem40 with 1.6 thanks per post
    1. trending_up 37,893 views
    2. thumb_up 457 thanks given
    3. group 34 followers
    1. forum 183 posts
    2. attach_file 199 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 150,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

My journaled journey to success

(login for full post details)
  #171 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 50,182 since Jun 2009
Thanks: 32,680 given, 99,536 received


gordo View Post
So things are looking better. I have managed to build an indicator that appears to be predicting the direction of an indicator. I have abandoned trying to make the MACD work and am using the TSI indicator which is based on the work of William Blau in his book, "Momentum, Direction and Divergence" (1995). The reason is a matter of expediency. AI works on normalized data with a range of -1 to +1. The conversion back and forth from normalized to de-normalized data requires the operator to know the min and max of the infeed data. TSI has a max of 100 and min of -100. [The truth of the matter is that if you look at the formula, it only has that because the result is multiplied times 100. So in fact, I could change the formula and not even use the normalization/de-normalization routines.] MACD does not have such limits so the values can be anything and thus makes predictions much harder because you need more infeed data.

Anyway, I took some one-minute tick data, converted the data into TSI values. Ran the artificial intelligence program and got trained data. I plugged the trained data into the Ninja indicator and got a plot. If you look at the first attachment you can see the two values. The top being the actual TSI data and the bottom being the predicted data. The second shows the two in the same panel. What I am looking for is for the predictor (ie the black line) to preceed the green line in changes to TSI. That is I want the AI program to tell me which direction TSI is going to go. The results are VERY hopeful.

I think I am cooking with gasoline now!

Is this a one bar in advance model? To me the indicators look like they align on time, meaning I am not seeing the prediction. Maybe you could post another screenshot that shows the actual alignment, or I have misunderstood.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote

Can you help answer these questions
from other members on futures io?
Tips for directional bias during Asian Session?
Traders Hideout
mahDualCandlePattern
NinjaTrader
value chart indicator by sim22
NinjaTrader
need help with easy language OR breakout code:)
Traders Hideout
 
Best Threads (Most Thanked)
in the last 7 days on futures io
Spoo-nalysis ES e-mini futures S&P 500
44 thanks
How many day traders are profitable?
28 thanks
Why you should move from futures to forex
15 thanks
Attached Orders
13 thanks
fx "traders" hate real profitable traders
8 thanks
 
(login for full post details)
  #172 (permalink)
 kareem40 
Market Wizard
Dallas, TX
 
Experience: Intermediate
Platform: NT, TS
Broker: IB, Oanda
Trading: ES, MES, MNQ, MYM
 
kareem40's Avatar
 
Posts: 661 since May 2010
Thanks: 1,873 given, 876 received

Nice to see you posting again.

Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #173 (permalink)
 MrYou 
NC, USA
 
Experience: None
Platform: None Yet
Trading: Guitar
 
MrYou's Avatar
 
Posts: 403 since Jun 2011
Thanks: 618 given, 196 received


@gordo

Any update?

Reply With Quote
 
(login for full post details)
  #174 (permalink)
 echoeversky 
Puyallup, Washington State
 
Experience: Intermediate
Platform: NinjaTrader
Broker: NINJATRADER/CQG
Trading: NQ, CL, ES, YM
 
echoeversky's Avatar
 
Posts: 115 since Sep 2009
Thanks: 113 given, 48 received


gordo View Post
So things are looking better. I have managed to build an indicator that appears to be predicting the direction of an indicator. .... The results are VERY hopeful.

I think I am cooking with gasoline now!


It's been a month.. how goes?

Follow me on Twitter Reply With Quote
 
(login for full post details)
  #175 (permalink)
 NJAMC 
Atkinson, NH USA
 
Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker: NinjaTrader Brokerage
Trading: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,970 since Dec 2010
Thanks: 3,036 given, 2,382 received


gordo View Post
So things are looking better. I have managed to build an indicator that appears to be predicting the direction of an indicator. I have abandoned trying to make the MACD work and am using the TSI indicator which is based on the work of William Blau in his book, "Momentum, Direction and Divergence" (1995). The reason is a matter of expediency. AI works on normalized data with a range of -1 to +1. The conversion back and forth from normalized to de-normalized data requires the operator to know the min and max of the infeed data. TSI has a max of 100 and min of -100. [The truth of the matter is that if you look at the formula, it only has that because the result is multiplied times 100. So in fact, I could change the formula and not even use the normalization/de-normalization routines.] MACD does not have such limits so the values can be anything and thus makes predictions much harder because you need more infeed data.

Anyway, I took some one-minute tick data, converted the data into TSI values. Ran the artificial intelligence program and got trained data. I plugged the trained data into the Ninja indicator and got a plot. If you look at the first attachment you can see the two values. The top being the actual TSI data and the bottom being the predicted data. The second shows the two in the same panel. What I am looking for is for the predictor (ie the black line) to preceed the green line in changes to TSI. That is I want the AI program to tell me which direction TSI is going to go. The results are VERY hopeful.

I think I am cooking with gasoline now!

Hi @gordo,

Great to see others looking into ML for trading solutions! I haven't read your entire thread but this last post I have a few questions. It looks like you are saying your plots are made with In-Sample "training data". Is this the case? If so, it does look like your solution is starting to converge. Did you try your indicator with out of sample data? If so, how does it look?

Was this a Neural Network? If so, you might be a little under fit on your solution. You are very close to matching the desired output. You might add one or two more nodes as it looks like your topology is close. If you have a hidden layer, I would add it there first to see if you get a little better match. I would not shoot for perfect as you are likely over-fitting the data and it will not generalize so out of sample data will not work well.

Keep going!



Keep up the ML work!

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #176 (permalink)
 NJAMC 
Atkinson, NH USA
 
Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker: NinjaTrader Brokerage
Trading: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,970 since Dec 2010
Thanks: 3,036 given, 2,382 received


gordo View Post
Was less than optimistic last night with my SunSpot example. Got to thinking about the data. That data is not like trading data. Trading data flows. It can't go from 150 to 20. It has to move from 150 to 20. So I did some testing. First I started out with an easy one...a sine wave. The results were not too bad.



The problem is this data is predictable. So I ran another model. I used MACD data I had collected. Results this time were pretty interesting.




Now that DOES look interesting. Here's the problem. This data is from a workbench modeling program. Can I do the same thing with the C# program I was working with so many months ago? Time to find out. Dust off the old program and see if I can run the same data and get similar results. Then I have to convert that program to a DLL to run in Ninja Trader. Possible? Maybe. At least I am going to give it a whirl.

My first attempt at making my C# program learn the MACD data did not turn out as I would have hoped. I may need to do some more work here!




As I reflect on where I am right now, I was close before but didn't realize I was close. Let's see if I can get better results.

G

Hi @gordo,

Sorry I didn't see your thread before. I am not sure if it is help to you now but I would say your 2nd chart is a classic "Over-Fit" sample. The "solution" to the problem has way to many dimensions than needed to solve the problem so you started to fit NOISE.

The 3rd chart is a classic "Under-fit" example. The "solution" your NN or whatever assumed the solutions would look like did not have enough degrees of freedom. You need to find a "solution" between these two examples. Less complex solution than your first (or add dampening) and more than your 2nd example.

If you are interested, you can find more information about Over/Under fit here:

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to NJAMC for this post:
 
(login for full post details)
  #177 (permalink)
 gordo 
Tonganoxie, Kansas
 
Experience: Intermediate
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: CL,6E
 
gordo's Avatar
 
Posts: 160 since Nov 2009
Thanks: 129 given, 401 received

Used my TSI data. Have the program running in C# now.

Where have I been? Lost in life.....sigh.....Loose heart on this project when I fail or can't figure things out. Haven't found a forum or a group of link minded folks to work with. I'm just using this as a journal. Big Mike's folks are GREAT...no SUPER GREAT when it comes to Ninja and indicators and just general slap on the back "Go get em" support....not so much with C# programming and the AI program in C# when I run into a wall. The Heaton Research site is okay, but the people there don't want to work the problem, they are just interested in the quick free lunch. A person gets spoiled when they spend time on Big Mike's.....too many good folks in one family.....I trudge on.

So I am trying to refind myself. Dusted off the C# program. Re-learned some of what I forgot...the rest....GONE! Now to get this program into an indicator in Ninja.

Here is the TSI run. Data looks pretty close to the eyeball.

Attached Thumbnails
Click image for larger version

Name:	TSI in C#.png
Views:	237
Size:	77.2 KB
ID:	160298  
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to gordo for this post:
 
(login for full post details)
  #178 (permalink)
Ringspun
London England
 
 
Posts: 2 since Oct 2014
Thanks: 1 given, 1 received

Hi Gordo

I hope the trading is going well. This is just a quick note to say thanks for posting a journal. I'm currently on Page 6 and I am finding it very insightful. I can relate to so many of the ups and downs (and early discipline problems, which you seem to have sorted out. I'm still working on that).

Great job. Good luck for the future.

C

Reply With Quote
The following user says Thank You to Ringspun for this post:
 
(login for full post details)
  #179 (permalink)
 gordo 
Tonganoxie, Kansas
 
Experience: Intermediate
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: CL,6E
 
gordo's Avatar
 
Posts: 160 since Nov 2009
Thanks: 129 given, 401 received

Turns out my previous post wasn't close...it was dead on exact/duplicate/copy...which means I did it wrong...sigh...

The Heaton-Research users group...gone! The founder decided to work on his PHD. What's up with that???

A son of mine is interested in trading which has rekindled my interest. The older I get the hard it is to concentrate. I am trying to solve problems that are really out of my expertise so I struggle greatly.

I think if I can get my Sine Wave to work, I can understand what is going on with the actual program. Here's a picture of what the program calculates. Now the question is, do I believe it? I know, I know! I have been down this path before...I just am STUCK!!

Back to the TSI indicator. I just can't come up with something that is SOOOO perfect. That is wrong and I know it is...

Attached Thumbnails
Click image for larger version

Name:	Sine.png
Views:	214
Size:	29.1 KB
ID:	211784  
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #180 (permalink)
 NJAMC 
Atkinson, NH USA
 
Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker: NinjaTrader Brokerage
Trading: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,970 since Dec 2010
Thanks: 3,036 given, 2,382 received


Hi @gordo,

Ah, another Machine Learning person out there... Don't find many of those...

What are you using for your data? Is this an RNN system maybe LSTM? I will try to read back in your thread to see if I can figure out what you are doing. Looks like a regression of sorts.

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Visit my futures io Trade Journal Reply With Quote


futures io Trading Community Trading Journals > My journaled journey to success


Last Updated on August 22, 2016


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
     



Copyright © 2022 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada), info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts