My journaled journey to success - Trading Journals | futures.io
futures io futures trading


My journaled journey to success
Started: by gordo Views / Replies:25,151 / 183
Last Reply: Attachments:199

Welcome to futures io.

Welcome, Guest!

This forum was established to help traders (especially futures traders) by openly sharing indicators, strategies, methods, trading journals and discussing the psychology of trading.

We are fundamentally different than most other trading forums:
  • We work extremely hard to keep things positive on our forums.
  • We do not tolerate rude behavior, trolling, or vendor advertising in posts.
  • We firmly believe in openness and encourage sharing. The holy grail is within you, it is not something tangible you can download.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.


You'll need to register in order to view the content of the threads and start contributing to our community. It's free and simple, and we will never resell your private information.

-- Big Mike

Reply
 199  
 
Thread Tools Search this Thread
 

My journaled journey to success

  #171 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 45,804 since Jun 2009
Thanks: 29,097 given, 81,389 received


gordo View Post
So things are looking better. I have managed to build an indicator that appears to be predicting the direction of an indicator. I have abandoned trying to make the MACD work and am using the TSI indicator which is based on the work of William Blau in his book, "Momentum, Direction and Divergence" (1995). The reason is a matter of expediency. AI works on normalized data with a range of -1 to +1. The conversion back and forth from normalized to de-normalized data requires the operator to know the min and max of the infeed data. TSI has a max of 100 and min of -100. [The truth of the matter is that if you look at the formula, it only has that because the result is multiplied times 100. So in fact, I could change the formula and not even use the normalization/de-normalization routines.] MACD does not have such limits so the values can be anything and thus makes predictions much harder because you need more infeed data.

Anyway, I took some one-minute tick data, converted the data into TSI values. Ran the artificial intelligence program and got trained data. I plugged the trained data into the Ninja indicator and got a plot. If you look at the first attachment you can see the two values. The top being the actual TSI data and the bottom being the predicted data. The second shows the two in the same panel. What I am looking for is for the predictor (ie the black line) to preceed the green line in changes to TSI. That is I want the AI program to tell me which direction TSI is going to go. The results are VERY hopeful.

I think I am cooking with gasoline now!

Is this a one bar in advance model? To me the indicators look like they align on time, meaning I am not seeing the prediction. Maybe you could post another screenshot that shows the actual alignment, or I have misunderstood.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #172 (permalink)
Arb-Trader
Dallas, TX
 
Futures Experience: Intermediate
Platform: NT, TS
Broker/Data: IB, Oanda
Favorite Futures: YM/EURUSD
 
Posts: 274 since May 2010
Thanks: 107 given, 147 received

Gordo

Nice to see you posting again.

Reply With Quote
 
  #173 (permalink)
Elite Member
NC, USA
 
Futures Experience: None
Platform: None Yet
Favorite Futures: Guitar
 
MrYou's Avatar
 
Posts: 403 since Jun 2011
Thanks: 618 given, 192 received


@gordo

Any update?

Reply With Quote
 
  #174 (permalink)
Elite Member
Puyallup, Washington State
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: NINJATRADER/CQG
Favorite Futures: NQ, CL, ES, YM
 
echoeversky's Avatar
 
Posts: 115 since Sep 2009
Thanks: 112 given, 47 received


gordo View Post
So things are looking better. I have managed to build an indicator that appears to be predicting the direction of an indicator. .... The results are VERY hopeful.

I think I am cooking with gasoline now!


It's been a month.. how goes?

Reply With Quote
 
  #175 (permalink)
Elite Member
Atkinson, NH USA
 
Futures Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker/Data: NinjaTrader Brokerage
Favorite Futures: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,890 since Dec 2010
Thanks: 2,944 given, 2,246 received

Futures Edge on FIO

What value do you place on the webinars on FIO?

 

gordo View Post
So things are looking better. I have managed to build an indicator that appears to be predicting the direction of an indicator. I have abandoned trying to make the MACD work and am using the TSI indicator which is based on the work of William Blau in his book, "Momentum, Direction and Divergence" (1995). The reason is a matter of expediency. AI works on normalized data with a range of -1 to +1. The conversion back and forth from normalized to de-normalized data requires the operator to know the min and max of the infeed data. TSI has a max of 100 and min of -100. [The truth of the matter is that if you look at the formula, it only has that because the result is multiplied times 100. So in fact, I could change the formula and not even use the normalization/de-normalization routines.] MACD does not have such limits so the values can be anything and thus makes predictions much harder because you need more infeed data.

Anyway, I took some one-minute tick data, converted the data into TSI values. Ran the artificial intelligence program and got trained data. I plugged the trained data into the Ninja indicator and got a plot. If you look at the first attachment you can see the two values. The top being the actual TSI data and the bottom being the predicted data. The second shows the two in the same panel. What I am looking for is for the predictor (ie the black line) to preceed the green line in changes to TSI. That is I want the AI program to tell me which direction TSI is going to go. The results are VERY hopeful.

I think I am cooking with gasoline now!

Hi @gordo,

Great to see others looking into ML for trading solutions! I haven't read your entire thread but this last post I have a few questions. It looks like you are saying your plots are made with In-Sample "training data". Is this the case? If so, it does look like your solution is starting to converge. Did you try your indicator with out of sample data? If so, how does it look?

Was this a Neural Network? If so, you might be a little under fit on your solution. You are very close to matching the desired output. You might add one or two more nodes as it looks like your topology is close. If you have a hidden layer, I would add it there first to see if you get a little better match. I would not shoot for perfect as you are likely over-fitting the data and it will not generalize so out of sample data will not work well.

Keep going!



Keep up the ML work!

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Reply With Quote
 
  #176 (permalink)
Elite Member
Atkinson, NH USA
 
Futures Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker/Data: NinjaTrader Brokerage
Favorite Futures: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,890 since Dec 2010
Thanks: 2,944 given, 2,246 received


gordo View Post
Was less than optimistic last night with my SunSpot example. Got to thinking about the data. That data is not like trading data. Trading data flows. It can't go from 150 to 20. It has to move from 150 to 20. So I did some testing. First I started out with an easy one...a sine wave. The results were not too bad.

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).


The problem is this data is predictable. So I ran another model. I used MACD data I had collected. Results this time were pretty interesting.

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).



Now that DOES look interesting. Here's the problem. This data is from a workbench modeling program. Can I do the same thing with the C# program I was working with so many months ago? Time to find out. Dust off the old program and see if I can run the same data and get similar results. Then I have to convert that program to a DLL to run in Ninja Trader. Possible? Maybe. At least I am going to give it a whirl.

My first attempt at making my C# program learn the MACD data did not turn out as I would have hoped. I may need to do some more work here!

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).



As I reflect on where I am right now, I was close before but didn't realize I was close. Let's see if I can get better results.

G

Hi @gordo,

Sorry I didn't see your thread before. I am not sure if it is help to you now but I would say your 2nd chart is a classic "Over-Fit" sample. The "solution" to the problem has way to many dimensions than needed to solve the problem so you started to fit NOISE.

The 3rd chart is a classic "Under-fit" example. The "solution" your NN or whatever assumed the solutions would look like did not have enough degrees of freedom. You need to find a "solution" between these two examples. Less complex solution than your first (or add dampening) and more than your 2nd example.

If you are interested, you can find more information about Over/Under fit here:
https://futures.io/elite-circle/23861-machine-learning-ai-discussion-generic.html

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Reply With Quote
The following user says Thank You to NJAMC for this post:
 
  #177 (permalink)
Elite Member
Tonganoxie, Kansas
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: CL,6E
 
gordo's Avatar
 
Posts: 160 since Nov 2009
Thanks: 129 given, 398 received

Progress Update

Used my TSI data. Have the program running in C# now.

Where have I been? Lost in life.....sigh.....Loose heart on this project when I fail or can't figure things out. Haven't found a forum or a group of link minded folks to work with. I'm just using this as a journal. Big Mike's folks are GREAT...no SUPER GREAT when it comes to Ninja and indicators and just general slap on the back "Go get em" support....not so much with C# programming and the AI program in C# when I run into a wall. The Heaton Research site is okay, but the people there don't want to work the problem, they are just interested in the quick free lunch. A person gets spoiled when they spend time on Big Mike's.....too many good folks in one family.....I trudge on.

So I am trying to refind myself. Dusted off the C# program. Re-learned some of what I forgot...the rest....GONE! Now to get this program into an indicator in Ninja.

Here is the TSI run. Data looks pretty close to the eyeball.

Attached Thumbnails
My journaled journey to success-tsi-c-.png  

Last edited by gordo; October 6th, 2014 at 10:01 PM.
Reply With Quote
The following user says Thank You to gordo for this post:
 
  #178 (permalink)
Trading Apprentice
London England
 
Futures Experience: Beginner
Platform: ProRealTime
Favorite Futures: Forex
 
Posts: 2 since Oct 2014
Thanks: 1 given, 1 received

Great Journal

Hi Gordo

I hope the trading is going well. This is just a quick note to say thanks for posting a journal. I'm currently on Page 6 and I am finding it very insightful. I can relate to so many of the ups and downs (and early discipline problems, which you seem to have sorted out. I'm still working on that).

Great job. Good luck for the future.

C

Reply With Quote
The following user says Thank You to Ringspun for this post:
 
  #179 (permalink)
Elite Member
Tonganoxie, Kansas
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: CL,6E
 
gordo's Avatar
 
Posts: 160 since Nov 2009
Thanks: 129 given, 398 received

Stuck in Neverland

Turns out my previous post wasn't close...it was dead on exact/duplicate/copy...which means I did it wrong...sigh...

The Heaton-Research users group...gone! The founder decided to work on his PHD. What's up with that???

A son of mine is interested in trading which has rekindled my interest. The older I get the hard it is to concentrate. I am trying to solve problems that are really out of my expertise so I struggle greatly.

I think if I can get my Sine Wave to work, I can understand what is going on with the actual program. Here's a picture of what the program calculates. Now the question is, do I believe it? I know, I know! I have been down this path before...I just am STUCK!!

Back to the TSI indicator. I just can't come up with something that is SOOOO perfect. That is wrong and I know it is...

Attached Thumbnails
My journaled journey to success-sine.png  
Reply With Quote
 
  #180 (permalink)
Elite Member
Atkinson, NH USA
 
Futures Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker/Data: NinjaTrader Brokerage
Favorite Futures: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,890 since Dec 2010
Thanks: 2,944 given, 2,246 received


Hi @gordo,

Ah, another Machine Learning person out there... Don't find many of those...

What are you using for your data? Is this an RNN system maybe LSTM? I will try to read back in your thread to see if I can figure out what you are doing. Looks like a regression of sorts.

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Reply With Quote

Reply



futures io > > > My journaled journey to success

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)
 

Spring Grains Outlook w/Sean Lusk @ Walsh Trading

Elite only

FIO Video Journal Challenge featuring NinjaTrader ($2,000+ of prizes)

April

Process above all else w/Anthony Crudele @ Futures Radio Show

Elite only

Machine Learning - Quantitative Trading w/Martin Froehler @ Quantiacs

Elite only

Market Dynamics w/Peter Davies @ Jigsaw Trading

Elite only

Ask Me Anything w/Patrick Rooney @ Trading Technologies

Apr 18

Ask Me Anything w/FuturesTrader71

Apr 19

Machine Learning w/Kris Longmore

Elite only

Market Analysis w/Dave Forss

Apr 25

Introducing iSystems with Stage 5 Trading

Apr 27
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Success and Opportunities wh Psychology and Money Management 9 February 16th, 2016 05:18 PM
Bizman70 The Journey to success of Trading Live bizman70 Elite Trading Journals 88 September 15th, 2014 09:22 AM
Meander System - any success stories? drolles Elite Automated Trading 6 April 18th, 2012 10:15 PM
Key To Success George Psychology and Money Management 1 December 27th, 2009 12:12 AM


All times are GMT -4. The time now is 05:36 AM.

no new posts
Page generated 2017-03-29 in 0.16 seconds with 20 queries on phoenix via your IP 54.145.148.103