STF discretionary spot Forex system development journal - futures io
futures io



STF discretionary spot Forex system development journal


Discussion in Trading Journals

Updated
      Top Posters
    1. looks_one bnichols with 249 posts (152 thanks)
    2. looks_two Big Mike with 17 posts (4 thanks)
    3. looks_3 Adamus with 14 posts (3 thanks)
    4. looks_4 NW27 with 6 posts (0 thanks)
      Best Posters
    1. looks_one sam028 with 4 thanks per post
    2. looks_two bnichols with 0.6 thanks per post
    3. looks_3 Big Mike with 0.2 thanks per post
    4. looks_4 Adamus with 0.2 thanks per post
    1. trending_up 41,538 views
    2. thumb_up 167 thanks given
    3. group 18 followers
    1. forum 292 posts
    2. attach_file 182 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

STF discretionary spot Forex system development journal

(login for full post details)
  #51 (permalink)
 bnichols 
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Should probably post before I'm counted MIA. BTW, the gaps in posts tend to correspond to periods when my thoughts about trading are consolidating, but also in this case as mentioned previously when vacation season is looming and I'm focused on earning the money to pay the cost, which (since investing is my sole income source) includes managing my stock portfolio somewhat more actively, irrelevant to trading STF spot Forex unfortunately. Finally, I've been investing quite a bit of time learning more about the R language and using it to study the statistics behind time series analysis, forecasting with Arima, Garch, and neural models applied to IMFs (Intrinsic Mode Functions) in particular. While so far results seem encouraging I hesitate to report until satisfied the techniques have some merit. **

At some point I'll do something with results so far from K-means clustering as a means of preprocessing feature vectors to feed the bot. In general the stumbling block remains that while entries (ideally at the exact point trend changes direction) do cluster, in practice these entries are born of conditions associated with the trend that is coming to an end. In other words, the clusters themselves are not sufficient to generate signals, and still plan to derive heuristics from a trading system (in this case BB) for the fuzzy inference engine that will process the clusters. As it sits a learning methodology for the bot is still up in the air.

Recently I reviewed the material I bought from Barry Burns with the intention of coming up with a better regime change policy (how to deal with the transition from trend to range trading and vice versa), and happily found what I was looking for. Hard to say why it wasn't obvious from day 1 (essentially, pay extremely close attention to momentum, that when wave counting "1,2,3..." is not the same as "A,B,C,...", especially stay immune to the associated head fakes). Overall the exercise reminded me that when learning how to trade (and probably more generally) we fool ourselves if we think we understand a concept the first time we encounter it, since in reality we learn iteratively, don't grasp vital nuances and subtleties until we're ready, have done the required ground work (which in turn calls to mind all those biblical references to seed falling on stony ground, pearls before swine, etc., etc. ) In other words, one thing trading has taught me is that if a system does not work for me out of the box I should not be too quick to blame the system.

Some basic trading rules the trip down memory lane (review of BB's material) reinforced for me include the following:

1. S/R levels are not cast in stone; they are certainly not "where price is going to change direction", but merely battle grounds where bulls & bears have at it. We can imagine ourselves to be military generals if it helps, whose tactics depend on how the battle for a particular hill is going but whose strategy for winning the war does not change imid-battle. From another perspective, placing an order before it's clear who the victor is (i.e., at market) is akin to betting on a horse race--certainly not trading (which, when I'm doing it right, is more like placing a bet once it's clear which horse is going to win barring him stepping in an unseen gopher hole )

2. Never place market orders, but instead use stop orders a couple of pips above/below ultra short term resistance/support, with momentum in our favour and timed so that stochastics rising/falling respectively, preferably from oversold/overbought levels.

3. Strictly speaking breakouts are low probability trades according to my understanding of BB's method. If it's going to break out we've already been in the trade for a while, starting to take profits just as others are piling in.

4. Job 1 is capital preservation, therefore (apologies to Desiderata) go placidly amid the noise & haste & missed opportunities and don't freak out when an initial stop is hit or whine when a trade exits at a trailing stop. Instead be glad you weren't on the wrong side of that 50 pip break to the upside from a dead stop--no doubt the universe is unfolding as it should. Rather, take comfort in your profits when they accrue, rejoice when a well executed trade consistent with your method captures in excess of 2 thirds of the favourable excursion, because when it starts to occur regularly it may no longer be dumb luck but a sign at long last all the hard work and strict self discipline are starting to pay off. Above all, be at peace with Momentum, whatever you conceive it to be.

5. Trade from one chart (probably the mid time frame chart), guided by the longer time frame chart but entering (and placing initial stop) from the shorter time frame chart.

6. Never trade against what the system (hence price) is telling you (e.g., thinking "this time it's wrong"). The system is never wrong, but until you've mastered it you are prone to misinterpreting what it is saying.

7. Job 1 is capital preservation (needs repeating), therefore take some profits at intermediate S/R levels no matter how close to your entry. If you blow the wad on nearby targets often enough, rethink your notion of what constitutes a setup, or learn how to get back in the game rapidly while the getting is good (which style if it becomes habit means commission costs are going to rise as a % of ROI). IMO the only reason to buy or sell into resistance/support respectively is because you're confident it will not hold, the potential reward if S/R breaks justifies the risk and you want a buffer between your entry and ensuing price action. More often than not however, you probably want S/R at your back.

8. You can't get blood from a stone. No trend and no momentum = don't trade, especially don't imagine our trade will cause the market to snap out of the doldrums. While I might hallucinate that my opinion matters as much as a national bank the market doesn't even know I exist, doesn't care, does not notice me stuck to the bottom of its shoe. If we want to bet on what on the face of it appears to be a coin toss (the "no-current-price-action-but-who-knows-what-might-happen-next" system), we ought to be clear how random entry trading systems work.

9. Practice, practice, practice in all sessions, and between sessions, until you are familiar with the tricks and habits used by your competitors, the effect of news, how price behaves in light of give market sentiment, and so on.

** For thrill seekers I've attached the most recent beta version of the strategy I use to extract and write the currently defined feature vector to disk (WriteZigZagRecord.cs) and the R language script file used to create experimental Arima models from EMD IMFs of daily stock quotes downloaded from Stockwatch.com (fstock.txt).

Notes:
1. fstock.txt should be renamed fstock.R (futures.io (formerly BMT) won't attach file types it doesn't recognize) and you should have or acquire a working knowledge of the R language to use it, including how to acquire the necessary packages (e.g., EMD, forecast, etc.)
2. WriteZigZagRecord.cs can be used to dump feature vectors in BackTest mode ("Use Historical" set to TRUE) or as a strategy in real time. I use the real time option when running experimental price prediction apps in parallel with NT, that detect that the feature file has been updated, load the latest vector and output an updated prediction.
3. I'm working on an OHLC neural net based one-step predictor in Visual C# and will post the VS project once it's more or less stable, whether or not it produces useful results, for comic relief.

Attached Files
Register to download File Type: cs WriteZigZagRecord.cs (20.3 KB, 14 views)
Register to download File Type: txt fstock.txt (5.2 KB, 15 views)
Visit my futures io Trade Journal Started this thread Reply With Quote

Can you help answer these questions
from other members on futures io?
Can anyone help convert this pine script to EasyLanguage?
EasyLanguage Programming
Futures calendar spreads with TS
TradeStation
Looking for combined bar type (UniRenko with HeikinAshi) NT8
NinjaTrader
Bookmap replay files
Platforms and Indicators
NFP
TradeStation
 
 
(login for full post details)
  #52 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 50,138 since Jun 2009
Thanks: 32,638 given, 98,939 received

You can now attach .R files directly.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to Big Mike for this post:
 
(login for full post details)
  #53 (permalink)
 bnichols 
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Many thanks Mike. If anyone has questions about the R code or the strategy I'll be happy to clarify.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #54 (permalink)
 bnichols 
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Preamble

The OHLC one step predictor is more or less stable so will publish the project Rev 0.9 (experimental), no restrictions on use, as is, no warranties.

This program is a precursor result and stems from the main system under development, which will probably resemble ANFIS when done (see e.g., https://ece.ut.ac.ir/Classpages/S86/ECE406/Papers/ANFIS.pdf)

Code is single threaded--doesn't put that much of a load on the computer--but may multi-thread it in the future.

If you're still on dial up sorry for the number of pictures.

Performance is not bad. See Item 8 under Use, Fig 15, below for P&L from a simple next-day-buy/sell strategy based on predictions of the trained network applied to novel test data (data the network has never seen before)

Project files

The system was written in C# using VS 2010 Ultimate and .NET 4. To modify you should employ something similar, or dissect the project to extract what you need.

I've attached 2 ZIP files containing the Project itself and the Install (published) version for anyone who wants to experiment without using an SDK.

ETA: Also attached data file used in the example below (daily OHLC data for TCK.B on the TSX)

Required 3rd party (Open Source) DLLs (Zedgraph & AForge) are embedded in the distribution, but the DLLs and the entire development environments for these packages can be downloaded from the respective sources if desired.

Please let me know if you have issues with either file. My development system is virus free as far as I know but as with any object downloaded from the Web you might want to scan before use.

Fig 1. Opening screen:






Known Bug/Clarity fix list
1. Result files are saved with double extension (i.e., " filename.csv.csv"). This does not appear to affect programs that need to open the file (e.g., Open Office Calc)
2. No header row is written to saved results (e.g., from the Test tab, where Test vectors are saved along with Target and Predicted parameter values)--may have to guess what column is what for now.
2. On Train tab, "Input Vector" label should probably be "Training Vector" and training set should probably load when the tab is opened rather than after the "Start Training" button is clicked.
3. Some buttons need to be disabled (and perhaps enabled) more appropriately
4. Not sure if it's possible to get into trouble clicking buttons out of sequence--needs more error checking.



Wish list:
1. Graph of training error in real time.
2. Presently does 1-step prediction of one parameter (e.g., predicts close from prior day(s) OHLC data). Should have easy way to predict & combine predictions for all 4 parameters to experiment with multi-step forecasts.
3. More tooltips for new users




Use

1. Run the program and choose your data file (data must be comma-separated; e.g., .CSV format).
.......1.1 file format is as follows:
..............1.1.1 First line is header row.
..............1.1.2 Columns must include Open, High, Low, Close data column, but header label not important
..............1.1.3 Subsequent lines contain comma-separated values
Example:

Quoting 
<ticker>,<date>,<exchange>,<open>,<high>,<low>,<close>,<change>,<vol>,<trades>
ABX,19860902,T,0,18.88,18.5,18.75,0,131840,0
ABX,19860903,T,0,19.25,18.75,19.13,0,147309,0
ABX,19860904,T,0,19.13,18.63,18.88,0,152850,0
ABX,19860905,T,0,19.38,18.75,18.88,0,165012,0

.......1.2 Loading data is a 2-step process
...............1.2.1 Select the file via the File > Load OHLC Data File
...............1.2.2 Tell program which columns correspond to Open, High, Low and Close if necessary (default is Stockwatch.com format)
...............1.2.3 Click the "Load Data" button

Fig 2. File scanned. Identify OHLC columns by modifying dropdown box selection :


Fig 3. Data Loaded (after satisfied OHLC columns are correct) & "Load Data" button clicked








2. Select Training set
........2.1 Left click mouse on Data list and drag to highlight data. Right click on highlighted data to open context menu.
................2.1.1 Note that data is moved--not copied--from original data set to target set (can be put back; see Item 4 and Fig. 8, below).

Fig 4. Select training set data by dragging mouse, right click for popup context menu


Fig 5. Training set updated after "Add to training set" selected











3. Select Test set
........3.1 Left click mouse on Data list and drag to highlight data. Right click on highlighted data to open context menu.

Fig 6. Select test set data by dragging mouse, right click for popup context menu


Fig 7. Test set updated after "Add to test set" selected








4. Restore Data if necessary
........3.1 Move data between sets by highlighting items and left clicking mouse on either Training or Test sets
................3.1.1 Note that data is moved--not copied--between sets

Fig 8. For example, move training back to Input Data set or to Test set








5. Network Architecture
........5.1 Click on Network Tab after data sets configured.
...............5.1 Probably no need to change anything here until you're ready to experiment.
...............5.2 "Lookback" is number of prior OHLC samples to use in the prediction of the selected parameter, where the predicted parameter is one of next period Open, High, Low or Close--default is use only prior day OHLC values, maximum 10 IIRC.
...............5.3 "Load Previously Saved Network" applies if you've saved a network after a previous training run (see Training, below). This lets you resume training from a previous state, or run new Test data with a previously trained net.

Fig 9. Network configuration





6. Training
........6.1 Click on Train Tab after inspecting Network architecture
...............6.1 Set number of iterations if desired. 10,000 is probably too low for most applications.
...............6.2 Iterations can be updated at any point (even while training is occurring) by changing the number and clicking the "Update Interations" button.
........6.2 Click the "Start Training" button to get the show on the road.


Fig 10. Training tab



Fig 11. Training underway (after clicking "Start Training" button




7. Test
........7.1 Click on Test Tab at any point after Training has started (or is complete)
...............7.1 Click the "Start Test" button to see how Training is going (has gone)
...............7.2 When watching individual predicted value change in the Prediction list, scroll to and left-click on the corresponding item in the Test vector list or Target value list to sync position of all 3 lists to that line (otherwise Predicted list will jump to first item every time list is refreshed--every 500 iterations)
...............7.3 Zoom on graph by right-click and drag. Left click to unzoom (select Unzoom from popup menu)

Fig 12. Test run on test data set


Fig 13. Close up of Actual vs Predicted test values graph


Fig 14. "Closer up" of Actual vs Predicted test values graph






8. Save results if desired.
.........8.1 Clicking "Save Vectors" where available on a tab will write the data displayed in the listboxes to a .CSV file of your choosing.
.........8.2 Examine the saved data in a spreadsheet if desired
.........8.3 Present version doesn't write a header row to identify columns--sorry for any head scratching that may cause at this point
.........8.4 For example, Fig 15 shows the result of a "buy/sell at the open, sell/buy and the close if next day prediction is up/down over today's close" strategy done in Open Office calc using results of the test run. Don't often see a Cumulative P&L curve like that and much more work needs to be done to determine what it means.

Fig 15. Cumulative P&L for strat driven by 1-step prediction of trained network and applied to novel test data

Attached Files
Register to download File Type: zip TDOHLCProject.zip (1.11 MB, 32 views)
Register to download File Type: zip SimpleOHLCInstall.zip (518.2 KB, 29 views)
Register to download File Type: zip TCKB20120601.zip (32.1 KB, 29 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #55 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 50,138 since Jun 2009
Thanks: 32,638 given, 98,939 received

@bnichols, quite an achievement, thank you for sharing!

What are your thoughts as for adapting this for use with real-time data? Is that ultimately your goal, or are you looking for other types of uses?

I'm wondering what led you to develop this NN predictor?

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #56 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 50,138 since Jun 2009
Thanks: 32,638 given, 98,939 received

Also I really think you should post that as its own thread, Neural Network OHLC Predictor, or etc, so it can be easily found in the future

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #57 (permalink)
 bnichols 
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Big Mike View Post
@bnichols, quite an achievement, thank you for sharing!

What are your thoughts as for adapting this for use with real-time data? Is that ultimately your goal, or are you looking for other types of uses?

I'm wondering what led you to develop this NN predictor?

Mike

As you surmise real time is a goal, and that's just a skip and a jump from here since I've interfaced trained nets with NT in the past--should probably post that interface as well.

The issue for me right now is whether real time (i.e., short time frame) OHLC data has different dynamics than daily data--others may have, but I've never quantified it (i.e., is what I perceive as noise actually data in another system of coordinates). While the issue as a trader is whether a net makes money, and the acid test is applying a given net to STF data and watching what happens, if it doesn't work I like to know why since often enough the fix is right under my nose. Hate to miss it. To date I've never built a net that handled STF price well enough to rely upon, but until now I've never had much time to devote to it. Never say die .

Regarding why the NN predictor, it's a test bed for ideas to be incorporated into a somewhat more complicated hybrid system still under development. I find daily data easier to deal with than 200 Tick, say, partly because the pace is slower, have all night to wonder about some development. I'd say I'm more into comfort than speed but more likely I simply don't think that fast :-/

ETA: Should add, wanted a decent NN predictor because anything is better than the Arima based system I'm using for swing trading the stock market, since non-stationary variance is a headache. Garch would likely improve things but I'm stronger in AI than traditional statistics.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #58 (permalink)
 bnichols 
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Big Mike View Post
Also I really think you should post that as its own thread, Neural Network OHLC Predictor, or etc, so it can be easily found in the future

Mike


Roger--in Journals or elsewhere? An AI forum here? There is an Encog (AI related) thread under NT programming, but this is only marginally related.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #59 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 50,138 since Jun 2009
Thanks: 32,638 given, 98,939 received


bnichols View Post
Roger--in Journals or elsewhere? An AI forum here? There is an Encog (AI related) thread under NT programming, but this is only marginally related.

Not journals - otherwise it is really up to you, put where you think it goes best, but I think it deserves its own thread.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #60 (permalink)
 bnichols 
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received



Big Mike View Post
Not journals - otherwise it is really up to you, put where you think it goes best, but I think it deserves its own thread.

Mike

Alrighty--will create a new thread under Platforms & Indicators, something like AI Platforms, if that's ok.

Visit my futures io Trade Journal Started this thread Reply With Quote


futures io Trading Community Trading Journals > STF discretionary spot Forex system development journal


Last Updated on May 7, 2013


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada), info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts