STF discretionary spot Forex system development journal - futures io
futures io futures trading



STF discretionary spot Forex system development journal


Discussion in Trading Journals

Updated
      Top Posters
    1. looks_one bnichols with 249 posts (152 thanks)
    2. looks_two Big Mike with 17 posts (4 thanks)
    3. looks_3 Adamus with 14 posts (3 thanks)
    4. looks_4 NW27 with 6 posts (0 thanks)
      Best Posters
    1. looks_one sam028 with 4 thanks per post
    2. looks_two bnichols with 0.6 thanks per post
    3. looks_3 Big Mike with 0.2 thanks per post
    4. looks_4 Adamus with 0.2 thanks per post
    1. trending_up 36,600 views
    2. thumb_up 167 thanks given
    3. group 18 followers
    1. forum 292 posts
    2. attach_file 182 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

STF discretionary spot Forex system development journal

(login for full post details)
  #1 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Intro

This journal sets out to share my experiences learning short timeframe discretionary spot Forex.

It was inspired first by the fact that after about 13 weeks of hard work, on paper I appear to be on the verge of consistent profitability trading a single pair (EUR/USD), and therefore sharing results at this point may help others at the same stage (or to reach the same stage) without taking them down too many blind alleys.

It was inspired second in the hope of providing a little entertainment. Hopefully not of the fall-on-my-face slapstick variety, but rather a record of the transition from "the verge of consistent profitability", a forward looking perspective with which most of us are familiar (e.g., early in one's trading career a relatively short-lived delusional state characterized by unusual optimism or euphoria, usually preciptitated by an accidental winning streak), to actual "consistent profitability"--a perspective accessible only to considerable hindsight and a stage considerably harder to achieve since it signals mastery of the target instrument & time frame.

Caveat

No successful system, method or approach that I know of is guaranteed to work except for the trader who developed it. I accept developers may prefer to market a trading system to recover R&D costs and make a living by means other than trading, since trading is relatively hard work. In light of this there are rumoured to be a spectrum of legitimate 3rd party systems & services out there tailored to the size of the knowledgeable investor's / trader's wallet & his or her investment style (as opposed to illegitimate systems geared to the naive trader's gullibility), but I've yet to prove any. In the case of 3rd party systems we purchase & operate ourselves, IMO unless the system comes with an experienced trader telling us which button to press and when to press it (e.g., access to a trading room), or comes with sufficiently agile expert system built in, one still faces the inconvenience of learning how to trade a particular instrument.

Background & Aim

I'm essentially retired, have an undergraduate degree in pure math (that I was told once in Canada qualifies me to sell insurance), a graduate degree in geophysics, 15 years experience with time series analysis and AI methods for data reduction & interpretation and the same amount of time helping run a small business (embedded systems) by managing a team of electronics engineers and programmers.

I've invested for decades, swing & day-traded stocks and auto-traded longish time frame spot currency in recent years.

I became obsessed with short time frame discretionary spot Forex trading a few months ago while trying to improve the efficiency of a Forex swing trading bot, when it became apparent it wasn't as easy as it looked.

In my experience bots that are profitable for longer time frame (1-4 hour bar) currency trading don't optimize for shorter time frames. From my perspective this occurs mainly because signal-to-noise ratio decreases rapidly with decreasing time frame. Bots live & die by the adaptability and hence output consistency of their preprocessors (indicators used to reduce price action to inputs required by the trading algorithm), and at some point the increasing number of statistical outliers makes e.g. particle oscillator-based indicators unusable.

Practically speaking (from my previous currency swing trader point of view), these "outliers" at best correlate with missed opportunities for significant gains, at worst cost us big time, but in any event are a constant drag on profitability.

So far it looks like short time frame currency movement in most circumstances is not amenable to time series analysis--price action does not reduce to tradeable statistics because the population is too small and too badly behaved, and the lag of generated signals is too great.

The approach I've taken therefore is to study the effectiveness of commonly available, relatively stationary indicators of the sort that inform various market profile systems (support/resistance levels, point of control & value areas, contribute to value area breakout/breakdown/reentry strategies, momentum however we choose to define it, and so on ). The idea is make it cheap to run with any instrument--not dependent on the quality of the data--and robust--not dependent on the availability of certain data (i.e., trade info for Forex feeds).

The thesis is that bot development is knowledge engineering--developing an algorithm, essentially a state machine, to embody what an expert knows and to mimic what an expert does with as few unhandled states as possible. Whether we avail ourselves of an expert and try to understand his or her methodology or we become an expert and mine our own experience, IMO bot development via either approach likely involves a similar level of effort and the choice is a matter of what turns our crank.

In this case I chose to try to become the expert and this journal intends to document the process.

The aim is therefore 2-fold: first, to learn how to trade short time frame spot Forex profitably; second, to abstract the process--to derive useful heuristics from whatever turns out to guide decision making.

Approach

The basic approach is predictably simple: read as much as possible and trade as much as possible.

I've made probably 3000+ live trades on paper at all hours over the last 13 weeks, of which NT captured and/or retains only 2418 due to occasional crashes and database corruption.

Trading was limited to 2 related pairs (EUR/CAD & EUR/USD) to avoid distractions. Along the way I wrote a few trivial indicators to clarify aspects of price action but have more or less settled on the following setup:

Charts:
- 4 & 8 BetterRenko
- 5 minute candlestick
- 1 hour candlestick
- 1 day candlestick

Indicators:
- MurreyMath
- Pivots
- PriorDayOHLC
- ShowBidAsk (if I wrote this it was derived from an indicator written by "Ben L.", sbgtrading@yahoo.com)
- VOL (stock volume indicator)

In subsequent posts I plan to share my reading list, discuss choice & use of the charts and indicators and summarize ongoing activities, including screenshots.

Visit my futures io Trade Journal Started this thread Reply With Quote

 
 
(login for full post details)
  #3 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,780 since Jun 2009
Thanks: 32,306 given, 97,567 received




Congrats on your new journal, I look forward to reading it

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #4 (permalink)
California
 
Experience: Intermediate
Platform: FxTrade/Ninjatrader
Broker: Oanda/MBT
Trading: Eur/Usd
 
Trankuility's Avatar
 
Posts: 269 since Jan 2011
Thanks: 54 given, 151 received

Awesome. I too trade spot Eur/Usd on a short time frame, so really look forward to seeing your journal develop.

Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #5 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Intro

Sorry for the delay in updating my journal. Even though I think putting my private trading journal into a form suitable for public consumption is a profitable exercise (it forces me to view my trading activity more objectively), it's turning out to be a challenge. Thinking is time consuming and e.g. illustrations take effort to construct. That said, trust journals get easier to maintain with practice

I'd like to illustrate my comments below about setups, entries & exits with some annotated chart images but that remains to be seen--at this point mainly want to put something at least back into the community. I'll be on vacation the entire month of July and thought I could summarize progress to date, such as it is, before taking off.

The bottom line is I haven't yet settled upon a reliable strategy (one with a “positive expectancy” according to Van Tharp) but after 4000-5000 paper trades results are beginning to be encouraging.

In the news: I started trading real money with some trepidation on 16 June not because I was ready but after the paper trading “experience” stalled (IMO crossed the line from experiment to gambling) and I decided I needed a kick in the pants. Since then I've made 250 real money trades with mixed results and am probably ready to appreciate sim trading again Seriously though, I wanted to test my current skill level under the gun and needed to address fears that were beginning to resurface about the possibility of trading "for real", that I suspect were contributing to the paper gambling. The good news is the foolishness instantly evaporated when I started trading with real money and so far the outcome tends to support the likelihood trading for a living may indeed be viable with continued hard work.


Instrument

I'm still day trading EUR/USD almost exclusively, from an 8 BetterRenko (BR) chart using 4 BR and 1 minute charts to pick entries and 4 hour and daily charts to indicate longer term trends (contemplating a 1 hour chart as well). USD index and Euro futures are always open in the background.

Indicators

I'm using the following combination of indicators, more or less in order of significance to the methodology:

1. Murrey Math (“MM”, updated hourly)
2. Pivots
3. Previous Day OHLC (“PDOHLC”)
4. Regression Channels (the main exception to my rule of depending on static indicators)
5. Bid Volume (plotted by default; would like to develop an indicator for currencies that incorporates both bid & ask volumes as the sum of differences perhaps, since I depend on BR bar volume for signals and feel like I'm seeing only half the picture)
6. Fib methods (on occasion)
7. ADL (mostly because I'm wondering whether e.g. apparent divergences are of any use )
8. 20, 50, 100 & 200 day moving averages & trend lines on daily chart for comparison with analyst comments
9. Daily & 4 hour candlestick analysis (on occasion)
10. Considering popular oscillators (stochastics, MACD) to have some idea what other traders might be looking at


Indicator Rational

The choice of indicators (like my preference for BR bars) stems from my focus on where exactly to enter & exit short time frame trades--more of a “static, vertical perspective”, in terms of what are the current impediments or lack thereof to price movement, than a longer time frame “left to right” perspective (in terms of historical patterns, including trends, informed by moving averages & oscillators that IMO aid setups more broadly by suggesting bias).

In other words this so-called “vertical (or price) perspective” follows from my so far unquantified belief that
1. In a sense time is less important than price movement since P&L is generated directly by up & down price movement whereas IMO time pertains to efficiency (the fact notwithstanding in reality any movement requires some amount of time)
2. currency price movement can be modelled by superimposing a random walk on top of an underlying directional bias over various time scales; and
3. the random component can be significant. See for example, “Random Walks” by John Norstad, 2011, (https://homepage.mac.com/j.norstad/finance/index.html); and,
4. to some extent Fib phenomena might be interpreted as pre-determined impulse response

In this regard I'd like to find time to explore the notion price movement might be better modelled in random walk terms by convolving selected impulse & bias functions with an S/R “vector” in the presence of noise, but unfortunately unless I were able to sell a book filled with such speculation it would not put food on the table

Methodology

While the longer term objective is to develop heuristics for automated trading so far I haven't been able to prove it's possible to trade short time frame EUR/USD in a total vacuum (strictly technically, without any knowledge of currency fundamentals). As a consequence, in addition to mainstream news about circumstances that may bias price movement I monitor Babypips.com for scheduled events and CNBC TV, stock & commodity exchanges for market reaction in real time. I also scan Fxstreet.com periodically for analyst comments and suggested S/R levels.

General

When trading 7-30 pip movements I try to stay 100% focused on price action without becoming obsessed. The door to my home office is closed, email is shut down and the radio & TV are off. If I feel concentration flagging or obsession setting in or lose a couple of trades in a row (the acid test) I either switch to sim trading for a while or take a break.

Also when trading for 7-30 pips I set an initial stop loss based on price pattern but no more than 2% of the account size, and a target 1 or 2 MM levels away unless I'm determined to scalp in a relatively fast moving market. When scalping

1. I don't use stops since they tend to get taken out by noise
2. do use a target to maximize profit (i.e., similarly avoid trailing stops because of noise)
3. keep the mouse over the Close button
4. recognize that since trading without a stop is a VERY BAD IDEA I never scalp unless I'm convinced I'm temporarily at the top of my game.

In general I've found the more compressed the setup (smaller the profit target) the faster losses mount up relative to profits and overall the riskier the trade, hence not an approach I should take for more than a couple of trades at a time before letting the market move without me for a while.

Basic Approach

Before trading

1. study news for any inter-session change in market sentiment
2. check for any scheduled events and note the time (Babypips.com)
3. scan FXstreet.com for comments and any S/R picks & trend lines mentioned, drawing picks & trend lines on the daily chart if significantly different than what's already there.
4. review overnight price action, noting any areas of consolidation or trends. Note price & volume behaviour around breakouts
5. study real-time price action & volume until I have some idea how they are responding to S/R levels in the immediate vicinity of price and what the basic state of the market is (i.e., ranging, trending, impending breakout), perhaps making a few preliminary sim trades to determine whether I'm “getting it”.

Setups

With EUR/USD what I'd call pure “range plays” IMO do not seem to occur. However, price does more or less bounce between S/R levels from time to time (“consolidates”), providing opportunities to earn 7-30 pips / trade during retracements (say) and to set up for breakouts.

I try to look for entries & exits in the vicinity of MM, Pivot, PDOHLC & psychological price levels in the direction of any prevailing trend, with an eye out for temporary S/R levels that seem (rarely) to pop out of nowhere. In other words, based on my own trade statistics I no longer enter a trade between S/R levels that are drawn on the chart and do not jump into “breakouts” I haven't planned, e.g. for fear of missing a big move, since more often than not this results in big losses. I'm extremely wary of psychological price levels whether they print automatically on the chart, appear in analyst comments or not.

Entries & Exits

For entries & exits I monitor time-based (1 minute) charts price action and volume as usual to get some idea of momentum but also for time-based patterns in the vicinity of an S/R level since they can be integrated into (and hence hidden from view) inside a BR bar.

I like BR bars for the way they deconstruct time-based bars, exposing any hidden structure.

I study how bars are being built on BR charts since the dynamics can offer tell-tale signals, in particular

1. whether volume is building significantly for a given BR bar, since this tends to signal a decision is pending either as a breakout through or rejection of the S/R level, or unexpected impediment is arising to continuation of a trade in transit between S/R levels;.
2. whether the tail is above or below the body (there is pressure building but movement is biased downward if the tail is above and upward if the tail is below);
3. whether the tail is short ("starts" mid-bar: doesn't reach one end of the bar or the other). This tends to strengthen the bias toward impending movement in item 2 above, but not always. If the tail is subsequently consumed momentum is failing.

While any moving channel-style indicator based on price variance might serve the same purpose I prefer to monitor price position relative to its linear regression channel on 2 scales (4 & 8 BR). I consider it to be a high probability setup when price, one side of both scale regression envelopes and an S/R level more or less coincide.

The regression channel is pretty much the only dynamic indicator I pay attention to. For me it illustrates how price probability distribution and local trend are bearing down on an S/R level, and often seems to illuminate developing price pattern.

Other

I startled myself by making $1400 in an hour or so on a single successful trade last week and decided to transfer $600 of it from my trading account to my bank account. While this might be unwise from a money management perspective at that point I wanted to get a taste of what profitable trading might feel like. It tastes GOOD

Perhaps needless to say I subsequently lost the $1400 & more, but the memory lingers.

For the record, recent excerpts from my private trading journal


Quoting 
June 17

Transition to Live Trading

Up to the point I went live (started trading real money) I was still capable of blowing up a paper account and this made me hesitant to start. Eventually it became more or less clear I was blowing up accounts because I was "fooling around"-- I had become bored and was treating trading like a game, losing mostly by gambling for the thrill of it rather than losing as part of the learning process.

While I view paper trading as absolutely essential to learn the ropes, or to recover one's resolve after a bad experience trading with real money, the "it's not real so it doesn't matter" factor is always present to some extent, for better or worse.

For those of us who have learned the value of paper trading (e.g., because we've been burned previously) any sense of unreality is more of a comfort than an impediment, whereas for newbies it can be a constant burr under the blanket. In hindsight, if it becomes an issue at any stage IMO that's where we should make the leap to live trading. Just don't burn the bridge back to paper.

Some things I've learned (and relearned, and learned again) about Psychology

Strive for complete objectivity--do not allow yourself to feel smugness or euphoria on the upside, or dread on the downside since these tend to be harbingers of disaster. Emotion is for thrill seekers, a distracting waste of mental energy at best and highly counterproductive when trading.

Keep in the back of your mind that even a trade that is at the moment wildly successful can back up on you at any time, or if in a winning streak you are potentially only 1 trade away from the start of a losing streak; just don't dwell on it and don't panic when it happens.

Focus 100% on preservation of capital first, profits second. In particular don't fall in love with a profit target (don't count your chickens before they're hatched). Until realized it is just a gleam in the eye. Instead (if necessary) focus on the P&L at the trailing stop, which most of the time is where you're going to exit a trend trade. Initially it is what saves you from (too much of) a loss and later will optimize profit when the move (and hence the trade) is over. That said, also practice setting informed targets, since meeting targets tends to maximize profits and builds moral.

Study price movement constantly and adjust trailing stop relative to S/R levels accordingly. If profits are increasing and "likely" to continue increasing advance the stop toward price as warranted without entering the "zone of price variability". In other words avoid the temptation of going for the long bomb when day trading (and probably when swing trading).

Taking a loss

After studying price action for some time before considering a trade, and having identified what appears to be a setup, more often than not these days (particularly when scalping) I will place an order initially without a stop loss or with a stop set to guard against a catastrophic error in judgement (i.e., at 2% of my account). This is not a good thing in a directionless market where expected profits are relatively small since stats are showing trading in a trendless market is unacceptably high risk. When losses start to mount and no later than when you've finally managed blow away the day's gains, STOP TRADING. Can't profit if price is not moving.

Psychological corollary of taking a loss -- profit expectations not met when trailing stop taken out. Don't let it get to you--focus on setting sensible targets.

Remember to breathe.


June 20

At 2:50 AM EST (just prior to London markets opening) USD index is still rising (since 14 June), EURO and CAD dropping, since the USD index is inversely proportional to a weighted sum of currencies including the EURO and CAD. If this keeps up the TSX may open lower later this morning due to lower prices for stocks that are sensitive to commodity price fluctuations. At the moment the main factor seems to be doubts resurfacing that Greece with be able to avoid "restructuring its debt" (euphemism more or less for default), possibly resulting in a "Lehman-style event", with similar nasty consequences for the world economy. Not clear to me why Greece is not simply booted from the Euro Zone so that it can devalue its way out of debt by printing muchas drachmas, except that it might encourage the rest of the PIIGS to take the same escape route.

As a consequence I'm short the EUR/USD a couple of contracts at the moment & may sell a few more if a trend develops (finally in at 1.42365 after some hemming & hawing, ambitious target of 1.3970 in case it melts down to May 23 low this week).

June 21

"Things generally positive this AM, which means I'm being stopped out of short term EUR/USD shorts in advance of the Greek confidence vote tonight after N. American markets close, since the sentiment is the vote will pass (Greek government will receive a mandate to enact further austerity measures and thereby earn a short term bailout).

IMO sentiment is positive mainly because for the financial establishment at least the alternative (collapse of the Greek government, followed by the Euro & eventually the ECB) is unthinkable and investors don't like to dwell on the unthinkable, even to the point of ignoring the likely immediate reaction of anti-austerity protesters camped outside the Greek parliament if it does pass, and the mounting evidence tossing more good money after bad will not stem Greece's descent into bankruptcy.

Edited to add: gold price (like oil) continues to be "up" over yesterday's close 15 minutes before the market opens but this is due primarily to decline in the USD index rather than demand for gold. Since gold is a commodity denominated in USD, when the USD drops relative to other currencies the "price" of gold on N. American markets may appear to rise since it requires more USD to purchase it, but this morning folks are actually selling gold (which, if the USD remained flat say, would tend to drive the price of gold down). See e.g. Gold, USD, Price gold, Silver, US Dollar, Oil, Platinum - Kitco KGX for a little more discussion."

Edited to add: at 5:50 PM EST the Greek vote was finally tallied and the government received its mandate. My EUR/USD buy stop was taken out, putting me long the Euro just as I wanted, whereupon the Euro immediately plunged 75 pips . I expect it to recover, perhaps rise to 1.4500...just a tad volatile right now...that's all...famous last words. Good news is riots did not materialize.

June 22

Waiting for FOMC report at 12:30 PM EST but more importantly Bernanke's take on it in a news conference at 2:15 PM EST, since he will likely hint at the likelihood (yea or nay) of QE3.

While markets are assuming it will be business as usual (e.g., no interest rate rise) the question is how much of that assumption is already baked into stock, commodity & currency prices. If markets receive additional reassurance (as they usual do from FOMC announcements) e.g., stock & commodity prices may rise. However, as I learned the hard way last night wrt the Euro, the old adage "buy on rumour, sell on news" is apparently still alive & kicking :-/ Regarding the Euro, whereas bullish jubilation that the Greek government survived the vote was instantly replaced by bearish doubts about the viability of the Greek economy, in the case of the FOMC report expectations about interest rates (bullish for stocks) may upon news be instantly replaced by bearish fears about the economy (is this a "soft patch" or prelude to a "double dip"), which could drive markets down. In any event remains to be seen.

Edited to add: at 12:30 PM EST report says no change to interest rates, economy is in the toilette but "we're optimistic". Every comment preceded by "However ... ". So far no real movment one way or another in USD so stock markets probably not affected yet but bond prices appear to be rising (driving down yield).

Edited to add: at the close of N. American financial markets there appears to be a delayed reaction to Bernanke's remarks resulting in a rising USD index. This started to drive commodity prices down and hence commodity based stocks on the TSX down with them until trading ended for the day. Currencies move almost 24 hours/day however and if the trend continues the TSX may gap down tomorrow morning. In the meantime will probably remain short EUR/USD until things change.


Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #6 (permalink)
The Netherlands
 
Experience: None
Platform: MultiCharts, TradingView
Trading: ...
 
Jura's Avatar
 
Posts: 774 since Apr 2010
Thanks: 2,347 given, 688 received


bnichols View Post
[...]
For the record, recent excerpts from my private trading journal
[......]

This is thread is looking promising, mainly because of these excerpts from your private trading journal. Thanks for that, it's fascinating to read that stuff.

So, I'm also looking forward to reading this journal.

Regards,

Reply With Quote
 
(login for full post details)
  #7 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Watching EUR/USD shortly after the market opened this evening observed that it broke 4.1600 and seemed determined to stay beneath it.

Based on the short article "Forex: EUR/USD, bears still in control" at FXStreet.com and the fact that last week 4.1600 had proven formidable support decided to sell if it broke through yesterday's close as well, even though my heart wasn't really in it (5 hrs sleep last night & it's late). The opportunity seemed too good to resist in that even if I messed up the entry (which I did) the market would likely forgive, which luckily it did. The entry (1/2 lot x 4) reflects the fact I was aware I wasn't on top of the game but gained confidence as price action continued to consolidate more or less below yesterday's close. 2 lots @ 22 pips real money was a decent wage for 30 minutes work so decided to pack it in for tonight rather than trust the market to forgive me twice.


Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #8 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Yesterday was a write-off except perhaps as a (re)learning experience. I made a few tentative scalps and lost more than I made (ca. ~ $50). The problem was psychological: first, I'm focused on reports that the EUR/USD is highly volatile due to circumstances in both Greece & the US and trading in a range that could bankrupt me (or would have a couple of years ago).

Second, I wanted to short the market yesterday and price refused to cooperate. I know from much bitter experience that even if I managed to consciously accept the fact EUR/USD was going up the knowledge somehow would not filter through to my "trading brain" and I'd likely stumble somewhere, so I packed it in early.

Today was different in that I managed to overcome my fear of market volatility (that seems to translate into a fear of market heights) long enough to make one long EUR/USD trade.

Setup

Overnight & just prior to taking the trade I talked myself into believing the Euro was likely going to rise when New York came online, given the following factors

1. There is yet another Greek vote on Wednesday and if folks were going to "buy on rumour" yet again then I would too (given that last time I "bought on news" I got creamed).

2. Comments on FXStreet proposed a selection of what I thought were realistic S/R levels long & short

3. US stock market futures were up and continued to rise into the open.

By 10 AM AST, half an hour before the NYSE opened it looked like price was consolidating around MM 2/8 and therefore planned to go long from there.

Entry

The entry was a bit of a fumble (see charts below).

I notice my heart still starts to pound in the moments prior to entering a real money trade and have to remind myself to breathe--in through the nose, out through the mouth...repeat... :-/

I try to enter on a retrace with 1/2 contract. Once it looks like price has retraced as far as it's going to go I tend to top up to 2 contracts. Today I topped up with 1 contract, for a total of 1.5 because I wasn't feeling that cold hard sensation of certainty (for lack of a better way to describe it) I think I probably should be feeling--a feeling I noticed for the first time when starting to trade real money last week.

Entry went as follows:

1. initial entry before the NY markets opened around 10:05 (overconfident 2 x 1/2 lot) at MM level 2/8. Price hit MM 3/8 (15 pips) before backing up over top of me and I closed the trade more or less at break-even.

2. Stupidly I hit the "Buy" button again at 10:20 AST with 1/2 contract at the same point I closed the previous trade, before determining whether 2/8 support was going to hold and sweated bricks as price sank, until the N Am stock markets opened at 10:30 AST. I suppose I thought it was going to be my 1 trade of the day and therefore if it weren't going to work out better find out ASAP, or something like that.

3. Given positive market open and general exuberance on CNBC hit the Buy button again on a full contract and decided to stick with 1.5 contracts "until further notice", which turned out to be the entire trade.

At this point I still didn't have a stop or a target.

Maintaining the trade

This trade reminded me that while I am intimately familiar with emotions trading evokes and may be somewhat better equipped to rein them in before they get out of hand these days, I still feel them.

I talked to myself and to the monitor constantly in the 1st 20 minutes while price rose to MM 3/8 and then started to retrace just as before. While price retraced I put a stop just under MM 2/8 (just under break-even). Despite the fact I've read one should not dwell on the $$$ I still do. I watched the trade claw its way up 25 pips to $375 and then back down to $120, and it was tough to place a stop below my entry knowing I was essentially deciding to throw away a sure 100 bucks on no more than a gut feeling.

Things got better after that.

Even before price reached Pivot R1, based on the general enthusiasm with which it was rising I finally placed a target at MM 8/8 (psych level 1.4400) and just below Pivot R2, which immediately made me think "pigs get slaughtered", but assuming I'd choose the exit by carefully managing the stop.

To make an hour long story short, as price advanced past each MM price level and began to approach the next I moved the stop up to just below the bottom MM level.

Exit

There were 2 occasions when I moved the stop to a price that was not just below a MM level: first when it approach MM 5/8 after clearing Pivot R1 at 1.4352 (placed stop at psych level 1.4350); and second after profit exceeded $1000 near the end of the trade--I set it just above MM 6/8 to guarantee $1050 after fees simply because I wanted to be able to tell my wife "I made over $1000 today"

In my defense the trade was getting a little long in the tooth, beginning to retrace from MM 7/8 before taking another run at it--but at the end of the day I didn't want to risk bragging rights. Even though when setting up I'd already decided to stop out rather than target out, not the most disciplined way to plan an exit :-/

It turns out price failed to breach MM 7/8 so feel somewhat vindicated, even though thinking a little more objectively I probably would have lowered the target from MM 8/8 to MM 7/8 to squeeze another 13 pips out of the trade rather than stop out unnecessarily.

Summary

The moral for this week's trading so far is we might do what we can to identify a setup, and may try hard not to botch execution too much, but the rest is up to the market: even at the top of our game it alone decides whether we make money, or whether we lose, and how much. I don't feel any particular pride or elation being above water so far this week--more just looking forward to vacation & getting away from it for a month.

Once again, smart money management aside, I transferred 1/2 the profits to my bank account, this time not only for the morale boost but partly to avoid the temptation to trade amounts greater than my expertise warrants.

Managing this kind of uncomplicated trend trade probably lends itself to a bot that isn't much more involved than an NT ATM strategy and I'll likely give some thought to implementing it. If so will post it here.

Edited to add: while I expected some sort of sell off when the move was over (in this case turned out to be about 50 pips) I avoid trading the retrace after big moves because

1. I'm not yet able to change mental gears reliably enough to try to capture a retrace successfully (have tried and tend to fail).

2. At this stage taking on another trade after a satisfactory trade completes amounts to "piggish" behaviour apparently & unnecessary risk to capital, since my personal stats show in general the larger the number of trades I undertake the more likely they will fail.


The 2 charts below show the trade (1 minute & 4 BR).

1 Minute chart:



4BR Chart:



Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #9 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Sim traded the entire session today except for one live short that netted $7. I'm not ready to trade high volatility events like the Greek vote live.

On the other hand sim trading netted $1200 when the EUR/USD dipped about 100 pips in 1/2 minute I happened to be short because I was testing the possibility the assumed price already reflected good news & the Euro would subsequently sell off as it did last week, but didn't expect that kind of drama. As it turns out the sell off did not materialize (price eventually rebounded) so decided to stay away until I could figure out what was going on.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #10 (permalink)
London, UK
 
Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker: IB/IQFeed
Trading: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 471 given, 786 received


Vacation for a month? You certainly have something to look forward to

Have you thought about the mental side of this trading process? There is a lot of material about putting your mind into the right frameset to trade. I just read a blog post by Lance Beggs where he talks about just that. Even while in the middle of a trading session, there are thought processes you can employ to bring your mind back into the space where you want it to be - i.e. ready to trade. You just reminded me of that with your comment about entering or not entering the next trade after a successful prior trade.

Also I wanted to say it's interesting to read about your exploits as you are at a point one step ahead of me.

Best of luck and I hope it keeps progressing.

You can discover what your enemy fears most by observing the means he uses to frighten you.
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to Adamus for this post:
 
(login for full post details)
  #11 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Adamus View Post
Vacation for a month? You certainly have something to look forward to

Have you thought about the mental side of this trading process? There is a lot of material about putting your mind into the right frameset to trade. I just read a blog post by Lance Beggs where he talks about just that. Even while in the middle of a trading session, there are thought processes you can employ to bring your mind back into the space where you want it to be - i.e. ready to trade. You just reminded me of that with your comment about entering or not entering the next trade after a successful prior trade.

Also I wanted to say it's interesting to read about your exploits as you are at a point one step ahead of me.

Best of luck and I hope it keeps progressing.

Re vacation, pointing the motorcycle toward Michigan in about an hour

Roger--the mental process (psychology) is #1 issue -- has always been a battle. I've learned not to try to scalp when my head isn't in it, for example, and to stop trading when every fibre in my body wants to short the market & price refuses to cooperate :-/ I should probably take a look at Beggs' book since I miss a lot of good moves because of it.

Yesterday set up a simple NT ATM to trade what seemed to me a directionless market with decent success (~ $400 in 9 or 10 trades), going short near the top of the previous price action and long near the bottom (parameters for entry & trailing stop below--note numbers are in 1/2 pips so divide by 2 to get pips). Might do better with 2 targets at 2 successive MM levels )15 & 30 pips) rather than 1 target at 2 MM levels away (30 pips).

Alrighty--have to run! Good luck with your trading this month. Should mention I found going live this time around was a sobering experience (in a good way). Trade much less & ratio good/bad trades rose significantly from paper.




Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #12 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,780 since Jun 2009
Thanks: 32,306 given, 97,567 received

FYI, your chart images from your external host are not working.



Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #13 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Big Mike View Post
FYI, your chart images from your external host are not working.



Mike

Thanks for the heads up Mike. Images are stored on a server at my house and it looks like my son inadvertently adjusted router settings (he's "looking after things" while I'm on on vacation :-/) Should have things fixed by week end.

Brian

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #14 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Since returning from vacation I've been watching the markets & paper trading almost exclusively (i.e., rather than real money), 10-16 hours/day and often more, occasionally to the point where I'll wake up from a nap in a panic, realizing that the price action I've been trading was imaginary before it sinks in that any orders I placed while asleep were also very likely imaginary (not prone to sleep-trading so far ).

Since returning I also got into the habit of writing long, rambling journal entries that eventually I decide not to post (because they're boring or otherwise imperfect ) :-/ While there might be something worth retrieving in them probably better to start this one online and work up the enthusiasm eventually to just hit the Submit button.

Back to basics -- trend, momentum, cycle, S/R, time frame

The main development is probably that I've gone back to basics, revisiting what I thought I understood about trend, cycle, momentum, S/R, time frame (what some apparently call "fractal" phenomena & relationships) and price action. I found the Top Dog Trading (Barry Burns) course to be helpful in that regard since it leverages very basic ideas into a relatively straight forward trading system. Interestingly--like everything else probably--while beginners can likely profit from the course in the sense of getting a good grounding rather than making money necessarily, it may benefit more advanced traders who've plateaued one way or another. Not a course recommendation, since everyone will have their own way around obstacles--just an observation. There's just something about the way Burns iterates upon the basics that works for me. As a consequence even his tireless self promotion and the endless stream of email spam exhorting me to buy products I already own or don't need doesn't bother me all that much. Yet :-/

The reason for revisiting basics is simply that an intellectual grasp of a thing doesn't necessarily translate into practical (i.e., in this case profitable), gut level or intuitive understanding. For me trading is something like riding a motorcycle on the street, which I've done most of my life. After 30 or 40 years (riding) I'm still learning and still need to return to basics occasionally if I'm going to stay sharp (& stay alive). Not sure why that is.

Things I've relearned that I thought I knew

In any event after watching videos of experienced traders using 5 or 10 different trading methods profitable trading seems to boil down to a few things, but primarily to the fact price has to behave well enough between order entry and target(s), and behave often enough, for a system to profit in the long run--i.e., price has to trend for the duration of the order, even if from a broader perspective the setup was a range trade, say.

Furthermore, unless one is trading individual ticks price will wiggle while it trends between entry and exit, the difference between a "wiggle" and a failed (stopped out) trade more or less related to time frame, quantifiable e.g. perhaps in terms of ATR (average true range) over a specific period. By definition wiggles in a price trend (essentially, advances followed by retraces followed by advances) have smaller range than movements defining the trend--advances exceed retraces. Choice of stop reflects our notion of what constitutes the line between wiggle (retrace) and failure of the setup (failure of price to advance toward the target)

Price is unpredictable no matter how we analyze it, meaning no matter what indicators are telling us or what our expectations, we never know for sure what price is going to do. We can however assign probabilities to price movement. For me probability expresses itself as a gut feeling or growing sense of inevitability that price is favouring one direction over another. This "favouring one direction over another" is of course measurable and quantifiable (i.e., in terms of cycles and basic indicators like RSI & Stochastics) but is not enough (or shouldn't be) to motivate us to press the order button unless it's accompanied by some assurance than price will meet the target (i.e., some assurance of range) and that it will do so "soon enough" (i.e., with some enthusiasm); in other words, we need reassurance price has some momentum in the favoured direction (e.g., indicated by good old MACD, onset of bifurcation at the edge of a market profile value area or cumulative delta if the instrument supports it, or even a tick- or bar-per-second indicator).

Targets intend to capture as much of the favourable excursion as possible over a given cycle, therefore reflect our notion of the point at which price will encounter S/R or otherwise run out of steam (sellers/buyers). The choice is based on whatever method we happen to be using and we can select targets based upon previous turning points, previous period OHLC, Gann lines (Murrey Math), VWAP bands, one market profile attribute or another, etc.

Method and discipline (ability to stick to a plan) may be necessary factors of consistently profitable trading but since "consistently profitable" means profits outweigh losses over the "long term" the implication is that money management is paramount, perhaps even sufficient if we believe theories about systems based on random entries that are discussed here from time to time. After testing the strategy developed by shodson during his last webinar (implements a random entry method due to Tom Basso & Van Tharp), if not a true believer that money management is sufficient to profitable trading I'm convinced it's at least absolutely necessary--that profitability is mostly about managing (potential) loss than maximizing profit--and at this point no stop, including a catastrophic stop, on any trade never risks more than 2% of the account. That said, perhaps like most, when scalping micro ranges I don't use any stop at all :-/

Reflections on system trading

To sum up, following Top Dog's trading guidelines at the moment I enter a trade with no less than 4 positions (trading spot Forex via IB "contracts" don't really exist), more often than not exhausting 2 at the first target and thereby eliminating risk altogether. Until I can better intuit what indicators tell me (and trade price action alone without them) I modified standard MACD, SMA and Stochastics indicators to perform like his proprietary indicators but have not yet mimicked his "wave" counter and may not. I don't think it would be counter to his copyright to attach my modified indicators to the thread and will do so after checking to see if the code needs tidying.

Like any other system the Top Dog approach allows me to accept lost "opportunities" (relatively large price movements that I fail to capitalize on, in retrospect) and failed trades with equanimity, or at least without as much self-flagellation as seat-of-the-pants trading. I mark failed trades as "failed" and both failed and profitable trades as "mistake" if the outcome was probably a consequence of breaking a rule. System trading reinforces the notion one simply has to learn to deal with emotions rather than fight to eliminate them (which may not be possible). Profits are ideally as consistent as the system predicts, the goal in self overcoming to master the system (to learn to stick to it) rather than to overcome one's self in some nebulous philosophical way.

A system provides a framework against which losses can be studied to determine whether we failed to pick up on some nuance (i.e., to test our grasp of the system) and over the long haul whether the system is somehow biased toward a pattern of price action and needs retuning if it is to be truly robust in any time frame or instrument, or (worst case) abandoned when price is not behaving.

Similarly a system is a framework against which profits can be measured to determine e.g. if tuning could conceivably improve return, given there can be a fine line between tuning a robust system and curve fitting. We know an otherwise reliable system has been wound a little too tight when it starts to cycle between periods of significant profits followed by periods of significant losses.

A system removes doubt by dictating the chart configuration & setups & provides a means for quantifying both the probability a trade will succeed and the potential yield but most importantly constrains worst case loss (slippage aside). This is helpful for those of us who still tend to agonize about the meaning of "cut losses early" and "let profits run" every time we enter a trade. Profits & losses are simply products of the system model, predictable until verified otherwise, and can be easily tailored to a given money management scheme assuming account balance meets minimum criteria.

Finally, a system helps us learn the difference between the emotion of confidence (or lack thereof) and the experience of probability (high & low) without going bust. For example, a system provides a safety net when perhaps after a streak of eye-popping winners we start to feel we've got it made in the shade (pitcher full of Koolaid), which is the surest sign euphoria has overpowered discipline and blotted out everything we know to be true about price action. We're more likely to snap out of it before we hit our daily loss limit instead of by the time there's no margin left to trade.

ETA: ADDENDUM

I've begun to implement an interpretation of (homage to? Whatever is least likely to be construed as property violation) the Top Dog trading system as a short time frame (200 - 1800 tick bar) automated multi-timeframe strategy, not because there's any reason to think it's better than any other system but because it's what's at hand and I need something for the London spot Euro market. So far any attempt to alter my hours to discretionary trade those tantalizing price movements has had a deleterious effect on quality of life :-/

Seriously, not sure to what extent it can be shared without violating copy- or intellectual property rights at this point but will share what seems fit as it becomes available.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #15 (permalink)
London, UK
 
Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker: IB/IQFeed
Trading: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 471 given, 786 received

I totally agree that you need price to move swiftly from your entry to your target to make your strategy work*. I think anything else would be an admission that you're taking a 50:50 punt. All those indicators you mention - are you making any progress in "internalising" them, for want of a better way of putting it, i.e. I assume you want to see what the indicators see in price action, so you can then delete the indicators.

* [EDIT]: There's a statistic for that called Maximum Adverse Excursion. If you have accumulated a batch of trade data and can generate that, you need to minimise it. Generally speaking you need to maximise your winning percentage and your win/loss size ratio, so hopefully your strategy will have a MAE that correlates negatively to your winning % and your win/loss size ratio anyway.

You can discover what your enemy fears most by observing the means he uses to frighten you.
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #16 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Roger the MAE Adamus. I spent some time a while ago trying to use MAE explicitly as feedback in a neural-net based autotrading (Encog) system ( ), unfortunately my dabbling in esoterica like neural nets is usually a pretty good sign I've taken my eye off the (trading) ball

In the (discretionary) method I'm working with now MAE shouldn't be an issue since stops are usually placed initially either 1 pip past the end of the bar opposite to the entry (which is placed 1 pip + spread past the other end of the bar) or at the "indicator stop"-- just beyond the last retrace, which ought to serve as S/R. In theory I just have to count failed trades to measure efficiency.

I'm not focused on "internalizing" indicators at the moment so much as trying to develop an automated strategy using them. Lord Kelvin said something to the effect we don't understand a thing until we can measure it. In my case (background in numerical modelling) I have to code a thing to make sense of it

At this point the main advantage of relying on indicators, and perhaps the first step in "internalizing" them, is that it forces me to wait until the "stars align" (a setup presents itself, in theory) before looking for an entry, and while waiting I can observe how the indicators reduce price action ("reduce" in the data analysis sense, of reducing raw price action to easier-to-digest information like trend, momentum, cycle, S/R). This circumstance may eventually allow me to dispense with indicators (trade price action alone) if my thought process eventually mimics what indicators are telling me, price action concealed to some extent in the minute details of indicator behaviour. If nothing else fewer entries = fewer mistakes in a given session

Regarding the bot, this time around rather than neural nets (so far) the distraction is fuzzy logic, partly to corral the huge number of parameters I've decided to include to describe the state of trend, cycle, momentum and S/R in 3 time frames (200, 600 and 1800 tick) for spot Euro. The other rationale for appealing to fuzzy logic is that hopefully it will turn out to be a simple and convenient (i.e., formal) way to construct & manipulate trading heuristics. At the moment I'm developing the "fuzzy core" as a DLL and "using" it in the strategy

I also have 6e (Euro futures) with Fat Tails's indicators applied running in the background mostly to observe any correlations--but so far no motivation to turn a multi time frame strategy (already pushing 3500 lines of code with parameter definitions & most of a fuzzy engine) into a multi-instrument strategy. Wisdom (of market profile) teaches us that while we may not be able to predict what price is going to do we can piggy back on the actions of larger participants (whether institutions placing sizable orders for corporate or sovereign clients according to one algorithm or another, that may impart a bias to price movement, or prop shops running stops for fun & profit, which may be responsible e.g. for relatively rapid, parabolic transitions from one area of consolidation to another). I believe it's Fat Tails' view that price movement is the consequence of willful action by one or more large, professional traders who are observing more or less the same features (i.e., value areas, say, rather than SMAs, since everyone uses different SMAs ). If so then no other (arbitrary) collection of time frames & indicators will be as accurate or efficient.

Edited to add: thinking out loud, with the bot one issue I still have to deal with is how to conceive the task overall, once the system has translated measurements into fuzzy numbers. When one is up to one's elbows in variables and code snippets it's a challenge to keep the goal in mind (project management not being my strong suit). Fuzzy systems are usually found in control systems (cameras, washing machines, subway train braking systems, robot/missile guidance systems etc). In each case measurements are converted via heuristics to control signals to steer the system to a pre-established goal whereas I tend to think in terms of modelling a physical process & making predictions about the future by driving the model with real world inputs--not quite the same can of worms, the difference between a prescription for action (control function--e.g., "to maximize profit go long or short so many units right now") and a weather forecast ("it's going to rain" or "price is likely going to rise", a forecast not providing instructions about whether to risk camping, suffer galoshes or place an order, explicit instructions being what computers require to function).

The hope in working with fuzzy values in this application is that associated "crisp" values can be interpreted more or less as probabilities (e.g., that at any given moment a suite of successive S/R levels have a specific probability of being attained and therefore an order with targets spanning these levels has a specific probability of success, the number of units involved constrained by money management policy, not necessarily attempting to model longer term profitability but instead perhaps simply optimizing order quantity for some measure of profitability using familiar techniques). In this case the trigger would be that "probability" exceeds a threshold, in the first instance perhaps quantity proportional simply to probability of success.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #17 (permalink)
London, UK
 
Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker: IB/IQFeed
Trading: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 471 given, 786 received

If you're concentrating on automating a NT strategy to do all that then I can see that you're probably not getting far substituting out your indicators. However I'm probably suffering from the usual price action snobbishness towards indicators but you don't see it as a priority to ditch them. I won't try to persuade you otherwise then.

The stuff with fuzzy logic sounds good and it'll be interesting to see how you develop it. Even though I've done a lot of mechanical trading system development myself, I never touched fuzzy logic - partly because of the technical challenge of implementing but partly because I was worried about the dangers of inadvertantly over-optimising / curve-fitting it. I assume your backtesting.

You can discover what your enemy fears most by observing the means he uses to frighten you.
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to Adamus for this post:
 
(login for full post details)
  #18 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

On the topic of applying fuzzy logic in an NT strategy, after a couple of false starts trying to roll my own core (let's call it "research" ) I decided to go with an available open source library that seems to be a pretty complete and easy to use implementation of a fuzzy core (primatives, membership, fuzzification, inference engine, defuzzification); namely, AForge.Fuzzy available from AForge.NET :: Framework. Since the source for AForge is also available anyone with a C# development environment (preferably MS Visual Studio, since the source comes with MSVS projects already defined) can tweak their own version.

In my experience fuzzy logic may be less prone to curve fitting than other approaches (including other AI techniques) prehaps because of its nature--the machine code equivalent of imprecise linguistics--but that remains to be seen

For the record, to install and experiment with the default AForge.Fuzzy DLL (compiled for Dot Net 2.0, so you might want to recompile for 4.0), as usual

 
Code
1. copy the DLL to the NT ...\bin\custom directory (for Windows 7 & other MS OS typically C:\Users\Owner\Documents\NinjaTrader 7\bin\Custom)
2. good idea to add it as an NT reference by
    2.1 right-clicking anywhere in a code editing window
    2.2 selecting References... at the bottom of the popup window (see 1st figure below)
    2.3 clicking on the Add... button in the resulting References window, which opens the Select Assembly window
    2.4 selecting AForge.Fuzzy.dll from (in this case) its location in the NT ...\bin\Custom directory in the Select Assembly window, followed by clicking the Open button (see 2nd figure below)
    2.5 clicking OK in the References window
3. restart NT to make the new reference available, if "using AForge.Fuzzy" statement in your code results in a "not found" error.
Adding a reference DLL to an NT strat:

Right click in code editor window...



Select DLL & click Open ...

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #19 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

More fuzzy trading concepts.

In order to create membership functions (mappings of raw inputs to fuzzy space) we have to know something about the range of a given input parameter in the "real world". For example, if we want to use e.g. the slope of the 50SMA as a proxy for trend we need to tell the strat what we mean by linguistics like "positive trend" in rules like

"If the trend is positive then favour long entries"
"If the trend is negative then favour short entries"

The first figure below shows a histogram (frequency plot) of the 10-bar slope of the 50SMA for the EUR/USD 200 tick chart over the last month or so, roughly unimodal with standard deviation of 0.3915 (pips per bar). We'll ignore the interesting wiggles along the sides for the time being.

Assuming we want to use a simple triangular membership function (second figure) to map trend observations to fuzzy inputs for the inference engine we have to decide how to divide the abscissa (x axis, pips/bar) among labels (fuzzy sets) denoted by "Falling Strongly", "Falling", "Falling Slightly", "Flat", "Rising Slightly", "Rising", "Rising Strongly", say. This division can be arbitrary, based on personal opinion of what constitutes e.g. a strongly falling trend, say, or the division can try to presage the rules by which it will be combined with other observations potentially to generate buy/sell orders, in part appealing to experience.

In other words, any 50SMA with a slope more negative than -1.0 pips/bar may characterize a trend that is "Falling Strongly", but in light of our candidate rule regarding negative trends ("If the trend is negative then favour short entries"), experience suggests it may not be smart to enter short when negative SMA slope is in the vicinity of an inflection point (i.e., maximum negative) or waning. Furthermore, a snapshot measurement that indicates "negative to flat" trend can mean opposite things depending on whether trend is accelerating from zero or decelerating (i.e., "trend may be beginning so prepare to sell heavily" versus "trend may be ending so prepare to exit short positions")

The bottom line is, we should not expect a strat to place orders based on one aspect of price action any more than we do. In terms of parameterizing the model the notion of trend will likely require a qualifying linguistic (namely, "accelerating" and "decelerating"), and trend is just the first of 5 notions (or "energies" according to Barry Burns) to be implemented (the others including momentum, cycle, S/R and fractal confirmation).

Histogram of 50SMA slopes for 1 month of 200 Tick EUR/USD data (x axis in pips/10000):


Membership plan for Trend using centroid to calculate fuzzy representation:

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #20 (permalink)
Site Moderator
 
 
sam028's Avatar
 
Posts: 3,665 since Jun 2009
Thanks: 3,787 given, 4,494 received

Very interesting @ bnichols, Ill be happy to see a piece of this fuzzy code integrated in a NinjaScript strategy, or at least how "heavy" is it to add this kind of logic. Your code seems to have more than 2000 lines, a bit scary .

See below a good, small and simple paper to understand fuzzy logic, "A Fuzzy Logic Based Trading System".
STF discretionary spot Forex system development journal-10.1.1.90.5035.pdf

Success requires no deodorant! (Sun Tzu)
Follow me on Twitter Reply With Quote
The following 4 users say Thank You to sam028 for this post:
 
(login for full post details)
  #21 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Thanks for the paper @sam028. It raises a few issues that should probably also be discussed here when the time comes, like choice of membership function (the authors use a Gaussian mapping, rather than triangular, and I plan to use trapezoidal rather than triangular since that's what's available in the 3rd party library). In the best of all possible worlds the shape of the membership function should be determined empirically but at this point I suspect any effect would be at best 2nd order.

The authors also mention their "convergence" model without giving details (the means by which fuzzy values input to the inference engine are derived from observations), the means by which the inference engine processes propositions ("rules") without elaborating and how they combine & defuzzify proposition outcomes to generate orders (Mamdani min/max implication), again with no discussion when the choice of fuzzification method & implication operator can in theory have significant impact on the accuracy & behaviour of the system.

It's interesting to note their choice of indicators for input to the system (MACD, CCI, RSI, Bollinger Bands) accounts for momentum and cycle ("spectral") characteristics but IMO there is no real measure of trend or attempt at inter-time frame confirmation. Like most autotrading strategies no doubt the system works when the market cooperates but tends to bring to mind Samuel Johnson's remark, to paraphrase ""...[a machine trading] is like a dog's walking on his hind legs. It is not done well; but you are surprised to find it done at all."

I spent the time since my last post dumping indicator outputs for EUR/USD 200, 600 & 1800 tick charts (various parameterizations of S- & EMAs, MACD & Stochastics) and their first and 2nd differences (slopes & accelerations) and staring at plots of them in Open Office Calc first to make sure they're usable, second to see how they interact and finally to define membership functions. My next post will likely be to summarize the membership functions and show how they're coded in Aforge.Fuzzy. After that the issue will be implementing the trading rules as "fuzzy propositions"--the meat & potatoes of any strategy.

Edited to add: a decent discussion (i.e., adequate, plain language, math doesn't make one's head spin, lots of examples) of fuzzy concepts is located here: PowerZone: Fuzzy Logic Module. It's for control applications rather than for trading but the principles are the same & are straight forward to extrapolate.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #22 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Just a quick post to reflect on using frequency binning (histograms) to determine extrema and ultimately fuzzy membership functions (mappings of input values onto fuzzy sets, or collections of labels).

Histograms also provide a basic means to estimate probability density function (PDF) and in general histograms of values of Close price produce non-unimodal distributions that e.g. @Fat Tails refers to as "time-Price opportunities" from which TWAP (time weighted average price) & POC (point of control) can be estimated. Histograms of SMAs of Close price are also in general non-unimodal, muted replicas of the Close price histogram, which might be expected since SMA values are essentially low pass filtered versions of price.

Histograms of first and second differences (i.e., slopes and "accelerations") tend to produce unimodal, practically "normal" distributions.

In contrast it turns out the histogram of MACD values themselves is also approximately unimodal whereas the histograms of Stochastics K & D values are significantly bimodal (Stochastics D a muted replica as might be expected), in the latter case in fact reminiscent of the PDF of a sine wave (see e.g. Atif's Blog: Probability Density Function (pdf) of Sine Wave). Even though MACD is sometimes referred to as a "momentum oscillator" the fact that its estimated PDF is characteristic of a random distribution suggests Stochastics is a true(r) oscillator and an adequate proxy for the notion of Cycle in our model.

Graph of MACD histogram for 600 tick Eur/USD (24106 sample population):




Graph of Stochastics K histogram for 200 tick Eur/USD (33005 sample population):
I suspect the spike at x=50 is real, due to a tendency of Stochs to flatten from time to time, rather than a sampling artifact.




Graph of Stochastics D histogram for 200 tick Eur/USD (33005 sample population):




Graph of histogram of one cycle of sinewave:
Artifact (drops to zero) in centre of plot is not real, due to histogram bin size



Further to the topic of PDF estimation, a somewhat more elaborate approach is "vector quantization" (VQ) used in lossy compression of sound & images. A variety of VQ employs competitive learning, so called, and is the basis of "learning vector quantization" (LVQ) which in turn is at the root of the "self organizing feature map" (SOM) approach to pattern recognition. SOM and other artificial intelligence (AI, essentially neural network based) methods have been used for financial market prediction for a long time (e.g., "Predicting Stock Prices Using A Hybrid Kohonen Self Organizing Map (SOM)", by Mark O. Afolabi and Olatoyosi Olude in "Proceedings of the 40th Hawaii International Conference on System Sciences - 2007", $30 from IEEEExplore, which IMO unfortunately reads like it was written by a couple of 3rd year engineering students who knew as little about financial markets as they did about AI :-/).

While this association with neural networks may be interesting & inviting (at least to me), it's also a warning not to let development of a potentially useful way to manipulate trading heuristics via fuzzy propositions devolve into yet another AI blind alley In any event the AForge.net library also contains a neural network modelling core, including SOM methods.

If nothing else, to the extent histograms used to parameterize fuzzy membership functions may also estimate PDFs well enough to calculate the probability price action will generate a given linguistic, we should hope to calculate the probability price action will cause a given proposition to execute (proposition execution probabilities some combination of probabilities associated with fuzzy input parameters comprising the propositions), proposition execution eventually resulting in buy/sell orders, and thereby estimate the strategy's level of trading activity. This is approximately what the inference engine already does (compute proposition outcomes in terms of likelihoods from its components).

By the same token, ideally we might attempt to "build backtesting in" to the strategy by examining the same price data set used to parameterize the membership functions, looking for a way to estimate the probability a given proposition will "succeed" ( i.e., that any order derived from the proposition's predicate will meet its targets), and hence "build in" some idea about profitability.

At first glance this might not be as nebulous, far-fetched or time wasting as it might seem since propositions here simply embody a system's trading rules, which in turn presumably are determined from some number of observations of live or historical price data, from which setups are deduced because they are seen to work 2 or 3 times and we conclude optimistically "maybe they work most of the time". Back- and forward testing, construed as a separate step in system development, tempers our initial optimism by helpfully ferreting out all the occasions the proposed setup fails.

What might be missing from the plan so far therefore is an empirically constrained element of prediction--supervised training, in other words. The issue I'm grappling with may be whether it's possible to incorporate training into the fuzzy system directly from data used to design the membership functions--particularly output membership functions--without resorting either to explicit backtesting or some neural AI technique (a slippery slope IMO that should be approached cautiously).

Edited to add: I've attached the code--a strategy called "TDWriteSMASlope"--I'm using to dump data for histogram calculation in case anyone wants to experiment. It's configured as a strategy because I find it more convenient to work from strategy analyzer than attach an indicator to a chart. Should import into NT 7, no need to install indicators included (generic to NT), creates a .CSV file for import into your favourite spreadsheet program but no guarantees because it's is quick, dirty and unoriginal (borrows features from other published code ..... thanks for the file write snippets @Ducman ). The only custom parameter for the strategy ("Collect By") can be "Daily", "Monthly" or "OneFile" and determines how much data is written to a single .CSV file. For "Daily", 3 files are written to a different folder in the NinjaTrader.Cbi.Core.UserDataDir + @"\Preprocessing\ folder (typically C:\Users\Owner\Documents\NinjaTrader 7\Preprocessing\) for every day, Instrument, PeriodType and Period (e.g.,EURUSD_Minute_5_values.csv, EURUSD_Minute_5_slope.csv, EURUSD_Minute_5_accn.csv) between the selected start and end of the strategy Time Frame. Any folder that doesn't exist will be created. The 3 files contain bar-by-bar data for Close price and values, slopes and accelerations for 3 SMAs, 1 EMA, MACD, MACD.Avg and Stochastics.K and Stochastics.D. Similarly "Collect By" set to "Monthly" will write 3 files to a different folder for every month, Instrument, PeriodType & Period in the Time Frame, and "OneFile" writes 3 files to a different folder for every year in the Time Frame.

Directory structure for data dump "Collect By" parameter:

Attached Files
Register to download File Type: zip Strat_WriteData.zip (13.8 KB, 13 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #23 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


bnichols View Post
Thanks for the paper @sam928. <....snip....>

Not sure why I can't edit post #21 to correct your mention @sam028, so I'll quote it instead.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #24 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Preamble

Since I last posted I've been manually paper trading spot EUR/USD using the system under development almost every waking hour, sleeping 4-6 hours at a time as required, studying the way selected indicators react to as many variations of price action as possible. While the market is capable of a very, very large number of variations at long last it seems patterns I've never seen before are becoming rare.

As the system slowly becomes "second nature" the number of successful trades increases and so does my confidence. At this stage the market has become familiar, friendly & fun rather than scary & inscrutable. That said, it seems we approach "consistent profitability" exponentially, by repeating mistakes until we finally learn from them and by reinforcing behaviours that work. As a general rule we increase the number of units traded from a bare minimum as confidence increases to the limit of what money management allows, which is where the joy of trading starts to kick in.

I still believe we can't create a bot that makes money if we don't know how to make money, any more than we can program a bot to solve a problem if we don't understand the problem. Computers are good for problems than have numerical (non analytic) solutions, as I believe trading to be. Which perhaps paradoxically is why non-analytic humans can be so good at it.

To reiterate, "the system" is essentially Barry Burns' ("BB") "5 energies" system (topdog.com). This is not a recommendation or an endorsement of this particular system, since IMO it is simply yet another repackaging of sound trading principles common to any system. I fully believe any legitimate system will make money once you've mastered it (not necessarily a tautology). The whole issue is mastering the system, which means learning how to trade, and in the process mastering oneself. For the purpose of this exercise BB's system provides the definition of a "vector" for implementation as a bot, as described below.

The cost of shifting my internal clock to trade both the London and New York sessions is substantial and more than ever like a lot of traders I'm looking forward to creating a bot from what I've learned, so that e.g. it can trade London while I trade New York.

Work in Progress--Fuzzy Decision Making

It's one thing to fuzzify inputs but quite another to design a fuzzy buy/sell/sit-on-your-hands decision process.

To recap, we've selected a set of indicators we hope are more or less necessary and sufficient to defining setups and generating order signals--in this case essentially MACD, Stochastics and 4 moving averages, plus some definition of S/R levels--in the usual 3 time frames ("fractals" in BB's terminology). What needs to be made explicit is what to look for--what are the significant combinations of values and rates-of-change of the indicators, and mainly--where are the S/R levels and their strength.

In terms of bot programming a "significant combination of values and rates-of-change" is simply a set of vectors that are likely to cluster between buy and sell signals.

A convenient way to define such a vector is by capturing indicator values at "significant moments". Assuming a "significant moment" is the point at which in hindsight we wished we'd bought or sold, then one way to identify some is by dumping values at peaks and troughs of e.g. the ZigZag indicator during a backtest, given the usual caveats of backtesting (past results etc., which is where exhaustive experience comes in).

Anyone who's used one variation of ZigZag or another will understand it matters what value we choose for the deviation (minimum pips for a zig or a zag). This matters to the algorithm under development as well and at this point needs more study to control profit expectation for a given setup.

To this end I rewrote the strategy mentioned previously, whose sole purpose is to dump data during a backtest, to include values of the ZigZag indicator. During proof-of-concept indicator value sets ("vectors") at ZigZag peaks and troughs were isolated in a spreadsheet, but since then I've written a standalone C# class to extract these vectors from NinjaTrader strategy output directly.

At this point the idea behind identifying "significant vectors" is to determine their centroids in so-called "N-space" (which we assume at first is Euclidean). I naively suppose I can write an indicator to show where price action is relative to these buy/sell centroids once they've been established (a process that amounts to backtesting and hence is worth as much as backtesting) as an aid to manual trading. While such an indicator might be a trading aid (i.e., to help overcome any residual tendency to trade impulsively) the main purpose is to design a decision-making core for a bot.

Again, success of the approach depends on the extent to which the chosen indicators are necessary and sufficient; in other words, the extent to which (narrowness with which) vectors cluster.

Where Fuzzy comes in

The N-space cluster has an associated distribution function, distance and slope of which can be assigned linguistic descriptors and thereby made part of the usual input-inference-output process.

Outputs

I've written a standalone program (C# using Visual Studio 2010 Ultimate with Net 4.0, using ZedGraph for visualization if you want to prepare) to perform K-means clustering, mainly to extract centroids and am working on a program to reduce raw data collection strategy output to clustering program inputs. Will publish the programs when they're stable. Still trying to figure out interesting anomalies that seem to become apparent with visualization, hence the importance of visualization. At this stage the question is, are the anomalies real and therefore useful for trading algorithm design, or is there a mistake in the N-dimensional clustering program.

Conclusion & prognosis

Programming is more fun than trading but at the end of the day I like a program that makes money. I'm optimistic about results so far.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #25 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Preamble

Since the last post I've been both trading as much as humanly possible and working on a Visual C# program to perform a number of statistical analyses of price data to determine the feasibility of writing a bot to trade the "5 energies" method.

I can trade while drinking alcohol, after being up for 24 hours and essentially out of the corner of my eye-- while watching TV or playing Spider Solitaire and still retain the necessary focus (based on cumulative profits) but yet still have episodes in which I can be wide awake & seemingly not distracted and yet lose my feeling for price and continue to trade--what I consider my worst habit: trading without "feeling price". If bad trading habits die hard then they are also prone to coming back from the dead.

The screenshots below summarize progress so far: namely,

1. importing feature vectors generated by a NinjaTrader strategy, 1 vector per bar over a given time period (as previously described)
2. creating buy/sell clusters (hopefully) from the data by filtering on ZigZag indicator extrema
3. applying K-Means analysis to extract bounding polygons & centroids of clustered data
4. calculating density functions for the clusters to act as fuzzy "shape functions"
5. visualizing the above

I prefer this approach to regular backtesting because it's just as quick and probably more efficient; by virtue of filtering by ZigZag extrema the data can be made self organizing.

In theory basing buy/sell decisions on the covolution of an instantaneous feature vector with a density function derived from clustered buy/sell population data (vectors prefiltered by ZigZag extrema) should produce a more robust strategy than a ZigZag indicator-based strategy alone.

Continuing Work

At the moment I'm still contemplating the most efficient way to implement the shape functions in a bot but have pretty much ruled out neural nets & some derivation of Self Organizing Maps, the rule of thumb being that if the goal can be met by a "conventional" computer algorithm then it should be done that way (net design in particular having a way of becoming frustrating & time consuming). I've also ruled out messing with data in higher dimensions than 3.

Therefore the approach being actively pursued at the moment involves determining the probability that a given feature input vector signals a buy or sell

1. first, by convolving a real-time feature vector in some sense with each of 190 x 3 = 570 predetermined 2-dimensional density functions corresponding to the same feature population clustered 2 at a time ("20 choose 2") for 3 fractals (1800, 600 and 200 ticks).
2. second, by estimating a probability from the results by Mamdani Min Implication (a standard approach to reducing fuzzy propositions)

To the extent the approach is feasible (and it's not clear yet that just because processed data clusters that it's useful) then some means has to be implemented to determine stops, profit targets and trailing stop behaviour but the following assumptions are made:

1. initial stops ought to be placed according to usual criteria (vicinity of a convenient S/R level above or below entry that also meets money management rules)

2. profit targets set at S/R levels consistent with the profit cluster(s) generating the highest probability buy/sell signals

3. trailing stop behaviour controlled by the trajectory of the instantaneous feature vector (vector generated at the close of the last bar) within the various density clusters.

Screenshots

Notes:

1. Data shown below (raw data, sample of unfiltered and 570-such filtered clusters) is derived from EUR/USD for the last 3 months of 2011, each analysis comprising 14,000-120,000 vectors depending on the fractal shown (1800- , 600- or 200-tick-based charts respectively)

2. Data shown has not been cleaned to remove outliers, as indicated occasionally by the extremely wide enclosing polygon boundaries.

3. It can be seen that in general the data clusters, which at first glance might seem to imply it should lend itself to buy/sell prediction (and AI methods in general), but in reality at this stage means further analysis is required to determine what part of the feature vector population not associated with a ZigZag extremum also lies within the cluster and hence whether the cluster predicts anything at all.

4. While the center plot (Parameter Chart) shows both clusters on the same X-Y axes and hence on the same scale and proper relative position, the Density Function plot scales and cluster position are relative to min & max of the respective enclosing polygon shown in the Parameter Chart.

5. Respective centroids are indicated by small black discs near the center of each buy/sell cluster

Figure 1. 118,000-vector STF Fractal (200 Tick bar) StochsD/MACD feature population in an 8-pip profit cluster (i.e., data filtered to show vectors that preceded at least a subsequent 8-pip price movement)
Non-close fitting enclosing polygon in the Parameter Chart indicates noise spikes that are naturally suppressed in the Density Function, which at this point is essentially a low-pass filter




Figure 2. LTF Fractal (1800 Tick bar) StochsDSlope/StochsD feature data in a 32-pip profit cluster (i.e., data restricted to vectors that preceded at least a subsequent 32-pip price movement)


Figure 3. LTF Fractal (1800 Tick bar) MACD/StochasticsD feature data in a 128-pip profit cluster (i.e., data restricted to vectors that preceded a subsequent minimum 128-pip price movement)


Figure 4. LTF Fractal (1800 Tick bar) unfiltered/unclustered StochsD/StochsDSlope feature data (approx 15,000 vectors)


Figure 5. LTF Fractal (1800 Tick bar) StochasticsD/StochasticsDSlope feature data (data from previous Figure 4) in a 128-pip profit cluster (i.e., data restricted to vectors that preceded a subsequent minimum 128-pip price movement)


Figure 6. MTF Fractal (600 Tick bar) unfiltered/unclustered StochsD/MACD feature data (approx 43,000 vectors)


Figure 7. MTF Fractal (600 Tick bar) StochasticsD/MACD feature data (previous data from Figure 6) in a 16-pip profit cluster (i.e., data restricted to vectors that preceded a subsequent minimum 16-pip price movement)


Figure 8. MTF Fractal (600 Tick bar) StochasticsD/MACD feature data (previous data from Figure 6)in a 32-pip profit cluster (i.e., data restricted to vectors that preceded a subsequent minimum 32-pip price movement)


Figure 9. MTF Fractal (600 Tick bar) 15EMASlope/200SMA feature data in a 32-pip profit cluster (i.e., data restricted to vectors that preceded a subsequent minimum 32-pip price movement)


Figure 10. MTF Fractal (600 Tick bar) StochsD/StochasticsDSlope feature data in a 32-pip profit cluster (i.e., data restricted to vectors that preceded a subsequent minimum 32-pip price movement)


Figure 11. MTF Fractal (600 Tick bar) StochsD/StochasticsDSlope feature data in a 128-pip profit cluster (i.e., data restricted to vectors that preceded a subsequent minimum 128-pip price movement)

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #26 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,780 since Jun 2009
Thanks: 32,306 given, 97,567 received

Spectacular!

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #27 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Thanks Mike

A few comments while I decompress, glass of rum in hand--not the first, lit cigarette on the lips, bearing in mind retail currency spot market reopens in less than 24 hours. The density plots in particular are easy on the eyes but as you might agree we won't be too impressed until the thing makes money, and many miles to go before we sleep easy in that regard. Which I suppose should prompt a caveat to the newbie quant trader: namely, IMO one ought to view statistics--no matter how lovely to look at--the same way we scrutinize a prospective spouse: skeptically, proof always in the home-made pudding if not the fact s/he's self supporting, forgives our mistakes. Don't chase algorithms any more than others chase indicators. IMO the path to profitability starts and ends with manual trading by the seat of your pants, as often as possible--only way to rewire your neurons to make money while you're awake and your wits are about you, and to write code that makes money while you sleep (with one eye open). First and foremost quants need to hearken back to your first profitable algorithm always, the same way you rely on your original spouse. Assuming you're still married to your first spouse, in which case if you're not there may be trading issues to overcome that this thread may not address, since even a booty-shaking, money-making program can lose in the wrong hands

To this end, on the question of where do feature vectors not corresponding to price extrema lie relative to the buy/sell (extrema) cluster centroids, overnight I added an "auxiliary vector" to each feature vector that simply records the "distance" in the time dimension of any vector from the vector corresponding to the last extremum, hypothesizing (i.e., "hoping like hell") that between extrema (i.e., while price is trending from entry toward a profit target) feature vectors would be well behaved. In other words, to see if feature vectors between price extrema would move predictably (more or less monotonically) from one cluster toward the other as price moves from entry to profit target, rather than (say worst case) leaping between clusters in a binary fashion. Practically speaking this analysis aims to quantify feasibility the algorithm can duplicate the trading rule of thumb, that if we miss the initial entry it may not be too late to participate if we lower profit expectations and be extra careful about placing the initial stop (i.e., assuming in general such a stop is going to introduce relatively higher risk than stopping the ideal entry). By the same token the analysis should also inform trailing stop behaviour, both goals requiring the vectors prove well behaved.

One problem I still have is allowing too much weight to the rule of trading "Don't let a profitable trade become unprofitable" and hence still too often getting trailing-stopped-out early, despite the rules of the method effectively relying on stops to put money in the bank rather than targets, to the detriment of cumulative profit. Effective management of trailing stops is crucial (if still considered more an art than science) since weak management tends to nullify impact of the high probability with which some number of us (most?) can pick S/R levels most of the time & (perhaps to a lesser extent) the likelihood they will breach, extreme vagaries of short time frame spot currency aside, so I'm looking forward to a little clarification It could turn out e.g Fibonacci discovered the rules of trailing-stop-management a long time ago (funny how exhaustive analysis often just proves what other folks take for granted), which would take a load off the CPU.

So far cursory visualization of the behaviour of these intermediate vectors looks good for selected features that we know to be highly correlated with price movement, like MACD and Stochastics. We expect increasing divergence from a straight line path between buy/sell clusters in first and second differences (slopes and accelerations) but it's hoped e.g. the statistical variance will lend some idea of how much price can be expected to wiggle between order entry and profit targets, hence provide an input into the trailing stop control. If nothing else variance may provide confidence weighting factors when combining probabilities of noisy density functions with more stable density functions. BTW, what separates born quants from normal, well-adjusted traders (assuming there is such a thing) is--quants stew over the fact every wiggle in a trend trade represents $$$ left on the table.

ETA: On a practical note, making the latest changes introduced what appears to be a harmless bug into the software, which nevertheless has to be tracked down (frustrating because development stops until it's repaired, for as long as it takes); one is never 100% sure if software (let alone statistics) show the "truth" or simply reflect our expectations, more doubt when they misbehave in seemingly inconsequential ways (being a true believer that the devil lurks in the details) :-/ In that regard we ought to note EUR/USD was trending down overall during the last 3 months of 2011, meaning the current trial population itself likely includes a large wavelength bias that still needs to be accounted for.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #28 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Bug fixed with no change in the prognosis for the method. An analysis of intermediate vectors (snapshots of parameter values between price extrema) indicating that they are well enough behaved, thoughts are turning toward implementation of a probability density function based mapping from parameter values to fuzzy variables. At this point therefore things get a little more technical.

It turns out after an actual parameter count the length of the feature vector being considered is closer to 30 than 20 and in danger of growing. The effect of this is to increase the number of density functions from 190 x 3 = 570 (20 choose 2 x the number of fractals) to 435 x 3 = 1305 (30 choose 2 x 3). Since a value must be retrieved from each 2D density function for each parameter pair bar by bar in real time while the strategy is running, the larger the number of functions the greater the load on computer resources. If in addition we want the system to learn (in this case, to update the density functions on the fly) then whatever means we choose to store & access system structures ought to be malleable.

Rather than make the effort right now to try to prune the list to a sensible number, or to reexamine the approach itself in light of the fact it might be unwieldy , I've instead decided to evaluate methods that might reduce the complexity (or at least deal with complexity effectively).

We're using 2D densities pretty much only because higher dimensions are more difficult to visualize, and IMO visualization is important. We've also been sticking to 2D partly because some math (e.g., cross product, if we needed it) apparently breaks down in higher dimensions, except e.g. for the 3rd and 7th dimensions. I have no clue what the significance of that is and at the moment no inclination to pursue it.

That said, it may be possible to exploit the multidimensional nature of the data without resorting to too much hyperspatial gymnastics if all we're doing essentially is data retrieval.

The purpose of the 2D density function (visualization aside) is ultimately to associate a probability that instantaneous values of 2 parameters (e.g., MACD and Stochastics) lie within a buy or sell cluster. Boiled down, right now the 2 instantaneous parameter values index a table, whereas from a broader perspective we'd prefer to index an N-dimensional table with a vector comprising N parameter values. While as mentioned previously we don't want to wind up lost in hyperspace or impaled on c# pointers, methods for collapsing multiple dimensions to a single dimension for the purpose of data storage and retrieval are well known (e.g., helical hyper-spatial codes, related to Peano codes, Hilbert curves and space filling curves in general). My initial exposure to these was in the early '90's, rubbing shoulders with a colleague who was helping Oracle develop their multidimensional search core, subsequently embodied in Version 7 I think.

In other words, if we can come up with a suitable key/value structure to encode all 1305 (or more) density functions in a single table then it may be possible to use C#'s Dictionary function to perform the correlation in "one" relatively fast operation. (I'm considering the Dictionary function only because I've read it's faster than the C# hash table implementation).

We expect the size of the dictionary to increase as the square of table size (i.e., with parameter resolution). For example, right now we're using a 100x100 array to store each 2D density function, which it's hoped adequately resolves parameter values, but otherwise means each index requires 7 bits--essentially one byte of memory before compression. If an index requires one byte then in theory we could increase density function resolution to 256x256 density values over the same parameter range, from 100x100, without affecting how we handle indices, but by so doing we increase dictionary size from 100x100x1305 = 13 million to 256x256x1305 = 85 million key/value pairs.

I think the trick to exploiting the N-dimensional nature of the data at this stage is simply developing an appropriate dictionary key, which amounts to choosing the right space filling curve to act as index. Specifically, we want the index to move through the table in such a way that adjacent points in "N-space" are also adjacent in terms of the index, implying among other things that the resolution with which we probe the N-dimensional probability function is determined by the resolution of the index. Related to this is the fact the density functions are sparse (mostly empty space, zero values). In this case it may be possible to view the structure as an N-dimensional bitmap and therefore accessible to available image and sound compression techniques.

In any event practical implementation of such a key structure is straight forward--far easier than this cursory description might imply, so looking forward to concrete results soon.

ETA: first attempt to implement the key structure will rely on the availability of BigInteger for Net 4.0 in VS 2010 (System.Numerics namespace) to create a Hilbert curve key structure, with the Dictionary definition
 
Code
public Dictionary<BigInteger, double> dictionary = new Dictionary<BigInteger, double>();
and simply interleaving bits from all density function table indices to create N-dimensional Morton numbers (so called) as the key.

While there are other approaches, BigInteger is convenient because e.g. 30 parameters will result in manipulation of 30*8 = 240 bit keys.

ETA: while it's possible to stoop to one of a host of algorithms on the internet to perform a brute force calculation of Morton number given an indeterminate number of dimensions at the moment I'd prefer to generalize the "shift and mask" algorithm only because it's possibly more elegant. Therefore the task at hand appears to be generalizing calculation of the masks to multiple dimensions. Figuring out bit shifting is not my strong suit so if any reader has done this before please feel free to share

The good news is the Net 4.0 implementation of BigInteger in VS 2010 appears to be robust.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #29 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Since the last post I decided that (regarding the bit interleave algorithm) elegance requires more work than brute force, is unwarranted this early in the game at the stage we're still trying to prove the concept, so that brute force is good enough--save elegance for the finished product. At least this is the principle IMO behind "canonical form" in which e.g. math treatises are published, by which an author sweeps all clues as to an idea's humble beginnings under the rug in order to give the impression it sprang as is, full blown from his or her brow and to leave his or her readers completely mystified about how it stems from or applies to anything practical.

In that respect, this is the working Morton number (bit interleave) code in c#. It assumes byte array "a" contains nb parameter coordinates corresponding to a point in N-dimensional parameter space and assumes C# operator precedence; i.e., bit-wise logic ("&") executes before left-shift ("<<") and arithmetic functions ("+", "*") before left-shift ("<<"):

 
Code
        private void interleaveN(byte[] a, int nb)
        {
            for (int i = 0; i < 8; i++) // unroll for more speed...
            {
                for (int j = 0; j < nb; j++)
                {
                    bKey |= (a[j] & 1U << i) << j + i * (nb - 1);
                }
            }
        }
In other news, while looking for a bug in the code that calculates price variance between entry and exit to guide trailing stop behaviour I was forced to reexamine how I was using ZigZag data to extract buy/sell vectors from the parameter data.

To make a long story short, the ZigZagHigh & ZigZagLow dataseries being used are "proto-series" in the sense they don't directly define local tops and bottoms of price movement, but instead record candidate tops and bottoms based on price action as interpreted by the ZigZag indicator. The ZigZag indicator then applies an algorithm to the 2 dataseries to synthesize the familiar zig-zag pattern.

Cluster and density plots posted previously show parameter vectors extracted at discontinuities in the raw ZigZag high & low dataseries--no further processing applied--with the effect that the buy/sell clusters mix long/short entries respectively with what I choose to interpret as candidate intermediate profit targets between entry and exit (the intermediate profit targets being points in the ZigZag dataseries where price consolidates, or undergoes a mini-retrace).

In other words, a lot of the points in the "buy clusters" for various parameter pairs are not strictly characteristic of long entry conditions, but instead characteristic of conditions in which in the very least a short trade should take profits. By the same token, a lot of the vectors in the "sell clusters" are not strictly characteristic of short entry conditions, but instead characteristic of conditions in which in the very least a long trade should take profits. In a sense this is good news, since if the method will work at all these restrictions further reduce ambiguity by limiting the range of conditions characterizing, perhaps even defining, permissible entries and advisable exits. This development embodies the trading rule, "Trade less--lose less."

We may surmise that remaining vectors, the majority of vectors in each cluster that don't correspond to entries or profit targets, might be deconstructed into "do nothing" or "consider adding to the position" conditions.

I expect to post plots of augmented density functions to show the effect of these changes once they're available.

Thus while additional processing (mimicking the algorithm in the NT indicator that draws the actual ZigZag pattern) has been required to separate vectors signalling actual long/short entries from sell/buy profit-taking in the opposite trade (consolidation following short/long entries) but in general characterizing price behaviour at S/R levels that may or may not be explicitly present in the parameter vector (and hence probability density) in the form of Previous Period OHLC, Murrey Math & pivots, overall it's still true for example that the closer a "buy" parameter vector approaches the centroid of the Sell cluster, the more conditions represented by the vector reflect a Sell entry condition within the context of the selected ZigZag deviation. To the extent these non-extremum discontinuities in the zigzag function can be construed as S/R levels not contributing directly to the probability function, they inherently dissect the density function beyond simple clustering, therefore still may contribute to decision making.

Finally, in still other news, I've decided rather than get bogged down unnecessarily trying to deal with data compression it's probably sufficient simply to populate our dictionary structure only with points in the density functions that have non-zero values, taking advantage of inherent benefits of the structure in C# (e.g., translating null returns to zeros if the need arises) instead of trying to enforce too much pseudo mathematical rigour.

The situation is depicted in the figure below, which shows the 1800 tick ("LTF fractal") chart for EUR/USD between approximately 10AM AST December 23 and 6PM AST December 30 in the top half of the illustration and corresponding 32 point (pip) deviation ZigZagHigh and ZigZagLow dataseries superimposed on a line chart of closes in the lower part. (Note: all indicators shown as well as their 1st and 2nd differences are used to parameterize the method. The ZigZag indicator--thin blue line on the top chart--is used as a filter as described during parameter vector pre-processing and does not appear explicitly in the eventual N-dimensional probability density). As a point of interest the 2 dataseries synthesize a channel resembling other familiar "persistent extrema" indicators (e.g., Donchian Channel).

The annotation in the bottom part of the illustration highlights one hypothetical short trade (straight, diagonal blue line marked "Trade") as defined by raw ZigZag dataseries.

In summary, the system clusters parameter data for 10's of thousands of such potential trades from historical tick data in the hope of assigning a probability to patterns--essentially the same as training a neural net without the headache of developing a suitable neural net architecture.


Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #30 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Doing sanity checks on the data, software, outputs for the last 24 hours so not much to report.

A first crack at the dictionary (BigInteger) based probability density encoding method produced mixed results. The algorithm used to produce keys is creating occasional duplicates--definite no-no for what should be a one-to-one mapping. Checking indicates the code is fine so the problem is very likely a logic error and if so I think I know where it is but need a clearer head to decide what the impact is and how to fix. Biggest fear is I've overlooked something fundamental that will send me back to the drawing board. The amount of dead code in the project increases steadily but too soon to prune.

Once issues with the encoding are worked out the next significant push will likely be to interface the dictionary with NT via a DLL, at this stage assuming the strategy will pass the current parameter vector to it as an array. At that point we'll have the first indication whether the method is going to work.

I traded a little on paper today to stay current, lately too distracted by system development and impaired by lack of sleep to trade real money. Although still evolving, even in this condition am confident and have little trouble breaking even no matter how much I trade. At the top of my game I make money so remain optimistic. It still appears the only difference between losing and consistent profitability is experience, and sticking to the letter of whatever method one has adopted.

My attitude toward losses, and hence stops, is in a state of flux. These days I set small stops (3-4 pips) and I have no problem taking a (small) loss because it seems to me there is no practical difference between a loss that is not yet realized and one that has been realized. Overall, commissions pale beside the cost of remaining in a losing trade too long. We not only waste time waiting for the market to "turn around" if it comes back at us but worse, while we remain in the market we tend not to be entirely objective about conditions, nor does it seem as easy to see pending setups that either confirm we were probably right in the first place or waken us to the fact we were wrong. Doubt or confusion is probably the best & earliest indicator the market has become choppy relative to our trading method, and one losing trade is the signal to sit back and take stock. The bottom line is while we are in a trade our account balance reflects what price does, for better or worse, and while we are not we are immune to what price does--one ought not take profits & losses personally, but we do.

That said, one change I made some time ago that may be obvious to experienced traders and that tripled profits happened after I noticed 45 of 50 ATM trades were exiting at the trailing stop (I've come to hate the words NT speaks, "Stop filled"). Rather than simply advancing the stop after some price advance, or after price cleared an S/R level--essentially hoping for one long-bomb trade, conserving the entire position until the bitter end--instead I now both take some profits and advance the stop when price advance pauses for one reason or another. In particular this avoids the hateful situation I've come to call "boiling the frog"--when after we've been up (in my case) 35 or 50 pips price slowly and inexorably retraces to take out the original entry point. A "small loss" in this circumstance is inexcusable.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #31 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Since the last post I've been trading less and working on the system more.

Briefly, the dictionary-with-bit-interleaved-key approach to storing the probability density works well enough and so far seems a highly efficient way to find a particular data point among (literally) millions, but I'm becoming a victim of my project management approach ("Let's just make a start and see what happens") and undisciplined programming style--spaghetti & unreachable code technique. The upshot is that data reduction--an automated process whereby raw parameter data output dumped by the NT backtest strategy is converted to density maps--is time consuming, perhaps up to 18 hours (running in debug from Visual Studio IDE; standalone release version runs about 4x faster) to produce the STF (short time frame fractal) data set. The application could exploit multithreading better than it does but this should have been worked out at the planning stage (what planning stage?) and would now mean unraveling the Gordian Knot of the core process.

The last few days have been mainly focused on administrative tasks--data structures and code entry points needed to interface with NinjaTrader. Among other things this requires finalizing the data object that represents the observed behaviour of parameters ("learned behaviour" is too probably strong a phrase at this stage). At the moment this "object" comprises 2 dictionaries--the Density dictionary (essentially a database of raw 2D pseudo-probability density functions) and the Parameters dictionary, the latter a database containing information about each 2D function in the Density dictionary.

Before that I spent some time looking at the shape of the functions and studying the way test parameter vectors migrate between buy & sell clusters, still looking for confirmation that price behaviour is predictable in some sense between highs and lows as a sanity check but (hoping for the best) also a way to describe variance (retraces between entry and target) for trailing stop control. Today while watching test vectors migrate between buy/sell clusters and watching EUR/USD oscillating around 1.3200 for a while before jumping to oscillate around 1.3250 it occurred to me the math of strange attractors might provide a way to quantify the behaviour in density space as well as in price-time space (i.e., on the chart) around "magic numbers" like 00 and 50 (in particular their numerical approximation), since the Morton number dictionary key probably lends itself to that, but decided that life is too short--will have to wait until the strategy is making money.

Hope to update this post with details & some illustrations ASAP but in the meantime here is an N-dimensional bit deinterleaving algorithm implemented in C#--seems to work 100% of the time but as usual no guarantees. It's been handy as a debugging tool, since so far in practice keys need only be computed, not decoded. The fiddle with the BigIntegers is because conventional long unsigned Int overflows during shifting for these bit lengths. As before the array a will contain the nb parameter vector coordinate values after decoding. Global variable bKey is the Morton number to be deconstructed.

 
Code
        private void deinterleaveN(byte[] a, int nb)
        {
            BigInteger bT = bKey;
            BigInteger bit = new BigInteger(0);
            BigInteger u1 = new BigInteger(1);
            for (int k = 0; k < nb; k++)
            {
                a[k] = 0;
                for (int i = 0; i < 8; i++)
                {
                    bit = (bT & (u1 << k + i * nb)) >> k + i * (nb - 1);
                    a[k] |= (byte)bit;
                }
            }
        }

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #32 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Regarding visualization, which consumes more coding effort right now than writing data reduction algorithms, shown below are 4 images of a 2D probability density plot (left hand side) next to a snapshot of an 1800 tick chart for EUR/USD between 01:52 AM and 11:02 AM AST, November 15 2011 (right hand side), corresponding to the Long Time Frame fractal.

The first image shows on the left side the projection of a buy condition parameter vector on the 2D density for StochasticsK (vertical axis) vs MACD (horizontal) filtered for a 16 pip profit target. The cursor in the chart on the right hand side marks the corresponding relative low price. Green lines connect the fields in the GUI with both the selected 2D Density (StochsK vs MACD) plot and the selected test vector with the corresponding projection onto the density plot (green square) and the event in the chart (marked by the cursor in the chart).



The second image shows on the left side the projection of a sell condition parameter vector on the 2D density for StochasticsK (vertical axis) vs MACD (horizontal) filtered for a 16 pip profit target. The cursor in the chart on the right hand side marks the corresponding relative high price. Green lines connect the fields in the GUI with both the selected 2d Density plot (StochsK vs MACD) and the selected test vector with the corresponding projection onto the density plot (green square) and the event in the chart (marked by the cursor in the chart).



The third image shows the projection of a buy condition parameter vector on the 2D density for the difference between 15EMA and Close price (vertical axis) vs MACD (horizontal) filtered for a 16 pip profit target on the left hand side of the image. The cursor in chart on the right hand side marks the corresponding relative low price. Green lines connect the fields in the GUI with both the selected 2D Density (15EMADiff vs MACD) plot and the selected test vector with the corresponding projection onto the density plot (green square) and the event in the chart (marked by the cursor in the chart).



The fourth image shows the projection of a sell condition parameter vector on the 2D density for the difference between 15EMA and Close price (vertical axis) vs MACD (horizontal) filtered for a 16 pip profit target on the left hand side of the image. The cursor in chart on the right hand side marks the corresponding relative high price. Green lines connect the fields in the GUI with both the selected 2D Density (15EMADiff vs MACD) plot and the selected test vector with the corresponding projection onto the density plot (green square) and the event in the chart (marked by the cursor in the chart)..



Notes

In the Density GUI the "Profit Target" textbox contents apparently no longer refer to anything (shows 32--should be 16). "Button1" redraws the Density map minus the centroids (yellow squares) or test vector projections (green squares).

We've used a radius of "50" in scaled probability space, which translates into a smoothing function approximately 1/10th the size of the Density plot, which accounts for the circular appearance of outliers. We should base the radius on estimated density (number of samples in the space divided by the volume of the space).

Disks (notably white disks) with chunks bitten out in the density plot are an artifact of the method used to combine buy and sell clusters and thus e.g. of outlier sell cluster data interfering with outlier buy cluster data and typically have a low probability of occurrence, so is not significant & ignored at the moment. They remind me of Pac Man.

The centroids of the buy (blue to pink color scheme) & sell (white/red/black color scheme) clusters in the density plots are indicated by yellow squares.

The projection of a single test vector onto both density planes during buy and sell events is shown as a green square (circles are more interesting to draw from scratch on a bitmap than I ought to make time for right now ). In both cases test vector projections (green squares) happens to fall squarely in the "strong buy"or "strong sell" area of the density plot.

The good news is, from the EUR/USD chart these events happens to coincide with 2 points at which one could have made money respectively entering a long or short trade if one exited at the right spot. Exits are a work in progress, bound up with trailing stop control.

Note however that these 2 events were chosen from the same dataset used to compute the density functions and is more of a sanity check on the implementation than proof that the system "works" in any sense.

In particular this example is for illustration only; we can't rely on the fact that 2 cherry-picked test vectors projected onto one or 2 density functions happen to meet expectations proves anything at all. There are (or will be) 325 x 4 x 3 = 3900 such density functions in the Density dictionary (the number of combinations of 23 parameters taken 2 at a time, times the selected number of profit target comprising the ZigZag filter, times the number of time frame fractals) and countless "test vectors".

It should be reiterated perhaps at this point the time and effort invested in this first attempt to code the system won't be required to keep the "knowledge base" of density functions updated, or to encode a new knowledge base with different parameters (and therefore in theory extensible without much effort to any system). It's expected however the "glue" (fuzzy propositions to come, that generate buy/sell signals from the combined outputs of bar-by-bar convolutions of parameter vectors appropriate to the system with the knowledge base) will require input from a trader familiar with the particular system. Not sure yet what form that will take.

Concluding Remarks
That said, the numbers generated by convolving a test vector with (what is essentially) an N-dimensional probability function will make sense to a computer (eventually) and is the best way I can think of to expose a computer to trading probabilities without resorting to conventional AI techniques, and having inspected a lot of these projections it's possible to say performance so far meets expectations.

In other words--its not perfect. I can find instances where test vectors in a suite of vectors corresponding to a price turning point that is broad relative to the time frame (or fractal)--low level chop--appear to give successive contradictory buy/sell signals if it were left up to them. This is to say problems that may plague trailing stop control may affect entry, but we won't be sure until we attempt to combine outputs of the convolution of an instantaneous parameter vector with the knowledge base (i.e. until we interface with NinjaTrader). This may be evidence of the worst-case scenario mentioned earlier (at least in the first instance that parameter transition from entry to exit is not well behaved), but IMO just overall more proof one does not trade a single probability function any more than one trades a single indicator, or trades chop, and that proof is always in the pudding. Some kind of low lag filter on these instantaneous parameter vectors will likely be required, and I'm assuming fuzzy logic will perform that function (make sense of the outputs). At least it's the hypothesis I'm sticking to at the moment.

Next work

A data structure to represent the output of a test vector (the "instantaneous parameter vector" generated when each bar closes in NinjaTrader) convolved with the Density dictionary (the knowledge base) is required, which means code to visualize it. This will initially take the form of a 3 fractal by 4 profit target by 325 probability value structure (one probability value corresponding to each pair of parameters projected onto the buy/sell probability plane of every combination of 23 parameters considered 2 at a time). It's not clear at the moment if the structure requires (or would permit) further reduction to make manipulation by fuzzy propositions viable. I'm hopeful because the system so far is nothing more than a somewhat queer shape function mapping measurements to fuzzy input values.

The density object part of the design still doesn't account for S/R levels, probably because while they vary they tend to be discrete (don't have 1st and 2nd differences). I'm still wondering to what extent the other 23 defined parameters account for price behaviour in the vicinity of S/R levels, and in fact how much redundancy is already in the selected parameters. In any event S/R levels, including magic numbers, make natural profit targets and can buttress entries

There may be evidence some parameters (typically differences between SMA and EMA and close price) routinely map to the opposite (wrong) buy/sell cluster, which makes no sense to me at the moment. Have to determine whether it's a bug or, if genuine, resign myself to using negative correlations in the fuzzy propositions that combine function outputs.

Time is included in the parameter vector but no attempt has been make to incorporate it the way some systems e.g. use time to cluster (in principle) the same traders, and presumably their styles, in a given session. There is some evidence of behaviours at certain hours of the day, in particular the possible tendency of various markets to make a beeline for the floor pivot.

Finally, rather than weight function outputs prior to synthesizing a buy/sell signal from combined probabilities will likely simply remove parameters that result in weak separation between buy/sell probability clusters. Overall IMO rules, including how much attention one pays to a given indicator depending on circumstances (e.g. stochastics during a breakout) ought to be embodied in fuzzy propositions, which is where the system's actual trading expertise resides. This is seen to be a particular problem for small profit targets (e.g., scalping during a period of chop, or range trading a channel), where any method that generates random entries or range trades the price histogram can profit if sound money management principles are followed, given some will argue "any currency trade is a range trade".

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #33 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

For a little comic relief, 2 images below illustrate the one trade I made so far today (while busy with other things :-/)

What interested me about the trade once it started to go south was that it provided an opportunity to study what I hypothesize to be "impulse response" price action (that a lot of people describe in Fib terms) after a relatively strong move across a significant S/R (in this case the floor pivot).

The first image shows the state of the trade just as price is about to hit an adjusted profit target, after a second entry to haul the trade out of the fire was necessary, when price unexpectedly ran back over me (twice). Stop loss was moved a couple of times and then out of the way altogether.

While I should not have been trading at all (distracted, trying to do 3 or 4 things at the same time), or expecting price to cross a significant support level to reach my (single) target or trading from the short time frame chart (200 ticks) the decision to make a 2nd entry took place only after I'd managed to focus and consulted longer time frame charts. Generally however this tactic (averaging) is a good way to score a full boat loss.



And as usual, what price inevitably did ...




[Edited to add] 2nd (spite?) trade, underway as we speak. Trading for pocket change at this time of the day for practice, since there is little or no energy in the market. Aside from spite (to get a piece of what I missed) this trade was entered because there is a decent resistance level fairly close by above, trend (50SMA) on 200 and 600 tick charts is down, momentum is negative and 200 and 600 tick stochs were peaking at the time of entry.





OOPS........darn again..........200 Tick chart at the close....gambling pennies missed the close (penny wise pound foolish as they say). All orders are GTC so the thing will wake from the dead in 15 minutes Will follow up once it concludes.

Note this is an example of how the NT/IB combo can go haywire if a trade is active at the close (apparent target has jumped in this case, and very often the apparent entry does as well )

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #34 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Preamble

Since I last posted I've done some trading mostly on paper, continuing to gain confidence with the method, in particular how indicators behave when a trend turns into consolidation and vice versa (unfortunately at the moment not enough hours in the day to post these and discuss). In general end of week results are break even to respectably profitable on hundreds of trades.

Otherwise have been spending every waking hour, and some sleeping hours it seems, working on bot implementation details, where (as they say) the devil resides.

This entry will be at first a written summary of notes from my pen and paper journal, scribblings on backs of envelopes and rough diagrams the bot code is rising from, but intend to supplement with illustrations later

State of development

The main stumbling block has been how to cope with the volume of data comprising the knowledge base (the probability densities) that needs to be preprocessed, reduced, interpreted and made available as an indicator or to the bot during operation, which volume is proportional to the number of parameters being considered. While a minimum number of parameters derived from indicators particular to the method (i.e., trend via 50MA Slope, momentum, cycle, S/R and fractal) must be included, ideally the bot should also embody sometimes hard to define nuances of price and indicator action that optimize entries and exits to minimize MAE and maximize ETD.

After puzzling too long about what these nuances might be, to get things moving I decided the top priority at this stage is to develop data management and interpretation tools and to write a basic API (application program interface) in order to get the system off the ground. Once NT has been interfaced and is producing results then performance & hence final parameter selection will become the main focus.

Therefore the parameter vector is restricted to 16 variables (including first differences as a proxy for "nuance" but still excluding consideration of S/R levels) at the moment, which generates a large enough knowledge base to guide development but not so large as to be unmanageable. Characteristics of the implementation at present therefore are as follows:

1. 3 fractals corresponding to 200-, 600- and 1800 tick bars
2. 4 profit targets (8, 16, 32 & 64 pips)
3. 16 parameters (indicator outputs and their first derivatives)
4. 120 2D probability density functions (256 x 256 = 65536 density values) per fractal, per profit target, for a total of 1440 2D (256x256 = 65536 values each) density functions
5. Total non-zero density values distributed among the density functions: approximately 54 million based on a 3 month price data sample over 3 time/tick frames.

Implementation details

At first the idea was to store the knowledge structures (density dictionary & its support structures) in a database (MS SQL 2008 Express) and have the bot interact with them as required via calls to a DLL, which in turn would either interface directly to SQL server or with it via a Windows Service. It soon became apparent however that a simple SQL server installation is not equipped to deliver the necessary volume of data at adequate rates (i.e., 96 minutes simply to initialize the data structure in memory reading from database). Therefore preprocessing now includes a step to serialize the density data structure as a binary file, reducing load time from an hour and a half to the few seconds required to deserialize a 90 megabyte binary file on disk to a structure in memory.

Also in the interest of performance--necessity, not elegance, the mother of invention--I scrapped the too-clever interleaved density dictionary keys in favour of ad hoc keys constructed by concatenating pertinent variables. That is, the density dictionary is still a list of projections of points comprising a (potentially) 60 dimensional probability object onto planes defined by parameter axes taken 2 at a time, but the dictionary key is of the form

ftttXXYYMMMNNN

where
f = 1 digit identifying the fractal (0, 1, 2) corresponding to 200-, 600- and 1800-tick data respectively
ttt = profit target derived from the ZigZag indicator used to filter the raw data into buy/sell clusters underlying the 60-D probability "object"
XX = index into 60 item array of potential parameters, hence associated with (what I imagine as) an axis of the 60-dimensional space and corresponding to the first of 2 parameter axes defining the plane upon which a point in 60-D space is projected
YY = index into the same 60 item array of potential parameters, also associated with (what I imagine as) an axis of the 60-dimensional space and corresponding to the second of 2 parameter axes defining the plane upon which a point in 60-D space is projected
MMM = coordinate of the 1st parameter (identified by XX) along the XX axis
NNN = coordinate of the 2nd parameter (identified by YY) along the YY axis.

Testing and the prelude to interface design

Once the system was able to project a point (selected by a parameter test vector) in the 60-D probability object onto selected planes the ability to read & synchronize test vectors from disk 3 fractals at a time was added, so that it became possible to play historical data from disk through the system as a precursor to data being delivered by NT in real time.

At the same time a means to visual system activity was conceived as an array of 12 (3 fractal x 4 target) 60x60 pixel bitmaps, the color of a single pixel corresponding to the probability value associated with the projection of a test vector onto a given 2D probability density function. In AI terms we might humour ourselves by interpreting the changing pattern of colors of the bitmap assembly as each test vector is processed to be a view of sensory neuron activity as our proto-bot reacts to price action. But mainly (as mentioned before), visualization aids debugging, gives a better idea of what the system is doing and helps decide how to proceed next.

After visualizing system response to corresponding chart price action for a while 2 things became apparent; namely,

1. there was an encouraging observable correlation overall of buy/sell probability and price extrema, with exciting across-the-board consensus when a long term trend was setting up
2. there was sometimes discouraging randomness in between, including apparent occasional inexplicable breakdowns in correlation.

Following the logic that progress is made only by explaining anomalies, and regarding Item 2 (the discouraging part), as useful as it may be the bitmap assembly represents 1440 variables whose values are constantly changing and is hard to read during chop. In at least one case I caught myself comparing a system snapshot with the wrong part of the charts. In addition, each pixel color was finally & deliberately pinned "hard buy" (blue) or "hard sell" (red) no matter what the actual probability of a buy or sell condition, to make them easier to see to aid debugging.

The importance of Item 2 is that it suggested the next step; namely, to give the 1440 individual probability values represented by pixels in the bitmap assembly voting rights, whether to advise the fuzzy logic master logic to enter a trade, how long to stay in a trade, trade direction as well as position size a matter by some means the will of the majority. Rather than suffer a democracy however (one node--one vote) it seemed more appropriate to assign each the same rights as newbie traders in a prop shop, and discipline them according to performance.

Such a node's activity is constrained by rules and consists in entering a paper trade when probability of profit exceeds a threshold, maintaining position size commensurate with evolving probability that the direction of the trade (long or short) will be profitable after commission, accumulating profits and losses as the position is liquidated accordingly and voting according to its "beliefs" in "elections" that influence real-money buy/sell/hold decisions made by the fuzzy logic "expert system".

As a consequence the following structure & set of rules was devised to aggregate system response in a form accessible to the fuzzy logic system--the real-money buy/sell/hold decision-making component--as well as to permit adaptation & learning:

1. The construct handling each of the 1440 probability values (each microcosmic assessment of price action, whether it represents a buy, sell or hold condition within some scope or context) was abstracted as a node in a larger decision engine and assigned an activation function & system of weights, similar to neuron function, bias & weighting in a neural net (or constraints on a trader in a prop shop)
2. Each node was further assigned membership in 3 peer groups: fractal (its time frame trading peers), target (its profit target peers), and "global" (the system)
3. Each node is free initially to enter or exit a "paper" trade, or modify its position; any entry, exit or adjustment intended to maintain a position consistent with the evolving probability that the action will generate profits or preserve capital.
4. Each node maintains a cumulative P/L record of these paper trades.
5. Each node adds its voice to consensus (votes for buy, sell, hold) by making its state available during a poll at each bar close, including position status & cumulative profit
6. Each node reports results immediately after each private paper trade (as explained below), and accepts the consequences of its actions via modifications to its decision making process, including its assumptions (i.e., the way the underlying probability is interpreted)

The node-level "paper trail" concept was adopted both as a means to weigh a node's input into the real-money trading process and to track the contribution of a given parameter pair to the performance of the system for structural audit purposes.

Thus while each node operates autonomously the scope of its activity and its influence on others must be regulated as a consequence of its actions, and subject as well as the opinions of its peers and of its "boss" (the overall system). Behaviour is regulated by a simple weighting mechanism intended to reward decisions that increase profits and discourage or reverse habitual decisions that decrease profits. In general

1. nodes that habitually do well relative to other nodes in its peer groups tend to carry bigger position sizes and their vote carries more weight. Successful nodes are able to discount the opinions of less successful nodes and even an idiot boss (a system that exhibits overall performance inferior to the node). Ideally by rewarding success system behaviour ought to come to mimic successful node behaviour.
2. nodes that do poorly,
2.1 first, to reduce negative impact on profitability:
2.1.1 develop poor confidence (reduce position sizes & trading frequency)
2.1.2 have the weight of their vote reduced (ability to influence others and real-money decisions)
2.1.3 must pay more attention to the opinion their peers regarding entry decisions and position sizes
2.2 second, to correct behaviour:
2.2.1 have their assumptions examined (i.e., the construct generating the probability underlying their decision making is modified, even to the extent of reversing the buy-sell dependence on the underlying probability)

Finally, at the "atomic" level each of the 65536 values comprising the 2D probability density functions corresponding to active parameters selected 2 at a time are assigned both a weight and a bias, which are modifiable under program control. This was done to provide a way for the system to mutate so to speak, beyond simple adaptation. Depending on the algorithm the possibility exists for the way the density function is interpreted (what the system "hears") to differ significantly from the original composition ("what the probability function says").

Notes

As mentioned I'll try to add illustrations and details of critical algorithms (for e.g., weighted entry thresholds and position sizing) as they become available.

The system stage described tries to blend basic trading principles with simple neural net design & training practice. If so it's unavoidable and convenient: the concepts are trivial to implement in code and yet the approach avoids the frustration of generic trial & error network design.

Given the number of degrees of freedom implicit in the weighting system I'm curious whether the system will stabilize in a desirable (profit-generating) state. IMO if the underlying trading method has a positive expectation (unlike e.g. gambling) then it should.

Theoretically in this design nodes (representing pairs of parameter choices) can become inactive, whereupon a "structural audit" will be indicated (pruning, or substituting other parameters).

Despite continuing encouraging signs there is still no reason to believe the system will work. The first indication should come once the algorithms (paper trading, voting, adaptation) described above are finished, debugged and tested. My wildest desire is it turns out better than flipping a coin :-/ If nothing else I've learned there's a big difference between sitting behind a desk wearing pretty clothes and telling software architects & engineers what to do (my previous lot in life), and doing it one's self.

Edited to add: In the cold light of day after a few hours sleep it now seems it may make no sense from a real world trading perspective to keep records ("paper trade") at the node level, since that would amount more or less to trading 2 indicators--a bare minimum for this (TopDog) approach and could introduce a negative bias that unfairly penalizes the node. It may be trading has to occur in the subset of nodes defined by the intersection of fractal and target peer groups--a team of 120 nodes, which might suggest a simple neural net and use of backpropagation to "discipline" individual nodes. Not much of a compromise since net architecture would be predefined--perhaps no messing around to find an optimum structure. In any event concept needs more thought.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #35 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

This weeks' trades so far

It's easiest to comment on one's trading activities when things are going relatively well, and noticing I'm sitting at 95% profitable trades for the 1st 2 days of the week (21 trades) will use this opportunity (taking a break from system development) to do so

I've been using a small account (typically $6000-$10,000) and trading 80,000 unit lots, usually no more than 2 at a time (odd lots possible with Interactive Brokers as long as the post leverage USD amount is at least $30,000) because something about this account size seems right--perhaps short time frame, small pip targets = small account. Mainly I've adopted a per-trade risk 2% or less and because of an understanding with my significant other a 2% loss will cost us a bottle of scotch rather than several months groceries. Leverage is approximately 29:1. This may pale compared to 400:1 available from some Forex brokers who cater to small accounts, but with a background in stocks it seems more than adequate to me.

As mentioned previously I've traded EUR/USD spot currency exclusively for a couple of years now so as not to be distracted by issues with other currencies, in the hope that its price action would become second nature--hypothesizing first it's possible to learn price action instinctively and second it is experience alone that allows us to master a currency, that it's better to master one currency than be a "jack of all currencies, master of none". IMO short time frame spot currency is an excellent test of the hypothesis since initially at least the market can seem frustratingly unpredictable.

I'm using Barry Burns' (TopDog) chart trading method, the claim to fame strictly that it happens to agree with my temperament, not because I think it's superior to any other method, and because it's cheap (invested approximately $700 in materials, not all of which were necessary). Some will argue any money spent on information that is freely available and essentially rehashes what we already know (or should know) is wasted,--like paying for a diet plan--the teacher/self taught debate. I think we teach ourselves in any case and any guru should be the first to admit that--but IMO however a method pays for itself if it speeds up the learning process, gives what we know a little form and structure, puts 1000 facts into some sort of context, especially lets us understand exceptions to the rules (e.g., when to trust Stochastics and when not), and thereby lessens the number of costly mistakes during apprenticeship.

The underlying principle is the same as any other method--enter high probability trades or be prepared to suffer the consequences--and the characteristics of high probability trades are simple:

1. trade with the trend, (longer term trend when trading ranges, which IMO together with short term price action encapsulates sentiment)
2. trade with momentum
3. trade with the cycle (e.g., enter on retraces)
4. enter & exit at or near S/R levels
5. for perspective bracket your medium time frame chart with longer and shorter time frame charts (BB likes factors of 3 for reasons that may sound vaguely new age but that's what I do)

In addition to knowing what the rules are, to trade the method successfully one must

1. follow the rules (easier said than done, since it means mastering our impulses and overcoming bad habits)
2. trade as much as possible to learn what the rules actually mean in light of price action
3. try to avoid thinking we "already know all this stuff" until we are consistently profitable
4. never quit, keeping in mind that when we fail it's our fault, not the system, and that we can overcome our faults with experience.

It's not necessary to use indicators to trade the method manually if you have "the gift", either by birth or extensive experience. I use them because bots require them, and I find it helps to think like a bot when writing bots. Edited to add: as a retired executive with a Type A personality, I began trading believing the market would "implement the memo" I sent, meaning make me profitable. It turns out the markets don't get memos. Instead it reduces all of us, we start out all over again in the mail room. Therefore how long it takes to become profitable depends on how long it takes to accept that, and how hard one is prepared to work to begin a new career. In my case extra detention to unlearn everything I thought I knew before learning a bunch of new stuff.

To summarize, the 2 images below show chart setup (last few trades) and NinjaTrader Account Performance Summary (excluding commission of $2.48/order) for this week's trading so far. MAE in the Account Performance summary is nothing to brag about :-/ The buy (blue) and sell (arrows) on the charts more or less correlate with indicators at the bottom of each chart and S/R levels as might be expected (buy when MACD is positive and or rising and Stochastics are bottoming, sell when MACD negative and/or falling and Stochastics are peaking, backstop with S/R) and can be deduced as entries and exits from the simple fact the one unsuccessful trade is not shown (wandered away from the computer while a trade was active--bad, bad idea).

I depend on magic numbers (00, 50, 90 and any others that materialize out of thin air) and Murrey Math for near term S/R levels, have come to respect Ultimate Support/Resistance when it appears at the same price on 200-, 600- and 1800- tick charts. Longer term I keep an eye on Prior Day OHLC and pivots (PP, S1, S2, R1, R2), which more often than not seem to become magnets at certain times of the day.

I expect corresponding price action when longs and shorts bail at the end of a session (e.g., Europe closing during the US market hours).

Other than the charts I keep an eye on real time DOW Industrials for departures from correlation when the US market is open, since differences in behaviour tend to be more informative than agreement, with the assumption these days the DOW & EUR/USD are in sync at the open (i.e., the open is built into EUR/USD at that moment) and tend to track each other for the first 30-60 minutes. For example, in this time if the DOW decides to close any gap up or down EUR/USD tends to follow. I also keep a TV tuned to CNBC during US market hours for news that might explain (or rarely, forewarn of) significant price movements but ignore the commentary almost 100%.





System Development

Just a note--for the time being I've decided to go with the neural net approach to fuzzifying probabilities generated by mapping parameter vectors onto each of the 1440 probability density planes. The structure of the net derived from the planes comprises several layers, the first (input layer) being 12 groups of 120 input nodes (neurons) accepting a raw probability (0 - 1, that a buy or sell condition with specified profit expectation is present for a given fractal), each group of 120 nodes defined by a common fractal and profit expectation. I may adopt e.g. a sigmoidal activation function for these inputs and weight the outputs of these input layer nodes as usual, but IMO to be summed for input to nodes in subsequent layers must be combined according to the rule for combining disjoint probability estimates that a given event will occur; i.e,;

 
Code
SUM = (P1 * P2 * P3 * ... * Pn)  / ( P1 * P2 * P3 * ... * Pn + (1-P1)*(1-P2)*(1-P3)*...*(1-Pn))
where the Pi (i=1,....,120) are input probabilities and SUM is the consensus probability that the buy/sell condition is in fact present.

Edited to add: Forgot to mention the one major achievement reflected by the last couple of days' earnings: keep profit expectations small. Double the investment and place the first profit target inside the closest S/R level. Listen to your inner voice when it comes to distributing the remaining targets at S/R levels further removed, and be aggressive about preserving capital.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #36 (permalink)
London, UK
 
Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker: IB/IQFeed
Trading: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 471 given, 786 received


bnichols View Post
I expect corresponding price action when longs and shorts bail at the end of a session (e.g., Europe closing during the US market hours).

Pardon my ignorance but what would that corresponding price action be?

I look at the Europe session start and wonder what is going on there a lot - I think it's a well known phenomenon that the market often goes to either the Asian high or the Asian low and then reverses the other way for the rest of the session. I sometimes speculate why and I think maybe it's the Japanese closing their positions out.

Is the same sort of idea you are thinking about for the European close? What time would that be? 17:00 London time?

You can discover what your enemy fears most by observing the means he uses to frighten you.
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #37 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Adamus View Post
Pardon my ignorance but what would that corresponding price action be?

I look at the Europe session start and wonder what is going on there a lot - I think it's a well known phenomenon that the market often goes to either the Asian high or the Asian low and then reverses the other way for the rest of the session. I sometimes speculate why and I think maybe it's the Japanese closing their positions out.

Is the same sort of idea you are thinking about for the European close? What time would that be? 17:00 London time?

Hi Adamus! Thanks for calling me out on this one So far for me the issue of what explains market movement is still a matter of debate and I appreciate your observations about the European market open--will look for that.

For me European market close at present is 11:30 EST (12:30 AST--my time, or 16:30 GMT)

I crave explanations of price action as much as anyone but have come to think of explanation as a crutch--or at least a bad habit--and I should try to avoid contributing to the problem. I (perhaps all of us) interpret explanations as predictive of market bias or sentiment under certain circumstances even though experience has taught me otherwise (i.e., I've lost a lot of money predicting what price will do before it does it). These days my only edge is recognizing when price action has passed some point of no return, or momentum & stochs some point of capitulation at a price extremum, and capitalizing on it before it's too late. All I should have said was "I'm wary of the European market close".

In this case, what I meant is when any price hiccup associated with the last 10-20 minutes of the close of the European market is in the opposite direction of the general trend, I suppose it's a consequence of day speculators exiting positions consistent with the trend (e.g., shorts exiting a downtrend buying the Euro and selling the US dollar) and assumes speculators are present who en mass or otherwise have positions significant enough to move the market. If so, the movement would be more pronounced on Fridays, when the effect is combined with weekly position holders exiting positions. In fact any such move could be due to any number of reasons, including my obsession with movements of 10 pips or more about that time, and at end of the day (literally) the reason matters not--either the market moves and we're in it on the right side of the trade or we're not (or it doesn't move ). This prejudice of mine may be inherited from my stock trading days. The bottom line is--as you already know--be prepared for price movement when markets open or close, and brace for disappointment when they don't

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #38 (permalink)
London, UK
 
Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker: IB/IQFeed
Trading: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 471 given, 786 received

Wasn't trying to catch you out, just asking what you meant. I have never even looked for repeated PA patterns at the European close, although I will do now!

You can discover what your enemy fears most by observing the means he uses to frighten you.
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #39 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Sorry -- "calling me out" was a probably poor choice of words Suspect my communication skills are getting rusty day after day talking only to my computer screen.

Interestingly after all that I got slammed today by exactly this--the rise in EUR/USD in the 45 minutes prior to the close today when price made a ~ 50 pip beeline for the floor pivot after trading downward most of the day. I was short at the time, not an issue usually since I know how to recover from such a situation--just have to be quick. This time I was groggy (party with family friend last night :-/) and simply did not react, made every rookie mistake in the book, immediately quit trading for the day.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #40 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Quick update regarding system development.

Having given more thought to the notion of using a neural network to condition ("fuzzify") inputs to the bot's fuzzy inference engine an issue arose that has awoken a bigger issue.

Briefly, the neural net approach came up because I decided it would be a good idea to automate the process of updating the system's "knowledge base"-- the collection of probability density functions derived from relatively current price action--and if update its knowledge base then why not update the way it interpreted the knowledge? There are well documented ways of doing this in the Artificial Intelligence literature, neural nets one simple and time honoured means.

Training a net e.g. by backpropagation requires adjusting internal weights to distribute the difference between desired outcome and outcome produced by the network in such a way as to minimize the difference. Implicit in this is that we know what the desired outcome is.

In the real world--trading--there is a big difference unfortunately between desired outcome and actual outcome.

Regarding the desired outcome, this project started out naively, supposing we could combine a number of probability estimates derived from a snapshot of price action (more precisely, a snapshot is a measurement of a number of parameters derived from certain indicators at a moment in time) to generate profitable order signals. Beyond concerns the selected parameters were necessary and sufficient it also bothered me that by definition a snapshot may not necessarily embody price dynamics in the moments approaching that magical "point of no return" (commitment to a direction with some amount of momentum, corresponding to an entry or exit), even if the approach may capture the context in which price action is occurring.

In other words, time is not a factor in a snapshot, and the "desired outcome" was conceived to be simply some combination of probabilities that conditions indicating a buy or sell with an associated profit expectation were present (inherited from filtering with the ZigZag filter). This is why first differences are present in the parameter vector (as the inventors of calculus might agree they are a convenient way to take a snapshot of velocity--rate of change of price--hopefully a measure of the "nuance" referred to in a previous post).

In the beginning I hadn't given any thought to how such a system might learn and adapt--it didn't seem relevant since the learning was embodied in the probability density "knowledge base".

The problem now with asking such a system to learn is it takes time for price to move--to reveal the "actual outcome"--either wiggling it's way toward a profit target (success) or stop loss (failure). Over time, as traders, through trial and error we want to reinforce decisions (hence actions) that lead to success and discourage decisions (hence actions) that lead to failure, and we should expect nothing less of the bot. Otherwise it will repeat the same action based on its probabilistic input over and over, expecting the same outcome, which may bear no resemblance to the actual outcome--one definition of insanity and perhaps why traditionally we're wary of robots.

This last description is characteristic of a class of AI devices known as automata, that have little or no memory of the past and have limited ability to learn.

The problem we face here is, if we're going to assign credit or blame to any data point in the knowledge base, or decision, or action that led to a given position (long or short some number of units), we have to associate the point at which the prediction proved successful or failed with the data, decision and action(s) in question no matter how long it takes. As they say in politics--we need a paper trail that allows us to finger-point.

On that topic, having given over my ability to earn a living by accepted means to my ability to predict price many years ago, I'd acknowledge the contribution of taxpayer funded academics who still maintain it's impossible, that price is random. Put your money where your mouth is--I'll take the wager. At least I die honourably

A somewhat more thoughtful approach to the machine implementation takes into account decisions and actions in the past that can be associated with a present successful or unsuccessful outcome. Marvin Minsky and others pioneered work on this in the 1950's & '60's with the notion of reinforced learning, although developments have ebbed and flowed since then with the availability of funding. More recently Richard Sutton (PhD 1984, "Temporal Credit Assignment in Reinforcement Learning") put more effort into it with a series of papers, including "Learning to Predict by the Methods of Temporal Differences" (1987) and "TD models: Modeling the world at a mixture of time scales" (1997), whose results I may adapt. Still reading.

The bottom line is straightforward neural nets fill the bill--can be used for prediction of future events--if we're prepared to finesse them, but I've finessed enough neural nets to want something different this time, and for once have the time to look at alternatives.

In other news, will likely adopt a softmax activation function for the nodes rather than the sigmoid function, since upon inspection the softmax function more likely preserves inputs as posterior probabilities and we're dealing with probabilities, assuming we can treat buy/sell/hold conditions as categories in AI parlance.

In still other news, stumbled across Mahalanobis distance as a better alternative to the pseudo Euclidean radius function I've been using to create the probability density function. Amazing to me that if you can conceive such a thing someone's already worked out the details. This means I have to go back and recompute the density functions--no big deal but wishing I still had an assistant

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #41 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Trading EUR/USD just now I made a few bucks using the wisdom I gleaned from the set of the jaw of some trader I saw on TV sometime ago--"most of the time a range trade does not break out." In fact the range did not break out. In a previous life I would have bet price went lower.

This perspective assists one of my past endeavours--to get to the bottom of "since I'm losing all the time should I do the opposite of what I do, and how do I know what the opposite is?"

The answer is: practice. Try conceiving the opposite of what you think and trading that. This would be one instance where paper trading saves you money.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #42 (permalink)
London, UK
 
Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker: IB/IQFeed
Trading: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 471 given, 786 received


bnichols View Post
Interestingly after all that I got slammed today by exactly this--the rise in EUR/USD in the 45 minutes prior to the close today when price made a ~ 50 pip beeline for the floor pivot after trading downward most of the day. I was short at the time, not an issue usually since I know how to recover from such a situation--just have to be quick. This time I was groggy (party with family friend last night :-/) and simply did not react, made every rookie mistake in the book, immediately quit trading for the day.

Mind if I call you out again? Are you using floor pivots on the cash instrument? Are you sure that works, especially when the difference between the future and the cash is large?

Hope the party was fun.


bnichols View Post
In still other news, stumbled across Mahalanobis distance as a better alternative to the pseudo Euclidean radius function I've been using to create the probability density function.

Sometimes what you say goes over my head by about a mile, i.e. you write stuff and I think "how come he's worried about that when all I can manage to worry about is whether the last candle closes up or down?" Anyway, you mention probability density functions a lot. Can you describe in one concise sentence what a probability density function is?


bnichols View Post
As an aside--if the "last edited" footnote at the bottom of each post continues to give me 1 minute to update a post I'm going to quit this forum in a couple of posts. Drives me freaking nuts. Not sure why.

That is one of the compromises of public forums. It's so you can't go back and edit your posts when you're in mid-argument with someone, at least not without leaving evidence. I think it's designed to cut down on trolling.

You can discover what your enemy fears most by observing the means he uses to frighten you.
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #43 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Adamus View Post
Mind if I call you out again? Are you using floor pivots on the cash instrument? Are you sure that works, especially when the difference between the future and the cash is large?

Hope the party was fun.

I trade spot exclusively. I tried keeping a futures window open for comparison but was surprised to discover futures hath mysteries unique unto itself--totally distracting. So the floor pivot refers to cash only. Otherwise like wearing 2 watches--never know what time it is.

Party was great! No arrests this time.



Adamus View Post
Sometimes what you say goes over my head by about a mile, i.e. you write stuff and I think "how come he's worried about that when all I can manage to worry about is whether the last candle closes up or down?" Anyway, you mention probability density functions a lot. Can you describe in one concise sentence what a probability density function is?

Think of the mice that have infested my house this winter. Collectively the population is large, but the number varies from room to room, highest density overall in the vicinity of the kitchen. The probability therefore that a mouse will wander into a trap under the kitchen table is greater than the probability a mouse will be caught under the couch in the living room and so far experiment verifies that. I suppose that's several sentences.

Regarding the learning math topics I discuss, may seem esoteric if one hasn't spent the better part of one's life puzzling over them I guess. On the other hand accounting principles that seem transparent to my wife are totally opaque to me--e.g., how to balance a cheque book--never studied them and shied away when given the opportunity. Somewhat like the intricacies of taking a bus--wouldn't hazard that.

Edited to add: Seriously though--I don't think I'd ever heard of Mahalanobis before last night--have to admit the name rolls off the tongue--and never realized I needed his metric until this moment, and that's a month after I was constructing the density functions, pondering what was the simplest way to record the distance of a test vector from the centroid of non-circular distribution. Reading the description of the metric it was obvious he had precisely this problem (of cluster analysis) in mind. BTW--I try to stick to jargon that gives a minimum 500,000 hits on Google


Adamus View Post
That is one of the compromises of public forums. It's so you can't go back and edit your posts when you're in mid-argument with someone, at least not without leaving evidence. I think it's designed to cut down on trolling.

You read all the way to the bottom of the post! I'm not used to arguing on this forum but would look forward to it--all I have is Forex charts for company. I've taken what you said to heart and deleted those sentences. Fit of hysteria that will pass after a nap, no doubt.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #44 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Since the last post I've been trading short time frame spot EUR/USD manually on a small ($6000) account with mixed results--more or less break even--and working non-stop on the bot.

The good news is I have little or no fear anymore of blowing out the account, but having overcome so many obstacles have yet to overcome what I assumed would be a short lived newbie problem; namely, after a relatively large number of successful trades losing everything and more in a few missteps. As usual, resolve returns once our backs are against the wall. This survival instinct may be our problem (we can't function unless we're in crisis) or it may be the key to our salvation.

The issue with manual trading continues to be that I drift off mentally and yet continue to trade. I suspect this is mainly because I'm still focused on gaining experience (to put into the bot, I tell myself) and less on making money (& keeping it), because it's becoming clear the losses occur on low probability trades (50% or less chance price will move in the "desired" direction, when in fact what we desire has nothing to do with it).

If price is not moving, momentum gone to zero, especially when we expect price will not move for some time to come, let's not anticipate a breakout on no news.

In other words while I know the difference between trading and gambling, when the market slows I have an inclination to gamble, whatever the underlying psychological cause. The market is not a casino and it is not influenced by any move I may make, and yet when the action dies I start to behave as if it were. Having written that down at long last I may begin to deal with it.

Regarding the bot, I've taken some time off from straight implementation of the 2D probability based strategy to study how it can learn, which turns out to be more involved than I first imagined. (Reminds me of the observation that if at the outset we had any real idea how difficult a given course of action might prove we would never take the first step )

The approach I took to bot learning ("adaptation") was borrowed from neural network back propagation, which eventually led to wondering if the inputs themselves should be weighted. This notion of weighting inputs probably sprang from a "eureka moment" in which I realized the buy/sell signals being generated by a prototype strategy based on the buy/sell clusters were more often than not exactly backwards. In other words, it finally dawned on me I was mixing (confusing) short/long signals with buy/sell signals respectively--the probability density based system was telling me buy/sell signals are rare events that emerge from short/long conditions, respectively, and that overall the likelihood represented by the e.g. buy cluster is that price is in a downtrend, not that it is a reason to buy.

In still other words, all the 2D PD system so far is capable of is reminding us when trading a given time frame one buys only after price has declined "enough" (objectively speaking), and sells only after price has advanced "enough". What any 1st grader knows. All the probability density clusters identify is short/long conditions from which buy/sell signals emerge.

Finally, since the clusters have (essentially) been low pass filtered (smudged) there may be no effective difference between a given point (initially representing an isolated vector representing a buy condition, say) in the "buy cluster" and neighbouring vectors representing "remain short conditions".

Believing we could perhaps "fix this" by adjusting the weights (or, the "meaning') of the 2D probability density functions input to the system I wrote a number of neural-net based models that finally involved individual parameter (or "feature") vectors. We might recall the previous 2D probability density functions are functions of 2 feature vectors as ordinate & abscissa, so latest work perhaps amounts to 1D buy/sell clustering of an individual parameter (think of a bimodal line graph of a distribution), although I haven't yet clustered the 1D data.

IMO this step uncomplicates the strategy, since it's a more direct representation of the Top Dog system our system attempts to model.

The main reason I haven't spent more time on theory (e.g. 1D clustering) is I ran into a technical issue that's taken a few days to work around. The issue involves deserialization in C# of objects stored as binary files on disk by DLLs referenced by NinjaTrader. To make a long story short--so far it appears NT absolutely refuses to deserialize objects in DLLs referred to from strategies.

Regarding deserializing objects in external DLLs loaded by NT, I wouldn't mind advice at this point, since I've tried everything I know, bearing in mind I know more than a little. "Unable to load assembly" error in this case has been traced to the attempt to deserialize the neural network previously trained in an application outside of NT. Serialized objects helpfully contain a reference to the assembly (in this case, AForge.Neuro) that serialized them and even though test apps deserialize the net without complaint, NT refuses to follow. The usual cause of an "Unable to load assembly" in this situation results from mismatched write/read DLL's, and the usual solution is to write an overriding binary formatter binder to force the object to load the currently defined object. Unfortunately NT appears to shoot itself in the head prior to exposing this option.

That said, I wrote a method to store the trained network as a text file and built the interface to NT just in time for the long weekend. The good news is the strategy loads without incident-especially none of the weirder output I've seen from NT on this project. The bad news it's the long weekend and price is absolutely flat. Not surprisingly all the bot returns is "hold" on all time frames.

Note to self--bot should be saying "Flat".

"NN.cs.zip" is the Visual 2010 code to implement the current neural net, assuming you've already loaded AForge.net.

Attached Files
Register to download File Type: zip NN.cs.zip (276.6 KB, 20 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #45 (permalink)
London, UK
 
Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker: IB/IQFeed
Trading: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 471 given, 786 received

Well done with the work-around - I expect you will or have already come to the same conclusion that I have - that NT7 is good for strategies and automated trading - but not that good.

You can discover what your enemy fears most by observing the means he uses to frighten you.
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #46 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Adamus View Post
Well done with the work-around - I expect you will or have already come to the same conclusion that I have - that NT7 is good for strategies and automated trading - but not that good.

Have to admit I haven't had this much trouble with NT since the early days, when it finally transpired my anti-virus was causing NT to abend with all kinds of ridiculous errors. Unfortunately it's been my experience there's no point in contacting NT support about issues of this sort, except in the hope another user lurking on the site has been through the same thing and can offer a solution.

This morning, having spent all weekend getting rid of the what appeared to b causing the trouble (replacing .Net binary deserialization of neural net weights in the DLL with boiler plate text file load) NT is once again whining that it can't find the right version of AForge.Neuro.dll. Needless to say, NT has been restarted, reloaded, reinstalled--reinitialized in every conceivable way--and is being provided the correct AForge DLL version ; test programs using the custom DLL and the same AForge DLL work flawlessly.

Not quite at the hair tearing stage--have beaten NT before and will do it again--but right now to paraphrase that old cartoon, "Wish NT would give me the same answer twice"


Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #47 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Update--as many times before defanged the strategy by removing all references to the custom DLL, removed it from strategy references, wiped NT of all related assemblies, restarted NT and connected to the broker, ran the strat with no errors, quit NT, reinstalled the AForge DLL (and every DLL in the Visual Studio Release directory) in the NT custom directory, restarted NT, edited the strat to use the custom DLL and added the custom & AForge.Neuro DLLs to the strat references, recompiled and restarted the strat with fingers crossed and bated breath.

Lo and behold, strat runs with no errors. Remains to be seen if it's profitable--the whole purpose of this latest battle with NT.

Not sure exactly how NT gets its DLL knickers in knot but there has to be a pattern here somewhere.

While this strat is intended simply to test a hypothesis about fuzzy learning it is still capable of generating buy/sell orders, presumably with some sort of edge. If that turns out to be the case (i.e, if it performs better than a coin toss) I'll post the code.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #48 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Just a quick update before I hit the sack (up for 24 hours or so--apologies if the following doesn't scan)--system development bogged down running data through neural nets since the last post with not much to show for it--certainly nothing that might be profitable, code based on AForge.Neuro still being modified.

In the meantime I harnessed my real money manual trading to a slightly modded version of the stock NT SimpleMA strategy running on my paper account in the background on 1800 tick EUR/USD bars. The 2 moving average periods and stop loss were optimized on year-to-date data (26 bars, 60 bars and 24 pips, respectively). Backtesting suggests it should be marginally profitable and so far forward testing indicates that it may be (see illustrations below for details). It is not robust in the sense that results vary with the the selected bar (time frame) and it has not been tested on other instruments.

SimpleMA IMO is basically a momentum system, not much different than trading MACD (also based on 2 moving averages), buying and selling what would amount to MACD zero crossings (MACD(26,60,30) printed at the bottom of the chart in the illustration below for reference). The rationale is to put my interpretation of price action in some sort of context, which I find improves my objectivity & reduces stress, resulting in better entries and exits. The context here is the strategy's opinion of what price is going to do, whether or not the direction is clear to me and whether or not I agree with its opinion. In any event, the strategy is not sophisticated and has a rather significant drawdown partly because it is configured to produce a profit on average, whereas I trade specific events. If I choose to enter a real money trade based on its signals I can hedge my bets by exiting before it does if, in my opinion, it looks like the trade is ending sooner than it is geared to respond in a given case.

Also, if I enter a trade based on its signals and the trade goes south I can blame the strategy

At the moment the strat and the real (short) trade are up about 60 pips. Will close the real trade before retiring but leave the strat running on the paper account.

Figure 1. Current EUR/USD 1800 tick chart showing strat behaviour, 2 moving averages, trades at crossovers, and corresponding MACD below


Figure 2. Backtest result for the raw strategy. Real money manual trading results so far tend to be better because losses are cut early.


Figure 3. Cumulative PnL for raw strat

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #49 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Since the last post I've been trading the EUR/USD macro style (the way I started years ago, on political and economic developments) rather than based on short time frame technicals (the subject of this journal) simply for income, with a relatively wide stop and large multi-tiered targets. Vacation season after all is fast approaching.

I'm using NT at the moment because it's acquiring data I want to analyse, and except for a glitch last night (subject of a rant in the "NT Truth" thread ) so far so good. After 2 of 3 targets hit, at this point have to decide whether to stick to my guns, assuming the consolidation below (1.28) 50 during N American market hours will eventually break down in the Asian session, or take the money and run.

EDITED TO ADD: BTW, I will close the trade before end of day and reopen if warranted, because it's my ongoing experience GTC Forex orders don't transit end-of-day well--much wackiness ensues (namely, some hours later the entry point jumps a random amount in a random direction, throwing NT for a loop, even though IB accounting seems to come up with the correct P&L when the trade eventually is closed). No doubt someone more wise to these things can explain it but for now, even though I suspect it's IB's fault I'll restart both NT & TWS. As my dad used to say--"If I paddle you all I'll get the right one."




If price does hit the 3rd target then in the absence of good news from Europe expect EUR/USD to go much lower (200 pips?)

Regarding system development I finally resorted to R (open source version of S--the stats analysis platform) to research ways to improve neural net performance--in fact the performance of any data pre-processor. Writing modules for NT may be interesting but debugging is not as much fun as it used to be.

Mainly I've been using Empirical Mode Decomposition (EMD) to reduce various series, including indicator outputs, to their Intrinsic Mode Functions (IMFs), applying transformations (including prediction) to the IMFs and then summing to reconstruct the new time series. The hypothesis is IMFs as inputs, rather than the noisy series from which they're derived, may allow individualized model parameterizations and hence more robust predictions. The hypothesis is being tested initially by comparing forecasts produced by standard GARCH and Arima models applied to both raw data and the derived functions.

To illustrate the concept, here is a test decomposition run last Friday on the request of a colleague on the last 20 years of daily close prices for the TSX (Canadian stock market index).

Bear in mind such models + the price of a latte will buy a cup of coffee :-/

The first pic shows the original record as the reconstruction of the first IMF and 1st residual. Remaining pics show successive IMFs and their "optimum" Arima-based 60 day forecasts. The final pic in the series shows the TSX forecast constructed from intermediate results, 85% and 95% confidence intervals indicated by red and yellow shading, respectively.

NOTE: the period of the dominant frequency (days) shown at the bottom of IMFs 7 and up is wrong, possibly an aliasing artifact due to the quick & dirty & independent spectral method used to determine the period. This does not affect results.

Original data + 1st IMF and residual


1st IMF + prediction


2nd IMF + prediction


3rd IMF + prediction


4th IMF + prediction


5th IMF + prediction


6th IMF + prediction


7th IMF + prediction


8th IMF + prediction


Residue + prediction


60 day TSX Forecast constructed from IMF predictions by summation:



While wonderful to look at, as expected the Q-Q plot from a GARCH analysis on TSX returns (1st difference of log data) shows that TSX data density deviates from normal (familiar S shaped curve), which means among other things we should be prepared to predict variance as a function of time as well for raw price data:

Q-Q plot from GARCH(1,1) analysis of TSX returns:

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #50 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Short spot EUR/USD since last night (1.5 contracts). After visiting PP earlier now appears to have cleared (what I pegged as) support at 1.2815. Targets ~ 1.2745 and 1.2690 (Gann line corresponding to Murrey Math 0/8, "Ultimate S/R" on H1 chart)

Hour chart:


Edited to Add: Half hour before the close of the N. American session 1st target hit (changed to 1/2 contract from 1 contract initially) at 85 pips. Things remaining equal will exit the rest of the trade in about 20 minutes (1 contract at ~ 100 pips or so)

600 Tick chart:

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #51 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Should probably post before I'm counted MIA. BTW, the gaps in posts tend to correspond to periods when my thoughts about trading are consolidating, but also in this case as mentioned previously when vacation season is looming and I'm focused on earning the money to pay the cost, which (since investing is my sole income source) includes managing my stock portfolio somewhat more actively, irrelevant to trading STF spot Forex unfortunately. Finally, I've been investing quite a bit of time learning more about the R language and using it to study the statistics behind time series analysis, forecasting with Arima, Garch, and neural models applied to IMFs (Intrinsic Mode Functions) in particular. While so far results seem encouraging I hesitate to report until satisfied the techniques have some merit. **

At some point I'll do something with results so far from K-means clustering as a means of preprocessing feature vectors to feed the bot. In general the stumbling block remains that while entries (ideally at the exact point trend changes direction) do cluster, in practice these entries are born of conditions associated with the trend that is coming to an end. In other words, the clusters themselves are not sufficient to generate signals, and still plan to derive heuristics from a trading system (in this case BB) for the fuzzy inference engine that will process the clusters. As it sits a learning methodology for the bot is still up in the air.

Recently I reviewed the material I bought from Barry Burns with the intention of coming up with a better regime change policy (how to deal with the transition from trend to range trading and vice versa), and happily found what I was looking for. Hard to say why it wasn't obvious from day 1 (essentially, pay extremely close attention to momentum, that when wave counting "1,2,3..." is not the same as "A,B,C,...", especially stay immune to the associated head fakes). Overall the exercise reminded me that when learning how to trade (and probably more generally) we fool ourselves if we think we understand a concept the first time we encounter it, since in reality we learn iteratively, don't grasp vital nuances and subtleties until we're ready, have done the required ground work (which in turn calls to mind all those biblical references to seed falling on stony ground, pearls before swine, etc., etc. ) In other words, one thing trading has taught me is that if a system does not work for me out of the box I should not be too quick to blame the system.

Some basic trading rules the trip down memory lane (review of BB's material) reinforced for me include the following:

1. S/R levels are not cast in stone; they are certainly not "where price is going to change direction", but merely battle grounds where bulls & bears have at it. We can imagine ourselves to be military generals if it helps, whose tactics depend on how the battle for a particular hill is going but whose strategy for winning the war does not change imid-battle. From another perspective, placing an order before it's clear who the victor is (i.e., at market) is akin to betting on a horse race--certainly not trading (which, when I'm doing it right, is more like placing a bet once it's clear which horse is going to win barring him stepping in an unseen gopher hole )

2. Never place market orders, but instead use stop orders a couple of pips above/below ultra short term resistance/support, with momentum in our favour and timed so that stochastics rising/falling respectively, preferably from oversold/overbought levels.

3. Strictly speaking breakouts are low probability trades according to my understanding of BB's method. If it's going to break out we've already been in the trade for a while, starting to take profits just as others are piling in.

4. Job 1 is capital preservation, therefore (apologies to Desiderata) go placidly amid the noise & haste & missed opportunities and don't freak out when an initial stop is hit or whine when a trade exits at a trailing stop. Instead be glad you weren't on the wrong side of that 50 pip break to the upside from a dead stop--no doubt the universe is unfolding as it should. Rather, take comfort in your profits when they accrue, rejoice when a well executed trade consistent with your method captures in excess of 2 thirds of the favourable excursion, because when it starts to occur regularly it may no longer be dumb luck but a sign at long last all the hard work and strict self discipline are starting to pay off. Above all, be at peace with Momentum, whatever you conceive it to be.

5. Trade from one chart (probably the mid time frame chart), guided by the longer time frame chart but entering (and placing initial stop) from the shorter time frame chart.

6. Never trade against what the system (hence price) is telling you (e.g., thinking "this time it's wrong"). The system is never wrong, but until you've mastered it you are prone to misinterpreting what it is saying.

7. Job 1 is capital preservation (needs repeating), therefore take some profits at intermediate S/R levels no matter how close to your entry. If you blow the wad on nearby targets often enough, rethink your notion of what constitutes a setup, or learn how to get back in the game rapidly while the getting is good (which style if it becomes habit means commission costs are going to rise as a % of ROI). IMO the only reason to buy or sell into resistance/support respectively is because you're confident it will not hold, the potential reward if S/R breaks justifies the risk and you want a buffer between your entry and ensuing price action. More often than not however, you probably want S/R at your back.

8. You can't get blood from a stone. No trend and no momentum = don't trade, especially don't imagine our trade will cause the market to snap out of the doldrums. While I might hallucinate that my opinion matters as much as a national bank the market doesn't even know I exist, doesn't care, does not notice me stuck to the bottom of its shoe. If we want to bet on what on the face of it appears to be a coin toss (the "no-current-price-action-but-who-knows-what-might-happen-next" system), we ought to be clear how random entry trading systems work.

9. Practice, practice, practice in all sessions, and between sessions, until you are familiar with the tricks and habits used by your competitors, the effect of news, how price behaves in light of give market sentiment, and so on.

** For thrill seekers I've attached the most recent beta version of the strategy I use to extract and write the currently defined feature vector to disk (WriteZigZagRecord.cs) and the R language script file used to create experimental Arima models from EMD IMFs of daily stock quotes downloaded from Stockwatch.com (fstock.txt).

Notes:
1. fstock.txt should be renamed fstock.R (futures.io (formerly BMT) won't attach file types it doesn't recognize) and you should have or acquire a working knowledge of the R language to use it, including how to acquire the necessary packages (e.g., EMD, forecast, etc.)
2. WriteZigZagRecord.cs can be used to dump feature vectors in BackTest mode ("Use Historical" set to TRUE) or as a strategy in real time. I use the real time option when running experimental price prediction apps in parallel with NT, that detect that the feature file has been updated, load the latest vector and output an updated prediction.
3. I'm working on an OHLC neural net based one-step predictor in Visual C# and will post the VS project once it's more or less stable, whether or not it produces useful results, for comic relief.

Attached Files
Register to download File Type: cs WriteZigZagRecord.cs (20.3 KB, 14 views)
Register to download File Type: txt fstock.txt (5.2 KB, 15 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #52 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,780 since Jun 2009
Thanks: 32,306 given, 97,567 received

You can now attach .R files directly.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to Big Mike for this post:
 
(login for full post details)
  #53 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Many thanks Mike. If anyone has questions about the R code or the strategy I'll be happy to clarify.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #54 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Preamble

The OHLC one step predictor is more or less stable so will publish the project Rev 0.9 (experimental), no restrictions on use, as is, no warranties.

This program is a precursor result and stems from the main system under development, which will probably resemble ANFIS when done (see e.g., https://ece.ut.ac.ir/Classpages/S86/ECE406/Papers/ANFIS.pdf)

Code is single threaded--doesn't put that much of a load on the computer--but may multi-thread it in the future.

If you're still on dial up sorry for the number of pictures.

Performance is not bad. See Item 8 under Use, Fig 15, below for P&L from a simple next-day-buy/sell strategy based on predictions of the trained network applied to novel test data (data the network has never seen before)

Project files

The system was written in C# using VS 2010 Ultimate and .NET 4. To modify you should employ something similar, or dissect the project to extract what you need.

I've attached 2 ZIP files containing the Project itself and the Install (published) version for anyone who wants to experiment without using an SDK.

ETA: Also attached data file used in the example below (daily OHLC data for TCK.B on the TSX)

Required 3rd party (Open Source) DLLs (Zedgraph & AForge) are embedded in the distribution, but the DLLs and the entire development environments for these packages can be downloaded from the respective sources if desired.

Please let me know if you have issues with either file. My development system is virus free as far as I know but as with any object downloaded from the Web you might want to scan before use.

Fig 1. Opening screen:






Known Bug/Clarity fix list
1. Result files are saved with double extension (i.e., " filename.csv.csv"). This does not appear to affect programs that need to open the file (e.g., Open Office Calc)
2. No header row is written to saved results (e.g., from the Test tab, where Test vectors are saved along with Target and Predicted parameter values)--may have to guess what column is what for now.
2. On Train tab, "Input Vector" label should probably be "Training Vector" and training set should probably load when the tab is opened rather than after the "Start Training" button is clicked.
3. Some buttons need to be disabled (and perhaps enabled) more appropriately
4. Not sure if it's possible to get into trouble clicking buttons out of sequence--needs more error checking.



Wish list:
1. Graph of training error in real time.
2. Presently does 1-step prediction of one parameter (e.g., predicts close from prior day(s) OHLC data). Should have easy way to predict & combine predictions for all 4 parameters to experiment with multi-step forecasts.
3. More tooltips for new users




Use

1. Run the program and choose your data file (data must be comma-separated; e.g., .CSV format).
.......1.1 file format is as follows:
..............1.1.1 First line is header row.
..............1.1.2 Columns must include Open, High, Low, Close data column, but header label not important
..............1.1.3 Subsequent lines contain comma-separated values
Example:

Quoting 
<ticker>,<date>,<exchange>,<open>,<high>,<low>,<close>,<change>,<vol>,<trades>
ABX,19860902,T,0,18.88,18.5,18.75,0,131840,0
ABX,19860903,T,0,19.25,18.75,19.13,0,147309,0
ABX,19860904,T,0,19.13,18.63,18.88,0,152850,0
ABX,19860905,T,0,19.38,18.75,18.88,0,165012,0

.......1.2 Loading data is a 2-step process
...............1.2.1 Select the file via the File > Load OHLC Data File
...............1.2.2 Tell program which columns correspond to Open, High, Low and Close if necessary (default is Stockwatch.com format)
...............1.2.3 Click the "Load Data" button

Fig 2. File scanned. Identify OHLC columns by modifying dropdown box selection :


Fig 3. Data Loaded (after satisfied OHLC columns are correct) & "Load Data" button clicked








2. Select Training set
........2.1 Left click mouse on Data list and drag to highlight data. Right click on highlighted data to open context menu.
................2.1.1 Note that data is moved--not copied--from original data set to target set (can be put back; see Item 4 and Fig. 8, below).

Fig 4. Select training set data by dragging mouse, right click for popup context menu


Fig 5. Training set updated after "Add to training set" selected











3. Select Test set
........3.1 Left click mouse on Data list and drag to highlight data. Right click on highlighted data to open context menu.

Fig 6. Select test set data by dragging mouse, right click for popup context menu


Fig 7. Test set updated after "Add to test set" selected








4. Restore Data if necessary
........3.1 Move data between sets by highlighting items and left clicking mouse on either Training or Test sets
................3.1.1 Note that data is moved--not copied--between sets

Fig 8. For example, move training back to Input Data set or to Test set








5. Network Architecture
........5.1 Click on Network Tab after data sets configured.
...............5.1 Probably no need to change anything here until you're ready to experiment.
...............5.2 "Lookback" is number of prior OHLC samples to use in the prediction of the selected parameter, where the predicted parameter is one of next period Open, High, Low or Close--default is use only prior day OHLC values, maximum 10 IIRC.
...............5.3 "Load Previously Saved Network" applies if you've saved a network after a previous training run (see Training, below). This lets you resume training from a previous state, or run new Test data with a previously trained net.

Fig 9. Network configuration





6. Training
........6.1 Click on Train Tab after inspecting Network architecture
...............6.1 Set number of iterations if desired. 10,000 is probably too low for most applications.
...............6.2 Iterations can be updated at any point (even while training is occurring) by changing the number and clicking the "Update Interations" button.
........6.2 Click the "Start Training" button to get the show on the road.


Fig 10. Training tab



Fig 11. Training underway (after clicking "Start Training" button




7. Test
........7.1 Click on Test Tab at any point after Training has started (or is complete)
...............7.1 Click the "Start Test" button to see how Training is going (has gone)
...............7.2 When watching individual predicted value change in the Prediction list, scroll to and left-click on the corresponding item in the Test vector list or Target value list to sync position of all 3 lists to that line (otherwise Predicted list will jump to first item every time list is refreshed--every 500 iterations)
...............7.3 Zoom on graph by right-click and drag. Left click to unzoom (select Unzoom from popup menu)

Fig 12. Test run on test data set


Fig 13. Close up of Actual vs Predicted test values graph


Fig 14. "Closer up" of Actual vs Predicted test values graph






8. Save results if desired.
.........8.1 Clicking "Save Vectors" where available on a tab will write the data displayed in the listboxes to a .CSV file of your choosing.
.........8.2 Examine the saved data in a spreadsheet if desired
.........8.3 Present version doesn't write a header row to identify columns--sorry for any head scratching that may cause at this point
.........8.4 For example, Fig 15 shows the result of a "buy/sell at the open, sell/buy and the close if next day prediction is up/down over today's close" strategy done in Open Office calc using results of the test run. Don't often see a Cumulative P&L curve like that and much more work needs to be done to determine what it means.

Fig 15. Cumulative P&L for strat driven by 1-step prediction of trained network and applied to novel test data

Attached Files
Register to download File Type: zip TDOHLCProject.zip (1.11 MB, 32 views)
Register to download File Type: zip SimpleOHLCInstall.zip (518.2 KB, 29 views)
Register to download File Type: zip TCKB20120601.zip (32.1 KB, 29 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #55 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,780 since Jun 2009
Thanks: 32,306 given, 97,567 received

@bnichols, quite an achievement, thank you for sharing!

What are your thoughts as for adapting this for use with real-time data? Is that ultimately your goal, or are you looking for other types of uses?

I'm wondering what led you to develop this NN predictor?

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #56 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,780 since Jun 2009
Thanks: 32,306 given, 97,567 received

Also I really think you should post that as its own thread, Neural Network OHLC Predictor, or etc, so it can be easily found in the future

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #57 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Big Mike View Post
@bnichols, quite an achievement, thank you for sharing!

What are your thoughts as for adapting this for use with real-time data? Is that ultimately your goal, or are you looking for other types of uses?

I'm wondering what led you to develop this NN predictor?

Mike

As you surmise real time is a goal, and that's just a skip and a jump from here since I've interfaced trained nets with NT in the past--should probably post that interface as well.

The issue for me right now is whether real time (i.e., short time frame) OHLC data has different dynamics than daily data--others may have, but I've never quantified it (i.e., is what I perceive as noise actually data in another system of coordinates). While the issue as a trader is whether a net makes money, and the acid test is applying a given net to STF data and watching what happens, if it doesn't work I like to know why since often enough the fix is right under my nose. Hate to miss it. To date I've never built a net that handled STF price well enough to rely upon, but until now I've never had much time to devote to it. Never say die .

Regarding why the NN predictor, it's a test bed for ideas to be incorporated into a somewhat more complicated hybrid system still under development. I find daily data easier to deal with than 200 Tick, say, partly because the pace is slower, have all night to wonder about some development. I'd say I'm more into comfort than speed but more likely I simply don't think that fast :-/

ETA: Should add, wanted a decent NN predictor because anything is better than the Arima based system I'm using for swing trading the stock market, since non-stationary variance is a headache. Garch would likely improve things but I'm stronger in AI than traditional statistics.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #58 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Big Mike View Post
Also I really think you should post that as its own thread, Neural Network OHLC Predictor, or etc, so it can be easily found in the future

Mike


Roger--in Journals or elsewhere? An AI forum here? There is an Encog (AI related) thread under NT programming, but this is only marginally related.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #59 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,780 since Jun 2009
Thanks: 32,306 given, 97,567 received


bnichols View Post
Roger--in Journals or elsewhere? An AI forum here? There is an Encog (AI related) thread under NT programming, but this is only marginally related.

Not journals - otherwise it is really up to you, put where you think it goes best, but I think it deserves its own thread.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #60 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Big Mike View Post
Not journals - otherwise it is really up to you, put where you think it goes best, but I think it deserves its own thread.

Mike

Alrighty--will create a new thread under Platforms & Indicators, something like AI Platforms, if that's ok.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #61 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,780 since Jun 2009
Thanks: 32,306 given, 97,567 received


bnichols View Post
Alrighty--will create a new thread under Platforms & Indicators, something like AI Platforms, if that's ok.

Use a couple keywords people will search for, like Neural Network for example.

This way people can find it (including me).

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #62 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Head's up. Having stewed all night about the too-good-to-be-true P&L it finally dawned on me the problem is the way I'm normalizing training and test input and output vectors (code snippet grabbed from a filter routine is totally inappropriate for nets). The target is being included in normalization calculations, essentially giving the net a peek at the future--huge no-no.

Will fix normalization before the project is moved to its new home.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #63 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Since the last post working on software, swing trading stocks and using Barry Burns' basic trend trade method to trade short time frame EUR/USD.

Software in development includes the following 3 items:

1. NN package mentioned last time, which (after removing its ability to see the future) turns out not to be so amazingly profitable after all. It needs a few more tweaks before publishing. Might be useful for someone wanting to experiment.

2. A wrapper written in Visual C# for a hodgepodge of programs in PHP and the R language to download end of day OHLCV data from my favourite provider (stockwatch.com, likely no better than any other end of day data provider) for all instruments & indices on all N American markets as well as all FX pairs (32000 or so items total), collate to individual OHLCV files for each instrument/index/pair, run a few indicators & and Arima model predictor on the updated files and finally screen the results for potential opportunities. The program is still in alpha (incomplete, several annoying bugs) and at this point am focused primarily on the Arima predictor, in particular writing code to quantify performance.

3. An NT strategy to implement the highest probability FX trading method I know of (i.e., trend trading )--not hard to write--just tedious--and unfortunately still mostly scribbles in my paper journal. IMO Barry Burns' basic method is as good as any in that regard so I've been using it more or less exclusively since the start of the month to confirm I know it (so I can code it).

On that topic (trading BB's trend method manually), since the first of the month it's averaged approximately 50% successful trades and lately (last few days, once I manged to focus), successful trades of the last 24 approached 80%, bearing in mind that I do not take a loss if I can help it and perhaps 20%-30% of those "successes" are trades that were stopped out at +2 pips.

Once again in defense of BB, if necessary, while some opine he might have packaged freely available techniques that may be as old as trading itself (albeit with a few proprietary refinements), and may be peddling nothing more than common sense, alas for many traders, including me, common sense is not necessarily our strong suit and at least in the beginning any sort of package (and IMO BB's style of teaching in this case) that reduces the myriad techniques out there to basics and thereby filters out most of the noise is a blessing, given the cacophony of the trading industry. In fact I like the fact the system is so simple--it doesn't distract from price action and didn't waste a lot of time installing & learning how it works. Like all systems BB's basic trend trading system clearly defines high probability setups, which (once I quit trying to game the system) stopped my over-trading. [While there are methods to range trade and to trade chaotic conditions the ones I've tested seemed stress- and inevitably mistake-inducing, perhaps simply due to the nature of the market] Also, no matter how sophisticated a trader we might imagine ourselves to be, it's hard to argue too heatedly against what becomes essentially a monotonic increasing cumulative P&L graph once we've learned to follow a few rules. The bottom line is, once we've mastered a method--any method--to the point we are consistently profitable, as far as the market cooperates one's income becomes pretty much a function of the number of contracts traded (just not so large that we become a target for some institution's bot ), and IMO that is all that matters.

The charts below show my implementation of BB's trend trader and a few typical trades, here relying mostly on Murrey Math for S/R and hence entry & targets, but also magic numbers, PriorDayOHLC and pivots (for those times in every session it seems price makes more or less a beeline e.g. for the floor pivot). I use 200 Tick, 600 Tick and 1800 Tick charts for short time frame, medium time frame and long time frame respectively, trading the 600 Tick chart and using the 200 Tick chart for placing entry and initial stops. I use as many targets as S/R levels between entry and "likely" beginning of the retrace (often 2 Murrey Math levels max).

The annotation on the chart tries to point out the alignment of BB's "5 energies" (trend, momentum, cycle, S/R and fractal) for a typical high probability trade; i.e.,

E1-50MA trending (rising, in the example below)
E2-MACD positive + rising (or bottoming) for an uptrend; negative + falling (or topping) for a downtrend
E3-K stochastic less than 45 and preferably less than 20 + rising + having crossed above D for an uptrend; greater than 55 and preferably greater than 80 + falling + having crossed below D for a downtrend
E4-nearest S/R for support and target placement
E5-confirmation of the mid time frame momentum and stochs as far as possible from both short and long time frame charts.


Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #64 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

The fixed NN Predictor project has been posted in its own thread here

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #65 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Last 2 trades on Friday happened because
- I wanted a little extra cash for the weekend (bad, bad reason for trading)
- there still seemed to be a little left in the trend before it would start feeling any headwind from (according to the buzz) significant resistance at 1.2672.

Both trades made money but both were not high probability according to my understanding of BB's system.

The first trade was a bet on a bounce off Murrey Math 0/8 in the direction of the trend, clumsy entry at 1.26375 with initial stop 6 pips below at 1.26315 and 2 targets initially at 7.5 and 15 pips above the entry. Bid touched the stop before reversing, which unnerved me. Moved the targets to 1.2650 but ultimately bailed entirely at 1.26435 for 6 pips in the face of what seemed at the time growing resistance, but in hindsight the trade was probably in less danger at that point than when entered

The second trade was contrary to trend mostly because I didn't want to get caught in end of day/ end of week volatility. I chanced it because momentum was slowing (what I perceived as divergence) at the top of the channel, short entry at 1.26515 (slightly above the magic number of 50), stop at 1.26555 (1 pip above last high, within my 6 pip limit), stochs & momentum dropping on the entry (STF) chart, targets 7.5 and 15 pip below. Both targets were hit and that was it for me for the day & the week.

I have a variety of simple ATM strategies I use for manual trading, choice depending on price action, S/R density and so on, which in this case placed a stop and 2 targets automatically when the trades were entered and moved the stop to break even + a couple of pips when the first target was (or ought to be) hit. The targets are adjusted to S/R levels as necessary, and occasionally the stop (toward the action) if it looks like price is getting ready to back up over me.


Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #66 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Back in the saddle after a motorcycle trip down the US east coast last month (including the entire Blue Ridge Parkway, Skyline Drive & Tail of the Dragon for good measure)--great opportunity to blow out the cobwebs and to remember why I trade (to afford to get away from trading ). First few days back at it were iffy but today I made 144 pips (spot EUR/USD) in the hour and 20 minutes spanning the non-event hosted by Mario Draghi. **

I feel pretty much nothing about today's gain since it simply confirms that at this stage the issue is not capturing pips when the market offers them but hanging on to them--a problem that reared its ugly head early on and continues to plague me from time to time. I rarely lose money by misreading a setup, far more often by over-trading (continually entering low probability trades) when bored, frustrated or tired. Having suffered the consequences of over-trading for a couple of years while learning the craft I'm only now beginning to recognize when I'm not fit to trade, at long last starting to accept that if I want to trade (or at least want to hang onto profits) then I have to obey the rules about diet, fitness, getting enough sleep, routine and trading only when the market is active. In short, I'm beginning to appreciate that knowledge and a system are not enough--profitable trading also requires lifestyle changes & self discipline, and no system is going to change that.

It's going to take some time (2 weeks?) to get back up to speed on technical issues--the neural net in another thread and automating the system I use here for manual spot Forex.

** ETA: loved Mr. Draghi's phrase to the effect the ECB is going to "take some months to design modalities"

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #67 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,780 since Jun 2009
Thanks: 32,306 given, 97,567 received

Welcome back

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #68 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Thanks Mike...good to be back.

Today's Post Mortem

This morning's trading session was a perfect example of what I talked about yesterday; namely, the sole impediment to consistent profits for me at this point is not making money but hanging onto profits. "Hanging onto profits" is the mental basket I prefer to store all the negative factors that impede profitable trading these days, psychological & otherwise, since in my view all such factors feed off each other so belong together, all are present & accounted for and all are pretty much in remediation. In other words all are issues contributing to a common effect (negative trading outcome) hence these days easier for me to conceive as a single problem, rather than as separate issues I need to gnash teeth and wring hands over. If that makes any sense.

In any event IMO the trading day actually starts the night before when one hits the sack (step 1 = good night's sleep), and last night I was unable to doze more than a few hours. Overslept & woke up in a panic, sucked back a coffee laced with maca, stumbled into the trading room and discerned through bleary eyes EUR/USD was accelerating downward rapidly with remarkable enthusiasm, no clue why (didn't know what time it was and had forgotten about the US jobs report, therefore didn't know this was the initial reaction which I might be able to play when on my toes but not otherwise). Before the trader brain could get in gear a finger clicked "Sell Market". To make a long story short I was down about 57 pips before the smoke cleared and once the coffee kicked in I spent the rest of the morning struggling to undo the damage.

The good news is spin control (which no doubt we all practice according to our level of experience and which provides a convenient measure of our level of experience if we insist on testing it) worked better than hoped. The reptilian part of the brain (or whatever part doesn't sleep) realized the mistake immediately and sounded the usual alarm, triggering another part of the brain to start logging the potential train wreck (whatever part functions like the black box in an aircraft). The log shows as expected the part of the brain controlling the mouse finger is insisting "I CAUSED THIS AND I CAN FIX THIS", still another is screaming "STOP PUSHING FREEEEKING BUTTONS", while what is presumably the Voice of Reason repeats "We'll do a situation analysis ASAP boys, but not before we've had our coffee" like a broken record. Emotions seem to pop out of fox holes like enemy soldiers while the analysis takes place except at this stage in our career no prisoners are taken--the sniper part of the brain is picking off each one as soon as it shows itself. Finally the trader shows up and begins to work and we gain back 49 pips. He reports "Should have been up those pips instead of down 8", but best he could do under the circumstances.

ETA: I imagine the familiar personality characteristic that precipitated the havoc this morning to be the lazy & impatient gambler, who apparently also doesn't sleep and believes in luck rather than smarts or hard work, probably because it's a more flattering explanation of why less deserving people than himself often seem to be better off. His bad behaviour is rewarded just often enough to reinforce it. In fact he's in it for the rush and once that passes, even when a gamble pays off must somehow dismiss the inexplicable feeling of guilt and unease that comes from receiving something one hasn't earned. May be next against the wall in our little revolution

ETA: Rather than record every trade prefer to mention only anything out of the ordinary. Instead will start listing daily P/L month to date. For August so far it's therefore +136 pips.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #69 (permalink)
London, UK
 
Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker: IB/IQFeed
Trading: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 471 given, 786 received

It's interesting to see the more experienced traders sometimes go in without a firewall in place between their thoughts and reality. I relate habit-building to a firewall - I still have a long way to go with it all though. I am fascinated by the human mind - specifically my own and anyone else who uses theirs in the same way. Have you thought of mechanical support - some sort of bio-feedback device that would measure your pulse/blood pressure/brain waves and give you a green light to trade? i.e. would detect and display when your thoughts were in the zone and not acting the way you describe? Or do you think that's an unnecessary crutch, when we should be building those discipline habits on our own?

You can discover what your enemy fears most by observing the means he uses to frighten you.
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #70 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Interesting idea, to use biofeedback. In my youth I might have tried it (there may still be a brainwave monitor kicking around here somewhere) but the closest I get these days is checking my blood pressure periodically. Wonder if my daughter (studying neuroscience at university) could offer any advice. Right now she's working on brain differences between bilingual people who learn a 2nd language early vs late in life--will try to convince her traders' brains are much more interesting

For the time being the objective test for me is the (estimated) probability of the trade--if I start taking anything other than high probability trades I know something is amiss, best to quit before they start turning into losing trades.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #71 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Continuing issues with NT this week--in this case connection problems (see thread elsewhere) led to creation of a phantom position in NT's record of my live trading account. In this case a position that shows up nowhere else except as a position in the Executions tab and which is used internally somehow in P&L trade calculations every time a trade is closed, totally screwing NT's accounting prior and since. As usual NT refuses to sync with the broker account despite restarts, DB resets etc. ad nauseum. Reinstalling NT (and losing all records kept) is only cure I've found so far for this particular problem.

The good news is I'm learning my way around the TWS trading interface -- ugh.

ETA: in other news took a hint from Big Mike's inquiry in another thread about value areas and started looking closely at the "volume profile" stock indicator as a constraint for trading what the system otherwise interprets as chaos (i.e., chop), especially for predicting jumps between "modes" (or areas of concentrated activity). These areas so far seem to correlate with a crude realtime activity indicator (essentially, activity = 1 / [seconds per tick bar]). Not exactly sure what volume it is measuring in the case of Forex (appears to be at least what is relayed from IB as instantaneous inside order sizes) but if the last couple of days are indicative the approach may prove profitable.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #72 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Trading activity described today in this post. Bottom line is I scalped 2 contracts all morning (80 trades, all in at entry with max 1.5 pip stop + 2 targets) and grossed ~73 pips, netted ~33 pips after commission (bandit of a broker made off with ~40 pips).

Traded from STF (200 tick) chart using price action and stochs, MACD and VolumeProfile in order of priority.

NT accounting is still FUBAR'd and will likely reinstall it over the weekend.

ETA: I've copied the chart image I posted to the thread alluded to above in order to comment briefly on it. I think a lot of traders would agree the style of trading indicated, while profitable at the end of the day, is very inefficient in terms of commission costs. However, it reflects something I've found to be true; namely, if one intends to trade (and I wake up intending to trade every morning the market is open) one probably ought to trade one's mindset rather than one's system. This means in particular that if the first 3 trades go against me that's it for the day. Otherwise if up to it and the market is cooperating (i.e., relatively large moves in response to events, say, clear sentiment, obvious signals) I'll start trading the "5 energies" system described previously. However, some days (like today) all I can focus on is minutiae of price movement, and those days I scalp, more or less trading the activity histogram of the current mode, depending on MACD for temporary sentiment, stochs for entry, entries and targets at local S/R levels , hopefully catching the trend when price migrates to (or establishes) a different mode but often as not scalping every wiggle along the way.



Detail of 200 tick chart showing what I mean by "scalping every wiggle" . With NT accounting misbehaving I'll have to wait for the broker to post the activity report to calculate trading stats (other than the loss to commissions) for the morning.


Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #73 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Mixed results Monday & Tuesday, partly because I broke a rule that because of it's apparent importance probably ought to move up the priority list; namely, don't trade outside your normal session. I've been trading outside the normal session (combined US/London session) for 2 reasons:

1. I was trying to look busy to avoid doing repairs to the deck at the back of the house; and,
2. I was curious whether at this stage of my trading career the scalping method used recently to trade the regular session (both New York and London online), when trader activity is 10 to 20 x activity of New York alone after the London close, would work in a slower market. The question occurs to me now and then because range and tick bar patterns have no time characteristic and the 2-part hypothesis to be proved or disproved is therefore that

1. patterns are all that matters.
2. patterns don't change when activity slows

This time around it turns out the approach (setting out to make a number of relatively small, short time frame trades, essentially scalping) did not work outside of normal session hours, partly because the slow pace drives me absolutely nuts (nuttery in this case characterized by a compulsion to gamble). Thus we chalk up another statistic on the "How not to trade" side of the ledger.

The last few days reminded me of trap I sometimes still fall into stems from the days I tried to predict what price was going to do in the hope of getting in at the earliest possible moment. Stats continue to suggest the odds of doing this (in my case) are less than 50%, probably much less. Worse, stops and targets tend to reflect expectation of the Big Move and hence are in general wholly unrealistic in light of what price is actually doing--all in all a recipe for disaster.

In an effort to regain the almost 50 pips lost so far this week today we're setting out to trade the mode a little more objectively. The figure below shows a 4-standard deviation regression channel (from a couple of hours before the N Am session started and the present) added for guidance and order preset at the top of the channel (top because of the slightly negative slope of the channel). Ordinarily I don't preset orders but wait for signs that a retrace is setting up, and intend to when the time comes. Being easily amused however, in the meantime the preset gives me something to occupy myself with (a point of reference for ongoing price action), which prevents me randomly clicking Buy/Sell buttons



ETA: In as much time as it took to write the first installment the setup was triggered with the following outcome:

First target hit, ATM strategy advanced the stop to breakeven + a couple of pips.



Second target hit.



Subsequent price action. Top of regression channel (blue line at very top of chart) now indicating mode bias is upward so looking for a long trade once robust (upward) movement sets up below the mean/median of the mode



Next 2 trades (first long at regression channel bottom wary of aggressive downward momentum, target 1 + trailing stop hit; second short in case transition to new mode occurring--target 1 + trailing stop hit.

Hate trading the thin edge of the wedge, especially around MM 0/8 ("Ultimate S/R"). Wouldn't hazard if any powerful magic numbers (00, 50, 75 or 90, say) in the vicinity





Final couple of trades of the day, both short: first non optimal for 1 target hit + trailing stop; second actually after London closed but higher probability, by preset ATM strategy while I wasn't in the room--both targets hit

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #74 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Generalities

[Started much earlier today] Trading the European open just now (not my cup of tea ordinarily since the price action lately rarely seems to justify being awake in the wee hours AST) reminds me of the discussion about chop, how to detect it & how to deal with it. I define chop as price action with respect to some time frame that doesn't generate profitable trading opportunities within the confines of one's system, the only gauge that works for me being that if I try to trade it and get whipsawed then it's chop (no longer whipsawed to death, fortunately, since these days I back off and reassess the first time a trade doesn't go as expected).

What has worked for me more or less so far this morning (spot EURUSD, 8 trades, 6 profitable, 2 breakeven) in the absence of anything I'd call a trend continues to be treating the position of price relative to VolumeProfile clarified by a 4-stddev regression channel as an indication of the probability price will move in a given direction (both indicators applied to the European session only, assuming there is going to be a disconnect between European and Pacific Rim trading patterns), further constrained by various S/R levels (Murrey Math, PriorDayOHLC) and by momentum in 3 time frames (200, 600 and 1800 tick chart).

The "edge" here, if we can call it that, derives from the awesome power of not trading (similar to the power of quitting, just that you can "not trade" as often as you like whereas presumably one quits only once per session ). Price can fluctuate at random between 4-stdev regression lines for a relatively long time before approaching one boundary or another, at which point, and only at which point, any potential trade is considered. Others use some mix of other volatility based channels (e.g. Bollinger Bands) in a similar way (to estimate the probability of a reversal versus a continuation or "breakout"), and some even have strong opinions and elaborate theories explaining why it works. IMO the bottom line is one ought to go with whatever works without necessarily trying to explain to anyone else why it works (and especially don't try to argue one technique is somehow superior to any other since such explanations simply cause the listeners' eyes to roll back in the head. IMO all techniques that consistently make money for the practitioner probably work because they've become part of a given trader's wiring, no more and no less, exactly the same in light of the only metric that matters--profits). That said, I suspect keeping an eye on VolumeProfile and regression channels on top of everything else works for me simply because the additional constraint biases against over-trading--invariably when I'm not at the top of my game--which still remains the occasional impediment to consistent profitability.

Upshot--the power of not trading in action

[At end of day] Having said all that I finished today one pip in the black after 42 trades, ca. 60% profitable (which stats may suggest what kind of day it's been)--a miracle, having missed all significant moves getting what little sleep I've had in the last 24 hours while they were occurring and eventually making all the mistakes I've been pontificating against . What may have saved my bacon (like landing a Cessna 170, any session I can walk away from is a success) was somehow managing to recognize when price pattern was shifting, working with tiny pip targets & microscopic stop loss. That and the triple fail safe--I've finally accepted that if I begin to feel anything at all, especially negative emotions, it's a huge red flag. Not necessary or even possible to control them at that point but essential to stop trading.

Tomorrow is another day and TGIF

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #75 (permalink)
Bridgwater, UK
 
Experience: Beginner
Platform: Ninjatrader
Broker: MB Trading
Trading: Forex
 
mokodo's Avatar
 
Posts: 384 since Jun 2011
Thanks: 525 given, 346 received

bnichols,

Just wanted to say I've been following your journal and enjoy it a great deal.

know thyself
Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #76 (permalink)
London, UK
 
Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker: IB/IQFeed
Trading: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 471 given, 786 received


bnichols View Post
What has worked for me more or less so far this morning (spot EURUSD, 8 trades, 6 profitable, 2 breakeven) in the absence of anything I'd call a trend continues to be treating the position of price relative to VolumeProfile clarified by a 4-stddev regression channel as an indication of the probability price will move in a given direction

Hold on, if you're trading spot EURUSD, how are you getting volume profile? Are you taking it from the EUR future?

You can discover what your enemy fears most by observing the means he uses to frighten you.
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #77 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Adamus View Post
Hold on, if you're trading spot EURUSD, how are you getting volume profile? Are you taking it from the EUR future?

Hi Adamus !

Roger that--good point and alluded to briefly in post 71 above, VolumeProfile plots whatever is returned by NT for spot Forex as MarketDataEventArgs.Volume in the OnMarketData event, which for Forex data may be what NT represents as instantaneous inside bid/ask volume in the lower right hand side of a chart (Chart Trader section) and for IB in particular (sampled data) is in any event highly suspicious (but which nevertheless, e.g., jtRealStats manages to create a volume ladder out of). Here we're interpreting whatever it actually is simply as a measure of market activity at a given price.

By the way, I know I'm giddy from lack of sleep when the question brings to mind the scene in HitchHikers Guide to the Galaxy where Jeltz (in charge of the Vogon demolition fleet) responds to the panic of the population of Earth when they learn the planet is seconds away from annihilation, with "There's no point in acting all surprised about it. The plans and demolition orders have been on display at your local planning office in Alpha Centauri for fifty of your Earth years, so you've had plenty of time to lodge formal complaints.''

B.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #78 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


mokodo View Post
bnichols,

Just wanted to say I've been following your journal and enjoy it a great deal.

Many thanks mokodo !

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #79 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Friday was a learning experience, starting badly (down about 36 pips or $360 after commission at one point) but ending on a positive note (up about 42 pips or $420, only because the market took pity and delivered almost $750 in 16 trades over the last 42 minutes of the session), 67 trades in all @ 57% winning, 2 contracts per trade, 2 targets, 1 contract per target.

What today made clear was that if I choose to trade like a rodeo cowboy (so to speak, meaning ignoring the rules & trading by the seat of my pants when there are no clear signals) I'm going to faceplant often & eat a lot of cow pie.

A trade-by-trade post mortem of the week is still in progress, which includes a comparison of trading results to backtest results for a trend trading bot (see attached strategy cobbled together this A.M. based on the anaSuperTrendU11 indicator)

BUT NOTE IF YOU WANT TO TEST THE STRAT: the strat calls an experimental version of anaSuperTrendU11 (not included since it's @Fat Tails' IP) that has been modified simply to give direct access to the UpTrend[] dataseries; specifically the scope of the UpTrend[] dataseries has been changed from private to public on liine 75 of the original; i.e., from

 
Code
private BoolSeries 					upTrend;
to

 
Code
public BoolSeries 					upTrend;



Once optimized on 1800 tick data for the last week no surprise the strategy traded a heck of a lot less during backtesting (averaging 1 trade per day, 80% winning, hence suffered much less loss to commissions) and doing significantly better in terms of P&L than I managed when a usable trend emerged on Friday.

Results for Friday


My trades vs bot trade on 1800 tick chart:


While the bot made $721 on 1 trade (2 contracts) and paid $9.92 in commissions, my trade-by-trade P&L looks like this:


Notes on attachment

Most of us know what bots are worth (anybody watching the machinations and share price of Knight Capital Group?). Guessing which is why high frequency bot programmers earn the big bucks, but live and die by the sword.

This is a barebones trend trader entirely dependent on Fat Tails' anaSuperTrendU11 indicator. To use as is you must modify a copy of the indicator as mentioned above.

To use with unmodified anaSuperTrendU11 change the existing OnBarUpdate() code below in TDSuperTrend.cs to detect a change in direction from an available anaSuperTrendU11 dataseries (i.e., StopDot or StopLine)

 
Code
        protected override void OnBarUpdate()
        {
			bool isUp = anaSuperTrendU11(BasePeriod,Multiplier,RangePeriod,ReverseIntraBar,ThisBaseType,ThisOffsetType,ThisRangeType).upTrend[0];

			if(isInit)
			{
				bool isGoLong = isUp && !isLastTrendUp;
				bool isGoShort = !isUp && isLastTrendUp;
				if(isGoLong)
				{
					string sigName = "L_"+nTrade.ToString()+"_"+nLong.ToString();
					nTrade++;
					nLong++;
					isLastTrendUp = true;
					EnterLong(Quantity,sigName);
				} else if(isGoShort){
					string sigName = "S_"+nTrade.ToString()+"_"+nShort.ToString();
					nTrade++;
					nShort++;
					isLastTrendUp = false;
					EnterShort(Quantity,sigName);
				}
			}
			else
			{
				isInit = true;
				if(isUp)
					isLastTrendUp = true;
				else
					isLastTrendUp = false;
			}
        }
It seemed so much simpler this morning to modify the indicator and thus inherit Fat Tails' impeccable logic than to reinvent the wheel in the strategy.

In any event if there's an issue let me know. The only claim to fame for anyone who doesn't already have a method in their own code snippet collection may be that the strategy shows one way to expose an indicator's enums to optimization.

Attached Files
Register to download File Type: cs TDSuperTrend.cs (8.0 KB, 14 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #80 (permalink)
Market Wizard
Berlin, Europe
 
Experience: Advanced
Platform: NinjaTrader, MultiCharts
Broker: Interactive Brokers
Trading: Keyboard
 
Fat Tails's Avatar
 
Posts: 9,855 since Mar 2010
Thanks: 4,238 given, 26,728 received


bnichols View Post
BUT NOTE IF YOU WANT TO TEST THE STRAT: the strat calls an experimental version of anaSuperTrendU11 (not included since it's @Fat Tails' IP) that has been modified simply to give direct access to the UpTrend[] dataseries; specifically the scope of the UpTrend[] dataseries has been changed from private to public on liine 75 of the original

Although upTrend is in fact private, the anaSuperTrendU11 has a public BoolSeries UpTrend defined in line 564 of the Properties section. You could have easily accessed the public BoolSeries as anaSuperTrendU11.UpTrend[0] without modifying the indicator. Please take note of the capital "U" of the public BoolSeries.

Reply With Quote
The following 2 users say Thank You to Fat Tails for this post:
 
(login for full post details)
  #81 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Fat Tails View Post
Although upTrend is in fact private, the anaSuperTrendU11 has a public BoolSeries UpTrend defined in line 564 of the Properties section. You could have easily accessed the public BoolSeries as anaSuperTrendU11.UpTrend[0] without modifying the indicator. Please take note of the capital "U" of the public BoolSeries.

Roger the capitalization and thanks Fat Tails No excuse and not sure why I modified the indicator anymore. Having a hard time sleeping--today I feel like there's a wool blanket being dragged over my brain

ETA: Just figured out one of the reasons I missed it....I generally use underscores ahead of private variables. Ancient habit and hard to break. My wife will be the first to tell you I don't see something if I don't think it's there

ETA: After all those excuses in spite of the fact there is no excuse (and after a glass of wine or 2) should point out what matters--when I need a powerful and reliable indicator that no one has thought of yet I look up what you've done lately Pretty much stop looking if you haven't done it.

Greatest respect.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #82 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Per @Fat Tails remarks, modified version of TDSuperTrend.cs is attached that will run with the original version of anaSuperTrendU11.

Attached Files
Register to download File Type: cs TDSuperTrend.cs (8.0 KB, 13 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #83 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Attached is a slightly less bare bones version of TDSuperTrend (TDSuperTrendQuanity.cs) that allows for a simple stop loss and intermediate profit target. Initial values are set based on backtesting 1800 tick data between April 22, 2012 and the present (all the data that survived the last reinstallation of NT --haven't had a chance to reload the rest).

This also just posted to the NinjaTrader forum in regard to performance of the bot in response to a comment by a trader using NT's Monte Carlo simulation:

Monte Carlo Simulation - NinjaTrader Support Forum


Quoting 
Not sure how often this thread gets looked at but having just started using NT's Monte Carlo Simulation it appeared near the top of a Google search.

In any event a couple of caveats come to mind (and please correct if it looks like I'm missing something) in case the reader is not already well aware of them; namely,

1. session definition is vital to the actual performance of the strategy (bot), since the behaviour of any indicators used by the bot in back- and forward testing varies with session definition (i.e., data outside defined session hours is not used to determine indicator values). The upshot of this is there is apparently no way in NT to back- or forward test a bot intended to be run on a session that is a subset of the indicator data domain. In other words, it does not appear meaningful to test a bot with e.g. 24/5 currency futures over multiple sessions with indicators calculated from the 24/5 data domain, if the trading session is also not 24/5 (i.e., instead restricted to the overlap between European and N American markets, say). In such a case one can however forward test by turning the bot on and off manually at the start and end of each session. That said, it is not an issue if one is aware of (and perhaps depending upon) the fact that in real life the indicators (and hence the bot) will react vigorously to any price gap between start of session and end of last session.

2. the probability curve displayed is a cumulative probability, so that for indicated probabilities over 50% the probability of the associated value occurring is actually 100 minus the indicated probability. E.g., the probability of a an $84,000 cumulative profit with 96% cumulative probability occurring is 100 - 96 = 4%.

ETA: by the way, it's been my experience with bots that if there is any chance of a worst case scenario occurring, it will, seemingly often enough near the outset when the account is most vulnerable, a perception that likely stems from the fact it's game over if we blow out the account. I've learned therefore to limit quantities so that the account is able to absorb the worst case draw down and still function from the get-go.

ETA: Plan to run the strat on IB's paper account this week while manually trading the cash account (or at least planning to trade manually, since I was fiddling with the strat when the only clear channel-trade opportunities so far today occurred). I don't use NT's SIM account because I never took the time to find out how the fills compare to broker paper/cash and want an independent transaction record due to the not insignificant probability NT's accounting will decide to take another walk in the snow (due to DB corruption--guessing app conflict is causing NT to get its panties in a bunch occasionally but so far unable to say which app this time)

Attached Files
Register to download File Type: cs TDSuperTrendQuantity.cs (9.9 KB, 13 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #84 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Nothing much to report except a couple of pedestrian trades for less than 10 pips @ 2 contracts after missing the only real setup of the day and that the bot (TDSuperTrendQuantity) managed to make a single trade on the paper account once I finally got it up and running (made about 16 pips before exiting the trade on end-of-session at 12:30 AST--see Figure). Not sure why the bot waited as long as it did to enter.

The "pedestrian trades" occurred outside my regular trading hours while experimenting with the description of ES activity at various hours of the day in @dario1's journal (first post in ES Unfair Advantage). I'm not sure how he created the list and as he notes there are a couple of gaps, but I like the idea because like all good science it lends itself to peer review and to association of probability functions with inter-session intervals usable by a bot, assuming one can suitably quantify notions like "retesting the days' highs" (shouldn't be too hard).


Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #85 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Did not trade today--didn't sleep well enough the night before. The bot (TSSuperTrendQuantity, trading on paper) also took the day off because there were no reversals in the session, which amounts to a missed opportunity since EUR/USD moved higher all day, hence should probably give some thought to adding trend continuation at session open.

Did poke around futures.io (formerly BMT) however (reading journals mostly) and attended the TopStepTrader webinar. Nothing better than reading about the impressive things other folks are doing to put one's own progress in the proper perspective

Out of curiosity I created a plot of EUR/USD activity vs time-of-day inferred from duration of 1800 tick bars from the first of the year to the present (figure below). Even unprocessed it's possible to discern routine peaks at certain times of the day and activity associated e.g. with events. I want to do a similar study of price movement vs time-of-day to see in particular if (as it seems a lot of traders claim) there is a usable correlation between inter- and intrasession ranges & trends (e.g., to get a better idea to what extent counter-trend movements are predictable), but that's a little more involved.


Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #86 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Excerpt from today's private journal pretty much sums it up (3 trades, hit day's loss limit, quit trading). For slaps & giggles will watch to see how the bot fares on paper for the rest of the session (it's presently long).


Quoting 
20120822

After attending TopStepTrading webinar yesterday and reviewing previous webinar from March decided to amp

up my discipline a notch, including journal keeping.

Initiated short open ended trade at 1.24542 @1.2 contracts at 9:07 AM
1. because bot TDSuperTrendQuantity was in a short paper trade
2. the setup correlated more or less with TD's approach
3. before I'd studied market sentiment but after glancing at volume profile
4. Before calculating what TDSuperTrend's initial stop meant for money management

Management commentary:


1. Stop at 1.2470 = 15.8 pips -> loss of $189.6 minus commissions, or 189.6/54720 = 0.35%
Since we're forward testing with cash assume a daily loss of 0.2%. Moved to .002*54720 + 1.24542 = 1.2463

(or 0.2%). If this trade fails that's it for the day.

2. By 9:39 TDSTQ had moved stop to 1.24674 in response to rising EUR/USD (hit MM 6/8 @ 1.24590). Since

this is still above my 0.2% daily loss target left my stop where is. Feeling of depression setting it at ~

-4 pips unrealized loss, then equanimity knowing the stop would take care of things if they get out of

hand.

3. Shortly after the last comment at 9:39 EUR/USD plunged (to +12 pips profit by 9:45) with every sign of

going lower. Depression turns to calm elation, then to worry. Moved stop to protection at +1.5 pips

(1.24515), when it looked like price was setting up a new lower value area median 1.2441, TD method

indicated downtrend.

4. Retrace peaked for the time being at @MM 3/8 at 9:52--thinking about moving stop to just above this, but

since TDSTQ stop is on the move will go with that if it gets between my entry and my stop.

5. By 10:06 price has retraced back to within 5.5 pips, 200 tick chart indicating a miniuptrend, 600 pip

chart stil indicating weak downtrend, 1800 tick chart indicating downtrend, and notice on othe brain's

back burner the part of the Lord's Prayer that begins "Lo, though I walk through the valley of the shadow

of death...." is repeating.

6: By 10:10 I noticed my entry and protective stop are sitting smack dab on the median of the mode of

today's value area--a sitting duck even if price is still polishing its shotgun (appears to have given up

on the retrace in the downtrend, heading away from me for the time being). Ordinarily I'd move the stop to

just above the last retrace high at 1.2448 but seems that would be penny wise pound foolish--why increase

the chance of getting stopped out unnecessarily, risking pennies against what could amount to significant

profit? Instead we'll wait and hope for price to break downward through previous low at 1.2431 just above

the floor pivot (another location where price could set up housekeeping--just has to lose interest in the

last value area and the scent of my stop sitting there).

7: 7 minutes before US open price appears to be consolidating between 1.2448 and 1.2435, US futures

negative meaning we can likely expect the Dow to try to recover the initial gap down in the first 30-60

minutes after open, probably correlated with upward pressure on the EUR/USD

8: At the open one eye on the Dow 1 minute chart and one eye on the EUR/USD, which is sitting at the top of

range mentioned previously, Dow as suspected aggressively trying to close the gap, EUR/USD rising but not

so rapidly.

9: Dow now collapsing at T+ 90 seconds, EUR/USD inching upward toward my stop. Hard not to move the stop

back to day's loss where TDSTQ stop now sits

10: A little relief at t+5 minutes after EUR/USD (attached to Dow by rubber band) came within 1.5 pips of

the stop but has now pulled back a little. 200 Tick chart indicators flatlining, 600 & 1800 Tick charts

suggesting beginning to resume downward trend, but still buoyant and too soon to say. Dow still showing

remarkable signs of life. Too bad. Hopefully we survive until a (downward) trend sets up. If not we'll

get stopped out and sit on our hands until the market initiators tip their hand one way or the other.

Should walk away & let market takes its course--get breakfast rather than sit here watching & working on an

ulcer.

11. at 10:43 Steeling myself to get stopped out, and stopped out for a profit after commission of $7. Gag

me with a spoon. Uptrend started on 200 tick chart, mixed signals on 600, signs of wicked retrace in a

downtrend on 1800 tick chart. EUR/USD and Dow have merrily climbed to premarket value area, entirely

closing the gap & more. So much for that new minivan.

12. 10:47 Price hits TDSTQ stop, and the bot goes long at significant S/R with all indicators pinned in

overbought area--silly bot. Still, all signs point to beginning of uptrend, so we sit at wait for a

retrace, potentialy to go long, avoiding the powerful urge to jump on the lemming train at this point.

FOMO is strong in this one. Smartass bot is up 11 pips, but it's early days. "First ones now will later

be last" is repeating in the back of my brain.

13: 10:55 watching price as Dow retreats from yesterday's close, having gorged now fat & happy. Looking

for signs it may retest the close, breakout probably necessary for EUR/USD to establish an uptrend.

14. 10:57 entry at 1.2474, just below magic number of 75, stop at 69 (5 pips) when 200 shows retrace over

and 600 confirms. Should NOT be taking setups from 200 tick chart Stop moved to +1.5 pips at

first opportunity since we're facing considerable uphill climb in front of likely S/R. Not exactly a high

prob entry. Dow plunging on 1 minute chart.

15: 11:01 stopped out for another $7 profit after commissions. Who'd have thought? Time for breakfast.

Bot still up 6.5 pips. Instead placed another long limit order at 1.2377, 4 pips above price and 4 pips

below MM 6/8 (I can hear @Fat Tails every time I mention MM ) --make the market work a little if it's

going to take my money.

16. 11:06 Price weaking, no momentum, no nothing except FOMO. Cancelled order. Bot still up 9.5 pips--

stupid bot.

17. 11:07 long limit order filled at 1.24835--absolutely my most hated entry (breakout). Initial stop

placed just above day's loss limit including previous gains at 1.24685. No real justification for the

entry, the exuse is "I'm going to quit for the day if that trade goes south."

18. 11:15 stop hit and that's it for me for today, down $158 after the smoke clears. Now where did I put that stick I beat myself with.

Notes:

Market Sentiment post first order entry:
Possible explanation of drop just before 9:AM EST from ForexLive by Adam Button || August 22, 2012 at

12:50 GMT

Euro Hits Stops Below 1.2440

Risk aversion spikes and the US dollar makes broad gains.
EUR/USD bids at 1.2420/30 with more stops below.
No fundamental drivers for the latest moves, which were led by breakdowns in Bund and Treasury yields.



Market Sentiment pre trade:

Orderboard from ForexLive:
EUR/USD: Bids 1.2420/30, sell stops below. Tech supp 1.2390/00 (55 day MA- 1.2390). Offers start

1.2480/00 ahead of a barrier at 1.2500 (part of a 1.20/25 DNT), also talk of SNB related offers from

supranational. Buy stops through 1.2510 ahead of offers 1.2535/50 (July 5 high 1.2538) and ahead of the

1.2550 barrier

Futures from CNBC @ 9:30 AM:
DOW (Mini)
1month

FUTURES FUTURES FAIR VALUE (-22.42)

13203.58 13177.0 -22.00 13176.58 13177.0 0.42
Last Updated: 08:24:23 AM

S&P 500 (Mini)
1month

FUTURES FUTURES FAIR VALUE (-1.63)

1413.17 1409.3 -3.20 1410.87 1409.3 -1.57
Last Updated: 08:23:47 AM

NASDAQ (Mini)
1month

FUTURES FUTURES FAIR VALUE (-3.25)

2772.2 2769.25 -4.75 2770.75 2769.25 -1.50
Last Updated: 08:24:59 AM

OIL 96.68 -0.16 -0.17%

GAS 3.0655 0.0003 +0.01%

NAT GAS 2.826 0.051 +1.84%

GOLD 1642.00 1.30 +0.08%

SILVER 29.40 -0.028 -0.1%


Events from BabyPips
Session Time Currency Event Impact Actual Forecast Prev.
London New York 10:00 USD Existing home sales med 4.51 mn

4.37 mn
New York 14:00 USD Minutes from prior (31 Jul-1 Aug) FOMC Meeting

released high
Sydney Tokyo 23:30 USD Fed's Evans discusses current economic conditions and

monetary policy in press briefing low

ForexStreet news bullish for euro but also for dollar:

Forex Flash: EUR/USD targets 1.2700 – BBH
Wed, Aug 22 2012, 12:26 GMT | FXstreet.com
Share on email Share on print Share on facebook_likeShare
Related News
Forex Flash: EUR/USD targets 1.2700 – BBH
Forex Flash: EUR a bright spot as of late – BNZ
EURUSD’s bull and bear fight….
FXstreet.com (Córdoba) - Greece outlook has gotten more constructive as the meetings start, according to

the BBH team. Juncker will be in Athens today while Greek PM Samaras will be in Berlin and Paris on Friday

and Saturday. Hollande and Merkel will also meet tomorrow in Berlin. "Despite the flurry of meetings, the

Greek saga will continue to play out over the coming weeks until the Troika decision and German court

ruling in September", they say.

"The technical picture seems to favor further euro gains near-term", BBH says. "The euro broke above the

$1.2445 area Tuesday that had capped the euro in August, which also coincided with a retracement level from

the June-August drop in EUR/USD", analysts comment. "That break targets the June 28 high near $1.2700, but

further gains could be a slow slog as markets await further developments regarding Draghi's proposed bond-

buying plan".

Forex Flash: Dollar is bullish against the majors – UBS
Wed, Aug 22 2012, 12:15 GMT | FXstreet.com
Share on email Share on print Share on facebook_likeShare
Related News
Forex Flash: EUR/USD targets 1.2700 – BBH
EURUSD’s bull and bear fight….
Forex Flash: Dollar is bullish against the majors – UBS
FXstreet.com (Barcelona) - FX markets consolidated the majority of yesterday’s gains, with the DXY trading

flat in the European session. Most major equity markets were trading slightly in the red at the time of

writing, though news flow was relatively light ahead of the FOMC minutes due later in the US session.

According to FX Strategist Chris Walker at UBS, “Our core views remain bullish for the dollar against the

euro, pound, yen and Swiss franc, while exercising a degree of caution with commodity currencies at current

levels against the greenback.”

The Eurozone debt crisis remains a focal point, with EUR/USD around 10 big figures lower, and Spanish 10y

bond yields 65 percentage points higher, than was the case in May. Lastly, “we remain confident that the

SNB will maintain the 1.2000 floor in EUR/CHF for the foreseeable future.” Walker adds.


ETA: Chart showing the carnage:



ETA: 11:42 AST Imagining the lads and lassies trading market profile are now just beginning to evaluate the opening range and its implications. Too bad I've already blown the wad & quit trading for the day

ETA: 12:00 AST notice divergence between EUR/USD and Dow may be setting in. Dow plunging while EUR/USD still gaining. Upshot is the damn bot is still ahead of the game by a pip.

ETA: 12:30 AST Bot ends the session on the European close down $293. Don't know whether to feel superior to the machine or to rue this additional evidence one can't program a bot to outstrip one's own knowledge & experience (psychological factors aside).

ETA: latest version of the bot attached (per Monte Python's reasoning same name to avoid confusion ). Now enters a trade at session open (7 AM AST) depending on prevailing trend. Optimized on 1800 tick data for all of 2012 to present (which had the effect of adjusting the preemptive stop and interim profit target slightly. Backtesting indicates positive expectancy, as expected but Monte Carlo reminds us train wrecks can happen, and in any event potential drawdown is still significant.

Attached Files
Register to download File Type: cs TDSuperTrendQuantity.cs (10.4 KB, 16 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #87 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Road to hell being paved with good intentions, done for the day or not couldn't pass up the reaction to FOMC so in at what may be support central at 1.2486, trailing stop currently at 1.2507, 1 & only target off in the boonies (approx the 100 day MA at 1.2615ish) but need a whole lot of other people's buy stops to get triggered to make it that far. I'll let you know how that works out :-/

ETA: discretion the better part of valor sacrificed my trailing stop to the gaping maw of price at 1.2519 for a gain of 33 pips before commissions, ca. 100 pips short of the target. You can take the boy out of short time frame trading, but you can't take short time frame trading out of the boy. Part of the problem is, while the spectre of QE might spike the markets I can't believe it's good for the economy, so guess I live in fear traders en masse will awaken one morning with a conscience (and sell at the prospect of central planners printing more money). Not being totally risk averse however, in the meantime my gold swing (long at the moment) stays put Also, seems to me gold traders think a lot slower than currency traders...not an insult but in the sense of the mills of the gods, that grinding exceeding slow but exceeding fine....welcome relief to downshift now and then (because traffic has slowed rather than in search of torque)

While I finished the day ahead, and IMO when trading cash that is an issue, overall I feel like I've betrayed myself by betraying the plan. Oh well--may be time to change the plan then, if the bottom line is "be true to yourself [so far as yourself is profitable]". Time to choose between a glass of wine and yard work.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #88 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

Woke up late, tired & totally uninspired today. Followed the bot into a single trade, one contract long @ 1.2561 at 11:26 AM AST, out @ 1.2583 at 12:29 PM (session close) for 22 pips (ca. $215 after commission)


Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #89 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received

One short trade with 1 contract this AM trading (what I consider) chop stopped out for 1/2 pip profit after commission--my bad for chickening out and putting the stop too close to the action. Having my angel hat on may sit on my hands now until events at 8:30. On the other hand, bot like a fool just rushed in to go short 2 contracts @ 1.25345 (on paper) even though the signal from anaSuperTrend is mixed, currently ahead ca. 10-18 pips, trailing stop at 1.26139 and dropping.

ETA: 7:24 AST Bot stop now at 1.2586 and dropping, price consolidating 15 pips below short entry. I might be tempted to take a little off the table at this point and will probably add the option of moving to breakeven to the bot together with adding to a winning trade if conditions warrant and see what backtesting says about that. Not sure at the moment how to define "add....if conditions warrant".

My (day-) trend trading system is showing mixed signals but overall favoring a short if & when the time comes (negative momentum and 50SMA trend on 200-, 600- and 1800 tick charts but cycle not aligned yet--stochs needs to turn down on 200- and 600- tick charts ideally somewhere over 55, and MACD slope turn negative). Bot stop 1.25629 and dropping.

ETA: guessing that if EUR/USD breaks down through 1.2520 bidders it will start triggering sell stops down to 1.2500--not sure what will happen after that. Trick is to stay aware of what price is actually doing however and not bake in the attitude "trend is negative and it's going to drop any moment now."

ETA: by 7:53 short 1 contract at 1.2522 on cash account, selling into support essentially from ca. 1 stdev above median of current mode (much prefer the thin edge of the wedge where activity is down 3-4 stdev, in this case shorting S1 at 1.25265 ), initial stop above S1 at 1.2528 or roughly 1/2 my daily loss limit of 0.2% of the account, 1/2 contract targets at 1.2513 and just above S2 at 1.24935. Not a high probablity entry and even as I type this price has started looking positively buoyant


ETA: 8:03 AST moved cash account trailing stop to break even after commission, well within striking distance of price but feeling fragile & just not in the mood to take a loss this AM. Bot stop at 1,25621--flattening out while price makes up its mind.

ETA: 8:13 AST stop hit for $7 profit after commission--could be stop hunters but got to face the fact it's more likely just chop. Bite me. Since I'm feeling like a sissy probably ought to be doing something less brutal than trading, playing in freeway traffic say. Not going to see another entry for a while ; in any event will see what happens after news 8:30 EST.

ETA: 8:25 AST Bot still active, stop at 1.25546, already ahead 24 pips just as the floor falls away taking out what was the first target in the trade that aborted. I picture one of those idiot TV reporters who prey on shell shocked victims of disasters sticking a microphone up my nostril and asking, "Soooooo...how does that make you feel?". Bot ahead 28 pips or so, stop at 1.25472.

ETA: 8:37 watching bot at +32 pips just added "Thinking too much" under "Bad Habits" in my private journal. Maybe a long enough crayon up the nose will solve that problem...worked for Homer Simpson. Price pausing in its plunge, feeling the effect of the proximity to 1.25000. Bot stop essentially at break even.

ETA: ca. 9:03 AST, my trades so far today (first figure) vs bot trade (second figure)



ETA: 9:18 AST, 3rd figure (below) shows 3rd trade of the day, for a few dollars more, somewhat by the book except didn't place the stop until after 1st target hit--not the smartest move but perfectly consistent with "crayon up the nose" approach. In any event not much further ahead. NOW we wait for news at 8:30 AST.



ETA: 9:43 news turns out to be a non-starter, a few more long/short trades stopped out for a few more pips, up $40 today so far. This is nuts. Bot still placidly short 2 contracts on paper, stop now at 1.25323, up about $600, but as I write this I notice price going through the floor yet again...yep...bot briefly up just over $1000 :-/. IIRC bot's first intermediate target at 57 pips (still has 10-11 pips to go), stop at 1.25268. Time for the human to look for another setup, likely short around 1,2504 - 1.2512. Likely won't reach that until N American stock markets have opened, while opening range is setting up and during any boost associated with stock market index gap closure, so will have to be a little circumspect.

ETA: 10:15 AM, short entry placed at 1.2511 to give the algos something to process and to prevent me clicking buttons, price playing silly buggers just below, but otherwise thumb twiddling 15 minutes to N. Am stock market open. Bot stop 1.2525, so "guaranteed" 10 pips. In terms of psychology the FOMO (Fear Of Missing Out) is not strong in this one today.

ETA: 10:39 AM still thumb twiddling (see figure below) while price consolidates below my entry (which, if paying attention and quick enough I may cancel and replace with a short stop if price rises too aggressively), bot stop unchanged at 1.2525, its profits having plunged $300 - $400 to under $700.




ETA: 10:58 AM nudged entry upward to 1.2512 (what I'm treating as a range trade), top of 4-stdev regression channel since trend trader tea leaves are saying "time not auspicious to consider a short". Boy I'm nervous nelly today.

ETA: 11:01 AM EST, short at 1.2512 for better or worse. Stop a little pricey for me (in excess of 0.2%) and a show stopper if hit, but in the vicinity of what the bot is using. Besides, if chicken little gets half a chance he will move it between entry and price at the first opportunity.



ETA: 11:18 AM EST sure enough, moved the stop to 1.2511 and got stopped out for another few bucks profit (see figure below). Range trades like catching knives--may have to take up juggling chainsaws to get my nerve back. Bottom line is momentum has gone to zero and I don't like 50% odds, and trend trader ouija board still points to "go long at the earliest opportunity" (which means waiting to place a long limit order above support and a couple of pips above price when it looks like any subsequent retrace is running out of steam, or if it looks like a range trade with an upward bias). Resisting the strong urge to buy a breakout with the Dow spiking at 11:38 AST. (Oh look.....there goes something shiny! Did anyway--took the breakout--hit one target + trailing stop for another few bucks. What a stupid move. Will be going back to the drawing board this weekend )

ETA: 11:58 AST, about to enter last 30 minutes of European session. In the mood to watch for any short lived effect of longs/shorts bailing ahead of the weekend, although EUR/USD not responding while US stock market indices going parabolic--never a good sign.

ETA: 12:06 AST packing it in for the day. Made about $15 / hour today after expenses but before taxes--better than a poke in the eye but nowhere near enough to support me in the style to which I'd like to become accustomed.

Bot bailed at 1.25235 for a profit of 11 pips (2 contracts) on paper and immediately went long at the same price. It will shut down at end of session (12:30 pm AST 16 minutes from now), and is already down almost $100. If it doesn't pull this off may make an adjustment to its program (e.g. don't switch teams in the last 30 minutes).



ETA: 12:15 before I could finish typing that EUR/USD took off and gained about 33 pips in less than a minute, in an otherwise controlled move that kissed what I have as the floor pivot at 1.2558. Bot is now up another $400. How the hell does it do that???

Hope it has nothing to do with Empire State Building shooting just now That said, hate to discuss it but if so suspect market will give up last gains once it's accepted it's a disgruntled employee rather than a terrorist act. Could be due e.g. to other news which is not really news (from ForexLive.com)


Quoting 
Rumor Mill Cranks Up Again: ECB Considering Setting Yield Band Targets Under New Program

Reuters headline Citing central bank sources.
Expect a denial via Bloomberg in 20-30 minutes…
There have been rumors since the Draghi press conference that the ECB could set a target for yields spreads over the benchmark German bund, so this is merely a rehash.
The move above 1.2525 takes the immediate downside pressure off of EUR/USD and strengthens the support at 1.2480/90. 1.2590/95 is resistance, the later level the 38.2% of the drop from 1.3487.
Share and Enjoy:

By Jamie Coleman || August 24, 2012 at 15:17 GMT
Category: All, Americas, Central Banks, Regions || Tags: ECB || 0 comments || Add comment

but in any event rotated the sign in my trading shop window from "Closed" to "Open", short one contract @ 1.25475 on a wing and a prayer (certainly not momentum).

Bot exited up $594 after commissions for the day at session close.




Status of short at ca 12:50 PM AST, girlish stop in place target inspired by Lotto 649 & a glass of wine--5 pm somewhere (I have the greatest respect for women, especially women traders, among my best friends, including my wife, but being male tend to respect big, dumb manly stops & targets--just don't place stops that way).


ETA: 13:16 PM AST volume gone to zero, bias still negative for EUR/USD, ahead about 14 pips. Divided about whether to fold 'em or settle in for a long winter's discontented nap until close of trading. Amuse myself by playing Leonard Cohen's "Closing Time" in the back of my brain, then out of the speakers (if you click the link to the song, suffer the ad or find the small red X at the upper right of the video).


ETA: 13:52 PM AST up 11 pips, like having teeth pulled while watching paint dry. Or watching grass grow. Reminds me of baseball--IMO they use artificial turf in baseball arenas because it doesn't distract from the game. For you landscape painters--what zero momentum is like when ahead of the game. Still....better than getting a shower, having something to eat on top of the toast and salsa at 7:00 AM and 5 cups of coffee and working on the deck--better than getting a life, in other words, although having second thoughts. Even turned the phone on, first call from a son who wants to drop by on the weekend to talk about currency trading--no issues there. Traders back from an early lunch enter at 2:00 PM AST (1 PM EST) with no enthusiasm one way or the other yet. Thinking of moving my stop down into the suicide zone. Compromise by moving it to 1.2539, just above suicide (8.5 pips profit before commission). The know it all in the back of my head tells me I don't trade this for a reason. I respond--you think you know it all, but you don't. Price humours me, reinforcing bad behaviour--now up 15 pips. I count the layers of steel between my elation at making 15 pips and the black depression of having my stop taken out at +8.5 pips and imagine how nice a show would feel right now. Participation flatlined 40 minutes ago.

I like an envelope--gives me something to test now and then.

I feel like the stereotypical wounded guy who gets left behind to defend a position in the usual sci-fi horror flick. with an automatic weapon and a grenade, while the love interest gets away--come and get me you bastards.

ETA: 2:47 PM Alrighty--there goes the Dow (rising like a phoenix, straight up). Luckily EUR/USD a slow learner. Closed the position in a heartbeat (that would be the grenade, for you movie afficiendos), "Afternoon move" popped up on my daytimer at the same instant.
Picture of EUR/USD before it knows what hit it:


Went long at 1.25345 to test the hypothesis.

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #90 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,780 since Jun 2009
Thanks: 32,306 given, 97,567 received



Enjoy your weekend!

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to Big Mike for this post:
 
(login for full post details)
  #91 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks
 
bnichols's Avatar
 
Posts: 638 since Feb 2010
Thanks: 64 given, 460 received


Big Mike View Post


Enjoy your weekend!

Mike

Thanks Mike....been a long week

Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #92 (permalink)
Dartmouth NS
 
Experience: Intermediate
Platform: MC, MC.Net, NT, TWS
Broker: IB / IQFeed / Kids
Trading: Forex, stocks