NexusFi: Find Your Edge


Home Menu

 





Outside the Box and then some....


Discussion in Trading Journals

Updated
      Top Posters
    1. looks_one iantg with 66 posts (325 thanks)
    2. looks_two SMCJB with 12 posts (15 thanks)
    3. looks_3 artemiso with 12 posts (34 thanks)
    4. looks_4 pen15 with 10 posts (2 thanks)
      Best Posters
    1. looks_one iantg with 4.9 thanks per post
    2. looks_two wldman with 3.7 thanks per post
    3. looks_3 artemiso with 2.8 thanks per post
    4. looks_4 SMCJB with 1.3 thanks per post
    1. trending_up 31,189 views
    2. thumb_up 410 thanks given
    3. group 64 followers
    1. forum 137 posts
    2. attach_file 52 attachments




 
Search this Thread

Outside the Box and then some....

  #131 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
Posts: 408 since Jan 2015
Thanks Given: 90
Thanks Received: 1,147

For context jefforey is discussing my recent post here: . I provided a sample market replay output using tick data of a particular class of high frequency trading, as he inquired if anyone had any profitable backtesting results, so I posted screen grabs of one of my systems as thought experiment.

This particular class of HF strategy is already deployed in many markets by many HF trading firms. It is a variation of a known method, and I posted this in response to the original premise of the thread he started.

To jefforey's quote below: You will typically lose 1 tick for entry with limit orders via toxic fills and 1-2 ticks for entry with market orders. Most people use a limit order as their profit target and a market order as their stop loss. While there is hardly any slippage on the profit target, there can be 1-5 ticks slippage on the stop loss depending on the trade. It is very reasonable to assume the overall slippage for most traders even using limit orders on 2/3 of their order types would be 2-3 ticks. $4.00 commissions are also a reasonable commission rate, though retail commissions with no volume breaks can be in the $3.50- $3.75 range as well. So I essentially agree with everything Jefferey said below regarding the cost structure for most retail traders.

Having said that, none of this applies to the system that I posted market replay results for. This type of system works off of a completely different set of rules, with different entry and exit logic than typical retail traders use. These tools, while they are only available to those of us that can program complex rule based order routing, are available to any retail trader that can code well. You can be a retail trader and design a working system that never crosses the spread on either entry or exit. The trade off is you don't guarantee a fill in every case, but you guarantee that you never lose the spread when you do get filled. (Which is more important, is the question that this thought experiment addresses)

The particular class of system I posted also does not suffer from any sort of market direction bias. Often people are afraid of using limit orders because you miss the perfect entry. This system has no bias towards timing, alpha signals or anything else, only towards keeping the spread. While it does miss fills and miss trades, it still gets > 100 trades per day. And even though this particular example will cover a $4.00 commission cost handily and still be profitable, any system like this operating at this volume would come with a seat license and a broker volume discount and have a final all in cost of < $2.00 dollars.

I am not encouraging people to pursue this particular path, (Leaning to code, to try to build complex systems that beat the spread like this), but rather to point out that by thinking outside of the box you can build profitable systems, but sometimes you have to kick conventional wisdom to the curve and try something different. I hope that this encourages others to try to test some wild and crazy ideas and debunk the status quo.

Ian




jefforey View Post
My opinion is that for a retail trader slippage of 2 ticks per trade is a conservative figure. Yes the slippage could be more. If your trade has one entry leg and one exit leg then 2 ticks, if there are more legs than there could be more slippage. If you use limit orders only than the simulation is useless because many of your signals won't result in fills in actual trading. For one contract trade that is $25 per trade of slippage. Adding brokerage $4 (round trip) takes the total to $29 per trade. That means on an average every trade must win at least 3 ticks ($ 37.5) for the strategy to be profitable.


In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my NexusFi Trade Journal Started this thread Reply With Quote
The following 2 users say Thank You to iantg for this post:

Can you help answer these questions
from other members on NexusFi?
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
Request for MACD with option to use different MAs for fa …
NinjaTrader
NexusFi Journal Challenge - April 2024
Feedback and Announcements
ZombieSqueeze
Platforms and Indicators
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Retail Trading As An Industry
66 thanks
NexusFi site changelog and issues/problem reporting
48 thanks
Battlestations: Show us your trading desks!
37 thanks
GFIs1 1 DAX trade per day journal
32 thanks
What percentage per day is possible? [Poll]
31 thanks

  #132 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
Posts: 408 since Jan 2015
Thanks Given: 90
Thanks Received: 1,147

I haven't posted here in a while, but I just wanted to give a quick update on what I am up to.

For the last 6 months I have been building a High Frequency trading system using ninjatrader. This sounds like a crazy idea, and in retrospect it is likely a very stupid idea. There are limitations that this platform has that I have been working to overcome through a variety of insane workarounds.

I finished the primary execution algorythm recently and I am trading live, testing different edges, betting strategies and seeing what I can actually pull off given my obvious disadvantage in this space.

But I can actually say that I have optimized my code well enough, and with decent co-location I hit < 1 milliseconds on 50% of my executions on my last run. When I first starting building this unholly beast, the best time I got was 10 milliseconds and average time was around 50.

I have had to solve for so much random crap it's crazy.

1. Ninjatrader like 90% of all retail platforms doesn't take advantage of the CME MBO feed, so I have to build an algorithmic approach to estimating my place in the queue. Getting this part even close to the ballpark has taken months of testing live with different settings. I have learned a ton through this process.
2. Ninjatrader doesn't allow you to do the necessary things you would need to do to run a market making system with the managed approach, so I have had to take the training wheels off and go 100% unmanaged. No big deal here, but unmanaged can throw you completely under the bus. There are no kids gloves here, if you do something stupid you pay for it.... (Funny story below included)
3. I use a one to many cancel to fill ratio on all my orders, so I am handling all my cancels full custom with no OCO logic at all.
4. I optimized my execution speed to get to < 1 Millisecond execution speed even with a code that is > 5,000 lines through some optimization steps.
5. Ninjatrader has a lag with their level 2 data feed. When you get to the transaction level, the level 2 volumes lag the OnBarUpdate methods (GetCurrentAskVolume() for example by as much as a few seconds. So I have had to put a band aid on that.
6. I have optimized around my message signals so I can react even before my order position updates from Long to Flat, or Flat to Long for example. I have the fastest possible position processing methods worked out, so I can get all my executions off as fast as possible with NT.
7. Running my strategy on a memory intensive 1 tick time series, because no matter what the fastest method to see a price level change comes from the OnBarUpdate, and the update frequency is dictated by the bar size. This kills my memory, but I have no choice. I have found some ways to optimize the use of this so it doesn't kill my performance.
8. Hitting the level 2 book constantly to track my place in the queue, another extremely memory intensive process, I have had to solve for.
9. At any point in time in the market I have as many as 6 open orders at different price levels in different directions for different reasons. When certain ones fill, I need to cancel others, reprice others, and leave some still open. So my decision tree on this is insane.
10. I have no concept of which order is the entry and which order is the exit. I run two orders at all times resting K levels out, and when either of these hits the transaction level I drop a 3rd order on the opposite side of the spread. So I always have a buy bid and sell ask at all times and have to stay agnostic to which is the entry and which is the exit. So which ever one gets hit first, I have to create a whole reaction series of events around this. I can't wait for my position status to tell me I am long or short either. This subroutine is too slow, so I have had to solve for this chaos.

If nothing else, building all of this in C# via NT at least gets me to understand all the mechanics that are needed to do this. If it doesn't pan out on this platform I have every single bit of logic defined and working to take this core market making execution algorythm onto a different platform. I would just have to reverse engineer it, but the logic is all defined.

I hit a few snags along the way. The most comical one was I hit a bad loop where I canceled and resubmitted and NT had my virtual orders at 1 to 1 but the exchange wasn't landing but 10% of the cancels for every new one submitted, so I ended up with 25 open orders, and the Risk settings in NT kicked in and disabled my strategy, so I was left naked. This all happened in like 1 second, and before I could manually try to close my position, I had 25 redundant messages telling me that my margin account had exceeded my risk limit, and my strategy would be shut down, blah, blah, blah... And the whole system temporarily froze from all the messages being sent. The funny part... for all my stupidity not checking my loop more carefully and all things considered, I actually got out of this mess with + $50 dollars on the bad trade.

NT crashed a few times on me trading live. Vs. 11 had a bug I reported and they fixed. Version 6 had another bug. I now only use version 10. It has been fine. Starting with version 11, and then 12 they added more features which eats more memory and I need to be as lean as possible.

But overall, most of the mundane leg work of just getting up and running is over. Now I am testing out different strategies with different edge cases and seeing what NT can actually pull off. My huge speed disadvantage is not lost on me. Even hitting 1 millisecond execution speed, my real disadvantage is the data feed lag. By the time I see a new price level, or a huge volume change, the real HF guys have already reacted to it and are ahead of me. So I can't compete by the normal methods of waiting to the last possible minute to land a cancel, and trying to use speed to get to the front of the line. I have to use a different style of trading, and work off of a different bet, if you will.

I have developed a few bets that work fine and still pay out even with extremely high full retail commission levels. The caveat is there are certain ratios, and levels in my expectancy decision tree calculation that I need to hit. For example my scratch rate has to be X. My fill rate on limit orders from the side that wins the price level has to be Y. Some of these can be optimized just by settings in my strategy, and some of these are simple execution speed and information speed related and can not be optimized any further than I already have them.

So over the next month or so, I will be running different bets and seeing which ones I can pull off. If I would have realized that it would have taken me 6 months just to build the core 5K lines of this strategy I would have built my own platform. But I had no clue what I was getting into. It's been interesting to say the least. Stay tuned!

Ian

In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my NexusFi Trade Journal Started this thread Reply With Quote
The following 8 users say Thank You to iantg for this post:
  #133 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,033 since Dec 2013
Thanks Given: 4,359
Thanks Received: 10,172


Interesting. So are you making or losing money?

Reply With Quote
  #134 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
Posts: 408 since Jan 2015
Thanks Given: 90
Thanks Received: 1,147

Honestly, I have only been testing the mechanics of everything so far and getting all the freaking bugs out. I have had 50 or so trades so far, and I have paid less < 200 dollars including commissions for the testing. This includes a ton of F ups on my part, and around half of this was just running mostly queue optimizations and testing execution speeds with very simple entry / exit logic. I am not crossing the spread, and I am quoting from both sides, so even with no optimization I mostly scratch, lose 1 tick or make 1 tick. My burn rate on my margin account at this rate would be almost for ever even if everything failed, it would be a very slow burn.

As of this week I am finally to the fun part where I can start testing different real market making tactics. I am going to be trying different types of bets. For example:

1. Bet 1: Get to the top 10% of the queue from the strong side, cancel everything else. I know I will eat 100% of the losers with no time to cancel, but I will also capture around 75% or better of the times that the strong side wins.

2. Bet 2: Stay in the 50% range of the Queue from the strong side. When I hit the transaction level I should have time to cancel when things look bad in some cases. Can I land a high enough cancel rate and fill rate on winners at this threshold.... With my latency, this one will be interesting... I guess we'll see.

Bet 3: ***bets 1 or 2 combined with: Submit weak side order immediately when the new price level is created. I will catch 100% of all losers, but likely 50% to 75% of all winners also.

Bet 4: ***bets 1 or 2 combined with: Only submit orders to the weak side when my queue tracker marks me at my estimated fill. This will mitigate my risk of getting whacked immediately if only 10 people show up to the weak side and it breaks. But I also run a slight risk of not getting my order even submitted in time in some cases.

Bet 5: ***bets 1 or 2 combined with: Only submit orders to the weak side if the volume is at least 25% of the strong side volume. There is higher risk of the weak side immediately breaking if the volume never gets to this level or better. Again, some latency risk here... I might not get my order out in time to have a shot at quoting both sides.

As you can imagine most of the strategy is not on picking direction, but rather managing your toxic fill rate. These bets would all play out well, if I had ideal speed of information and execution, but since I don't I have to get creative and see what I can pull off given my disadvantage.

I am debating on what aspects of everything I should be sharing... I could share either some of the technical aspects or some of the strategy aspects but not likely both without running the risk of revealing too much. I tend to not care about revealing too much typically because hardly any one is crazy enough to try this kinda stuff, and it may not even pan out considering my obvious disadvantages speed wise.

I know exactly where the lines are on some of these bets in terms of the pass or fail for the strategy being profitable. I just don't know what NT can achieve in terms of overall average execution speed and the average speed reacting to information. In a 0 latency environment, (NT Market Replay) I am hitting between 5 to 7 dollars per trade on some of these bets I will be testing. I am hitting fair execution speeds, but my information speed sucks and this is the part that worries me the most.




SMCJB View Post
Interesting. So are you making or losing money?


In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my NexusFi Trade Journal Started this thread Reply With Quote
The following 4 users say Thank You to iantg for this post:
  #135 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,033 since Dec 2013
Thanks Given: 4,359
Thanks Received: 10,172

Good Luck to you. Please keep us informed.

Reply With Quote
The following user says Thank You to SMCJB for this post:
  #136 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
Posts: 408 since Jan 2015
Thanks Given: 90
Thanks Received: 1,147

So I have been pretty deep in some weird rabbit holes for the last few months. I have built a few trading strategies that work but now I have hit some challenges with the NT data model. So I am having some latency issues I am solving for.

Case in point: https://forum.ninjatrader.com/showthread.php?p=541346

The last few posts explain the issue in detail but basically both the OnBarUpdate and OnMarketData subroutines can at different times lead or lag each other. Knowing which one to use to get the most recent update to the bid / ask price is the challenge.... So this requires a complex work around, as they have stated that they won't fix the issue.

I am pretty much at the point where I have a fair view of the micro-structure of the ES and now I am just working on which bets I can pull off given the latency that arises from this data structure issue.

Anyway, I may share some of this research on here, or post an ES micro-structure thread if there any interest. It is very difficult to get a clear view of the market and some of the rabbit holes I have gone down might help others.

I am debating what aspects to share.... Either I can share more of my general strategy and the betting logic, or more of the code and technical stuff but not both for obvious reasons. So where there might be more interest I will continue sharing.

More to come soon...

Ian

In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my NexusFi Trade Journal Started this thread Reply With Quote
The following 4 users say Thank You to iantg for this post:
  #137 (permalink)
 
ratfink's Avatar
 ratfink 
Birmingham UK
Market Wizard
 
Experience: Intermediate
Platform: NinjaTrader
Broker: TST/Rithmic
Trading: YM/Gold
Posts: 3,633 since Dec 2012
Thanks Given: 17,423
Thanks Received: 8,425

Hi @iantg, always appreciate reading your detailed stuff, thanks!

The two main issues I see are:

1) Using any 'GetCurrentBid()/GetCurrentAsk()' calls from inside OnBarUpdate is largely pointless, as OnBarUpdate will only be called for 'Last' price changes, regardless of speed or any other settings.

2) Whatever OnMarketData or OnMarketDepth calls you rely on will still be subject to the vagaries of the Windows system thread scheduler, so you can forget any guaranteed timing requirements; I also suspect this may be responsible for some of the larger delays you see.

Tough world out there!

Cheers

Travel Well
Visit my NexusFi Trade Journal Reply With Quote
The following 2 users say Thank You to ratfink for this post:
  #138 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
Posts: 408 since Jan 2015
Thanks Given: 90
Thanks Received: 1,147

Hi ratfinik,

Thanks for the quick feedback.... You're hitting exactly on the type of "pick your poison" scenario that we are in with NT..... and this may very well be par for the course with just about any retail level trading software. Heck, even the pro's likely struggle with this a bit. But your absolutely right about the OnBarUpdate being largely useless in the context of what I am trying to use it for since it only pulls from the "last price". And the only reason it is even in the conversation at all is because it has to be.... The OnMarketData / OnMarketDepth feeds can catch so much overhead that the simple GetCurrentBid() / GetCurrentAsk() calls from OnBarUpdate actually front run them, which is ridiculous!

Just about every time the GetCurrentBid() or GetCurrentAsk() takes a new update, the associated and price and volume will typically be 100 milliseconds to 2 to 3 seconds ahead of anything running from any other feed. All of this of course assumes you are running a 1 tick time series.

I'll likely post some code with examples later to show what I am solving for.

Ian



ratfink View Post
Hi @iantg, always appreciate reading your detailed stuff, thanks!

The two main issues I see are:

1) Using any 'GetCurrentBid()/GetCurrentAsk()' calls from inside OnBarUpdate is largely pointless, as OnBarUpdate will only be called for 'Last' price changes, regardless of speed or any other settings.

2) Whatever OnMarketData or OnMarketDepth calls you rely on will still be subject to the vagaries of the Windows system thread scheduler, so you can forget any guaranteed timing requirements; I also suspect this may be responsible for some of the larger delays you see.

Tough world out there!

Cheers


In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my NexusFi Trade Journal Started this thread Reply With Quote
The following 2 users say Thank You to iantg for this post:





Last Updated on June 23, 2018


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts