Unfiltered data feeds and HFT, detecting algos with orderflow? - Reviews of Brokers and Data Feeds | futures io social day trading
futures io futures trading


Unfiltered data feeds and HFT, detecting algos with orderflow?
Updated: Views / Replies:5,615 / 17
Created: by cashinvestor Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Unfiltered data feeds and HFT, detecting algos with orderflow?

  #1 (permalink)
Elite Member
Spokane, WA
 
Futures Experience: Intermediate
Platform: ninjatrader, tradestation, esignal
Favorite Futures: Futures, Futures Options
 
cashinvestor's Avatar
 
Posts: 13 since Jul 2009
Thanks: 0 given, 10 received

Unfiltered data feeds and HFT, detecting algos with orderflow?

I recently attended a webinar for Shadow Traders. The CIO, Barbara, says that she is primarily a programmer, not from a trading background. And since she has coded for some trading houses has inside knowledge of HFT structure and behavior. Also that trading is no longer a technical analysis game, but a programming game. I can attest that writing strategies using standard TA tools (like stochastics and macd) are very hard to make profitable. It seems like more time is spent trying to stay out of bad trades than getting into good trades.

I'm also a programmer and know where she is coming from. I'm not a Wall Street coder, but I am doing a lot of study about it. Goldman makes $50M a day front running stock quotes. The Quants are PhD physicists, mathematicians, statisticians, etc. They are coding up some nasty looking math equations. But their algorithms work best when the market is acting nice. As soon as things get wonky, their models don't apply any more.

I'd like to hear from any experienced coders from Wall Street. Is it true that the HFT robots can be detected by watching the order flow database? I googled "order flow database" and only got back a single page of hits, most of the references being on people's resumes. Can Shadow Traders really detect when the HFT engines start trading, or is increased volume activity just the outside behavior and they are attributing it to HFT.

I know that an unfiltered data feed is better overall, but is it really a make-or-break scenario? Tradestation has a lot of customers trading every day. Can they really be all losers because they aren't getting rocket speed unfiltered data?

Is watching the order flow database in this fashion only good for scalping? If I want to catch 60 ticks and stop for the day, do I really care if the HFT's are running the market around. I'm looking for setups and building stop movements around the bigger move.

I don't have enough inside knowledge and experience to validate what Barbara is saying. Does anybody here?

Thanks.

Reply With Quote
The following 2 users say Thank You to cashinvestor for this post:
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Membership Revoked
Oslo, Norway
 
Futures Experience: Advanced
Platform: CQG, Excel
Favorite Futures: CL
 
Lornz's Avatar
 
Posts: 1,198 since Apr 2010
Thanks: 500 given, 1,235 received


Since Shadow Traders have been mentioned in at least three different threads this past week, I caved in and did a search on Youtube:



I am amazed that she sells products; she seems completely clueless. These fraudsters piss me off! Apparently @tigertrader knows who she is, and he did not seem impressed either.

If you have a decent feed, you can get access to cancels. I would guess that she simply looks at increased activity (number of ticks p/s, e.g.), maybe combined with analysis of the fluctuating order depth (doubtful).

As retailers we can just forget about competing on speed. Take a step back and try to catch some of the bigger moves...

Reply With Quote
 
  #4 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,237 since Jun 2009
Thanks: 29,350 given, 83,150 received

Moderator Note


This thread will NOT be about Shadow Traders. Use existing threads. All posts made about Shadow Traders will be deleted from this thread. This is not a thread to discuss a vendor.




Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #5 (permalink)
Membership Revoked
Oslo, Norway
 
Futures Experience: Advanced
Platform: CQG, Excel
Favorite Futures: CL
 
Lornz's Avatar
 
Posts: 1,198 since Apr 2010
Thanks: 500 given, 1,235 received


Big Mike View Post
Moderator Note


This thread will NOT be about Shadow Traders. Use existing threads. All posts made about Shadow Traders will be deleted from this thread. This is not a thread to discuss a vendor.




Mike

Yeah, I guess my post was a little out of place. But it seems that they have been overrepresented on this board the past week or two.

My point was just that I wouldn't really give much credence to her opinion based on her portrayed "knowledge".

Not that I claim to be all-knowing on the subject either.

Reply With Quote
 
  #6 (permalink)
Elite Member
Spokane, WA
 
Futures Experience: Intermediate
Platform: ninjatrader, tradestation, esignal
Favorite Futures: Futures, Futures Options
 
cashinvestor's Avatar
 
Posts: 13 since Jul 2009
Thanks: 0 given, 10 received

I don't want to talk about them. I want to know about the order flow database.

How do you look at it from a programmers view inside of ninjatrader? Is there any sample code for looking at transactions, volume, etc that might come close to this?

In MetaStock I created a chart that plotted each tick as a dot and price action looked like a wide spray paint swatch that moved up and down. So that was my first inkling to how transactions were being recorded. But I wasn't programming the platform back then.

I've thought about the bid/ask model and how I might dig into it from ninja. Is there any clues in there that might give an edge?

Reply With Quote
The following user says Thank You to cashinvestor for this post:
 
  #7 (permalink)
Membership Revoked
Oslo, Norway
 
Futures Experience: Advanced
Platform: CQG, Excel
Favorite Futures: CL
 
Lornz's Avatar
 
Posts: 1,198 since Apr 2010
Thanks: 500 given, 1,235 received


cashinvestor View Post
I don't want to talk about them. I want to know about the order flow database.

How do you look at it from a programmers view inside of ninjatrader? Is there any sample code for looking at transactions, volume, etc that might come close to this?

In MetaStock I created a chart that plotted each tick as a dot and price action looked like a wide spray paint swatch that moved up and down. So that was my first inkling to how transactions were being recorded. But I wasn't programming the platform back then.

I've thought about the bid/ask model and how I might dig into it from ninja. Is there any clues in there that might give an edge?

I don't really know what she means with "order flow database". I assume she is referring to Level III. To be quite honest, I am not really sure how much information certain firms gets from the exchanges. Maybe someone else can chime in on that.

I started looking into the possibilities of pursuing something like this, but I quickly discovered that it was hopeless. Just the cost of an in-memory database like kdb+ is about $200k/year. You also have one-stop shops like OneTick and Deltix. The latter looks really interesting for my purposes, but it costs around $6-8000/mo, excluding data. Add about the same amount for a good data feed and it adds up quickly.

I'm not really a programmer, so I can't really say what people can do with NT or not. But last time I checked, it didn't have CEP capabilities!

I must warn that I am really out of my depth here, and I am simply offering a layman's perspective. However, I did spend a considerable amount of time last year researching various options.

Keep in mind that the feeds we retailers use are crap for such purposes. There are missed ticks and services like IQFeed only timestamps to the second. NxCore does sub-second timestamps and is priced reasonably. I'm not really aware of any other comparable solution in that price range.

As for my personal experience, there used to be a lot of information in the DOM, but it has changed drastically with the advent of HFT. Quote stuffing ruined a lot of my analysis. I export the DOM (along with other data) to Excel, and there were a lot more useful things one could gain from that several years ago. Things have gotten more "primitive" now, but I still make money. Now I am reliant on catching the bigger swings, though.

Portraying ticks as dots, representing the size of the order, is a very intuitive way of looking at price action. @Vegas has made a product that does just that.

Are you a programmer by trade? I became increasingly frustrated with the limitations of all the platforms I've used, I finally decided to take some CS classes this year. It seems that nearly all of my ideas require custom programming not supported by retail platforms, and I am tired of doing things semi-manually in Excel.

Reply With Quote
 
  #8 (permalink)
Trading Apprentice
New York, NY USA
 
Futures Experience: Advanced
Platform: R-Trader
Favorite Futures: Currency - EUR
 
Posts: 11 since Mar 2012
Thanks: 4 given, 6 received

For direct trade routes search the exchange web sites

For direct trade routes search the exchange web sites. for example, cme site - search trading platforms.

it gives a full list of direct data feeds

Reply With Quote
 
  #9 (permalink)
Trading Apprentice
New York, NY USA
 
Futures Experience: Advanced
Platform: R-Trader
Favorite Futures: Currency - EUR
 
Posts: 11 since Mar 2012
Thanks: 4 given, 6 received

i looked at shadow trader before, about 4 years ago. i can not believe they still around. for rookies...anyone know a moving over cross over with fancy background coding

Reply With Quote
 
  #10 (permalink)
Elite Member
Spokane, WA
 
Futures Experience: Intermediate
Platform: ninjatrader, tradestation, esignal
Favorite Futures: Futures, Futures Options
 
cashinvestor's Avatar
 
Posts: 13 since Jul 2009
Thanks: 0 given, 10 received


yes, I am a programmer by trade. Even though I've been a software engineer for 30 years, I've been out of the loop as a full time programmer. So there are a lot of technologies that I don't know. I've only been coding on ninja for a short time. But I like C# as a language and it has career enhancing properties.

From my research, I know that most of the HFT programs are running as C++ under Linux on big PC-based hardware hanging off the exchange. This is the fastest it can get.

The London Exchange tried going to a Microsoft infrastructure and it died. They had to scrap it and go to a Linux based structure.

So writing C#/.Net/Windows over the open internet is not a good formula for fast transations. But that's not what we are doing anyway.

I'm also looking into what protocols the HFT's are using, like FIX. This is supported by the CME and many vendors use it. But I'm thinking that to go that route, you have to write a custom C++ app for talking to the exchange directly and not using any visual charts, etc.

There is an open source java-based framework that does a similar thing. No graphics, just algorithms and order management.

Right now I'd just like to experiment with what ninja can do at the bid/ask level and see how far that can go. One interesting example is the indicator that plots the speed of order flow. I can't remember what it's called, but it is freeware. When orders pick up speed, he displays the bars in a different color and I think there is an audio alert.

I would like to do something like measure the orders based on whether they are at the bid or at the ask. This should give indication as to where the transaction flow might be headed.

Reply With Quote
The following user says Thank You to cashinvestor for this post:

Reply



futures io > > > > Unfiltered data feeds and HFT, detecting algos with orderflow?

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

Dec 7

Linda Bradford Raschke: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Treasury data feeds elementbrdr Reviews of Brokers and Data Feeds 6 October 16th, 2011 07:44 PM
European Data Feeds Laurent Reviews of Brokers and Data Feeds 18 November 12th, 2010 08:49 PM
Two data feeds on same network ahrinc Reviews of Brokers and Data Feeds 20 November 11th, 2010 01:20 PM
Paid Data Feeds Michael.H The Elite Circle 27 June 8th, 2010 08:44 PM
Which data feeds chugged thru today? Day Trading Fool Reviews of Brokers and Data Feeds 24 May 17th, 2010 04:11 PM


All times are GMT -4. The time now is 09:21 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-11-18 in 0.14 seconds with 19 queries on phoenix via your IP 54.80.157.133