NexusFi: Find Your Edge


Home Menu

 





Unfiltered Data


Discussion in Brokers

Updated
      Top Posters
    1. looks_one Big Mike with 10 posts (110 thanks)
    2. looks_two MethylSeasons with 5 posts (1 thanks)
    3. looks_3 Beljevina with 4 posts (8 thanks)
    4. looks_4 RM99 with 3 posts (2 thanks)
      Best Posters
    1. looks_one Big Mike with 11 thanks per post
    2. looks_two NoBias with 10 thanks per post
    3. looks_3 Beljevina with 2 thanks per post
    4. looks_4 RM99 with 0.7 thanks per post
    1. trending_up 74,271 views
    2. thumb_up 133 thanks given
    3. group 35 followers
    1. forum 43 posts
    2. attach_file 4 attachments




 
Search this Thread

Unfiltered Data

  #31 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,322 since Jun 2009
Thanks Given: 33,143
Thanks Received: 101,476


MethylSeasons View Post
So it appears that using either Rithmic or Zenfire with IB would be the best and cheapest option... Would you agree?

Now I just need to see if I can use either or those with IB.

Zenfire is dead after NinjaTrader acquired Mirus (who owned Zen-Fire).



I don't know what "use with IB" means. But it sounds like you think you can use a data feed other than IB with TWS, and that is definitely never going to happen.

If you mean "use with IB" as in IB for execution, and data feed for data, then of course that is fine so long as you don't use TWS. You can do it with Sierra Chart for example (which I use) or with NinjaTrader for example (which many others use). Both of which are off-topic for this thread, which is supposed to be about unfiltered data.

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote

Can you help answer these questions
from other members on NexusFi?
ZombieSqueeze
Platforms and Indicators
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
NexusFi Journal Challenge - April 2024
Feedback and Announcements
Request for MACD with option to use different MAs for fa …
NinjaTrader
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Retail Trading As An Industry
66 thanks
NexusFi site changelog and issues/problem reporting
48 thanks
Battlestations: Show us your trading desks!
37 thanks
GFIs1 1 DAX trade per day journal
32 thanks
What percentage per day is possible? [Poll]
31 thanks

  #32 (permalink)
 eagle 
Europe
 
Experience: Intermediate
Platform: IRT, Ninja
Trading: ZC
Posts: 8 since Jul 2011
Thanks Given: 22
Thanks Received: 1


Big Mike View Post
Here is my opinion.

Filtering isn't about getting rid of bad ticks. It's about latency and completeness (accuracy).

Interactive Brokers is widely known for filtering data. It's snapshot data, meaning that they aggregate info, and send you a 'snapshot' of what it looks like every 250ms or so. This means you are not getting all the ticks, not getting all the L2 depth changes, not even getting all the volume. If you trade daily bars or even minute data, then you won't care about this. But if you are using tick based data: ie tick, range, renko, volume or other exotic bar types, then you should not be using IB for data.

On top of that, IB has big restrictions on their historical data. For this reason, my conclusion is that IB filters data simply due to lack of resources on their end, and probably because overwhelming majority of their customers are using minute based or daily based data.

IB is a combination execution + feed provider.

Next we have the Rithmic's and Zen Fire's of the world. They send every tick. They are well known in the industry as unfiltered data. They also have a reputation for being ultra low latency (fast). This is very appealing to a large amount of more 'advanced' traders that want to use tick based charting.

However, they don't have their own historical servers (some platforms provide it for their users, it varies). They also do filter some L2 depth changes. In order to be as fast as possible, and get you that tick as fast as possible, the trade off is to not send some of the L2 changes. Even most of the 'advanced' traders today are not measuring L2 changes so pretty much no one is complaining about this or cares about it.

Rithmic/Zen Fire is an execution/risk system more than it is a data feed. The focus on the feed is execution, which is also why the focus on the feed is latency. They do this extremely well. I've heard rumors for many years that Zen Fire is branching out and adding other exchanges or breadth tools soon. Maybe they will? But for now, it's really designed as execution only, although a lot of people use it for data as well and are completely happy with it, provided they also use a platform (like NinjaTrader) that gives them historical data.

Next we have the IQFeed's and Kinetick's (oem) of the world. They send every tick. They also have an extensive historical database and ticker plants. They all major exchanges. They have something like 500 breadth tickers.

IQFeed's (and Kinetick) business is data. Not execution. Since their focus is on data, they send every L2 update. They send every tick. They send everything, and they also send it in the right sequence. It is the best quality feed currently available to a retail trader, in my opinion. They do give up some latency when compared to Rithmic/Zen Fire, as there is a price to be paid for completeness.

IQFeed also provides historical bid/ask backfill. If you don't know what that is, you don't need it. Only some platforms support it, such as Sierra Chart, Investor/RT, MarketDelta. NinjaTrader 8 is supposed to support it. If you need this then IQFeed is not only an excellent choice, but pretty much the only choice.

As for TradingTechnologies TT FIX, it is an execution feed. It's focus is on execution, although many do use it for data as well.

As for which feed is right for you, only you can answer that.

Mike

Hi is someone able to provide any more recent info on this:
I am looking at the options offered by AMP Global. I am looking for an unfiltered feed and the options are
CQG , TTnet and Rithmic.
Latency of the feed would be less of a concern on my side. More of a concern would be all the ticks should be non aggregrated. Accurate Bid / Ask trade classification

Can anyone advise if these feeds are all unfiltered? Any views on the Bid Ask trade classification

Thank you in advance

Reply With Quote
  #33 (permalink)
 
Beljevina's Avatar
 Beljevina 
Toronto, Canada
 
Experience: Advanced
Platform: NinjaTrader, SierraCharts
Broker: Amp/CQG, IB
Trading: CL, GC, 6E, ES, RTY, EURUSD, GBPJPY, AUDUSD
Posts: 352 since Sep 2010



eagle View Post
Hi is someone able to provide any more recent info on this:
I am looking at the options offered by AMP Global. I am looking for an unfiltered feed and the options are
CQG , TTnet and Rithmic.
Latency of the feed would be less of a concern on my side. More of a concern would be all the ticks should be non aggregrated. Accurate Bid / Ask trade classification

Can anyone advise if these feeds are all unfiltered? Any views on the Bid Ask trade classification

I use the CQG from Amp option, as provided to it's Ninjatrader users (now only available to new NT7 users that have purchased their own NT7 license.). The feed is slightly different from the full (and much more expensive) CQG datafeed: it is made up of realtime incoming data and is provided by CQG directly - it is completely unfiltered, meaning you get all trade ticks and all the bid/ask info in that is available. This allows me to have completely accurate footprint charts showing exact buyers and sellers at every price level (which I record via GOM, for the entire opening duration of the market through each and every week). I have compared my footprint charts to the other datafeeds you have mentioned, and they are almost always exactly the same.

The difference with the Amp CQG feed for NT7 users is that whenever NT7 makes a request for historical data, it comes from NT's own housed servers, from data they have recorded for that instrument. They used to have issues where their servers would go down every so often on the weekend or an overnight and then when you request historical data, it is not available to you: this problem appears to have been rectified by them and it has been quite some time where this has now not been an issue.

Follow me on Twitter Reply With Quote
  #34 (permalink)
 eagle 
Europe
 
Experience: Intermediate
Platform: IRT, Ninja
Trading: ZC
Posts: 8 since Jul 2011
Thanks Given: 22
Thanks Received: 1

Thank you Beljevina for providing some more insight.

I am wanting to use it with IRT who routes CQG data through the marketdelta trader platform. Interestingly I got a reply from IRT which said I should go with Rithmic.
With IRT there is the option to backfill through I believe DTN. The only concern with Rithmic is the additional per side pricing fee which adds up over time.

When you have tested you CQG feed, have you eyeballed it ( which I tend to do) or have you dug into data on different feeds with excel or something similar. What sort of discrepancies have you noticed or picked up with regards any of this?

Thanks

Reply With Quote
  #35 (permalink)
 traderwerks   is a Vendor
 
Posts: 692 since Jun 2009
Thanks Given: 436
Thanks Received: 465


Beljevina View Post
The difference with the Amp CQG feed for NT7 users is that whenever NT7 makes a request for historical data, it comes from NT's own housed servers, from data they have recorded for that instrument. They used to have issues where their servers would go down every so often on the weekend or an overnight and then when you request historical data, it is not available to you: this problem appears to have been rectified by them and it has been quite some time where this has now not been an issue.

I am still having a problem with NT7 & NT8 requests to the Ninja servers. They have been missing some data and I have been getting an error on the front month contract since the last week of December. @NinjaTrader siad there is a problem , but they do not have an ETA on a fix. It has been a week already so I don't think they are in a hurry.

Math. A gateway drug to reality.
Reply With Quote
  #36 (permalink)
 
Beljevina's Avatar
 Beljevina 
Toronto, Canada
 
Experience: Advanced
Platform: NinjaTrader, SierraCharts
Broker: Amp/CQG, IB
Trading: CL, GC, 6E, ES, RTY, EURUSD, GBPJPY, AUDUSD
Posts: 352 since Sep 2010


eagle View Post
I am wanting to use it with IRT who routes CQG data through the marketdelta trader platform. Interestingly I got a reply from IRT which said I should go with Rithmic.
With IRT there is the option to backfill through I believe DTN. The only concern with Rithmic is the additional per side pricing fee which adds up over time.

When you have tested you CQG feed, have you eyeballed it ( which I tend to do) or have you dug into data on different feeds with excel or something similar. What sort of discrepancies have you noticed or picked up with regards any of this?

I don't know enough about MarketDelta and how it does it's cumulative delta/footprint (if you are using that feature), and hence how it builds, collects, stores that data, either real-time or historically; I mention this as a contrast to GOMi's fabulous tools for NT footprint and delta, which I completely depend on, and find essentially flawless in their operation. Hopefully you have a technical understanding of why they would recommend Rithmic; I mention that because help-desks usually have a bias and often steer their customers one way, for non-technical reasons.

My CQG/Amp usage over the past 4+ years I'd say is somewhat beyond generic eyeballing: I've compared the footprint chart's bid x ask prints in entire bars, trying to focus in on what I knew to be fast moving periods in the market, and comparing colleague's charts either on SierraChart and/or MarketDelta, and matching up those bid x ask levels. I've done this for ES, CL, ZN, 6E and others. Yes, there are rare differences, but, on say a $CL print of 720 x 674 at $34.70, if it's 5 to 10 to 20 contracts more or less, I don't see a reason to be concerned; usually I can find the discrepancy in the tick above or below, which, is indicative of an incredibly fast moving market as well as uncontrollable software where the GOMi indicator 'slotted' the up/down ticks perhaps in the opening or closing seconds into a price level above or below, or, it is in the next bar if that was the opening or closing tick. In other words, because delta is not part of the datafeed, there can be minor software impacts that will make a 5 minute footprint chart appear different on NT vs SC vs MD. My sense is that it is better than 99% similar. And, if one is different versus the other (by 1% at an incredibly fast point in the market), the question is ... so what? Will it impact one's profitability? I don't think so. My advice is that there is a far greater spectrum of impact we as a trader introduce to the mix, making potentially bad decisions completely unrelated to incredibly small variances in a footprint or delta print: note I am assuming your chart's individual bar's high, low and other non-footprint data is completely the same. Also, things like OCO-server side, and latency frankly I feel are more important. I hear you on the pricing differences, but brokers can adjust those for their clients to be more competitive (just ask them) and, with ES at $12.50 per tick, I will put in limited effort/concern towards a 20¢ full-turn price variance.


traderwerks View Post
I am still having a problem with NT7 & NT8 requests to the Ninja servers. They have been missing some data and I have been getting an error on the front month contract since the last week of December. @NinjaTrader siad there is a problem , but they do not have an ETA on a fix. It has been a week already so I don't think they are in a hurry.

I'm not aware of any chart gaps, but, I could be wrong and not seeing it, or, because I also run my NT7 during every waking minute of the futures markets and my charts will get built from historical data when I do restart NT7, perhaps I am not impacted. If you want to PM with details, I'd be happy to see if I'm experiencing the same issue, or, if I could help or somehow shed light on the topic.

Follow me on Twitter Reply With Quote
  #37 (permalink)
 eagle 
Europe
 
Experience: Intermediate
Platform: IRT, Ninja
Trading: ZC
Posts: 8 since Jul 2011
Thanks Given: 22
Thanks Received: 1

Thank you Beljevina for the clarifications.

You've obviously spent some time with this by the looks of things , and bring through some great understanding.
You point as well on the tick size and price per trade is a valid one as well. i sometimes tend to forget its just the cost of doing business.

Thanks for the lucid explanations, Good luck with your trading

Reply With Quote
  #38 (permalink)
mks212
NY, NY
 
Posts: 12 since Apr 2015
Thanks Given: 6
Thanks Received: 1

This is a helpful post. I was looking at charts/data in tradestation and then going into Interactive Brokers to place some trades and I realized that the high and low of day columns did not match. At first I thought it was due to IB considering the premarket and tradestation using the actual trading day.

I then looked at 5 min charts in both platforms and those don't match either. I find that disconcerting since many traders use 5 min bars and there is no way to know which one is accurate.

I downloaded about a year's worth of daily prices of BIIB from both brokers' software and compared them. I am surprised to see that even on daily bars there is a decent amount of difference, particularly in the opening prices. I have attached the sheet with differences. Tradestation is the first set, IB is in the second set starting in column G and the IB Value less the respective Tradestation Value is the third set starting in column M.

As an aside, I find that Tradestation and ThinkorSwim data matches for whatever that is worth.

Attached Files
Elite Membership required to download: Difference in Daily data From IB and Tradestation.xlsx
Reply With Quote
The following user says Thank You to mks212 for this post:
  #39 (permalink)
 
xplorer's Avatar
 xplorer 
London UK
Site Moderator
 
Experience: Beginner
Platform: CQG
Broker: S5
Trading: Futures
Frequency: Never
Duration: Never
Posts: 5,925 since Sep 2015
Thanks Given: 15,421
Thanks Received: 15,235


Beljevina View Post
I use the CQG from Amp option [...] The feed is slightly different from the full (and much more expensive) CQG datafeed: it is made up of realtime incoming data and is provided by CQG directly - it is completely unfiltered, meaning you get all trade ticks and all the bid/ask info in that is available.

What difference in price do you see?

Reply With Quote
  #40 (permalink)
 
Beljevina's Avatar
 Beljevina 
Toronto, Canada
 
Experience: Advanced
Platform: NinjaTrader, SierraCharts
Broker: Amp/CQG, IB
Trading: CL, GC, 6E, ES, RTY, EURUSD, GBPJPY, AUDUSD
Posts: 352 since Sep 2010



xplorer View Post
What difference in price do you see?

I am not exactly sure how to determine the precise price for an equivalent CQG set of instrument and data feeds; their page at Products | [AUTOLINK]CQG[/AUTOLINK], Inc. doesn't seem to specifically offer pricing solely for the datafeed, but may bundle it with their other products. In general terms and without doing specific research for you, the cost for datafeeds can be as little as $60/month (Kinetick?, but may not include markets like the Russell, Brent, FDAX, etc) and from past memory, Rithmic, which is likely upwards of $90). Bear in mind too, that a datafeed is one thing, but, entering and exiting trades is determine by whatever broker you choose, and thus their infrastructure to successfully route your trade to your instrument's exchange.

My point simply in my posts(s) was that there are many bundled solutions from brokers or other sources, that will provide for an incredibly high quality datafeed at minimal cost. And, my advice is to pick a reputable unfiltered data source, and not spend silly amounts of time exploring that avenue, when, it is the quality and success of each of our abilities to successfully trade in to and out of positions, that are the real cornerstone of making it or breaking it as a trader.

Follow me on Twitter Reply With Quote





Last Updated on May 6, 2020


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts