I keep seeing these advertisements for unfiltered data.....

Isn't the whole point of filtering the data to ensure that a rogue/erroneous data point won't ruin your day? If you have a stop in place and a bad piece of data comes in, that exceeds your stop, most platforms treat stop orders as "market if touched or exceeded" orders.

What's the whole issue with having "unfiltered" data?

Thanks Mike, so as long as I'm not scalping or involved in the weeds, then filtered data (like Tradestation) should be sufficient for my purposes?

My trading strategies aren't made or broken based on the accuracy of the next few hundred miliseconds or my order in the book at the next ask/bid.

My charts and indys are populated historically and use end of bar calc's, so I should be good right?

I'm not shopping, I was just curious and the whole data quality/filtered/unfiltered thing is new to me.

I would like to take advantage of data timestamped down to the second rather than the minute however, which would allow me to explore some multi-timeframe ventures on advanced charts.

So all that I've read about rogue/errant data points is just hype?

From what little research i did, I saw horror stories about people using unfiltered data and bad ticks coming in and screwing with their DOMs/Matrixes, etc.

If you are using TradeStation, then you can choose any data provider you want, so long as it is TradeStation. You can also choose any broker you want, so long as it is TradeStation. TradeStation is in all-in-one solution, and as such isn't really a leader in any field, in my opinion.

Mike

I feel the pain of all that, what I'm asking is that if I'm not scalping or dependent upon hyper quick calculations and execution if it's going to make an appreciable difference for me?

I guess for everyone's benefits, which "types" of traders benefit most from having unfiltered data and which types is it really a non-issue?

Having never used TradeStation, I can't say. But my understanding there is an option in the menu somewhere to disable filtering and send you all the ticks.

Asking about scalping is the wrong question. I mean, I guess it is possible to scalp on a minute chart. The better question is tick data, or not tick data. I cannot comment on TradeStation's data feed.

As for which traders benefit, I believe I covered everything in my original reply. ?

Mike

Here is my opinion.

Filtering isn't about getting rid of bad ticks. It's about latency and completeness (accuracy).

Interactive Brokers is widely known for filtering data. It's snapshot data, meaning that they aggregate info, and send you a 'snapshot' of what it looks like every 250ms or so. This means you are not getting all the ticks, not getting all the L2 depth changes, not even getting all the volume. If you trade daily bars or even minute data, then you won't care about this. But if you are using tick based data: ie tick, range, renko, volume or other exotic bar types, then you should not be using IB for data.

On top of that, IB has big restrictions on their historical data. For this reason, my conclusion is that IB filters data simply due to lack of resources on their end, and probably because overwhelming majority of their customers are using minute based or daily based data.

IB is a combination execution + feed provider.

Next we have the Rithmic's and Zen Fire's of the world. They send every tick. They are well known in the industry as unfiltered data. They also have a reputation for being ultra low latency (fast). This is very appealing to a large amount of more 'advanced' traders that want to use tick based charting.

However, they don't have their own historical servers (some platforms provide it for their users, it varies). They also do filter some L2 depth changes. In order to be as fast as possible, and get you that tick as fast as possible, the trade off is to not send some of the L2 changes. Even most of the 'advanced' traders today are not measuring L2 changes so pretty much no one is complaining about this or cares about it.

Rithmic/Zen Fire is an execution/risk system more than it is a data feed. The focus on the feed is execution, which is also why the focus on the feed is latency. They do this extremely well. I've heard rumors for many years that Zen Fire is branching out and adding other exchanges or breadth tools soon. Maybe they will? But for now, it's really designed as execution only, although a lot of people use it for data as well and are completely happy with it, provided they also use a platform (like NinjaTrader) that gives them historical data.

Next we have the IQFeed's and Kinetick's (oem) of the world. They send every tick. They also have an extensive historical database and ticker plants. They all major exchanges. They have something like 500 breadth tickers.

IQFeed's (and Kinetick) business is data. Not execution. Since their focus is on data, they send every L2 update. They send every tick. They send everything, and they also send it in the right sequence. It is the best quality feed currently available to a retail trader, in my opinion. They do give up some latency when compared to Rithmic/Zen Fire, as there is a price to be paid for completeness.

IQFeed also provides historical bid/ask backfill. If you don't know what that is, you don't need it. Only some platforms support it, such as Sierra Chart, Investor/RT, MarketDelta. NinjaTrader 8 is supposed to support it. If you need this then IQFeed is not only an excellent choice, but pretty much the only choice.

As for TradingTechnologies TT FIX, it is an execution feed. It's focus is on execution, although many do use it for data as well.

As for which feed is right for you, only you can answer that.

Mike

Mike, awesome summary, thanks,

Anything you could add about CQG? I guess the only thing I'd add that's worth taking into account with the data feeds is that with CQG you can store OCO orders on the server, which is not offered by Zenfire (nor Rithmic?).

I would imagine CQG would be comparable to Rithmic / close to IQ Feed ? Or would I be wrong ?

Cheers,

It seems the brokers that filter data, can manipulate it as well. Anyone have an opinion about this?

I decided to try out a demo of ninja trader w/ zen-fire (mirus) and compare similar ticks to Td Ameritrade and it def. looks different.

Feed that is filtered, seems to me is inaccurate and can affect traders buy sell positions.

kinetick provides historical bid/ask backfill? and real-time?

I need unfiltered data with bid/ask backfill real-time because I trade with CD. Is kinetick a good option?
I doubt between kinetick and iqfeed.

thanks

kinetick provides historical bid/ask backfill? and real-time?

I need unfiltered data with bid/ask backfill real-time because I trade with CD. Is kinetick a good option?
I doubt between kinetick and iqfeed.

thanks

Here is my opinion.

Filtering isn't about getting rid of bad ticks. It's about latency and completeness (accuracy).

Interactive Brokers is widely known for filtering data. It's snapshot data, meaning that they aggregate info, and send you a 'snapshot' of what it looks like every 250ms or so. This means you are not getting all the ticks, not getting all the L2 depth changes, not even getting all the volume. If you trade daily bars or even minute data, then you won't care about this. But if you are using tick based data: ie tick, range, renko, volume or other exotic bar types, then you should not be using IB for data.

On top of that, IB has big restrictions on their historical data. For this reason, my conclusion is that IB filters data simply due to lack of resources on their end, and probably because overwhelming majority of their customers are using minute based or daily based data.

IB is a combination execution + feed provider.

Next we have the Rithmic's and Zen Fire's of the world. They send every tick. They are well known in the industry as unfiltered data. They also have a reputation for being ultra low latency (fast). This is very appealing to a large amount of more 'advanced' traders that want to use tick based charting.

However, they don't have their own historical servers (some platforms provide it for their users, it varies). They also do filter some L2 depth changes. In order to be as fast as possible, and get you that tick as fast as possible, the trade off is to not send some of the L2 changes. Even most of the 'advanced' traders today are not measuring L2 changes so pretty much no one is complaining about this or cares about it.

Rithmic/Zen Fire is an execution/risk system more than it is a data feed. The focus on the feed is execution, which is also why the focus on the feed is latency. They do this extremely well. I've heard rumors for many years that Zen Fire is branching out and adding other exchanges or breadth tools soon. Maybe they will? But for now, it's really designed as execution only, although a lot of people use it for data as well and are completely happy with it, provided they also use a platform (like NinjaTrader) that gives them historical data.

Next we have the IQFeed's and Kinetick's (oem) of the world. They send every tick. They also have an extensive historical database and ticker plants. They all major exchanges. They have something like 500 breadth tickers.

IQFeed's (and Kinetick) business is data. Not execution. Since their focus is on data, they send every L2 update. They send every tick. They send everything, and they also send it in the right sequence. It is the best quality feed currently available to a retail trader, in my opinion. They do give up some latency when compared to Rithmic/Zen Fire, as there is a price to be paid for completeness.

IQFeed also provides historical bid/ask backfill. If you don't know what that is, you don't need it. Only some platforms support it, such as Sierra Chart, Investor/RT, MarketDelta. NinjaTrader 8 is supposed to support it. If you need this then IQFeed is not only an excellent choice, but pretty much the only choice.

As for TradingTechnologies TT FIX, it is an execution feed. It's focus is on execution, although many do use it for data as well.

As for which feed is right for you, only you can answer that.

Mike

Big Mike:

I trade with Range bars and I would think that even if IB data is filtered....it would still spit out the new range tick data so that it should be fairly accurate. Even if there are 10 missing ticks....one tick should still print the range value within that bar to make it accurate?

I was hoping I could use range bars with Ninjatrader and an IB feed.

Thanks.
Joules360

It's snapshot data. You are missing ticks, and also only receiving them in a "burst" fashion (every 250ms if I recall, or maybe it's every 100ms). You aren't missing just a few ticks, you are missing a bunch of them.

If you use exotic bars, like range bars or non-time based bars of any kind, you really should use a better data feed.

Mike

Rithmic, Zen Fire, CTS, IB, and BigTick (Mirus) all offer server side OCO I believe, but it will depend on your platform as to what is supported.

I have no comment on CQG except that I would never use AMP Futures to get it.

Mike

Hoping someone can help me.

I'm a client with IB and using ninja. What is the cheapest way I can get unfiltered data for ES, CL, and forex? I know kinetick is an option but they are fairly expensive for me right now considering im a new trader and am not working with much capital.

Create a Ninjatrader demo for futures and for FX and you have data for free and when the demo expire renew it.

I need to trade through NT. I use a live account through IB.

If you have multibroker licence first connect to ninjatrader demo and then connect to IB live account

NT demo gives unfiltered tick data? Better than IB?

Have you read post #3 of this very thread? Check it out.

Mike

Have you read post #3 of this very thread? Check it out.

Mike

So it appears that using either Rithmic or Zenfire with IB would be the best and cheapest option... Would you agree?

Now I just need to see if I can use either or those with IB.

Here is my opinion.

Filtering isn't about getting rid of bad ticks. It's about latency and completeness (accuracy).

Interactive Brokers is widely known for filtering data. It's snapshot data, meaning that they aggregate info, and send you a 'snapshot' of what it looks like every 250ms or so. This means you are not getting all the ticks, not getting all the L2 depth changes, not even getting all the volume. If you trade daily bars or even minute data, then you won't care about this. But if you are using tick based data: ie tick, range, renko, volume or other exotic bar types, then you should not be using IB for data.

On top of that, IB has big restrictions on their historical data. For this reason, my conclusion is that IB filters data simply due to lack of resources on their end, and probably because overwhelming majority of their customers are using minute based or daily based data.

IB is a combination execution + feed provider.

Next we have the Rithmic's and Zen Fire's of the world. They send every tick. They are well known in the industry as unfiltered data. They also have a reputation for being ultra low latency (fast). This is very appealing to a large amount of more 'advanced' traders that want to use tick based charting.

However, they don't have their own historical servers (some platforms provide it for their users, it varies). They also do filter some L2 depth changes. In order to be as fast as possible, and get you that tick as fast as possible, the trade off is to not send some of the L2 changes. Even most of the 'advanced' traders today are not measuring L2 changes so pretty much no one is complaining about this or cares about it.

Rithmic/Zen Fire is an execution/risk system more than it is a data feed. The focus on the feed is execution, which is also why the focus on the feed is latency. They do this extremely well. I've heard rumors for many years that Zen Fire is branching out and adding other exchanges or breadth tools soon. Maybe they will? But for now, it's really designed as execution only, although a lot of people use it for data as well and are completely happy with it, provided they also use a platform (like NinjaTrader) that gives them historical data.

Next we have the IQFeed's and Kinetick's (oem) of the world. They send every tick. They also have an extensive historical database and ticker plants. They all major exchanges. They have something like 500 breadth tickers.

IQFeed's (and Kinetick) business is data. Not execution. Since their focus is on data, they send every L2 update. They send every tick. They send everything, and they also send it in the right sequence. It is the best quality feed currently available to a retail trader, in my opinion. They do give up some latency when compared to Rithmic/Zen Fire, as there is a price to be paid for completeness.

IQFeed also provides historical bid/ask backfill. If you don't know what that is, you don't need it. Only some platforms support it, such as Sierra Chart, Investor/RT, MarketDelta. NinjaTrader 8 is supposed to support it. If you need this then IQFeed is not only an excellent choice, but pretty much the only choice.

As for TradingTechnologies TT FIX, it is an execution feed. It's focus is on execution, although many do use it for data as well.

As for which feed is right for you, only you can answer that.

Mike

Hi is someone able to provide any more recent info on this:
I am looking at the options offered by AMP Global. I am looking for an unfiltered feed and the options are
CQG , TTnet and Rithmic.
Latency of the feed would be less of a concern on my side. More of a concern would be all the ticks should be non aggregrated. Accurate Bid / Ask trade classification

Can anyone advise if these feeds are all unfiltered? Any views on the Bid Ask trade classification

The difference with the Amp CQG feed for NT7 users is that whenever NT7 makes a request for historical data, it comes from NT's own housed servers, from data they have recorded for that instrument. They used to have issues where their servers would go down every so often on the weekend or an overnight and then when you request historical data, it is not available to you: this problem appears to have been rectified by them and it has been quite some time where this has now not been an issue.

I am wanting to use it with IRT who routes CQG data through the marketdelta trader platform. Interestingly I got a reply from IRT which said I should go with Rithmic.
With IRT there is the option to backfill through I believe DTN. The only concern with Rithmic is the additional per side pricing fee which adds up over time.

When you have tested you CQG feed, have you eyeballed it ( which I tend to do) or have you dug into data on different feeds with excel or something similar. What sort of discrepancies have you noticed or picked up with regards any of this?

I am still having a problem with NT7 & NT8 requests to the Ninja servers. They have been missing some data and I have been getting an error on the front month contract since the last week of December. @NinjaTrader siad there is a problem , but they do not have an ETA on a fix. It has been a week already so I don't think they are in a hurry.

I use the CQG from Amp option [...] The feed is slightly different from the full (and much more expensive) CQG datafeed: it is made up of realtime incoming data and is provided by CQG directly - it is completely unfiltered, meaning you get all trade ticks and all the bid/ask info in that is available.

What difference in price do you see?

I am not exactly sure how to determine the precise price for an equivalent CQG set of instrument and data feeds; their page at Products | CQG, Inc. doesn't seem to specifically offer pricing solely for the datafeed, but may bundle it with their other products. In general terms and without doing specific research for you, the cost for datafeeds can be as little as $60/month (Kinetick?, but may not include markets like the Russell, Brent, FDAX, etc) and from past memory, Rithmic, which is likely upwards of $90). Bear in mind too, that a datafeed is one thing, but, entering and exiting trades is determine by whatever broker you choose, and thus their infrastructure to successfully route your trade to your instrument's exchange.

My point simply in my posts(s) was that there are many bundled solutions from brokers or other sources, that will provide for an incredibly high quality datafeed at minimal cost. And, my advice is to pick a reputable unfiltered data source, and not spend silly amounts of time exploring that avenue, when, it is the quality and success of each of our abilities to successfully trade in to and out of positions, that are the real cornerstone of making it or breaking it as a trader.

I am using Ninja with CQG and I trade with a 6500 volume chart, during the day I will reload the historial data maybe 5 times to make sure I am getting all the correct info. I copy my chart and look the next day at the trades and always the chart candles change. So if I get Kinetick can do I need 2 ninja lic to run one with CQG to trade and one to see Kinetick feed to see the correct data? If I get kinetick when I download the historical data with the chart look the same tomorrow as it will live?
Thanks
Happy Rick

Here is my opinion.

Filtering isn't about getting rid of bad ticks. It's about latency and completeness (accuracy).

Interactive Brokers is widely known for filtering data. It's snapshot data, meaning that they aggregate info, and send you a 'snapshot' of what it looks like every 250ms or so. This means you are not getting all the ticks, not getting all the L2 depth changes, not even getting all the volume. If you trade daily bars or even minute data, then you won't care about this. But if you are using tick based data: ie tick, range, renko, volume or other exotic bar types, then you should not be using IB for data.

On top of that, IB has big restrictions on their historical data. For this reason, my conclusion is that IB filters data simply due to lack of resources on their end, and probably because overwhelming majority of their customers are using minute based or daily based data.

IB is a combination execution + feed provider.

Next we have the Rithmic's and Zen Fire's of the world. They send every tick. They are well known in the industry as unfiltered data. They also have a reputation for being ultra low latency (fast). This is very appealing to a large amount of more 'advanced' traders that want to use tick based charting.

However, they don't have their own historical servers (some platforms provide it for their users, it varies). They also do filter some L2 depth changes. In order to be as fast as possible, and get you that tick as fast as possible, the trade off is to not send some of the L2 changes. Even most of the 'advanced' traders today are not measuring L2 changes so pretty much no one is complaining about this or cares about it.

Rithmic/Zen Fire is an execution/risk system more than it is a data feed. The focus on the feed is execution, which is also why the focus on the feed is latency. They do this extremely well. I've heard rumors for many years that Zen Fire is branching out and adding other exchanges or breadth tools soon. Maybe they will? But for now, it's really designed as execution only, although a lot of people use it for data as well and are completely happy with it, provided they also use a platform (like NinjaTrader) that gives them historical data.

Next we have the IQFeed's and Kinetick's (oem) of the world. They send every tick. They also have an extensive historical database and ticker plants. They all major exchanges. They have something like 500 breadth tickers.

IQFeed's (and Kinetick) business is data. Not execution. Since their focus is on data, they send every L2 update. They send every tick. They send everything, and they also send it in the right sequence. It is the best quality feed currently available to a retail trader, in my opinion. They do give up some latency when compared to Rithmic/Zen Fire, as there is a price to be paid for completeness.

IQFeed also provides historical bid/ask backfill. If you don't know what that is, you don't need it. Only some platforms support it, such as Sierra Chart, Investor/RT, MarketDelta. NinjaTrader 8 is supposed to support it. If you need this then IQFeed is not only an excellent choice, but pretty much the only choice.

As for TradingTechnologies TT FIX, it is an execution feed. It's focus is on execution, although many do use it for data as well.

As for which feed is right for you, only you can answer that.

Mike