Unfiltered Data - Reviews of Brokers and Data Feeds | futures io social day trading
futures io futures trading


Unfiltered Data
Updated: Views / Replies:25,927 / 41
Created: by RM99 Attachments:4

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 4  
 
Thread Tools Search this Thread
 

Unfiltered Data

  #1 (permalink)
Fortitudo et Honor
Austin, TX
 
Futures Experience: Advanced
Platform: TradeStation
Favorite Futures: Futures
 
Posts: 882 since Mar 2011
Thanks: 128 given, 702 received

Unfiltered Data

I keep seeing these advertisements for unfiltered data.....

Isn't the whole point of filtering the data to ensure that a rogue/erroneous data point won't ruin your day? If you have a stop in place and a bad piece of data comes in, that exceeds your stop, most platforms treat stop orders as "market if touched or exceeded" orders.

What's the whole issue with having "unfiltered" data?

"A dumb man never learns. A smart man learns from his own failure and success. But a wise man learns from the failure and success of others."
Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,237 since Jun 2009
Thanks: 29,350 given, 83,150 received



RM99 View Post
I keep seeing these advertisements for unfiltered data.....

Isn't the whole point of filtering the data to ensure that a rogue/erroneous data point won't ruin your day? If you have a stop in place and a bad piece of data comes in, that exceeds your stop, most platforms treat stop orders as "market if touched or exceeded" orders.

What's the whole issue with having "unfiltered" data?

Here is my opinion.

Filtering isn't about getting rid of bad ticks. It's about latency and completeness (accuracy).

Interactive Brokers is widely known for filtering data. It's snapshot data, meaning that they aggregate info, and send you a 'snapshot' of what it looks like every 250ms or so. This means you are not getting all the ticks, not getting all the L2 depth changes, not even getting all the volume. If you trade daily bars or even minute data, then you won't care about this. But if you are using tick based data: ie tick, range, renko, volume or other exotic bar types, then you should not be using IB for data.

On top of that, IB has big restrictions on their historical data. For this reason, my conclusion is that IB filters data simply due to lack of resources on their end, and probably because overwhelming majority of their customers are using minute based or daily based data.

IB is a combination execution + feed provider.

Next we have the Rithmic's and Zen Fire's of the world. They send every tick. They are well known in the industry as unfiltered data. They also have a reputation for being ultra low latency (fast). This is very appealing to a large amount of more 'advanced' traders that want to use tick based charting.

However, they don't have their own historical servers (some platforms provide it for their users, it varies). They also do filter some L2 depth changes. In order to be as fast as possible, and get you that tick as fast as possible, the trade off is to not send some of the L2 changes. Even most of the 'advanced' traders today are not measuring L2 changes so pretty much no one is complaining about this or cares about it.

Rithmic/Zen Fire is an execution/risk system more than it is a data feed. The focus on the feed is execution, which is also why the focus on the feed is latency. They do this extremely well. I've heard rumors for many years that Zen Fire is branching out and adding other exchanges or breadth tools soon. Maybe they will? But for now, it's really designed as execution only, although a lot of people use it for data as well and are completely happy with it, provided they also use a platform (like NinjaTrader) that gives them historical data.

Next we have the IQFeed's and Kinetick's (oem) of the world. They send every tick. They also have an extensive historical database and ticker plants. They all major exchanges. They have something like 500 breadth tickers.

IQFeed's (and Kinetick) business is data. Not execution. Since their focus is on data, they send every L2 update. They send every tick. They send everything, and they also send it in the right sequence. It is the best quality feed currently available to a retail trader, in my opinion. They do give up some latency when compared to Rithmic/Zen Fire, as there is a price to be paid for completeness.

IQFeed also provides historical bid/ask backfill. If you don't know what that is, you don't need it. Only some platforms support it, such as Sierra Chart, Investor/RT, MarketDelta. NinjaTrader 8 is supposed to support it. If you need this then IQFeed is not only an excellent choice, but pretty much the only choice.

As for TradingTechnologies TT FIX, it is an execution feed. It's focus is on execution, although many do use it for data as well.

As for which feed is right for you, only you can answer that.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.


Last edited by Big Mike; August 24th, 2012 at 01:46 PM.
Reply With Quote
The following 84 users say Thank You to Big Mike for this post:
 
  #4 (permalink)
Fortitudo et Honor
Austin, TX
 
Futures Experience: Advanced
Platform: TradeStation
Favorite Futures: Futures
 
Posts: 882 since Mar 2011
Thanks: 128 given, 702 received

Thanks Mike, so as long as I'm not scalping or involved in the weeds, then filtered data (like Tradestation) should be sufficient for my purposes?

My trading strategies aren't made or broken based on the accuracy of the next few hundred miliseconds or my order in the book at the next ask/bid.

My charts and indys are populated historically and use end of bar calc's, so I should be good right?

I'm not shopping, I was just curious and the whole data quality/filtered/unfiltered thing is new to me.

I would like to take advantage of data timestamped down to the second rather than the minute however, which would allow me to explore some multi-timeframe ventures on advanced charts.

So all that I've read about rogue/errant data points is just hype?

From what little research i did, I saw horror stories about people using unfiltered data and bad ticks coming in and screwing with their DOMs/Matrixes, etc.

"A dumb man never learns. A smart man learns from his own failure and success. But a wise man learns from the failure and success of others."
Reply With Quote
 
  #5 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,237 since Jun 2009
Thanks: 29,350 given, 83,150 received


RM99 View Post
Thanks Mike, so as long as I'm not scalping or involved in the weeds, then filtered data (like Tradestation) should be sufficient for my purposes?

My trading strategies aren't made or broken based on the accuracy of the next few hundred miliseconds or my order in the book at the next ask/bid.

My charts and indys are populated historically and use end of bar calc's, so I should be good right?

I'm not shopping, I was just curious and the whole data quality/filtered/unfiltered thing is new to me.

I would like to take advantage of data timestamped down to the second rather than the minute however, which would allow me to explore some multi-timeframe ventures on advanced charts.

So all that I've read about rogue/errant data points is just hype?

From what little research i did, I saw horror stories about people using unfiltered data and bad ticks coming in and screwing with their DOMs/Matrixes, etc.

If you are using TradeStation, then you can choose any data provider you want, so long as it is TradeStation. You can also choose any broker you want, so long as it is TradeStation. TradeStation is in all-in-one solution, and as such isn't really a leader in any field, in my opinion.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
The following 4 users say Thank You to Big Mike for this post:
 
  #6 (permalink)
Fortitudo et Honor
Austin, TX
 
Futures Experience: Advanced
Platform: TradeStation
Favorite Futures: Futures
 
Posts: 882 since Mar 2011
Thanks: 128 given, 702 received


Big Mike View Post
If you are using TradeStation, then you can choose any data provider you want, so long as it is TradeStation. You can also choose any broker you want, so long as it is TradeStation. TradeStation is in all-in-one solution, and as such isn't really a leader in any field, in my opinion.

Mike

I feel the pain of all that, what I'm asking is that if I'm not scalping or dependent upon hyper quick calculations and execution if it's going to make an appreciable difference for me?

I guess for everyone's benefits, which "types" of traders benefit most from having unfiltered data and which types is it really a non-issue?

"A dumb man never learns. A smart man learns from his own failure and success. But a wise man learns from the failure and success of others."
Reply With Quote
 
  #7 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,237 since Jun 2009
Thanks: 29,350 given, 83,150 received


RM99 View Post
I feel the pain of all that, what I'm asking is that if I'm not scalping or dependent upon hyper quick calculations and execution if it's going to make an appreciable difference for me?

I guess for everyone's benefits, which "types" of traders benefit most from having unfiltered data and which types is it really a non-issue?

Having never used TradeStation, I can't say. But my understanding there is an option in the menu somewhere to disable filtering and send you all the ticks.

Asking about scalping is the wrong question. I mean, I guess it is possible to scalp on a minute chart. The better question is tick data, or not tick data. I cannot comment on TradeStation's data feed.

As for which traders benefit, I believe I covered everything in my original reply. ?

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
The following 2 users say Thank You to Big Mike for this post:
 
  #8 (permalink)
Trading Apprentice
Paris, France
 
Futures Experience: Advanced
Platform: Tradestation, MultiCharts, TWS, ButtonTrader
Broker/Data: IB
Favorite Futures: ES, EUR/USD, Oil, Gold, Stocks
 
NoBias's Avatar
 
Posts: 16 since Feb 2011
Thanks: 1 given, 13 received


Big Mike View Post
Having never used TradeStation, I can't say. But my understanding there is an option in the menu somewhere to disable filtering and send you all the ticks.

Asking about scalping is the wrong question. I mean, I guess it is possible to scalp on a minute chart. The better question is tick data, or not tick data. I cannot comment on TradeStation's data feed.

As for which traders benefit, I believe I covered everything in my original reply. ?

Mike

TradeStation is ok for time based charts [Longer term]

It doesn't build volume charts correctly [but close enough for govt. work]
{edit: didn't a few years back when we tested and compared ((7.x)). I no longer use volume or tick charts, so can not speak for current build} - but TS is TS...

If one requires higher quality data, then your recommendation for IQF is correct.

Regarding option to disable filtering and send you all the ticks, I never could find a specific filter, but believe in File/Preferences / Tradestation Nework preference / Streaming Data Optimization / "none" is as close as it gets.

SS's attached:

Attached Thumbnails
Unfiltered Data-ts-network-preference-high.png   Unfiltered Data-ts-network-preference-standard.png   Unfiltered Data-ts-network-preference-none.png  

Last edited by NoBias; August 29th, 2012 at 02:50 PM. Reason: Disclaimer ref: volume charts
Reply With Quote
The following 8 users say Thank You to NoBias for this post:
 
  #9 (permalink)
waverider
Sydney
 
Futures Experience: Advanced
Platform: NinjaTrader; TWS
Broker/Data: Kinetick, IB
Favorite Futures: US Stocks, FX, Futures
 
Posts: 45 since Dec 2009
Thanks: 28 given, 6 received

Zenfire, Rithmic, CQG

Mike, awesome summary, thanks,

Anything you could add about CQG? I guess the only thing I'd add that's worth taking into account with the data feeds is that with CQG you can store OCO orders on the server, which is not offered by Zenfire (nor Rithmic?).

I would imagine CQG would be comparable to Rithmic / close to IQ Feed ? Or would I be wrong ?

Cheers,



Big Mike View Post
Here is my opinion.

Filtering isn't about getting rid of bad ticks. It's about latency and completeness (accuracy).

Interactive Brokers is widely known for filtering data. It's snapshot data, meaning that they aggregate info, and send you a 'snapshot' of what it looks like every 250ms or so. This means you are not getting all the ticks, not getting all the L2 depth changes, not even getting all the volume. If you trade daily bars or even minute data, then you won't care about this. But if you are using tick based data: ie tick, range, renko, volume or other exotic bar types, then you should not be using IB for data.

On top of that, IB has big restrictions on their historical data. For this reason, my conclusion is that IB filters data simply due to lack of resources on their end, and probably because overwhelming majority of their customers are using minute based or daily based data.

IB is a combination execution + feed provider.

Next we have the Rithmic's and Zen Fire's of the world. They send every tick. They are well known in the industry as unfiltered data. They also have a reputation for being ultra low latency (fast). This is very appealing to a large amount of more 'advanced' traders that want to use tick based charting.

However, they don't have their own historical servers (some platforms provide it for their users, it varies). They also do filter some L2 depth changes. In order to be as fast as possible, and get you that tick as fast as possible, the trade off is to not send some of the L2 changes. Even most of the 'advanced' traders today are not measuring L2 changes so pretty much no one is complaining about this or cares about it.

Rithmic/Zen Fire is an execution/risk system more than it is a data feed. The focus on the feed is execution, which is also why the focus on the feed is latency. They do this extremely well. I've heard rumors for many years that Zen Fire is branching out and adding other exchanges or breadth tools soon. Maybe they will? But for now, it's really designed as execution only, although a lot of people use it for data as well and are completely happy with it, provided they also use a platform (like NinjaTrader) that gives them historical data.

Next we have the IQFeed's and Kinetick's (oem) of the world. They send every tick. They also have an extensive historical database and ticker plants. They all major exchanges. They have something like 500 breadth tickers.

IQFeed's (and Kinetick) business is data. Not execution. Since their focus is on data, they send every L2 update. They send every tick. They send everything, and they also send it in the right sequence. It is the best quality feed currently available to a retail trader, in my opinion. They do give up some latency when compared to Rithmic/Zen Fire, as there is a price to be paid for completeness.

IQFeed also provides historical bid/ask backfill. If you don't know what that is, you don't need it. Only some platforms support it, such as Sierra Chart, Investor/RT, MarketDelta. NinjaTrader 8 is supposed to support it. If you need this then IQFeed is not only an excellent choice, but pretty much the only choice.

As for TradingTechnologies TT FIX, it is an execution feed. It's focus is on execution, although many do use it for data as well.

As for which feed is right for you, only you can answer that.

Mike


Reply With Quote
 
  #10 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,237 since Jun 2009
Thanks: 29,350 given, 83,150 received



waverider View Post
Mike, awesome summary, thanks,

Anything you could add about CQG? I guess the only thing I'd add that's worth taking into account with the data feeds is that with CQG you can store OCO orders on the server, which is not offered by Zenfire (nor Rithmic?).

I would imagine CQG would be comparable to Rithmic / close to IQ Feed ? Or would I be wrong ?

Cheers,

Rithmic, Zen Fire, CTS, IB, and BigTick (Mirus) all offer server side OCO I believe, but it will depend on your platform as to what is supported.

I have no comment on CQG except that I would never use AMP Futures to get it.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
The following 3 users say Thank You to Big Mike for this post:

Reply



futures io > > > > Unfiltered Data

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

Dec 7

Linda Bradford Raschke: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Unfiltered data feeds and HFT, detecting algos with orderflow? cashinvestor Reviews of Brokers and Data Feeds 17 May 3rd, 2014 01:11 PM
Chart Bar OHLC data DOESNT match with the value in data box Koepisch NinjaTrader 4 May 12th, 2012 03:52 AM
Data Manager fails to receive data from eSignal rsbtrader TradeStation 1 April 17th, 2012 03:03 PM
IQfeed vs IB data as data provider for Multicharts arjfca Reviews of Brokers and Data Feeds 21 January 16th, 2012 11:39 AM
Best Data Provider to compliment Zen-Fire for Mkt Internals & EOD Data? waverider Reviews of Brokers and Data Feeds 17 October 14th, 2010 11:33 AM


All times are GMT -4. The time now is 08:30 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-11-19 in 0.18 seconds with 20 queries on phoenix via your IP 54.161.3.96