Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
The issue as I understand it, is NT was getting bogged down/slow. This is an NT problem or code running under NT, not an IQFeed problem.
When I was receiving 200 L2 streams, I was using 24 cores to do so with multiple threads to IQFeed servers. Mind you, a lot of the activity on my server was the fact it was recording the data. Still, it wouldn't be possible to efficiently receive that many symbols without using multiple threads with IQFeed.
That shouldn't be the case, it's just barely several Mbps of data and it's just a TCP client handle read + tokenizer + cast in your processing step, a single-threaded implementation should be more than sufficient. I've done 500 concurrently on a single Nehalem core.
Don't believe it, I just satisfied myself that the Market Replay environment is not at all reliable as regards timestamps once you go multi-instrument. It holds up ok so long as all the series are for the same instrument, (apart from the OMD bip duplication issue that @NJAMC pointed out) but it falls apart as soon as any other instrument is added. I have checked this with Ninja's own data downloaded for the US market and with my own eSignal self-recorded data from Eurex and the Deutsche Bourse.
e.g. this is the sort of simple output that shows event order and timestamp order mismatches clearly:
In this case bip 0-2 are ES, bip 3 is DJIA and bip 4 is SPX, all simple TICK series, not even the time based series issue to mess it up and all Historical data issues eliminated. Showing a maximum 19 sec hiccup in this recorded US intraday session. Obviously we don't worry about single second reports as that is NT's limited replay resolution but make no assumptions appears to be in order.
Just to note that the same data shows no problems whatsoever so long as the test is kept intra-series, within any of the series. Points pretty heavily at MR imho. I am revising my future test plans and architecture as a result.
edit: Reposted in more relevant thread with further detail:
Reposting in here as a more relevant thread, I seem to have satisfied myself that the Market Replay environment is not at all reliable as regards timestamps once you go multi-instrument. It holds up ok so long as all the series are for the same instrument, …
