Even if the values exist if we load datas from Zen-Fire they are wrong. I suspect it has something to do with how Ninja view a daily bar. If you hover your cursor on a daily bar, you'll see the time shown in the data box is 9:30 which is obviously wrong and consequently the real Open or High/Low created during the Globex session are obliterated so to speak. But i am not sure if it really explains the problem mentionned by Fat Tails.
- Daily bars will be time stamped based off the session template assigned to the data series. By default, we use an ETH template for the ES for example which would then time stamp the daily bar with the close of the session in your local time zone. Since you are in Quebec, this would be 3:15 PM. By chance are you using a custom session template?
- Zen-Fire historical daily bars are built from RTH session (not ETH) session and thus any the OHLC values can and likely will be different more often than not from a daily bar built on ETH session. This explains why the values are different. We do have on our list to provide ETH at some time in the future, no ETA on this right now.
The following user says Thank You to NinjaTrader for this post:
Here are some screen captures i did this morning before the opening regarding the OHLC disparities we get in Ninja when connecting to Kinetick and Zen-Fire.
First i connected to Kinetick to download Yesterday's OHLC for the three instruments i trade, namely the ES, TF and 6E. I exported the three tables from Ninja's database in Excel while i was still connected to Kinetick to form this trio:
Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).
Next, i disconnected from Kinetick and connected to Zen-Fire with no chart being opened. Then, I opened a new chart with daily bars loading only the last day to the present time before the session starts at 9:30EST.
I scrolled through the list of my three instruments to make sure that the ES, TF and 6E would send a request to the server to refresh Ninja's database. Finally, i opened the Historical Data Manager/Edit section to edit any disparities that were introduced by Zen-Fire.
The charts included show what needed to be edited. As you will note, since a couple of days the OHL do not seem to be changed by Zen-Fire. This was not the case prior to April 4, 2011. This is a recent change i have observed. The yellow background color shows what has been changed manually by me, myself and i.
If any of you knows how i could avoid this painfull procedure then i am all ear.
- Just to clarify, the data is different between the sources since Zen has RTH based daily bars where Kinetick is currently defaulted to ETH daily bars
- That being said, as you connect to Zen-Fire, I see that a few days of Zen-Fire daily data is loaded and overwriting data downloaded from Kinetick
- I have asked development to respond to me why this is the case. The person responsible for this part of our code is off the next few days attending the London Traders Expo so I will not be able to respond to this until mid next week. Thanks for your patience.
The following user says Thank You to NinjaTrader for this post:
Just to confirm that this is an important point. I can use Kinetick Daily Data together with intraday data from Interactive Brokers by double connecting: first connecting to Kinetick and second connecting to Interactive Brokers.
This prevents that (false) daily data is loaded from Interactive Brokers and I will only get Kinetick data written to the historical data base. This is an excellent solution that works for me.
I only need the discipline to always first-connect to Kinetick EOD.
Differences Interactive Brokers / Zenfire
(1) It seems that there is a problem with double connecting to Kinetick EOD / Zenfire (waiting for confirmation).
(2) The Zenfire interface overwrites the daily data from yesterday, when connected. The Interactive Brokers interface does not overwrite this data. (confirmed)
Thanks for your patience. This is expected behavior in an attempt to ensure that data is accurate. The reason this occurs is because there is no way to know if you actually have a complete 4/6 bar or not. When you get the data from Kinetick, there is no way to know if you got that daily bar in the middle of the day on 4/6 instead of after the complete day on 4/7. Since NT is not able to differentiate these scenarios, when you load up a chart with 4/7 it goes off and reloads 4/6 as well to ensure that the 4/6 data is actually complete. This is why the request ends up being of 4/6 and 4/7 and that is why your 4/6 data is getting replaced by the Zen connectionís data instead of being persisted as the Kinetick data.
If we did not do this, then there could be a situation where users could have incorrect daily data since their prior day daily bar was a middle of the day snapshot. I see the downside of this in the approach you have taken but I see a greater downside in not reloading the prior day's bar.
But then it would be a great enhancement for most of the data feeds, if you could simply opt out to receive daily data. This would allow to connect to Kinetick after the close, update daily data and just use it.
All the daily data loaded via Interactive Brokers, Zenfire, TT, etc. currently causes one or another pain. And the high quality data by Kinetick is difficult to use due to the interference with the other data sources.
The following user says Thank You to Fat Tails for this post:
I thought I would put this in the thread, and email I sent to NT Support requesting details of their MD (or President of whoever). Talk about nails down to the quick...aaaaaarrgh!
"I did all this; stopped 42 live strategies, set the old database as OLD.sdf, restarted, set all the strategies again; now I'm back to a problem I have had before, that is GBPJPY currencies showing up as incorrect in the >Trade Performance as it is not set to the correct pip amount. So, my reporting will be wrong for another day or another month or whatever it is. So, since running live strategies for 4 months I have not been able to pull off 1 single accurate report in that time. Either the DB needs repairing, or the currency reporting is out by so many pips so a loss shows up as $59000 instead of $590 or something. Instead I have to go back to my broker every week & ask him to print off reports - which he can only provide me in pdf format cos he can't get any actual data reports to give me. So I have to highlight the pdf material & paste it page by page into Word, and then transfer that into Excel to get an accurate report.
I manage a $15m currency fund that is fully AFSL licensed, that is completely automated in MT4. Every week I save off reports & email them to my accountant who provides me a complete audit in half an hour, which I then send to unit holders. In 2 years this thing has never missed a beat, not one cent difference between my brokers account & what MT4 tells me. Just to give you an idea, I run many of my strategies on minute charts too, so there are thousands of trades every single month.
Contrast that with the $30k account I run on NT7 (same broker) where I test strategies in NT7 & set them live but never really understand how they perform. So I now am employing a full-time developer to get my strategies that work to be re-coded from NT7 to MT4. If only I could get NT7 to work in a simple and timely fashion, & do the things it is supposed to do I would use it. For my fund. Full time.
But it's still not right.
Do you have an address of your President, or Managing Director or whatever you call them over there so I can write him a letter please? I feel he should know about these sorts of things, unless he is genuinely not interested."