First off, Fat Tails thank you very much for such detailed and helpful responses. A combination of your information, and another reliable source (10 year professional futures trader) has finally put this to rest for me. Your advice was completely accurate. Regarding self-fulfilling, any professional trader/institution pushing size is watching a back-adjusted continuous contract which makes complete sense now that I understand the process. The back-adjusted (equalized) contract must be used or S/R, gaps and composite volume profiles will be at incorrect levels. Yes the past data shifts and that is the somewhat tricky part to understand since the numerical value of the level may change (especially as more rollover periods pass). Although the numerical level changes the data shift is exactly the same and the S/R, gaps and composite volume profile levels will then be accurate.
Looking at a back-adjusted RTH only, Daily chart is a beautiful thing. Long-term unfilled gaps are at correct levels, major Fib levels are suddenly spot on (a drastic difference in comparison to a Daily, non-adjusted ETH chart).
Couple of questions:
- looking back through the instrument manager tree the default merge date NT is using is rollover and most often coincides with the session when net volume shifts to the new contract. Isnt this exactly when I want the merge to occur instead of expiration? Remember, main concern is accurate S/R, gaps and composite volume profiles going forward.
- how exactly do I check if my Daily database contains the settlement dates and not the closes?
- since Zenfire now has Daily/historical data item (c) is irrelevant?
The following user says Thank You to neko333 for this post:
I am quite happy you had this confirmed by another source. Learning is a step-by-step process. Usually two steps forward and one step backward, unlearning. This is particularly true for trading.
I have coded several fib and fib confluence indicator, and they only work correctly with backajdusted futures. This suggests that the big guys use them, but I have not send around a questionary so far.
Checking your daily data base
The only primary source, which is always correct, is the exchange. let us take ES for simplicity. The following link shows the correct values published for last Friday.
The settlement for ES 09-10 is 1070.25, the close is 1071.00 . For ES the close is a good proxy for the settlement, but this not the case for all contracts. It really depends on the settlement period. For CME the settlement period can be found here:
ES: settlement period is 15:14:30 to 15:15:00 CT, close is 15:15:00 (just a matter of seconds)
ZB: settlement period is 13:59:00 CT to 14:00:00 CT, close is 16:00 CT (2 hours later)
CL: settlement period is 14:28:00 ET to 14:30:00 ET, close is 17:15:00 ET (2 hours 45 min later)
TF: settlement period is 16:14:00 ET to 16:15:00 ET, close is 18:00 ET (1 hours 45 min later)
I do not use Zenfire, but to my knowledge NinjaTrader will not display correct daily data from Zenfire. What you can do is
- first connect to Kinetick-EOD, which has the correct daily data with the settlement prices and
- second connect to Zenfire
NinjaTrader will then load daily data from Kinetick and intraday data from Zenfire.
Correct Rollover Dates
For financial futures and metals (GC, SI, HG) this is no problem, as the rollover dates are known. It really gets tricky for energies and agricultural. Because of the physical delivery the price of the old contract may wildly swing up and down, if there are supply shortages or oversupply. Therefore rollover should be done latest at volume crossover. I use volume crossover as the rollover date (volume new front month contract > volume old contract), NinjaTrader calculates the offset from the difference of the settlement prices (if that is the close in your daily data base) of the session prior to rollover.
I am neither trading CL nor agrculturals, but if I did, I would make sure that I talked to some professionals to find out the correct rollover schedule.
Not sure why you say NinjaTrader does not display correct daily data from Zenfire. You can get several years of Daily back-adjusted data.
Kineticks continuous contract is not adjusted and can`t be adjusted at this time. Also, Kineticks Daily bars are ETH instead of much more useful RTH Daily bars.
Here is a post from NT forums.
Yes this is a change in NinjaTrader 7. NinjaTrader automatically merges back contracts to the current month when data is requested that would exceed the amount of data available in the current contract expiry.
Let me know if I can be of further assistance.
Brett, NinjaTrader Customer Service
How does the attached ES continous back-adjusted RTH Daily chart compare to yours? Should be spot on. Might be a little harder to compare if you only have ETH Daily bars.
Kinetick: Continuous contracts are non-adjusted. However you can download about 500 days of data for single contracts. Set NinjaTrader to MergePolicy MergeBackAdjusted and you will get MergeBackadjusted contracts and can even modify rollover dates. That is what I am doing. Is all ETH data as reported by the exchange.
Only applies with MergePolicy set to MergeBackAdjusted, when loading single contracts.
I only use ETH daily bars. However, if you want to use RTH daily bars, you can build them yourself. But this is a bit tedious. Anyhow, here is the method:
- set instrument settings session template to RTH
- export tick or minute data for all single contracts via Historical Data Manager
- reimport that same data as daily data via the Historical Data Manager
You now have a historical data base with daily RTH bars.
A couple of weeks later. I`ve went through all of your information in both threads several times. Lets see if got this right. A couple of questions to follow:
My Daily historical data in NT-7 is based on the pit (14:30) close. So when I pull up a Daily back-adjusted chart I have RTH only candles which are by default adjusted according to NT`s rollover date. Since I have not manually changed any of the offsets/dates NT is using the difference between the 14:30 closes on the session prior to rollover. Pretty close to an accurate offset but as you explained the precise offset is the difference between the prior sessions settlement price which is actually the value weighted average of the final two minutes until the 14:30 RTH close. So if there is a price spike within the last few minutes the settlement may differ from the 14:30 close by several ticks. Nonetheless, manually going through a few of them by comparing my CL 14:30 RTH close with CMEs settlement report they are usually pretty close. As you mentioned, I also have no use for a perpetual contract and never hold CL positions overnight. I am only interested in rolling the contracts as precisely as possible for S/R, gaps and composite volume profiling.
So it seems all I need to do at this point is `fine tune` NTs back adjustment by manually putting in the offsets according to settlement differences instead of 14:30 closes and also rolling a day earlier if there is a clear volume shift before the normal rollover date. Looking back through the recent 5 rollovers there was only one month where the volume shifted before rollover day (on the 5/19 rollover volume shifted on the session prior). Please advise if I finally got all this right.
1. Since the CME only provides the recent 5 days settlement prices (unless you pay for Datamine) is there a resource where I can find accurate historical settlements for the CL?
2. If there is a volume shift to the new contract on the day before rollover I would be using settlement prices on the day prior to the volume shift which would actually be two days prior the rollover day. Is this correct?
Guess I should be grateful that the CL rolls so often, sure forced me to learn a lot about the process.
This is not my understanding. The daily chart (Kinetick) is built from the high and low of the full (ETH) session and the 14:30 settlement price. NT7 uses daily data for calculating the offset, so it should be correct.
Due to the physical delivery constraints in Cushing, the old contract is often volatile prior to the last trading date, so for calculating offsets, it is not a bad idea to roll as early as possible. Some data providers as Pinnacle roll even several days earlier. It is a shame that NYMEX has not expanded its delivery options to large sea ports.
Kinetick-EOD data, LOL. The daily close is the settlement price of CL.
I think there is a difference between finding the rollover-date for trading and the rollover-date for data adjustment.
Rollover day for trading
As I said CL ís physically deliverable. During the last days the old contract can show abnormal volatility. There have been magnificent short squeezes (you need to deliver in that stupid depot in Oklahoma and there is no stuff because the pipeline is shut down) or an increasing contango (depot is full and you don't fins a buyer).
If you are not a trader who has access to the physical market, you are likely better of not to trade the old contract prior to expiry and roll as early as possible. This means you could even roll prior to volume cross-over, if liquidity is satisfactory. But in any case you want to move to the new contract with volume crossover.
Rollover day for back-adjusting
If Cushing has a problem, the offset can be distored during the last days. You don't want to base your offset on a short squeeze. So again earlier is better. You can also do a study and write down the different offsets for the week prior to rollover (as per volume crossover) and compare them. If these values are pretty stable, you should have a good offset. If they are jumping back and forth when approaching expiry, it is no more useful. I have not done this, but you could, if you are trading CL.
The following user says Thank You to Fat Tails for this post: