Using OnMarketData() on Historical data with a recording engine - NinjaTrader Programming | futures io social day trading
futures io futures trading


Using OnMarketData() on Historical data with a recording engine
Updated: Views / Replies:28,299 / 115
Created: by gomi Attachments:23

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors – all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you don’t need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 23  
 
Thread Tools Search this Thread
 

Using OnMarketData() on Historical data with a recording engine

  #71 (permalink)
Elite Member
Portland Oregon, United States
 
Futures Experience: Beginner
Platform: Ninjatrader®
Broker/Data: CQG, Kinetick
Favorite Futures: Gameplay Klownbine® Trading of Globex
 
Zondor's Avatar
 
Posts: 1,327 since Jul 2009
Thanks: 1,246 given, 2,635 received

Musings regarding Gom Indicators on volume based charts

I think that the optimal setup for GomRecorderIndicators used to write data, is to put them on tick interval charts. There is no need to use writing recorders on any other type of chart than a tick chart. All of my recorders that have write data set to true are on dedicated tick interval recorder charts in one workspace that is not used for anything else, and that runs in the background.

I found that when my recording indicator and data consuming indicator are both on tick charts, I am now getting perfect matches on almost all bars between (buyvolume + sellvolume) and Volume[0]. To get there, I needed to implement RJay's FirstTickOfBar fix for NT7, and also to count the BetwenBidAsk and Unknown ticks, using Gomi's "direction" to determine whether they are buys or sells.

The file created by the GomRecorder needs to be able to be used by data consuming indicators that could be on any type of chart. Charts with intervals of ticks and seconds should work fine. On minute based charts there are some problems due to the lower precision of the time stamps. However you can get around this by using seconds intervals, such as 300 for a 5 minute chart. This forces Ninnie to use the one second time stamps. Gomi pointed this out a long time ago.

With volume interval charts there are problems when a spillover trade with volume of more than one causes the volume counted to go over the nominal interval volume of the chart. Example: We have a 100 volume chart and are up to 98 contracts traded. The next tick has a volume of 25. If all of those are included in the current bar of the volume indicator, it will have a total volume of 123, but Volume [0] will still be 100, as it is for all bars.

The child indicators such as CD, GomDeltaIndicator, etc are where the adjustments should be made on volume based charts, when one trade spills over to the next bar. I don't see how the GomRecorder itself can compensate for this, since it has no way of knowing what kind of chart it will be called upon to supply data to.

When I get around to it I will see if I can code changes to the BuySellVolume indicator and to GomDeltaIndicator to make them work correctly on constant volume charts. I am interested in the GomDeltaIndicator because it is the data source for the GomVolumeLadder, which is often used on volume interval charts.

Regarding range, Renko, etc charts, I don't even want to think about them right now.

Reply With Quote
The following user says Thank You to Zondor for this post:
 
  #72 (permalink)
Elite Member
CA
 
Futures Experience: Master
Platform: Marketdelta and Ninja
Broker/Data: Velocity
Favorite Futures: NQ
 
Posts: 670 since Apr 2010
Thanks: 64 given, 521 received

How many days do you load on the tick charts.. Do you set it to ETH or default? Does it have to be on a 1 tick chart?


Last edited by Michael.H; October 26th, 2010 at 02:42 AM.
Reply With Quote
 
  #73 (permalink)
Elite Member
San Francisco Bay Area
 
Futures Experience: Intermediate
Platform: NT,TOS,IB
Favorite Futures: ES,CL,TF
 
Posts: 278 since Jun 2010
Thanks: 154 given, 267 received


Zondor:

I am surprised that type of chart you record on should make a difference. The recorder is based on Market Data events and those are kind of independent of the bar logic in NT. Do you have any specific observations which made you arrive at that conclusion or is it based more on empirical observations about what works and what does not?

I made a small change on the volume ladder to add the last tick to the new bar. I do not mess with the old bar since it does not matter that much.

Reply With Quote
 
  #74 (permalink)
Elite Member
Portland Oregon, United States
 
Futures Experience: Beginner
Platform: Ninjatrader®
Broker/Data: CQG, Kinetick
Favorite Futures: Gameplay Klownbine® Trading of Globex
 
Zondor's Avatar
 
Posts: 1,327 since Jul 2009
Thanks: 1,246 given, 2,635 received

settings for active recording charte


Quoting 
How many days do you load on the tick charts.. Do you set it to ETH or default? Does it have to be on a 1 tick chart?

Assuming that you are asking about the charts that I use for recording: They have an interval of about 100 ticks and only go back about 50 bars, so that they will load fast. To make sure I don't miss anything I usually use the default 24/7 session template.

Before I shut down Ninnie I close all the workspaces except the Recorder workspace. Then it's the only one that loads automagically® when Ninnie restarts.

Then I start up the other workspaces one by one and leave the Recorder one running in the background. The only reason I look at it once in a while is to check that everything is Recording OK.

Reply With Quote
 
  #75 (permalink)
Elite Member
Paris
 
Futures Experience: None
Platform: NinjaTrader
 
Posts: 1,173 since Oct 2009
Thanks: 227 given, 4,089 received

Hi guys,

A good news : I've been working to try to improve the tick engine of the recorder, for now I've been able to get 100% perfect match of Vol[0] as compared to gomi volume, for second charts, tick charts, range charts and volume charts (by splitting volume). I also improved the non-splitting volume chart by sending the ticks a bit differently.

A few charts : first indie is Vol[0], second indie is Volume read from Gom tick engine, and third indie is the diff : so should be 0.

Of course this is work in progress...

Attached Thumbnails
Using OnMarketData() on Historical data with a recording engine-secondchart.png   Using OnMarketData() on Historical data with a recording engine-tickchart.png   Using OnMarketData() on Historical data with a recording engine-rangechart.png   Using OnMarketData() on Historical data with a recording engine-volumesplitting.png   Using OnMarketData() on Historical data with a recording engine-oldticksending.png   Using OnMarketData() on Historical data with a recording engine-newticksending.png  
Reply With Quote
The following 8 users say Thank You to gomi for this post:
 
  #76 (permalink)
Elite Member
San Francisco Bay Area
 
Futures Experience: Intermediate
Platform: NT,TOS,IB
Favorite Futures: ES,CL,TF
 
Posts: 278 since Jun 2010
Thanks: 154 given, 267 received


gomi View Post
Hi guys,

A good news : I've been working to try to improve the tick engine of the recorder, for now I've been able to get 100% perfect match of Vol[0] as compared to gomi volume, for second charts, tick charts, range charts and volume charts (by splitting volume). I also improved the non-splitting volume chart by sending the ticks a bit differently.

A few charts : first indie is Vol[0], second indie is Volume read from Gom tick engine, and third indie is the diff : so should be 0.

Of course this is work in progress...

Gomi:

I was also thinking of making the gom interface sensitive to the type of bar being used to work around these last tick issues; it is great you are already attempting it.

One of the biggest challenges with the gom indicators is keeping the tick data base current. I was wondering if you would consider a mode where the recorder uses one file per day of data, instead of one file for the entire history of the instrument.

It would be much easier to maintain a clean tick database; you can just download and save the data from your vendor after midnight, while recording the current data live. This will also allow you to update historical data without shutting off the recorder. The biggest challenge I face is keeping the database current, without shutting off the live recorder.

Ideally the on disk file-structure for that will involve instrument specific sub-folders within the GOMFOLDER directory.
I can pitch in some coding time if that helps, but I do not understand all the nuances of the GomRecorder to do it independently.

Reply With Quote
The following user says Thank You to aviat72 for this post:
 
  #77 (permalink)
Elite Member
Portland Oregon, United States
 
Futures Experience: Beginner
Platform: Ninjatrader®
Broker/Data: CQG, Kinetick
Favorite Futures: Gameplay Klownbine® Trading of Globex
 
Zondor's Avatar
 
Posts: 1,327 since Jul 2009
Thanks: 1,246 given, 2,635 received

Splitting up the Gom Files

That's great news, Gomi.

I also like the idea of one file per day for Gom files, would make it easier to manage the data, fix gaps, etc.

Reply With Quote
The following 2 users say Thank You to Zondor for this post:
 
  #78 (permalink)
Elite Member
San Francisco Bay Area
 
Futures Experience: Intermediate
Platform: NT,TOS,IB
Favorite Futures: ES,CL,TF
 
Posts: 278 since Jun 2010
Thanks: 154 given, 267 received

gomi:

I think we still have a problem. I have a CL template with 3 sessions (6:00PM to 9:AM, 9:00AM to 2:30PM, 2:30 PM to 5:15PM EST). Today (Friday) the last day of the week, the recorder stopped sending ticks at 9:00AM when the session rolled over the pit session.

Earlier on, I used to face problems on Monday morning ES (again a 3 session template). I think that is fixed, but the Friday problem remains.

Reply With Quote
 
  #79 (permalink)
Market Wizard
Columbus, OH
 
Futures Experience: None
Platform: NT 8, TOS
Favorite Futures: ES
 
Silvester17's Avatar
 
Posts: 3,332 since Aug 2009
Thanks: 4,579 given, 10,313 received


aviat72 View Post
gomi:

I think we still have a problem. I have a CL template with 3 sessions (6:00PM to 9:AM, 9:00AM to 2:30PM, 2:30 PM to 5:15PM EST). Today (Friday) the last day of the week, the recorder stopped sending ticks at 9:00AM when the session rolled over the pit session.

Earlier on, I used to face problems on Monday morning ES (again a 3 session template). I think that is fixed, but the Friday problem remains.

I had the exact same problem with TF and 3 session template. today is the first day that everything is working fine. don't know if it has something to do with NT7 rc1 version. are you using the same version?

Reply With Quote
 
  #80 (permalink)
Elite Member
San Francisco Bay Area
 
Futures Experience: Intermediate
Platform: NT,TOS,IB
Favorite Futures: ES,CL,TF
 
Posts: 278 since Jun 2010
Thanks: 154 given, 267 received



Silvester17 View Post
I had the exact same problem with TF and 3 session template. today is the first day that everything is working fine. don't know if it has something to do with NT7 rc1 version. are you using the same version?

gomi made some changes in the recorder file. That fixed the monday bug but there is this Friday issue now.

Reply With Quote

Reply



futures io > > > > > Using OnMarketData() on Historical data with a recording engine

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
NT Providing Bad Price & Volume Data in OnMarketData!!! RJay NinjaTrader Programming 18 June 5th, 2017 10:53 AM
Historical CL Tick Data and Minute Data Big Mike The Elite Circle 112 July 26th, 2014 07:07 PM
BuySellVolumeTotalG39 Lost data recording ability JohnPS The Elite Circle 2 April 26th, 2011 12:30 AM
Historical Tick Data togier Reviews of Brokers and Data Feeds 2 December 3rd, 2010 05:09 PM
Anyone not receive pre-market data while recording with NinajTrader via IQFeed today? richw Reviews of Brokers and Data Feeds 1 May 10th, 2010 08:48 PM


All times are GMT -4. The time now is 07:39 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-11 in 0.15 seconds with 20 queries on phoenix via your IP 54.226.132.197