Using OnMarketData() on Historical data with a recording engine - futures io
futures io



Using OnMarketData() on Historical data with a recording engine


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one gomi with 34 posts (87 thanks)
    2. looks_two aviat72 with 10 posts (2 thanks)
    3. looks_3 danjurgens with 9 posts (3 thanks)
    4. looks_4 Michael.H with 8 posts (0 thanks)
      Best Posters
    1. looks_one gomi with 2.6 thanks per post
    2. looks_two Zondor with 1 thanks per post
    3. looks_3 danjurgens with 0.3 thanks per post
    4. looks_4 aviat72 with 0.2 thanks per post
    1. trending_up 38,632 views
    2. thumb_up 104 thanks given
    3. group 28 followers
    1. forum 115 posts
    2. attach_file 23 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Using OnMarketData() on Historical data with a recording engine

(login for full post details)
  #31 (permalink)
 gomi 
Market Wizard
Paris
 
Experience: None
Platform: NinjaTrader
 
Posts: 1,255 since Oct 2009
Thanks: 276 given, 4,462 received

Here's a new version of the Recorder.

Added

File Formats


* Ability to read directy QCollector , QCollectorIQ and IRT files
* Millisec format, which will try to use the current clock to record millisecond timestamps. The recorder will try to adapt the lag between the exchange and the PC to keep timestamps matched (see for an indy that monitors the lag). And thanks Zondor, there was indeed a bug ;-)

Methods

Added GomOnBarUpdateDone method to allow better performance.

So now the whenever OnBarUpdate is called on the indicator, it calls first GomOnBarUpdate, then it does its internal stuff like reading Gom files, and when everything is finished it calls GomOnBarUpdateDone

Ideally, you should put in GomOnBarUpdate the code thas is used to initialize each new bar, then in GomOnMarketData the code that updates your own data objects, and in GomOnBarUpdateDone the code that plots your internal objects by setting values to time series.

See enclosed GomDeltaVolume to see example implementation.

Attached Thumbnails
Click image for larger version

Name:	millisec.png
Views:	423
Size:	39.2 KB
ID:	15178  
Attached Files
Register to download File Type: cs GomRecorderIndicator.cs (67.3 KB, 130 views)
Register to download File Type: cs GomDeltaVolume.cs (8.7 KB, 112 views)
Started this thread Reply With Quote
The following 12 users say Thank You to gomi for this post:

Can you help answer these questions
from other members on futures io?
TOS color palette
ThinkOrSwim
Tick counter that works with directions?
NinjaTrader
NT8 SuperDOM Column programming - display image
NinjaTrader
Market data replay for HSI, MSI, HHI and MCH
NinjaTrader
Testing Market Types
Elite Quantitative Trading
 
Best Threads (Most Thanked)
in the last 7 days on futures io
The Beast Slayer, Lances NQ Trading Journal
42 thanks
Scalping ES
16 thanks
Machine Learning Journal
15 thanks
es vs anything else
11 thanks
Design a DayTrader Scalping Trend Pivot Indicator
11 thanks
 
(login for full post details)
  #32 (permalink)
 zikonc 
san ramon
 
Experience: Beginner
Platform: ninja
Broker: zen
Trading: es, 6e
 
Posts: 136 since Oct 2009
Thanks: 98 given, 76 received


gomi View Post

So now the whenever OnBarUpdate is called on the indicator, it calls first GomOnBarUpdate, then it does its internal stuff like reading Gom files, and when everything is finished it calls GomOnBarUpdateDone.

Hi Gomi, would you say that this new function GomOnBarUpdateDone would help VolumeLadder?.......if so, perhaps you can tell us where in the code you would add it ...... thanks a bunch .....

Reply With Quote
 
(login for full post details)
  #33 (permalink)
 gomi 
Market Wizard
Paris
 
Experience: None
Platform: NinjaTrader
 
Posts: 1,255 since Oct 2009
Thanks: 276 given, 4,462 received


no the ladder uses it own plot methods. It's more for indicators that use standard plotting methods and Ninja dataseries.

The problem is that if you put your plotting code in GomOnMarketData you will end plotting too often. So it's best to put it in GomOnBarUpdateDone.

Started this thread Reply With Quote
The following user says Thank You to gomi for this post:
 
(login for full post details)
  #34 (permalink)
 Michael.H 
CA
 
Experience: Master
Platform: Marketdelta and Ninja
Broker: Velocity
Trading: NQ
 
Posts: 670 since Apr 2010
Thanks: 64 given, 527 received

my chart doesn't say gom millisecond recorder on it. Im not using the delta volume indicator, just the gom cd. I know before you didn't have to actually load the volume recorder on the chart, but do you have to on this one?

Reply With Quote
 
(login for full post details)
  #35 (permalink)
 gomi 
Market Wizard
Paris
 
Experience: None
Platform: NinjaTrader
 
Posts: 1,255 since Oct 2009
Thanks: 276 given, 4,462 received

no, but did you launch a recompilation using F5 on any indicator ?

Started this thread Reply With Quote
 
(login for full post details)
  #36 (permalink)
 rsi77 
Chicago
 
Experience: Intermediate
Platform: Ninja Trader & TradeStation
Broker: Amp Futures/Zen-Fire
Trading: ES
 
Posts: 56 since Jun 2009
Thanks: 17 given, 48 received

Hi Gomi,
Will new indicators need to be written or updated for this recorder, or will the current GOM package benefit from this recorder. (probably a stupid question but I don't quite understand how the GOM stuff works...I'm just glad it does!)

Ron

Reply With Quote
 
(login for full post details)
  #37 (permalink)
 Michael.H 
CA
 
Experience: Master
Platform: Marketdelta and Ninja
Broker: Velocity
Trading: NQ
 
Posts: 670 since Apr 2010
Thanks: 64 given, 527 received

I actually went to edit ninjascripts, and did a compile manually. I then did f5 on all my charts, still won't say millisecond recorder. its says using binary on the side. Here's a pic


Gomi.. sorry.. stupid mistake. I didn't choose milisecond on the bottom. Few question...
1. will it replace the old bid/ask data if i switch formats to millisecond? found out it creates a new file... so disregard.
2. Is there a file converter that allows you to convert old binary data to be ported to millisecond data so it can use it as a lookback?
3. Also, if I have q collector, will it read it as soon as q collector is finished downloading it( after hitting F5 of course)? I don't have q collector yet. Reason i ask is simply because technically, it takes a few seconds to download data, and with this feature, there's no longer a need to leave computer on running 24/5
4.If we use q collector to backfill, can it ammend to millisecond data or is it a totally different file format (meaning you download for missing data, then as new data comes in using the recorder, that picks off where the qcollector left of)?

Is there a file format that you actually prefer or think works best?




this is truely amazing, you managed to overcome ninja's greatest problem, by yourself.....
I was talking to tim about this.... amazing how one person can do so much, but a team of coders at ninja couldn't fix this....

Attached Thumbnails
Click image for larger version

Name:	Capture.PNG
Views:	268
Size:	19.5 KB
ID:	15284  
Reply With Quote
 
(login for full post details)
  #38 (permalink)
 gomi 
Market Wizard
Paris
 
Experience: None
Platform: NinjaTrader
 
Posts: 1,255 since Oct 2009
Thanks: 276 given, 4,462 received


rsi77 View Post
Hi Gomi,
Will new indicators need to be written or updated for this recorder, or will the current GOM package benefit from this recorder. (probably a stupid question but I don't quite understand how the GOM stuff works...I'm just glad it does!)

Ron

No nothing needs to be rewritten. All current indicators can benefit from this one.

Started this thread Reply With Quote
 
(login for full post details)
  #39 (permalink)
 gomi 
Market Wizard
Paris
 
Experience: None
Platform: NinjaTrader
 
Posts: 1,255 since Oct 2009
Thanks: 276 given, 4,462 received


Michael.H View Post
1. will it replace the old bid/ask data if i switch formats to millisecond? found out it creates a new file... so disregard.
2. Is there a file converter that allows you to convert old binary data to be ported to millisecond data so it can use it as a lookback?
3. Also, if I have q collector, will it read it as soon as q collector is finished downloading it( after hitting F5 of course)? I don't have q collector yet. Reason i ask is simply because technically, it takes a few seconds to download data, and with this feature, there's no longer a need to leave computer on running 24/5
4.If we use q collector to backfill, can it ammend to millisecond data or is it a totally different file format (meaning you download for missing data, then as new data comes in using the recorder, that picks off where the qcollector left of)?

Is there a file format that you actually prefer or think works best?

2. no. Millisecond data has to be written real time. It may actually work though if you use an input file (IRT, QCollector etc ) and your input file has milliseconds timestamps.
3. YOu will have a problem because the indicators keep a lock on the file so it won't be updateable by QCollector
I wanted to update the indicator with all the nice job of gooni ( ) but actually forgot;-)
Anyway, for now, you can't update your file using QCollector while it's used by any Gom indicator
4. I don't think QCollector does millisec.
5. File format preference : this millisec stuff is for guys that want to play around with millisec data. It may not be usable at all(check your network jitter with then monitoring app). If you don't need it you can stick to binary

Careful anyway using directyl QCollector files : in my case with esignal, 1 week is 500Mo, so parsing of the file is very long. Binary uses a few Mo for the same week, so you can read contract-long data with not too much pain.

Started this thread Reply With Quote
 
(login for full post details)
  #40 (permalink)
 gomi 
Market Wizard
Paris
 
Experience: None
Platform: NinjaTrader
 
Posts: 1,255 since Oct 2009
Thanks: 276 given, 4,462 received


Here's a new Converter that handles Millisec format.

You also have a command line mode using
GomFileConverter.exe inputfile inputformat outputformat ticksize

I also included the VS2010 project

Attached Files
Register to download File Type: zip GomFileConverterProject.zip (7.86 MB, 261 views)
Register to download File Type: zip GomFileConverter.zip (17.2 KB, 200 views)
Started this thread Reply With Quote
The following 8 users say Thank You to gomi for this post:


futures io Trading Community Platforms and Indicators NinjaTrader > Using OnMarketData() on Historical data with a recording engine


Last Updated on March 6, 2011


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
     



Copyright © 2022 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada), info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts