Using OnMarketData() on Historical data with a recording engine - futures io
futures io



Using OnMarketData() on Historical data with a recording engine


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one gomi with 34 posts (87 thanks)
    2. looks_two aviat72 with 10 posts (2 thanks)
    3. looks_3 danjurgens with 9 posts (3 thanks)
    4. looks_4 Michael.H with 8 posts (0 thanks)
      Best Posters
    1. looks_one gomi with 2.6 thanks per post
    2. looks_two Zondor with 1 thanks per post
    3. looks_3 danjurgens with 0.3 thanks per post
    4. looks_4 aviat72 with 0.2 thanks per post
    1. trending_up 35,988 views
    2. thumb_up 104 thanks given
    3. group 28 followers
    1. forum 115 posts
    2. attach_file 23 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Using OnMarketData() on Historical data with a recording engine

(login for full post details)
  #101 (permalink)
 aviat72 
San Francisco Bay Area
 
Experience: Intermediate
Platform: NT,TOS,IB
Trading: ES,CL,TF
 
Posts: 280 since Jun 2010
Thanks: 155 given, 271 received


danjurgens View Post
BDB accepts duplicate keys and duplicate key/data pairs just fine. You just have to use a construct they call a cursor to access all the data with duplicate keys. The conflict between realtime writers and backfill writers resolves itself when you force the backfill writer to provide all the tick data for each unique time stamp at once. Then you just delete and replace all the data for each time stamp with the back filled data.

Yes. That is exactly what I had in mind; overwriting the existing data is important.


Quoting 

It's theoretically possible you would lose some ticks if you back fill up to the current real time second, but I don't see that being possible in my initial implementation. To download back fill for the current partial second and get it processed and to the server while a real time writer is still writing that tick is race the user can avoid by not back filling up to right now, just go a second back.

There will be some latency from the time you make the request and till you get the data. You can not be sure of the time-stamp of the information returned by the historical data request. What you could do is buffer the live data while you are requesting historical data and then using time-stamps, and some local tick matching, find where the historical data tick ends and then store the buffered live data.

Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #102 (permalink)
 paolfili 
italy
 
Experience: Intermediate
Platform: Ninjatrader
Broker: Zen-Fire
Trading: ES
 
Posts: 33 since Nov 2009
Thanks: 51 given, 11 received

GomCD CalculateDeltaIndicator method can use BidAsk (Price >=Ask Price<=Bid), UpDownTick (TickUp=+1,TickDown=-1) UpDownTickWithContinuation...
I need to count the Ticks on Bid and and the Ticks on Ask. (No Up or Down....in the while the Bid/ask can change).
Some suggestions on this subject before I modify the GomdeltaIndicator to insert this culculation mode?

Thanks

Paolo

Reply With Quote
 
(login for full post details)
  #103 (permalink)
 danjurgens 
Austin, TX
 
Experience: Advanced
Platform: TradeStation, TWS
Broker: TradeStation, IB .. for now
Trading: 6E, ES
 
Posts: 28 since May 2010
Thanks: 0 given, 28 received


pfx111 View Post
GomCD CalculateDeltaIndicator method can use BidAsk (Price >=Ask Price<=Bid), UpDownTick (TickUp=+1,TickDown=-1) UpDownTickWithContinuation...
I need to count the Ticks on Bid and and the Ticks on Ask. (No Up or Down....in the while the Bid/ask can change).
Some suggestions on this subject before I modify the GomdeltaIndicator to insert this culculation mode?

Thanks

Paolo

So you want to filter out ticks that are AboveAsk/BelowBid/Between?

Without having the code right in front of me I might be slightly off the mark, but I think you need to Add a calculationModeType for Delta at Bid/Ask assume you call it AtBidAsk.

then in the CalcDelta function add a case for your new type as such:
else if ((calcmode==GomCDCalculationModeType.AtBidAsk) && (tickType!=TickTypeEnum.Unknown))
{
if ((tickType==TickTypeEnum.AtBid))
delta =-volume;
else if ((tickType==TickTypeEnum.AtAsk))
delta=volume;
}

Reply With Quote
 
(login for full post details)
  #104 (permalink)
 Zondor 
Portland Oregon, United States
 
Experience: Beginner
Platform: NinjatraderŽ
Broker: CQG, Kinetick
Trading: Gameplay KlownbineŽ Trading of Globex
 
Zondor's Avatar
 
Posts: 1,347 since Jul 2009
Thanks: 1,245 given, 2,721 received

The tickTypes are BelowBid, AtBid, BetweenBidAsk, AtAsk, and AboveAsk.

So, in GomOnMarketData() you just respond to those incoming ticks with the tickTypes representing the types of trades you want to count.

Or you respond to them selectively. For example, trades that are above the ask and below the bid can be counted in the Grande Totals of buyvolume and sellvolume, but also can be counted separately. If you do this, in a bar chart showing buyvolumes and sellvolumes, the "outside" trades can be shown as comprising PORTIONS of the buyvolume and sellvolume bars, whilst also being counted within the totals of buyvolume and sellvolume. See the Buy/Sell Volume indicators I have posted in the Elite downloads section for code examples.

Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following 2 users say Thank You to Zondor for this post:
 
(login for full post details)
  #105 (permalink)
 RJay 
Hartford, CT. USA
 
Experience: Intermediate
Platform: NinjaTrader
Broker: AMP/CQG, Kinetick
Trading: RTY
 
RJay's Avatar
 
Posts: 688 since Jun 2009
Thanks: 748 given, 781 received


pfx111 View Post
GomCD CalculateDeltaIndicator method can use BidAsk (Price >=Ask Price<=Bid), UpDownTick (TickUp=+1,TickDown=-1) UpDownTickWithContinuation...
I need to count the Ticks on Bid and and the Ticks on Ask. (No Up or Down....in the while the Bid/ask can change).
Some suggestions on this subject before I modify the GomdeltaIndicator to insert this culculation mode?

Thanks

Paolo

I assume you want the ticks that are completed orders on the bid and ask sides of the market.

Non Gomi indicator.

------------------------------------------------------------------------------------------------------------

if (e.MarketDataType == MarketDataType.Ask)AskPrice = e.Price;

if (e.MarketDataType == MarketDataType.Bid)BidPrice = e.Price;

if (e.MarketDataType == MarketDataType.Last)LastPrice = e.Price;

if (e.MarketDataType != MarketDataType.Last)return;


if((LastPrice >= AskPrice) && (AskPrice!=0)) AskTickTotal ++;
else if((LastPrice <= BidPrice) && (BidPrice!=0)) BidTicktotal--;

--------------------------------------------------------------------------------------------------------

Gomi indicator:

---------------------------------------------------------------------------------------------------------

if ((tickType==TickTypeEnum.AboveAsk) || (tickType==TickTypeEnum.AtAsk)) AskTickTotal ++ ;

else if ((tickType==TickTypeEnum.BelowBid) || (tickType==TickTypeEnum.AtBid)) CurrentBidVolumeBidTicktotal--;

----------------------------------------------------------------------------------------------------------

RJay

Reply With Quote
The following user says Thank You to RJay for this post:
 
(login for full post details)
  #106 (permalink)
 danjurgens 
Austin, TX
 
Experience: Advanced
Platform: TradeStation, TWS
Broker: TradeStation, IB .. for now
Trading: 6E, ES
 
Posts: 28 since May 2010
Thanks: 0 given, 28 received

My first attempt at the Database using Tick server is available for your downloading pleasure:


Reply With Quote
The following user says Thank You to danjurgens for this post:
 
(login for full post details)
  #107 (permalink)
 gomi 
Paris
 
Experience: None
Platform: NinjaTrader
 
Posts: 1,248 since Oct 2009
Thanks: 274 given, 4,420 received

Here's a new beta version of the recorder
features:
* Added "Volume Split" setting, that will split ticks on constant volume charts. It works on historical (gom file) and real time data. Disable it if you do any kind of volume filtering.
* Improved the tick rendering in "non split" mode, to minimize difference between non-split volume and the constant volume series NT maintains.
* Rewrote tick rendering engine. Now 100% the same as Volume[0] on second,tick,volume and range charts. In history and real time. With CalculateOnBarClose=true or false
* NT7 "new tick model" with OnMarketData firing before OnBarUpdate is now OK. NT 6.5 compatibility has been dropped
*Added Ninja file format availability in Read Only mode. This way you can export data from NT and use it directly in Gom indicators (extension .Ninja.txt) to check how the tick engine handles the ticks
* Dropped the "compress ticks" mode that was pretty much useless and made the recording functions a real pain to write, so simplified the recording methods as well
*Added a GomOnStartUp for those wishing access to NT7's OnStartUp
*Changed session handling so there shouln't be problems with multiple session templates

I also include a new DeltaIndicator class, that implements Zondor's "Delta Completion" idea, meaning that when the ticktype is unknown or betweenbidask, we use updowntick mode to assign a delta. So each tick has a delta, and when you sum buy delta and sell delta, you get the total volume of the bar. This setting can be changed (It's called delta completion)

I also include GomVol, that basically plots volume that you can directly compare to Volume[0]. It also show how to correctly implement recorder indies :
*Initialize bars in GomOnBarUpdate
*Do your internal stuff in GomOnMarketData
*Set the values to the indicator time series in GomOnBarUpdate Done

I also include GomVolTest, that will show bar and cumulative difference between Volume[0] and GomVol[0]. It is reset on each session start

I also include GomDeltaVolume for basic delta checks.

Any feedback is most welcome.

Attached Files
Register to download File Type: zip beta.zip (18.3 KB, 76 views)
Started this thread Reply With Quote
The following 19 users say Thank You to gomi for this post:
 
(login for full post details)
  #108 (permalink)
 Michael.H 
CA
 
Experience: Master
Platform: Marketdelta and Ninja
Broker: Velocity
Trading: NQ
 
Posts: 670 since Apr 2010
Thanks: 64 given, 526 received

I get this error when i imported the indicator into the folder... I was trying to use your GOm CD and ladder

Attached Thumbnails
Click image for larger version

Name:	Capture.PNG
Views:	104
Size:	6.0 KB
ID:	25713  
Reply With Quote
 
(login for full post details)
  #109 (permalink)
 gomi 
Paris
 
Experience: None
Platform: NinjaTrader
 
Posts: 1,248 since Oct 2009
Thanks: 274 given, 4,420 received

? Maybe your template file is corrupt. Could you try again on a plain new chart ?

Started this thread Reply With Quote
 
(login for full post details)
  #110 (permalink)
 Silvester17 
Market Wizard
Columbus, OH
 
Experience: None
Platform: NT 8, TOS
Trading: ES
 
Silvester17's Avatar
 
Posts: 3,585 since Aug 2009
Thanks: 5,079 given, 11,380 received


gomi View Post
Here's a new beta version of the recorder
features:
* Added "Volume Split" setting, that will split ticks on constant volume charts. It works on historical (gom file) and real time data. Disable it if you do any kind of volume filtering.
* Improved the tick rendering in "non split" mode, to minimize difference between non-split volume and the constant volume series NT maintains.
* Rewrote tick rendering engine. Now 100% the same as Volume[0] on second,tick,volume and range charts. In history and real time. With CalculateOnBarClose=true or false
* NT7 "new tick model" with OnMarketData firing before OnBarUpdate is now OK. NT 6.5 compatibility has been dropped
*Added Ninja file format availability in Read Only mode. This way you can export data from NT and use it directly in Gom indicators (extension .Ninja.txt) to check how the tick engine handles the ticks
* Dropped the "compress ticks" mode that was pretty much useless and made the recording functions a real pain to write, so simplified the recording methods as well
*Added a GomOnStartUp for those wishing access to NT7's OnStartUp
*Changed session handling so there shouln't be problems with multiple session templates

I also include a new DeltaIndicator class, that implements Zondor's "Delta Completion" idea, meaning that when the ticktype is unknown or betweenbidask, we use updowntick mode to assign a delta. So each tick has a delta, and when you sum buy delta and sell delta, you get the total volume of the bar. This setting can be changed (It's called delta completion)

I also include GomVol, that basically plots volume that you can directly compare to Volume[0]. It also show how to correctly implement recorder indies :
*Initialize bars in GomOnBarUpdate
*Do your internal stuff in GomOnMarketData
*Set the values to the indicator time series in GomOnBarUpdate Done

I also include GomVolTest, that will show bar and cumulative difference between Volume[0] and GomVol[0]. It is reset on each session start

I also include GomDeltaVolume for basic delta checks.

Any feedback is most welcome.

question from a clueless:

if you have other indicators using the recorder with the code "protected override void OnStartUp()", is it OK with the new recorder just to change that code to "protected override void GomOnStartUp()" ?

Reply With Quote
 
(login for full post details)
  #111 (permalink)
 gomi 
Paris
 
Experience: None
Platform: NinjaTrader
 
Posts: 1,248 since Oct 2009
Thanks: 274 given, 4,420 received

yes. that's exactly it.

Started this thread Reply With Quote
The following user says Thank You to gomi for this post:
 
(login for full post details)
  #112 (permalink)
 Silvester17 
Market Wizard
Columbus, OH
 
Experience: None
Platform: NT 8, TOS
Trading: ES
 
Silvester17's Avatar
 
Posts: 3,585 since Aug 2009
Thanks: 5,079 given, 11,380 received


gomi View Post
yes. that's exactly it.

thank you very much. really appreciate your work here.

Reply With Quote
The following user says Thank You to Silvester17 for this post:
 
(login for full post details)
  #113 (permalink)
 gomi 
Paris
 
Experience: None
Platform: NinjaTrader
 
Posts: 1,248 since Oct 2009
Thanks: 274 given, 4,420 received

Hi,

Upgraded to GomRecorder 2.0 in Elite section


Big improvement is the ability to record daily files instead of one big file as of today.

Enjoy !

Started this thread Reply With Quote
The following 4 users say Thank You to gomi for this post:
 
(login for full post details)
  #114 (permalink)
 Gibby 
BC
 
Experience: Advanced
Platform: Ninja
Broker: Amp with Zenfire and IB
Trading: ES, currencies, forex
 
Posts: 69 since Sep 2009
Thanks: 12 given, 11 received

Is there a way to get a delta bar of the average of the last x bars?
Kind of like you have done with the GOMCMDA but instead on the GOMDeltaVolume, to produce a GOMDVMA if you will?
This would show some nice volume delta divergence at equal swing points I think.

Reply With Quote
 
(login for full post details)
  #115 (permalink)
redbull4444
Toronto
 
 
Posts: 6 since Sep 2010
Thanks: 2 given, 1 received

interested in gom_MP had few questions,

is there a step by step guide on using DTN-IQ data feed to backfill historical tick data to GOM_MP?

I believe you need to use the gom recorder? Is there a step-by-step guide for this?

thanks

Reply With Quote
The following user says Thank You to redbull4444 for this post:
 
(login for full post details)
  #116 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 50,004 since Jun 2009
Thanks: 32,468 given, 98,285 received


redbull4444 View Post
interested in gom_MP had few questions,

is there a step by step guide on using DTN-IQ data feed to backfill historical tick data to GOM_MP?

I believe you need to use the gom recorder? Is there a step-by-step guide for this?

thanks

You need to be Elite to access the Gom stuff, but you can see some examples here in Gom User Manual.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote


futures io Trading Community Platforms and Indicators NinjaTrader > Using OnMarketData() on Historical data with a recording engine


Last Updated on March 6, 2011


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
 

Journal Challenge w/$1,800 in prizes!

April
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts