Using OnMarketData() on Historical data with a recording engine - NinjaTrader Programming | futures io social day trading
futures io futures trading


Using OnMarketData() on Historical data with a recording engine
Updated: Views / Replies:28,298 / 115
Created: by gomi Attachments:23

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors – all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you don’t need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 23  
 
Thread Tools Search this Thread
 

Using OnMarketData() on Historical data with a recording engine

  #101 (permalink)
Elite Member
San Francisco Bay Area
 
Futures Experience: Intermediate
Platform: NT,TOS,IB
Favorite Futures: ES,CL,TF
 
Posts: 278 since Jun 2010
Thanks: 154 given, 267 received


danjurgens View Post
BDB accepts duplicate keys and duplicate key/data pairs just fine. You just have to use a construct they call a cursor to access all the data with duplicate keys. The conflict between realtime writers and backfill writers resolves itself when you force the backfill writer to provide all the tick data for each unique time stamp at once. Then you just delete and replace all the data for each time stamp with the back filled data.

Yes. That is exactly what I had in mind; overwriting the existing data is important.


Quoting 

It's theoretically possible you would lose some ticks if you back fill up to the current real time second, but I don't see that being possible in my initial implementation. To download back fill for the current partial second and get it processed and to the server while a real time writer is still writing that tick is race the user can avoid by not back filling up to right now, just go a second back.

There will be some latency from the time you make the request and till you get the data. You can not be sure of the time-stamp of the information returned by the historical data request. What you could do is buffer the live data while you are requesting historical data and then using time-stamps, and some local tick matching, find where the historical data tick ends and then store the buffered live data.

Reply With Quote
 
  #102 (permalink)
Elite Member
italy
 
Futures Experience: Intermediate
Platform: Ninjatrader
Broker/Data: Zen-Fire
Favorite Futures: ES
 
Posts: 33 since Nov 2009
Thanks: 51 given, 11 received

Gompackage - suggestions

GomCD CalculateDeltaIndicator method can use BidAsk (Price >=Ask Price<=Bid), UpDownTick (TickUp=+1,TickDown=-1) UpDownTickWithContinuation...
I need to count the Ticks on Bid and and the Ticks on Ask. (No Up or Down....in the while the Bid/ask can change).
Some suggestions on this subject before I modify the GomdeltaIndicator to insert this culculation mode?

Thanks

Paolo

Reply With Quote
 
  #103 (permalink)
Elite Member
Austin, TX
 
Futures Experience: Advanced
Platform: TradeStation, TWS
Broker/Data: TradeStation, IB .. for now
Favorite Futures: 6E, ES
 
Posts: 28 since May 2010
Thanks: 0 given, 27 received



pfx111 View Post
GomCD CalculateDeltaIndicator method can use BidAsk (Price >=Ask Price<=Bid), UpDownTick (TickUp=+1,TickDown=-1) UpDownTickWithContinuation...
I need to count the Ticks on Bid and and the Ticks on Ask. (No Up or Down....in the while the Bid/ask can change).
Some suggestions on this subject before I modify the GomdeltaIndicator to insert this culculation mode?

Thanks

Paolo

So you want to filter out ticks that are AboveAsk/BelowBid/Between?

Without having the code right in front of me I might be slightly off the mark, but I think you need to Add a calculationModeType for Delta at Bid/Ask assume you call it AtBidAsk.

then in the CalcDelta function add a case for your new type as such:
else if ((calcmode==GomCDCalculationModeType.AtBidAsk) && (tickType!=TickTypeEnum.Unknown))
{
if ((tickType==TickTypeEnum.AtBid))
delta =-volume;
else if ((tickType==TickTypeEnum.AtAsk))
delta=volume;
}

Reply With Quote
 
  #104 (permalink)
Elite Member
Portland Oregon, United States
 
Futures Experience: Beginner
Platform: Ninjatrader®
Broker/Data: CQG, Kinetick
Favorite Futures: Gameplay Klownbine® Trading of Globex
 
Zondor's Avatar
 
Posts: 1,327 since Jul 2009
Thanks: 1,246 given, 2,635 received

Using Gomi's tickTypes

The tickTypes are BelowBid, AtBid, BetweenBidAsk, AtAsk, and AboveAsk.

So, in GomOnMarketData() you just respond to those incoming ticks with the tickTypes representing the types of trades you want to count.

Or you respond to them selectively. For example, trades that are above the ask and below the bid can be counted in the Grande Totals of buyvolume and sellvolume, but also can be counted separately. If you do this, in a bar chart showing buyvolumes and sellvolumes, the "outside" trades can be shown as comprising PORTIONS of the buyvolume and sellvolume bars, whilst also being counted within the totals of buyvolume and sellvolume. See the Buy/Sell Volume indicators I have posted in the Elite downloads section for code examples.

Reply With Quote
The following 2 users say Thank You to Zondor for this post:
 
  #105 (permalink)
 Vendor: www.innovative-trading-solutions-online.com 
Hartford, CT. USA
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: AMP/CQG, Kinetick
Favorite Futures: TF
 
RJay's Avatar
 
Posts: 677 since Jun 2009
Thanks: 726 given, 740 received


pfx111 View Post
GomCD CalculateDeltaIndicator method can use BidAsk (Price >=Ask Price<=Bid), UpDownTick (TickUp=+1,TickDown=-1) UpDownTickWithContinuation...
I need to count the Ticks on Bid and and the Ticks on Ask. (No Up or Down....in the while the Bid/ask can change).
Some suggestions on this subject before I modify the GomdeltaIndicator to insert this culculation mode?

Thanks

Paolo

I assume you want the ticks that are completed orders on the bid and ask sides of the market.

Non Gomi indicator.

------------------------------------------------------------------------------------------------------------

if (e.MarketDataType == MarketDataType.Ask)AskPrice = e.Price;

if (e.MarketDataType == MarketDataType.Bid)BidPrice = e.Price;

if (e.MarketDataType == MarketDataType.Last)LastPrice = e.Price;

if (e.MarketDataType != MarketDataType.Last)return;


if((LastPrice >= AskPrice) && (AskPrice!=0)) AskTickTotal ++;
else if((LastPrice <= BidPrice) && (BidPrice!=0)) BidTicktotal--;

--------------------------------------------------------------------------------------------------------

Gomi indicator:

---------------------------------------------------------------------------------------------------------

if ((tickType==TickTypeEnum.AboveAsk) || (tickType==TickTypeEnum.AtAsk)) AskTickTotal ++ ;

else if ((tickType==TickTypeEnum.BelowBid) || (tickType==TickTypeEnum.AtBid)) CurrentBidVolumeBidTicktotal--;

----------------------------------------------------------------------------------------------------------

RJay

Reply With Quote
The following user says Thank You to RJay for this post:
 
  #106 (permalink)
Elite Member
Austin, TX
 
Futures Experience: Advanced
Platform: TradeStation, TWS
Broker/Data: TradeStation, IB .. for now
Favorite Futures: 6E, ES
 
Posts: 28 since May 2010
Thanks: 0 given, 27 received

My first attempt at the Database using Tick server is available for your downloading pleasure:

https://futures.io/ninjatrader-programming/6684-i-stab-gomrecording-db-backend.html#post76982

Reply With Quote
The following user says Thank You to danjurgens for this post:
 
  #107 (permalink)
Elite Member
Paris
 
Futures Experience: None
Platform: NinjaTrader
 
Posts: 1,173 since Oct 2009
Thanks: 227 given, 4,089 received

Here's a new beta version of the recorder
features:
* Added "Volume Split" setting, that will split ticks on constant volume charts. It works on historical (gom file) and real time data. Disable it if you do any kind of volume filtering.
* Improved the tick rendering in "non split" mode, to minimize difference between non-split volume and the constant volume series NT maintains.
* Rewrote tick rendering engine. Now 100% the same as Volume[0] on second,tick,volume and range charts. In history and real time. With CalculateOnBarClose=true or false
* NT7 "new tick model" with OnMarketData firing before OnBarUpdate is now OK. NT 6.5 compatibility has been dropped
*Added Ninja file format availability in Read Only mode. This way you can export data from NT and use it directly in Gom indicators (extension .Ninja.txt) to check how the tick engine handles the ticks
* Dropped the "compress ticks" mode that was pretty much useless and made the recording functions a real pain to write, so simplified the recording methods as well
*Added a GomOnStartUp for those wishing access to NT7's OnStartUp
*Changed session handling so there shouln't be problems with multiple session templates

I also include a new DeltaIndicator class, that implements Zondor's "Delta Completion" idea, meaning that when the ticktype is unknown or betweenbidask, we use updowntick mode to assign a delta. So each tick has a delta, and when you sum buy delta and sell delta, you get the total volume of the bar. This setting can be changed (It's called delta completion)

I also include GomVol, that basically plots volume that you can directly compare to Volume[0]. It also show how to correctly implement recorder indies :
*Initialize bars in GomOnBarUpdate
*Do your internal stuff in GomOnMarketData
*Set the values to the indicator time series in GomOnBarUpdate Done

I also include GomVolTest, that will show bar and cumulative difference between Volume[0] and GomVol[0]. It is reset on each session start

I also include GomDeltaVolume for basic delta checks.

Any feedback is most welcome.

Attached Files
Register to download File Type: zip beta.zip (18.3 KB, 73 views)
Reply With Quote
The following 19 users say Thank You to gomi for this post:
 
  #108 (permalink)
Elite Member
CA
 
Futures Experience: Master
Platform: Marketdelta and Ninja
Broker/Data: Velocity
Favorite Futures: NQ
 
Posts: 670 since Apr 2010
Thanks: 64 given, 521 received

I get this error when i imported the indicator into the folder... I was trying to use your GOm CD and ladder

Attached Thumbnails
Using OnMarketData() on Historical data with a recording engine-capture.png  
Reply With Quote
 
  #109 (permalink)
Elite Member
Paris
 
Futures Experience: None
Platform: NinjaTrader
 
Posts: 1,173 since Oct 2009
Thanks: 227 given, 4,089 received

? Maybe your template file is corrupt. Could you try again on a plain new chart ?

Reply With Quote
 
  #110 (permalink)
Market Wizard
Columbus, OH
 
Futures Experience: None
Platform: NT 8, TOS
Favorite Futures: ES
 
Silvester17's Avatar
 
Posts: 3,332 since Aug 2009
Thanks: 4,579 given, 10,313 received



gomi View Post
Here's a new beta version of the recorder
features:
* Added "Volume Split" setting, that will split ticks on constant volume charts. It works on historical (gom file) and real time data. Disable it if you do any kind of volume filtering.
* Improved the tick rendering in "non split" mode, to minimize difference between non-split volume and the constant volume series NT maintains.
* Rewrote tick rendering engine. Now 100% the same as Volume[0] on second,tick,volume and range charts. In history and real time. With CalculateOnBarClose=true or false
* NT7 "new tick model" with OnMarketData firing before OnBarUpdate is now OK. NT 6.5 compatibility has been dropped
*Added Ninja file format availability in Read Only mode. This way you can export data from NT and use it directly in Gom indicators (extension .Ninja.txt) to check how the tick engine handles the ticks
* Dropped the "compress ticks" mode that was pretty much useless and made the recording functions a real pain to write, so simplified the recording methods as well
*Added a GomOnStartUp for those wishing access to NT7's OnStartUp
*Changed session handling so there shouln't be problems with multiple session templates

I also include a new DeltaIndicator class, that implements Zondor's "Delta Completion" idea, meaning that when the ticktype is unknown or betweenbidask, we use updowntick mode to assign a delta. So each tick has a delta, and when you sum buy delta and sell delta, you get the total volume of the bar. This setting can be changed (It's called delta completion)

I also include GomVol, that basically plots volume that you can directly compare to Volume[0]. It also show how to correctly implement recorder indies :
*Initialize bars in GomOnBarUpdate
*Do your internal stuff in GomOnMarketData
*Set the values to the indicator time series in GomOnBarUpdate Done

I also include GomVolTest, that will show bar and cumulative difference between Volume[0] and GomVol[0]. It is reset on each session start

I also include GomDeltaVolume for basic delta checks.

Any feedback is most welcome.

question from a clueless:

if you have other indicators using the recorder with the code "protected override void OnStartUp()", is it OK with the new recorder just to change that code to "protected override void GomOnStartUp()" ?

Reply With Quote

Reply



futures io > > > > > Using OnMarketData() on Historical data with a recording engine

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
NT Providing Bad Price & Volume Data in OnMarketData!!! RJay NinjaTrader Programming 18 June 5th, 2017 10:53 AM
Historical CL Tick Data and Minute Data Big Mike The Elite Circle 112 July 26th, 2014 07:07 PM
BuySellVolumeTotalG39 Lost data recording ability JohnPS The Elite Circle 2 April 26th, 2011 12:30 AM
Historical Tick Data togier Reviews of Brokers and Data Feeds 2 December 3rd, 2010 05:09 PM
Anyone not receive pre-market data while recording with NinajTrader via IQFeed today? richw Reviews of Brokers and Data Feeds 1 May 10th, 2010 08:48 PM


All times are GMT -4. The time now is 05:52 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-11 in 0.15 seconds with 20 queries on phoenix via your IP 54.82.79.109