Using OnMarketData() on Historical data with a recording engine - NinjaTrader Programming | futures io social day trading
futures io futures trading


Using OnMarketData() on Historical data with a recording engine
Updated: Views / Replies:28,343 / 115
Created: by gomi Attachments:23

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors – all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you don’t need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 23  
 
Thread Tools Search this Thread
 

Using OnMarketData() on Historical data with a recording engine

  #81 (permalink)
Elite Member
Austin, TX
 
Futures Experience: Advanced
Platform: TradeStation, TWS
Broker/Data: TradeStation, IB .. for now
Favorite Futures: 6E, ES
 
Posts: 28 since May 2010
Thanks: 0 given, 27 received

Regarding multiple files

I too like the idea of multiple files for the data to allow data updates while running. But it's a complicate scenario that still doesn't help you out if you lose your internet connection temporarily during a particular session.

To that end I intend to create a DB back end for data storage. A quick investigation has me leaning towards Berkeley DB (open to other suggestions, BDB may be extreme overkill here, but it's the best documented free non-SQL DB I found). My initial implementation plans are for 1 db file per month per symbol, which is much nicer than 1 large file for backup purposes especially with a backup system like Mozy. The key benefit of using a DB on the back end is that it will allow concurrent updates to the data, which would enable data back fill on any time frame instead of an arbitrary time since the last file split being locked out. Once that back fill completes a chart refresh would get you new data. I plan to extend GomFileConverter for back fill capability (which basically comes for free once the recorder indicator is updated).

Reply With Quote
The following user says Thank You to danjurgens for this post:
 
  #82 (permalink)
Elite Member
Portland Oregon, United States
 
Futures Experience: Beginner
Platform: Ninjatrader®
Broker/Data: CQG, Kinetick
Favorite Futures: Gameplay Klownbine® Trading of Globex
 
Zondor's Avatar
 
Posts: 1,327 since Jul 2009
Thanks: 1,246 given, 2,635 received

database for historical bid ask data

danjurgens, I am extremely interested in this and would like to collaborate with you.

I have quite a bit of experience with RDBMS including architecture, administration, coding of front end applications, etc. Have worked with Access, SQLServer, and Oracle. It's been a while, but I eat this stuff up.

Reply With Quote
 
  #83 (permalink)
Elite Member
Austin, TX
 
Futures Experience: Advanced
Platform: TradeStation, TWS
Broker/Data: TradeStation, IB .. for now
Favorite Futures: 6E, ES
 
Posts: 28 since May 2010
Thanks: 0 given, 27 received


I think I'll get a chance to sit down and take my first stab at this on the weekend. For this project I don't really see the benefit of the RDMS since the relational aspects are pretty one dimensional. Symbol(1)->Ticks(N). Berkeley DB should provide all that is needed with Key->Data storage a mapping of symbols to different databases. It manages caching (configurable) and concurrent access, which are the performance and data integrity sticking points.

My background is in embedded real time development so the C# learning curve will slow me down a bit, but I hope to have a prototype working sometime this weekend. I'll post what I come up with as soon as I have it.

On a side note, it would be nice to have a version management system for the Gom files, has anyone considered setting something up on sourceforge or anything? I have a pretty mixed collection where I get some files from NT's forums, updates from here, etc.

Dan

Reply With Quote
 
  #84 (permalink)
Elite Member
Paris
 
Futures Experience: None
Platform: NinjaTrader
 
Posts: 1,177 since Oct 2009
Thanks: 228 given, 4,097 received


aviat72 View Post
gomi:

I think we still have a problem. I have a CL template with 3 sessions (6:00PM to 9:AM, 9:00AM to 2:30PM, 2:30 PM to 5:15PM EST). Today (Friday) the last day of the week, the recorder stopped sending ticks at 9:00AM when the session rolled over the pit session.

Earlier on, I used to face problems on Monday morning ES (again a 3 session template). I think that is fixed, but the Friday problem remains.

Do you mean a chart with 3 CL, each with its own session time settings ?
Depending on which session the recorder is running, it won't send ticks outside of the associated session. For instance if you have a GomCD associated with the CL running in the 9:00-2:30 session, it won't send ticks outside of this session.

Or am I not understanding the problem ?

Reply With Quote
 
  #85 (permalink)
Market Wizard
Columbus, OH
 
Futures Experience: None
Platform: NT 8, TOS
Favorite Futures: ES
 
Silvester17's Avatar
 
Posts: 3,332 since Aug 2009
Thanks: 4,579 given, 10,313 received


gomi View Post
Do you mean a chart with 3 CL, each with its own session time settings ?
Depending on which session the recorder is running, it won't send ticks outside of the associated session. For instance if you have a GomCD associated with the CL running in the 9:00-2:30 session, it won't send ticks outside of this session.

Or am I not understanding the problem ?

hi gomi,

I have the same problem. 1 chart, 1 instrument and a 3 session template. (template with different sessions, from 20:00 - 09:30, 09:30 - 16:15 and 16:15 - 18:00)

Attached Thumbnails
Using OnMarketData() on Historical data with a recording engine-2010-11-03_1523.png  
Reply With Quote
 
  #86 (permalink)
Elite Member
Paris
 
Futures Experience: None
Platform: NinjaTrader
 
Posts: 1,177 since Oct 2009
Thanks: 228 given, 4,097 received

Thanks.

Not sure this is supported by ninja : I tried on the ES and I don't get Session Break bars, ans Bars.SessionBreak never gets true.

If you want all ticks you can always set "disable tick filter" to true.

Reply With Quote
 
  #87 (permalink)
Trading Apprentice
Willowbrook, IL
 
Futures Experience: None
Platform: Windows 7, Android
Favorite Futures: Mouse and Keyboard
 
tarantino's Avatar
 
Posts: 32 since Sep 2010
Thanks: 71 given, 39 received


gomi View Post
Thanks.

Not sure this is supported by ninja : I tried on the ES and I don't get Session Break bars, ans Bars.SessionBreak never gets true.

If you want all ticks you can always set "disable tick filter" to true.

How about if(Bars.FirstBarOfSession) ?

Reply With Quote
 
  #88 (permalink)
Elite Member
italy
 
Futures Experience: Intermediate
Platform: Ninjatrader
Broker/Data: Zen-Fire
Favorite Futures: ES
 
Posts: 33 since Nov 2009
Thanks: 51 given, 11 received

OT: Cumulative Delta splitter

Sorry.Probably OT,but gomi presence can be useful for some suggestions.
I' m interested in having in the same panel some CD of the different 'size players' (i.e. on five level <5,6<x<49,50<x<99,100<x<299,x>300 - kind of Volume Splitter on Cumulative Delta with different lines color for different players).
Is there any way to achieve without big revolution on the code?
Can gomi or some other CD expert programmer (Zondor?) give me some hints?
Thanks


Last edited by paolfili; November 7th, 2010 at 05:59 PM.
Reply With Quote
 
  #89 (permalink)
 R.I.P. 1960-2010 
 
Futures Experience: Advanced
Platform: NinjaTrader7
Broker/Data: Mirus RCG/Zen-Fire
Favorite Futures: CL & 6e, looking at ES, ZB and AU again.
 
ZTR's Avatar
 
Posts: 2,099 since Nov 2009
Thanks: 1,104 given, 1,390 received


pfx111 View Post
Sorry.Probably OT,but gomi presence can be useful for some suggestions.
I' m interested in having in the same panel some CD of the different 'size player' (i.e. on five level <5,6<x<49,50<x<99,100<x<299,x>300 - kind of Volume Splitter on Cumulative Delta with different lines color for different players).
Is there any way to achieve without big revolution on the code?
Can gomi or some other CD expert programmer (Zondor?) give me some hints?
Thanks

Quick way: You could have more than one Gom CD in the lower panels with different size filters. I have posted ladders with the GOM CD, both filtered by size.

There are differences...

R.I.P. Andy Zektzer (ZTR), 1960-2010.
Please visit this thread for more information.
Attached Thumbnails
Using OnMarketData() on Historical data with a recording engine-vol_filter.jpg  
Reply With Quote
 
  #90 (permalink)
Elite Member
Austin, TX
 
Futures Experience: Advanced
Platform: TradeStation, TWS
Broker/Data: TradeStation, IB .. for now
Favorite Futures: 6E, ES
 
Posts: 28 since May 2010
Thanks: 0 given, 27 received

Update on DB back-end.


I've started working on the DB back-end for gomRecorderIndicator. It got a little more complicated when I figured out that bringing up the DB had to be serialized, that added a lot of complexity if one wanted to keep it as an NT indicator, so I'm taking a slightly different route. I'm in the process of creating it as stand alone server app. I've got the server side up and running, although not really debugged at all. I'm working on the client side now.

The downside is I probably won't have anything I can distribute until next weekend.

Once I do that I'll start a new thread instead of continuing to hijack this one!

Reply With Quote

Reply



futures io > > > > > Using OnMarketData() on Historical data with a recording engine

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
NT Providing Bad Price & Volume Data in OnMarketData!!! RJay NinjaTrader Programming 18 June 5th, 2017 10:53 AM
Historical CL Tick Data and Minute Data Big Mike The Elite Circle 112 July 26th, 2014 07:07 PM
BuySellVolumeTotalG39 Lost data recording ability JohnPS The Elite Circle 2 April 26th, 2011 12:30 AM
Historical Tick Data togier Reviews of Brokers and Data Feeds 2 December 3rd, 2010 05:09 PM
Anyone not receive pre-market data while recording with NinajTrader via IQFeed today? richw Reviews of Brokers and Data Feeds 1 May 10th, 2010 08:48 PM


All times are GMT -4. The time now is 11:22 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-16 in 0.13 seconds with 20 queries on phoenix via your IP 54.226.34.209