NexusFi: Find Your Edge


Home Menu

 





Using OnMarketData() on Historical data with a recording engine


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one gomi with 34 posts (87 thanks)
    2. looks_two aviat72 with 10 posts (2 thanks)
    3. looks_3 danjurgens with 9 posts (3 thanks)
    4. looks_4 Michael.H with 8 posts (0 thanks)
      Best Posters
    1. looks_one gomi with 2.6 thanks per post
    2. looks_two Zondor with 1 thanks per post
    3. looks_3 danjurgens with 0.3 thanks per post
    4. looks_4 aviat72 with 0.2 thanks per post
    1. trending_up 51,118 views
    2. thumb_up 104 thanks given
    3. group 28 followers
    1. forum 115 posts
    2. attach_file 23 attachments




 
Search this Thread

Using OnMarketData() on Historical data with a recording engine

  #81 (permalink)
 danjurgens 
Austin, TX
 
Experience: Advanced
Platform: TradeStation, TWS
Broker: TradeStation, IB .. for now
Trading: 6E, ES
Posts: 28 since May 2010
Thanks Given: 0
Thanks Received: 28

I too like the idea of multiple files for the data to allow data updates while running. But it's a complicate scenario that still doesn't help you out if you lose your internet connection temporarily during a particular session.

To that end I intend to create a DB back end for data storage. A quick investigation has me leaning towards Berkeley DB (open to other suggestions, BDB may be extreme overkill here, but it's the best documented free non-SQL DB I found). My initial implementation plans are for 1 db file per month per symbol, which is much nicer than 1 large file for backup purposes especially with a backup system like Mozy. The key benefit of using a DB on the back end is that it will allow concurrent updates to the data, which would enable data back fill on any time frame instead of an arbitrary time since the last file split being locked out. Once that back fill completes a chart refresh would get you new data. I plan to extend GomFileConverter for back fill capability (which basically comes for free once the recorder indicator is updated).

Reply With Quote
Thanked by:

Can you help answer these questions
from other members on NexusFi?
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
Trade idea based off three indicators.
Traders Hideout
ZombieSqueeze
Platforms and Indicators
PowerLanguage & EasyLanguage. How to get the platfor …
EasyLanguage Programming
How to apply profiles
Traders Hideout
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Spoo-nalysis ES e-mini futures S&P 500
29 thanks
Tao te Trade: way of the WLD
24 thanks
Just another trading journal: PA, Wyckoff & Trends
24 thanks
Bigger Wins or Fewer Losses?
21 thanks
GFIs1 1 DAX trade per day journal
17 thanks
  #82 (permalink)
 
Zondor's Avatar
 Zondor 
Portland Oregon, United States
 
Experience: Beginner
Platform: Ninjatrader®
Broker: CQG, Kinetick
Trading: Gameplay Klownbine® Trading of Globex
Posts: 1,333 since Jul 2009
Thanks Given: 1,246
Thanks Received: 2,731

danjurgens, I am extremely interested in this and would like to collaborate with you.

I have quite a bit of experience with RDBMS including architecture, administration, coding of front end applications, etc. Have worked with Access, SQLServer, and Oracle. It's been a while, but I eat this stuff up.

Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #83 (permalink)
 danjurgens 
Austin, TX
 
Experience: Advanced
Platform: TradeStation, TWS
Broker: TradeStation, IB .. for now
Trading: 6E, ES
Posts: 28 since May 2010
Thanks Given: 0
Thanks Received: 28


I think I'll get a chance to sit down and take my first stab at this on the weekend. For this project I don't really see the benefit of the RDMS since the relational aspects are pretty one dimensional. Symbol(1)->Ticks(N). Berkeley DB should provide all that is needed with Key->Data storage a mapping of symbols to different databases. It manages caching (configurable) and concurrent access, which are the performance and data integrity sticking points.

My background is in embedded real time development so the C# learning curve will slow me down a bit, but I hope to have a prototype working sometime this weekend. I'll post what I come up with as soon as I have it.

On a side note, it would be nice to have a version management system for the Gom files, has anyone considered setting something up on sourceforge or anything? I have a pretty mixed collection where I get some files from NT's forums, updates from here, etc.

Dan

Reply With Quote
  #84 (permalink)
 gomi 
Paris
Market Wizard
 
Experience: None
Platform: NinjaTrader
Posts: 1,270 since Oct 2009
Thanks Given: 282
Thanks Received: 4,505


aviat72 View Post
gomi:

I think we still have a problem. I have a CL template with 3 sessions (6:00PM to 9:AM, 9:00AM to 2:30PM, 2:30 PM to 5:15PM EST). Today (Friday) the last day of the week, the recorder stopped sending ticks at 9:00AM when the session rolled over the pit session.

Earlier on, I used to face problems on Monday morning ES (again a 3 session template). I think that is fixed, but the Friday problem remains.

Do you mean a chart with 3 CL, each with its own session time settings ?
Depending on which session the recorder is running, it won't send ticks outside of the associated session. For instance if you have a GomCD associated with the CL running in the 9:00-2:30 session, it won't send ticks outside of this session.

Or am I not understanding the problem ?

Started this thread Reply With Quote
  #85 (permalink)
 
Silvester17's Avatar
 Silvester17 
Columbus, OH
Market Wizard
 
Experience: None
Platform: NT 8, TOS
Trading: ES
Posts: 3,603 since Aug 2009
Thanks Given: 5,139
Thanks Received: 11,527


gomi View Post
Do you mean a chart with 3 CL, each with its own session time settings ?
Depending on which session the recorder is running, it won't send ticks outside of the associated session. For instance if you have a GomCD associated with the CL running in the 9:00-2:30 session, it won't send ticks outside of this session.

Or am I not understanding the problem ?

hi gomi,

I have the same problem. 1 chart, 1 instrument and a 3 session template. (template with different sessions, from 20:00 - 09:30, 09:30 - 16:15 and 16:15 - 18:00)

Attached Thumbnails
Click image for larger version

Name:	2010-11-03_1523.png
Views:	237
Size:	32.0 KB
ID:	25243  
Reply With Quote
  #86 (permalink)
 gomi 
Paris
Market Wizard
 
Experience: None
Platform: NinjaTrader
Posts: 1,270 since Oct 2009
Thanks Given: 282
Thanks Received: 4,505

Thanks.

Not sure this is supported by ninja : I tried on the ES and I don't get Session Break bars, ans Bars.SessionBreak never gets true.

If you want all ticks you can always set "disable tick filter" to true.

Started this thread Reply With Quote
  #87 (permalink)
tarantino
Willowbrook, IL
 
Posts: 32 since Sep 2010
Thanks Given: 71
Thanks Received: 39


gomi View Post
Thanks.

Not sure this is supported by ninja : I tried on the ES and I don't get Session Break bars, ans Bars.SessionBreak never gets true.

If you want all ticks you can always set "disable tick filter" to true.

How about if(Bars.FirstBarOfSession) ?

Reply With Quote
  #88 (permalink)
 paolfili 
italy
 
Experience: Intermediate
Platform: Ninjatrader
Broker: Zen-Fire
Trading: ES
Posts: 33 since Nov 2009
Thanks Given: 51
Thanks Received: 11

Sorry.Probably OT,but gomi presence can be useful for some suggestions.
I' m interested in having in the same panel some CD of the different 'size players' (i.e. on five level <5,6<x<49,50<x<99,100<x<299,x>300 - kind of Volume Splitter on Cumulative Delta with different lines color for different players).
Is there any way to achieve without big revolution on the code?
Can gomi or some other CD expert programmer (Zondor?) give me some hints?
Thanks

Reply With Quote
  #89 (permalink)
 
ZTR's Avatar
 ZTR 
 
Experience: Advanced
Platform: NinjaTrader7
Broker: Mirus RCG/Zen-Fire
Trading: CL & 6e, looking at ES, ZB and AU again.
Posts: 2,096 since Nov 2009
Thanks Given: 1,099
Thanks Received: 1,393


pfx111 View Post
Sorry.Probably OT,but gomi presence can be useful for some suggestions.
I' m interested in having in the same panel some CD of the different 'size player' (i.e. on five level <5,6<x<49,50<x<99,100<x<299,x>300 - kind of Volume Splitter on Cumulative Delta with different lines color for different players).
Is there any way to achieve without big revolution on the code?
Can gomi or some other CD expert programmer (Zondor?) give me some hints?
Thanks

Quick way: You could have more than one Gom CD in the lower panels with different size filters. I have posted ladders with the [AUTOLINK]GOM[/AUTOLINK] CD, both filtered by size.

There are differences...

R.I.P. Andy Zektzer (ZTR), 1960-2010.
Please visit this thread for more information.
Attached Thumbnails
Click image for larger version

Name:	Vol_Filter.jpg
Views:	245
Size:	74.6 KB
ID:	25439  
Visit my NexusFi Trade Journal Reply With Quote
  #90 (permalink)
 danjurgens 
Austin, TX
 
Experience: Advanced
Platform: TradeStation, TWS
Broker: TradeStation, IB .. for now
Trading: 6E, ES
Posts: 28 since May 2010
Thanks Given: 0
Thanks Received: 28


I've started working on the DB back-end for gomRecorderIndicator. It got a little more complicated when I figured out that bringing up the DB had to be serialized, that added a lot of complexity if one wanted to keep it as an NT indicator, so I'm taking a slightly different route. I'm in the process of creating it as stand alone server app. I've got the server side up and running, although not really debugged at all. I'm working on the client side now.

The downside is I probably won't have anything I can distribute until next weekend.

Once I do that I'll start a new thread instead of continuing to hijack this one!

Reply With Quote




Last Updated on March 6, 2011


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts