Indicator performance bottleneck - NinjaTrader Programming | futures io social day trading
futures io futures trading


Indicator performance bottleneck
Updated: Views / Replies:2,094 / 24
Created: by Adamus Attachments:1

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 1  
 
Thread Tools Search this Thread
 

Indicator performance bottleneck

  #11 (permalink)
Elite Member
London, UK
 
Futures Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker/Data: IB/IQFeed
Favorite Futures: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 470 given, 778 received


ratfink View Post
Just another quick point, getting things up into the milli-second rather than micro-second levels usually takes memory or even hard page faults to become an issue, have you eliminated that side of things? (i.e. the TM is showing (100/number-of-cores)% flat-out for Ninja and no hard faults so it is all cpu). Would be unusual but not unknown, just depends what you are doing.

I had to look up what hard faults are and then I discovered that my resource monitor doesn't display properly, according to Microsoft the work-around for an incompatibility with IE10 is that I reduce my display font size to 100%, but Windows now wants me to log out and log back in so this will have to wait.

I see a 25% CPU utilisation which is 100% on one core when loading the indi. As I said can't tell yet about hard faults (what should I see?)

You can discover what your enemy fears most by observing the means he uses to frighten you.
Reply With Quote
 
  #12 (permalink)
Elite Member
Birmingham UK
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: IG/eSignal
Favorite Futures: Dax
 
ratfink's Avatar
 
Posts: 3,336 since Dec 2012
Thanks: 11,276 given, 7,090 received


Adamus View Post
I see a 25% CPU utilisation which is 100% on one core when loading the indi. As I said can't tell yet about hard faults (what should I see?)

Then it is just a total cpu hog so the hard faults will likely be showing zero anyway and can be ignored. Definitely check for redundant drawing, calculation and object allocation operations, poor data fetch/create methods, excessive routine calling, unexpected looping, etcetera. Tests including dumb prints on small sets, eliminations and and binary chop style commenting out, your mission, should you choose to accept it, starts now.

Travel Well
Reply With Quote
The following user says Thank You to ratfink for this post:
 
  #13 (permalink)
Elite Member
London, UK
 
Futures Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker/Data: IB/IQFeed
Favorite Futures: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 470 given, 778 received



cory View Post
set a calculation kill switch on after calculation on history data is done.

hi cory, at first I didn't understand and just thought 'oh well' but then I saw you have an awesome 3,777 thank-yous and a higher thank-you to post ratio so I figure what you said is probably well worth heeding and I better find out what you mean. What confuses me is that all the calculations I do on history data need to be on every live bar as well, and in fact I don't think I've ever written anything for Ninja where that wasn't so. I do occasionally use the 'Historical' flag to switch when there is something hacky going on with historical vs live data or processing, but usually not. Could you expand a little on what you meant? Thanks.

You can discover what your enemy fears most by observing the means he uses to frighten you.
Reply With Quote
 
  #14 (permalink)
Market Wizard
virginia
 
Futures Experience: Intermediate
Platform: ninja
Favorite Futures: ES
 
cory's Avatar
 
Posts: 5,215 since Jun 2009
Thanks: 628 given, 6,312 received


Adamus View Post
hi cory, at first I didn't understand and just thought 'oh well' but then I saw you have an awesome 3,777 thank-yous and a higher thank-you to post ratio so I figure what you said is probably well worth heeding and I better find out what you mean. What confuses me is that all the calculations I do on history data need to be on every live bar as well, and in fact I don't think I've ever written anything for Ninja where that wasn't so. I do occasionally use the 'Historical' flag to switch when there is something hacky going on with historical vs live data or processing, but usually not. Could you expand a little on what you meant? Thanks.

see if this help

Quoting 
.....

public class NoGapBarbyBar : Indicator
{
#region Variables
// Wizard generated variables
// User defined variables (add any user defined variables below)
private DateTime currentDate = Cbi.Globals.MinDate;
private DateTime now;
private DateTime prevDay;
private DateTime prev2Day;
private DateTime prev3Day;


.....
#endregion

/// <summary>
/// This method is used to configure the indicator and is called once before any bar data is loaded.
/// </summary>
protected override void Initialize()
{
CalculateOnBarClose = true;
Overlay = true;


now = DateTime.Today; // today date
prevDay = now.AddDays(-1); // yesterday date
prev2Day = now.AddDays(-2);
prev3Day = now.AddDays(-3);
....


}

/// <summary>
/// Called on each bar update event (incoming tick)
/// </summary>
protected override void OnBarUpdate()
{

// do one (firsttickofbar) for each live bar (bar date = today date) only
//
if (FirstTickOfBar && Time[0].Date == now)
{

// Print("open 0 "+Open[0]+ " close 1 " +Close[1] + " open 1 "+Open[1] + "Close 2 "+Close[2] + " open 2 "+Open[2]);
// do calculation
.....

} // end first tick



} // end on bar update

...


Reply With Quote
The following user says Thank You to cory for this post:
 
  #15 (permalink)
Elite Member
London, UK
 
Futures Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker/Data: IB/IQFeed
Favorite Futures: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 470 given, 778 received

Thanks Cory for that. I see what you mean but I am doing it the way it is for the very reason that I can do that. I need to seriously think about whether I want to carry on that way. I also found the main culprit so if I can't fix it, I'll be into major re-design territory.

Now got a smaller screen resolution to make the resource manager work and no page faults. I might have to get more powerful reading glasses though.


ratfink View Post
Tests including dumb prints on small sets, eliminations and and binary chop style commenting out, your mission, should you choose to accept it, starts now.

I should employ Dilbert Jayavarman via freelancer.com to help.

Dumb prints are my forte - no worries there - although I suspect the eliminations I've managed so far owe more to the curry I had last night

Binary chop style commenting out sounds like an art form.

My experiments so far revealed some fairly embarrassing dog food but nothing that would go further than a peer code review if I was gainfully employed plus I've learnt too much about C#

It seems that .NET linq allows you to do all sorts of cool stuff but it runs like a snail. In all my C# Ninjascripting naivety I just wanted to find out if 2 lists of objects are the same, so I did this:

 
Code
IEnumerable<SRLevel> firstNotSecond = mergedLevelsList.Except(
	otherMergedLevels);
IEnumerable<SRLevel> secondNotFirst = 
	otherMergedLevels.Except(mergedLevelsList);
if (firstNotSecond.Count() > 0) return false;
if (secondNotFirst.Count() > 0) return false;
return true;
That alone would have been alright but every now and again the routine needs to go over info from all the bars - which is exactly what everyone here was warning about.


I might be able to send back my laptop cooler when this is sorted.

You can discover what your enemy fears most by observing the means he uses to frighten you.
Reply With Quote
The following user says Thank You to Adamus for this post:
 
  #16 (permalink)
Elite Member
London, UK
 
Futures Experience: Beginner
Platform: NinjaTrader, home-grown Java
Broker/Data: IB/IQFeed
Favorite Futures: EUR/USD
 
Adamus's Avatar
 
Posts: 1,085 since Dec 2010
Thanks: 470 given, 778 received

Plot() performance

I have achieved great performance with the OnBarUpdate after re-coding some of the loops I was doing, but I am still not satisfied with the performance of the Plot()

Can anybody who's been there give me a faster way of doing this:

'level' is my simple data class containing the prices to draw at and the bars to display on. I keep them in a list for each bar in a dictionary indexed by bar.

 
Code
for (int i = 0; i < BarsArray[0].Count; i++) //loop over all bars
{
    if (i < ChartControl.FirstBarPainted) continue;
    if (i > this.LastBarIndexPainted) continue;
    List<SRLevel> levels = srLevels[i];
    for (int x = 0; x < levels.Count; x++)
    {
        SRLevel level = levels[x];
        int leftX = ChartControl.GetXByBarIdx(BarsArray[0], i);
        int rightX = ChartControl.GetXByBarIdx(BarsArray[0], i + 1);
        int width = rightX - leftX; 
        // higher price is lower Y - Y origin is top
        int startY = ChartControl.GetYByValue(this, level.topY) - displayMargin; 
        int stopY;
        if (level.topY.Compare(level.bottomY, PRECISION) == 0)
            stopY = startY + displayMargin;
        else
            stopY = ChartControl.GetYByValue(this, level.bottomY) + displayMargin; 
        int height = stopY - startY;
        if (height < 1) height = 1;
        int labelY = 0; 
        //LastBarPainted only references primary dataseries
        if (level.behaviour == SRLevelBehaviour.SUPPORT)
        {
            brush.Color = supportColor;
            labelY = stopY + textFont.Height / 2; // labelY is for text
        }
        else if (level.behaviour == SRLevelBehaviour.RESISTANCE)
        {
            brush.Color = resistanceColor;
            labelY = startY - textFont.Height;
        }  
        graphics.FillRectangle(brush, leftX, startY, width, height);
    }
}
It seems ChartControl.GetXByBarIdx() and GetYByValue() are costly operations. Is there any other way of doing this?

You can discover what your enemy fears most by observing the means he uses to frighten you.

Last edited by Adamus; December 5th, 2013 at 10:24 AM. Reason: added some more code to make it hopefully more readable
Reply With Quote
 
  #17 (permalink)
Market Wizard
virginia
 
Futures Experience: Intermediate
Platform: ninja
Favorite Futures: ES
 
cory's Avatar
 
Posts: 5,215 since Jun 2009
Thanks: 628 given, 6,312 received

ninja performance tip
Last Historical Bar? - NinjaTrader Support Forum

Reply With Quote
The following user says Thank You to cory for this post:
 
  #18 (permalink)
Elite Member
@ Germany
 
Futures Experience: Beginner
Platform: NinjaTrader
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: FDAX
 
Posts: 439 since Nov 2011
Thanks: 254 given, 368 received


Adamus View Post
...
It seems ChartControl.GetXByBarIdx() and GetYByValue() are costly operations. Is there any other way of doing this?

@Adamus: Nice coding so far. You can test that you don't call the Get.. functions every time. If you call it 2 times for 2 consecutive IDXs then you can interpolate the other values. You can also check if "List<SRLevel> levels = srLevels[i];" is copying your list (worse) - there is no need for this, work with srlevels[i] directly.

Reply With Quote
 
  #19 (permalink)
Elite Member
Cedar Rapids, iowa
 
Futures Experience: Intermediate
Platform: Ninjatrader
Broker/Data: Ninjatrader - Continuum
Favorite Futures: 6E, TF, 6J
 
Tasker_182's Avatar
 
Posts: 626 since Aug 2009
Thanks: 426 given, 1,126 received


cory View Post
see if this help

Hey Cory,

Your example references FirstTickOfBar and appears to be CalcOnbarclose=true. According to the ninja reference this would not work.

"10.4.3.26 FirstTickOfBar
Definition
Indicates if the incoming tick is the first tick of a new bar. This property is only of value in
scripts that run tick by tick which is when the CalculateOnBarClose property is set to false."

Did I miss something?

Regards,

"Money is better than poverty, if only for financial reasons." - Woody Allen
Reply With Quote
 
  #20 (permalink)
Market Wizard
virginia
 
Futures Experience: Intermediate
Platform: ninja
Favorite Futures: ES
 
cory's Avatar
 
Posts: 5,215 since Jun 2009
Thanks: 628 given, 6,312 received



Tasker_182 View Post
Hey Cory,

Your example references FirstTickOfBar and appears to be CalcOnbarclose=true. According to the ninja reference this would not work.

"10.4.3.26 FirstTickOfBar
Definition
Indicates if the incoming tick is the first tick of a new bar. This property is only of value in
scripts that run tick by tick which is when the CalculateOnBarClose property is set to false."

Did I miss something?

Regards,

use FirstTickOfBar so you don't have to set CalcOnbarclose=true.

Reply With Quote

Reply



futures io > > > > > Indicator performance bottleneck

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Performance impact of calling multiple instances of an external indicator Big Mike NinjaTrader Programming 12 June 27th, 2015 04:47 AM
Strategy Performance JamesCBO NinjaTrader 11 October 20th, 2013 05:48 AM
How do you see the overnight performance? arnie Traders Hideout 3 January 28th, 2013 11:26 PM
Relative Performance Indicator soulartist NinjaTrader 4 November 22nd, 2010 10:21 PM
Performance Indicator MXASJ NinjaTrader Programming 2 April 16th, 2010 01:13 AM


All times are GMT -4. The time now is 11:04 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-11 in 0.16 seconds with 20 queries on phoenix via your IP 54.145.16.43