Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
I wrote an indicator to display support and resistance levels and a few other types of horizontal lines on the chart. It's got pretty complex but that's not to say that it's necessarily a pile of spaghetti - I've started from scratch on a couple of occasions when I've found that the design I had was no longer working.
At this point the indicator that does everything required of it finally, after months of work, but it's got a bad performance problem in that the time to process each historical bar creeps up steadily, making it a pain to load more than about 1500 bars (less 3 days on a 3-min chart), which is stifling my ability to use the indicator to do historical research on the setups I use it for.
I log the time each OnBarUpdate takes. It starts off at 20 millisecs for each OBU and varies around 20 ms, but every 10 or 20 bars it increases by roughly 10 ms. After about 1000 bars it gets up to over 100 ms per OBU. That's like a second for every ten bars, so another day (480 bars) takes almost a minute and then longer and longer.
Despite knowing the indi inside out, I don't really know where to look to see what is causing the problem. I'm not a C# expert so I don't know what the expensive operations are in the code. What foxes me is that I can see from my logging that the indicator can have nothing more to do from one bar to the next over perhaps 100 bars, but the performance degrades anyway.
A few of the things I'm doing that might be causing problems are:
using several lists and searching them with delegate/predicate methods
using dictionaries in the same way
instantiating new objects a lot
these objects are based on a class with 2 subclasses, so lots of inheritance going on
using multiple time-frames
pre-loading data myself with code using GetBars() (this is quick but I thought it might have unknown side-effects)
That list represents my ignorance of C# rather than my insightful experience. Here's what it's doing:
I can post more info, data, logs, whatever - if any C# coders out there have come across this sort of unidentifiable bottle-neck before, I'd love to hear about it and perhaps get some tips on where to investigate.
You can discover what your enemy fears most by observing the means he uses to frighten you.
Can you help answer these questions from other members on NexusFi?
Just sounds like you need to work out a lot more clearly what it needs to do, rather than what it is doing. I can offer speed-up tips, but to be honest you want to be in the micro-second ranges at worst anyway, so I'm guessing at this stage it's orders of magnitude too slow and clearly running geometrically or exponentially so needs a major redesign. All the bullet points you mention are found in many unwieldy OO implementations, I suggest you do some pondering and look for some KISS approaches instead.
And check for obvious no-no's like redrawing every Ninja rectangle from the beginning of time on every tick and such like....
OK I can see you come from the kill it fast and kill it hard school of software development
However the Plot is fine. I've got good durations for each call to Plot(). It's just historical bar processing.
In my defence though, the complexity of the indicator is necessary to keep the visual display on the chart simple, e.g. if I've got 5 lines across 10 points, it'll merge them so you see one zone on the chart, not 5 clustered lines. But I will put my mind to it on how to KISS it. I don't want to be guilty of trying to justify a pile of horseshit.
Another thing I just discovered: the bar update duration drops back down to sub-millisecond levels once it starts processing live bars.
Why? What is the difference between historical and live bars OnBarUpdates?
You can discover what your enemy fears most by observing the means he uses to frighten you.
@Adamus It's impossible to help you out without looking at the source codes. You can check if your code iterates thru the whole set of historical bars at new bar arrival (include some function loop counter). Have you used the debugger to resolve your problems?
You bet. In 40 years I have met many writers of correct software but, since OO and Ghz CPUs, a lot less writers of (correct && efficient) software.
On the other hand you may just be unlucky at this stage and have a simple gotcha in there that just needs hunting down. It sounds like you know what you are doing, maybe just need to dig deeper. My first thought was that you are exponentially processing all bars on every tick or something, but I'm only guessing at a distance so can't really help that much.
There are several, but one that comes to mind is the drawing of off-screen graphical artefacts, Ninja doesn't seem too good at clipping, e.g. particularly of rectangles/triangles, which is why I mentioned them. You can speed up a lot of things by not drawing unnecessarily, but it is always worth proving what is taking the time first just by commenting out calls, etc. Lots of red-herrings to find and eliminate when you really start to run a speed-up exercise! It can always be our own code or just unrealistic expectations of somebody else's.
Just another quick point, getting things up into the milli-second rather than micro-second levels usually takes memory or even hard page faults to become an issue, have you eliminated that side of things? (i.e. the TM is showing (100/number-of-cores)% flat-out for Ninja and no hard faults so it is all cpu). Would be unusual but not unknown, just depends what you are doing.
Hi Koepisch, good comment, thanks. I shall check while I am going through adding more detailed performance monitoring.
The code runs to 2800 lines. I assume nobody would want to look at it - I can't imagine I'd want to look at somebody else's dogfood. If I'm wrong, I'd gladly attach it - just say the word.
You can discover what your enemy fears most by observing the means he uses to frighten you.
