Ninjatrader Threading issue with Custom columns, can anyone help? - futures io
futures io futures trading



Ninjatrader Threading issue with Custom columns, can anyone help?


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one KhaosTrader with 8 posts (2 thanks)
    2. looks_two rleplae with 4 posts (0 thanks)
    3. looks_3 Zondor with 2 posts (1 thanks)
    4. looks_4 Quick Summary with 1 posts (0 thanks)
    1. trending_up 2,336 views
    2. thumb_up 5 thanks given
    3. group 3 followers
    1. forum 15 posts
    2. attach_file 0 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Ninjatrader Threading issue with Custom columns, can anyone help?

(login for full post details)
  #1 (permalink)
San Jose
 
Experience: Intermediate
Platform: NinjaTrader, Esignal
Trading: Stocks
 
Posts: 107 since Jan 2012
Thanks: 40 given, 21 received

Hi,

I am writing a custom column, where use my own indicator.

When I grab data from my indicator, I get a ThreadingLockRecursionException.

I get the data as follows...

Suggestions Please...

Thank you in advance...

 
Code
protected override void OnMarketData(MarketDataEventArgs marketDataUpdate)
		{

			
				if (!IsFirstTickOfBar)
				{
					return;
				}
			
			
			enuColumnAlertAction ColumnAlertAction = enuColumnAlertAction.None;
			enuTradeDirection TradeDirection = enuTradeDirection.Unknown;
			

		
			double InfoVal = Khaos_QuantumCalc.Market_Analyzer_Setup_Info[1]; // <-- problem

Started this thread Reply With Quote

Can you help answer these questions
from other members on futures io?
Automatic pattern research with NT
Elite Algorithmic NinjaTrader Trading
Neurological explanations for trading Success
Psychology and Money Management
Which kind of Blockvolume is visible? - order processing
Emini and Emicro Index
What blockvolume is visible? - order processing
NinjaTrader
NT8 indicator to simulate a Funding Evaluation where to …
NinjaTrader
 
 
(login for full post details)
  #3 (permalink)
Pretoria Gauteng
 
Experience: Intermediate
Platform: Sierra Charts
Trading: NQ
 
Popsicle's Avatar
 
Posts: 249 since May 2016
Thanks: 2,426 given, 545 received


Hi @KhaosTrader,

Try providing a lock for the indicator call you are making. At the top of the class (or anywhere outside a method) declare a variable that you can use as a lock:

 
Code
private static object _syncLock = new object();
Then change this line of code:

 
Code
double InfoVal = Khaos_QuantumCalc.Market_Analyzer_Setup_Info[1];
to:

 
Code
double InfoVal = 0.0d;
lock(_syncLock)
{
      InfoVal = Khaos_QuantumCalc.Market_Analyzer_Setup_Info[1];
}
//Use the value in the InfoVal variable here....
This will ensure that only one thread at a time can call that piece of code. Please just note that this may have a bit of a performance impact on your code as it effectively creates a "one call at a time queue" basically into that call. It does correctly ensure that only one thread at a time calls the code though.

If this does not fix the issue, you might have code in your indicator that recursively calls itself which is a totally different beast to solve.

Hope this helps,
Popsicle

Follow me on Twitter Reply With Quote
The following 2 users say Thank You to Popsicle for this post:
 
(login for full post details)
  #4 (permalink)
Gits (Hooglede) Belgium
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 2,991 since Sep 2013
Thanks: 2,437 given, 5,801 received


KhaosTrader View Post
Hi,

I am writing a custom column, where use my own indicator.

When I grab data from my indicator, I get a ThreadingLockRecursionException.

I get the data as follows...

Suggestions Please...

Thank you in advance...

 
Code
protected override void OnMarketData(MarketDataEventArgs marketDataUpdate)
		{

			
				if (!IsFirstTickOfBar)
				{
					return;
				}
			
			
			enuColumnAlertAction ColumnAlertAction = enuColumnAlertAction.None;
			enuTradeDirection TradeDirection = enuTradeDirection.Unknown;
			

		
			double InfoVal = Khaos_QuantumCalc.Market_Analyzer_Setup_Info[1]; // <-- problem

Probably there is (already) a lock in that code ?

Can you dig deeper ?

what is behind Khaos_QuantumCalc.Market_Analyzer_Setup_Info[1] ?

Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #5 (permalink)
San Jose
 
Experience: Intermediate
Platform: NinjaTrader, Esignal
Trading: Stocks
 
Posts: 107 since Jan 2012
Thanks: 40 given, 21 received

Hi,

Thank you guys for your help..

I didnt put any locks in any of my code prior to this point...

I added the locks, and I still got errors, but not as many errors...

Here is my modified code...

I put the following line outside the scope of the method...

 
Code
private static object _syncLock = new object();
And here is the modified code in the method...

 
Code
		protected override void OnMarketData(MarketDataEventArgs marketDataUpdate)

		{

	if (!IsFirstTickOfBar)
			{
				return;
			}
			
			if (CurrentBar < 300)
			{
				return;
			}
			
				
			int InfoValue = 0;
			int SqueezeState = 0;
		lock(_syncLock)
		{
			 InfoValue =  (int) Khaos_QuantumCalc.Market_Analyzer_Setup_Info[0];
			SqueezeState = (int)Khaos_QuantumCalc.Market_Analyzer_Squeeze_Info[0];
		}

Started this thread Reply With Quote
 
(login for full post details)
  #6 (permalink)
Gits (Hooglede) Belgium
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 2,991 since Sep 2013
Thanks: 2,437 given, 5,801 received

I don't think it's a good idea to put a lock and serialize the 'on market data' event
this will create a single bottle neck in your system

If you read your indicator, that does not come from OnMarketData anyway

It would be good to know, what is happening when you are reading the value
of your indicator... and why there is a conflict

Normaly (according to my understanding)
you need a lock, when two threads are accessing the same data
and more precisely, if at least one of the two threads is changing
the data. If two thread are 'reading' the data, that is not a concurrency
issue.

Here i think you are doing something in the code for 'reading' your indicator
i would be interested to see your 'getter' piece of code, what happens there..

Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #7 (permalink)
San Jose
 
Experience: Intermediate
Platform: NinjaTrader, Esignal
Trading: Stocks
 
Posts: 107 since Jan 2012
Thanks: 40 given, 21 received

Hi rleplae,

Thank you for your helping.. I really appreciate it.

The code calls my indicator that is quite complex, it datamines various bar patterns and uses all sorts of intelligent filtering to come up with setups. I like to use it to trade forex, and I basically have about 17 or so pairs I trade. I have the custom column on the market analyzer such that it will act as a scanner and tell me how many bars ago the setup was, what direction the setup is, and a rating of the setup.

The cool thing is I can put in several columns with different time frames, thereby having my scanner show me my setups on dozens of charts, and its really quite handy.

Started this thread Reply With Quote
 
(login for full post details)
  #8 (permalink)
Gits (Hooglede) Belgium
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 2,991 since Sep 2013
Thanks: 2,437 given, 5,801 received


KhaosTrader View Post
Hi rleplae,

Thank you for your helping.. I really appreciate it.

The code calls my indicator that is quite complex, it datamines various bar patterns and uses all sorts of intelligent filtering to come up with setups. I like to use it to trade forex, and I basically have about 17 or so pairs I trade. I have the custom column on the market analyzer such that it will act as a scanner and tell me how many bars ago the setup was, what direction the setup is, and a rating of the setup.

The cool thing is I can put in several columns with different time frames, thereby having my scanner show me my setups on dozens of charts, and its really quite handy.

Something must be there in that code...

Just on another direction, if you only have one single column for one single instrument, are you also running into the same issue ? or is it only, when multiple columns for multiple instruments are updating simultaneously ?

Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #9 (permalink)
San Jose
 
Experience: Intermediate
Platform: NinjaTrader, Esignal
Trading: Stocks
 
Posts: 107 since Jan 2012
Thanks: 40 given, 21 received

i checked it with one column with 18 instruments... I didnt check with 1 column with 1 instrument, should I try that? it might take a while for the problem to come up , if it will come up with just 1 instrument...

Started this thread Reply With Quote
 
(login for full post details)
  #10 (permalink)
Gits (Hooglede) Belgium
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 2,991 since Sep 2013
Thanks: 2,437 given, 5,801 received



KhaosTrader View Post
i checked it with one column with 18 instruments... I didnt check with 1 column with 1 instrument, should I try that? it might take a while for the problem to come up , if it will come up with just 1 instrument...

if it never comes up with 1 column 1 instrument
then it's the indicator that is not thread safe

it's worth doing the test to eliminate that possibility

Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #11 (permalink)
San Jose
 
Experience: Intermediate
Platform: NinjaTrader, Esignal
Trading: Stocks
 
Posts: 107 since Jan 2012
Thanks: 40 given, 21 received

Ok I will test this overnight on Forex 5 minute interval and get back to you.. thank you


Sent from my iPad using futures.io futures trading

Started this thread Reply With Quote
The following user says Thank You to KhaosTrader for this post:
 
(login for full post details)
  #12 (permalink)
Portland Oregon, United States
 
Experience: Beginner
Platform: Ninjatrader®
Broker: CQG, Kinetick
Trading: Gameplay Klownbine® Trading of Globex
 
Zondor's Avatar
 
Posts: 1,347 since Jul 2009
Thanks: 1,245 given, 2,721 received

Try this, if it works it should eliminate the need for the lock.



 
Code
        ....   variable declarations...

        private Khaos_QuantumCalc KQC;


        ...     OnStateChange....

        if(State==State.Configure)
               {   KQC=Khaos_QuantumCalc();

               }


        protected override void OnMarketData(MarketDataEventArgs marketDataUpdate)

		{

	if (!IsFirstTickOfBar || CurrentBar < 300) return;			{
				
			
	lock(_syncLock)
        	{   InfoValue =    (int) KQC.Market_Analyzer_Setup_Info[0];
		    SqueezeState = (int) KQC.Market_Analyzer_Squeeze_Info[0];
		}

"If we don't loosen up some money, this sucker is going down." -GW Bush, 2008
“Lack of proof that something is true does not prove that it is not true - when you want to believe.” -Humpty Dumpty, 2014
“The greatest shortcoming of the human race is our inability to understand the exponential function.”
Prof. Albert Bartlett
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to Zondor for this post:
 
(login for full post details)
  #13 (permalink)
San Jose
 
Experience: Intermediate
Platform: NinjaTrader, Esignal
Trading: Stocks
 
Posts: 107 since Jan 2012
Thanks: 40 given, 21 received

Hi Zondor,

I have it this way... and still the problem, I am not sure if I am doing it where i am using the "init" concept in your referenced post.. but here is my code..

please if you have another idea or if i am not employing your suggestion correctly, let me know how to do it.

Thank you..

Code Below:

Declare

 
Code
public class KhaosQuantum : MarketAnalyzerColumn
	{
		
		private Khaos_Quantum_Calculator Khaos_QuantumCalc;
...
...
Reference

 
Code
		else if (State == State.Configure)
			{
		Khaos_QuantumCalc = Khaos_Quantum_Calculator
				(
					9, // BarsLength
					20, // MA Short Length
					50, // MA Medium Length
					200, // MA Long Length
....
....

Method Logic

 
Code
		protected override void OnMarketData(MarketDataEventArgs marketDataUpdate)

		{
			

				
			
			if (!IsFirstTickOfBar || CurrentBar < 300)
			{
				return;
			}
			
				
			int InfoValue = 0;
			int SqueezeState = 0;
		
		lock(_syncLock)
		{
			 InfoValue =  (int) Khaos_QuantumCalc.Market_Analyzer_Setup_Info[0];
			SqueezeState = (int)Khaos_QuantumCalc.Market_Analyzer_Squeeze_Info[0];
		}

...
...

*** Also note, it seems that when I click reload ninjascript, I really get errors, I just put into testing on a 10 second interval with about 18 or so instruments, when I hit reload ninjascript, and while it is already reloadingi again, many errors occur... Here is the log info:

More info....

It seems if I press the "refresh" on the Market Analyzer it triggers these errors...

Here is the log file...
2016-10-18 14:02:56:851|2|4|Session Break (Version 8.0.0.14)
2016-10-18 14:02:59:038|1|4|Verifying license at primary server...
2016-10-18 14:02:59:308|1|4|Type=Simulation State=Verified Start Date=12/31/2004 End Date=11/30/2099
2016-10-18 14:02:59:418|1|4|Global simulation mode enabled
2016-10-18 14:03:03:511|1|4|Vendor assembly 'System.Speech' version='4.0.0.0' loaded.
2016-10-18 14:03:04:319|1|2|Using IS (is-us-003.ninjatrader.com/31658)
2016-10-18 14:03:04:357|1|4|Automated trading disabled
2016-10-18 14:03:04:358|1|4|Auto connecting 'My FXCM'...
2016-10-18 14:03:04:401|1|2|My FXCM: Primary connection=Connecting, Price feed=Connecting
2016-10-18 14:03:04:651|1|32|Order entry hot keys disabled
2016-10-18 14:03:04:653|1|4|Auto close enabled=False
2016-10-18 14:03:12:147|1|2|My FXCM: Primary connection=Connected, Price feed=Connected
2016-10-18 14:03:12:227|1|2|Time to auto close position='00:00:00', Enabled=False
2016-10-18 14:03:12:228|1|2|Using HDS (hds-us-003.ninjatrader.com/31655)
2016-10-18 14:05:54:630|3|4|Failed to call 'Add' method: System.Threading.LockRecursionException: Write lock may not be acquired with read lock held. This pattern is prone to deadlocks. Please ensure that read locks are released before taking a write lock. If an upgrade is necessary, use an upgrade lock in place of the read lock.
at System.Threading.ReaderWriterLockSlim.TryEnterWriteLockCore(TimeoutTracker timeout)
at System.Threading.ReaderWriterLockSlim.TryEnterWriteLock(TimeoutTracker timeout)
at NinjaTrader.Data.BarsSeries.Add(Bars bars, Double open, Double high, Double low, Double close, DateTime time, Int64 volume, Double tickSize, Boolean isBar, Double bid, Double ask)
2016-10-18 14:31:57:361|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:421|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:472|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:537|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:622|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:628|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: You are accessing an index with a value that is invalid since it is out-of-range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.
2016-10-18 14:31:57:698|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:723|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:773|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:816|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:844|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:921|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:57:977|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:58:010|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:58:095|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: Object reference not set to an instance of an object.
2016-10-18 14:31:58:095|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: You are accessing an index with a value that is invalid since it is out-of-range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.
2016-10-18 14:31:58:095|3|4|Market analyzer column 'KhaosQuantum': Error on calling 'OnMarketData' method on bar -1: You are accessing an index with a value that is invalid since it is out-of-range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.

Started this thread Reply With Quote
 
(login for full post details)
  #14 (permalink)
San Jose
 
Experience: Intermediate
Platform: NinjaTrader, Esignal
Trading: Stocks
 
Posts: 107 since Jan 2012
Thanks: 40 given, 21 received

I tried with 1 instrument, error comes up.

What I am going to do is put in an indicator column and just have that work for me.

Started this thread Reply With Quote
The following user says Thank You to KhaosTrader for this post:
 
(login for full post details)
  #15 (permalink)
Portland Oregon, United States
 
Experience: Beginner
Platform: Ninjatrader®
Broker: CQG, Kinetick
Trading: Gameplay Klownbine® Trading of Globex
 
Zondor's Avatar
 
Posts: 1,347 since Jul 2009
Thanks: 1,245 given, 2,721 received

You did not make the change I suggested, and don't seem to understand what I said.

I don't think that this is a threading issue.

The indicator crashes because it does not have error handling code. If you trapped the errors, it would probably work.

Refer to a C# tutorial to learn about error handling.

To eliminate the Object Reference exceptions, as a first step add this condition to beginning of OnMarketData and OnBarUpdate methods:

if(CurrentBar == -1) return;

If that doesn't get rid of them, something else is wrong - just like it says, failure to have set an object reference to an instance of an object..

That's about all I can offer based on the information you posted since the problems are in code not revealed..

"If we don't loosen up some money, this sucker is going down." -GW Bush, 2008
“Lack of proof that something is true does not prove that it is not true - when you want to believe.” -Humpty Dumpty, 2014
“The greatest shortcoming of the human race is our inability to understand the exponential function.”
Prof. Albert Bartlett
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #16 (permalink)
San Jose
 
Experience: Intermediate
Platform: NinjaTrader, Esignal
Trading: Stocks
 
Posts: 107 since Jan 2012
Thanks: 40 given, 21 received

Hi Zandor,

---> I did do a try catch block and posted the results follow up on ninjatrader forum. Eventually ninjatrader suggested I make a new indicator that calls the main indicator, and just use an indicator column.

With the suggested approach, it works fine.

In the ninjatrader form thread, the support person said this

" KhaosTrader,

Thank you for your patience.

To sum it up, even if the custom column has the thread of the instrument and indicator locked we still don't have the bars loaded. We could make a bars request but those bars would not be associated with the indicator and thus we would really be re-writing the entire indicator into the column (which may be an approach you wish to take).

However, I am going to make a feature request to allow for Series<T> types to be the value that is used from the Indicator Column Plot option as well. This would resolve your entire scenario as you could just have an indicator that looks for the conditions and sets a Series<string> to the desired string for the column.

"



The thread is located here:

Custom Column Question - NinjaTrader Support Forum

As you can see Ninjatrader is thinking about allowing the return of a string value for a custom indicator column. That would be excellent if they do that.

Thank you for your patience and help Zandor.

Started this thread Reply With Quote


futures io Trading Community Platforms and Indicators NinjaTrader > Ninjatrader Threading issue with Custom columns, can anyone help?


Last Updated on October 21, 2016


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
 

Battlestations! Show us your trading desk - $1,500 in prizes!

March
 

Importance of Finding Your Own Way w/Adam Grimes

Elite only
 

Journal Challenge w/Jigsaw

April
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts