Global Dictionary

February 26th, 2015, 11:21 PM

Hello Everyone:

I'm attempting to export multiple data points (say, EMA 20, EMA 5) from multiple tick charts (say, ES 6 tick, ES 8 tick) to be processed into a receiving chart (say, ES 2 tick) and am having trouble doing this. The previous forum post on a similar query ("Global Dictionary sender and receiver" started in Jul 2011) included very helpful links by Fredrick Braden. However, it discussed exporting multiple data points from ONE tick chart but I wish to do so with more than one.

At the moment:
Eg 1. my test code has succeeded in exporting, importing, and plotting multiple data points from one chart into another. This uses a hard coded GD name. (code below)
Eg 2. When I attempt to so with a more generic GD name (say, "ES-6"), TS plots 0.00 in the receiving graph. The difference between Eg1 and Eg2 is in the Receiver code “Method void ItemChg”. In Eg1, I’m using “EMA10” and “EMA05” in the cases. However, if I do this in Eg2, TS gives me a “null” error. In Eg2, at the moment, I’ve “ES_6” and “ES_8” which are accepted by TS but the output being plotted is 0.00. (code below)

I would have attached the TS code file and screenshots of the output but as a new user, I'm unable to do so. So, I've embedded the code of the routines. If any seasoned user can help me resolve the issue, I would greatly appreciate your assistance.

Thank you!

======================
Eg 1 - Sender Code
---------------------------------------
using elsystem ;
using elsystem.collections;

Input:
AvgLong ( 20 ),
AvgShort ( 05) ;

variables:
Intrabarpersist double EMA10 ( 0 ),
Intrabarpersist double EMA05 ( 0 ),
string GDictName ( "myGDictName"),
GlobalDictionary myGDict ( null );

Method void UpdateGDict ( )
Begin
myGDict.Items [ "EMA10" ] = EMA10;
myGDict.Items [ "EMA05" ] = EMA05;
End;

Method void PlotOutput ()
Begin
EMA10 = Xaverage(close, AvgLong);
EMA05 = Xaverage(close, AvgShort);
Plot1(EMA10, "EMA 10");
Plot2(EMA05, "EMA 05");
Plot3(GDictName, "Name");
End;

Once Begin
myGDict = GlobalDictionary.Create ( true, GDictName);
End;

PlotOutput ( );
UpdateGDict ( );

======================
Eg 1 - Receiver Code
---------------------------------------
using elsystem ;
using elsystem.collections;

variables:
Intrabarpersist double EMA10 ( 0 ),
Intrabarpersist double EMA05 ( 0 ),
string GDictName ( "myGDictName" ),
GlobalDictionary myGDict ( null );

Method void ItemChg ( elsystem.Object Sender, ItemProcessedEventArgs Args)
Begin
Switch ( Args.Key )
Begin
Case "EMA10": EMA10 = myGDict.Items [ Args.Key ] astype double;
Case "EMA05": EMA05 = myGDict.Items [ Args.Key ] astype double;
End;
PlotOutput ( ) ;
End;

Method void PlotOutput ( )
Begin
Plot1(EMA10, "E Slow");
Plot2(EMA05, "E Fast");
End;

Once Begin
myGDict = GlobalDictionary.Create (true, GDictName);
myGDict.ItemChanged += ItemChg;
End;

======================
Eg 2 - Sender Code
---------------------------------------
using elsystem ;
using elsystem.collections;

Input:
AvgLong ( 20 ),
AvgShort ( 05 ),
Key ("ES_6");

variables:
Intrabarpersist double EMA10 ( 0 ),
Intrabarpersist double EMA05 ( 0 ),
GlobalDictionary myGD1 ( null),
GlobalDictionary myGD2 ( null);

Method void UpdateGDict ( )
Begin
myGD1.Items [ "EMA10" ] = EMA10;
myGD2.Items [ "EMA05" ] = EMA05;
End;

Method void PlotOutput ()
Begin
EMA10 = Xaverage(close, AvgLong);
EMA05 = Xaverage(close, AvgShort);
Plot1(EMA10, "EMA 10");
Plot2(EMA05, "EMA 05");
Plot3(Key, "name");
End;

Once Begin
myGD1 = GlobalDictionary.Create ( false, Key) ;
myGD2 = GlobalDictionary.Create ( false, Key) ;
End;

PlotOutput ( );
UpdateGDict ( );

======================
Eg 2 - Receiver Code
---------------------------------------
using elsystem ;
using elsystem.collections;

variables:
string Key1 ( "ES_6" ) ,
string Key2 ( "ES_8" ),
Intrabarpersist double EMA10_6 ( 0 ),
Intrabarpersist double EMA05_6 ( 0 ),
Intrabarpersist double EMA10_8 ( 0 ),
Intrabarpersist double EMA05_8 ( 0 ),
GlobalDictionary myGD1 ( null ),
GlobalDictionary myGD2 ( null );

Method void ItemChg ( elsystem.Object Sender, ItemProcessedEventArgs Args)
Begin
Switch ( Args.Key )
Begin
Case Key1:
Begin
EMA10_6 = myGD1.Items [ Args.Key ] astype double;
EMA05_6 = myGD2.Items [ Args.Key ] astype double;
End;
Case Key2:
Begin
EMA10_6 = myGD1.Items [ Args.Key ] astype double;
EMA05_6 = myGD2.Items [ Args.Key ] astype double;
End;
End;
PlotOutput ( ) ;
End;

Method void PlotOutput ( )
Begin
Plot1(EMA10_6, " E10 - 6");
Plot2(EMA05_6, " E05 - 6");
Plot3(EMA10_8, " E10 - 8");
Plot4(EMA05_8, " E05 - 8");
End;

Once Begin
myGD1 = GlobalDictionary.Create (false, Key1);
myGD1.ItemChanged += ItemChg;
myGD2 = GlobalDictionary.Create (false, Key2);
myGD2.ItemChanged += ItemChg;
End;

February 26th, 2015, 11:52 PM

There is a glitch in the matrix.

In Eg 2, you are creating 2 dictionaries with the same key and if this analysis technique is applied twice with 2 different keys, you will end up with both myGD1 and myGD2 containing a reference to the same dictionary:

Code

Once Begin
myGD1	=	GlobalDictionary.Create ( false, Key)	;
myGD2	=	GlobalDictionary.Create ( false, Key)	;
End;

My guess is that you are collecting the values of 2 EMAs from 2 different charts so you should be creating only one dictionary here. Then, once this is applied to 2 charts with different keys, there would be 2 dictionaries created, one for each key and each containing 2 EMA values.

Then in the receiving code also, I think the switch statement is switching over the dictionary key names and not the actual dictionary key values.

Code

Method void ItemChg ( elsystem.Object Sender, ItemProcessedEventArgs Args)
Begin
Switch ( Args.Key )
Begin
Case Key1:	
Begin 
EMA10_6	=	myGD1.Items [ Args.Key ] astype double;
EMA05_6	=	myGD2.Items [ Args.Key ] astype double;
End;
Case Key2:	
Begin 
EMA10_6	=	myGD1.Items [ Args.Key ] astype double;
EMA05_6	=	myGD2.Items [ Args.Key ] astype double;
End;	
End;
PlotOutput ( ) ;
End;

Here, Args.Key is being compared to Key1 and Key2 which would be "ES_6" and "ES_8". I think you should be comparing the keys with "EMA10" and "EMA05" instead. Plus, I would separate the event handler for both dictionaries into 2 different methods just so I know which dictionary's change event I am handling. The way you have it, both events will be handled by the same code and you might be retrieving values from one of the GDs when they have not really changed.

On another note, you could achieve all this with just 1 dictionary by using different keys like ES_6-EMA10, ES_6-EMA05, ES_8-EMA10, ES_8-EMA05. Have one sender chart populate 2 key values for ES_6 and the other one for ES_8.

Good luck.

February 27th, 2015, 09:56 PM

Hulk

On another note, you could achieve all this with just 1 dictionary by using different keys like ES_6-EMA10, ES_6-EMA05, ES_8-EMA10, ES_8-EMA05. Have one sender chart populate 2 key values for ES_6 and the other one for ES_8.

Good luck.

Hello Hulk:

Thank you for your suggestion! It worked.
While I've tested and optimized my strategy on a single time period (say, ES_6 or ES_8), now, as I seek to combine the signals from multiple periods, I just realized that Sender/Receiver functions only generate signals going forward. Is there any way I can get them to send/receive historical data for back testing purposes? I'm trying to replicate TS's built in Strategy Optimization on multiple tick periods.

Thank you in advance!

February 27th, 2015, 11:14 PM

DrQuantTrader

Hello Hulk:

Thank you for your suggestion! It worked.
While I've tested and optimized my strategy on a single time period (say, ES_6 or ES_8), now, as I seek to combine the signals from multiple periods, I just realized that Sender/Receiver functions only generate signals going forward. Is there any way I can get them to send/receive historical data for back testing purposes? I'm trying to replicate TS's built in Strategy Optimization on multiple tick periods.

Thank you in advance!

You can use ADE (All-Data-Everywhere). You will find this on the TS forum. There are limitations though. It works best on time based charts. There is a way of using it on tick charts also but it is tricky and depends upon your exact needs how well it will work for you.

Basically, the way it works is that, you store your "sender" data into a shared collection of collections (its very robust in that way) and you retrieve these values from elsewhere. The storage is in maps (dictionaries). The retrieval program has logic built in to return the "nearest" key data and it is pretty accurate if the keys are logically created. For instance, if you store data from a 1 min sender chart and retrieve it in a 5 minute chart, it will know which 1 minute data to return. And vice-versa too. But for tick charts, it becomes tricky. The good thing is that you can decide what the keys are made up of in the sender program.

This can be used on historical data. It is pretty self explanatory if you know data structures and the TS wiki has a lot of information on how to best use it for your needs.

Depending upon how experienced you are and how much time you have allocated for this, you can build your own external storage mechanism for historical data. This can be done by integrating TS with your own .NET program and communicate back and forth via WCF. Take a look at my old journal for how this can be done (

Hulk's trading journal

First hurdle cleared. I have managed to push data from TS to The Hulk, although at this time, it is a stateless call. I managed to pass the date/time, bar identifier and the close of each bar through a COM wrapper to a C# program. The C# program concatenated …

)

Once you get into this and need help, let me know.

February 28th, 2015, 01:11 AM

Hulk

You can use ADE (All-Data-Everywhere). You will find this on the TS forum. There are limitations though. It works best on time based charts. There is a way of using it on tick charts also but it is tricky and depends upon your exact needs how well it will work for you.

Ok. I'll do some research on this. Thank you!

Hulk

Depending upon how experienced you are and how much time you have allocated for this, you can build your own external storage mechanism for historical data. This can be done by integrating TS with your own .NET program and communicate back and forth via WCF. Take a look at my old journal for how this can be done ([link removed])

Once you get into this and need help, let me know.

Unfortunately, the post seems to be an Elite post and I'm unable to access it. Would you be able to email me the article?

February 28th, 2015, 04:41 PM

DrQuantTrader

Unfortunately, the post seems to be an Elite post and I'm unable to access it. Would you be able to email me the article?

I sent you a PM with relevant information from this post I was referring to.

Also wanted to offer you an alternate method. Since the global dictionary is nothing buy a key/value pair, you could use it to do what you want without externalizing anything by using it as a dictionary of dictionaries (or a multi map).

For simplicity's sake, lets assume that your sender charts are both 1 minute charts. You run your sender EL code by applying it to the sender charts first. For sender chart 1, you enter a key of "SC_1" into your GD and the value for this key is a regular dictionary object. This inner dictionary object can contain a key/value pair for each bar in this chart. Do the same thing for sender chart 2 and call that key "SC_2" in the GD. Then in the receiver chart, you can access the values stored by each chart by first accessing the GD with key "SC_1" to get a map of all the values stored by the first sender chart and the the second sender chart by using "SC_2".

You will just have to make sure that the GD is populated by the sender before the receiver attempts to access it. Also, the keys will have to be known before hand so the receiver chart knows what to retrieve for each bar. Its much easier if all charts are time based charts but for tick charts, it becomes tricky and you have to come up with a defined mechanism for the sender keys.

Its worth a shot I think.

Good luck.

February 28th, 2015, 05:17 PM

Hulk

I sent you a PM with relevant information from this post I was referring to.

Thank you, Hulk! Yes, I've received it. It's a lot of information and I'm slowly processing it.

Hulk

Also wanted to offer you an alternate method. Since the global dictionary is nothing buy a key/value pair, you could use it to do what you want without externalizing anything by using it as a dictionary of dictionaries (or a multi map).

At the moment, I'm researching ADE which might be the cleanest way of storing historical data for back-testing. Of course, as you've mentioned, most of the discussion is on time charts. Once I've understood how ADE works, I'll experiment with tick charts.

March 4th, 2015, 03:33 PM

Hulk

Since the global dictionary is nothing buy a key/value pair, you could use it to do what you want without externalizing anything by using it as a dictionary of dictionaries (or a multi map). ...
You will just have to make sure that the GD is populated by the sender before the receiver attempts to access it.

Hulk,
In order to backtest the strategy, I'm developing the ADE code in the TypeZeroSync framework. I'm a bit uncertain about:

If every individual sender chart will have its own text/class file (e.g., ADE.Usefile_ES_8, ADE.Usefile_ES_6 etc)?
If yes, to the above, then, how will the start of every file be synced during send/receive processes?
How is intrabar data update permitted/stopped during backtesting?
For backtesting, I presume, all sending charts will be started serially; once all data is stored in text files, then, the receiving chart will be activated for trading?

Do you have access to any sample code which discusses data transmission between multiple charts at the same time?

March 4th, 2015, 07:54 PM

DrQuantTrader

[*] If every individual sender chart will have its own text/class file (e.g., ADE.Usefile_ES_8, ADE.Usefile_ES_6 etc)?

I didnt have ADE use files. I instead just had it save sender data in memory using the default mechanism. In this case, an "ELC Server" (ELC stands for EL Collections) starts up in the background and holds all data stored by senders in collections in memory. So the sender code would look something like this:

Code

Inputs:
	ADEClassName("ES_6");

Vars:
	Class(ADEClassName),
	double vEma10(0),
	double vEma05(0),
	MyMap(MapSN.New),
	Interval(ADE.Daily); // This will probably change in your case

	//... do calcs here for ema etc

	Value1 = MapSN.Put(MyMap, "EMA10", vEma10);
	Value1 = MapSN.Put(MyMap, "EMA05", vEma05);
	Value1 = ADE.PutBarInfo(Class, GetSymbolName, Interval, ADE.BarID, MyMap);

DrQuantTrader

[*] If yes, to the above, then, how will the start of every file be synced during send/receive processes?

There are some quirks even when storing into memory. As you work with ADE, you should be able to work around those. But, in general, you dont have to worry about syncing.

DrQuantTrader

[*] How is intrabar data update permitted/stopped during backtesting?

Manual backtesting or by using strategies? If using strategies then I dont know. I actually havent used ADE for auto backtesting but my guess is that if data exists in a collection and the strategy requests it, then it should have it. It should not behave any differently than an indicator requesting stored data.

DrQuantTrader

[*] For backtesting, I presume, all sending charts will be started serially; once all data is stored in text files, then, the receiving chart will be activated for trading?

Not sure I totally understand this one since I am not sure if you are referring to manual or auto backtesting. Once you load your code into the sender charts, the ADE.PutBarInfo call stores data into ELC collections and then your other code in the receiver charts would call ADE.GetBarInfo to retrieve the stored data. Like I said, there are some quirks getting started with this but as you dig in, you will figure them out.

DrQuantTrader

Do you have access to any sample code which discusses data transmission between multiple charts at the same time?

Not really but its no different than data transmission from 1 chart to another. if you take the code snippet I posted above as an example, you can apply that code to 2 sender charts with different class names (ADEClassName input) and retrieve them one by one in your receiver chart in a single piece of code. So your receiver code would look something like this:

Code

Inputs:
	ADE1ClassName("ES_6"),
	ADE2ClassName("ES_8");

Vars:
	Class1(ADE1ClassName),
	Class2(ADE2ClassName),
	double v1Ema10(0),
	double v1Ema05(0),
	double v2Ema10(0),
	double v2Ema05(0),
	MyMap1(MapSN.New),
	MyMap2(MapSN.New),
	Interval1(ADE.Daily),
	Interval2(ADE.Daily),

	Value1 = ADE.GetBarInfo(Class1, GetSymbolName, Interval1, ADE.BarID, MyMap1); //Get the first sender data map
	Value1 = ADE.GetBarInfo(Class2, GetSymbolName, Interval2, ADE.BarID, MyMap2); //Get the second sender data map
	// the calls above will fail with an error if your sender code hasnt created maps named "ES_6" and "ES_8" yet.

	v1Ema10 = MapSN.Get(MyMap1, "EMA10"); // Get the emas from the first map
	v1Ema05 = MapSN.Get(MyMap1, "EMA05");
	v2Ema10 = MapSN.Get(MyMap2, "EMA10"); // Get the emas from the second map
	v2Ema05 = MapSN.Get(MyMap2, "EMA05");

	// the calls above will not fail with an error but the variable values will be null if data for this bar does not exist in the maps
	
	// now you can plot them or do whatever you want

This sample is for time based charts and I havent checked for syntax and other errors but hopefully it should give you a better idea of how this is supposed to work.

Global Dictionary

Discussion in EasyLanguage Programming

Global Dictionary