A lot of code separation and cleaning. Now you can use the FileManager in VS project with zero reference to Ninja Trader assemblies.
GomRecorderIndicator :
This version provides futures merging capability, …
Changelog : All A lot of code separation and cleaning. Now you can use the FileManager in VS project with zero reference to Ninja Trader assemblies.
GomRecorderIndicator :
This version provides futures merging capability, meaning
it will automagically load ES 12-10.xxx files before ES 03-11 rollback date, and ES 09-12, ES 06-12 etc...
it will obey Ninja merging settings, and apply continuous offsets as needed. So you get exact prices either using DoNotMerge, MergeBackAdjusted and MergeNonBackAdjusted
This feature is embedded in the Recorder, so if you implement a new DataManager you will benefit from it
GomCD :
Corrected the size of the bars.
Corrected the size filter.
Converter
Added parameter to set file mode when used from command line.
GomCD
ComCDHA
GomDataManager(not an indicator)
GomDataManagerNinjaBridge(not an indicator)
GomDeltaIndicator(do not instantiate)
GomDeltaMomentum
GomDeltaVolume
GomFileManager(not an indicator)
GomFileManagerNinjaBridge (not an indicator)
GomRecorderIndicator
Installation of this package should provide smooth upgrade from 1.3b or 2.x
GomFileConverter.exe.zip : the converter GomFileConverterProject.zip : the converter project GomDataManagerSample.cs : if you want to study how to implement a data manager. Do not install otherwise.
2.1 versions of Test and Misc packages can be found here ; these packages haven't been changed.
* corrected problem on QCollector esignal
* now millisecond time stamps sent real time as well
* added a three-state status when recording : OK,KO and NotNeeded. With the new version, the write file is not opened until there is a tick to write. So at first status is NotNeeded, then after the first tick we get OK or KO. This avoids seeing OK when actually the recorder isn't recording.
*Changed the file format and recording mode to Parameters instead of Settings category, so now they've gone up in the properties. The advantage is that you can select them when you do a composed indicator, like EMA of GomCD., or EMA of any Recorder based indicator. You can now choose format and mode. As a result dropped GomCDMA.
*you can instantiate GomCDs using
gcd = GomCD(GomCDCalculationModeType.BidAsk, GomCDChartType.CumulativeChart, "Binary", Gom.FileModeType.OnePerDay, GomFilterModeType.None, 1, false);
Q: I did run into a problem with the Millisec recording format. I could not call historic info from either the single file or the day file of recorded data.
A: You can't populate millisec with replay
Q: I also ran into a problem with the GomCD that came with 2.1. In non cumulative mode, the bars appear to be are a little wacky. I am attaching a screen shot. Bars are not respecting the zero line.
A: This has always been the case. Bars are HiLo bars, the open is set to 0, so depending on what the delta does, the hi and the low will be above/below the zero line. You can check this by disabling the ForceHiLo mode (see chart)
Q: what's up with the white bars? is this a new feature?
A: Always been the case as well. In this case the open of the bar=close of bar so color is neutral color. Again, you can check this by disabling ForceHiLo mode
Q: Using market replay data from yesterday, I replayed full screens of data. I then refreshed one of two identical charts recording binary data. Results were disappointing. This recorder appears to be having difficulty handling the faster moving markets.
A: Please check this is not a case of ninja modifying timestamps.
To test, export the ninja historical data, put the TFxx.Last.txt in your gom folder and rename it TFXX.Ninja.txt.
Then set your indicators to use the ninja export file.
If the result is better, most certainly this is the case of ticks not having the same timestamp in OnMarketData and in the Ninja database.
For more info on this and Ninja tick dropping problem, check here : Does NinjaTrader drop ticks real-time ? - Page 2 - NinjaTrader Support Forum
Here's a new release.
Modifications :
* corrected problem on QCollector esignal
* now millisecond time stamps sent real time as well
* added a three-state status when recording : OK,KO and NotNeeded. With the new version, the write file is not opened …
Here is the changelog:
Package is NOT compatible with 6.5
*Added a recording mode setting that allows to record one file per day, or one big file (like 1.3b mode). Default is to create one file per day, which allows simpler data exchange and file maintenance. Files are split at 00:00 GMT
*Added the Millisec format, where Time[0] is replaced by DateTime.Now, so obviously will only record correctly real-time. The recorder will try to find the offset between the computer and the Exchange and show it on the plots.
Millisec indicators are Millisec file format (binary), and Flat (make sure you select Use Millisec)
Note on Millisec indicators : the millisec format is only useful if :
*you use millisec designed indicators
*your network jitter is small : your lag with the exchange must remain constant. Check the indicated lag on the chart, or for a visual analysis you can check GomTimeMeasure here : [COLOR=#000000]Exchange Data Delay Test[/COLOR] . It will give a histogram of your network lag. For millisec to be any useful, your network lag must stay constant, or move a few millisecs. If it moves around the mean value about 500 ms, the millisec format is pretty useless, because you're not recording with millisec precision.
*The recorder is now able to read IR, QCollector, QCollectorIQ and Ninja export files directly, in read-only mode
*Remember you MUST enable Write Data to true for one indicator per instrument, or recording won't happen.
*As in 1.3b, the recorder can't write date before the last known tick. In 1.3b the only solution was to delete the whole data file and run replays to repopulate, now with 2.0 you only have to delete the file corresponding to the date you want to repopulate. Careful : files are split at 00:00 GMT
* added a three-state status when recording : OK,KO and NotNeeded. With the new version, the write file is not opened until there is a tick to write. So at first status is NotNeeded, then after the first tick we get OK or KO. This avoids seeing OK when actually the recorder isn't recording.
*Changed the file format and recording mode to Parameters instead of Settings category, so now they've gone up in the properties. The advantage is that you can select them when you do a composed indicator, like EMA of GomCD., or EMA of any Recorder based indicator. You can now choose format and mode. As a result dropped GomCDMA.
*You can instantiate GomCDs using
gcd = GomCD(GomCDCalculationModeType.BidAsk, GomCDChartType.CumulativeChart, "Binary",Gom.FileModeType.OnePerDay, GomFilterModeType.None, 1, false);
*Added a GomOnStartUp and GomOnTermination for those wishing access to NT7's OnStartUp and OnTermination
*Added OnMarketData done to allow better performance.
When OnBarUpdate triggers, the following happens :
Recorder calls GomOnBarUpdate
Recorder sends ticks through GomOnMarketData
Recorder calls GomOnBarUpdateDone
So to have best performance, initialize your internal counters on OnBarUpdate, process them internally in GomOnMarketData and set values to the time series in GomOnMarketDataDone.
*There is now a "standard" method to be read ticks: GomOnMarketData(Gom.MarketDataType e) and access e.Time, e.Volume, e.TickType and e.Price
* Added "Volume Split" setting, that will split ticks on constant volume charts. It works on historical (gom file) and real time data. Disable it if you do any kind of volume filtering.
* Improved the tick rendering in "non split" mode, to minimize difference between non-split volume and the constant volume series NT maintains.
* Rewrote tick rendering engine, so that now Gom Volume is 100% the same as Volume[0] on second,tick,volume and range charts. In history and real time. With CalculateOnBarClose=true or false.
* NT7 "new tick model" with OnMarketData firing before OnBarUpdate is now OK.
* Dropped the "compress ticks" mode that was pretty much useless and made the recording functions a real pain to write, so simplified the recording methods as well
*Improved overall performance
*Changed session handling so there shouldn't be problems with multiple session templates
Implements Zondor's "Delta Completion" idea, meaning that when the ticktype is unknown or betweenbidask, we use updowntick mode to assign a delta. So each tick has a delta, and when you sum buy delta and sell delta, you get the total volume of the bar. This setting can be changed (It's called delta completion)
First package to upgrade from 1.3. Basicalley upgrade GomRecorder and GomCD If you come from 1.3b you must run this first Caution : the new recorder setting is to write 1 file per day ; if you want to use your existing big 1.3 file, you must set Recording Mode to "SingleFile" instead of "OnePerDay"
GomDataManager : Data Abstraction Layer
GomFileManager : A class implementing the DataManager; defines the file formats
GomRecorderIndicator : Instantiates a GomDataManager class to use a data provider
GomDeltaIndicator: provides delta calculations
GomPrice: shows the Gom price
GomVol : shows Gom volume to be able to compare to Volume[0]
GomVolDiff: shows Diff and Cumulative diff of Gom Volume and Volume [0]
I also include GomDataManagerSample, a class that will create a new Data Provider for the recorder as an example.
If placed in the Indicator directory, a new "Sample" format will appear, that sends 1 tick per second.
This indicators uses new properties that will automagically appear in any GomRecorder indy property box.
For convenience purposes I've decided to move all GomRecorder indicator binaries to here.
Reminder: GomRecorder indicators are developped using the GomRecorderIndicator framework and allow developers to access an emulated OnMarketData() on historical bars.
EDIT : 1.3b restores NT7 functionality for GOMCDHA and GOMCDMA
Attached Thumbnails
Click image for larger version Name: gompackage1.2.png Views: 661 Size: 77.4 KB ID: 10341
Attached Files
File Type: zip GomDeltaBarsType1.0.zip (2.0 KB, 601 views)
File Type: zip GomFileConverter1.1.zip (13.6 KB, 449 views)
File Type: zip GomPackage1.1.zip (27.2 KB, 522 views)
File Type: zip GomPackage1.3b.zip (31.3 KB, 905 views)
Last edited by gomifromparis; 03-28-2010 at 08:02 AM.
GomRecorderIndicator : the base indicator.
The purpose of this indicator is to allow the use of a pseudo OnMarketData on historical data.
To achieve this, a file containing ticks is recorded and reread during historical bar construction.
WARNING : on opening of the file, GomRecorded checks what is the last date recorded, and it will only record data after this date. So "backfill" using replay files is not possible.
* There are 3, which are, from more verbose to less verbose : Flat, Short and Binary.
* Flat format is the only really human-readable format.
* You can convert from one format to another using the file converter (more on this later)
* DateTimes are recorded using UTC time zone, allowing to share files between users
The file contains the ticks with time, volume, and a value to representing where the tick happened in the current bid/ask context.
The algorithm is :
Code:
if (ask<bid) // should not happen but does
{
if (price<ask) tickType=TickTypeEnum.BelowBid;
else if (price==ask) tickType=TickTypeEnum.AtAsk;
else if (price<bid) tickType=TickTypeEnum.BetweenBidAsk;
else if (price==bid) tickType=TickTypeEnum.AtBid;
else tickType=TickTypeEnum.AboveAsk;
}
else if (bid<ask) //normal case
{
if (price<bid) tickType=TickTypeEnum.BelowBid;
else if (price==bid) tickType=TickTypeEnum.AtBid;
else if (price<ask) tickType=TickTypeEnum.BetweenBidAsk;
else if (price==ask) tickType=TickTypeEnum.AtAsk;
else tickType=TickTypeEnum.AboveAsk;
}
else //bid==ask, should not happen
{
if (price<bid) tickType=TickTypeEnum.BelowBid;
else if (price>ask) tickType=TickTypeEnum.AboveAsk;
else tickType=tickType=TickTypeEnum.BetweenBidAsk;
}
* Time granularity is the second.
* If "compress ticks" is set, all ticks occurring at the same second and with the same ticktype are aggregated on one tick with the sum of all aggregated ticks volume.
* This makes the file smaller but ruins the ability to do volume filtering on the indicators
* Files are stored in My Documents folder, unless you set the GOMFOLDER environment variable, which will be used then.
Creat a New "User variable" called GOMFOLDER. The "Value" will equal the path of the folder where the Gom data files will be stored ( ie. C:\GOMFOLDER or C:\User\????\MyDocuments\GomData ). See How to Add, Remove or Edit Environment variables in Windows 7? for further instructions.
* Files are named Instrumentname.FileFormat.txt or .dat
* When Time filtering is on, the recorder doesn't send tick to the indicator outside of session hours.
* If it is off, all ticks are use, so in all the AH ticks are sent to the first bar of the session.
* Only one of the Recorder derived indicators per instrument is allowed to write in the file. You must indicate which one it is by setting the write flag to true.
* If the indicator is unable to gain write access, "Recording KO" is showed. Usually typing F5 solves the case and a "Recording OK" should appear.
* The RecorderIndicator shouldn't need to be instantiated on a chart, except as a "recording-only" indicator.
* As the file has a 1 second resolution, there can be a problem on bars that are less than a second long. This can easily happen : for instance, bars that have their volume split by NT are 0 second long.
* The recorder will try to split volume and ticks on constant volume and tick charts, so that all volume is not sent on only one bar.
[hide][top]How do I create a GomRecorder Indicator :
see GomVWAP for an example.
* Basically you must derive your indicator from GomRecorderIndicator instead of Indicator.
* Then you must use methods GomInitialize(), GomOnBarUpdate() and GomOnMarketData(TickTypeEnum tickType,double price,int volume) instead of standard NT methods, and you're done.
* If you need to access the exact time of the tick that is sent, you can use GomOnMarketDataWithTime(DateTime tickTime, TickTypeEnum tickType,double price,int volume,bool firstTickOfBar)
* All indicators will inheritate all the GomRecorder properties.
GomVWAP
* Simple example of GomRecorder indicator.
* VWAP is reset on the start of a new session.
GomVolumeProfile
* This indicator is a GomRecorder modded version of the NT VolumeProfile.
* If Reinit session is set to true, the values are reset on each session break.
* You can instantiate one indicator with Reinit Session set to true, and another indicator with Reinit Session set to false with different colors, and in that way you can see both full chart volume and daily volume on the same chart.
Last edited by gomifromparis; 12-06-2009 at 01:33 PM.
This indicator is used to plot Delta (volume signed +/- depending if is considered buying volume(+) or selling volume(-).
The data is plotted in the same format than the main price data : Candlesticks, OHLC and HiLo are supported.
Calculation Mode
* BidAsk : if (tick >= ask) the volume is buying ; if (tick <= bid) the volume is selling
* UpDownTick : if (tick > previous tick) volume is buying ; if (tick < previous) tick volume is selling
* UpDownTickWithConinuation : if (tick > previous tick) volume is buying and any further volume@tick is buying ; if (tick < previous)tick volume is selling and any further volume@tick is selling
Delta Calculation
* CumulativeChart : Delta values are summed bar after bar
* NonCumulativeChart : Delta is reset on the beginning of each bar.
Reinit on session break
* Delta is set to 0 on the beginning of a new session.
* A Zero Line will be drawn
Size Filter Mode
* None: all ticks are used
* OnlyLargerThan : only ticks with volume strictly larger than Volume Filter Size will be used
* OnlySmallerThan : only ticks with volume strictly smaller than Volume Filter Size will be used.
* Size filtering requires that tick compression is disabled.
Bid/Ask Backup mode
* If the GomCD is incomplete, you can complete it with some NT extracted data with the file converter.
* This data doesn't contain bid/ask info, but you can tell the GomCD thas if no Bid/Ask info is available, it must fallback to using UpDownTick or UpDownTickWithContinuation.
CandleSticks : Enhance HiLo Bars
This setting doubles the size of the HiLo bar of the candlestick when in candlestick mode.
CandleSticks : Show Outline
Enables or disables the drawing of the candlestick outline when in candlestick mode
Force HiLo for non cumulative
When in non cumulative mode, forces the GomCD format to HiLo to avoid drawing glitches.
Forced HiLo Bar size
When in forced HiLo mode, determines the width of the bars.
Paint Type
* Determines the color of GomCD bars
* None : All bars painted with bar outline color
* UpDown : if (Close > Open) Color is UpColor ; if (Close < Open) Color is DownColor
* StrongUpDown : if (Close > High[1]) Color is UpColor ; else if (Close < Low[1]) Color is DownColor ; else Color is bar outline color
Exposed Dataseries
The GomCD exposes 5 DataSeries for use in other indicators : DeltaClose, DeltaOpen, DeltaHigh, DeltaLow, DeltaValue(=DeltaClose)
GomCDSMA
* This is an example of how to use the GomCD values in other indicators : here it is an SMA using the GomCD DeltaClose time series.
* The SMA can be overlayed over a CD chart
GomCDHA (Heikin Ashi)
* This code is an adaptation from ModHA indicator.
* It's used to plot a Heinkin Ashi version of GomCD
Last edited by gomifromparis; 12-02-2009 at 11:13 AM.
GomDeltaVolume
This indicator is another representation of the delta. It is not cumulative.
Delta Mode
* True : Shows Abs(Buying Volume - Selling Volume). Resulting bar is green or red depending on Sign(Buying Volume - Selling Volume)
* False : Shows Buying and Selling volume separately, with positive green bars and negative red bars
Show total volume :
If True, shows total volume of the bar as a blue bar. Mostly useless if Delta Mode is False.
GomDeltaBarType
This indicator will create a new Bar Type (Volume, Time , Range etc) called GomDelta that will construct bars at a constant delta.
Of course you can't calculate the bid/ask delta in the historical bar construction since bid/ask is not available, so UpDownTickWithContinuation model is used as a proxy.
Purpose of this stand alone app is to convert data to GomRecorder format.
It requires the 3.5 Framwork because it uses some Time Zone conversion classes.
Possible uses include:
* Converting Binary data to Flat for editing purposes
* Converting Flat to Binary for size purposes
* Converting NT, IRT and QCollector files to GomRecorder format for backfill purposes.
Input File Format :
Flat, Short, Binary :
Those are GomRecorder Files
Ninja :
Allows converting Ninja export format to GomRecorder format. Bid/Ask won't be available, obviously, but you can use UpTick option in GomCD as a fallback
IRT :
Investor R/T format
Collector :
Qcollector format. I use this weekly to get a clean file using Qcollector.
Then I add it to the previous week file using binary add in Windows :
copy /bin file1.dat+file2.dat file3.dat
Format is DateTime;Price;Volume;Bid;Ask and looks like this :
30/11/2009 22:30:26;1093;3;;
30/11/2009 22:30:26;;;0;1093
30/11/2009 22:30:27;;;1092,75;0
Input File Culture, InputFile TZ :
Use this if you're converting data that comes from a computer in another country
Compress tick :
Activates tick compression. Don't use if you're planning to do volume size filtering, as some ticks will be aggregated.
Tick size:
Needed for short and binary format as prices aren't recorded as floats but as a number of ticks.
This article is a stub. Please edit the article to improve it and add additional details.
Images will be added later.
Formatting is inconsistent. Please help.
Here's a new release.
Modifications :
* corrected problem on QCollector esignal
* now millisecond time stamps sent real time as well
* added a three-state status when recording : OK,KO and NotNeeded. With the new version, the write file is not opened …
