Design a DayTrader Scalping Order Flow Indicator - futures io
futures io



Design a DayTrader Scalping Order Flow Indicator


Discussion in Traders Hideout

Updated
      Top Posters
    1. looks_one hyperscalper with 111 posts (156 thanks)
    2. looks_two Chof with 22 posts (6 thanks)
    3. looks_3 justtrader with 14 posts (3 thanks)
    4. looks_4 TWDsje with 13 posts (9 thanks)
      Best Posters
    1. looks_one bobwest with 1.5 thanks per post
    2. looks_two hyperscalper with 1.4 thanks per post
    3. looks_3 TWDsje with 0.7 thanks per post
    4. looks_4 Chof with 0.3 thanks per post
    1. trending_up 11,640 views
    2. thumb_up 196 thanks given
    3. group 35 followers
    1. forum 193 posts
    2. attach_file 64 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Design a DayTrader Scalping Order Flow Indicator

(login for full post details)
  #11 (permalink)
 hyperscalper 
boise idaho
 
Experience: Advanced
Platform: NinjaTrader C# Custom
Broker: NinjaTrader LeeLoo Rithmic
Trading: Nasdaq Futures NQ/MNQ
 
Posts: 247 since Apr 2020
Thanks: 14 given, 331 received

COMMENTS ON SIMPLE BIG LOT INTEGRATORS FOR INVENTORY ANALYSIS

Full Inventory Analysis, is a very complex and memory intensive activity, whereby
for example, each Market Maker Buy is matched with a MM Sell from which the two
matching volumes (open Inventory) are taken out of the Price Trade history.

This "matching" is the equivalent of "making the Spread" which Market Maker does
all the time, you'll recall, by always Buying the BID and always Selling the ASK/OFFER
price, thus profiting by the amount of the spread on average.

Following the 80/20 rule, or perhaps the 90/10 rule, and considering the shorter
timeframes involved; THANKFULLY a Full Inventory Analysis, which looks at Risk
factors (how much money is being Lost due to VWAP adversity) is
NOT REALLY WORTHWHILE on these shorter Accumulation / Distribution
timeframes.

This fact greatly SIMPLIFIES our task, which collapses to the much simpler
Integration of Signed Trade Volumes over a recent Window of time; a calculation
which is quite simple to accomplish.

WHAT ARE WE REALLY LOOKING FOR HERE?

OK, let's face that question head on. Intuitively, and in fact, we see that there is
a "flurry" of activity at TOPs and BOTTOMs; often referred to as the "fight" between
Buyers and Sellers. What is happening here?

1) Trade frequency per unit time increases at these prospective Tops and Bottoms, but
that is not particularly informative.

2) DIRECTIONALITY of the signed Trade Volumes increases as Retail players BUY at the
last (wrong) moment, i.e. at the Top; and THE MIX of Trade Volumes SHIFTS to contain
a much higher percentage of BIG LOT trades.

3) the reason for 2) above, is that Market Maker also moves significant SIZE on the
Book, or the DOM, the Market Depth in order to interact with this flurry of Retail
activity PREFERENTIALLY.

4) this is true because Market Maker INTENDS TO CHANGE THE PRICE TREND, and so
wants to SELL to the Retail population, as much as Possible, at as High a price as possible.
(and the opposite at a bottom...)

WE ARE THEREFORE LOOKING FOR an Acceleration of Signed Volume MM Selling Distribution at a
"top" (more Retail Buying in larger lot sizes); and the opposite at a bottom,
an Acceleration of Signed Retail Selling volume, and increases in Big Lot sizes as Retail Sellers accelerate their
Selling at the bottom, enabling MM to "scoop up" Buying volume at as low a Price as possible
before turning the Price trend.

AS A GENERAL SUMMARY, we are looking for an Acceleration in Distribution by MM Selling at Tops,
and an Acceleration in Accumulation by MM at Bottoms; using the Retail population as the
unfortunate "sheep-like" counter-party because, as we've said, the Retail population most
generally Buys when Price rises (especially if a Break Out seems possible); and Sells when
Price falls (especially if a Break Down seems probable).

IT IS THIS ACCELERATION IN "BIG LOT" *RATES* OF ACCUMULATION AND DISTRIBUTION
that we are seeking to characterize by this Indicator.

Naturally, anybody can try to punch holes in this, and it's true that perhaps NO SINGLE INDICATOR
should be relied upon; however, this effect is Universal and in the very nature of trading, so
we will see it nearly everywhere.

[EDIT] So.. IN CASE IT HASN'T BEEN MADE OBVIOUS TO THIS POINT IN TIME...
Market Maker is, in fact, TRADING AGAINST YOU, taking a Position against the
ENTIRE Retail Market, getting Long at Bottoms, and Short at Tops; so she is
your Mortal Enemy, and any clues as to her situation (e.g. Inventory imbalance)
gives you an edge.

FINAL QUESTION: What do you think results in the "Micro Sawtooth" structure of
Price movement? It should be obvious at this point that MM creates these Price
movements in order to take Long/Short positions against the entire Retail Trading
population simultaneously... And if that's not an "Aha!" moment, then what is? LOL

hyperscalper

Started this thread Reply With Quote
The following 4 users say Thank You to hyperscalper for this post:

Can you help answer these questions
from other members on futures io?
Does anyone have the code for Anchored Vwap to use in Qu …
Traders Hideout
one chart to another
NinjaTrader
Chart Comparer
NinjaTrader
What are the main Eurobond futures and their respective …
Traders Hideout
Google Keyword Search Alerts
Traders Hideout
 
 
(login for full post details)
  #12 (permalink)
 hyperscalper 
boise idaho
 
Experience: Advanced
Platform: NinjaTrader C# Custom
Broker: NinjaTrader LeeLoo Rithmic
Trading: Nasdaq Futures NQ/MNQ
 
Posts: 247 since Apr 2020
Thanks: 14 given, 331 received

PRE-RELEASE INFORMATION ON THE IMPLEMENTATION

A typical Trader using NinjaTrader 8 may not have created an Indicator. You are
out of luck... but it's not that bad if you take it "one step at a time".

The code I'll be releasing is, first of all, "kind of a hack" since I would not design
this in a single file, I would use Interfaces, I would use refactoring, and a bunch
of other things in multiple files..... and I'd do it in Visual Studio, and I'd
probably generate a DLL, .... BUT let's just not worry about all that now !!!

We have decided to do this all in a single file, which means Classes must be declared
"inline" as so-called "inner classes" and the size of the code file very quickly becomes
huge; even with the relatively basic things we're trying to do...

So I'd recommend that you dust off your existing skills or, if you're not a coder, to
regularly take time to become proficient; and just keep moving forward.

UNLESS YOU CAN IMPLEMENT YOUR OWN INDICATOR IDEAS, YOU'LL ALWAYS
BE AT THE MERCY OF SOMEONE ELSE, so Jump In... the Water's Fine

WHAT ARE THE CONCEPTS WE'RE INTRODUCING THAT WILL BE STRANGE, AND
PROBABLY "UNSUPPORTED" HERE?

NO INTERACTIVE USER INTERFACE for Indicators, means that if we do want to
interact while the Indicator is running, we have to "roll our own". We decided to
use a simple Properties file concept where the text entries on each line have the
form of "key=value"; and we have a periodic reader which will re-read that file
and assign any new values; which are then applied to our running Indicator.

WHY? We want to avoid the weirdness and bugs associated with going back to
the Properties dialog in NinjaTrader 8. Trust me, you don't want to have to deal
with that.....

ALSO WHY? Because the Dynamic range of the market during a session changes very
considerably, and we want (at a minimum) to 1) change the vertical scaling of our
output), and 2) to change the "retention interval" within our Trade Flow Integrator,
and 3) adjust our "Big Lot" minimum threshold as well.....

NinjaTrader support would say that the method being used here is "unsupported", so
don't be surprised; and just don't ask. Ask hackers, perhaps in this forum instead.
Most of you Hackers are "White Hat" hackers... just kidding... LOL

MULTIPLE ASYNCHRONOUS THREADS to implement the Properties file periodic updater.
This updater has a "loop" which executes in a completely independent "thread" of
execution, absolutely "asynchronously" to the Indicator's callbacks to the main
elements OnMarketData and OnBarUpdate, etc... The code Waits for a while,
which can be interrupted early, and then re-reads the Properties file. If you want
to investigate Thread usage like this, it's easy once you understand what's going on...

USING A TEXT EDITOR (recommended Notepad++) which you'll be able to use to
EDIT the Properties text file. When you SAVE the update, then the Indicator's
property reader loop will READ the file again, parse it, and then there's a primitive
callback into the Indicator body, which will enable the Indicator to correctly update
its internal variables. If you screw up your edits, internal exceptions should be
generated; but the system should not crash... Fingers crossed !

MULTIPLE COPIES OF THE INDICATOR presumably on multiple different symbols,
keeping in mind that I'm only focussed on NQ or MNQ contracts, but there isn't
anything inherently different in other symbols, since we're only reading the fairly
standard Time and Sales, but any problems represent an "opportunity" for you
to modify things to suit your needs... A DIFFERENT FILENAME should be used
for Properties on different symbols, or perhaps running on the SAME symbol,
but with DIFFERENT parameters.... keep that in mind.

INITIALIZATION AND DE-INITIALIZATION should not really be required here, since
we no longer need to Stop and Restart the Indicator in order to change its behavior.
NinjaTrader 8 uses what I can only describe as a "very weird" "object cloning" method
of activating Indicators, Strategy's, etc. and YOU DON'T WANT TO KNOW THE
DETAILS cuz they will cause lots of Anxiety !!! The fact that we are not having to
stop and restart the Indicator means we completely side-step those thorny issues,
and TRUST ME, they cause a lot of problems when you're allocating Objects like
we are, and potentially (as you really SHOULD) implementing your Indicator
as a DLL file..... 'Nuff said.

IT WON'T BE LONG before a strictly Alpha/Beta sample will be posted, and then
we'll find all the Bugs. I just wanted to get y'all started; and what we really want
to focus upon is the PREDICTIVE NATURE OF THE SIGNAL we're trying to generate !!!

hyperscalper

Started this thread Reply With Quote
The following 2 users say Thank You to hyperscalper for this post:
 
(login for full post details)
  #13 (permalink)
 hyperscalper 
boise idaho
 
Experience: Advanced
Platform: NinjaTrader C# Custom
Broker: NinjaTrader LeeLoo Rithmic
Trading: Nasdaq Futures NQ/MNQ
 
Posts: 247 since Apr 2020
Thanks: 14 given, 331 received


THE PROPERTIES FILE AS IT IS INITIALLY WRITTEN, however I did change
the MULTIPLIER in the Editor from 1 to 1.5

Note that it is written to the Custom folder at the path you can also see
at the top of the Notepad++ editor screenshot.

THAT IS TO SAY, it would be your personal login Documents path for NinjaTrader
down in the bin then Custom folders.

BEST PRACTICE IS: If you modify the code in the NinjaScript editor and compile; then
it's best to go to the attach Chart(s) and REMOVE the existing indicator (because it
is running the OLD one), APPLY and then ADD the Indicator again (be sure to put in
the desired File Name).

NinjaTrader uses "reflection" and also "object cloning" to retain the state of Indicators,
so best when you make code changes to "quasi-religiously" REMOVE, and then ADD again.

[EDIT] Yes, you're waiting for the code; just let me check it out first; otherwise, I'll
have Egg Yolks on my face, for sure... LOL

hyperscalper

Attached Thumbnails
Click image for larger version

Name:	Trade-flow-properties-file.PNG
Views:	88
Size:	27.7 KB
ID:	318013  
Started this thread Reply With Quote
The following 2 users say Thank You to hyperscalper for this post:
 
(login for full post details)
  #14 (permalink)
 hyperscalper 
boise idaho
 
Experience: Advanced
Platform: NinjaTrader C# Custom
Broker: NinjaTrader LeeLoo Rithmic
Trading: Nasdaq Futures NQ/MNQ
 
Posts: 247 since Apr 2020
Thanks: 14 given, 331 received

HERE IS THE INDICATOR ADDED TWICE ON SAME CHART, but with different
properties files.

Prop file 1:
# comment line
RETENTION_SECONDS=30
MULTIPLIER=1
BIGLOT_MINIMUM=2
TAPER_SIZE=True

Prop file 2:
# comment line
RETENTION_SECONDS=300
MULTIPLIER=1
BIGLOT_MINIMUM=2
TAPER_SIZE=True

So I think it can be released, with the understanding that this is strictly "as is" code.

hyperscalper

Attached Thumbnails
Click image for larger version

Name:	Trade-flow-2-retentions.PNG
Views:	115
Size:	53.2 KB
ID:	318014  
Started this thread Reply With Quote
The following 3 users say Thank You to hyperscalper for this post:
 
(login for full post details)
  #15 (permalink)
 hyperscalper 
boise idaho
 
Experience: Advanced
Platform: NinjaTrader C# Custom
Broker: NinjaTrader LeeLoo Rithmic
Trading: Nasdaq Futures NQ/MNQ
 
Posts: 247 since Apr 2020
Thanks: 14 given, 331 received

OK, ALL EXCITED? HERE'S THE CODE.

I'll attach the text of the Indicator in a ZIP file, but I'll also see how much of
it I can paste between some code tags... Here goes:

[EDIT] I'm seeing this POLL off to the right; so I guess that's just a forum thing...

 
Code
#region Using declarations
using System;
using System.IO;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Gui;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Gui.SuperDom;
using NinjaTrader.Gui.Tools;
using NinjaTrader.Data;
using NinjaTrader.NinjaScript;
using NinjaTrader.Core.FloatingPoint;
using NinjaTrader.NinjaScript.DrawingTools;
#endregion

//This namespace holds Indicators in this folder and is required. Do not change it. 
namespace NinjaTrader.NinjaScript.Indicators
{
	public class TradeFlowTutorial : Indicator
	{
		protected override void OnStateChange()
		{
			if (State == State.SetDefaults)
			{
				Description									= @"in Futures.IO an exercise in developing a simple Trade Flow analyzer.";
				Name										= "TradeFlowTutorial";
				Calculate									= Calculate.OnEachTick;
				IsOverlay									= false;
				DisplayInDataBox							= false;
				DrawOnPricePanel							= true;
				DrawHorizontalGridLines						= false;
				DrawVerticalGridLines						= false;
				PaintPriceMarkers							= true;
				ScaleJustification							= NinjaTrader.Gui.Chart.ScaleJustification.Left;
				//Disable this property if your indicator requires custom values that cumulate with each new market data event. 
				//See Help Guide for additional information.
				IsSuspendedWhileInactive					= false; // always run
				FileName					= "TradeFlowProps.txt";
			}
			else if (State == State.Configure)
			{
				indexCounter = 0; // start at zero
				// index named variables automatically assign in order
				AddPlot(new Stroke(Brushes.Black), PlotStyle.Hash, "zeroIndex"); // index
				zeroIndex = indexCounter++; // starts with zero; plot idx
				Plots[zeroIndex].Width = 2;
				
				AddPlot(new Stroke(Brushes.Purple), PlotStyle.Line, "inventory"); // index
				inventoryIndex = indexCounter++; // starts with zero; plot idx
				Plots[inventoryIndex].Width = 4;
				
				init();
			}
			else if (State == State.Realtime)
			{
				isReady = true;
			}
			else if (State == State.Terminated)
			{
				isReady = false;
				deInit();
			}
		}
		
		#region Properties
		[NinjaScriptProperty]
		[Display(Name="FileName.txt", Description="Properties file in default folder", Order=1, GroupName="Parameters")]
		public string FileName
		{ get; set; }
		#endregion

		private int indexCounter = 0;
		
		private int zeroIndex = 0; // assigned in config
		private int inventoryIndex = 0;
		
		private bool isReady = false;
		
		#region Startup and Initialization
		private void init() {
			customRootPath = Environment.ExpandEnvironmentVariables(
					"%USERPROFILE%\\Documents\\NinjaTrader 8\\bin\\Custom\\");
			myIntegrator = new VolumeExpiryIntegrator(1000, retentionSecs);
			// allocate objects
			if (myPropertyReader==null) {
				myPropertyReader = new SimplePropertyReader(this);
				myPropertyReader.setPropertiesFileNameDotTxt(FileName);
				// now assign default property values
				myPropertyReader.setKeyValue(retentionSecsProp, ""+retentionSecs); // stringify it
				myPropertyReader.setKeyValue(multiplierProp, ""+multiplier);
				myPropertyReader.setKeyValue(bigLotMinimumProp, ""+bigLotMinimum);
				myPropertyReader.setKeyValue(taperSizeProp, ""+taperSize);
				myPropertyReader.start();
			}
		}
		#endregion
		
		#region Synchronization and PROPERTY Management
		// begin of synchronization support
		// use getUpdateMutex()
        private object updateMutex = new object();
		
		private SimplePropertyReader myPropertyReader = null;

        // asynchronous property updates use this
        // synchronization mechanism
		// in Visual Studio, of course, we'd use
		// interface definitions, and multiple
		// files, but we're staying in a single file
		// for usage within the NinjaScript environment
        protected object getUpdateMutex() {
            return updateMutex; 
        }
		
		// CONTROL VARIABLES MODIFIABLE BY PROPERTY MANAGER
		private static string retentionSecsProp = "RETENTION_SECONDS";
		private int retentionSecs = 60; // default 60 seconds
		private static string multiplierProp = "MULTIPLIER";
		private double multiplier = 1.0; // default unity multiplier
		private static string bigLotMinimumProp = "BIGLOT_MINIMUM";
		private int bigLotMinimum = 2; // default unity multiplier
		private static string taperSizeProp = "TAPER_SIZE";
		private bool taperSize = true; // default useTaper
		
		protected void updatePropertiesCallback() {
			if (myPropertyReader==null) return;
			lock(getUpdateMutex()) {
				// a thread which already holds the lock
				// will always get the same lock again....
			// the file-based property manager will
			// firstly lock(getUpdateMutex())
			// and then it will call this routine
			// where your property values can
			// be updated safely
			//	myPropertyReader.setKeyValue(retentionSecsProp, retentionSecs);
			//	myPropertyReader.setKeyValue(multiplierProp, multiplier);
			//	myPropertyReader.setKeyValue(bigLotMinimumProp, bigLotMinimum);
			//if ( !myPropertyReader.hasChanged()) return;	
			try {
				int retentionSecsTmp = myPropertyReader.getIntegerProperty(retentionSecsProp);
				if (retentionSecsTmp<10) retentionSecsTmp = 10; // ten sec min
				retentionSecs = retentionSecsTmp;
				myIntegrator.setRetentionSecs(retentionSecs); // change the Integrator
			} catch (Exception ex) {} // do nothing
			try {
				double multiplierTmp = myPropertyReader.getDoubleProperty(multiplierProp);
				if (multiplierTmp<0.1) multiplierTmp = 0.1;
				multiplier = multiplierTmp;
				// used in OnBarUpdate
			} catch (Exception ex) {} // do nothing
			try {
				int bigLotMinimumTmp = myPropertyReader.getIntegerProperty(bigLotMinimumProp);
				if (bigLotMinimumTmp<1) bigLotMinimumTmp = 1;
				bigLotMinimum = bigLotMinimumTmp;
				// used when reporting Signed Trade Volumes
			} catch (Exception ex) {} // do nothing
			try {
				taperSize = myPropertyReader.getBooleanProperty(taperSizeProp);
				myIntegrator.setUseTaper(taperSize);
			} catch (Exception ex) {} // do nothing
				
				
			} // end of any property value updates
		}
		// end of synchronization support
		#endregion

		#region NinjaScript Output Window printing
		private static bool USE_NINJA_PRINT = true;
		
        private void ninjaPrint(string msg) { // primary tab
            if (!USE_NINJA_PRINT) return;
            PrintTo = PrintTo.OutputTab1;
            Print(msg);
        }

        private void ninjaPrint2(string msg) { // secondary tab
            if (!USE_NINJA_PRINT) return;
            PrintTo = PrintTo.OutputTab2;
            Print(msg);
        }
		#endregion
		
		#region The Trade Volume (tapering) Integrator
		private VolumeExpiryIntegrator myIntegrator = null;
		
		/* This version will enforce a strict moving
		time window, and optionally taper the trailing
		portion of that window gradually to zero to
		avoid artifacts associated with size leaving
		the time window*/
		protected /*inner*/ class VolumeExpiryIntegrator {
			private double[] dataArray = null;
			private DateTime[] timestampArray = null;
			// these are treated as "circular buffers"
			bool useTaperFlag = false; // TODO
			int size = 0;
			int ptr = 0;
			int _expiryMsecs = 60000;
			public VolumeExpiryIntegrator(int sizeArg, int expirySecsArg) { // constructor
				size = sizeArg;
				_expiryMsecs = 1000 * expirySecsArg;
				dataArray = new double[size];
				timestampArray = new DateTime[size];
				for(int i=0; i<size; i++) {
					dataArray[i] = 0;
					DateTime now = DateTime.UtcNow;
					timestampArray[i] = now;
				}
			}
			
			public void addData(double dataArg) {
				// RULE: ptr should always index the
				// next valid array index, not the
				// most recent one.  ptr always
				// increments.
				if (ptr>=size) {
					ptr = 0;
				}
				timestampArray[ptr] = DateTime.UtcNow;
				dataArray[ptr++] = dataArg; // increment after usage
				if (ptr>=size) {
					ptr = 0;
				}
			}
			
			public void setRetentionSecs(int seconds) {
				_expiryMsecs = seconds * 1000;
			}
			
			public void setUseTaper(bool flag) {
				useTaperFlag = flag;
			}
			
			public double getSum() {
				// must work backwards in time from most
				// recent until items are expired, or we
				// traverse the entire circular ring buffer
				DateTime now = DateTime.UtcNow; // this is REAL TIME ONLY
				double sum = 0;
				int tmpPtr = ptr - 1; // most recent
				if (tmpPtr<0) tmpPtr = size - 1;
				else if (tmpPtr>=size) tmpPtr = 0; // array bounds
				int taperBeginAge = (int)(_expiryMsecs * 0.75); // if tapering...
				bool expired = false;
				// always using tmpPtr here !
				while( !expired && tmpPtr!=ptr) {
					DateTime ts = timestampArray[tmpPtr];
					double item = dataArray[tmpPtr--];
					// REMINDER: we are moving from most recent
					// backward in time here !!
					if (tmpPtr<0) tmpPtr = size - 1; // array bounds
					TimeSpan elapsed = ( now - ts );
					int ageMsecs = (int)elapsed.TotalMilliseconds;
					if ( ageMsecs > _expiryMsecs ) {
						expired = true; continue; // and exit loop
					}
					if ( tmpPtr == ptr ) {
						// we exhaused all of our circular buffer (unlikely)
						expired = true; continue; // and exit loop
					}
					double finalItem = item;
					if (useTaperFlag) { // taper last 25% of time window
						// OK, so we'll taper the oldest 25% of the
						// time window trade volume linearly down to zero
						// e.g. Expiry might be 10000, so begin tapering beyond age 7500
						if (ageMsecs > taperBeginAge) { // e.g. from 7500 msecs into a 10000 max window
							int numerator = _expiryMsecs - ageMsecs; // progressing 2500 toward zero
							if (numerator<0) numerator = 0;
							int denominator = _expiryMsecs - taperBeginAge; // e.g. 10000 - 7500 = 2500
							double ratio = (double)numerator / (double)denominator;
							finalItem *= ratio; // LINEAR TAPER
						}
					}
					sum += finalItem; // summation of signed (tapered) trade volume
				}
				return sum;
			}
		}
		#endregion

		#region Calculations and Utilities
        private static double EXP = 0.75; // exponent less than 1 and >0.5

        // non-linear signed size compression
        private static double compressSizeMinOneSigned(double valueArg)
        {
			if ( EXP > 1 ) EXP = 1.0; // no compression at all
			if ( EXP < 0.5 ) EXP = 0.5; // square root
			bool isNegative = ( valueArg < 0 );
            double absValue = Math.Abs(valueArg); // calculate on positive
            if (absValue < 1) absValue = 1; // enforce minimum of size 1 (contract)
            double x = Math.Pow(absValue + 0.0001, EXP); // just adding a smidgeon
            // 1^0 = 1 so 1 should be the minimum possible
            if (x < 1) x = 1;
            return ( isNegative? -x : x); // restore the sign
        }
		#endregion

		#region Shutdown de-Initialization
		
		private void deInit() {
			myIntegrator = null;
			if (myPropertyReader!= null) {
				myPropertyReader.stop(); // kill the thread
				myPropertyReader = null;
			}
			// stop and deallocate objects
		}
		#endregion
		
		private static bool LOG_RAW_VOLUME = true;
		
		private int lastRawTradeVol = 0;
		
		#region Processing Trade Flow
		private void reportTrade(int tradeRawVol, double price) {
			if (Math.Abs(tradeRawVol) < bigLotMinimum) return; // reject volume
			if ( LOG_RAW_VOLUME ) {
				if (tradeRawVol>0) {
					ninjaPrint("Retail Buy "+tradeRawVol+" @"+price);
				}
				else {
					ninjaPrint("       Retail Sell "+tradeRawVol+" @"+price);
				}
			}
			lastRawTradeVol = tradeRawVol;
			double compressedVol = compressSizeMinOneSigned( tradeRawVol );
			reportToIntegrator( compressedVol );
		}
		
		private void reportToIntegrator(double compressedVolArg) {
			myIntegrator.addData(compressedVolArg);
		}
		#endregion
		
		private double lastBidPrice = 0;
		private double lastAskPrice = 0;
		private string customRootPath = null;

		#region OnMarketUpdate callback
		protected override void OnMarketData(MarketDataEventArgs marketDataUpdate)
		{
			if ( !isReady ) return;
			lock(getUpdateMutex()) { // no property updates now
				MarketDataType type = marketDataUpdate.MarketDataType;
				if (type==MarketDataType.Last) { // Trade
					if (lastBidPrice==0) return;
					if (lastAskPrice==0) return;
					if (lastAskPrice <= lastBidPrice) return;
					double tradePrice = marketDataUpdate.Price; // price of the Trade
					int tradeVolume = (int)marketDataUpdate.Volume;
					double mid = 0.5 * ( lastAskPrice + lastBidPrice ); // mid price
					if (tradePrice < mid ) {
						// trade toward BID negative Retail Sell
						reportTrade( -tradeVolume, tradePrice );
					}
					else {
						// trade toward ASK/OFFER positive Retail Buy
						reportTrade( tradeVolume, tradePrice );
					}
				}
				else if (type==MarketDataType.Bid) {
					lastBidPrice = marketDataUpdate.Price; // BID price
				}
				else if (type==MarketDataType.Ask) {
					lastAskPrice = marketDataUpdate.Price; // ASK/OFFER price
				}
			} // end lock
			return;
		}
		#endregion

		protected override void OnBarUpdate()
		{
			lock(getUpdateMutex()) { // no property updates now
				Values[zeroIndex][0] = 0;
				double inventory = myIntegrator.getSum();
				Values[inventoryIndex][0] = inventory * multiplier;
			}
		}

		#region periodic SimplePropertyReader
	
	private static bool DBG_KV = false; // debug only
    protected class SimplePropertyReader {

        // USAGE: Create object
        // setPropertiesFileNameDotTxt
        //   use name "." (default directory) \\ double backslash
        //   desiredName.txt
        // if file exists, then 
        //   the given properties will be ignored, and the
        //      properties dictionary contents taken from the file
        //   otherwise, the given properties are written to
        //   a newly created properties file
        // setPropertiesSamplingSecs
        // call start() once only
        // on termination, call stop()
		TradeFlowTutorial parent = null;
        public SimplePropertyReader(TradeFlowTutorial parentArg) {
			parent = parentArg;
            // client should set all properties, prior to start()
        }

        // please call start only once
        public void start() {
            ThreadStart ts = new ThreadStart(propertiesReaderLoop);
            Thread runner = new Thread(ts);
            isRunning = true;
            runner.Start();
        }

        private string fileName = "myProperties.txt";

        // if used, must do this BEFORE start()
        public void setPropertiesFileNameDotTxt(string fileNameArg) {
            fileName = fileNameArg;
        }

        // if used, do this BEFORE calling start();
        public void setPropertiesSamplingSecs(int secondsArg) {
            propertiesSampleDelayMsecs = secondsArg * 1000;
        }

        private bool isRunning = false;
        private int propertiesSampleDelayMsecs = 20000;

        public void stop() {
            isRunning = false;
            notifyUpdate(); // break thread wait; and exit
        }
        private object threadWaitObject = new object();
        private bool isNotifiedLocked = false;
        public void notifyUpdate() {
            lock(threadWaitObject) {
                    isNotifiedLocked = true;
                    Monitor.Pulse(threadWaitObject);
            }
        }

        private void propertiesReaderLoop() {
            try {
				parent.ninjaPrint("propertiesReaderLoop started");
                while (isRunning) {
                    try {
                        lock (threadWaitObject) {
                            if (!isNotifiedLocked) {
                                Monitor.Wait(threadWaitObject, propertiesSampleDelayMsecs, true); // every XX secs or sooner
                                isNotifiedLocked = false;
                            }
                            if (!isRunning) continue; // thread will stop
                                                      // read property file
                            if ( !updateFromPropertiesFile()) continue; // error in file or first time
                            lock(parent.getUpdateMutex()) {
                                  parent.updatePropertiesCallback();
                            }
                        }
                    }
                    catch(Exception ex) {

                    }
                }
            }
            catch(Exception ex) {
                // nothing
            }
        }

        // client must set ALL properties and values BEFORE start()
        public void setKeyValue(string keyArg, string value) {
            lock (propertiesDictionary) {
                string key = keyArg.ToUpper().Trim();
                string strValue = "" + value;
                if (propertiesDictionary.ContainsKey(key)) {
                    propertiesDictionary[key] = strValue;
                } else {
                    propertiesDictionary.Add(key, strValue);
                }
            }
        }

        private Dictionary<string, string> propertiesDictionary = new Dictionary<string, string>();

        public int getIntegerProperty( string key) {
            return (int)getDoubleProperty(key);
        }

        public double getDoubleProperty(string keyArg) {
            string key = keyArg.ToUpper().Trim();
			double retDbl = 0;
            lock (propertiesDictionary) {
                if (propertiesDictionary.ContainsKey(key)) {
					try {
						retDbl = Double.Parse(propertiesDictionary[key]);
					} catch(Exception ex) { throw; } // propogate exception to caller
				}
            }
			return retDbl;
        }

        public bool getBooleanProperty(string keyArg) {
            string key = keyArg.ToUpper().Trim();
            bool retBool = false;
            lock (propertiesDictionary) {
                if (propertiesDictionary.ContainsKey(key)) {
                    try {
                        retBool = Boolean.Parse(propertiesDictionary[key].Trim().ToLower());
                    } catch (Exception ex) { throw; } // propogate exception to caller
                }
            }
            return retBool;
        }

        private bool updateFromPropertiesFile() {
			parent.ninjaPrint("updateFromPropertiesFile called... ");
            // if the properties file doesn't exist, then we create one
            // and populate with the provided properties in form xxx=yyy
			bool successful = true;
            string fullFilePath = parent.customRootPath + fileName;
            if ( !File.Exists(fullFilePath)) { // default dir; backslash is special escape char
				try {
                // write a new one
				parent.ninjaPrint("updateFromPropertiesFile new file "+fullFilePath);
                using (StreamWriter outputFile = new StreamWriter(fullFilePath)) {
                    outputFile.WriteLine("# comment line");
                    lock (propertiesDictionary) {
                        foreach (var item in propertiesDictionary) {
                            string key = item.Key;
                            string value = item.Value;
                            outputFile.WriteLine(key + "=" + value);
                        }
                        outputFile.Close();
                    }
                }
				} catch(Exception ex) { successful = false;
					parent.ninjaPrint("Exception creating file: "+ex.Message);}
                return successful;
            }
            // existing file, need to re-read and update our properties dictionary
            if (File.Exists(fullFilePath)) { // should definitely exist !!!
                                              // write a new one
				try {
                using (StreamReader inputFile = new StreamReader(fullFilePath)) {
                    lock (propertiesDictionary) {
                        //propertiesDictionary.Clear();
                        string line = " ";
                        while ((line = inputFile.ReadLine()) != null) {
							if (line.StartsWith("#")) continue; // skip comment
                            string trimmedLine = line.Trim();
                            if (trimmedLine.Length == 0) continue;
                            if (!trimmedLine.Contains("=")) return false;
                            string[] splitString = line.Split('=');
                            string key = splitString[0].ToUpper().Trim();
                            string value = splitString[1].ToLower().Trim(); // e.g. FALSE to false
							if (DBG_KV) {
								parent.ninjaPrint("Key: "+key+" value: "+value);
							}
							if ( !propertiesDictionary.ContainsKey(key) ) {
                            	propertiesDictionary.Add(key, value);
							}
							else {
								propertiesDictionary[key] = value;
							}
                        }
                        inputFile.Close();
                    }
                }
				} catch(Exception ex2) { successful = false; 
					parent.ninjaPrint("Exception: "+ex2.Message);}
            }
			return successful;
        }
    } // end class

		#endregion
		
	}
}

#region NinjaScript generated code. Neither change nor remove.

namespace NinjaTrader.NinjaScript.Indicators
{
	public partial class Indicator : NinjaTrader.Gui.NinjaScript.IndicatorRenderBase
	{
		private TradeFlowTutorial[] cacheTradeFlowTutorial;
		public TradeFlowTutorial TradeFlowTutorial(string fileName)
		{
			return TradeFlowTutorial(Input, fileName);
		}

		public TradeFlowTutorial TradeFlowTutorial(ISeries<double> input, string fileName)
		{
			if (cacheTradeFlowTutorial != null)
				for (int idx = 0; idx < cacheTradeFlowTutorial.Length; idx++)
					if (cacheTradeFlowTutorial[idx] != null && cacheTradeFlowTutorial[idx].FileName == fileName && cacheTradeFlowTutorial[idx].EqualsInput(input))
						return cacheTradeFlowTutorial[idx];
			return CacheIndicator<TradeFlowTutorial>(new TradeFlowTutorial(){ FileName = fileName }, input, ref cacheTradeFlowTutorial);
		}
	}
}

namespace NinjaTrader.NinjaScript.MarketAnalyzerColumns
{
	public partial class MarketAnalyzerColumn : MarketAnalyzerColumnBase
	{
		public Indicators.TradeFlowTutorial TradeFlowTutorial(string fileName)
		{
			return indicator.TradeFlowTutorial(Input, fileName);
		}

		public Indicators.TradeFlowTutorial TradeFlowTutorial(ISeries<double> input , string fileName)
		{
			return indicator.TradeFlowTutorial(input, fileName);
		}
	}
}

namespace NinjaTrader.NinjaScript.Strategies
{
	public partial class Strategy : NinjaTrader.Gui.NinjaScript.StrategyRenderBase
	{
		public Indicators.TradeFlowTutorial TradeFlowTutorial(string fileName)
		{
			return indicator.TradeFlowTutorial(Input, fileName);
		}

		public Indicators.TradeFlowTutorial TradeFlowTutorial(ISeries<double> input , string fileName)
		{
			return indicator.TradeFlowTutorial(input, fileName);
		}
	}
}

#endregion
hyperscalper

Attached Files
Register to download File Type: zip TradeFlowTutorial.zip (6.7 KB, 43 views)
Started this thread Reply With Quote
The following 4 users say Thank You to hyperscalper for this post:
 
(login for full post details)
  #16 (permalink)
 hyperscalper 
boise idaho
 
Experience: Advanced
Platform: NinjaTrader C# Custom
Broker: NinjaTrader LeeLoo Rithmic
Trading: Nasdaq Futures NQ/MNQ
 
Posts: 247 since Apr 2020
Thanks: 14 given, 331 received

OK, JUST SOME FINAL COMMENTS BEFORE BUGS COME ROLLING IN.

Screenshot attached shows 2 instances of TradeFlow with different "retention" intervals.

Typically each Indicator instance is in its own Panel.

But the image shows that if you click "on the data series" (the purple line) then you
can DRAG that onto the Price chart, so you have them overlain, which I personally
find better in seeing the relationships against Price more precisely.

ENJOY !!!

[EDIT] If you had any doubt, you can see how powerfully Retail Players are SELLERS
on Price decline, by viewing the attachment... Yep, most of them are "sheeple" LOL

[EDIT2] Attached some more realtime NQ info, showing Price versus Net Inventory
Trade Flow over 5 minutes expiration window

hyperscalper

Attached Thumbnails
Click image for larger version

Name:	Trade-flow-overlay.PNG
Views:	104
Size:	65.2 KB
ID:	318019   Click image for larger version

Name:	Sellers-on-price-decline.PNG
Views:	80
Size:	46.2 KB
ID:	318021   Click image for larger version

Name:	Trade-flow-NQ.PNG
Views:	75
Size:	156.8 KB
ID:	318023  
Started this thread Reply With Quote
The following user says Thank You to hyperscalper for this post:
 
(login for full post details)
  #17 (permalink)
 hyperscalper 
boise idaho
 
Experience: Advanced
Platform: NinjaTrader C# Custom
Broker: NinjaTrader LeeLoo Rithmic
Trading: Nasdaq Futures NQ/MNQ
 
Posts: 247 since Apr 2020
Thanks: 14 given, 331 received

PERFORMANCE CONCERNS CAN BE MONITORED WITH NT8 FROM
THE NINJASCRIPT OUTPUT WINDOW, right click and select
the NinjaScript Utilization Monitor.

This will display cumulative ongoing Milliseconds relative usage across
ALL Indicators in the system.

Our little Indicator "TradeFlowTutorial" seen in the screenshot is consuming
very little relative CPU; even though it's not really highly optimized, and may
not need to be....

hyperscalper

Attached Thumbnails
Click image for larger version

Name:	NinjaScript-utilization-monitor.PNG
Views:	66
Size:	20.8 KB
ID:	318020  
Started this thread Reply With Quote
The following user says Thank You to hyperscalper for this post:
 
(login for full post details)
  #18 (permalink)
 trendisyourfriend 
Legendary Market Wizard
Quebec
 
Experience: Intermediate
Platform: NinjaTrader wt Rancho Dinero's profiling tools
Broker: AMP/CQG
Trading: ES, NQ, YM
 
trendisyourfriend's Avatar
 
Posts: 4,142 since Oct 2009
Thanks: 3,820 given, 5,384 received

What is the difference between your indicator and what Ninjatrader already has with its OrderFlowCumulativeDelta.

Using different size as filter?

Reply With Quote
The following user says Thank You to trendisyourfriend for this post:
 
(login for full post details)
  #19 (permalink)
 hyperscalper 
boise idaho
 
Experience: Advanced
Platform: NinjaTrader C# Custom
Broker: NinjaTrader LeeLoo Rithmic
Trading: Nasdaq Futures NQ/MNQ
 
Posts: 247 since Apr 2020
Thanks: 14 given, 331 received


trendisyourfriend View Post
What is the difference between your indicator and what Ninjatrader already has with its OrderFlowCumulativeDelta.

Using different size as filter?

Good question. I have no idea what NinjaTrader engineering has coded...

Yes. 1) size filter is key; and 2) a strict integration time window (with taper)
are a couple of things; including also adjustability of the retention interval
and size filter while the Indicator is running in real time... just a few things.

As I have discussed we are concerned with teasing out "Big Lot Accumulation and
Distribution patterns" and ideally the "acceleration" of those effects at Tops and
Bottoms.

I've laid out the code; but it doesn't make a concerted effort to measure Acceleration
in Accumulation or Distribution more precisely; so that's an enhancement you could
play with.

I'm wide open to hear, or see screenshots comparing our TradeFlowTutorial tool with
existing Indicators, and whether anyone has significant "insights" as to how we
can better hone in on the effects which I've outlined from Theory through to the
very basic implementation we have here...

Believe me, I wouldn't be satisfied with the basic tool outlined above... It's a tutorial
which has "simple" enough code, for the average Trader to follow, while challenging
the "standard approaches" in a few key ways...

LIVE AND LEARN... the process never stops !!

I AM VERY INTERESTED IN YOUR IDEAS AND RESPONSES; i don't really have an
"agenda" here, other than finding "trade signals" which make money... But I do very
much like to have a "Theory" behind, and a "rationale" for, the Indicators I am using,
and preferably know exactly what they are measuring...

hyperscalper

Started this thread Reply With Quote
The following 4 users say Thank You to hyperscalper for this post:
 
(login for full post details)
  #20 (permalink)
 hyperscalper 
boise idaho
 
Experience: Advanced
Platform: NinjaTrader C# Custom
Broker: NinjaTrader LeeLoo Rithmic
Trading: Nasdaq Futures NQ/MNQ
 
Posts: 247 since Apr 2020
Thanks: 14 given, 331 received


SOME THINGS TO FIDDLE WITH

The "Big Lot" size threshold is a very interesting thing to fiddle with.
You'd be surprised, when the REALLY BIG LOTS start coming through,
just how much that tends to "nail" a major Top or Bottom

The "retention interval" on the Window, especially when coupled with a
relatively high Big Lot size filter, will cause such an Indicator to "spike"
at these significant turning points.

Consider that you may want the Indicator to stay at ZERO until this "Big Lot stuff"
starts to happen....

[EDIT] Notice in the attached screenshot how the 30 second retention interval
with a 10 Big Lot threshold starts to "spike"...

[EDIT2] REMEMBER... those "spikes" represent Retail Players, and MARKET MAKER
trades AGAINST them... that's the theory anyway...

ISSUES? Sure, there are many when dealing with the Nasdaq futures market,
and its WIDE DYNAMIC RANGE, which is why the ability to TWEAK the parameters
during a Day Trading session is really required.

I HAVE FOUND with my highly custom setup; that Tops and Bottoms in
Accumulation/Distribution of Big Lots is a very good Trigger Signal. Of
course, I have all of that built into an extremely complex Triggering
system which combines a handful of simultaneous additional conditions,
against which my system, which is co-located; can quickly react...

But this simple TradeFlowTutorial Indicator is intended for more "normal"
Traders who want to have some additional confirmation of where Local
Tops and Bottoms may exist, especially if you are a Scalper...

For me, ANALYTICS are the most important thing in the world; but then once they
are "good enough" the ability to MANAGE ORDERS in line with the Analytics
"signals" may also be required, since the sheer speed and frankly the Brutality
of the Nasdaq market, especially at the open, makes Manual trading impossible.

But that's not the Trader for which I've designed that simple Indicator; but it
is something you could consider if you put that code in a Strategy module, rather
than an Indicator; and attempt to "trigger" trades based on that, and a few
other signal variables.

BUT THEREIN IS A "Rabbit Hole" which you should realize requires significant
commitment.... so I don't expect most Traders will want to jump down it !!

hyperscalper

Attached Thumbnails
Click image for larger version

Name:	Big-lot-spikes.PNG
Views:	89
Size:	52.5 KB
ID:	318026  
Started this thread Reply With Quote
The following 2 users say Thank You to hyperscalper for this post:


futures io Trading Community Traders Hideout > Design a DayTrader Scalping Order Flow Indicator


Last Updated on November 29, 2021


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada), info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts