There are three fascinating subjects related to the markets that permanently catch my mind. I want to deal with one of them here to study the main characteristics of a trend.

The three topics are

(1) Behavioural Patterns, Rationality and Irrationality

Rational and irrational behavioural patterns create the primary feedback mechanisms required to create a non-random market. Fear and greed, mental accounting, loss aversion, risk tolerance, overconfidence and anchoring are some of these patterns that every trader should understand.

(2) Game Theory

Trading is a n-person-zero-sum game - at least if you include the brokers earning the commissions from your trade as participants in the game. To trade successfully, one needs to know who wil co-operate with you and who will stand out against you. Game theory provides a framework to analyse and define potential scenarios how other participants in the game may act or react. Once a scenario is defined, it allows to identify price levels where this scenario is no longer valid and appropriate action should be taken.

(3) Nonlinear Dynamics of Complex Systems

Leon Walras' Éléments d'économie politique pure was the point of departure for equilibrium economics. By applying equations from differential calculus he condemned economic science to follow a path, which since has been contradictory to empirical evidence. There is no equilibrium in economics, but markets are inherently unstable. So Walras got it all wrong: He should have taken the mathematical formalism developped for statistical physics and thermodynamics, and economists would not have created false models for over a century. Even the Black-Scholes model is based on Gaussian distributions, which only can be used to describe random action. With Gaussian distribution you can create wonderful models, so you don't even need to chekc for empirical evidence. Economics is the still half way between astrology and science.
With Gaussian distributions there are no feedback mechanisms, no fat tails or black swans. LTCM would still be alive today and the financial crisis would never have happened.

Non-linear systems add a second layer of feedback mechanisms on the primary mechanisms based on behavioral patterns. The second layer is an endogenous property of non-linear systems. This all sounds like a dry theory, but I think it is important to know this to trade the markets.

Besides, if this sounds too far-fetched for you: In his book "The Alchemy of Finance", George Soros tries to establish his theory of reflexivity. This certainly does not merit to be considered as a theoretical framework, but it shows that George Soros has developped a deep understanding for reciprocal feedback mechanisms of markets. He actually uses different scenarios to enter his trades, and - most important - knows at what point these scenarios are no longer valid. Most of the scenarios are based on feedback mechanisms, and the book is definitely worth reading for the trades, not for its half-baked theories.

Before I start with my exploration of the trend, here is some literature on nonlinear system dynamics. There are four books that I recommend in particular:

Philip Ball: Critical Mass Eric D. Beinhocker: The Origins of Wealth
Mark Buchanan: Ubiquity (related to the subject) Benoit R. Mandelbrot: The (mis)Behavior of Markets - A Fractal View of Risk, Ruin and Reward
Paul Ormerod: Why Most Things Fail (related to the subject)
Didier Sornette: Why Stock Markets Crash (don't read this if you are not at least half a mathematician) Nassim Nicholas Taleb: The Black Swan

To be continued........

The following 7 users say Thank You to Fat Tails for this post:

The question of of determinig trends is a difficult subject, so I want to start with some examples.

The standard trend definition is well-known

Uptrend: Higher high and higher low
Downtrend: Lower low and lower high

The advantage of this definition is that it is a fractal one, not based on any continuous function such as a moving average. The inconvenience of this definition is that most ABC corrections would start a new trend, as it does not indicate whether the shift in value was temporary (correction) or longer lasting (new trend).

So let us have a look at Brent Crude today:

Chart 1 shows the fractal view: There was a shift on value on Monday. This shift occured during one hour. Afterwards the price remained in a trading range. So the fractal view shows us a two bar trend and a temporary equilibrium-like situation that lasted two days.

Chart 2 shows the continuous view of the same price action. I have added a linear regression (blue, predictive, may overshoot price) and an EMA (red, lagging).

Uptrend: Linear regression above EMA and upsloping
Downtrend: Linear regression below EMA and downsloping

This chart suggests a completely different picture compared to the first one. Which picture is true?

The following 2 users say Thank You to Fat Tails for this post:

This topic is difficult, but I will try to continue, because I believe it will give me a better understanding of the way markets work.

Moving Averages and Linear Regression

The most widely used trend filters are probably moving averages. Unfortunately a moving average only tells you what has happened in the past, moving averages lag. They are useful to enter a trend, if that trend continues, but they may as well lead you into whipsaw moves. An improvement over a moving average is a linear regression indicator which predicts a move assuming that the momentum will continue. Where a moving average is based on price, the linear regression is based on its first derivative. You can use the difference between moving average and linear regression to detect the maturity of a trend.

The main problem of using moving averages arises with parabolic or climactic moves. Moving averages will not catch or recognize the climax but still move upwards ages after the sudden move has occured. The simple moving average is the worst of all. It is like a dog that barks at the visitors (upmove), when they arrive and then barks again when a visitor leaves, as the prior values drop out of the calculation of the moving average. The reality here is a sudden up move followed by a trading range. The moving average however is gradually rising. We are reasoning within a framework of high school mathematics, where we have applied calculus to continuous functions, and this framework cannot really be applied here.

A fractal approach to trend

The best known concept is the directional movement by Welles Wilder and its application, which is the ADX. ADX is probably the most widely used indicator for filtering trends. One well known example is the "Holy Grail" of Linda Raschke. I am not explaining ADX here, but simply want to deal with the shortcomings of the concept.

The positive directional movement measures the high of the current bar against the high of the previous bar. So apply this to a reversal bar! What you will find, is that the reversal bar with a long upper tail generates a positive directional movement, where as its meaning is a short signal, because the new high was not maintained, but rejected.

Modifying the Directional Movement

We come back with this to the statistical significance of high, low and close for each data series. A simple bar hides a statistical distribution of prices, each of the prices has a probability as exposed by the market delta method. The point of control has the highest probability and high and low have the lowest probabilities. So it is an obvious weakness of the concept of directional movement that it uses high and low. Ideally you would want to use the point of control. The close or the typical price is certainly a better proxy of the point of control than the high or low!

Note: A floor pivot is a simplified formula for a supposed point of control. Today you may replace it with the VWAP of the session. If a bar represents one session, you might also have a look of that VWAP (calculated from the ticks of the bar) and take it as a reference.

To keep things simple, I will modify the directional movement and ignore the low probability highs and lows but focus on the close versus the median value of the prior bar. This has several advantages:

(1) Temporary excursions out of the range of the prior bar will not be counted, but I only measure the difference between the latest price (= close) and the midvalue of the prior range (=median).

(2) After a downbar a bar's contribution will only be counted as positive, if its close exceeds the midrrange. This implies that it has retraced the prior bar by at least 50%.

Also I want to measure participation of traders through volume, so I will multiply the difference between Close[0] and Median[1] wíth the volume.So below you will find an example of a trend indicator built from a modified directional movement. You can see that the trend does not develop in a smooth continuous way, but that it has sudden moves followed by trading ranges.

Some examples

(1) For 6E a positive trend developped shortly after NYSE opened. Trend strength reached its first maximum after 15 minutes, slowed down and regained speed. The positive trend remained intact and offered an entry for pullback traders. About 2 hours after the open the positive trend resumed. The lime bars show that the trend consisted of some "trend explosions" followed by longer periods of stagnation.

(2) For ES there was some volatility when the stock market opened, but there was no clear direction. Volatility and volume both decreased for several hours. A strong trend only developped during the last 45 minutes of the market.

(3) For CL you could notice a strong negative trend, which was repeatedly reinforced by outbursts of downward volatility which resulted in a negative shift of trading ranges. If you had used the indicator as a trendfilter, you would have certainly been on the right side.

The indicator still lacks significant levels - such as 15/25 for the ADX - that allow to determine, whether a trend is strong enough or whether price can be considered within a trading range. This can be achieved through normalization and shall be the next step of this journey.

To be continued .....

The following 3 users say Thank You to Fat Tails for this post:

There is a little simulation that shows how agent and rule based systems behave. This is the island of sugarscape. The sugarscape terrain is a two-dimensional grid comprising rows and columns of cells, which represent the unit of location for a citizen. A citizen may occupy one cell, which cannot be shared.

Cells have three attributes: Sugar, spice and pollution. These are visually identifiable on the screen. The agents' behaviour is rule based. According to their visual abilities (inherited) they will look around and try to find the richest source of sugar and harvest it. To survive and prosper they need a minimum amount of sugar (metabolism inherited) to consume. If they are rich enough, they can mate and reproduce. A further improvement of the game introduces spice and barter trades to swap sugar for spice.

Sugarscape was first presented 1996 by Axtell and Epstein in their book "Growing Artificial Societies". They have recently published a new book "Generative Social Science: Studies in Agent-Based Computational Modeling".

So what has Sugarscape to do with trading? Trading as well is a multi-agent game, and in a way comparable to Sugarscape, even if the rules that trader follow can not be compared to those of the agents of sugarscape. But, if you actually run a simulation of sugarscape, you will find many similarities with trading. There a long periods of equilibrium, where the number of agents shows only minor fluctuations. Then, suddenly the population will decline to 15% without any reason that can be discerned. Remember, the sugarscape model does not undergo any external shocks to show this behaviour, but the sudden transition form quasi equilibrium to a strong trend has endogenous causes. These are the nonlinear, positive or negative feedback mechanisms that are based on the set of rules which govern the microbehaviour of the agents. In the same way, traders react to the market by using hard rules (based on algorithms) or soft rules (based on emotions such as greed and fear).

If you want to run a simulation of Sugarscape, this is easy, because it has been implemented in Java and can be accessed through the web:

I can't blame Walras for not making use of statistical thermodynamics.. to do that, he would have needed a time machine. Most of Boltzmann's work was first published in the 1870's, and wasn't widely distributed until the 1890's. Even then, much of the physics community attacked his work until years after that (some say this contributed to his suicide). Meanwhile, Walras was publishing his work at the same time, in the early 1870's, probably doing much of the research in the years before that. In a different country, in a different field, in a different language, with no internet or anything. So...

The following user says Thank You to Richard for this post:

I don't understand why your chosen estimate of the location during the previous bar is the Median, but the estimate of the location during the current bar is the Close. What makes the two cases different, given that both times you've got an unknown distribution and you want to summarize where price is?

The following user says Thank You to Richard for this post:

This an excellent question. On a smaller timeframe a bar is a proxy for a trading range. If the following bar closes outside that range we have a shift in value (= trending price action), if the following bar closes within the range (= inside close) we do not have any shift in value, so the directional movement should be zero. This is basically what Welles Wilder did, except that he used the temporary shifts generated by high and low.

So the first concept I tried was to modify the directional movement in a way that I replaced the condition High[0]>High[1] by Close[0]> High[1] and similar for the lows. The result was not satisfactory, however, as the indicator only gave delayed signals.

So I decided not to completely ignore the price action of the inside closes. Often the midpoint of a trading range can be considered a key level generating support or resistance, so to interpret the price action within the range of the prior bar, I decided to compare the close of the current bar to the midpoint. To filter out insignificant price moves I also used volume as a multiplier.

Neither midpoint nor close is identical with the point of control or VWAP of the bar, but they both come close and both have a higher probability then high or low. Comparing the midpoint to the close also gives me a numerical difference of 1.5 periods, as I associate the midrange with the midperiod and the close with the endpoint of the bar period of the following bar.

So I think this is the best I can do with 8 data points. Any better suggestion?

The following user says Thank You to Fat Tails for this post:

You are absolutely right. I should not have blamed Walras for applying a mathematical formalism borrowed from physics, as he actually contributed to the development of economics. It is certainly not the fault of Walras that economics has got stuck to equilibrium models for too long a time. Thanks for correcting this.

statistical physics to model markets is just as delusional as anything else you can pull out of the hat...Nothing but trading the known econometric skimping on the variables problem for a completely unmeasurable number of variables...
Game theory and markets is not a stone that has not been turned over, look up minority games. Problem is it suffers from the same problems of statistical physics.
If anything IMO information theory is at least worth knowing..
To my mind trends are caused by known information being absorbed by liquidity restraints, coupled with an unknowable noise/garbage multiplier...