Combining discretionary trading, risk management and ML as an art - futures io
futures io futures trading



Combining discretionary trading, risk management and ML as an art


Discussion in Emini and Emicro Index

Updated
      Top Posters
    1. looks_one aquarian1 with 40 posts (39 thanks)
    2. looks_two Big Mike with 1 posts (0 thanks)
    3. looks_3 Quick Summary with 1 posts (0 thanks)
    4. looks_4 TheTradeSlinger with 1 posts (1 thanks)
    1. trending_up 5,223 views
    2. thumb_up 40 thanks given
    3. group 14 followers
    1. forum 41 posts
    2. attach_file 18 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Combining discretionary trading, risk management and ML as an art

(login for full post details)
  #1 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

I am learning the Weka program via the MOCC (free courses)
The program comes from a New Zealand University of Waikato.


There are 3 courses:
Data Mining with Weka
More Data Mining with Weka
Advanced Data Mining with Weka.

The purpose of this thread is to gather thoughts on doing data mining for trading which seems to be a sparsely covered topic.


Department of Computer Science
University of Waikato, New Zealand
https://cs.waikato.ac.nz/

Class1.2


Slides (PDF):
https://goo.gl/IGzlrn

https://twitter.com/WekaMOOC
Data Mining with Weka

Table of Contents
1. summary of findings in using datamining for predictive trading

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following 3 users say Thank You to aquarian1 for this post:

Journal Challenge February 2021 results (so far):
Competing for $1500 in prizes from Topstep
looks_oneSBtrader82 's Trading Journalby SBtrader82
(167 thanks from 31 posts)
looks_twoJust BEING a Trader: Letting Go!!by iqgod
(116 thanks from 33 posts)
looks_3Wisdom is Emptinessby Mtype
(68 thanks from 25 posts)
looks_4Deetee’s DAX Trading Journal (time based)by Deetee
(31 thanks from 17 posts)
looks_5Journal for peanuts1956by peanuts1956
(23 thanks from 13 posts)
 
 
(login for full post details)
  #3 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received


In the second course
More Data Mining with Weka (4.5: Counting the cost)


There is an example of a credit file with 1000 instances (this would represent 1000 people applying for loans).

The cost of a rejecting a good credit risk (a mis-classifcation) is considered to be 1/5 of the cost of accepting a bad credit risk. (so a missed opportunity is 1/5 of making a bad decision)

Using one ML algorithm (J48) without consideration to cost you could correctly select 70%
and missing a good application cost you $1000
and accepting a bad application costs you $5000
you would have total costs of $1,027,000 on 1000 applications.


(I multiplied the $1 and $5 by $1000)
--------------
If you apply a cost matrix to the prb then the ML minimizes total costs by sacrificing total correct for less bad applications accepted.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #4 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

how the above post might connect to trading:

For traders with limited capital
  1. the cost of taking a setup that turns to a costly loss can be higher than
  2. the cost of not taking a setup that would have given a small profit

More experienced traders realize you cannot just set a stop whereever you like eg 1:5 W:L size ratio. Where a stop can be placed is a function of the instrument the place in the price movement and many other factors.

If stops are too tight for these factors you keep getting stopped out quickly. - dying a death of a thousand cuts.


However different setups in your arsenal can have different:
  1. expectations of working out
  2. amount that they produce

So I see a potential analogy to the above post. The cost of losing is much higher than not winning =missing out on an opportunity.

Many losses of $500 in a row (or a high percentage of them to wins) can quickly knock out a small capital trader.
It would be like a small new startup bank with limited capital -they just can take too many losses at the start until they build up their capital. They are better to pass on doubtful setups.

Also it is psychologically expensive eroding your confidence.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #5 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,753 since Jun 2009
Thanks: 32,299 given, 97,501 received

Why not add to one of the existing big discussion threads in the Elite Automated Trading section?

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #6 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

So this is why I made this thread. I don't see using ML for trading as a simple throw numbers in the top and turn the crank. One needs to understand the complexity of trading to intelligently apply ML to it.

That is the reason I am working slowly through the course. I am trying to pause and think of examples of its application and mis-application. I see a danger in just rushing into applying ML without knowing the dangers and limitations.

I have already learned many valuable points:
  • Using ZeroR to set a baseline
  • Selecting a subset of attributes rather than the entire db
  • not using the training set for testing
  • how NaiveBayes can give very good results even when the assumption of Independence is clearly violated
  • etc

I can am starting to see the complexity of the many ML techniques.
when that complex matrix is applied to the complexity of the many, many factors that go into trading
we get a complexity to the second power.

Though that gives challenges it gives potential.

Finding the right ML techniques to complement your trading strategies will probably prove to be an art.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #7 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

@Big Mike

Thanks Mike.

The reason is in the post after yours. (you are too quick for me!). :-)

I want to stay apart from the existing threads on automation that I see being much more linear -with more depth to the discussion and not only about automated trading.

e.g,

"Trading System Lab is the most widely respected trading system data mining tool and is said to cost $60k/year. "

I think there is room for a thread on the intricacies of intelligently applying the factors of ML, discretionary trading, trading psychology and risk management.

However, I am perfectly OK if you move this to a non-elite trading journal if you feel this is the wrong place.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following 2 users say Thank You to aquarian1 for this post:
 
(login for full post details)
  #8 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

When I looked at the credit model example I could see a parallel to trading.

In the credit model 1000 loan applications are the training set (the training set would be be like your data base (=db)of past trades for your strategy).
So this is 1000 historical setups, 1000 instances.
In the credit db 700 are good loans and 300 are bad so that is like your db with 700 setups were "good" and 300 were "bad".

For each loan you had characteristics (credit rating, own or rent, reason for the loan etc.) -these are called attributes.
For your setup you had attributes (above/below 20Ma, momentum positive or negative, etc)





Now each loan had a result (called the class attribute) repaid=good, not repaid=bad
and each of your trades has a result win/loss.

Each trade has a potential cost (as well as a potential payoff). So if you lose 5 pts on a "Bigswings" that don't work out and lose 1pt on "scaplers" that don't work out then this is the costs.

So in the lesson they are teaching that if you know your costs for each outcome and they are different (5 vs 1) then rather go for the highest accuracy in your ML algorithm you would seek to minimize the total cost. On best accuracy alone you might 70% predictive accuracy but with lowest total cost you might have 60.9%. accuracy.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #9 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

Last night I did the lesson on neutral networks. The instructor is not very impressed. He did his Master Thesis on them (at U of Calgary - which seems to have a strong ML comp sci dept), and made an improved algo. What he is impressed with is the name - thinks its brilliant! neural nets? not so much so.

In any case they were called perceptron in 1957 when first started, then fell out of favour with a paper showing their limitations and then came back into vogue when a method around the limitation (of linear boundary) was solved with the "kernal" trick. {They are akin to support vector machines which also use boundaries for classification.} They can be multilayer (hidden layers in addition to the input and output layer). Additional layers greatly increase the number of permutations and therefore computations - though may not add much to predictive accuracy in datamining. They are akin to linear regression analysis and go through repetitive learning cycles to adjust weights vectors based on the error rate (using gradient decent and later error increase to terminate the searching =epoch cycles).




So what's all this mean?

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #10 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received


Well not much - at least for datamining.

I take notes as I am doing the lessons (these lessons are youtube videos).

2 neutral net options
1. The Voted Perceptron (under function Classifiers) and it got 86% on the ionosphere
2. SMO is another choice and it got 88.6%


BY COMPARISON two other algos that are not NN:
1. Logistics = 88.9%
2. SimpleLogistic = 88.3%

So on that db the 2 nn algos didn't perform better than the Logistics algo.

------------------- my takeaway ------

I have noticed in trading many people are attracted to complicated sounded things. So if their indicator was called "heuristic adaptive quantum stochastic" indicator -- boy it must be good !

but a brilliant name that conjures how sophisticated one is seem to pander more to one's ego that to results. Certainly Kiss can't be over-applied to ST day trading for there is complexity in the markets. However, watching others via journals etc - one of the biggest dangers to a trader is too much ego. (BTW I'm not calling confidence, ego)

Especially for new traders jumping to complexities before getting a sold handle on basics (S/R, double tops etc) can be a recipe for account blow-up.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #11 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

I think in this thread I am trying to put forward that our knowledge of trading, having been in the trenches helps us when we move forward in applying possible tools such as ML.

There are a lot of dynamics to trading - risk management, stop-running algos, surprise moves, your own broker trading against you with "may act as principle" and are allowed to "internalize" a trade - meaning catching the edge from 2 clients and never sending it to the exchange, money management, psychology, lack of a clear road map, central bank market manipulation, volatility suppression and asset class correlation convergence (via risk parity funds, and interest rate suppression, buybacks and index hedging), low capitalization, etc. etc. The list is large.

So it may be that trading experience and knowledge will help us better understand the limitations of ML, the potentials of ML and the correct application.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #12 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

There are 3 courses on Weka. I am now doing the 3rd one. In the most recent lesson we have gotten to time series analysis.

In lesson1.2 he does stuff one step at a time - manually so you can see the process and in the next lesson he uses the forecast plug-in model.

First lesson manually
First he does a regression and then a manually lagged one. The db is the airline data and it has a 12 month cyclically to it, and so he creates a 12 month lagged series (attribute). Then deletes the first 12 months (instances) as linear regressions does bad things with missing data.

Here's the data straight-line regression:



Here's the result of the 12 month lagged and then regressed



With the lagging you get in effect and cyclical model and quite a good fit.

source:
Advanced Data Mining with Weka (1.2: Linear regression with lags)

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #13 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

In ML you often to the training of your model with 10-fold cross validation. (reusing 9 of 10 data slices in different combinations.) However you can't do this with time series data because the order matters.

So with time series data you hold out part of your data set for testing (training on the rest).

As you know in trading your data can go through "phases". So you can have:
an uptrend phase followed by
a sideways phase followed by
another upward phase followed by
a sideways phase followed by
as an example.

If you were building a model for the Close to Close (C:C), then ignoring the phases will not give you a model that simply holds back the last 15% of your data and training on the rest, will ignore the difference in phases (Note 1).

All traders understand these phases (market context) and so understand just dumping all the data in and turning the crank may not give you the results you are looking for. This is the problem of "averaging" across phases.

------------
Note1: At least to my knowledge at this time

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #14 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

The automatic time series forecasting package at default settings can give a very low error (when trained on entire data set - which you shouldn't do) and over-fitting. Overfitting is a bad thing where your model is so tight to the data that it will probably not forecast well.

The automatic program creates this by generating a large number of attributes, giving a very complex model and misleadingly high fit on the entire dataset. If we hold out 24 attributes for testing we can see the difference showing overfitting with too many attributes (attributes would be like columns in a spreadsheet).



source:
Advanced Data Mining with Weka (1.3: timeseriesForecasting package)

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #15 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

  • Use ZeroR to establish a baseline
  • NaiveBayes often works very well even when the independence assumption clearly doesn't apply.
  • In time series analysis you have to put your thinking hat on and be careful to only have in meaningful variables (attributes). Otherwise you can easily overfit and have a model that doesn't forecast well.
  • Time series errors propogate.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #16 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

ZeroR counts occurrences and predicts all most on the most common.

So if you had 1000 days and 600 up days and 400 down days it would predict (classify) all days as up days - as it is the most numerous and it would have 60% correct.

Let's say you are looking for a ML for predicting up days on ES and your model starts on March 2009.
If you just run an ML and get 65% up days you might shout "Eureka!" until you run ZeroR and realize that 65% were up days and your model didn't give you new insights.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #17 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

How might ML help us, we discretionary traders?

Two possible approaches (among several) come to mind:
  • testing our setup to establish win frequency
  • combining key inputs to help generate a rule set

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #18 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

There is a time forecasting plug-in package for Weka.

It comes with a db of apple stock prices in 2011. date, O/h/l/c and adjclose.

If you open in an just run the forecast the package will interpolate values for the date holes, ie weekends and holidays. This will throw off your results so you tell the prg to ignore weekend, date1, date2, ... (dates of the holidays when the market is closed.) The instructor predicted the closing price.

For day traders we are interested in intraday values. All the values are assigned to the same date
that is o,h,l,C are all on the same row - even those these happen at different times.

I was thinking it would be nice to have a third dimension, time (of the day) but I don't know how to set this up.

source:
Advanced Data Mining with Weka (1.3: timeseriesForecasting package)

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #19 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

Advanced Data Mining with Weka (2.1: Incremental classifiers in Weka)



Advanced Data Mining with Weka (2.2: Weka’s MOA package)



MOA can be run on multiple computers or on just one. It can be run from Explorer, CLI, and with the java API.
massiveOnlineAnalysis: MOA (Massive On-line Analysis).
URL: https://moa.cms.waikato.ac.nz/

Massive On-line Analysis is an environment for massive data mining. MOA provides a framework for data stream mining and includes tools for evaluation and a collection of machine learning algorithms. Related to the WEKA project, also written in Java, while scaling to more demanding problems. IMPORTANT: This package requires a separate download of the MOA jar file in order to work. After installing the massiveOnlineAnalysis Weka package: 1) download MOA from https://www.cs.waikato.ac.nz/~abifet/MOA, copy the "moa.jar" file from the MOA distribution to $HOME/.weka/packages/massiveOnlineAnalysis/lib, 3) re-start Weka.

Advanced Data Mining with Weka (2.3: The MOA interface)


In prequential evaluation with each incoming instance we test and then we train.


..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #20 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

As I understand MOA:

One can use it for infinite datastreams. It process each new instance and tests and then learns.

As one can connect it with the JAVA api - I would imagine it can listen to the incoming datastream -( though I don't have the fainest idea how to do this ) updating your ML model in real-time.

It handles concept drift - which for traders would be changes in market conditions and so your model is adaptive to changes in trend and volatility etc

It can also split the workload amongst several machines

-- all the above is just my understanding of it and the programmers out there could say better. ---------

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #21 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

At this point in the thread I think I need to add a real world example.

The course has been good so far - but now has it's own 'concept drift' with very brief videos by the new lecturers.

In the lesson where they did time series analysis (apple stock), I sensed how the instructors knew machine learning and stats but did not connect to real world and specifically trading. And that of course is the key to this thread - moving from the abstract to the real world.

(As a small digression I remember thinking how educators could do a ral service by interviewing people who are working in their own fields and ask them what are both the good and bad of their professions - what the reality is.)

So I have created an ARFF file (the Weka format) with CL daily OHLC & vol. Weka has readers to pull in CSV formats and Excel formats (among others such as db). However, the CSV convertor gave errors and the Excel showed my date as numeric (so the format one sees for a date 25-Oct-2107 or 10/25/2017 etc is not how the spreadsheet stores it 43034=numeric not date).

So it was quite a bother to find a work-a-around to create a file with the date as a date and not numeric.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #22 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

As I was starting to learn about ML I listened to a series of fast paced youtubes.

I thought I would include then if they are of interest to others.


intro to deep learning


Intro in Deep learning 2


Sentiment Analysis 3

waldeinsamkeit - alone in the woods connecting to nature

Math used in neural nets 4


Predicting Stock Prices - Learn Python for Data Science Siraj Raval


How to Make an Image Classifier - Intro to Deep Learning #6


This isn't on ML but interesting
HOw to learn


[yt]https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A/videos[/yt]

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #23 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

I have added info about ML into the summary page

logit

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #24 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

In the 3rd Weka course they show how to use the forecasting package and give an example of Apple stock daily values and forecasting tomorrow's closing price with the previous closes. The absolute mean error is looked at as a measure of success of the forecasting.

For daytraders having an estimate of today's closing price is not the first thing on our list of things we would like estimates of. Now for the apple stock the error is considered small by the instructor (not knowing about trading and in reference to other kinds of ML.

I got to thinking what is the baseline error?
If you give a trader a O/H/L/C daily series the very first thing to see is what is the change between one days close and another.

1. a simple prediction without ML is using yesterday's change as the estimate of today's.

others might be:
2. yesterdays close as an estimate of today's (we know this won't be correct except on most days but in a flat trend the mean absolute error could be low).
3. just using today's open as an estimate of the close
4. today's open + yesterday's change from open to close.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #25 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

If you would like to try time series analysis - forcasting, you could download Weka, and then the forecasting add-in package. With those you can watch the youtube lesson :

Advanced Data Mining with Weka (1.3: timeseriesForecasting package)


I suggest pausing and duplicating what the instructor is doing with apple stock prices.

I have created an ARFF file for CL daily prices (see attached zip) for you to play/learn with.

Weka has data-loaders for
CSV, DB, and XLS (Excel) files. (I haven't tried the db loader). Though the xls works fine for most files, for ones with a date column it see the date as numeric not date and so I went through some machinations with notepad and then manually created the ARFF header.




Here is the Weka Explorer showing how you open the CLdaily.arff file and the Forecasting tab (which shows once you add -in the package.)


..........
peace, love and joy to you
.........
Attached Files
Register to download File Type: zip Cldaily.zip (19.8 KB, 2 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #26 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

I have run the CLdaily file 3 times once with each of the learning schemes.
The goal is to get the lowest Mean Abs Error.

All 3 learning schemes are about the same on test data (the last 30 percent of the data held-out for testing).
$0.79, $0.84, $0.81 (linreg, SMO, NN respectively).

on the (learning) data (first 70% of instanced) the results are
$0.98, $0.97, $1.49 (linreg, SMO, NN respectively).

(The entire 3 run results are attached in a zip)


..........
peace, love and joy to you
.........
Attached Files
Register to download File Type: zip CLruns.zip (54.2 KB, 2 views)
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #27 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

I would think an obvious benchmark (equivalent to ZeroR for classifiers) for closing data is yClose.

So what happens if we just use yesterday's close as an estimate for today's close?



the ML systems didn't outperform
$0.79, $0.84, $0.81 (linreg, SMO, NN respectively)
to our benchmark $0.76

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #28 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

This is to my point about using ML intelligently.

Though ML people who are good with their systems might think $0.79, $0.84, $0.81 mean absolute errors (linreg, SMO, NN respectively) are good results, we as traders with common sense and real world experience know better with regards to trading.

Predicting the close isn't much good to a trader. Even if it were and I told you my estimate of the close was on average $0.79 from the actual you wouldn't be impressed.

$50.35 (the prediction for 21st of Sept 2017 from linreg -see data of CLrun1.txt) plus and minus $0.79 is a range from $51.14 to $49.56

I think the close on the 29th will be $50.35 but with an average error of 79 cents it could be as high as $51.14 and as low as $49.56. (and that is only the average, mean, error. It could be much wider)

---------
Actual data
2017-09-20 close = 50.69 chg 0.79
2017-09-21 close = 50.55 chg -0.14
----------

Pretty useless for trading info.
Don't apply ML blindly.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following 2 users say Thank You to aquarian1 for this post:
 
(login for full post details)
  #29 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

Apriori builds rule sets from the data.

The data must be nominal (not numeric). You can discretize your data. This is a binning process that employs enthropy heuristic (chooses split points that have the largest information gain)



Course 2.2 More Data Mining with Weka (2.2: Supervised discretization and the FilteredClassifier)


3.4: Learning association rules

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #30 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

11:47 AM 11/14/2017

In working with my dataset I found that if I used the discretization filter on the class attribute, the filter would discretize all other non-nominal attributes. I didn't want this so I am back to my spreadsheet holding my database and going through the discretizing the class attribute. The first step is to decided on the binning. Being familiar with the data of the class attribute I realized I wanted unequal binning size.

The goal is a model with the most predictive value to me for trading. Therefore, without knowing the results of the model run I can still make some decisions on binning f the class attribute.

If there are too many bins then I will not have enough instances in a bin. Too few and I will not have any predictive value.

I have 594 instances in my training dataset. I have decided on 12 bins (not all the same size).
Here is the frequency distribution of the data at equal bins each unit


The vertical blue lines are were I decided to set the bin edges

Here is the frequency distribution of the binning

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #31 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

(In relation to the prior post - I may have not needed to go back to the spreadsheet)


Today I worked with weka again.

I found that if I first changed the relationship to:
(no class)
unsupervised discretized the attribute I wanted, then made it the class attribute I could then use the supervised discretization which included information gain in the binning.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #32 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

I've still been working with the ML.

Most I have been using Apriori which has no class attribute (unsupervised) and looks for associations.

At first, I was thinking I wasn't using it correctly and then I drop the minimum confidence level from the default 90% downwards and then I got some rules. (BTW you can't have numeric attributes - it will tell you this in the Log -so you want to use the filter "Numeric to Nominal" to process any numeric attributes.)


I had thought the using the discretization filter would be the way to bin, but I didn't think the results were very good for data with wide narrow tails - though perhaps it is - something to learn about later.

I wanted to create an example for readers to apply these ideas to, so I have chosen the overnight gap.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #33 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

As most traders know there is an overnight gap fill idea.

The o/n gap is the difference between yesterday's close and today's open (RTH).

So for Dec 08 the gap was 7.00 (Close=2639.75 Dec 7 to Open=2646.75 Dec 8)

temp1.png

The idea in the gap fill is that most of the time the price will usually return to close the gap at some time during the day - that is at some point close the 7pt gap by falling to 2639.75.

Now a variation on the gap-fill setup is the progap. This is where it "gaps and goes" meaning on days with a "large" gap the gap fill trade won't work. So in the example above is +7 is a "large" gap it won't come back to Yclose of 2639.75.

So what is "large"? You can:
get all the gaps for past days (remembering to eliminate contract roll-over days)
bin the gaps
look for results with binned data.

So you might create bins of:
>+7.0 =very large up gap
+4 to +7.0 = large up gap
+0.0 to 3.75 = up gap
-0.25 to -3.75 = down gap
-4.0 to -7.0 = large down gap
<-7.0 = very large down gap

Here I've made 6 bins.
Bins do not need to be equal size.
You can also choose equal frequency binning (with the bin edges such that there are the same number of instances (observations) in each bin.

So we are taking a semi-continuous numeric series and nominalizing it into 6 types of gaps.

It depends on what you are trying to do (or for us trade).

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following 2 users say Thank You to aquarian1 for this post:
 
(login for full post details)
  #34 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

Once you have your set of rules then you need to decide how to trade that info.

Will you wait for an up move and then sell? If yes how much of an up move?
Will you sell 1 at open, and a second at +3.75 to the open, and a third at +4.50 to the open - should it get there?
Will you use indicators for the timing of the sell?

Of course your stop limits on trades will come into play - too wide a stop and you suffer on a bad trade and too tight and it pushes trough and then reverses to win with you no longer on board.

If you're a Goldman Sachs (https://en.wikipedia.org/wiki/Goldman_Sachs) and can toss $100 mil into the experiment and take a good size drawdown.

If you're tiny retail almost any drawdown is "down and out." :-)

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following 2 users say Thank You to aquarian1 for this post:
 
(login for full post details)
  #35 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

I've been working with my database and ML

(Below are examples only. I didn't use these i.e. -daytype, Y%ClsChg)

Let's say you are predicting daytype: if it is an up day, down day or flat day as measured by close-to-close.
So you might decide an up day is C:C of +3 or greater, down -3 of less and flat -3 to +3.

It seems the binning doesn't work the way I want (or at least I don't know how to make it work the way I want)
Consider a potential continuous variable e.g. the percentage change in the close.
-so values could be -15%, -6.8%, 0%, +5.4% etc


To use association ML such at J48, you need nominal values - not numeric.
A filter will nominalize these giving '-15%", '-6.8", etc

I was hoping the discretize would bin them - eg into ten bins. but it doesn't seem to work -except for equal frequency binning.

As with all things perhaps a difficulty is an opportunity.

So I work on binning (frequency distribution) within the spreadsheet - my database.


------------- on the next bit I'm not sure if I'm breaking some statistical rule or not ------------------
First I counted up, down, flat - so perhaps out of 750 instances you might have: 200 down, 300 flat, 250 up.
Then I would take my predictive attribute, yesterday's %chg in close (Y%ClsChg) and sort it eg ascending and use the vlookup function to give me the bin edges that would be approximately the same distribution 200,300 and 250.

I used these bin edges with a formula to discretize Y%ClsChg into -1,0,+1
(You could have a different number of bins, e.g. instead of 3 bins 5bins: -2,-1,0,1,+2)

------------- end of next bit ---------------------------

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following 2 users say Thank You to aquarian1 for this post:
 
(login for full post details)
  #36 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

I thought I would post a summary of my findings so far. (Please understand I am not an expert with statistics in reading the below.)

1. There is a strong limitation in the understanding of trading by those who are experts in ML algos. For example, a common example is predicting the close of a stock and the forecasting lagging and so on are done for that. For futures trading for the retail trader the predictions of the close are next to useless.

2. For numeric data generally the ML experts look at it as a regression problem. However, at least in my approach, it it not. Looking at forecasting as classification problem becomes much more difficult as you must discretize your data and nominalize it. The choice of binning of your class attribute will greatly impact the usefulness of your final model(s).

As an example, if you have one bin for your class attribute you would be 100% accurate even if you are using the height of pine trees in Washington state as your predictor. If you have only two classes -e.g. "up day" and "down day" your accuracy will be at minimum the size of your largest class (using ZeroR). So if your data has 60% up days and 40% down days, simply predicting every day as an up day gives you a 60% accuracy.

So you need several classes for your class attribute (the thing you are trying to predict).

What is the correct number?
The answer lies in your trading and your data and the compromise of the two.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following 2 users say Thank You to aquarian1 for this post:
 
(login for full post details)
  #37 (permalink)
Huntington WV
 
Experience: Advanced
Platform: TradeStation
Trading: ES, CL
 
TheTradeSlinger's Avatar
 
Posts: 430 since Jun 2015
Thanks: 714 given, 589 received

@aquarian1,

Fantastic thread!

I've always thought about this but have never been able to test it: what if you "showed" the ML algo your winning trades and somehow had it identify/classify why they were winners, then have it look for more opportunities like your winning trades?

I probably butchered that explanation in terms of ML lingo, but do you see what I am saying?

Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to TheTradeSlinger for this post:
 
(login for full post details)
  #38 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received


TheTradeSlinger View Post
@aquarian1,

Fantastic thread!

I've always thought about this but have never been able to test it: what if you "showed" the ML algo your winning trades and somehow had it identify/classify why they were winners, then have it look for more opportunities like your winning trades?

I probably butchered that explanation in terms of ML lingo, but do you see what I am saying?

I understand your question.

So there are ML algos that they simply tell it to win. E.g. a video game and you just let it play tons of games and it will soon (like overnight) play perfectly, starting from terrible.

Another case is that they they let it play "Go" against itself and it beat a (the) world champion.
Of course there are limited dimensions to these games.
(I don't know how to do this ML BTW).


The thing about your question is the information has to be in what you show it.

That is we as humans might think that such and such combination of indicators contains the information that produced the successful trade and we may have made the trade and won, yet the reason we won wasn't in the indicator setup but was something else.

It (the ML) needs lots of data and the data must have the answer hidden in the data that you give it for it the ML to find the patterns.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #39 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

One thing I should mention was something I saw in a video about the dangers of ML.
The example given was an image classification ML.

The speaker showed a picture of a wolf and said - "why might you label this as a picture of a wolf?"
The shape of the snout perhaps?
the eyes?

The programers added some lines and ask the ML why it classified it as a wolf.
It showed the snow.

All the images it was trained on of wolves showed snow in the background.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #40 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

I wanted to test is the surpervised discretization would break the same if I reduced the instances of the greatest count in the class attribute to equal the average count of the class attribute

I had 755 instances and 3 distinct classes giving 252 per class average.
Initially I had count of:
260=-1
165=0
330=+1
in the class attribute. I deleted 78 of the +1's bringing its count to 252.

The attribute I was testing for supervised discretization was initially broken at
-21.5 and 5.63 into 3 groups
and showed correctly classified at the top group +5.63 to +infinity of 67% accuracy.

Then (at the reduced count of 252) the attribute I was testing for supervised discretization was broken at
-21.5 and 5.75 into 3 groups
and showed correctly classified at the top group +5.75 to +infinity of 60.5% accuracy.

Conclusion
It works better when there is more variance in the dominant count of the class attribute (e.g. 330) then if all counts are about the same size (e.g.252). I think this makes sense as the supervised discretization is trying to maximize the information. However, it seems to be somewhat of an over-fitted giving a sense of greater predictive power than is really the case.

I think my rule of thumb is to have enough distinct classes in the class attribute to be meaningful broken into approximately equal bins. (Depending on what you are trying to predict and I think 5 distinct classes to be about correct balance in workable rules for the current class I am examining).

With 5 distinct classes I believe equal binning into 5 bins (of the predictive attributes) works well.

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
(login for full post details)
  #41 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

In this thread I had used the example of up days, down days, inside days, outside days and other for binning.
In the database there are 752 records. (one per day).

In looking for an estimate of the RTH midpt I have been doing some analysis - regression analysis.

There were
up days = (199 days)
down days = (160 days)

..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #42 (permalink)
Point Roberts, WA, USA
 
Experience: Advanced
Platform: IB and free NT
Broker: IB
Trading: ES
 
aquarian1's Avatar
 
Posts: 4,026 since Dec 2010
Thanks: 1,504 given, 2,579 received

I have posed a stats question in the information help box:


..........
peace, love and joy to you
.........
Visit my futures io Trade Journal Started this thread Reply With Quote


futures io Trading Community Traders Hideout Emini and Emicro Index > Combining discretionary trading, risk management and ML as an art


Last Updated on March 31, 2018


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
 

Journal Challenge w/$1500 prizes from Topstep!

February
 

Battlestations! Show us your trading desk - $1,500 in prizes!

March
 

Call Option Buying: The New Pain Trade? w/Carley Garner

Elite only
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts