Combining discretionary trading, risk management and ML as an art - Emini Index Futures Trading | futures io social day trading
futures io futures trading


Combining discretionary trading, risk management and ML as an art
Updated: Views / Replies:521 / 17
Created: by aquarian1 Attachments:7

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 7  
 
Thread Tools Search this Thread
 

Combining discretionary trading, risk management and ML as an art

  #1 (permalink)
The fun is in the numbers
Point Roberts, WA, USA
 
Futures Experience: Advanced
Platform: IB and free NT
Broker/Data: IB
Favorite Futures: ES
 
aquarian1's Avatar
 
Posts: 3,136 since Dec 2010
Thanks: 1,137 given, 1,696 received
Forum Reputation: Legendary

Combining discretionary trading, risk management and ML as an art

I am learning the Weka program via the MOCC (free courses)
The program comes from a New Zealand University of Waikato.


There are 3 courses:
Data Mining with Weka
More Data Mining with Weka
Advanced Data Mining with Weka.

The purpose of this thread is to gather thoughts on doing data mining for trading which seems to be a sparsely covered topic.


Department of Computer Science
University of Waikato, New Zealand
http://cs.waikato.ac.nz/

Class1.2
https://www.youtube.com/watch?v=nHm8otvMVTs

Slides (PDF):
http://goo.gl/IGzlrn

https://twitter.com/WekaMOOC
Data Mining with Weka

Keep your mind in the future, in the now.

Last edited by aquarian1; October 9th, 2017 at 08:58 PM.
Reply With Quote
The following 2 users say Thank You to aquarian1 for this post:
 
  #2 (permalink)
Quick Summary
Quick Summary Post

I am hoping that this thread be a combination of ideas
risk management
trading psychology
discretionary trading
and how ML can possibly assist - (but not replace) decisions in these areas.

Therefore even if you are not planning to automate your trading (which I am not at this time) this thread could still be of use to you. It is not a pure ML thread at all but mostly a trading thread with ML as a possible adjunct in certain areas.

You may not know anything about ML but have questions about it and how it might help you in your trading. Though I myself will probably know know the answer, hopefully others will. Don't worry if you are a beginning trader and don't worry if your question might be silly. Asking a question can start a flow with others asking theirs and an exchange of ideas.

There are limitations to approaching trading with pure "think of something - back test - repeat"
Intelligently we wish to understand if we are seeing shadows in our setups.

As all variables and indicators (correction most variables) are price,time,volume based there is an interplay - interrelationship between them. We do expect that they will not be independent. However has an indicator added new useful information to our set-up or is the information added contained within another we are using?

Rather than replacing the human the Ml algorithms could possible guide them towards making better setups.

Naturally there are many questions a non auto-trader might like to pose (perhaps not specific at all) and this could be a place.

So even if you know nothing about ML and still have a question don't let that lack of ML stop you from posting your question.

(Clearly I am new in exploring this tool but others may to able to answer how ML could help with part of your trading strategy of trade management strategy.

@rleplae I would like to thank Ron for guiding us to this resource (Weka)


Last edited by aquarian1; October 20th, 2017 at 07:35 PM.
 
  #3 (permalink)
The fun is in the numbers
Point Roberts, WA, USA
 
Futures Experience: Advanced
Platform: IB and free NT
Broker/Data: IB
Favorite Futures: ES
 
aquarian1's Avatar
 
Posts: 3,136 since Dec 2010
Thanks: 1,137 given, 1,696 received
Forum Reputation: Legendary

Cost trade-off


In the second course
More Data Mining with Weka (4.5: Counting the cost)
https://www.youtube.com/watch?v=N3R_Z8OlrIg

There is an example of a credit file with 1000 instances (this would represent 1000 people applying for loans).

The cost of a rejecting a good credit risk (a mis-classifcation) is considered to be 1/5 of the cost of accepting a bad credit risk. (so a missed opportunity is 1/5 of making a bad decision)

Using one ML algorithm (J48) without consideration to cost you could correctly select 70%
and missing a good application cost you $1000
and accepting a bad application costs you $5000
you would have total costs of $1,027,000 on 1000 applications.
Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).


(I multiplied the $1 and $5 by $1000)
--------------
If you apply a cost matrix to the prb then the ML minimizes total costs by sacrificing total correct for less bad applications accepted.
Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).

Keep your mind in the future, in the now.
Reply With Quote
 
  #4 (permalink)
The fun is in the numbers
Point Roberts, WA, USA
 
Futures Experience: Advanced
Platform: IB and free NT
Broker/Data: IB
Favorite Futures: ES
 
aquarian1's Avatar
 
Posts: 3,136 since Dec 2010
Thanks: 1,137 given, 1,696 received
Forum Reputation: Legendary

Cost trade-off in trading

how the above post might connect to trading:

For traders with limited capital
  1. the cost of taking a setup that turns to a costly loss can be higher than
  2. the cost of not taking a setup that would have given a small profit

More experienced traders realize you cannot just set a stop whereever you like eg 1:5 W:L size ratio. Where a stop can be placed is a function of the instrument the place in the price movement and many other factors.

If stops are too tight for these factors you keep getting stopped out quickly. - dying a death of a thousand cuts.


However different setups in your arsenal can have different:
  1. expectations of working out
  2. amount that they produce

So I see a potential analogy to the above post. The cost of losing is much higher than not winning =missing out on an opportunity.

Many losses of $500 in a row (or a high percentage of them to wins) can quickly knock out a small capital trader.
It would be like a small new startup bank with limited capital -they just can take too many losses at the start until they build up their capital. They are better to pass on doubtful setups.

Also it is psychologically expensive eroding your confidence.

Keep your mind in the future, in the now.
Reply With Quote
 
  #5 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,168 since Jun 2009
Thanks: 29,280 given, 82,799 received

Why not add to one of the existing big discussion threads in the Elite Automated Trading section?

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #6 (permalink)
The fun is in the numbers
Point Roberts, WA, USA
 
Futures Experience: Advanced
Platform: IB and free NT
Broker/Data: IB
Favorite Futures: ES
 
aquarian1's Avatar
 
Posts: 3,136 since Dec 2010
Thanks: 1,137 given, 1,696 received
Forum Reputation: Legendary

So this is why I made this thread. I don't see using ML for trading as a simple throw numbers in the top and turn the crank. One needs to understand the complexity of trading to intelligently apply ML to it.

That is the reason I am working slowly through the course. I am trying to pause and think of examples of its application and mis-application. I see a danger in just rushing into applying ML without knowing the dangers and limitations.

I have already learned many valuable points:
  • Using ZeroR to set a baseline
  • Selecting a subset of attributes rather than the entire db
  • not using the training set for testing
  • how NaiveBayes can give very good results even when the assumption of Independence is clearly violated
  • etc

I can am starting to see the complexity of the many ML techniques.
when that complex matrix is applied to the complexity of the many, many factors that go into trading
we get a complexity to the second power.

Though that gives challenges it gives potential.

Finding the right ML techniques to complement your trading strategies will probably prove to be an art.

Keep your mind in the future, in the now.
Reply With Quote
 
  #7 (permalink)
The fun is in the numbers
Point Roberts, WA, USA
 
Futures Experience: Advanced
Platform: IB and free NT
Broker/Data: IB
Favorite Futures: ES
 
aquarian1's Avatar
 
Posts: 3,136 since Dec 2010
Thanks: 1,137 given, 1,696 received
Forum Reputation: Legendary

@Big Mike

Thanks Mike.

The reason is in the post after yours. (you are too quick for me!). :-)

I want to stay apart from the existing threads on automation that I see being much more linear -with more depth to the discussion and not only about automated trading.

e.g,
https://futures.io/traders-hideout/30722-data-mining-indicators-vs-historical-data.html
"Trading System Lab is the most widely respected trading system data mining tool and is said to cost $60k/year. "

I think there is room for a thread on the intricacies of intelligently applying the factors of ML, discretionary trading, trading psychology and risk management.

However, I am perfectly OK if you move this to a non-elite trading journal if you feel this is the wrong place.

Keep your mind in the future, in the now.
Reply With Quote
The following user says Thank You to aquarian1 for this post:
 
  #8 (permalink)
The fun is in the numbers
Point Roberts, WA, USA
 
Futures Experience: Advanced
Platform: IB and free NT
Broker/Data: IB
Favorite Futures: ES
 
aquarian1's Avatar
 
Posts: 3,136 since Dec 2010
Thanks: 1,137 given, 1,696 received
Forum Reputation: Legendary

continuing the cost-sensitive model

When I looked at the credit model example I could see a parallel to trading.

In the credit model 1000 loan applications are the training set (the training set would be be like your data base (=db)of past trades for your strategy).
So this is 1000 historical setups, 1000 instances.
In the credit db 700 are good loans and 300 are bad so that is like your db with 700 setups were "good" and 300 were "bad".

For each loan you had characteristics (credit rating, own or rent, reason for the loan etc.) -these are called attributes.
For your setup you had attributes (above/below 20Ma, momentum positive or negative, etc)

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).




Now each loan had a result (called the class attribute) repaid=good, not repaid=bad
and each of your trades has a result win/loss.

Each trade has a potential cost (as well as a potential payoff). So if you lose 5 pts on a "Bigswings" that don't work out and lose 1pt on "scaplers" that don't work out then this is the costs.

So in the lesson they are teaching that if you know your costs for each outcome and they are different (5 vs 1) then rather go for the highest accuracy in your ML algorithm you would seek to minimize the total cost. On best accuracy alone you might 70% predictive accuracy but with lowest total cost you might have 60.9%. accuracy.

Keep your mind in the future, in the now.
Reply With Quote
 
  #9 (permalink)
The fun is in the numbers
Point Roberts, WA, USA
 
Futures Experience: Advanced
Platform: IB and free NT
Broker/Data: IB
Favorite Futures: ES
 
aquarian1's Avatar
 
Posts: 3,136 since Dec 2010
Thanks: 1,137 given, 1,696 received
Forum Reputation: Legendary

Neural Networks

Last night I did the lesson on neutral networks. The instructor is not very impressed. He did his Master Thesis on them (at U of Calgary - which seems to have a strong ML comp sci dept), and made an improved algo. What he is impressed with is the name - thinks its brilliant! neural nets? not so much so.

In any case they were called perceptron in 1957 when first started, then fell out of favour with a paper showing their limitations and then came back into vogue when a method around the limitation (of linear boundary) was solved with the "kernal" trick. {They are akin to support vector machines which also use boundaries for classification.} They can be multilayer (hidden layers in addition to the input and output layer). Additional layers greatly increase the number of permutations and therefore computations - though may not add much to predictive accuracy in datamining. They are akin to linear regression analysis and go through repetitive learning cycles to adjust weights vectors based on the error rate (using gradient decent and later error increase to terminate the searching =epoch cycles).

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).

https://www.youtube.com/watch?v=mo2dqHbLpQo

So what's all this mean?

Keep your mind in the future, in the now.

Last edited by aquarian1; October 11th, 2017 at 12:38 PM. Reason: what is nn without a nice complex picture?
Reply With Quote
 
  #10 (permalink)
The fun is in the numbers
Point Roberts, WA, USA
 
Futures Experience: Advanced
Platform: IB and free NT
Broker/Data: IB
Favorite Futures: ES
 
aquarian1's Avatar
 
Posts: 3,136 since Dec 2010
Thanks: 1,137 given, 1,696 received
Forum Reputation: Legendary

Neural Networks 2


Well not much - at least for datamining.

I take notes as I am doing the lessons (these lessons are youtube videos).

2 neutral net options
1. The Voted Perceptron (under function Classifiers) and it got 86% on the ionosphere
2. SMO is another choice and it got 88.6%


BY COMPARISON two other algos that are not NN:
1. Logistics = 88.9%
2. SimpleLogistic = 88.3%

So on that db the 2 nn algos didn't perform better than the Logistics algo.

------------------- my takeaway ------

I have noticed in trading many people are attracted to complicated sounded things. So if their indicator was called "heuristic adaptive quantum stochastic" indicator -- boy it must be good !

but a brilliant name that conjures how sophisticated one is seem to pander more to one's ego that to results. Certainly Kiss can't be over-applied to ST day trading for there is complexity in the markets. However, watching others via journals etc - one of the biggest dangers to a trader is too much ego. (BTW I'm not calling confidence, ego)

Especially for new traders jumping to complexities before getting a sold handle on basics (S/R, double tops etc) can be a recipe for account blow-up.

Keep your mind in the future, in the now.

Last edited by aquarian1; October 11th, 2017 at 12:41 PM.
Reply With Quote

Reply



futures io > > > > Combining discretionary trading, risk management and ML as an art

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)
 

RandBots: AutoTrading Bots to diversify your trading

Oct 24
 

TopstepTrader: Profiting Without Risking Your Own Money

Oct 26

Building Open Source Indicators for NT8 w/Fat Tails @ Lizard Indicators

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Webinar: Machine Learning & The Insidious Threat of Data Mining Bias w/Kris Longmore Big Mike Elite Automated Trading 13 September 6th, 2017 04:00 PM
WEKA rleplae Terms (Glossary) 0 August 19th, 2017 02:49 AM
Bitcoin trading/mining Big Mike Cryptocurrency Trading 117 November 11th, 2015 04:14 AM
data mining, indicators vs historical data saj1011 Traders Hideout 10 July 31st, 2014 12:38 PM
Data Mining /A.I. in Ninja for automation? jungian NinjaTrader 4 March 17th, 2012 11:36 AM


All times are GMT -4. The time now is 09:38 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-10-23 in 0.08 seconds with 19 queries on phoenix via your IP 54.81.110.114