Its been a while since I posted anything at FIO. Last year, I realized that my rules-based algos will soon be outperformed by smarter ones, probably based on machine learning. So, I am going to teach myself machine learning and convert my algos to be based on ML.
This journal is to document what I learn and eventually, how it is applied to market data. Its mostly going to be about education related to programming ML and application using trivial data, not really about trading strategies. Hopefully, this invites some meaningful conversation or companionship along the way.
My background:
I have been trading since 2009. Struggled during the initial 5-6 years. I was very lucky to get a job as a trader.
I trade many algos simultaneously. All of my algos are rules-based and semi-automated (manual execution)
I have been a programmer for 25+ years
My trading and programming background helps but there are still a lot of challenges...
I have been teaching myself but I keep getting lost because how vast the subject is and how new all this is to me. This is the main reason for starting this journal. Plus, getting old does not help either. And, math is not my friend unfortunately. I have recently taken many courses starting from high school algebra to calculus to linear algebra and although I still dont understand most of it, I think I know enough to implement machine learning algorithms.
Another challenge is that I am a Java programmer primarily. My algos run in a real-time linux/java environment. I can probably use Python for ML and interop using Jpype (, which is probably the smoothest and fastest integration between Python and Java, but it does require that the whole program needs to start in the Python runtime, which I dont like.
AWS has an open source library called DJL (. This is a Java wrapper over Apache MXNet, PyTorch and TensorFlow. Basically, the DJL API abstracts the usage of the underlying libraries and provides a unified interface. So the same code can be run against any of the underlying libraries. This is very appealing to me. The problem is documentation but I think enough exists to get started at least.
Based on all that I have read and learnt so far about the ML world, my goal is to sufficiently master Recurrent Neural Networks and Reinforcement Learning. As of now, I believe thats the best application of ML to trading.
The path to RNNs and RL isnt trivial. This is how I think I want to progress:
Linear Regression
Logistic Regression
Supervised ML
Supervised Deep Learning/Neural Networks
Cluster Analysis/Unsupervised ML
Unsupervised Deep Learning/NNs
RNNs
RL
Time Series Forecasting (using all of above)
For each topic, I want to learn the concepts, implement those in Java using DJL and hope to document my progress here. And, I really hope I can maintain my focus during this journey :becky:.
EDIT (01/22/22): DJL proved to be a little immature at this time. Switched to DL4J/ND4J backend (.
EDIT (03/18/2022): DL4J is also not ready for advanced usage. Switched to Python.