I'm new here and wanted to start a thread for thinking "outside the box." Sometimes we have epiphanies that may or may not be of relevance to trading. I had a thought today and don't know if much research has been done into it, but perhaps others have.
It started after I downloaded the ES tick data available here. I noticed that the compressed files were roughly 10% of the size of the actual tick data files when fully expanded.
I went on to do a bit of research on Wikipedia about compression (Data compression - Wikipedia, the free encyclopedia (:)
"In computer science and information theory, data compression, source coding,[1] or bit-rate reduction involves encoding information using fewer bits than the original representation.[2] Compression can be either lossy or lossless. Lossless compression reduces bits by identifying and eliminating statistical redundancy."
Interesting... compression algorithms identify and eliminate statistical redundancy. I wonder what those redundancies are and if there are edges to be gained from such redundancies...
Just thinking out loud--feel free to chime in. I hope we can grow this thread together.