I know I should probably learn MATLIB or SPSS or some other statistical tool, but I've never learned those tools and it turns out I'm really good with SQL Server, so I'm looking at building a data mine of market data and metrics that I am interested in researching.
Currently, I am just doing this on the freely available SQL Express. If/when I need more BI capabilities and SSIS I will migrate to a full developer version of SQL Server.
So, what I've done so far is export a few years of historical data out of NT7 continuous contract data, imported them into SQL, and am now working on creating daily measurements (bars) of the market via stored procedures into fact tables.
For example, here's my script that processes the daily OHLCV data and stores various SMAs that might be of interest.
Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).
I have a lot more to do, and can provide updates of my progress, but I was just wondering if anybody else likes analyzing the markets with SQL.
The following 7 users say Thank You to shodson for this post:
Have you looked at LINQ? Once you have the data you need in SQL you can use external apps to manipulate it and report. LINQ/C#, R to SQL, Matlab to SQL, etc are all at your disposal in way not easily done with the NT database itself.
If you like free stuff R might be worth a look before Matlab.
The following 2 users say Thank You to MXASJ for this post:
I've played with LINQ but, like most other object-data mapping attempts created by Microsoft, I hit a wall that prompted me to throw it away. For object-relational mapping (ORM) I have created my own code generator that I've used for years, based on similar approaches used by Ruby on Rails and other frameworks inspired by enterprise design patterns (Fowler). Microsoft's "entity framework" looks perhaps maybe they're getting closer, but if you don't have your own ORM framework for .NET I'd recommend Subsonic.
The following user says Thank You to shodson for this post:
@shodson thanks a lot for this thread. I know it is a very old thread, but could you please post an update. Did you make any progress in doing analysis with SQL ? Or you found it to be very limiting and therefor dumped it midway ?
I am also interested in doing some analysis within SQL Server for the EOD database that I have. I think that a lot of work could be done within such databases, which would be difficult to do in the normal platforms like ninja etc. Since you did this years ago, therefor I am very interested in knowing about your overall practical experience in this regards.
I use MySQL extensively in my own platform (second thread).
Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.
Need help? 1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first. 2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses. 3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make. 4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance. 5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers. 6) Help using the forum? Watch this video to learn general tips on using the site.
If you want to support our community, become an Elite Member.
The following 2 users say Thank You to Big Mike for this post:
Yes I made progress and it's a core part of my trading today. Instead of finding it limiting, I find it very expansive and additive to my trading toolbelt instead of just relying on vendor-provided software. If you have the skills, time, and passion for statistics and quantitative methodologies I highly recommend something along these lines.
The following 2 users say Thank You to shodson for this post: