NexusFi: Find Your Edge


Home Menu

 





How to create a database for backtesting?


Discussion in Traders Hideout

Updated
      Top Posters
    1. looks_one Big Mike with 3 posts (1 thanks)
    2. looks_two rleplae with 3 posts (0 thanks)
    3. looks_3 GoldenRatio with 2 posts (0 thanks)
    4. looks_4 Quick Summary with 1 posts (0 thanks)
    1. trending_up 8,128 views
    2. thumb_up 6 thanks given
    3. group 7 followers
    1. forum 12 posts
    2. attach_file 1 attachments




 
Search this Thread

How to create a database for backtesting?

  #11 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,397 since Jun 2009
Thanks Given: 33,173
Thanks Received: 101,537


rleplae View Post
I am able to put something on a chart...

I am not able to create a chart from zero (like a basic series/instrument)
Let say i create synthetic data and want that charted in ninjatrader...

What would you recommend ?

Better to let someone that uses NinjaTrader answer.

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Better Renko Gaps
The Elite Circle
New Micros: Ultra 10-Year & Ultra T-Bond -- Live Now
Treasury Notes and Bonds
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
Deepmoney LLM
Elite Quantitative GenAI/LLM
Futures True Range Report
The Elite Circle
 
  #12 (permalink)
 
rleplae's Avatar
 rleplae 
Gits (Hooglede) Belgium
Legendary Market Wizard
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
Posts: 3,003 since Sep 2013
Thanks Given: 2,442
Thanks Received: 5,863


Big Mike View Post
Better to let someone that uses NinjaTrader answer.

Mike

A datafeed connector is a solution....

Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #13 (permalink)
schematics12
Los Angeles, CA
 
Posts: 14 since Jan 2016
Thanks Given: 0
Thanks Received: 5


Flat text file-based storage with some clever file naming convention and maybe a directory hierarchy to split contracts by markets and types (i.e. futures from stocks from options, etc...) is a more scalable solution than a traditional DB. Most of the time when you work with historical data (time series) - you don't need to query data, or update /delete records, you just load it up as a sequence of data points, and reading files line-by-line works great. Most languages have support for streaming, so you don't need to load all data into memory at once..

So there is often no need for a SQL-based (or even a non-SQL based) database. Market price data, once obtained and cleaned, is static. You can zip it to save more space, and it's easy to archive. In addition, no DB means one less system to setup, maintain, backup, worry about fail-over, etc... Flat files do not consume CPU power or memory until they are used, unlike a DB engine. You can also easily load comma- or tab- separated files in Excel, or any other tool of your choice.


By the way found a really great tool for getting stocks/futures/options/Forex historical data as csv text files from Interactive Brokers, highly recommend it. You can forget about pacing violation or request/response size limits , it splits large data requests into many smaller once automatically behind the scenes:

Interactive Brokers Historical Data Downloader

Reply With Quote




Last Updated on March 21, 2016


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts