How to create a database for backtesting? (Page 2) - Traders Hideout | futures trading

Go Back

> Futures Trading, News, Charts and Platforms > Traders Hideout

How to create a database for backtesting?
Started:October 9th, 2015 (09:27 AM) by GoldenRatio Views / Replies:823 / 12
Last Reply:March 21st, 2016 (10:57 PM) Attachments:1

Welcome to

Welcome, Guest!

This forum was established to help traders (especially futures traders) by openly sharing indicators, strategies, methods, trading journals and discussing the psychology of trading.

We are fundamentally different than most other trading forums:
  • We work extremely hard to keep things positive on our forums.
  • We do not tolerate rude behavior, trolling, or vendor advertising in posts.
  • We firmly believe in openness and encourage sharing. The holy grail is within you, it is not something tangible you can download.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community. It's free and simple, and we will never resell your private information.

-- Big Mike

Thread Tools Search this Thread

How to create a database for backtesting?

Old October 10th, 2015, 05:12 PM   #11 (permalink)
Site Administrator
Manta, Ecuador
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
Big Mike's Avatar
Posts: 45,425 since Jun 2009
Thanks: 28,834 given, 79,538 received

rleplae View Post
I am able to put something on a chart...

I am not able to create a chart from zero (like a basic series/instrument)
Let say i create synthetic data and want that charted in ninjatrader...

What would you recommend ?

Better to let someone that uses NinjaTrader answer.


Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote

Old October 10th, 2015, 05:14 PM   #12 (permalink)
Elite Member
Omer עומר / Israel י
Futures Experience: Master
Platform: NinjaTrader, Proprietary,
Broker/Data: Ninjabrokerage/IQfeed + Synthetic datafeed
Favorite Futures: 6A, 6B, 6C, 6E, 6J, 6N, 6S, ES, CL, ZB
rleplae's Avatar
Posts: 1,457 since Sep 2013
Thanks: 1,063 given, 2,271 received

Big Mike View Post
Better to let someone that uses NinjaTrader answer.


A datafeed connector is a solution....

Reply With Quote

Old March 21st, 2016, 10:57 PM   #13 (permalink)
Trading Apprentice
Los Angeles, CA
Futures Experience: Advanced
Platform: Interactive Brokers
Favorite Futures: ES, CL, SB, KC, Forex
Posts: 14 since Jan 2016
Thanks: 0 given, 5 received

Flat text file-based storage with some clever file naming convention and maybe a directory hierarchy to split contracts by markets and types (i.e. futures from stocks from options, etc...) is a more scalable solution than a traditional DB. Most of the time when you work with historical data (time series) - you don't need to query data, or update /delete records, you just load it up as a sequence of data points, and reading files line-by-line works great. Most languages have support for streaming, so you don't need to load all data into memory at once..

So there is often no need for a SQL-based (or even a non-SQL based) database. Market price data, once obtained and cleaned, is static. You can zip it to save more space, and it's easy to archive. In addition, no DB means one less system to setup, maintain, backup, worry about fail-over, etc... Flat files do not consume CPU power or memory until they are used, unlike a DB engine. You can also easily load comma- or tab- separated files in Excel, or any other tool of your choice.

By the way found a really great tool for getting stocks/futures/options/Forex historical data as csv text files from Interactive Brokers, highly recommend it. You can forget about pacing violation or request/response size limits , it splits large data requests into many smaller once automatically behind the scenes:

Interactive Brokers Historical Data Downloader

Reply With Quote

Reply > Futures Trading, News, Charts and Platforms > Traders Hideout > How to create a database for backtesting?

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Upcoming Webinars and Events (4:30PM ET unless noted)

Corey Rosenbloom: Tips, Tactics, and Real Trades: Trading Intraday Trend Reversals

Elite only

An Afternoon with FIO trader bobwest

Elite only

NinjaTrader 8: Programming Profitable Trading Edges w/Scott Hodson

Elite only

Anthony Drager: Executing on Intermarket Correlations & Order Flow, Part 2

Elite only

Adam Grimes: Five critically important keys to professional trading

Elite only

Machine Learning Concepts w/FIO member NJAMC

Elite only

MarketDelta Cloud Platform: Announcing new mobile features

Dec 1

NinjaTrader 8: Features and Enhancements

Dec 6

Similar Threads
Thread Thread Starter Forum Replies Last Post
why Backtesting stocks difficult vs backtesting Futures emini_Holy_Grail Traders Hideout 8 July 4th, 2015 02:14 PM
Tick Database Storage MXASJ Platforms and Indicators 40 July 20th, 2012 12:30 PM
Database - A Simple Query ab456 Traders Hideout 1 October 15th, 2011 02:04 PM
Database for NT bomberone1 NinjaTrader Programming 6 April 29th, 2011 09:11 AM
My 2 cents in mysql database iwannatoscript NinjaTrader Programming 5 March 7th, 2010 08:56 AM

All times are GMT -4. The time now is 08:48 PM.

Copyright © 2016 by All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts

Page generated 2016-10-20 in 0.11 seconds with 26 queries on phoenix via your IP