[Other] Time Series database for tick and trade data. - Platforms and Indicators | futures io social day trading
futures io futures trading


Time Series database for tick and trade data.
Updated: Views / Replies:5,120 / 36
Created: by gregid Attachments:2

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you dont need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 2  
 
Thread Tools Search this Thread
 

Time Series database for tick and trade data.

  #11 (permalink)
Elite Member
Stockholm
 
Futures Experience: Advanced
Platform: NinjaTrader,Sierra Chart, Python
Broker/Data: IB
Favorite Futures: ES & Stocks
 
Posts: 86 since Apr 2013
Thanks: 80 given, 71 received


artemiso View Post
Golden advice here.



People who want to store tick data maybe there can be advantages to use a special database. The issue with store tick data is difficult to find a data source that can supplies many years of tick data.. I need more history that that so daily/minute data is more than enough for my purpose and then I can get 20-50 years of history..

Reply With Quote
 
  #12 (permalink)
Trading Apprentice
western NY
 
Futures Experience: Intermediate
Platform: NinjaTrader, Python
Favorite Futures: equities, bonds, currency, commodities
 
Posts: 42 since Apr 2015
Thanks: 9 given, 25 received


Optiondreamer View Post
What about Tea Files? I use it and works fine....

Looks pretty cool.
If you google TeaFile vs hdf5 there is a nice discussion on this topic on nuclear phynance and the creator of TeaFile chimes in.

That same thread they mention scidb Paradigm4: Home. That has quant finance in mind but might be overkill for an individual.

As far as not being important what you decide on I also think there is a technological fun factor not to be dismissed.
Some people get a kick out of maximizing the performance of their car engines but any engine that runs can easily go the speed limit. To say you don't need to bother maxing out the performance of the engine while true completely misses the point.


Last edited by Darthtrader4beta; May 3rd, 2015 at 03:39 PM.
Reply With Quote
 
  #13 (permalink)
Elite Member
Wrocław, Poland
 
Futures Experience: Intermediate
Platform: NinjaTrader, Racket
Favorite Futures: Ockham's razor
 
gregid's Avatar
 
Posts: 651 since Aug 2009
Thanks: 321 given, 604 received



Darthtrader4beta View Post
Looks pretty cool.
If you google TeaFile vs hdf5 there is a nice discussion on this topic on nuclear phynance and the creator of TeaFile chimes in.

That same thread they mention scidb Paradigm4: Home. That has quant finance in mind but might be overkill for an individual.

As far as not being important what you decide on I also think there is a technological fun factor not to be dismissed.
Some people get a kick out of maximizing the performance of their car engines but any engine that runs can easily go the speed limit. To say you don't need to bother maxing out the performance of the engine while true completely misses the point.

True that! Some do it with their cars, others with computer rigs, yet others with their software, and algorithms (strategies anyone? ) After all - isn't it how the progress is made? Especially if it brings joy to your life

[Comments on the lack of social life are not welcomed ]

Reply With Quote
 
  #14 (permalink)
Elite Member
Wrocław, Poland
 
Futures Experience: Intermediate
Platform: NinjaTrader, Racket
Favorite Futures: Ockham's razor
 
gregid's Avatar
 
Posts: 651 since Aug 2009
Thanks: 321 given, 604 received

SciDB is at the moment my most serious contender for the next project. For "array type" languages like python+numpy+pandas or julia this looks to be the most natural way of working with persistent data.

Here a presentation on timeseries and SciDB:



Another interesting project from the python world:
Ecosystem — Blaze 0.7.3 documentation

Reply With Quote
The following 5 users say Thank You to gregid for this post:
 
  #15 (permalink)
Elite Member
San Francisco, CA
 
Futures Experience: Advanced
Platform: SC, eSignal
Broker/Data: IB
Favorite Futures: Spreads
 
Posts: 46 since Jan 2015
Thanks: 44 given, 38 received

Some obvious well-known contenders that come to mind:

RRD
BDB

When it comes to efficiency/overhead, a row-based database (which most SQL accessed ones are) is not ideal for tick data.

Reply With Quote
 
  #16 (permalink)
Trading Apprentice
western NY
 
Futures Experience: Intermediate
Platform: NinjaTrader, Python
Favorite Futures: equities, bonds, currency, commodities
 
Posts: 42 since Apr 2015
Thanks: 9 given, 25 received

damn, SciDB and python in that video looks like the sex machine.

Reply With Quote
 
  #17 (permalink)
Trading Apprentice
Paris, France
 
Futures Experience: Intermediate
Platform: MetaTrader
Favorite Futures: Currency Futures
 
Posts: 2 since Sep 2016
Thanks: 0 given, 2 received

Hi,

sorry for seeing this thread so late!

What is your opinion about Arctic ?
Search "arctic datastore"

Thanks

Reply With Quote
The following user says Thank You to femto for this post:
 
  #18 (permalink)
Trading Apprentice
Paris, France
 
Futures Experience: Intermediate
Platform: MetaTrader
Favorite Futures: Currency Futures
 
Posts: 2 since Sep 2016
Thanks: 0 given, 2 received

An other option may be Feather, it's "fast, interoperable binary data frame storage for Python, R, and more powered by Apache Arrow."

Any opinion about this one also?

Reply With Quote
The following user says Thank You to femto for this post:
 
  #19 (permalink)
Elite Member
Wrocław, Poland
 
Futures Experience: Intermediate
Platform: NinjaTrader, Racket
Favorite Futures: Ockham's razor
 
gregid's Avatar
 
Posts: 651 since Aug 2009
Thanks: 321 given, 604 received

From a cursory look Feather looks more like an alternative to hdf5.
Arctic looks like a more mature solution though. Nice slideshow about arctic:
Arctic: High-performance IoT and financial data storage with Python a?

Reply With Quote
 
  #20 (permalink)
Elite Member
Jacksonville, Florida United States
 
Futures Experience: Intermediate
Platform: NT8, Fully custom
Broker/Data: NT Brokerage, Interactive Brokers
Favorite Futures: ES
 
Jasonnator's Avatar
 
Posts: 69 since Dec 2014
Thanks: 19 given, 43 received

PostgreSQL


Set up PostgreSQL according to the attachment and you're set.

Attached Thumbnails
Time Series database for tick and trade data.-nosql-way-postgresql.pdf   Time Series database for tick and trade data.-superfast-postgresql.pdf  

Last edited by Jasonnator; September 29th, 2016 at 08:25 AM. Reason: add Superfast PostgreSQL pdf
Reply With Quote
The following user says Thank You to Jasonnator for this post:

Reply



futures io > > > [Other]       Time Series database for tick and trade data.

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)
 

RandBots: AutoTrading Bots to diversify your trading

Oct 24
 

TopstepTrader: Profiting Without Risking Your Own Money

Oct 26

Building Open Source Indicators for NT8 w/Fat Tails @ Lizard Indicators

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
CTA - Series 3, Series 7, Series 9, Series 10, Series 56 NASD exams certifications Big Mike Traders Hideout 41 January 15th, 2015 09:27 AM
Tick data for ES a few years in time elitetradernyc NinjaTrader Programming 2 May 6th, 2013 04:23 PM
DataBase of TICK Data bomberone1 The Elite Circle 4 June 17th, 2012 09:31 PM
Is the 6E 12.50 a tick? Best time to trade it Texas time? skyfly Currency Futures 7 August 7th, 2011 09:49 AM
PLT_NTDataCollect store bid ask tick data to its local database? rcabri Trading Reviews and Vendors 3 December 27th, 2010 10:00 AM


All times are GMT -4. The time now is 10:39 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-10-22 in 0.12 seconds with 20 queries on phoenix via your IP 54.157.81.13