Real-time trade 'recorder'? - Traders Hideout | futures io social day trading
futures io futures trading


Real-time trade 'recorder'?
Updated: Views / Replies:2,364 / 6
Created: by Dionysus Attachments:2

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 100,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Real-time trade 'recorder'?

  #1 (permalink)
Monterey, CA
 
Trading Experience: Intermediate
Platform: Ninjatrader
Favorite Futures: CL
 
Posts: 80 since Jan 2014
Thanks: 28 given, 97 received

Real-time trade 'recorder'?

I'm not looking to have anything programmed/created by anyone here, so I hope this thread is in the proper location and can remain . . . apologies if I've err'ed. .

I'm curious as to whether or not anyone has, or knows about or has come across, any utility or program that pulls trade execution data in real-time for importation/processing elsewhere, to allow for statistical display? Ideally for Ninjatrader, as we're still using NT as our platform for executions, for the moment. It'd need to be able to handle hundreds of strategies running simultaneously.

My goal is to create a page of real-time trading statistics, broken down into several categories ranging from the most broad (cumulative total profit/loss across all strategies), to the the most narrow (a specific strategy) and everything in between (groupings by instrument, by market-direction, by instrument and market direction, etc).

For this, of course, I need the hard data of all trade executions as the foundation, and I want this to be collected automatically in real-time. I'm almost certain I'll need custom solution created, so I'm about to head to my tried and true companion 'Upwork' and similar sites, but I'd love to hear if there's anything currently out there that can accomplish the data collection portion of this task. . would save quite a bit of time/effort. Or, if anyone has gone a similar route and has any advice, I'm all ears.

A hearty thanks in advance!


Last edited by Dionysus; January 13th, 2017 at 05:00 PM.
Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
New York, NY
 
Trading Experience: Beginner
Platform: Vanguard 401k
Broker/Data: Yahoo Finance
Favorite Futures: Mutual funds
 
Posts: 1,045 since Jul 2012
Thanks: 717 given, 2,142 received


The results need to be server-rendered if you want a real-time streaming dashboard. I think the simplest setup is to do this with basic Apache/HTTP. The backend that computes your statistics could be anything, really. Something like Go, Ruby, Python should sit well with a HTML/AngularJS frontend; if you need plots, for these type of statistics, you don't really have to write your own fancy UI code, you can use existing libraries. Aside from how you do threading, I think the number of strategies has very little to do with how well your solution handles this. You can communicate over JSON which will be one of the slower parts of the entire stack, but it will probably still be seamlessly fast to the naked eye if you do it right.

I've coded up something similar to what I've described above in a couple of days that produced real-time analytics down to every trade but also fetched historical summaries. Here's a screenshot of it computing summary statistics on an entire day across multiple data centers. I uncensored 1 data center where the strategies sent out about 1 million orders and canceled about 880k orders in just one data center, which notwithstanding fills would probably be around 0.1% to 0.3% of the CME's order activity across every symbol nowadays.

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).


And I could drill down on the positions and individual orders of individual strategies like this seamlessly as well:

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).



As for how you collect the data, the simplest way to do this is to store this after you serialize the outgoing messages or deserialize the incoming messages. Unfortunately, a third party vendor like NT most definitely uses some kind of binary format behind the scenes that you won't be able to decode easily. So the easiest way you might be able to do this while relying on them is to write a custom strategy or indicator that runs in the background and uses their API to access the incoming fills, and have your trading strategy serialize the data associated with outgoing orders.

You'll want to store only the lowest level information like that and then compute whatever's higher level like strategy PnL in the server backend.

Reply With Quote
The following 3 users say Thank You to artemiso for this post:
 
  #4 (permalink)
Administrator: Retired Backtester
Rennes France
 
Trading Experience: Advanced
Platform: NinjaTrader
Broker/Data: IB/Kinetick
Favorite Futures: Futures
 
sam028's Avatar
 
Posts: 3,568 since Jun 2009
Thanks: 3,738 given, 4,328 received


artemiso View Post
...Unfortunately, a third party vendor like NT most definitely uses some kind of binary format behind the scenes that you won't be able to decode easily. So the easiest way you might be able to do this while relying on them is to write a custom strategy or indicator that runs in the background and uses their API to access the incoming fills, and have your trading strategy serialize the data associated with outgoing orders.
...

No need to decode anything: the OnOrderUpdate() builtin method can be used to send execution data (using ZeroMQ for example) then have a ZMQ listener (I'll wrote it in Python but it could be anything else) which will store everything, SQL, CSV or whatever.
AngularJS would be a good option for displaying the data, Excel would do the job too (simpler but not that nice).

Success requires no deodorant! (Sun Tzu)
Follow me on Twitter Reply With Quote
The following 3 users say Thank You to sam028 for this post:
 
  #5 (permalink)
New York, NY
 
Trading Experience: Beginner
Platform: Vanguard 401k
Broker/Data: Yahoo Finance
Favorite Futures: Mutual funds
 
Posts: 1,045 since Jul 2012
Thanks: 717 given, 2,142 received


sam028 View Post
No need to decode anything: the OnOrderUpdate() builtin method can be used to send execution data (using ZeroMQ for example) then have a ZMQ listener (I'll wrote it in Python but it could be anything else) which will store everything, SQL, CSV or whatever.

Yes, this is what I mean by using NT's API to access the fills or serialize the outgoing orders' state.

Reply With Quote
The following user says Thank You to artemiso for this post:
 
  #6 (permalink)
Monterey, CA
 
Trading Experience: Intermediate
Platform: Ninjatrader
Favorite Futures: CL
 
Posts: 80 since Jan 2014
Thanks: 28 given, 97 received

Excellent, thank you both. . all I needed was a clean way to gather the 'root' data of order execution information on an ongoing basis in real-time, to be processed (in a separate stage) in various ways afterwards, and I think I now have all the info I need to build the proper structure to allow for this. . . appreciated.

If anyone else has done such things and has any additional tips, I'd love to hear 'em, just to assure I don't reinvent any wheels here, but I now have enough to go on

Reply With Quote
 
  #7 (permalink)
Adelaide South Australia
 
Trading Experience: Beginner
Platform: CQG NinjaTrader TT IR/T
Favorite Futures: Futures
 
Posts: 6 since Dec 2014
Thanks: 0 given, 7 received

Also been looking for something like this for a long time, even a live equity curve as you're trading intra-day would be awesome, may even find patterns in your own trading around certain times of the day.

Pity I suck at programming so I'm useless.

Reply With Quote

Reply



futures io > > > Real-time trade 'recorder'?

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)
 

futures io is celebrating 10-years w/ over $18,000 in prizes!

Right now
 

$250 Amazon Gift Cards with our "Thanks Contest" challenge!

Right now
 

Show us your trading desks and win over $5,000 in prizes w/Jigsaw Trading

September
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Real Time Data at once matevisky Reviews of Brokers and Data Feeds 6 August 8th, 2013 01:28 AM
New.. is it time for real money?? MotorsportsX Beginners and Introductions 16 November 30th, 2012 03:30 PM
REAL TIME NT INTO EXCEL gparkis NinjaTrader Programming 2 June 19th, 2012 06:40 PM
Monitoring the earth real time kbit Off-Topic 2 June 10th, 2012 10:51 PM
Genetic Optimization vs real time kandlekid NinjaTrader Programming 7 November 16th, 2010 11:08 AM


All times are GMT -4. The time now is 01:17 AM. (this page content is cached, log in for real-time version)

Copyright © 2019 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts