A new (open source?) trading platform - Platforms and Indicators | futures io social day trading
futures io futures trading


A new (open source?) trading platform
Updated: Views / Replies:46,296 / 140
Created: by Big Mike Attachments:5

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 5  
 
Thread Tools Search this Thread
 

A new (open source?) trading platform

  #41 (permalink)
Membership Revoked
Szczecin
 
Futures Experience: Intermediate
Platform: Ninja, writing own now
 
Posts: 212 since Mar 2010
Thanks: 47 given, 102 received


MXASJ View Post
Not sure this is the best place to post this but here you go:

Has anyone looked at StreamInsight, Microsoft's shot at the CEP space? Is anyone aware of it being used in trading apps?

Thanks.

You will not see that outside of closed shops (due to high licensing costs - iompractical to put into a product) and there it is way too early to see it in wide adoption. Development takes time.

Reply With Quote
The following user says Thank You to NetTecture for this post:
 
  #42 (permalink)
Elite Member
Asia
 
Futures Experience: Beginner
Platform: NinjaTrader, TOS
 
Posts: 798 since Jun 2009
Thanks: 109 given, 786 received

NetTecture et al...

Would you mind sharing a bird's eye view of what considerations are needed in the design of a trading platform? I've made it past "Hello World" programming but I'm very new to this and large project stuff is something I'm trying to get my head around.

I know this is a very broad question, so I'll make some very broad statements to see if it leads to something.

At its heart a trading platform needs to receive inputs (price update data, account update data, order update data, etc), do something with that data, and then output something (orders, a chart, a db write, etc).

On the input side, there is no universal standard for messages. Some may be FIX, some may be a broker API, some may be a datafeed API, some may be static data from a db, etc. "Connectors" that translate data/events into a common format are required.

The "core" of the trading platform processes data/events in that common format.

The "connectors" are then used on the output side, perhaps sending things back out as a FIX message or broker API-specific message, or being output to a graphics engine for charting or any trade visualization, etc.

So where would one even start if you wanted to build your own platform? A UML diagram and a specification document? Would you start at the "core" and work outwards?

What are the performance constraints, best practices, industry consensus, etc to keep in mind in all aspects of such a project?

I continue to look at Tradelink (tradelink - Project Hosting on Google Code) for general architecture ideas but I'm not smart enough to know if it is brilliant or not. What is actually important? The database structure? The messaging bus? Unique ways to handle concurrency and multithreading on a per "connector" basis?

Any thoughts, ideas, or suggested reading is most welcome.

Thanks in advance for your time.

Reply With Quote
The following 2 users say Thank You to MXASJ for this post:
 
  #43 (permalink)
Membership Revoked
Szczecin
 
Futures Experience: Intermediate
Platform: Ninja, writing own now
 
Posts: 212 since Mar 2010
Thanks: 47 given, 102 received



MXASJ View Post
NetTecture et al...

Would you mind sharing a bird's eye view of what considerations are needed in the design of a trading platform? I've made it past "Hello World" programming but I'm very new to this and large project stuff is something I'm trying to get my head around.

I know this is a very broad question, so I'll make some very broad statements to see if it leads to something.

At its heart a trading platform needs to receive inputs (price update data, account update data, order update data, etc), do something with that data, and then output something (orders, a chart, a db write, etc).

On the input side, there is no universal standard for messages. Some may be FIX, some may be a broker API, some may be a datafeed API, some may be static data from a db, etc. "Connectors" that translate data/events into a common format are required.

The "core" of the trading platform processes data/events in that common format.

The "connectors" are then used on the output side, perhaps sending things back out as a FIX message or broker API-specific message, or being output to a graphics engine for charting or any trade visualization, etc.

So where would one even start if you wanted to build your own platform? A UML diagram and a specification document? Would you start at the "core" and work outwards?

What are the performance constraints, best practices, industry consensus, etc to keep in mind in all aspects of such a project?

I continue to look at Tradelink (tradelink - Project Hosting on Google Code) for general architecture ideas but I'm not smart enough to know if it is brilliant or not. What is actually important? The database structure? The messaging bus? Unique ways to handle concurrency and multithreading on a per "connector" basis?

Any thoughts, ideas, or suggested reading is most welcome.

Thanks in advance for your time.

This is very broad a topic and I think if from the way you ask it some explaantions may not stick well due to lack of knowledge.

First, I think, you need to agree on an architecture and what you want to do. Plus WHY - there are soem platforms out. They lack featuers, but identifying WHY you do something is core.

Then in the end you deal with a message passing / Event driven architecture. You need a decent set of messages. Fix etc. is a start. The all lack interoperability, but at the end an order is an order, so within your system you want ONE define view on it, regardless of what the end connected system says. I go with a large C# library defining classes for everything. They possibly can not cary all data, but they carry all data that I need.

Another tricky thing is how to deal with threads. WIthout multi threading you get into perforamcne hell. WIth multi threading you get into programming hell - unless you plan that properly. THat said, planning that is not that hard for something like trading.

Dealing with financial data is also challenging performance wise, unless you go end user (few symbols). Updates even for ES are not that many.... but getting some complete exchange is another ball game (one reason I do my own). Data storage is tricky here. Wnat something that is end user usable? Flat files are pretty much it, or db blobs. Useing a db server ?Get a lot of discs. I currently run 6 discs for data, soon going to 8... only for the data files (logs, tempdb resides on other discs). Not only the volume is tricky, also the amount of updates.

In the end, it is not a small proejct whatever side you tackle it. Iti s not a project for someone just after his "hello world".

Reply With Quote
The following 3 users say Thank You to NetTecture for this post:
 
  #44 (permalink)
Elite Member
Madison, WI
 
Futures Experience: Advanced
Platform: Sierra Charts, ALT
Favorite Futures: ES
 
aslan's Avatar
 
Posts: 614 since Jan 2010
Thanks: 342 given, 1,077 received

Where to start

This is a big task, but you have to start somewhere.

I would start by looking at the different feeds you want to support. Look at how symbols are done, how historical and live data are provided, how orders are processed. I have looked at IQ, Rithmic, ZenFire, and Fix. They are all similar, but enough different that you need to transform their API into a common interface that the core uses. Your core code should not deal with any of the APIs directly.

Seems strange, but just dealing with the symbol universe is kind of a pain because each API does symbols differently. You need to pick a format that is your base, and then be able to map all of the other feeds to your symbol mapping. This is especially true if you are going to support multiple feeds concurrently (like MC). How do you store your symbol info?

The next step is to decide how to store data. You can use flat files, files per date, database, or whatever. Personally, I don't think a DB is the right choice as you are just adding another layer that does not provide much, but it could make sense based on your goals. One of the key issues here is keeping all ticks in order, but most feeds still use second based timestamps. This is where a straight files are nice, because you just write them to the file. In a db, you need to add another field to maintain the order of the ticks. Also, what timestamp granularity do you want to keep track of?

You can work thru most of the above without even thinking about the GUI much, but you have to tackle that at some point as well. The GUI is also related to your threading model, which is critical to your performance. Are you going to go windows only or include others like OS X/Linux? If you are going to go windows, then I like the WPF architecture as you can do so much with the GUI, and the threading works well. The main drawback is you are locked in to windows/C#. The alternative is to go old school with C++ and use the older windows model (like SierraCharts).

The next step is work thru the charting, order processing, DOM, position tracking, .....

This is a huge task, and before you even start, you have to have your goals laid out, and then start peeling the onion. As you attack pieces, be sure to come back to the top level occasionally to make sure you have not pigeon holed yourself.

Reply With Quote
The following 2 users say Thank You to aslan for this post:
 
  #45 (permalink)
Membership Revoked
Szczecin
 
Futures Experience: Intermediate
Platform: Ninja, writing own now
 
Posts: 212 since Mar 2010
Thanks: 47 given, 102 received


aslan View Post
This is a big task, but you have to start somewhere.

I would start by looking at the different feeds you want to support. Look at how symbols are done, how historical and live data are provided, how orders are processed. I have looked at IQ, Rithmic, ZenFire, and Fix. They are all similar, but enough different that you need to transform their API into a common interface that the core uses. Your core code should not deal with any of the APIs directly..

I question your selection. Fix is ok, Rithmic, too. IQ, too. But Zen-Fire is redundant - it internally IS RIghtmic, uses the API, so you gain little real insight.

I actually gained a lot by integrating (ongoing work) NxCore. VERY interesting.


aslan View Post
Seems strange, but just dealing with the symbol universe is kind of a pain because each API does symbols differently. You need to pick a format that is your base, and then be able to map all of the other feeds to your symbol mapping. This is especially true if you are going to support multiple feeds concurrently (like MC). How do you store your symbol info?

That actually is painfull. It goes gfurther as often between technologies the symbols also propagate to the broker. Trade YM on Rightmic and TradeMaeven, and the symbols are different ON THE ACCOUNT STATEMENTS. Nice, isn't it.


aslan View Post
The next step is to decide how to store data. You can use flat files, files per date, database, or whatever. Personally, I don't think a DB is the right choice as you are just adding another layer that does not provide much, but it could make sense based on your goals. One of the key issues here is keeping all ticks in order, but most feeds still use second based timestamps. This is where a straight files are nice, because you just write them to the file. In a db, you need to add another field to maintain the order of the ticks. Also, what timestamp granularity do you want to keep track of?

No feed I know of uses second timestamp granularity. Neither RIghmit nor Zen-Fire do (NInja does, but that happens after the feed and it is a major issue wit h the platform, together with the fact that tehy do NOT maintani order - bid/ask and quotes are stored in separate streams, apparently). No idea about IQ. IB does 1/10th of a second snapshots, NxCore 25ms timestamps with order, Righmic microsecond as they arrive.

Reply With Quote
The following user says Thank You to NetTecture for this post:
 
  #46 (permalink)
Elite Member
Madison, WI
 
Futures Experience: Advanced
Platform: Sierra Charts, ALT
Favorite Futures: ES
 
aslan's Avatar
 
Posts: 614 since Jan 2010
Thanks: 342 given, 1,077 received


NetTecture View Post
I question your selection. Fix is ok, Rithmic, too. IQ, too. But Zen-Fire is redundant - it internally IS RIghtmic, uses the API, so you gain little real insight.

I started out with ZenFire, so it is what I knew. You are right though, in that ZF is completely built on top of Rithmic, so if you can deal with the Rithmic API, ZenFire gets you nothing, unless your broker wont give you the Rithmic connection strings for the ZenFire server. The two APIs do a few things differently, but in general the R API is much cleaner.


Quoting 
No feed I know of uses second timestamp granularity.

IQ does, Tradestation does.

Reply With Quote
 
  #47 (permalink)
Membership Revoked
Szczecin
 
Futures Experience: Intermediate
Platform: Ninja, writing own now
 
Posts: 212 since Mar 2010
Thanks: 47 given, 102 received


aslan View Post
IQ does, Tradestation does.

Sure about those?

I espeically dislike "Tradestation does". Tradestation is not a feed. This is like saying "NinjaTrader does", but at the end this is NInjaTrader fucking up the stored data. So, how you know that TradeStation "feed" (which to my knowledge you hae no API to) does it?

When I started my quest back into trading, I determined I am not inteested in a feed that comes with an API that requires more than a token local application / window (i.e. something like nanex installing a local price SERVER is ok - it has a window showing data throughput etc., but something like IB is not where you run requests through a local trading application).

Reply With Quote
 
  #48 (permalink)
Elite Member
Pt Vala
 
Futures Experience: Advanced
Platform: Proprietary
Favorite Futures: CL, EUR, DAX
 
Posts: 58 since Apr 2010
Thanks: 12 given, 30 received

Regarding the general design of a trading platform I am absolutely with Aslan so far (already built a platform on my own and in the process of building it learnt some lessons the hard way):

- Main point is to abstract from any existing API, symbology, data formats. Otherwise the platform will be very hard to adapt for changing environments.
- Symbology has to be universal, logical and straightforward
- Storing data has to be as simple as possible. Trying to be sophisticated is detrimental.
- Store every single bit of information that is available at the maximum resolution possible. You will at some time in the future find a use for it.
- If the data feed does not supply timestamps use the processor clock and store together with the data.



Some ideas for the design:

Symbology
Have a format that can be used consistently for everything. Examples (hopefully self explaining):
FUT:CME:ES:201103
OPT:CBOE:SPY:CALL:201101:122
STK::AAPL

Data storage
Store every tick from a day in just one plain file (binary) with a name like TICKDATA 2010 12 21.DAT.
Header contains a list of all symbols together with a unique code that identifies them in the data file.
This makes retrieval a little more difficult.
It makes much easier data storage for longer timeframes and backup.
You see what you have and it's easy to transfer data to another computer, exchange or join data.





NetTecture View Post
No feed I know of uses second timestamp granularity. Neither RIghmit nor Zen-Fire do (NInja does, but that happens after the feed and it is a major issue wit h the platform, together with the fact that tehy do NOT maintani order - bid/ask and quotes are stored in separate streams, apparently). No idea about IQ. IB does 1/10th of a second snapshots, NxCore 25ms timestamps with order, Righmic microsecond as they arrive.


Don't know why such misinformation should be disseminated:

- Zenfire has millisecond timestamps.
- IB has no timestamps, ticks come irregularily. Sometimes 2 snapshots/sec, sometimes even 12/sec depending on market activity, symbol and other factors that are only known by IB.

Zenfire:
Table of Content
>Zenfire namespace>TickEventArgs>TimeStamp
This leads to:
DateTime Structure (System)
(64bit MS.NET DateTime structure)

Reply With Quote
The following user says Thank You to uexkuell for this post:
 
  #49 (permalink)
Elite Member
Madison, WI
 
Futures Experience: Advanced
Platform: Sierra Charts, ALT
Favorite Futures: ES
 
aslan's Avatar
 
Posts: 614 since Jan 2010
Thanks: 342 given, 1,077 received


NetTecture View Post
Sure about those?

I espeically dislike "Tradestation does". Tradestation is not a feed. This is like saying "NinjaTrader does", but at the end this is NInjaTrader fucking up the stored data. So, how you know that TradeStation "feed" (which to my knowledge you hae no API to) does it?

When I started my quest back into trading, I determined I am not inteested in a feed that comes with an API that requires more than a token local application / window (i.e. something like nanex installing a local price SERVER is ok - it has a window showing data throughput etc., but something like IB is not where you run requests through a local trading application).

Yes, very sure about IQ. TS actually has more granularity in the feed, but they store data with sec granularity.

Totally agree that TS is NOT a good starting point.

The key point here, is the granularity does not matter, because you still have to maintain order of quotes, and unless you have an unrealistic granularity you will always have a few quotes with the same timestamp. So, you have a sorting issue if you rely on the timestamp field alone (i.e. in a db).

Reply With Quote
The following user says Thank You to aslan for this post:
 
  #50 (permalink)
Membership Revoked
Szczecin
 
Futures Experience: Intermediate
Platform: Ninja, writing own now
 
Posts: 212 since Mar 2010
Thanks: 47 given, 102 received



uexkuell View Post

Don't know why such misinformation should be disseminated:

- Zenfire has millisecond timestamps.

Zenfire:
Table of Content
>Zenfire namespace>TickEventArgs>TimeStamp
This leads to:
DateTime Structure (System)
(64bit MS.NET DateTime structure)

Reality check: DateTime does NOT store milliseconds. Really. It stores TICKS. I suggest reading the documentation you point to for clarification. MS is just the smallest increment exposed OUTSIDE TICKS. Ticks are 100ns large, finer than any clock available. Not milliseconds. NICE try.

DateTime.Ticks Property (System):
A single tick represents one hundred nanoseconds or one ten-millionth of a second. There are 10,000 ticks in a millisecond.

No clue where you got the idea System.DateTime stores milliseconds. Again, maybe Zen-Fire cuts off the ticks. my own Rithmic code did not (now pulling data from Nanex).

I have quite some stored Rithmic data here from development, and the timestamps go a lot finer than milliseconds. Not sure whether Zen-Fire cuts them off (never used the API too long, way too clumsy with the object model). I now use nxCore which has a theoretical MS resolution, but gives the "minimum effective resolution" in every daily tape start, which is currently set at 25 (i.e. the milliseconds counter only increments in increments of 25).

Actually I think of storing the order of items in the ticks. Even bwlow MS that gives me 10.000 ticks per millisecond, 250.000 if using up the complete resolution, so I can store timetamp and maintain order in one field.

Another item - I second storing everything with as much resolution as possible Order and delay MAY come in handy writing a decent simulator. Yes, your order takes time, but the exact order is necessary to make sure you can give good simulated fills.

Final note: If you are multi threaded - Windows Scheduler Resolution is about 55ms. Smallest task switch interval. Want more? High prioerity thread, never switching (critical section) and using spinlocks when needed. Effectively uses tons of power (CPU core at 100%) and blocks one core. OTOH the only way to go smaller. IF that is needed. IO etc. still make it mostly a futile attempt.

Reply With Quote

Reply



futures io > > > A new (open source?) trading platform

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
OEC Open E Cry Trading Platform Ranger Open E Cry 77 July 24th, 2011 09:43 AM
Range bars before the time a open the Ninja Platform? Barrosco Traders Hideout 4 March 29th, 2011 09:04 AM
Volitility indicator question - Open - Open / open - close / close to open andresimon NinjaTrader 2 September 27th, 2010 01:33 AM
New Trading platform Michael.H The Elite Circle 19 September 16th, 2010 01:54 PM


All times are GMT -4. The time now is 05:06 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-13 in 0.18 seconds with 20 queries on phoenix via your IP 54.90.92.204