Speeding up NinjaTrader with a RAM drive (ramdisk) - NinjaTrader | futures io social day trading
futures io futures trading


Speeding up NinjaTrader with a RAM drive (ramdisk)
Updated: Views / Replies:25,334 / 110
Created: by pawnbroker Attachments:6

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors – all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you don’t need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 6  
 
Thread Tools Search this Thread
 

Speeding up NinjaTrader with a RAM drive (ramdisk)

  #71 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,240 since Jun 2009
Thanks: 29,355 given, 83,237 received

I've been trying out PrimoCache. Under most circumstances it makes more sense than a dedicated ramdisk, and will offer the same performance benefits.

I am using a delayed write setting of 60 seconds. I see NinjaTrader writing 50,000 blocks per minute during large backtests, so this can really help both in terms of performance as well as lifespan (SSD wear level).

I am using an 8GB cache and with my normal day to day stuff, my cache hit ratio is only around 25%. If I increased my cache size to 16GB I would have much higher ratio, as I've only read 16.4GB from disk total since my last reboot. The problem is, I need around 20GB of memory to start Sierra Chart when it is loading all my charts (once it loads them, usage is only 1GB), so the max I can safely use for the cache disk is around 8GB. I have 32GB total.

Anyway, you might try it. No need for symlink and it will work with any program, not just ones you've configured to use the ramdrive.

Naturally you should always have a backup of any important data before using something like this.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
The following 2 users say Thank You to Big Mike for this post:
 
  #72 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,240 since Jun 2009
Thanks: 29,355 given, 83,237 received


Big Mike View Post
I've been trying out PrimoCache. Under most circumstances it makes more sense than a dedicated ramdisk, and will offer the same performance benefits.

I am using a delayed write setting of 60 seconds. I see NinjaTrader writing 50,000 blocks per minute during large backtests, so this can really help both in terms of performance as well as lifespan (SSD wear level).

I am using an 8GB cache and with my normal day to day stuff, my cache hit ratio is only around 25%. If I increased my cache size to 16GB I would have much higher ratio, as I've only read 16.4GB from disk total since my last reboot. The problem is, I need around 20GB of memory to start Sierra Chart when it is loading all my charts (once it loads them, usage is only 1GB), so the max I can safely use for the cache disk is around 8GB. I have 32GB total.

Anyway, you might try it. No need for symlink and it will work with any program, not just ones you've configured to use the ramdrive.

Naturally you should always have a backup of any important data before using something like this.

Mike

Hmm oops, math a bit wrong. Total read 16.5GB but cached read just under 4GB, so it means since boot I've 16GB from disk, but after the initial read was performed, only 4GB has been re-read and thus was delivered from the cache. So 8GB is plenty for my daily needs looks like, could even get by with less.

I think this is mainly attributed to Sierra Chart, because the data is roughly 50GB but they store it compressed (NTFS compressed) on disk, at about 18GB total. This is why on startup it balloons to 20GB of memory usage, due to Windows decompressing the data stream as it builds my charts. And since I usually only start Sierra Chart once every month or two (about as often as I reboot) then there is no real "caching", because it has to read it from disk once anyway to feed it to the cache, and I don't re-read it because I don't ever restart Sierra.

Your particular applications/daily use could be different.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #73 (permalink)
Elite Member
Brooklyn, New York/USA
 
Futures Experience: Intermediate
Platform: NinjaTrader
Favorite Futures: Stocks
 
Posts: 1 since Sep 2013
Thanks: 0 given, 0 received

Set Up


Hi,

I have installed dataram drive but do not know what settings to use speed up NT7. Where do I move the db file? Can someone list the steps?

Example

Basic Settings

*Disk Size 1.5MB*
*FAT16 Partition*

Format Options

*Create Temp Dir*

Load/Save

*??*

ETC....?

Any help would be appreciated.

Reply With Quote
 
  #74 (permalink)
Elite Member
Calgary, Canada
 
Futures Experience: Intermediate
Platform: Ninja Trader
Broker/Data: AMP - CQG
Favorite Futures: 6E, 6J, ES & TF
 
Posts: 5 since May 2011
Thanks: 3 given, 1 received

Speeding up NT-7

Thanks for the thread! I've been wondering what I could do to help speed things up with NT-7. I am running Win-7 Pro & have 32gb of RAM. Thinking of buying the latest extreme chip from Intel & up the RAM to 64.
I run about 140 charts imbedded into one chart for two different markets.

thanks,
CK


Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).







Reply With Quote
 
  #75 (permalink)
Trading Apprentice
Toronto,Canada
 
Futures Experience: Advanced
Platform: ninjatrader
Broker/Data: Ninjatrader
Favorite Futures: futures (CL and ES),forex,Options
 
Posts: 36 since Aug 2011
Thanks: 14 given, 12 received

NT feezing issues is fast markets

I have been experiencing freezing issues with NY using zenfire data in fast market environment.
I was told by NT support, that it is the 3rdparty indicators that are causing this issue.The developer is not totally convinced about that.
After reading the posts in this thread about using Ram disk, does it resolve those freezing issues in fast markets using zenfire?
I used to have 6 workspaces opened with all indicators when I was using IB filtered data without any issue, now with zenfire I am using a single workspace, removed most of my 3rdparty indicators, and still having issues

I am running windows 7 64 BT on 8 GB ram with Intel i7-3820 CPU @ 3.6 GHz

Any feedback and help on this is highly appreciated.

Thanks in advance
Guss

Reply With Quote
 
  #76 (permalink)
Elite Member
Toronto, Canada
 
Futures Experience: Advanced
Platform: NinjaTrader, MT4
Broker/Data: Amp/CQG, Citibank
Favorite Futures: CL, GC, 6E, ES, TF, EURUSD, GBPJPY, AUDUSD
 
Beljevina's Avatar
 
Posts: 349 since Sep 2010
Thanks: 379 given, 564 received


Guss View Post
I have been experiencing freezing issues with NY using zenfire data in fast market environment.
I was told by NT support, that it is the 3rdparty indicators that are causing this issue.The developer is not totally convinced about that.
After reading the posts in this thread about using Ram disk, does it resolve those freezing issues in fast markets using zenfire?
I used to have 6 workspaces opened with all indicators when I was using IB filtered data without any issue, now with zenfire I am using a single workspace, removed most of my 3rdparty indicators, and still having issues

I am running windows 7 64 BT on 8 GB ram with Intel i7-3820 CPU @ 3.6 GHz

It definitely sounds like indicators clogging things up. I've been here, and still kind of am today. The fact that you had no issues with IB 'condensed' or 'averaged' data is the telltale sign to me. Once you introduced a feed that is completely unfiltered, sending in all ticks - which increase significantly for often many instruments at the same time - as surges in volume (both buying and selling) come through in ES, CL, other indexes, etc - is the tip off to me.

I have not yet implemented this RAM disk thingy myself for other reasons, but I do not see it helping with the issue you have, as it is to speed loading of data from the SSD/harddisk (and writing), and I don't see the rush of market ticks or bid/ask data as a write throughput issue. When that data rushes in, it's NT doing something with it, that is causing the freezes, white screens/redraws, round blue "I'm busy" Windows cursor, etc.

I'm assuming most if not all indicators are set to 'Calculate on bar close = true'. Given the indicator removals you've mentioned and the drastic reduction in workspaces - on a highly capable CPU - I wonder if recreating & replacing each chart with a new chart that contains the exact same indicators would help, and then saving that as a completely new workspace. I mention this because I have had suspicions in the past where I feel there is something else 'clogging up' the chart in it's associated template XML file. I have seen too many quirky things with NT7 charts that are part of my workspace where, for example, I'm humming along day after day for weeks or months, and then all of a sudden, some indicators disappear off my chart. I have seen this occur on multiple machines and different installs. NT7 forums have reports like this, with the answer coming back 'pack up your log and template files' and send them to us. That is of course a time killing approach that realistically few undertake since, recreating or fixing the displayed chart is the quickest action. It's possible since the XML files are getting updated, parsed, etc by NT7, that it makes a mistake. And, if it can drop indicators, it can maybe be running others we're not seeing, or, having something else sucking up valuable CPU cycles. I have over the years undertaken this, and, it has cleared things up, for a time. Often, the 'clogging' returns over time, or, other unexplained issues start occurring.

I'm not looking to gripe, but it's a huge shortcoming that @NinjaTrader does not have some sort of utility to show CPU cycles used by each chart, or, say a breakdown by indicators. Hell, it is such a robust program, and if a much smaller memory footprint program like Chrome can offer it (via SHIFT-ESC), it's really upsetting that NT cant provide us an add on utility or build it into a release update. It would offer unbelievably untold benefit to us, as, we all have vastly different indicator needs & configurations.

One thing I am quite certain of, is that based on your described circumstances, there is no way you should be seeing that type of performance hit, which, ultimately points to a NT7 flaw, or inability to coexist with something, somewhere.

Reply With Quote
The following user says Thank You to Beljevina for this post:
 
  #77 (permalink)
Trading Apprentice
Toronto,Canada
 
Futures Experience: Advanced
Platform: ninjatrader
Broker/Data: Ninjatrader
Favorite Futures: futures (CL and ES),forex,Options
 
Posts: 36 since Aug 2011
Thanks: 14 given, 12 received

Thank you for your informative reply, much appreciated
will build a new workspace over the weekend, and test the performance again
Regards
Guss



Beljevina View Post
It definitely sounds like indicators clogging things up. I've been here, and still kind of am today. The fact that you had no issues with IB 'condensed' or 'averaged' data is the telltale sign to me. Once you introduced a feed that is completely unfiltered, sending in all ticks - which increase significantly for often many instruments at the same time - as surges in volume (both buying and selling) come through in ES, CL, other indexes, etc - is the tip off to me.

I have not yet implemented this RAM disk thingy myself for other reasons, but I do not see it helping with the issue you have, as it is to speed loading of data from the SSD/harddisk (and writing), and I don't see the rush of market ticks or bid/ask data as a write throughput issue. When that data rushes in, it's NT doing something with it, that is causing the freezes, white screens/redraws, round blue "I'm busy" Windows cursor, etc.

I'm assuming most if not all indicators are set to 'Calculate on bar close = true'. Given the indicator removals you've mentioned and the drastic reduction in workspaces - on a highly capable CPU - I wonder if recreating & replacing each chart with a new chart that contains the exact same indicators would help, and then saving that as a completely new workspace. I mention this because I have had suspicions in the past where I feel there is something else 'clogging up' the chart in it's associated template XML file. I have seen too many quirky things with NT7 charts that are part of my workspace where, for example, I'm humming along day after day for weeks or months, and then all of a sudden, some indicators disappear off my chart. I have seen this occur on multiple machines and different installs. NT7 forums have reports like this, with the answer coming back 'pack up your log and template files' and send them to us. That is of course a time killing approach that realistically few undertake since, recreating or fixing the displayed chart is the quickest action. It's possible since the XML files are getting updated, parsed, etc by NT7, that it makes a mistake. And, if it can drop indicators, it can maybe be running others we're not seeing, or, having something else sucking up valuable CPU cycles. I have over the years undertaken this, and, it has cleared things up, for a time. Often, the 'clogging' returns over time, or, other unexplained issues start occurring.

I'm not looking to gripe, but it's a huge shortcoming that @NinjaTrader does not have some sort of utility to show CPU cycles used by each chart, or, say a breakdown by indicators. Hell, it is such a robust program, and if a much smaller memory footprint program like Chrome can offer it (via SHIFT-ESC), it's really upsetting that NT cant provide us an add on utility or build it into a release update. It would offer unbelievably untold benefit to us, as, we all have vastly different indicator needs & configurations.

One thing I am quite certain of, is that based on your described circumstances, there is no way you should be seeing that type of performance hit, which, ultimately points to a NT7 flaw, or inability to coexist with something, somewhere.


Reply With Quote
 
  #78 (permalink)
Elite Member
Melbourne, Victoria, Australia
 
Futures Experience: None
Platform: TWS, NinjaTrader
Broker/Data: numerous
Favorite Futures: index and currency futures, stocks, options, warrants
 
kickmic's Avatar
 
Posts: 265 since May 2011
Thanks: 42 given, 336 received


Big Mike View Post
I've been trying out PrimoCache. Under most circumstances it makes more sense than a dedicated ramdisk, and will offer the same performance benefits.


Mike

Hi Mike,

Thanks for mentioning this option. I'll have a crack at setting this up later, if I get stuck I may have a couple of questions

Reply With Quote
 
  #79 (permalink)
Banned: trolling
NYC + NY / USA
 
Futures Experience: Advanced
Platform: "I trade, therefore, I AM!"; Theme Song: "Atomic Dog!"
Favorite Futures: EMD, 6J, ZB
 
kronie's Avatar
 
Posts: 798 since Oct 2009
Thanks: 216 given, 498 received


pawnbroker View Post
Robocopy is very fast if you run that frequently as a scheduled task, but it would be better to use real time synchronisation as that would reduce the loss of data to an absolute minimum. WIth real time sync, there would be no need for a slow "save on shutdown" option.

Primo Ramdisk has an option for real time replication of data to a hard drive, but I have not tested that yet. There are also products that do real time sync, such as BeyondSync and that could be used to keep a "safe" copy on a SSD. So you could create a RAM drive and use native sync with Primo or another application like BeyondSync with a freeware RAM drive.


A RAM drive can work well with NT, since that writes new data once and it read the same data many times. In addition, the amount of data written by NT and Gomi is low, so real time sync would not have an effect on performance.

If Visual Studio's performance has a bottle neck relating to the disk, you can try a RAM disk safely using Junction to point the existing directory to a replica on a RAM drive. If you don't gain a benefit, just delete the symbolic link on the C drive and restore the files from the RAM drive to the original location.

If it does give a large gain, try the Primo disk or BeyindSync or robocopy to keep a safe copy.

Full and Free Software Downloads » >Primo Ramdisk Professional Edition 5.1.0

just on this description page alone, this one, might actually be worth paying for, what with all the additional features that accompany the actual allocation and maintenance of the virtual ram disk

perhaps I'll report back on my findings

Reply With Quote
 
  #80 (permalink)
Elite Member
Milan Italy
 
Futures Experience: Intermediate
Platform: Ninjatrader, Sierra Chart
Favorite Futures: ES
 
Posts: 523 since Feb 2010
Thanks: 470 given, 629 received


Hi all,

I have a notebook with a SSD as the primary disk and a SD for some extra storage. Would it make sense to move the GOMFOLDER and NT db to the SD in order to save wear and tear on the SSD? And would it possibly make some difference to NT getting a bit sluggish in high volume bursts?

Thanks

Reply With Quote

Reply



futures io > > > > Speeding up NinjaTrader with a RAM drive (ramdisk)

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
.....SEC May Ticket Speeding Traders kbit News and Current Events 0 February 23rd, 2012 04:08 PM
NT7.4 - RAM hog kashter NinjaTrader 40 April 14th, 2011 02:10 PM
Drive cory Off-Topic 3 October 1st, 2010 04:17 PM
ram nt7 vps MyTraderAdvisor NinjaTrader 5 August 18th, 2010 04:49 PM
Toyota Blames Driver Error for Unwanted Speeding Quick Summary News and Current Events 0 July 14th, 2010 03:20 AM


All times are GMT -4. The time now is 09:40 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-17 in 0.18 seconds with 20 queries on phoenix via your IP 54.226.34.209