NexusFi: Find Your Edge


Home Menu

 





NinjaTrader with 16+ cores


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one Big Mike with 37 posts (33 thanks)
    2. looks_two JohnnyB with 17 posts (9 thanks)
    3. looks_3 NetTecture with 13 posts (2 thanks)
    4. looks_4 Eubie with 3 posts (0 thanks)
      Best Posters
    1. looks_one wavey with 1 thanks per post
    2. looks_two Big Mike with 0.9 thanks per post
    3. looks_3 JohnnyB with 0.5 thanks per post
    4. looks_4 NetTecture with 0.2 thanks per post
    1. trending_up 44,205 views
    2. thumb_up 52 thanks given
    3. group 26 followers
    1. forum 102 posts
    2. attach_file 20 attachments




 
Search this Thread

NinjaTrader with 16+ cores

  #51 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,463 since Jun 2009
Thanks Given: 33,239
Thanks Received: 101,662


TimeTrade View Post
You way is good, if you need ONE multicore system with one operating system...

Which I do most the time, and is superior for workstation use. Backtesting for example isn't useful if you have to split the job over multiple blade servers.

When I need virtualization, I will use VMWare Server under Server 2008 R2.

I also bet I built this quad opteron, 48 real core, system for only a tiny fraction of what those blades cost

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Started this thread Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Quant vue
Trading Reviews and Vendors
About a successful futures trader who didn´t know anyth …
Psychology and Money Management
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
Cheap historycal L1 data for stocks
Stocks and ETFs
MC PL editor upgrade
MultiCharts
 
  #52 (permalink)
 TimeTrade 
Erfurt Germany
 
Experience: Master
Platform: rtMagic, NinjaTrader
Broker: AMP/CQG, IAB
Trading: ES, 6E, FDAX, FGBL
Posts: 338 since Aug 2011
Thanks Given: 54
Thanks Received: 461


Big Mike View Post
Which I do most the time, and is superior for workstation use. Backtesting for example isn't useful if you have to split the job over multiple blade servers.

When I need virtualization, I will use VMWare Server under Server 2008 R2.

I also bet I built this quad opteron, 48 real core, system for only a tiny fraction of what those blades cost

Mike

the "new" price of bladecenters is very very expansive, my 2years old used centers cost here in germany ~2200USD

yes, i also use this with virtualized images and i have the advantage of more redundancy. 2 or 4 power supplies, 14 hot swappable blades, if one has an hard or software error, all other 13 other runnig without this problem

Reply With Quote
  #53 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,463 since Jun 2009
Thanks Given: 33,239
Thanks Received: 101,662




Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Started this thread Reply With Quote
Thanked by:
  #54 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,463 since Jun 2009
Thanks Given: 33,239
Thanks Received: 101,662


TimeTrade View Post
yes, i also use this with virtualized images and i have the advantage of more redundancy. 2 or 4 power supplies, 14 hot swappable blades, if one has an hard or software error, all other 13 other runnig without this problem

Yes of course, for servers. But no need for my workstation.

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Started this thread Reply With Quote
  #55 (permalink)
 JohnnyB 
Chicago, IL
 
Experience: Advanced
Platform: prop
Trading: Options
Posts: 38 since Jan 2012
Thanks Given: 23
Thanks Received: 18

Opterons will be a great build. I believe you're using MultiCharts and Sierra Charts. I can't wait to see how they utilize the 48 available cores. Please provide the details during the build and testing.

I'm envious of the new build but I like my SR-2s. I'll wait until Intel offers an unlocked E5 before building anything new.

Reply With Quote
  #56 (permalink)
 
whatnext's Avatar
 whatnext 
Rockland county , New York
 
Experience: Intermediate
Platform: NT 7
Trading: CL, 6E, SI, ZC
Posts: 230 since Mar 2011
Thanks Given: 338
Thanks Received: 81

(This is a little off topic, not sure where else to post it.)

I have spent a good amount of time trying to figure out the best single processor setup for trading with NT. Like trade station, NT utilizes less than 2 cores and doesn't benefit from hyper threading, as is not the case for MultiCharts or Investor R/T.

Currently using a HP Z210 Convertible Minitower Workstation with the xeon E3-1270 CPU – but may keep it as a back-up for the following build.

On some multi core benchmarks a stock SandyBridge i5-2500K is as fast as the Xeon version of the Bloomfield i7-960 (3550) overclocked to 4.4 ghz - even without the aid of multithreading. That's damn impressive.

The Bloomfield’s (45 nm thick) are over clocked by adjusting the BCLK and memory speed and when overclocked (often to 4.0-4.4 ghz) they are locked at that speed.

Sandy Bridge microarchitecture (35 nm thick) is overclocked, not by adjusting the BLCK (much if at all), but by adjusting the turbo boost. At stock an i7-2600K will go up to 3.8 ghz (core 1), 3.7 (core 2), 3.6 (core 3), 3.5 (core 4): while at full load. Overclocking can get all 4 cores to the same max speed you decide on.

The SandyBridge's idle at 1.6ghz and (unlike the Bloomfields) throttle up to whatever you overclocked it to when under full load. This keeps the thinner chip cooler, cleaner and less noisy. However, there is more of a delay in getting to the max frequency when it’s needed, compared to the Bloomfields.

I have found nothing about the real world difference for a program like NT, which usually only uses a few % of the processor once running, aside from backtesting. I think it could be significant though.

Turns out there is a way around this is: Disabling EIST, SpeedStep, C3 / C6 states and all other turbo/energy saving settings will lock the multiplier on SandyBridge LGA1155 processors to run at that speed, without the slower throttling up and down.

The hyperthreading feature adds about 7 degrees to the chip’s temperature. You can either disable this feature in windows for the 2600K/2700K i7 chips or save $100 and use an i5-2500K.

One person who used this method found the i5-2500K was stable during a 24 hour Prime95 stress test at around 4.4ghz / 1.30 volts with a $30 Cool Master 212 hyper CPU cooler. Not all motherboards allow this – the $300 Gigabyte P67A-UD7 rev 1.0 does though.

The following aftermarket non-water based CPU coolers (all are quite and under $100) offer about as good cooling as the Corsair H100 and are better than the H80 (full size case needed to fit them): Noctua NH-D14, Thermalright Silver Arrow, Prolimatech Super Mega HSF and Prolimatech Genesis.

The Genesis model also cools RAM and MOSFET. Both Prolimatech models don’t come with fans included – the Thermalright 140mm by 160mm EBR fans used on the Silver Arrow will work better than ones from Noctua and make less noise. For upgrading fans in the computer case, the Akasa APACHE S-Flow Fan Blade PWM 140mm will move more air more quietly than a Noctua 140mm and a COUGAR CF-V12HP 120mm beats a Noctua 120mm.

Phobya, Noiseblocker, Nanoxia, and (if you can find it) Feser Triebwerk all make good fans too – in finding a good compromise of acoustical noise dba, RPMs, static pressure and airflow / CFM (make sure the pin connector is correct).

For true water cooling the XSPC kits are popular – costing about $300. Rigging a fan in the case to blow over the mother board is good for cooling the other components’ when going with a water loop setup.

This is a good guide for overclocking Sandy Bridge chips – it’s a trial and error process: P67 Sandy Bridge Overclocking Guide For Beginners

"Be right and sit tight." - Jesse Livermore
Reply With Quote
  #57 (permalink)
 NetTecture 
Szczecin
 
Experience: Intermediate
Platform: Ninja, writing own now
Posts: 211 since Mar 2010

I think this is the main problem. The new Sandy Bridges are SO far ahead of AMD that even the PileDriver update wont make AMD even remotely competitive. Add Ivy Brige chips soonish and things get nasty. REALLY nasty.

LGA 2011 is compatible with Ivy Bridge as drop in update, so you also get future stability. I amdeeply impressed with my i7-3930 - that thing totally kills anything else I have around, and Opterons are sadly not a lot faster than the Phenom II which is years old. AMD dropped the ball. Majorly. Deadly, although I hope not - i really like AMD.

Need to get a server this year for trading / colo in USA. Renting now, will buy... once the Ivy Bridge are available.

Reply With Quote
  #58 (permalink)
 
Fat Tails's Avatar
 Fat Tails 
Berlin, Europe
Market Wizard
 
Experience: Advanced
Platform: NinjaTrader, MultiCharts
Broker: Interactive Brokers
Trading: Keyboard
Posts: 9,888 since Mar 2010
Thanks Given: 4,242
Thanks Received: 27,103

Running NinjaTrader 7 on a Xeon Quadcore under Windows 7 64-bit. The picture below shows the CPU utilization during a backtest run through an optimizer, which took about 20 minutes.

The CPU utilization of the 4 cores was around 80% over the whole period. The temperature of the processor went up to 65°C from a normal 35°C, CPU power consumption was 50W up from 15W.


Reply With Quote
  #59 (permalink)
 
whatnext's Avatar
 whatnext 
Rockland county , New York
 
Experience: Intermediate
Platform: NT 7
Trading: CL, 6E, SI, ZC
Posts: 230 since Mar 2011
Thanks Given: 338
Thanks Received: 81

The information on NT using less than 2 cores was given to me by falcon trading computes when they were building a system. The most I could get out of the very helpful NT staff was that 6 cores didn't beat 4.

My fault for assuming this also apples to backtesting - thanks FT.

I tried the multicharts quarter year try out (before they had a DOM) and couldn't find any info any where on how to use it. The sales rep who followed up to get the full sale was a deflective a-hole when I said "sure I'll get it - just need somewhere where I can figure out how to use it.

"Be right and sit tight." - Jesse Livermore
Reply With Quote
  #60 (permalink)
 
patbateman's Avatar
 patbateman 
NY, NY
 
Experience: None
Platform: T4, NinjaTrader
Trading: All!
Posts: 235 since Mar 2012
Thanks Given: 113
Thanks Received: 200



Big Mike View Post
Is anyone running NinjaTrader with more than 16 cores?

For example, a dual or quad Opteron 62xx (Interlagos) box?

Mike

Yes actually, I was running it on Amazon's EC2 and a couple other clusters. I think 32 or 64 cores, can't remember. I have screenshots and will share. I made a thread on the NT7 forums about why NT7 won't go past 30-40% cpu usage on larger systems. Other users with one large processor (i7 98xX) have experienced the same thing. I was a little harsh in my comments to them I think, but that's just because I really like the software and want it to be as professional as the more expensive products (like Bloomberg EMSX).

I'm going to be building a new system next quarter (possibly should start a thread then). I'm thinking most likely a server style board with two Xeon 5660s. It will be around twice as expensive as a "normal" i7 workstation, but in terms of compute power, it will be very inexpensive. The two things I want to optimize are screen area, cpu and disk usage. RAM will probably half use the board, and video usage is not a priority. An alternative possibility would be a microATX with the Silverstone Ft03s case, to save space, however that would throw any heavy cpu usage out the door.

Reply With Quote




Last Updated on October 1, 2014


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts