NexusFi: Find Your Edge


Home Menu

 





Can multiprocessors improve the speed of NT7 strategy optimization?


Discussion in NinjaTrader

Updated
    1. trending_up 3,569 views
    2. thumb_up 4 thanks given
    3. group 1 followers
    1. forum 16 posts
    2. attach_file 1 attachments




 
Search this Thread

Can multiprocessors improve the speed of NT7 strategy optimization?

  #1 (permalink)
cobrasmarketview
Toronto, ON, Canada
 
Posts: 9 since Sep 2014
Thanks Given: 0
Thanks Received: 2

I'm now running i7 6 core PC optimizing one of my trading systems. It takes whole night for one optimization. I checked, all 12 logical cores are used by NT7 when doing the optimization. CPU time up to 80%.

My question is: If I buy, say 2 processors, total 2*12 cores workstation, will it improve the speed? My guess is a 48 logical cores, should improve the optimization speed 4 times comparing to my current 12 logical cores machine. However, I'm a little afraid that NT cannot fully utilize the 2 processors, maybe it sill uses only 12 logic cores. So before I buy the machine, I hope someone here can confirm me that NT7 can fully take advantage of the multiprocessors machine when doing the optimization.

Thanks.

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
NexusFi Journal Challenge - April 2024
Feedback and Announcements
Exit Strategy
NinjaTrader
Futures True Range Report
The Elite Circle
Are there any eval firms that allow you to sink to your …
Traders Hideout
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
39 thanks
NexusFi site changelog and issues/problem reporting
26 thanks
The Program
18 thanks
GFIs1 1 DAX trade per day journal
18 thanks
  #2 (permalink)
 
ratfink's Avatar
 ratfink 
Birmingham UK
Market Wizard
 
Experience: Intermediate
Platform: NinjaTrader
Broker: TST/Rithmic
Trading: YM/Gold
Posts: 3,633 since Dec 2012
Thanks Given: 17,423
Thanks Received: 8,425


cobrasmarketview View Post
I'm now running i7 6 core PC optimizing one of my trading systems. It takes whole night for one optimization. I checked, all 12 logical cores are used by NT7 when doing the optimization. CPU time up to 80%.

My question is: If I buy, say 2 processors, total 2*12 cores workstation, will it improve the speed? My guess is a 48 logical cores, should improve the optimization speed 4 times comparing to my current 12 logical cores machine. However, I'm a little afraid that NT cannot fully utilize the 2 processors, maybe it sill uses only 12 logic cores. So before I buy the machine, I hope someone here can confirm me that NT7 can fully take advantage of the multiprocessors machine when doing the optimization.

Thanks.

My best guess is that it's bottle-necked on memory performance, i.e. MMU cache level busts and main board DDRN thoughput.

Travel Well
Visit my NexusFi Trade Journal Reply With Quote
  #3 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,396 since Jun 2009
Thanks Given: 33,172
Thanks Received: 101,536



cobrasmarketview View Post
I'm now running i7 6 core PC optimizing one of my trading systems. It takes whole night for one optimization. I checked, all 12 logical cores are used by NT7 when doing the optimization. CPU time up to 80%.

My question is: If I buy, say 2 processors, total 2*12 cores workstation, will it improve the speed? My guess is a 48 logical cores, should improve the optimization speed 4 times comparing to my current 12 logical cores machine. However, I'm a little afraid that NT cannot fully utilize the 2 processors, maybe it sill uses only 12 logic cores. So before I buy the machine, I hope someone here can confirm me that NT7 can fully take advantage of the multiprocessors machine when doing the optimization.

Thanks.

Short answer is no, there will be no benefit with NT7. See this thread:



That said, supposedly NT8 is completely redesigned and would see an advantage with more cores.

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
Thanked by:
  #4 (permalink)
cobrasmarketview
Toronto, ON, Canada
 
Posts: 9 since Sep 2014
Thanks Given: 0
Thanks Received: 2


Big Mike View Post
Short answer is no, there will be no benefit with NT7. See this thread:



That said, supposedly NT8 is completely redesigned and would see an advantage with more cores.

Mike

Thank you for the answer. But I only want to improve the optimization speed, not the charting or trading speed. According to you test here: #65 of the link you give me (I cannot post link yet), it seems that NT7 does use all 48 cores when doing the optimization. Unless I misunderstood your screenshot, it's not from NT7 test, it's the other software?

Reply With Quote
  #5 (permalink)
cobrasmarketview
Toronto, ON, Canada
 
Posts: 9 since Sep 2014
Thanks Given: 0
Thanks Received: 2

OK, I tested via Amazon Cloud, it's 36 cores. The screenshot shows clearly NT7 can use all the cores, well, at least 36 cores.

It didn't take 100% for all cores simply because the more threads, the more syn is needed, so if design the trading system well, i.e. avoid calling outside indicators, making all simple indicators, such as SMA, EMA, MACD inline, then my guess is it'd take more CPU time.

Attached Thumbnails
Click image for larger version

Name:	CPUTime.png
Views:	177
Size:	41.2 KB
ID:	157881  
Reply With Quote
  #6 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,396 since Jun 2009
Thanks Given: 33,172
Thanks Received: 101,536


cobrasmarketview View Post
OK, I tested via Amazon Cloud, it's 36 cores. The screenshot shows clearly NT7 can use all the cores, well, at least 36 cores.

It didn't take 100% for all cores simply because the more threads, the more syn is needed, so if design the trading system well, i.e. avoid calling outside indicators, making all simple indicators, such as SMA, EMA, MACD inline, then my guess is it'd take more CPU time.

Why not test it on the MACrossOver strategy included. It doesn't get much simpler. I believe you'll find the results disappointing with your 36 core test. What is the clock speed of these cores?

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #7 (permalink)
cobrasmarketview
Toronto, ON, Canada
 
Posts: 9 since Sep 2014
Thanks Given: 0
Thanks Received: 2


Big Mike View Post
Why not test it on the MACrossOver strategy included. It doesn't get much simpler. I believe you'll find the results disappointing with your 36 core test. What is the clock speed of these cores?

Mike

Yeah, the result is disappointing. The Amazon Cloud machine is E5 5680 v2 at 2.8G if I remember correctly.

Running one of my test on my i7 3930K 6 core needs 5 hours.
While running the same test on the 32 threads Amazon Cloud needs 4 hours.

So not much difference.

I'm setting a physical multiprocessors machine now to see if Amazon Cloud can reflect indeed what it clams to be E5 5680.

Reply With Quote
  #8 (permalink)
 
ratfink's Avatar
 ratfink 
Birmingham UK
Market Wizard
 
Experience: Intermediate
Platform: NinjaTrader
Broker: TST/Rithmic
Trading: YM/Gold
Posts: 3,633 since Dec 2012
Thanks Given: 17,423
Thanks Received: 8,425


cobrasmarketview View Post
Yeah, the result is disappointing. The Amazon Cloud machine is E5 5680 v2 at 2.8G if I remember correctly.

Running one of my test on my i7 3930K 6 core needs 5 hours.
While running the same test on the 32 threads Amazon Cloud needs 4 hours.

So not much difference.

I'm setting a physical multiprocessors machine now to see if Amazon Cloud can reflect indeed what it clams to be E5 5680.

Like I mentioned earlier, everybody forgets the RAM/MMU/cache performance issues and would rather ignore them. Cores are no more of a Holy Grail for OO system performance than anything is in trading.

The only thing the task manager shows you is that there are active threads in relation to the system idle thread. It tells you nothing about the state of the cpu core on behalf of the active thread and it tells you nothing about the utility or quality of work being achieved. [Assuming the Cloud machine was not shared of course.]

Given that OS/Ninja OO/C#/etcetera are fixed, for these cases the only measure you need be interested in is elapsed time.

I will be looking for improvements in NT8 but not miracles.

Travel Well
Visit my NexusFi Trade Journal Reply With Quote
Thanked by:
  #9 (permalink)
cobrasmarketview
Toronto, ON, Canada
 
Posts: 9 since Sep 2014
Thanks Given: 0
Thanks Received: 2


ratfink View Post
Like I mentioned earlier, everybody forgets the RAM/MMU/cache performance issues and would rather ignore them. Cores are no more of a Holy Grail for OO system performance than anything is in trading.

The only thing the task manager shows you is that there are active threads in relation to the system idle thread. It tells you nothing about the state of the cpu core on behalf of the active thread and it tells you nothing about the utility or quality of work being achieved. [Assuming the Cloud machine was not shared of course.]

Given that OS/Ninja OO/C#/etcetera are fixed, for these cases the only measure you need be interested in is elapsed time.

I will be looking for improvements in NT8 but not miracles.

I won't count on NT8 to improve much on optimization, it might be nothing to do with them, it's underlying, .net framework problem, I guess.

Reply With Quote
  #10 (permalink)
 
ratfink's Avatar
 ratfink 
Birmingham UK
Market Wizard
 
Experience: Intermediate
Platform: NinjaTrader
Broker: TST/Rithmic
Trading: YM/Gold
Posts: 3,633 since Dec 2012
Thanks Given: 17,423
Thanks Received: 8,425



cobrasmarketview View Post
I won't count on NT8 to improve much on optimization, it might be nothing to do with them, it's underlying, .net framework problem, I guess.

Not quite, the same applies to any general object-oriented architecture and language.

You can take all cores to 100% with multi-threaded software under NT7 as it stands, but it depends what the threads are doing. If they need random access to OO structures then the MMU caches will be busted nearly all the time, whereas if they only need sequential/local access then the cores can fly without stalling.

Travel Well
Visit my NexusFi Trade Journal Reply With Quote




Last Updated on September 25, 2014


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts