Walk Forward Optimization - memory consumption - NinjaTrader Programming | futures io social day trading
futures io futures trading


Walk Forward Optimization - memory consumption
Updated: Views / Replies:1,437 / 2
Created: by Mondrakete Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Walk Forward Optimization - memory consumption

  #1 (permalink)
Trading Apprentice
Germany
 
Futures Experience: Beginner
Platform: Ninja Trader
Favorite Futures: Stocks
 
Posts: 24 since Feb 2011
Thanks: 1 given, 0 received

Walk Forward Optimization - memory consumption

Hallo,

I'm trying to run a walk forward optimization on daily data from 1983 till now.
I try to simultaniously optimize 2 parameters (1. param: from 0,1 to 1; increment 0,1; 2nd param: 1,1 to 10, increment 0,1). Optimization period is 250 days, test period is 80 days.

During every attempt to do the optimization RAM memory is filled up until NT throws the OutOfMemoryException.

The reason for this obviously are indicators used within the strategy that read values from a txt file.
During OnStartUp() of the indi the txt file is read into a string array. This array is used afterwards in OnBarUpdate() for
indicator calculations. The array also contains daily data and each day is represented by one row of data. The array itself consists of 5 columns and 7000 lines. Does anybody know how to calculate the amount of memory that is consumed by this array? I'm actually surprised that it's big enough to freeze the system (32bit, Win XP, 3GB RAM).

Single backtesting this strategy works fine, using the GA during walk forward optim freezes the memory.

Anybody with experience and ideas in that field who knows how to reduce memory consumption?

Here comes the script that constructs the string array from the text-file.

 
Code
public string[,] tabelle(string pfad, string trennzeichen)
    {
      //initialisieren
      string[,] table={{"",""},{"",""}};
      
      try
      {
                  
        string[] fileContent = File.ReadAllLines(pfad);
        int linecount=fileContent.GetLength(0);
        //Spaltenzahl bestimmen
        string[] datensatz=Regex.Split(fileContent[0], trennzeichen);
        int rowcount=0;
        foreach (string wort in datensatz)
        {rowcount++;}
                    
        //table dimensionieren
        table = new string[linecount,rowcount];
          
        string zeile;
          
        for (int i = 0; i < linecount; i++) 
        {
          zeile=fileContent[i];
          datensatz=Regex.Split(zeile, trennzeichen);
            
          int j =0;
          foreach (string wort in datensatz)
          {
          table[i,j]=datensatz[j];//Print(table[i,j]); //Datenmatrix erstellen
          j=j+1;
          }
        }
      }
      catch (System.Exception excep)
        {
          MessageBox.Show(excep.Message);  
        }
        
      return table;
    }
Within OnBarUpdate() relevant the relevant row of the array is avaluated and strings are transformed into double values.

Many thanks in advance for suggestions and help!

Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Trading Apprentice
Denver, Colorado, USA
 
Futures Experience: Intermediate
Platform: Ninjatrader
Broker/Data: Kinetick
Favorite Futures: Forex
 
Posts: 36 since Aug 2011
Thanks: 6 given, 17 received

32bit vs 64bit


The main issue is probably that you are using 32bit NT and Windows. Its maxed out at around 3.25 GB maximum space in Windows 32bit, and a limit of .NET on your configuration is that it limits you to using 2 GB. NT is based on the .NET framework. You may want to consider upgrading to a fully 64bit trading system, since the RAM limit is quite high.

Memory Limits for Windows Releases (Windows)

Reply With Quote

Reply



futures io > > > > > Walk Forward Optimization - memory consumption

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Walk Forward Experiment Silver Dragon Elite Automated Trading 327 September 9th, 2012 12:56 AM
Walk the Walk, traders showing their profits and sim/demo bluemele Traders Hideout 48 July 25th, 2011 06:43 AM
UN: Global Resource Consumption Will Triple kbit News and Current Events 0 May 13th, 2011 07:31 PM
Walk Forward caprica NinjaTrader 4 August 31st, 2009 06:51 PM
Memory Loss - Humor jonesr9 Off-Topic 2 August 12th, 2009 12:28 PM


All times are GMT -4. The time now is 10:19 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-10 in 0.07 seconds with 19 queries on phoenix via your IP 54.221.73.186