Walk Forward Optimization - memory consumption - futures io
futures io



Walk Forward Optimization - memory consumption


Discussion in NinjaTrader

Updated
    1. trending_up 1,964 views
    2. thumb_up 0 thanks given
    3. group 2 followers
    1. forum 2 posts
    2. attach_file 0 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Walk Forward Optimization - memory consumption

(login for full post details)
  #1 (permalink)
Mondrakete
Germany
 
 
Posts: 24 since Feb 2011
Thanks: 1 given, 0 received

Hallo,

I'm trying to run a walk forward optimization on daily data from 1983 till now.
I try to simultaniously optimize 2 parameters (1. param: from 0,1 to 1; increment 0,1; 2nd param: 1,1 to 10, increment 0,1). Optimization period is 250 days, test period is 80 days.

During every attempt to do the optimization RAM memory is filled up until NT throws the OutOfMemoryException.

The reason for this obviously are indicators used within the strategy that read values from a txt file.
During OnStartUp() of the indi the txt file is read into a string array. This array is used afterwards in OnBarUpdate() for
indicator calculations. The array also contains daily data and each day is represented by one row of data. The array itself consists of 5 columns and 7000 lines. Does anybody know how to calculate the amount of memory that is consumed by this array? I'm actually surprised that it's big enough to freeze the system (32bit, Win XP, 3GB RAM).

Single backtesting this strategy works fine, using the GA during walk forward optim freezes the memory.

Anybody with experience and ideas in that field who knows how to reduce memory consumption?

Here comes the script that constructs the string array from the text-file.

 
Code
public string[,] tabelle(string pfad, string trennzeichen)
    {
      //initialisieren
      string[,] table={{"",""},{"",""}};
      
      try
      {
                  
        string[] fileContent = File.ReadAllLines(pfad);
        int linecount=fileContent.GetLength(0);
        //Spaltenzahl bestimmen
        string[] datensatz=Regex.Split(fileContent[0], trennzeichen);
        int rowcount=0;
        foreach (string wort in datensatz)
        {rowcount++;}
                    
        //table dimensionieren
        table = new string[linecount,rowcount];
          
        string zeile;
          
        for (int i = 0; i < linecount; i++) 
        {
          zeile=fileContent[i];
          datensatz=Regex.Split(zeile, trennzeichen);
            
          int j =0;
          foreach (string wort in datensatz)
          {
          table[i,j]=datensatz[j];//Print(table[i,j]); //Datenmatrix erstellen
          j=j+1;
          }
        }
      }
      catch (System.Exception excep)
        {
          MessageBox.Show(excep.Message);  
        }
        
      return table;
    }
Within OnBarUpdate() relevant the relevant row of the array is avaluated and strings are transformed into double values.

Many thanks in advance for suggestions and help!

Reply With Quote

Can you help answer these questions
from other members on futures io?
For the experts of Metastock
Platforms and Indicators
1 min chart v-shape scan help please
ThinkOrSwim
Eurex Denali vs dxFeed
Sierra Chart
Experience with AGN (Introducing Broker)
Brokers
MacdBB V402 NT804 + gradient + pullback not working on N …
NinjaTrader
 
 
(login for full post details)
  #3 (permalink)
Antisyzygy
Denver, Colorado, USA
 
 
Posts: 36 since Aug 2011
Thanks: 6 given, 17 received


The main issue is probably that you are using 32bit NT and Windows. Its maxed out at around 3.25 GB maximum space in Windows 32bit, and a limit of .NET on your configuration is that it limits you to using 2 GB. NT is based on the .NET framework. You may want to consider upgrading to a fully 64bit trading system, since the RAM limit is quite high.

Memory Limits for Windows Releases (Windows)

Reply With Quote


futures io Trading Community Platforms and Indicators NinjaTrader > Walk Forward Optimization - memory consumption


Last Updated on September 7, 2011


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing

Your Charts Should Speak To You w/Bob Cometta

Elite only
 

Our 12-year anniversary w/ $$,$$$ prizes (check soon)

August
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada), info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts