NexusFi: Find Your Edge


Home Menu

 





Walk Forward Optimization - memory consumption


Discussion in NinjaTrader

Updated
    1. trending_up 2,457 views
    2. thumb_up 0 thanks given
    3. group 2 followers
    1. forum 2 posts
    2. attach_file 0 attachments




 
Search this Thread

Walk Forward Optimization - memory consumption

  #1 (permalink)
Mondrakete
Germany
 
Posts: 24 since Feb 2011
Thanks Given: 1
Thanks Received: 0

Hallo,

I'm trying to run a walk forward optimization on daily data from 1983 till now.
I try to simultaniously optimize 2 parameters (1. param: from 0,1 to 1; increment 0,1; 2nd param: 1,1 to 10, increment 0,1). Optimization period is 250 days, test period is 80 days.

During every attempt to do the optimization RAM memory is filled up until NT throws the OutOfMemoryException.

The reason for this obviously are indicators used within the strategy that read values from a txt file.
During OnStartUp() of the indi the txt file is read into a string array. This array is used afterwards in OnBarUpdate() for
indicator calculations. The array also contains daily data and each day is represented by one row of data. The array itself consists of 5 columns and 7000 lines. Does anybody know how to calculate the amount of memory that is consumed by this array? I'm actually surprised that it's big enough to freeze the system (32bit, Win XP, 3GB RAM).

Single backtesting this strategy works fine, using the GA during walk forward optim freezes the memory.

Anybody with experience and ideas in that field who knows how to reduce memory consumption?

Here comes the script that constructs the string array from the text-file.

 
Code
public string[,] tabelle(string pfad, string trennzeichen)
    {
      //initialisieren
      string[,] table={{"",""},{"",""}};
      
      try
      {
                  
        string[] fileContent = File.ReadAllLines(pfad);
        int linecount=fileContent.GetLength(0);
        //Spaltenzahl bestimmen
        string[] datensatz=Regex.Split(fileContent[0], trennzeichen);
        int rowcount=0;
        foreach (string wort in datensatz)
        {rowcount++;}
                    
        //table dimensionieren
        table = new string[linecount,rowcount];
          
        string zeile;
          
        for (int i = 0; i < linecount; i++) 
        {
          zeile=fileContent[i];
          datensatz=Regex.Split(zeile, trennzeichen);
            
          int j =0;
          foreach (string wort in datensatz)
          {
          table[i,j]=datensatz[j];//Print(table[i,j]); //Datenmatrix erstellen
          j=j+1;
          }
        }
      }
      catch (System.Exception excep)
        {
          MessageBox.Show(excep.Message);  
        }
        
      return table;
    }
Within OnBarUpdate() relevant the relevant row of the array is avaluated and strings are transformed into double values.

Many thanks in advance for suggestions and help!

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
ZombieSqueeze
Platforms and Indicators
Trade idea based off three indicators.
Traders Hideout
Exit Strategy
NinjaTrader
Pivot Indicator like the old SwingTemp by Big Mike
NinjaTrader
How to apply profiles
Traders Hideout
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Spoo-nalysis ES e-mini futures S&P 500
29 thanks
Just another trading journal: PA, Wyckoff & Trends
25 thanks
Tao te Trade: way of the WLD
24 thanks
Bigger Wins or Fewer Losses?
23 thanks
GFIs1 1 DAX trade per day journal
18 thanks
  #3 (permalink)
Antisyzygy
Denver, Colorado, USA
 
Posts: 36 since Aug 2011
Thanks Given: 6
Thanks Received: 17


The main issue is probably that you are using 32bit NT and Windows. Its maxed out at around 3.25 GB maximum space in Windows 32bit, and a limit of .NET on your configuration is that it limits you to using 2 GB. NT is based on the .NET framework. You may want to consider upgrading to a fully 64bit trading system, since the RAM limit is quite high.

Memory Limits for Windows Releases (Windows)

Reply With Quote




Last Updated on September 7, 2011


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts