Walk Forward Optimization - memory consumption - futures io
futures io

Walk Forward Optimization - memory consumption

Discussion in NinjaTrader

    1. trending_up 1,964 views
    2. thumb_up 0 thanks given
    3. group 2 followers
    1. forum 2 posts
    2. attach_file 0 attachments

Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

Search this Thread

Walk Forward Optimization - memory consumption

(login for full post details)
  #1 (permalink)
Posts: 24 since Feb 2011
Thanks: 1 given, 0 received


I'm trying to run a walk forward optimization on daily data from 1983 till now.
I try to simultaniously optimize 2 parameters (1. param: from 0,1 to 1; increment 0,1; 2nd param: 1,1 to 10, increment 0,1). Optimization period is 250 days, test period is 80 days.

During every attempt to do the optimization RAM memory is filled up until NT throws the OutOfMemoryException.

The reason for this obviously are indicators used within the strategy that read values from a txt file.
During OnStartUp() of the indi the txt file is read into a string array. This array is used afterwards in OnBarUpdate() for
indicator calculations. The array also contains daily data and each day is represented by one row of data. The array itself consists of 5 columns and 7000 lines. Does anybody know how to calculate the amount of memory that is consumed by this array? I'm actually surprised that it's big enough to freeze the system (32bit, Win XP, 3GB RAM).

Single backtesting this strategy works fine, using the GA during walk forward optim freezes the memory.

Anybody with experience and ideas in that field who knows how to reduce memory consumption?

Here comes the script that constructs the string array from the text-file.

public string[,] tabelle(string pfad, string trennzeichen)
      string[,] table={{"",""},{"",""}};
        string[] fileContent = File.ReadAllLines(pfad);
        int linecount=fileContent.GetLength(0);
        //Spaltenzahl bestimmen
        string[] datensatz=Regex.Split(fileContent[0], trennzeichen);
        int rowcount=0;
        foreach (string wort in datensatz)
        //table dimensionieren
        table = new string[linecount,rowcount];
        string zeile;
        for (int i = 0; i < linecount; i++) 
          datensatz=Regex.Split(zeile, trennzeichen);
          int j =0;
          foreach (string wort in datensatz)
          table[i,j]=datensatz[j];//Print(table[i,j]); //Datenmatrix erstellen
      catch (System.Exception excep)
      return table;
Within OnBarUpdate() relevant the relevant row of the array is avaluated and strings are transformed into double values.

Many thanks in advance for suggestions and help!

Reply With Quote

Can you help answer these questions
from other members on futures io?
For the experts of Metastock
Platforms and Indicators
1 min chart v-shape scan help please
Eurex Denali vs dxFeed
Sierra Chart
Experience with AGN (Introducing Broker)
MacdBB V402 NT804 + gradient + pullback not working on N …
(login for full post details)
  #3 (permalink)
Denver, Colorado, USA
Posts: 36 since Aug 2011
Thanks: 6 given, 17 received

The main issue is probably that you are using 32bit NT and Windows. Its maxed out at around 3.25 GB maximum space in Windows 32bit, and a limit of .NET on your configuration is that it limits you to using 2 GB. NT is based on the .NET framework. You may want to consider upgrading to a fully 64bit trading system, since the RAM limit is quite high.

Memory Limits for Windows Releases (Windows)

Reply With Quote

futures io Trading Community Platforms and Indicators NinjaTrader > Walk Forward Optimization - memory consumption

Last Updated on September 7, 2011

Upcoming Webinars and Events

NinjaTrader Indicator Challenge!


Your Charts Should Speak To You w/Bob Cometta

Elite only

Our 12-year anniversary w/ $$,$$$ prizes (check soon)


Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada), info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts