Walk Forward Optimization - memory consumption - NinjaTrader Programming | futures.io
futures.io futures trading

Go Back   futures.io

> Futures Trading, News, Charts and Platforms > Platforms and Indicators > NinjaTrader > NinjaTrader Programming

Walk Forward Optimization - memory consumption
Started:August 6th, 2011 (06:51 AM) by Mondrakete Views / Replies:1,203 / 2
Last Reply:September 7th, 2011 (05:04 PM) Attachments:0

Welcome to futures.io.

Welcome, Guest!

This forum was established to help traders (especially futures traders) by openly sharing indicators, strategies, methods, trading journals and discussing the psychology of trading.

We are fundamentally different than most other trading forums:
  • We work extremely hard to keep things positive on our forums.
  • We do not tolerate rude behavior, trolling, or vendor advertising in posts.
  • We firmly believe in openness and encourage sharing. The holy grail is within you, it is not something tangible you can download.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community. It's free and simple, and we will never resell your private information.

-- Big Mike

Thread Tools Search this Thread

Walk Forward Optimization - memory consumption

Old August 6th, 2011, 06:51 AM   #1 (permalink)
Trading Apprentice
Futures Experience: Beginner
Platform: Ninja Trader
Favorite Futures: Stocks
Posts: 24 since Feb 2011
Thanks: 1 given, 0 received

Walk Forward Optimization - memory consumption


I'm trying to run a walk forward optimization on daily data from 1983 till now.
I try to simultaniously optimize 2 parameters (1. param: from 0,1 to 1; increment 0,1; 2nd param: 1,1 to 10, increment 0,1). Optimization period is 250 days, test period is 80 days.

During every attempt to do the optimization RAM memory is filled up until NT throws the OutOfMemoryException.

The reason for this obviously are indicators used within the strategy that read values from a txt file.
During OnStartUp() of the indi the txt file is read into a string array. This array is used afterwards in OnBarUpdate() for
indicator calculations. The array also contains daily data and each day is represented by one row of data. The array itself consists of 5 columns and 7000 lines. Does anybody know how to calculate the amount of memory that is consumed by this array? I'm actually surprised that it's big enough to freeze the system (32bit, Win XP, 3GB RAM).

Single backtesting this strategy works fine, using the GA during walk forward optim freezes the memory.

Anybody with experience and ideas in that field who knows how to reduce memory consumption?

Here comes the script that constructs the string array from the text-file.

public string[,] tabelle(string pfad, string trennzeichen)
      string[,] table={{"",""},{"",""}};
        string[] fileContent = File.ReadAllLines(pfad);
        int linecount=fileContent.GetLength(0);
        //Spaltenzahl bestimmen
        string[] datensatz=Regex.Split(fileContent[0], trennzeichen);
        int rowcount=0;
        foreach (string wort in datensatz)
        //table dimensionieren
        table = new string[linecount,rowcount];
        string zeile;
        for (int i = 0; i < linecount; i++) 
          datensatz=Regex.Split(zeile, trennzeichen);
          int j =0;
          foreach (string wort in datensatz)
          table[i,j]=datensatz[j];//Print(table[i,j]); //Datenmatrix erstellen
      catch (System.Exception excep)
      return table;
Within OnBarUpdate() relevant the relevant row of the array is avaluated and strings are transformed into double values.

Many thanks in advance for suggestions and help!

Reply With Quote

Old August 6th, 2011, 06:51 AM   #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.


Old September 7th, 2011, 05:04 PM   #3 (permalink)
Trading Apprentice
Denver, Colorado, USA
Futures Experience: Intermediate
Platform: Ninjatrader
Broker/Data: Kinetick
Favorite Futures: Forex
Posts: 36 since Aug 2011
Thanks: 6 given, 15 received

32bit vs 64bit

The main issue is probably that you are using 32bit NT and Windows. Its maxed out at around 3.25 GB maximum space in Windows 32bit, and a limit of .NET on your configuration is that it limits you to using 2 GB. NT is based on the .NET framework. You may want to consider upgrading to a fully 64bit trading system, since the RAM limit is quite high.

Memory Limits for Windows Releases (Windows)

Reply With Quote


futures.io > Futures Trading, News, Charts and Platforms > Platforms and Indicators > NinjaTrader > NinjaTrader Programming > Walk Forward Optimization - memory consumption

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Upcoming Webinars and Events (4:30PM ET unless noted)

An Afternoon with FIO trader bobwest

Elite only

NinjaTrader 8: Programming Profitable Trading Edges w/Scott Hodson

Elite only

Anthony Drager: Executing on Intermarket Correlations & Order Flow, Part 2

Elite only

Adam Grimes: Five critically important keys to professional trading

Elite only

Machine Learning Concepts w/FIO member NJAMC

Elite only

MarketDelta Cloud Platform: Announcing new mobile features

Dec 1

NinjaTrader 8: Features and Enhancements

Dec 6

Similar Threads
Thread Thread Starter Forum Replies Last Post
Walk Forward Experiment Silver Dragon Elite Automated Trading 327 September 8th, 2012 11:56 PM
Walk the Walk, traders showing their profits and sim/demo bluemele Traders Hideout 48 July 25th, 2011 05:43 AM
UN: Global Resource Consumption Will Triple kbit News and Current Events 0 May 13th, 2011 06:31 PM
Walk Forward caprica NinjaTrader 4 August 31st, 2009 05:51 PM
Memory Loss - Humor jonesr9 Off-Topic 2 August 12th, 2009 11:28 AM

All times are GMT -4. The time now is 02:04 PM.

Copyright © 2016 by futures.io. All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts

Page generated 2016-10-22 in 0.08 seconds with 19 queries on phoenix via your IP