how can I export ES indiv contracts (from NT7) as 1 continuous csv? - NinjaTrader | futures io social day trading
futures io futures trading


how can I export ES indiv contracts (from NT7) as 1 continuous csv?
Updated: Views / Replies:483 / 7
Created: by Wil70 Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

how can I export ES indiv contracts (from NT7) as 1 continuous csv?

  #1 (permalink)
Elite Member
SF (CA/USA)
 
Futures Experience: Beginner
Platform: NT / Proprietary
Broker/Data: IB / may be Kinetic
Favorite Futures: ES, CL, NQ, YM, 6E
 
Posts: 24 since Nov 2015
Thanks: 161 given, 5 received

how can I export ES indiv contracts (from NT7) as 1 continuous csv?

Hello, how can I export NT7 tick data from none continuous contract as 1 csv continuous contract ?

Note: I downloaded some continuous and none continuous contract, I can export both but I found a bug, I notice as I download new tick data the old data are being removed (from the continuous contract)....so now I Can not rely on my ES ##-## as it is missing some older data I did have, and if I export the last current ES individual contract then this is all I have in the file, I would like to export all ticks I have from ES as 1 continuous contract.

thanks for your help
w

Reply With Quote
 
  #2 (permalink)
Elite Member
SF (CA/USA)
 
Futures Experience: Beginner
Platform: NT / Proprietary
Broker/Data: IB / may be Kinetic
Favorite Futures: ES, CL, NQ, YM, 6E
 
Posts: 24 since Nov 2015
Thanks: 161 given, 5 received

My goal is to export 1 continuous contracts file with only ticks.
As there is a known bug when downloading a continuous contract (like they overwrite older data) with NT UI, I decided to download individual contracts, then to export their ticks by running a backtest (with a slightly modified version of the SampleStreamwritter code from the documentation so it can handle ticks)
But i always end up having this:


"Failed to call method 'Initialize' for indicator 'SampleStreamWriter': The process cannot access the file 'C:\Users\Wilhelm\Documents\NinjaTrader 7\MyTestFile.txt' because it is being used by another process.
Failed to call method 'Initialize' for indicator 'SampleStreamWriter': The process cannot access the file 'C:\Users\Wilhelm\Documents\NinjaTrader 7\MyTestFile.txt' because it is being used by another process."

 
Code
        protected override void Initialize()
        {
            CalculateOnBarClose	= true;
			path= Cbi.Core.UserDataDir.ToString() + this.Instrument.FullName +".txt";
            Console.Out.WriteLine(path);
            sw = File.AppendText(path);
        }
		
        /// <summary>
        /// Called on each bar update event (incoming tick)
        /// </summary>
        protected override void OnBarUpdate()
        {

        }
		
        protected override void OnMarketData(MarketDataEventArgs e)
        {
            if (e.MarketDataType == MarketDataType.Last)				
            {
				sw.WriteLine(ToDay(e.Time) + "," + e.Price);
             }
        }
		// Necessary to call in order to clean up resources used by the StreamWriter object
		protected override void OnTermination() 
		{
			// Disposes resources used by the StreamWriter
			if (sw != null)
			{
				sw.Dispose();
				sw = null;
			}
		}
What other process is locking the file?
Is there another way to export tick data out of NT7 in a csv file?

Thanks.

Reply With Quote
 
  #3 (permalink)
Administrator: Retired Backtester
 Vendor: speedytradingservers.com 
Rennes France
 
Futures Experience: Advanced
Platform: NinjaTrader
Broker/Data: IB/Kinetick
Favorite Futures: Futures
 
sam028's Avatar
 
Posts: 3,363 since Jun 2009
Thanks: 3,566 given, 3,976 received


Maybe another Ninja thread, if you forgot the Dispose() in a previous version?
The easier is to export from Tools -> Historical Data Manager , I don't see why you'll like to use a strategy to do this.

Success requires no deodorant! (Sun Tzu)
Reply With Quote
 
  #4 (permalink)
Elite Member
SF (CA/USA)
 
Futures Experience: Beginner
Platform: NT / Proprietary
Broker/Data: IB / may be Kinetic
Favorite Futures: ES, CL, NQ, YM, 6E
 
Posts: 24 since Nov 2015
Thanks: 161 given, 5 received


sam028 View Post
Maybe another Ninja thread, if you forgot the Dispose() in a previous version?
The easier is to export from Tools -> Historical Data Manager , I don't see why you'll like to use a strategy to do this.

Thanks Sam, my goal is to export all my ZB ticks from jan 2015 to now in a csv file.
Tools -> Historical Data Manager won't work because this imply I use the continuous contract of ZB (to have a nice back adjusted serie), but when I download new data for ZB ##-## from Continuum it always keep 1.5 years of date and overwrite my older data in ZB ##-## (like let say I have Jan2015 to June 2016, then I decide to download all the data missing from now to June 2016, it will erase Jan 2015 to April2015, so I will have left 1.5 years of tick data (from now to May 2015))...basically older data disappear with ZB##-## but they do not disappear in the individual contracts.
So I'm basically looking for a way to export those ticks as one continuous contract by using the individual contracts so I do not have the issue of disappearance of older ticks...The idea was to backtest with a little script from ZB 12-16 so I got a nice continuous contract that goes back to jan 2015. But I have this error of file locked by another process, even after restarting the NT and the machine

Reply With Quote
 
  #5 (permalink)
Administrator: Retired Backtester
 Vendor: speedytradingservers.com 
Rennes France
 
Futures Experience: Advanced
Platform: NinjaTrader
Broker/Data: IB/Kinetick
Favorite Futures: Futures
 
sam028's Avatar
 
Posts: 3,363 since Jun 2009
Thanks: 3,566 given, 3,976 received

Answer
This post has been selected as an answer to the original posters question Answer


Wil70 View Post
Thanks Sam, my goal is to export all my ZB ticks from jan 2015 to now in a csv file.
Tools -> Historical Data Manager won't work because this imply I use the continuous contract of ZB (to have a nice back adjusted serie), but when I download new data for ZB ##-## from Continuum it always keep 1.5 years of date and overwrite my older data in ZB ##-## (like let say I have Jan2015 to June 2016, then I decide to download all the data missing from now to June 2016, it will erase Jan 2015 to April2015, so I will have left 1.5 years of tick data (from now to May 2015))...basically older data disappear with ZB##-## but they do not disappear in the individual contracts.
So I'm basically looking for a way to export those ticks as one continuous contract by using the individual contracts so I do not have the issue of disappearance of older ticks...The idea was to backtest with a little script from ZB 12-16 so I got a nice continuous contract that goes back to jan 2015. But I have this error of file locked by another process, even after restarting the NT and the machine

You can find which process is using this file in opening the Windows Resource Monitor, from the CPU tab. Select all the processes, then take a look at the "Associated Handles", sort by "Handle name" and search for your .txt file.

I did something similar in the past and it worked, I used a sw.Close() instead of sw.Dispose() but it should do the same thing.

Success requires no deodorant! (Sun Tzu)
Reply With Quote
 
  #6 (permalink)
Elite Member
SF (CA/USA)
 
Futures Experience: Beginner
Platform: NT / Proprietary
Broker/Data: IB / may be Kinetic
Favorite Futures: ES, CL, NQ, YM, 6E
 
Posts: 24 since Nov 2015
Thanks: 161 given, 5 received


sam028 View Post
You can find which process is using this file in opening the Windows Resource Monitor, from the CPU tab. Select all the processes, then take a look at the "Associated Handles", sort by "Handle name" and search for your .txt file.

I did something similar in the past and it worked, I used a sw.Close() instead of sw.Dispose() but it should do the same thing.

Yeah working! Thanks Sam, I had 2 mistakes:
1- I didn't use close as you did, I just used Dispose, now I use both...working!
2- I move the export code in OnBarUpdate instead of OnMarketData because when you backtest, even if it is tick by tick, it will create bars of 1 tick.....(OnMarketData is never call)
Thanks again
Best
w

Note to future reader: if you know a better way to do it, please do not hesitate to post here. thanks.


Last edited by Wil70; October 31st, 2016 at 07:02 PM.
Reply With Quote
 
  #7 (permalink)
Administrator: Retired Backtester
 Vendor: speedytradingservers.com 
Rennes France
 
Futures Experience: Advanced
Platform: NinjaTrader
Broker/Data: IB/Kinetick
Favorite Futures: Futures
 
sam028's Avatar
 
Posts: 3,363 since Jun 2009
Thanks: 3,566 given, 3,976 received


Wil70 View Post
Yeah working! Thanks Sam, I had 2 mistakes:
1- I didn't use close as you did, I just used Dispose, now I use both...working!
2- I move the export code in OnBarUpdate instead of OnMarketData because when you backtest, even if it is tick by tick, it will create bars of 1 tick.....(OnMarketData is never call)
Thanks again
Best
w

Note to future reader: if you know a better way to do it, please do not hesitate to post here. thanks.

Dispose() should close the file too, but if you want to add both...
Correct, OnMarketData will be called after each order book updates, it's useless in your case.

Success requires no deodorant! (Sun Tzu)

Last edited by sam028; October 31st, 2016 at 07:33 PM. Reason: typo
Reply With Quote
 
  #8 (permalink)
 Vendor: www.probabletrades.com 
OC, California, USA
 
Futures Experience: Advanced
Platform: IB/TWS, NinjaTrader, thinkorswim
Favorite Futures: stocks, options, futures, VIX
 
shodson's Avatar
 
Posts: 1,858 since Jun 2009
Thanks: 480 given, 3,247 received

You should wrap your File.AppendText() calls around a using block. This will handle all of the disposing and resource releasing automatically once the block is exited. See the example code at https://msdn.microsoft.com/en-us/library/system.io.file.appendtext(v=vs.110).aspx

 
Code
using (StreamWriter sw = File.AppendText(path))
{
   sw.WriteLine(ToDay(e.Time) + "," + e.Price);
}
And you never have to call dispose()

If that feels like too much file opening and closing/releasing, consider using a StringBuilder and writing it out to the file OnTermination()

 
Code
private StringBuilder sb = new StringBuilder();
then in OnMarketData()

 
Code
sb.AppendLine(ToDay(e.Time) + "," + e.Price);
Then in OnTermination()

 
Code
using (StreamWriter sw = File.AppendText(path))
{
   sw.WriteLine(sb.ToString());
}
if you're writing lots of data that string could get really big, maybe check the size of it and write it to disk and clear it mid-stream like every megabyte, in OnMarketData()

 
Code
sb.AppendLine(ToDay(e.Time) + "," + e.Price);
if (sb.Length > 1024 * 1024) {
   using (StreamWriter sw = File.AppendText(path))
   {
      sw.WriteLine(sb.ToString());
   }
   sb.Clear();
}

Reply With Quote

Reply



futures io > > > > how can I export ES indiv contracts (from NT7) as 1 continuous csv?

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

Dec 7

Linda Bradford Raschke: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
NinjaTrader 7 Continuous Contracts grausch NinjaTrader 2 September 19th, 2016 08:33 AM
Continuous Contracts steve2222 Terms (Glossary) 0 July 4th, 2016 04:04 AM
Export chart data plotted from signal (Excel/CSV/etc)? cestnoir MultiCharts 1 April 12th, 2016 06:23 AM
ZigZag indicator export to txt/csv PK007 EasyLanguage Programming 2 January 21st, 2015 02:49 PM
multitimeframe live CSV export rirhasiu NinjaTrader 5 May 16th, 2013 03:15 PM


All times are GMT -4. The time now is 01:24 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-11-25 in 0.13 seconds with 35 queries on phoenix via your IP 54.80.146.251