How can I convert a .csv file to text for import into NT? - NinjaTrader | futures io social day trading
futures io futures trading


How can I convert a .csv file to text for import into NT?
Updated: Views / Replies:10,146 / 17
Created: by gunsnmoney Attachments:6

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 6  
 
Thread Tools Search this Thread
 

How can I convert a .csv file to text for import into NT?

  #11 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,218 received

You can also look here:
https://futures.io/platforms-indicators/21616-data-converter-format-another-format.html

And ultimately, eventually, here:
https://futures.io/elite-circle/21502-standalone-tick-minute-data-converter-cross-platform.html

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #12 (permalink)
Elite Member
Crete, IL/USA
 
Futures Experience: Intermediate
Platform: NinjaTrader, Mt4
Broker/Data: Tradestation/Tradestation, NinjaTrader, FXCM and Tallinex
Favorite Futures: ES, CL, EUR/USD, TF
 
spinnybobo's Avatar
 
Posts: 171 since Aug 2009
Thanks: 99 given, 48 received


sburtt View Post
Hi spinnybobo,

thanks for taking the time to post this useful material. When trying to replicate what you are doing I have an issue. When I run the code in basicNinjaParse.jave , I get the following message error:

Error: Main method not found in class basicNinjaParse, please define the main method as:
public static void main(String[] args)

Are you able to help me to understand what I'm doing wrong

Thanks for your time, John

Hi John.
It cannot find the main method for some reason. Did you name it .java or .jave ?
Just remember that there is only 1 public class per .java file and the name of the .java file has to be exactly the same name of the public class. So, I have an example below of what everything would be if the 1st class is called public class Data. With this, the fail name is: Data.java

The 2nd file is public class getData. With this, the main method is inside of it and is the starting point of the program. The file name is getData.java

Here is the 1st class

 
Code
public class Data {
	
	private String Date;
	private String Open;
	private String High;
	private String Low;
	private String Close;
	private String Volume;
	
	//default constructor  having the same name of the class
	public Data(){
		this.Date = null;
		this.Open = null;
		this.High = null;
		this.Low = null;
		this.Close = null;
		this.Volume = null;
		
	}//regular constructor
	public Data(String Date, String Open, String High, String Low, String Close, String Volume)
	{
		this.Date = Date;
		this.Open = Open;
		this.High = High;
		this.Low = Low;
		this.Close = Close;
		this.Volume = Volume;
	}
	public String getDate(){
		return this.Date;
	}
	public String getOpen(){
		return this.Open;
	}
	public String getHigh(){
		return this.High;
	}
	public String getLow(){
		return this.Low;
	}
	public String getClose(){
		return this.Close;
	}
	public String getVolume(){
		return this.Volume;
	}
}
2nd class

 
Code
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.*;
import java.util.Vector;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.File;

//these will work once you import the jar file from www.csvreader.com 
import com.csvreader.CsvReader;
import com.csvreader.CsvWriter;


public class getData {

	private static Vector<Data> dailyData;
	
	public void inputData(){
		//the vector to hold the data
		dailyData = new Vector<Data>();
		try
		{
		CsvReader products = new CsvReader("C:\\AAPL.txt");
		products.readHeaders();
			while(products.readRecord())
			{
				dailyData.add(new Data(
					products.get("Date"),
					products.get("Open"),
					products.get("High"),
					products.get("Low"),
					products.get("Close"),
					products.get("Volume")));
				
				
			}
		products.close();
		}
		catch(FileNotFoundException e){
			e.printStackTrace();
		}
		catch(IOException e){
			e.printStackTrace();
		}
		
	}
	public void outputData(){
		//nothing yet
		for(int i = 0; i<dailyData.size();i++){
			System.out.println(dailyData.get(i).getDate()+";"+dailyData.get(i).getOpen()+";"+dailyData.get(i).getHigh()+";"+dailyData.get(i).getLow()+";"+dailyData.get(i).getClose()+";"+dailyData.get(i).getVolume());
		}
		String outputFile = "AAPL_ninjaTrader.txt";
		//check if file exists
		boolean alreadyExists = new File(outputFile).exists();
		try
		{
			CsvWriter csvOutput = new CsvWriter(new FileWriter(outputFile, true), ';');
			
			//if the file didn't already exist, then we need to write out the header line
			if(!alreadyExists)
			{
				csvOutput.write("Date");
				csvOutput.write("Open");
				csvOutput.write("High");
				csvOutput.write("Low");
				csvOutput.write("Close");
				csvOutput.write("Volume");
				csvOutput.endRecord();
			}
			//else assume that the file already has the correct header line
			//write out a few records
			for(int i = 0; i<dailyData.size();i++){
				csvOutput.write(dailyData.get(i).getDate());
				csvOutput.write(dailyData.get(i).getOpen());
				csvOutput.write(dailyData.get(i).getHigh());
				csvOutput.write(dailyData.get(i).getLow());
				csvOutput.write(dailyData.get(i).getClose());
				csvOutput.write(dailyData.get(i).getVolume());
				
				csvOutput.endRecord();
			}
			csvOutput.close();
		}
	
		catch(IOException e)
		{
			e.printStackTrace();
		}
	}
	public static void main(String args []){
		getData load = new getData();
		load.inputData();
		load.outputData();
		
	}
}
I uploaded a file called AAPL.TXT
Put this in the C://AAPL.TXT directory. Create a project in Eclipse called whatever you want. Add a class to the project called Data. Add another class called getData

Copy and paste both into Eclipse.

There are errors because the .jar file is not added. Go to the zip file attached and unzip. Go to the subdirectory of Eclipse project you created and create a folder called "lib".

Copy and paste the javacsv.jar to this folder. Now go back to Eclipse and expand the project from the Package Explorer view. Right click on project and hit "refresh" so you see the lib folder. Right click on top of JRE Package Explorer and go to Build Path, and then Configure Build Path. Add JARs... and find the lib folder in the project and then click on the javacsv.jar and press OK.

The errors should disappear. Left click on the Project folder in Eclipse and then find the Green button called Run. Hit it and it should run. If you go back to the subdirectory for Eclipse and go into your project, you should see a file called AAPL_ninjaTrader.txt

This has the formated data ready to be imported into Ninja Trader. Just erase the first line of the header and erase the space as well. It should work fine.

Let me know if you have a problem

Spencer

Attached Files
Register to download File Type: zip lib.zip (12.0 KB, 13 views)
Register to download File Type: txt AAPL.TXT (316.9 KB, 11 views)
Reply With Quote
 
  #13 (permalink)
Elite Member
London, UK
 
Futures Experience: Advanced
Platform: NinjaTrader
Favorite Futures: EUR/USD
 
Posts: 58 since Jul 2012
Thanks: 3 given, 4 received




Thanks Mike very useful, but I see there has been little progress up to now. However, it is a great idea, and I would really appreciate if somebody could develop a generic converter.

Reply With Quote
 
  #14 (permalink)
Elite Member
London, UK
 
Futures Experience: Advanced
Platform: NinjaTrader
Favorite Futures: EUR/USD
 
Posts: 58 since Jul 2012
Thanks: 3 given, 4 received

Thank you!


spinnybobo View Post
Hi John.
It cannot find the main method for some reason. Did you name it .java or .jave ?
Just remember that there is only 1 public class per .java file and the name of the .java file has to be exactly the same name of the public class. So, I have an example below of what everything would be if the 1st class is called public class Data. With this, the fail name is: Data.java

The 2nd file is public class getData. With this, the main method is inside of it and is the starting point of the program. The file name is getData.java

Here is the 1st class

 
Code
public class Data {
	
	private String Date;
	private String Open;
	private String High;
	private String Low;
	private String Close;
	private String Volume;
	
	//default constructor  having the same name of the class
	public Data(){
		this.Date = null;
		this.Open = null;
		this.High = null;
		this.Low = null;
		this.Close = null;
		this.Volume = null;
		
	}//regular constructor
	public Data(String Date, String Open, String High, String Low, String Close, String Volume)
	{
		this.Date = Date;
		this.Open = Open;
		this.High = High;
		this.Low = Low;
		this.Close = Close;
		this.Volume = Volume;
	}
	public String getDate(){
		return this.Date;
	}
	public String getOpen(){
		return this.Open;
	}
	public String getHigh(){
		return this.High;
	}
	public String getLow(){
		return this.Low;
	}
	public String getClose(){
		return this.Close;
	}
	public String getVolume(){
		return this.Volume;
	}
}
2nd class

 
Code
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.*;
import java.util.Vector;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.File;

//these will work once you import the jar file from www.csvreader.com 
import com.csvreader.CsvReader;
import com.csvreader.CsvWriter;


public class getData {

	private static Vector<Data> dailyData;
	
	public void inputData(){
		//the vector to hold the data
		dailyData = new Vector<Data>();
		try
		{
		CsvReader products = new CsvReader("C:\\AAPL.txt");
		products.readHeaders();
			while(products.readRecord())
			{
				dailyData.add(new Data(
					products.get("Date"),
					products.get("Open"),
					products.get("High"),
					products.get("Low"),
					products.get("Close"),
					products.get("Volume")));
				
				
			}
		products.close();
		}
		catch(FileNotFoundException e){
			e.printStackTrace();
		}
		catch(IOException e){
			e.printStackTrace();
		}
		
	}
	public void outputData(){
		//nothing yet
		for(int i = 0; i<dailyData.size();i++){
			System.out.println(dailyData.get(i).getDate()+";"+dailyData.get(i).getOpen()+";"+dailyData.get(i).getHigh()+";"+dailyData.get(i).getLow()+";"+dailyData.get(i).getClose()+";"+dailyData.get(i).getVolume());
		}
		String outputFile = "AAPL_ninjaTrader.txt";
		//check if file exists
		boolean alreadyExists = new File(outputFile).exists();
		try
		{
			CsvWriter csvOutput = new CsvWriter(new FileWriter(outputFile, true), ';');
			
			//if the file didn't already exist, then we need to write out the header line
			if(!alreadyExists)
			{
				csvOutput.write("Date");
				csvOutput.write("Open");
				csvOutput.write("High");
				csvOutput.write("Low");
				csvOutput.write("Close");
				csvOutput.write("Volume");
				csvOutput.endRecord();
			}
			//else assume that the file already has the correct header line
			//write out a few records
			for(int i = 0; i<dailyData.size();i++){
				csvOutput.write(dailyData.get(i).getDate());
				csvOutput.write(dailyData.get(i).getOpen());
				csvOutput.write(dailyData.get(i).getHigh());
				csvOutput.write(dailyData.get(i).getLow());
				csvOutput.write(dailyData.get(i).getClose());
				csvOutput.write(dailyData.get(i).getVolume());
				
				csvOutput.endRecord();
			}
			csvOutput.close();
		}
	
		catch(IOException e)
		{
			e.printStackTrace();
		}
	}
	public static void main(String args []){
		getData load = new getData();
		load.inputData();
		load.outputData();
		
	}
}
I uploaded a file called AAPL.TXT
Put this in the C://AAPL.TXT directory. Create a project in Eclipse called whatever you want. Add a class to the project called Data. Add another class called getData

Copy and paste both into Eclipse.

There are errors because the .jar file is not added. Go to the zip file attached and unzip. Go to the subdirectory of Eclipse project you created and create a folder called "lib".

Copy and paste the javacsv.jar to this folder. Now go back to Eclipse and expand the project from the Package Explorer view. Right click on project and hit "refresh" so you see the lib folder. Right click on top of JRE Package Explorer and go to Build Path, and then Configure Build Path. Add JARs... and find the lib folder in the project and then click on the javacsv.jar and press OK.

The errors should disappear. Left click on the Project folder in Eclipse and then find the Green button called Run. Hit it and it should run. If you go back to the subdirectory for Eclipse and go into your project, you should see a file called AAPL_ninjaTrader.txt

This has the formated data ready to be imported into Ninja Trader. Just erase the first line of the header and erase the space as well. It should work fine.

Let me know if you have a problem

Spencer

spinnybobo,
thank you very much! Sorry it took me time to revert, I've become a dad and the last 3 weeks have been very intense for me.. I've managed to run the program following your instruction.


If it doesn't bother you too much, I would really appreciate your help with an issue I have
.

I have a lot of 1minute data for different securities in the following format:

"Date","Time","O","H","L","C","U","D"
04/20/1998,0919,1.0982,1.0982,1.0982,1.0982,1,0
04/20/1998,0920,1.0982,1.0982,1.0982,1.0982,1,0
04/20/1998,0922,1.0980,1.0981,1.0980,1.0981,1,1
04/20/1998,0924,1.0983,1.0983,1.0983,1.0983,2,0

Would you be able to quickly change a few variables in the code you've sent me in order to recognize input data such as the one above (that for your guide is EUR/USD) and generate an output file able to run in ninjatrader?

I would really be thankful and happy to share the data I have if you would like.

Again thanks, John
input.txt

Reply With Quote
 
  #15 (permalink)
Elite Member
Crete, IL/USA
 
Futures Experience: Intermediate
Platform: NinjaTrader, Mt4
Broker/Data: Tradestation/Tradestation, NinjaTrader, FXCM and Tallinex
Favorite Futures: ES, CL, EUR/USD, TF
 
spinnybobo's Avatar
 
Posts: 171 since Aug 2009
Thanks: 99 given, 48 received


sburtt View Post
spinnybobo,
thank you very much! Sorry it took me time to revert, I've become a dad and the last 3 weeks have been very intense for me.. I've managed to run the program following your instruction.


If it doesn't bother you too much, I would really appreciate your help with an issue I have
.

I have a lot of 1minute data for different securities in the following format:

"Date","Time","O","H","L","C","U","D"
04/20/1998,0919,1.0982,1.0982,1.0982,1.0982,1,0
04/20/1998,0920,1.0982,1.0982,1.0982,1.0982,1,0
04/20/1998,0922,1.0980,1.0981,1.0980,1.0981,1,1
04/20/1998,0924,1.0983,1.0983,1.0983,1.0983,2,0

Would you be able to quickly change a few variables in the code you've sent me in order to recognize input data such as the one above (that for your guide is EUR/USD) and generate an output file able to run in ninjatrader?

I would really be thankful and happy to share the data I have if you would like.

Again thanks, John
Attachment 86838

Hey John,

Hey, congrats on becoming a Dad! that is awesome. wow, you must be a busy guy right now.
So, did you get your data from PiTrading.com ? I got a bunch of data from them for my Masters Thesis and wrote code to parse the data, but later learned they had a special Ninja Trader parser they built for their clients.

I ask because your data seems to be in the same format as their data you can just use their parser. You download the attached file, unzip it, then put the ascii2nt.exe on your desktop. Then the the input.txt file and drag it over the ascii2nt.exe file. Then take your finger off the mouse. Then you should see a file show up on your desktop called inputNT.exe

This is now in the NinjaTrader format. The other thing this parser does is it checks for logical errors like if the low is greater then the high. It basically changes it so it is logical, but it might not be the exact price that happened in the past. This is needed because ninja will not let you import a dataset if it has logical errors.

Also, you don't double click on the asii2nt.exe file. It is not a GUI (graphical user interface) and does not install. It just runs when you drop a text file over it. So it is portable.

Last, is the last line U and D important? So, I attached the output inputNT.txt and the last line should be volume and as you can see it took the 2nd to the last column and made that volume, and cut off the very last line.

If this is ok then great. If it is not and you wanted something different, then a program would have to be written for this. I did write something for 1 min parsing that I initially used, but later started using their parser.

thanks
Spencer

Attached Files
Register to download File Type: zip ascii2nt.zip (1.9 KB, 46 views)
Register to download File Type: txt inputNT.txt (458.1 KB, 43 views)
Reply With Quote
 
  #16 (permalink)
Elite Member
London, UK
 
Futures Experience: Advanced
Platform: NinjaTrader
Favorite Futures: EUR/USD
 
Posts: 58 since Jul 2012
Thanks: 3 given, 4 received


spinnybobo View Post
Hey John,

Hey, congrats on becoming a Dad! that is awesome. wow, you must be a busy guy right now.
So, did you get your data from PiTrading.com ? I got a bunch of data from them for my Masters Thesis and wrote code to parse the data, but later learned they had a special Ninja Trader parser they built for their clients.

I ask because your data seems to be in the same format as their data you can just use their parser. You download the attached file, unzip it, then put the ascii2nt.exe on your desktop. Then the the input.txt file and drag it over the ascii2nt.exe file. Then take your finger off the mouse. Then you should see a file show up on your desktop called inputNT.exe

This is now in the NinjaTrader format. The other thing this parser does is it checks for logical errors like if the low is greater then the high. It basically changes it so it is logical, but it might not be the exact price that happened in the past. This is needed because ninja will not let you import a dataset if it has logical errors.

Also, you don't double click on the asii2nt.exe file. It is not a GUI (graphical user interface) and does not install. It just runs when you drop a text file over it. So it is portable.

Last, is the last line U and D important? So, I attached the output inputNT.txt and the last line should be volume and as you can see it took the 2nd to the last column and made that volume, and cut off the very last line.

If this is ok then great. If it is not and you wanted something different, then a program would have to be written for this. I did write something for 1 min parsing that I initially used, but later started using their parser.

thanks
Spencer


Spencer it works! Thanks! This really saved me loads of time.. I see you're US based, I live in London, If you ever come to London please send me a message I will invite you out for dinner, it's the least I can do.

I'm not sure about the source of data, I has been given to me by a friend that works in a Bank (btw I work in a bank to), however I will ask him and let you know. Thanks

Reply With Quote
 
  #17 (permalink)
Elite Member
Crete, IL/USA
 
Futures Experience: Intermediate
Platform: NinjaTrader, Mt4
Broker/Data: Tradestation/Tradestation, NinjaTrader, FXCM and Tallinex
Favorite Futures: ES, CL, EUR/USD, TF
 
spinnybobo's Avatar
 
Posts: 171 since Aug 2009
Thanks: 99 given, 48 received


sburtt View Post
Spencer it works! Thanks! This really saved me loads of time.. I see you're US based, I live in London, If you ever come to London please send me a message I will invite you out for dinner, it's the least I can do.

I'm not sure about the source of data, I has been given to me by a friend that works in a Bank (btw I work in a bank to), however I will ask him and let you know. Thanks

Hey John

hey, glad it worked out for ya.
thanks, I will keep it in mind when it comes time for me to visit London :-)
until then, take care
Spencer

Reply With Quote
 
  #18 (permalink)
Patience my young padawan
Houston, TX/USA
 
Futures Experience: Intermediate
Platform: NinjaTrader TWS Barchart
Broker/Data: IB / IB
Favorite Futures: ES & looking into commodities
 
ElChacal's Avatar
 
Posts: 314 since Nov 2014
Thanks: 386 given, 185 received

@NinjaTrader could it be this easy instead?

https://www.multicharts.com/trading-software/index.php/OpenInt

Reply With Quote

Reply



futures io > > > > How can I convert a .csv file to text for import into NT?

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to avoid conflict while writing to text file from NT indicator zsrs NinjaTrader Programming 4 February 18th, 2014 03:06 AM
Simple template to convert NT Strategy to NT IUndicator visounds NinjaTrader Programming 4 March 31st, 2011 08:16 AM
Convert this NT code to EL, Please lizmerrill EasyLanguage Programming 8 January 20th, 2011 02:33 PM
When I try to input a zip file I get this import error .... skyfly NinjaTrader Programming 12 July 22nd, 2010 07:34 AM
Import NinjaScript File Warning...arrgh! gannman NinjaTrader 6 June 18th, 2010 03:30 AM


All times are GMT -4. The time now is 08:57 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-11 in 0.16 seconds with 20 queries on phoenix via your IP 54.83.122.227