NexusFi: Find Your Edge


Home Menu

 





here's way to convert tick data from NT to comma-separated format, with Mac or Linux


Discussion in Platforms and Indicators

Updated
    1. trending_up 2,132 views
    2. thumb_up 2 thanks given
    3. group 1 followers
    1. forum 1 posts
    2. attach_file 0 attachments




 
Search this Thread

here's way to convert tick data from NT to comma-separated format, with Mac or Linux

  #1 (permalink)
 theorist 
Los Angeles, California, USA
 
Experience: Beginner
Platform: TradeStation, AmiBroker
Trading: ES
Posts: 9 since Jun 2012
Thanks Given: 17
Thanks Received: 11

If you have access to a Mac or Linux box, here's a single line of shell code (using awk) that will convert a tick file from NT (Ninja Trader) format to comma-separated (just substitute the name of your NT-formatted input file, and the desired name of your comma-formatted output file):


awk '{print $1"," $2}' input_file.txt | awk -F ";" '{print $1"," $2","$3}' > output_file.txt

For instance, if this is the contents of the input file:

20090610 153000;00936.00;21
20090610 153000;00935.75;1
20090610 153010;00935.50;1
...

Then this will be the contents of the output file:

20090610,153000,00936.00,21
20090610,153000,00935.75,1
20090610,153010,00935.50,1
.....

The input file will be untouched.

Awk is very powerful, and can do many other data file manipulations as well.

Here's how you would actually use it on a Mac:
Create a new folder directly under your home directory (it can be anywhere, but this makes it easy). Let's assume you name this folder "data"; save your input file there.
Open the Terminal application (Applications->Utilities->Terminal).
At the shell prompt (which ends with "$" -- not to be confused with the "$" in awk!), type the following, and hit return:
cd data
This will put you in "data" (note it's case-sensitive) ("cd" means "change directory")
To confirm the file is there, type the following and hit return ("ls" means "list"):
ls
This gives you a list of all the files and subfolders within data
Then enter the awk command given above and hit return.

It's the same on Linux, once you open a shell.

For those of you who are curious, here's how it works:

$N in awk means the Nth field in a line. In looking at the NT-formatted input file shown above, awk defaults to thinking there are two fields in each line (because we haven't specified a field separator, it takes the space as the separator). So the first part of the command, awk '{print $1"," $2}' input_file.txt, means take the input file, and, going through it line-by-line, print the first field, then a comma, then the second field; the output of this by itself would look like, for the first line: 20090610,153000;00936.00;21. Now we have to convert the semicolons to commas. So we pipe (using "|") the output of the first awk command to the second, where we tell it (using the -F flag) that the field separators are semicolons. So now awk thinks there are three fields, where the date and time together are the first, the price is the second, and the volume is the third. So the second awk command says to print the first field, then a comma, then the second, then a comma, and then the third (for each line in the input). Finally, the ">" symbol says to output the result to what follows (so here we specify the output file name) (note that ">" means "overwrite and output", while ">>" means "append.")

Started this thread Reply With Quote
Thanked by:

Can you help answer these questions
from other members on NexusFi?
Cheap historycal L1 data for stocks
Stocks and ETFs
Trade idea based off three indicators.
Traders Hideout
REcommedations for programming help
Sierra Chart
How to apply profiles
Traders Hideout
About a successful futures trader who didnt know anythin …
Psychology and Money Management
 




Last Updated on July 1, 2012


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts