NexusFi: Find Your Edge


Home Menu

 





Data converter from a format to another format


Discussion in Platforms and Indicators

Updated
      Top Posters
    1. looks_one Nicolas11 with 10 posts (24 thanks)
    2. looks_two aventeren with 5 posts (2 thanks)
    3. looks_3 Big Mike with 4 posts (0 thanks)
    4. looks_4 Zwaen with 2 posts (0 thanks)
    1. trending_up 11,007 views
    2. thumb_up 26 thanks given
    3. group 7 followers
    1. forum 22 posts
    2. attach_file 9 attachments




 
Search this Thread

Data converter from a format to another format

  #1 (permalink)
 
Nicolas11's Avatar
 Nicolas11 
near Paris, France
 
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769

This thread is a possible place to obtain a small software allowing the conversion of data from a format to another format.

For the avoidance of doubt, it’s free.

For instance, let’s suppose that you have a 1-Go text file with data formatted like:
05/24/2012 23:45 716,50 713,00 714,25 715,75 12563
… and you want to convert into a text file with the following format:
1120525,0045,716.50,713.00,714.25,715.75,12563
(time in output file shall be one hour later than in the input file).

Then...

1. In this thread, post:
For input file:
- the 5 first lines of your data file
- format of the lines
- explanation for each field
- identification of the delimiter (tabulation, comma, etc.)
- has the data file a first line with titles of the fields or not
- size of the input file
For output file:
- format of the lines
- explanation for each field
- identification of the delimiter (tabulation, comma, etc.)
- should the data file have a first line with titles of the fields or not

An example of such request is given in the following message.

2. Then, after some time (typically: from 1 day to 1 week), I will post in this thread:
- a JAR file (it is executable, same as EXE file),
- source code in Java.
The source code is useless to execute the JAR file. It is just given for reference.

3. Then, you will have to:
- save the JAR file in the same directory as your data file
- rename your data file as "input.txt"
- execute the JAR file (by double-clicking on it)

Note: the execution of the JAR file requires Java to be installed on your computer. It is usually the case. Otherwise, it could be downloaded there: Download Free Java Software

The application will generate in the same directory a file called "output.txt" in the required format.
Take care: any file already named "output.txt" in the directory will be deleted and replaced.

Note: if your input file is very big, the execution could take some time (for instance: 25 minutes for a 1 Go file). But you can check that the process is progressing by refreshing your Windows Explorer window (press F5): you should see the size of the output file growing until it reaches approximatively the size of the input file.

An example of such JAR file and source code is given in a following message.

4. Obviously, you will have to check the content of output.txt

5. These instructions may be updated in the "Quick Summary" below.

6. I do not promise to provide this service forever. If there are too many requests or if it is not possible any more for me to continue for any reason, I will kindly ask Big Mike to close this thread.

In case of problems in the execution of the JAR file, I will do my best to help, but in limited time. If we do not find a solution quickly, I will perhaps have to give up. As a reminder, this service is free.

Nicolas

Visit my NexusFi Trade Journal Started this thread Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Pivot Indicator like the old SwingTemp by Big Mike
NinjaTrader
Trade idea based off three indicators.
Traders Hideout
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
ZombieSqueeze
Platforms and Indicators
MC PL editor upgrade
MultiCharts
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Just another trading journal: PA, Wyckoff & Trends
28 thanks
Tao te Trade: way of the WLD
24 thanks
Diary of a simple price action trader
21 thanks
My NQ Trading Journal
14 thanks
GFIs1 1 DAX trade per day journal
9 thanks
  #3 (permalink)
 
Nicolas11's Avatar
 Nicolas11 
near Paris, France
 
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769


This message is an example of request.

Hi,

Could you please provide with an executable file allowing the following conversion?

Input file:
Five first lines:

Quoting 
Date Time High Low Open Close Vol
05/24/2012 22:30 716,50 713,00 714,25 715,75 26540
05/24/2012 22:45 716,50 713,00 714,25 715,75 12563
05/24/2012 23:00 716,50 713,00 714,25 715,75 13562
05/24/2012 23:15 716,50 713,00 714,25 715,75 9856

Field delimiter is: tabulation
Fields are: date MM/DD/YYY, hour HH:mm, H, L, O, C, volume
The file contains a first line with the titles of the fields
The size of the input file is 1Go

Output file:
1120525,0045,716.50,713.00,714.25,715.75,12563
Field delimiter is: comma
Field are: date YYYMMDD, hour HHmm, H, L, O, C , volume
Time is in output file shall be one hour later than in the input file
Note: commas within the numbers shall be replaced by dots
No first line with the titles of the fields

Thanks in advance.

Visit my NexusFi Trade Journal Started this thread Reply With Quote
Thanked by:
  #4 (permalink)
 
Nicolas11's Avatar
 Nicolas11 
near Paris, France
 
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769

This message is an example of answer.

Hi,

In the enclosed ZIP file, you will find an executable JAR file and a source code (.java file).

Next steps on your side:
- Save the JAR file in the same directory as your data file.
- Rename your data file as "input.txt"
- Execute the JAR file (by double-clicking on it)

Note: the source code (.java file) is useless for the conversion, and is just given for reference.

Note: the execution of the JAR file requires Java to be installed on your computer. It is usually the case. Otherwise, it could be downloaded there: Download Free Java Software

The application will generate in the same directory a file called "output.txt" in the required format.
Take care: any file already named "output.txt" in the directory will be deleted and replaced.

Note: if your input file is very big, the execution could take some time (for instance: 25 minutes for a 1 Go file). But you can check that the process is progressing by refreshing your Windows Explorer window (press F5): you should see the size of the output file growing until it reaches approximately the size of the input file.

It is your responsibility to check the content of output.txt

The 5 first lines of the input file:

Quoting 
Date Time High Low Open Close Vol
05/24/2012 22:30 716,50 713,00 714,25 715,75 26540
05/24/2012 22:45 716,50 713,00 714,25 715,75 12563
05/24/2012 23:00 716,50 713,00 714,25 715,75 13562
05/24/2012 23:15 716,50 713,00 714,25 715,75 9856

have been converted as follows:

Quoting 
1120524,2330,716.5,713,714.25,715.75,26540
1120524,2345,716.5,713,714.25,715.75,12563
1120525,0000,716.5,713,714.25,715.75,13562
1120525,0015,716.5,713,714.25,715.75,9856

On my computer, the conversion of a 1 Go input file has taken 25 minutes.

Nicolas

Attached Files
Elite Membership required to download: DataConverter_for_Nicolas11_v01.zip
Visit my NexusFi Trade Journal Started this thread Reply With Quote
Thanked by:
  #5 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,440 since Jun 2009
Thanks Given: 33,207
Thanks Received: 101,599

Continuing from

Source file is MultiCharts export:

 
Code
"Date","Time","Price","Volume"
1/3/2010,17:00:00,1113.75,2
1/3/2010,17:00:00,1113.75,2
1/3/2010,17:00:00,1113.75,1
1/3/2010,17:00:00,1113.75,1
1/3/2010,17:00:00,1113.75,1
Includes header line

Output file should be standard NinjaTrader yyyyMMdd HHmmss;price;volume

Are there any file size limits? I need to know if I need to break up to < 2GB or < 4GB files.

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #6 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,440 since Jun 2009
Thanks Given: 33,207
Thanks Received: 101,599

Request 2

Source is QCollector:
 
Code
06/01/2012	05:17:21	12100.00	1	1	12060.00	12100.00	4202	0	0
06/05/2012	09:14:20	12000.00	1	1	12000.00	12018.00	999	0	0
06/07/2012	22:02:33	12442.00	5	5	12442.00	12450.00	2445	0	0
06/10/2012	17:00:00	12524.00	1	1	0.00	12524.00	84	0	0
06/11/2012	06:35:54	12500.00	1	2	12500.00	12552.00	635	0	0
06/11/2012	08:15:50	12498.00	1	4	12498.00	12561.00	671	0	0
No header line. Tab delimited.
Date, Time hh:mm:ss, price, volume. The remaining columns need to be discarded.

Output is standard NinjaTrader.

Again, thank you.

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #7 (permalink)
 
Nicolas11's Avatar
 Nicolas11 
near Paris, France
 
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769

Hi @Big Mike,

Thanks for giving me the opportunity to test this.

Let's first just handle your "request 1" to see if it works.

I have made three assumptions:
(i) date format in MultiCharts export is dd/MM/yyyy and not MM/dd/yyyy (I cannot check now).
(ii) you do not want header in the output file
(iii) as you wrote, Ninja Trader format is "yyyyMMdd HHmmss" with no ";" in the middle (I do not know Ninja Trader)

If one of these assumptions is wrong, just tell me.

Normally, "size does not matter". So no need to break down the files. However, the conversion can take some time. So perhaps you would want to test on a small file first.

In the enclosed ZIP file, you will find an executable JAR file and a source code (.java file).

Next steps on your side:
- Save the JAR file in the same directory as your data file.
- Rename your data file as "input.txt"
- Execute the JAR file (by double-clicking on it)

Note: the source code (.java file) is useless for the conversion, and is just given for reference.

Note: the execution of the JAR file requires Java to be installed on your computer. It is usually the case. Otherwise, it could be downloaded there: Download Free Java Software

The application will generate in the same directory a file called "output.txt" in the required format.
Take care: any file already named "output.txt" in the directory will be deleted and replaced.

Note: if your input file is very big, the execution could take some time (for instance: 25 minutes for a 1 Go file). But you can check that the process is progressing by refreshing your Windows Explorer window (press F5): you should see the size of the output file growing until it reaches approximately the size of the input file.

The 5 first lines of the input file:


Quoting 
"Date","Time","Price","Volume"
1/3/2010,17:00:00,1113.75,2
1/3/2010,17:00:00,1113.75,2
1/3/2010,17:00:00,1113.75,1
1/3/2010,17:00:00,1113.75,1
1/3/2010,17:00:00,1113.75,1

have been converted as follows:


Quoting 
20100301 170000;1113.75;2
20100301 170000;1113.75;2
20100301 170000;1113.75;1
20100301 170000;1113.75;1
20100301 170000;1113.75;1



Nicolas

Attached Files
Elite Membership required to download: DataConverter_for_BigMike_req01_v01.zip
Visit my NexusFi Trade Journal Started this thread Reply With Quote
Thanked by:
  #8 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,440 since Jun 2009
Thanks Given: 33,207
Thanks Received: 101,599


Nicolas11 View Post
Hi @Big Mike,

Thanks for giving me the opportunity to test this.

Let's first just handle your "request 1" to see if it works.

I have made three assumptions:
(i) date format in MultiCharts export is dd/MM/yyyy and not MM/dd/yyyy (I cannot check now).
(ii) you do not want header in the output file
(iii) as you wrote, Ninja Trader format is "yyyyMMdd HHmmss" with no ";" in the middle (I do not know Ninja Trader)

Sorry about that. Request 1:

It is m/d/yyyy. Take note, not mm/dd/yyyy.

No header in output file.

Correct, the first field for ninja has a space between date and time.

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #9 (permalink)
 
Nicolas11's Avatar
 Nicolas11 
near Paris, France
 
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769

For request 1...

Updated output:

Quoting 
20100103 170000;1113.75;2
20100103 170000;1113.75;2
20100103 170000;1113.75;1
20100103 170000;1113.75;1
20100103 170000;1113.75;1

And updated executable enclosed (v02).

Nicolas

Attached Files
Elite Membership required to download: DataConverter_for_BigMike_req01_v02.zip
Visit my NexusFi Trade Journal Started this thread Reply With Quote
Thanked by:
  #10 (permalink)
 
Nicolas11's Avatar
 Nicolas11 
near Paris, France
 
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769


For "request 2"...

Enclosed please find a dedicated executable (_req02_v01).

The following input lines
 
Code
06/01/2012	05:17:21	12100.00	1	1	12060.00	12100.00	4202	0	0
06/05/2012	09:14:20	12000.00	1	1	12000.00	12018.00	999	0	0
06/07/2012	22:02:33	12442.00	5	5	12442.00	12450.00	2445	0	0
06/10/2012	17:00:00	12524.00	1	1	0.00	12524.00	84	0	0
06/11/2012	06:35:54	12500.00	1	2	12500.00	12552.00	635	0	0
06/11/2012	08:15:50	12498.00	1	4	12498.00	12561.00	671	0	0
have been converted into:
 
Code
20120601 051721;12100.00;1
20120605 091420;12000.00;1
20120607 220233;12442.00;5
20120610 170000;12524.00;1
20120611 063554;12500.00;1
20120611 081550;12498.00;1
Nicolas

PS - Mike, for some reasons, I will have to make a break on this conversion subject between 16:00 EST today and Wednesday morning. If you wish some changes today, do not hesitate! Anyway, we could resume on Wednesday.

Attached Files
Elite Membership required to download: DataConverter_for_BigMike_req02_v01.zip
Visit my NexusFi Trade Journal Started this thread Reply With Quote
Thanked by:




Last Updated on March 30, 2015


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts