Disaster Strikes! Backup your data - futures io
futures io futures trading



Disaster Strikes! Backup your data


Discussion in Tech Support

Updated
      Top Posters
    1. looks_one Big Mike with 10 posts (21 thanks)
    2. looks_two Peter2150 with 5 posts (6 thanks)
    3. looks_3 stephenszpak with 3 posts (1 thanks)
    4. looks_4 keymoo with 3 posts (4 thanks)
      Best Posters
    1. looks_one Big Mike with 2.1 thanks per post
    2. looks_two Silver Dragon with 1.5 thanks per post
    3. looks_3 keymoo with 1.3 thanks per post
    4. looks_4 Peter2150 with 1.2 thanks per post
    1. trending_up 17,313 views
    2. thumb_up 42 thanks given
    3. group 15 followers
    1. forum 40 posts
    2. attach_file 1 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Disaster Strikes! Backup your data

(login for full post details)
  #1 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,733 since Jun 2009
Thanks: 32,283 given, 97,470 received

The scenario:

You walk into your office, and your computer is dead. The system won't boot. The hard drives are completely worthless.

Forget about salvaging data off the HDD's -- they're gone. It's over. The drives have given up the ghost. Done. Toast. Finished.

Now, the question:

What would you do if your computer just crashed suddenly and without warning, and you lost everything on the computer with no hopes of salvaging any data that was stored on the hard drives.

I know there are a lot of techs in the forum. So let's just forget about RAID or trying to recover data from the drives, moving the drive to another computer, sending it out for professional recovery, or any of that. It doesn't matter if you were using SSD's or regular mechanical drives, for the purpose of this thread we will assume the data is permanently gone.

So for the average user, what do you do? Think about what you may have lost:

- Local emails (Outlook, Thunderbird clients)
- Pictures, videos, albums
- Music library
- Trading software
- Historical tick data
- Strategies, Indicators
- Software license codes you own
- Bookmarks in your browser
- Passwords and user names to all your sites

Always be prepared
If any of the data on your computer is important to you, then you need to have a backup.

Most common backups are to external USB drives, or eSATA drives. You attach them to the computer, and run some software to backup critical files to the external drive. This scenario should work great for the average home user, unless a real disaster strikes like a flood or tornado and you were unable to physically move the storage media to a safe place.

Let's say you have an external USB drive, or maybe you backup to a remote server. The next question is: How often? Daily? Weekly? Monthly? Let's say your last backup was a week or more ago, are you OK with losing whatever work you've done on your computer in the last week?

Then of course, the next question is: What are you backing up? Simply having an external backup drive and backing up often is not enough, not if you aren't even backing up the right files. Some backup programs may by default exclude important directories to you. For example, Sierra Charts ignores Windows best practices and by default installs itself to the root of your C: drive. 99% of backup software would never include this location in its default backup set. The same can be said of MultiCharts. MultiCharts saves its files in a non-traditional "ProgramData" directory, while not as bad as the root of the drive like Sierra Chart, it is still a path that most backup software will exclude from default backups.

The fact is, most backup software only backs up your "Documents" directory by default. So if you use NinjaTrader, you are covered. But if you use MC or SC, not so much. What about if you use local email clients like Outlook or Thunderbird? By default, a few backup applications will not include those directories either, which are buried in your %appdata% directory in your profile.

My point with this post is just to try and help some of you out. Disk drives are cheap. Your time isn't. You should invest a few minutes and a few dollars now to come up with a solid backup plan to save you from the day you lose everything.

If anyone has specific ideas or recommendations for backup drives, software, or even internet cloud backup suites, feel free to post them here. (reminder: we do not allow referral links)

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Started this thread Reply With Quote
The following 11 users say Thank You to Big Mike for this post:

Journal Challenge February 2021 results (so far):
Competing for $1500 in prizes from Topstep
looks_oneSBtrader82 's Trading Journalby SBtrader82
(151 thanks from 28 posts)
looks_twoJust BEING a Trader: Letting Go!!by iqgod
(110 thanks from 31 posts)
looks_3Wisdom is Emptinessby Mtype
(66 thanks from 24 posts)
looks_4Deetee’s DAX Trading Journal (time based)by Deetee
(31 thanks from 15 posts)
looks_5Journal for peanuts1956by peanuts1956
(23 thanks from 13 posts)
 
 
(login for full post details)
  #2 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,733 since Jun 2009
Thanks: 32,283 given, 97,470 received

My personal solution is more complex than the average user needs.

I have a remote media server with 8TB of storage that I use to house not just my media, but also a backup of critical files from my primary workstation.

I use a free piece of software called Syncrify. It is basically rsync over HTTP. There is a server side component, and a client side component. On my Debian 6 server, I am running the server component, and on my Windows 7 x64 client I have the client component.

Nightly the Syncrify client will scan my workstation for any changed files, and send that changed data across my local gigabit network to the media server. The whole process takes just a couple of minutes, since very little data changes on a daily basis.

If you want to buy Syncrify, you can also enable snapshotting on the server side. Or you could just do it natively in your server file system, like with ZFS or Btrfs. Since I am mainly just interested in disaster recovery, I don't care about snapshots so much on the server, as my Windows 7 client already has snapshots on the NTFS file system ("Previous Versions" in Windows speak).

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Started this thread Reply With Quote
The following 9 users say Thank You to Big Mike for this post:
 
(login for full post details)
  #3 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,733 since Jun 2009
Thanks: 32,283 given, 97,470 received


Here is a link to Newegg's external hard drives:

Newegg.com - External Hard Drives

On the left you can select the size and interface. If you know your computer supports it, eSATA will be faster than USB.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #4 (permalink)
Aurora, Il USA
 
Experience: Advanced
Platform: TradeStation
Trading: futures
 
kbit's Avatar
 
Posts: 5,902 since Nov 2010
Thanks: 3,294 given, 3,357 received

Thanks for bringing this up Mike.
I have an extenal HD and occasionally I also put what I stuff consider critcal on a couple cds. That's all I need really......but it's good to be reminded to make sure everything is up to date.

Reply With Quote
 
(login for full post details)
  #5 (permalink)
Legendary Data Wizard!!!
Cincinnati Ohio
 
Experience: Intermediate
Platform: TastyWorks
Broker: TastyWorks
Trading: FX, Stocks, Options
 
Silver Dragon's Avatar
 
Posts: 1,984 since Feb 2011
Thanks: 5,675 given, 4,798 received


Big Mike View Post
My point with this post is just to try and help some of you out. Disk drives are cheap. Your time isn't. You should invest a few minutes and a few dollars now to come up with a solid backup plan to save you from the day you lose everything.


If anyone has specific ideas or recommendations for backup drives, software, or even internet cloud backup suites, feel free to post them here. (reminder: we do not allow referral links)

Mike


I use Carbonite. 60 bucks a year. Well worth it for the peace of mind. Online Backup ? Data & File Backup Software | Carbonite

I can acess my backup files from any other computer i might be on. This has come in handy when working on a remote computer where I needed to get some code.

One thing about Carbonite; it does not backup 100% of your harddrive. Only what it thinks in the most important files are. For example: It will not backup MP3's unless you tell it to.

Ninja directory where strategies and indicators are stored is completely backed up automatically. 10 minutes after I create a new strategy it is backed up to the cloud. Love it!

SD

nosce te ipsum

You make your own opportunities in life.
Visit my futures io Trade Journal Reply With Quote
The following 2 users say Thank You to Silver Dragon for this post:
 
(login for full post details)
  #6 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,733 since Jun 2009
Thanks: 32,283 given, 97,470 received

Did you build the computer yourself, or did you buy it (Dell, HP, etc)?

If you bought it from an OEM, then most likely it did not include original Windows media to install the OS. It only includes "recovery media". So keep in mind that even if you are backing up all your precious data, you still need to re-install the operating system after you fix the problem. How long will you be "out of commission" until you get back online?

If you built it yourself, hopefully you still have the original install media --- and that it wasn't an "upgrade" version which requires a previous copy of Windows to exist on the HDD, since you'd be replacing the HDD in this scenario with a new (empty) one.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #7 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,733 since Jun 2009
Thanks: 32,283 given, 97,470 received

Good deal. I've tried Acronis online backup and Mozy online backup. Both were quite terrible, with huge shortcomings like not supporting files over 2GB in size and some other quite ridiculous (in this day) problems.

I eventually cancelled both, as it was just not reliable.

The other issue was fast forward to the scenario in post #1. How do you get all your data back from the cloud? In my case, I was storing hundreds of GB in the cloud. And their servers were so slow it would take days and days of downloading in order to get all that data back on my system.

I think that for the average user who is only backing up maybe 10-20GB, the cloud makes a lot of sense.

I've heard that Windows 8 is moving into the cloud in a big way. That is the only thing I like about what I've heard about Win8 so far. Everything else seems really terrible, and it seems like I will be staying on Windows 7 for a long time to come.

But no doubt, the cloud is where it's at.

It is worth mentioning that Dropbox is working on a similar solution to not just share files (like they do now), but act as a backup for your important folders.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #8 (permalink)
Legendary Data Wizard!!!
Cincinnati Ohio
 
Experience: Intermediate
Platform: TastyWorks
Broker: TastyWorks
Trading: FX, Stocks, Options
 
Silver Dragon's Avatar
 
Posts: 1,984 since Feb 2011
Thanks: 5,675 given, 4,798 received


Big Mike View Post
Did you build the computer yourself, or did you buy it (Dell, HP, etc)?

If you bought it from an OEM, then most likely it did not include original Windows media to install the OS. It only includes "recovery media". So keep in mind that even if you are backing up all your precious data, you still need to re-install the operating system after you fix the problem. How long will you be "out of commission" until you get back online?

If you built it yourself, hopefully you still have the original install media --- and that it wasn't an "upgrade" version which requires a previous copy of Windows to exist on the HDD, since you'd be replacing the HDD in this scenario with a new (empty) one.

Mike

If you did buy a OEM and only have the recovery media, you can purchase windows7 Professional OEM disk from Newegg. Runs 140 bucks for 32 bit and 189 for 64 bit. This is much cheaper then the retail version which costs 299 and up.

Only difference between retail and OEM is it does not come with Microsoft support.

Newegg.com - Operating Systems, OS Software, Microsoft Windows 7


SD

nosce te ipsum

You make your own opportunities in life.
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to Silver Dragon for this post:
 
(login for full post details)
  #9 (permalink)
Washington DC
 
Experience: Intermediate
Platform: NinjaTrader
 
Posts: 214 since Jun 2009
Thanks: 110 given, 117 received

I think I am more thorough than most. But I absolutely can't afford to lose data, nor can I afford downtime.

I have two desktops, and a desktop replacement laptop. All three machines have identical setup's and folder names. The laptop is just for emergency use.

On both desktops, I run an imaging program from Storagecraft, called Shadowprotect. It has a feature called continuous incrementals, and it takes an incremental image every 30 minutes in about 7 minutes. At the end of the day it collapses them into a daily. I just let it run. From these I can restore the computer from the image to previous points in time, as well as mount the images and pull data from them. These images are stored on a 2nd internal drive, and also synced to an external drive.

Then I use a synchronize program to sync to two external drives on the primary business machine. I then sync two the other machines external drive, and c: drive. The reason I do this is just because of what has happened. Turn on the main business machine in the am and it's dead. I can just go to the 2nd one and continue uninterrupted.

Finally I have cloud storage via two different companies. I use Idrive and Jungledisk. I picked these as they both offer private encrytion and satisfactory retentions policy's. Both retain multiple versions. They were chosen with geographical considerations. Idrive servers are in California and Jungledisk in Atlanta and Chicago. Although Jungledisk might seem like a strange name, the company behind them is Rackspace which is well known to the corporate world.


IMPORTANT POINT. All these backup methods have had the restore capability tested and proven. Without testing the restore, you have no backup.

Pete

Reply With Quote
The following 5 users say Thank You to Peter2150 for this post:
 
(login for full post details)
  #10 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,733 since Jun 2009
Thanks: 32,283 given, 97,470 received


Great feedback, thanks for sharing guys. Hopefully it will help save someone some trouble.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Started this thread Reply With Quote
The following user says Thank You to Big Mike for this post:
 
(login for full post details)
  #11 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,733 since Jun 2009
Thanks: 32,283 given, 97,470 received

Just a quick note:

Carbonite seems limited to 4GB files, not 100% clear
iDrive 10GB
Jungledisk 5GB

This kind of thing still upsets me. I have many files over this size that are important data to me (compressed archives generally). I suspect for most it is not an issue, but thought I would report my findings.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #12 (permalink)
Colorado
 
Experience: Advanced
Platform: MultiCharts
Broker: IB & Iqfeed
Trading: ES , CL
 
sptrader's Avatar
 
Posts: 543 since Apr 2010
Thanks: 1,094 given, 696 received

I've debated using the cloud like Carbonite, but for now, I use a 500gig WD USB3 external drive (about $50), that I plug in to backup weekly.. Takes about 15 minutes for a complete drive image (80gb or so of data) using "Casper" backup software.. I leave my trading computer on 24/7; I usually change drives every 3-4 years, to prevent problems..I also export my new studies or other important items, to my laptop upon completion..
If you're really paranoid about losing files, the external drive is small enough, to fit easily in a small, fire-proof safe..
Quick, cheap and painless.

Reply With Quote
The following 2 users say Thank You to sptrader for this post:
 
(login for full post details)
  #13 (permalink)
the coin hunter
virginia
 
Experience: Intermediate
Platform: ninja
Trading: NQ
 
cory's Avatar
 
Posts: 6,028 since Jun 2009
Thanks: 845 given, 7,891 received

my hd crashed, I used another machine and this product to recover most of data EaseUS provides professional Data Recovery, Partition Manager, Data Backup Software Products.

Reply With Quote
 
(login for full post details)
  #14 (permalink)
Switzerland
 
 
Posts: 12 since Dec 2011
Thanks: 5 given, 7 received

Since I do not rely on trading (it's just a hobby), there is no need for exteme fail over solutions. But what seem to work for me:

1.) running an imaging software like Norton Ghost on my primary trading platform (desktop pc) as well as on my notebook. I have a basic image where only Win7 SP1 and Office 2010 with the latest patches are installed and registered (if you want to have a fresh and clean installation). Then i have an up2date image (two weekly, daily is exaggerated for me) of my desktop PC. If you are relying heavily on it, you could have a spare disk where you can restore your ghost image in less than 5 minutes from the image stored on another backup disk in your PC or from the Nas (you could even think of having a spare disk around for generic emergency cases).

2.) Nas with raid5/raid6 (depending on your needs) that provides SMB/CIFS services (network drive) to store non-essential stuff (since desktop PC has an SSD) and generic data. Besides that, specific config files/indicators are also backuped there. (my music collection gets backuped infrequently to an external drive that I store then at an offsite location, since I really love my music collection and already had to start three times over due to disk failures and theft)

3.) The Nas replicates only the most important files to Amazon S3 in an encrypted form (ssl transfer and encrypted files). If you have a "full Linux" distribution for your server or Windows Server, Zmanda/Amanda Backup Server Community Edition is worth a look.

- I also maintain a KeePass (password safe) list, which is backuped trough Dropbox (no problem since it is encrypted). Source Code of programming projects are stored on off-site repositories (SVN) like origo (origo.ethz.ch), that should be backuped by them (and you can also get a whole svn dump).

- Last but not least, I have also a GSM/UMTS/LTE internet stick for my notebook, so in case of emergency and internet disconnect, I am still able to handle orders etc...

My solution seems like an overkill for private users and hobby traders, but hey, everyone has his own needs for data redundancy.

Reply With Quote
 
(login for full post details)
  #15 (permalink)
Washington DC
 
Experience: Intermediate
Platform: NinjaTrader
 
Posts: 214 since Jun 2009
Thanks: 110 given, 117 received


Big Mike View Post
Just a quick note:

Carbonite seems limited to 4GB files, not 100% clear
iDrive 10GB
Jungledisk 5GB

This kind of thing still upsets me. I have many files over this size that are important data to me (compressed archives generally). I suspect for most it is not an issue, but thought I would report my findings.

Mike

Hi Mike

Not familiar with Carbonite, but the other two yes.

On Idrive Pro which I use, it says it has been tested up to 10gb, not that there is a limit. The upload time on 10gb would be horrific

On Jungledisk Desktop Edition which is what I use, there is only a 5gb limit using the network feature. Otherwise there is no limit. Also you can break up archives like zip/rar files into smaller pieces.

Jungledisk has two backup modes you can set up. What they call a legacy mode which offers certain features like the ability to search thru the backup for files. An organized local disk makes this unnecessary to me. This mode stores the files uncompressed, and uses more space

The other mode is the backup vault which is more limited in it's feature set, but stores the files compressed. This mode has worked fine for me.

Jungledisk is the most versatile, albeit the less user friendly. It is also very price friendly.

Pete

PS. If you want me to test one of them for you, I will.

Reply With Quote
 
(login for full post details)
  #16 (permalink)
Madison, WI
 
Experience: Advanced
Platform: ALT
Trading: ES
 
aslan's Avatar
 
Posts: 616 since Jan 2010
Thanks: 351 given, 1,109 received


Big Mike View Post
For example, Sierra Charts ignores Windows best practices and by default installs itself to the root of your C: drive. 99% of backup software would never include this location in its default backup set. The same can be said of MultiCharts. MultiCharts saves its files in a non-traditional "ProgramData" directory, while not as bad as the root of the drive like Sierra Chart, it is still a path that most backup software will exclude from default backups.

The fact is, most backup software only backs up your "Documents" directory by default. So if you use NinjaTrader, you are covered. But if you use MC or SC, not so much.

"Windows best practices" -> that is comical.

There are good reasons SC does it that way, but I think it is good that you are pointing out this pitfall though. SC users can of course place their data directory anywhere they want by changing one preference in the Global Settings. So, if you wanted to place it in the Documents dir, you certainly could do that. Personally, I love having my entire install in a single dir, that I can dump to a thumb drive and copy somewhere else. I do the same thing for some other apps as well.

Reply With Quote
 
(login for full post details)
  #17 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,733 since Jun 2009
Thanks: 32,283 given, 97,470 received


aslan View Post
"Windows best practices" -> that is comical.

There are good reasons SC does it that way

What is the good reason that SC defaults to C:\SierraChart, instead of C:\Program Files (x86)\SierraChart? And that it stores it's data in the Program Files directory, instead of in the well accepted and industry standard Documents directory for user data?

I like Sierra. I know you like it too. But it seems like a poor design choice, and now they are just sticking with it because change is hard (?). We know the SC devs are smart guys, but we also know they are not receptive to change or critical feedback. Twenty years ago I liked to have well organized directories off the root too, in DOS 3.3 or so. But in the Windows world, it is not the standard and is frowned upon.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #18 (permalink)
Dallas,TX
 
Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker: Zaner/Zen Fire
Trading: ES,6E,6B,GC,CL
 
Posts: 598 since Nov 2009
Thanks: 176 given, 120 received

also, you can look at iomega usb drive with vclone that can take image of whole PC
Iomega v.Clone App Portable-izes Your Entire PC
2 benefits
1) restore if your PC fails
2) run PC from usb drive when you are travelling

Reply With Quote
 
(login for full post details)
  #19 (permalink)
Madison, WI
 
Experience: Advanced
Platform: ALT
Trading: ES
 
aslan's Avatar
 
Posts: 616 since Jan 2010
Thanks: 351 given, 1,109 received


Big Mike View Post
What is the good reason that SC defaults to C:\SierraChart, instead of C:\Program Files (x86)\SierraChart? And that it stores it's data in the Program Files directory, instead of in the well accepted and industry standard Documents directory for user data?

That "(x86)" did not use to be there. Notice how windows programs might be there or in the old program folder etc. Thats why I say it is comical. My backup program should not need to know about special dirs etc. Upgrading to a new PC is a nightmare.

Ever use a MAC? Most apps are the same, everything under one tree. Sure makes moving an app to another system easy. Also, to uninstall, you blow away the entire dir. Want to make a copy of an SC instance, copy the dir and you are off and running. Want to take it with you on a thumb drive, copy one dir done.

In any case, you could install SC into that dir, as you have a choice when you install, and as I mentioned you can also move your data dir wherever you want it. Once you install once, further upgrades from within the program know where the install is and install in place.


Quoting 
But it seems like a poor design choice, and now they are just sticking with it because change is hard (?). We know the SC devs are smart guys, but we also know they are not receptive to change or critical feedback.

Use some sugar ;-)

Reply With Quote
 
(login for full post details)
  #20 (permalink)
Bala, PA, USA
 
Experience: Intermediate
Platform: NinjaTrader
Broker: Mirus, IB
Trading: SPY, Oil, Euro
 
monpere's Avatar
 
Posts: 1,858 since Jul 2010
Thanks: 300 given, 3,338 received


emini_Holy_Grail View Post
also, you can look at iomega usb drive with vclone that can take image of whole PC
Iomega v.Clone App Portable-izes Your Entire PC
2 benefits
1) restore if your PC fails
2) run PC from usb drive when you are travelling

Got hit last week with a cpu intensive virus, that kept crashing the machine because it was overheating the cpu. Mawarebytes, Norton, and Windows Security Essentials would not detect it. Had to reinstall vista from my recovery disk. Good thing this happened during the holidays, an I also had backups.

Reply With Quote
 
(login for full post details)
  #21 (permalink)
Massachusetts (USA)
 
Experience: None
Platform: NinjaTrader
Trading: YM
 
stephenszpak's Avatar
 
Posts: 778 since Jun 2009
Thanks: 144 given, 356 received


monpere View Post
Got hit last week with a cpu intensive virus, that kept crashing the machine because it was overheating the cpu. Mawarebytes, Norton, and Windows Security Essentials would not detect it. Had to reinstall vista from my recovery disk. Good thing this happened during the holidays, an I also had backups.

You might want to switch to Kaspersky. I can still use the 2011 version for about 80
days, but the new one is already out. I have Kaspersky Internet Security, which takes
care of internet and viruses. I think that is confusing, but that's how they named it.


Buy Kaspersky Software | Kaspersky Lab United States


Peter2150 protection as a summary. I like the idea of 2 (in his case 3) separate computers. But like a number
of procedures on this thread, it isn't easy to setup for a non-tech. (Not asking for help, just commenting.)

I have two desktops...On both desktops, I run an imaging program from Storagecraft, called Shadowprotect....Then I use a synchronize program to sync to two external drives on the primary business machine. I then sync two the other machines external drive, and c: drive......Finally I have cloud storage via two different companies. I use Idrive and Jungledisk.

Reply With Quote
 
(login for full post details)
  #22 (permalink)
Massachusetts (USA)
 
Experience: None
Platform: NinjaTrader
Trading: YM
 
stephenszpak's Avatar
 
Posts: 778 since Jun 2009
Thanks: 144 given, 356 received


emini_Holy_Grail View Post
also, you can look at iomega usb drive with vclone that can take image of whole PC
Iomega v.Clone App Portable-izes Your Entire PC
2 benefits
1) restore if your PC fails
2) run PC from usb drive when you are travelling

Article from link provided:

Iomega v.Clone App Portable-izes Your Entire PC

It'd be easy to get knotted up in jargon describing v.Clone, so let's stick with the concept for now: It's a syncable, complete image of your PC, which you can boot run from basically any other PC. And it's free.

To anyone who's used mainstream backup software, this probably sounds too good to be true. In a way, yes, it is. But the tradeoff is minimal, and the payoff is more than worth it. Here's the deal:

v.Clone EMC owns Iomega, which makes portable hard drives. EMC also owns VMWare, which makes virtualization software around. v.Clone is essentially a portable installation of VMWare, meaning that you can plug your v.Clone-loaded Iomega portable HDD into most any Windows computer, run the app, and boot into your saved virtual machine. The secret, though, lies in what you're booting into—namely, a perfect copy of your main PC.

In other words, v.Clone lets you make a full copy of your primary PC—including apps, media, settings, etc—to a VMWare image, run said copy from other Windows-based computers, and upon reconnection with the primary PC, sync any changes you made while running the virtual machine back to your main PC. Likewise, any changes you've made on your host PC can be synced to your virtual machine, so your v.Clone image is more or less a virtualized, up-to-date clone of your main PC. We haven't been able to try it out yet, but Iomega assures us that the whole system isn't as ass-slow as it sounds like it could be, because the syncing process is incremental—if you only change a few things on your virtualized image, syncing it with your host PC won't take more than a few minutes.

There are a couple of catches here: the software may be called v.Clone, but it's not creating a traditional, raw image clone. If your main PC faceplants and destroys itself, your v.Clone image will let you salvage your data, but it won't let do a full restore, like Norton Ghost or Time Machine. And despite the fact that portable HDDs are all exactly the same, this software will only work with Iomega drives. But still, kinda genius! (If you happen to this particular brand of totally commodified product!)

v.Clone is available for owners of Iomega USB portable hard drives, including the full eGo and Prestige lines.

Reply With Quote
The following user says Thank You to stephenszpak for this post:
 
(login for full post details)
  #23 (permalink)
Massachusetts (USA)
 
Experience: None
Platform: NinjaTrader
Trading: YM
 
stephenszpak's Avatar
 
Posts: 778 since Jun 2009
Thanks: 144 given, 356 received

There are USB flash drives which are quite large. But as has been said, the main problem
is getting the correct software, so that all your stuff actually is backed up.


128 GB USB flash drive in attachment, and also at link below.

Newegg.com - Kingston HyperX DataTraveler HyperX 3.0 128GB USB 3.0 Flash Drive Model DTHX30/128GB

Attached Thumbnails
Click image for larger version

Name:	Kingston drive.png
Views:	133
Size:	197.5 KB
ID:	59703  
Reply With Quote
 
(login for full post details)
  #24 (permalink)
Hamburg, Germany
 
Experience: Intermediate
Platform: self-developed
Trading: Futures, Forex
 
GuerillaTrading's Avatar
 
Posts: 6 since Nov 2011
Thanks: 0 given, 2 received

There is an easy solution for this problem. Get a trading workstation fail-proof in the cloud and you don't have to think about backups again.
Not to mention the other benefits you get (high performance, faster order execution, high-availability....).

For my private pc, I've had good experience with duplicati. Very easy to use and it is also possible to encrypt your whole backup with AES-256!

Reply With Quote
The following user says Thank You to GuerillaTrading for this post:
 
(login for full post details)
  #25 (permalink)
Northern Germany
 
Experience: Intermediate
Platform: NT
Trading: FDAX, CL
 
vvhg's Avatar
 
Posts: 1,583 since Mar 2011
Thanks: 1,016 given, 2,816 received


GuerillaTrading View Post
There is an easy solution for this problem. Get a trading workstation fail-proof in the cloud and you don't have to think about backups again.

Nothing is fail-proof. At the end everything is down to probabilities, not only in trading, but there it's most obvious.


Vvhg


Sent from my iPad using Tapatalk HD

Hic Rhodos, hic salta.
Reply With Quote
The following 2 users say Thank You to vvhg for this post:
 
(login for full post details)
  #26 (permalink)
Montreal, Canada
 
Experience: Intermediate
Platform: Multicharts
Broker: Interactive Broker
Trading: Forex
 
Posts: 263 since Sep 2010
Thanks: 440 given, 91 received


Peter2150 View Post
I think I am more thorough than most. But I absolutely can't afford to lose data, nor can I afford downtime.

I have two desktops, and a desktop replacement laptop. All three machines have identical setup's and folder names. The laptop is just for emergency use.

On both desktops, I run an imaging program from Storagecraft, called Shadowprotect. It has a feature called continuous incrementals, and it takes an incremental image every 30 minutes in about 7 minutes. At the end of the day it collapses them into a daily. I just let it run. From these I can restore the computer from the image to previous points in time, as well as mount the images and pull data from them. These images are stored on a 2nd internal drive, and also synced to an external drive.

Then I use a synchronize program to sync to two external drives on the primary business machine. I then sync two the other machines external drive, and c: drive. The reason I do this is just because of what has happened. Turn on the main business machine in the am and it's dead. I can just go to the 2nd one and continue uninterrupted.

Finally I have cloud storage via two different companies. I use Idrive and Jungledisk. I picked these as they both offer private encrytion and satisfactory retentions policy's. Both retain multiple versions. They were chosen with geographical considerations. Idrive servers are in California and Jungledisk in Atlanta and Chicago. Although Jungledisk might seem like a strange name, the company behind them is Rackspace which is well known to the corporate world.


IMPORTANT POINT. All these backup methods have had the restore capability tested and proven. Without testing the restore, you have no backup.

Pete

Whow! I should had read this tread and your input before. You seem to have a nice and appropriate solution to a disaster recovery

I will try to reproduce your installation. Any clarification you could add?

I will print your message and start to proceed to the installation. I'm in the middle of a crisis and my backup solution did not work at all.

Martin

Reply With Quote
 
(login for full post details)
  #27 (permalink)
Montreal, Canada
 
Experience: Intermediate
Platform: Multicharts
Broker: Interactive Broker
Trading: Forex
 
Posts: 263 since Sep 2010
Thanks: 440 given, 91 received

Hello again Peter. I would like some clarification


Quoting 
On both desktops, I run an imaging program from Storagecraft, called Shadowprotect. It has a feature called continuous incrementals, and it takes an incremental image every 30 minutes in about 7 minutes. At the end of the day it collapses them into a daily. I just let it run.

My understanding:
- Both PC prepare an image and wrote it under a 2' Internal hard drive.


Quoting 
These images are stored on a 2nd internal drive, and also synced to an external drive. Then I use a synchronize program to sync to two external drives on the primary business machine. I then sync two the other machines external drive, and c: drive. The reason I do this is just because of what has happened.

- There I lost you. Is it done all automatically? How a change in one PC is reflected on the 2'. With your solution, both PC are synchronized. A file modification in either of them is reflected on the other one.

Could you elaborate on the synchronization process


Quoting 
Finally I have cloud storage via two different companies.

- Is it use for your individual file or your disk image.


Quoting 
IMPORTANT POINT. All these backup methods have had the restore capability tested and proven. Without testing the restore, you have no backup.

I truly re-enforce this wording. I taught that my external in line backup solution was a good one until I had to experience it

Martin

P.S. Excuse my English. It is not perfect

Reply With Quote
 
(login for full post details)
  #28 (permalink)
Australia
 
Experience: Advanced
Platform: NinjaTrader, Multicharts
Trading: Spot Forex, Gold, Silver
 
Posts: 176 since Oct 2010
Thanks: 114 given, 114 received

I use a 4-bay Synology NAS. The drives are in RAID-5.

All my media and documents etc are on the NAS and I access them through WinFS shared folders on all my workstations. It is quite good because all your stuff is backed up automatically simply by using the shared folders. It also has built-in recycle bin so if you erase a file by mistake, recovery is very easy to do.

Other features are

- built in media server, itunes server
- OpenVPN server - so you can come in from the outside and have secure access to your entire LAN
- IOS & Android apps to access your data via smartphone
- It is ARM cpu running linux so you can run your apps like torrents or apache/php/mysql etc

Reply With Quote
 
(login for full post details)
  #29 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,733 since Jun 2009
Thanks: 32,283 given, 97,470 received


Big Mike View Post
Just a quick note:

Carbonite seems limited to 4GB files, not 100% clear
iDrive 10GB
Jungledisk 5GB

This kind of thing still upsets me. I have many files over this size that are important data to me (compressed archives generally). I suspect for most it is not an issue, but thought I would report my findings.

Mike

Best I can tell, iDrive removed the 10GB limit (from their website at least). I just purchased the 150GB annual plan, and will report back as I test things. I have multiple files over 4GB and 10GB, so we will find out in coming days.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #30 (permalink)
Washington DC
 
Experience: Intermediate
Platform: NinjaTrader
 
Posts: 214 since Jun 2009
Thanks: 110 given, 117 received


Big Mike View Post
Best I can tell, iDrive removed the 10GB limit (from their website at least). I just purchased the 150GB annual plan, and will report back as I test things. I have multiple files over 4GB and 10GB, so we will find out in coming days.

Mike

Mike that should be an issue in any case. Almost all the archiving programs like Winrar and winzip, all you you to break up files into smaller sizes. I've used this and rarely see any problem putting them back together.

Pete

Reply With Quote
 
(login for full post details)
  #31 (permalink)
Near Dallas, Texas, US
 
Experience: Advanced
Platform: NinjaTrader
Broker: ZenFire
Trading: CL
 
Gary's Avatar
 
Posts: 1,070 since Jun 2009
Thanks: 502 given, 2,234 received


Peter2150 View Post
Mike that should be an issue in any case. Almost all the archiving programs like Winrar and winzip, all you you to break up files into smaller sizes. I've used this and rarely see any problem putting them back together.

Pete

Peter,

Except that requires more time, CPU, and potentially quite a bit of extra space. Ideally, I would think most want to backup the data without having to spend more resources before, during, and after backup. And, then do the same in reverse when restoring.

Gary

As consistently profitable traders.. "We get paid to wait, and we wait to get paid."
Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #32 (permalink)
Bedford, UK
 
Experience: Beginner
Platform: SierraChart/TT Feed
Trading: NQ
 
keymoo's Avatar
 
Posts: 129 since Mar 2012
Thanks: 212 given, 213 received

One easy solution and better than any other online backup I've found including Carbonite.

Backblaze. Easiest Online Backup Service - Backblaze

Simply awesome.

Follow me on Twitter Reply With Quote
The following user says Thank You to keymoo for this post:
 
(login for full post details)
  #33 (permalink)
Denver, CO USA
 
 
Posts: 15 since Sep 2012
Thanks: 1 given, 9 received

I've had so many times where a backup came in real handy (I play around with my comp a lot as a techie so a lot of times I need to restore everything after I screw something up). System images are the way to go for sure! An image backup will back up everything (not just data, but all your programs 2) for you so all you'll have to do is reformat and restore.

I use symantc backup exec system recovery for my main computer (windows XP). I've been using window's backup tool on my windows 7 laptop, but haven't had to restore yet so my personal opinion on window's built in backup is reserved for now.

Reply With Quote
 
(login for full post details)
  #34 (permalink)
Washington DC
 
Experience: Intermediate
Platform: NinjaTrader
 
Posts: 214 since Jun 2009
Thanks: 110 given, 117 received


keymoo View Post
One easy solution and better than any other online backup I've found including Carbonite.

Backblaze. Easiest Online Backup Service - Backblaze

Simply awesome.

very weak in versioning. Only keeps the last 4 weeks. Idrive keeps up to 30 copies, only charging for the first file.
Jungle Disk(Rackspace) allows you to set the number of versions, or keep all of them. This is the best flexibility.

Reply With Quote
The following user says Thank You to Peter2150 for this post:
 
(login for full post details)
  #35 (permalink)
Bedford, UK
 
Experience: Beginner
Platform: SierraChart/TT Feed
Trading: NQ
 
keymoo's Avatar
 
Posts: 129 since Mar 2012
Thanks: 212 given, 213 received


Peter2150 View Post
very weak in versioning. Only keeps the last 4 weeks. Idrive keeps up to 30 copies, only charging for the first file.
Jungle Disk(Rackspace) allows you to set the number of versions, or keep all of them. This is the best flexibility.

Nothing comes remotely close to BackBlaze for value for money. I'm not affiliated to them in any way, I'm just very happy with them. There's no way I'd pay the huge costs of Jungle or iDrive for little extra benefit.

I have 600 GB of data to back up over the internet (i.e. off site). This data is growing with my trading videos, logs, screenshots, etc..

Backblaze: $4 per month per PC (unlimited data)
Jungle Disk Personal: $89.25 per month (cost rises with data increases)
iDrive Pro Family Pack 500 GB Limit: $15.95 per month.
iDrive Pro Business 1000 GB Limit: $79.95 per month
Carbonite: $5 per month. Good price, but it's dog slow

I only need the last 4 weeks because if I accidentally delete something, I usually know within that month. If I delete something and it's before that month I have my own internal backups taken care of by BackupPC to my local backup server. Internet backup (for me) is there as a last resort. For example fire, flood, backup server corruption and simultaneous failure of my main PC (unlikely).

My local backups take care of versioning for the past 3 years of data.

Follow me on Twitter Reply With Quote
The following 3 users say Thank You to keymoo for this post:
 
(login for full post details)
  #36 (permalink)
Washington DC
 
Experience: Intermediate
Platform: NinjaTrader
 
Posts: 214 since Jun 2009
Thanks: 110 given, 117 received


keymoo View Post
Nothing comes remotely close to BackBlaze for value for money. I'm not affiliated to them in any way, I'm just very happy with them. There's no way I'd pay the huge costs of Jungle or iDrive for little extra benefit.

I have 600 GB of data to back up over the internet (i.e. off site). This data is growing with my trading videos, logs, screenshots, etc..

Backblaze: $4 per month per PC (unlimited data)
Jungle Disk Personal: $89.25 per month (cost rises with data increases)
iDrive Pro Family Pack 500 GB Limit: $15.95 per month.
iDrive Pro Business 1000 GB Limit: $79.95 per month
Carbonite: $5 per month. Good price, but it's dog slow

I only need the last 4 weeks because if I accidentally delete something, I usually know within that month. If I delete something and it's before that month I have my own internal backups taken care of by BackupPC to my local backup server. Internet backup (for me) is there as a last resort. For example fire, flood, backup server corruption and simultaneous failure of my main PC (unlikely).

My local backups take care of versioning for the past 3 years of data.

Couple of things. I use online backups for exactly same reason. For me data loss is critical

Your pricing comparisons are very problematic. Either wrong or incomplete.

Jungle Disk: Totally incorrect. I use the Desktop Edition. It is $3/month plus storage with the first 5gb free. I retain all versions, backup once or twice a day, and my monthly billl is up to $4.15 a month. Far cry from what you quoted.

Idrive: You are correct, but most people might not need the family plan, and can use the regular plan which is only $5.95 for 500gb

Idrive Pro: Seems outrageous at the price for a 1000gb until you realize it is for 99 machines.

There are some other factors that need to be considered if you value your back. Geography.

Idrive is located in California, and they say there servers are Seismically protected, and I have no reason to doubt them, but the Japan mess demonstrated that there are differences in the protection

On JungleDisk, I store on Rackspace servers, and there servers are located up and down the central part of the US.

Even though I do significant back ups, I do the cloud for protection, and I want them to be geographically diverse.

Pete

Reply With Quote
 
(login for full post details)
  #37 (permalink)
Bedford, UK
 
Experience: Beginner
Platform: SierraChart/TT Feed
Trading: NQ
 
keymoo's Avatar
 
Posts: 129 since Mar 2012
Thanks: 212 given, 213 received


Peter2150 View Post
Your pricing comparisons are very problematic. Either wrong or incomplete.

Jungle Disk: Totally incorrect. I use the Desktop Edition. It is $3/month plus storage with the first 5gb free. I retain all versions, backup once or twice a day, and my monthly billl is up to $4.15 a month. Far cry from what you quoted

Really? According to https://www.jungledisk.com/personal/desktop/pricing/

JungleDisk
Storage
Only $0.15 per GB-Month of storage used
First 5GB Storage FREE

If you do the maths it works out very expensive. I've just checked and I have over 700 GB to back up. That's over $100 per month.

Follow me on Twitter Reply With Quote
 
(login for full post details)
  #38 (permalink)
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,733 since Jun 2009
Thanks: 32,283 given, 97,470 received

It looks like IDrive for Windows is causing me some grief with extraordinary memory consumption (10GB).

Does anyone have any updated or recent reviews for a cloud service where I can purchase 500GB of storage and it supports 4GB files, with no exclusion on file type (videos, archives, etc).

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #39 (permalink)
Karlsruhe, Germany
 
Experience: Intermediate
Platform: MultiCharts, AmiBroker
Broker: IB/IQFeed
Trading: FDAX, ES
 
medias's Avatar
 
Posts: 60 since Jul 2009
Thanks: 30 given, 49 received


Big Mike View Post
It looks like IDrive for Windows is causing me some grief with extraordinary memory consumption (10GB).

Does anyone have any updated or recent reviews for a cloud service where I can purchase 500GB of storage and it supports 4GB files, with no exclusion on file type (videos, archives, etc).

Mike

Try this:
https://www.wuala.com/en/

I am using this service for several years for a daily backup. Very stable, no issues so far.

Reply With Quote
The following user says Thank You to medias for this post:
 
(login for full post details)
  #40 (permalink)
Detroit, Michigan
 
Experience: Beginner
Platform: Ninja Trader
Trading: Futures
 
Posts: 299 since Nov 2011
Thanks: 497 given, 286 received

BigMike's scenario happened to me about two months ago - my hard drive went on my laptop.

My backup was to a simple external USB hard drive (but unfortunately the backup was about six months old - due to a move and just plain forgetting to backup again). Most of my important files were backed up but there were a few that weren't.

The most important thing I did prior to the hard drive crash that saved my behind was consistently backing up my emails to a Gmail account. All my incoming POP email also goes to my Gmail account and although inconvenient I bcc my Gmail account on all outgoing POP email. This activity saved me tons of time after my computer got back up to speed as it relates to finding old emails I needed going forward.

After the new hard drive was installed, it took a few days to get the computer back up to snuff. The only things I changed was backing up documents and emails more often to an external USB.

The crash was a pain (it was the first ever for me) but manageable.

Reply With Quote
 
(login for full post details)
  #41 (permalink)
Chicago, USA
 
Experience: Advanced
Platform: TS, NT, TOS, SSE, IRT
Trading: ES, currency futures, options, stocks
 
Posts: 195 since Dec 2010
Thanks: 328 given, 273 received

I’m still using my HP MediaSmart Server EX487 that I bought in 2009 to back up all my computers and notebooks. It was one of the best products HP ever made. It wakes up all your computers on a specified time, and backs up an image of each one, so you can recover the entire HD if need be. In addition, you can open individual images of the hard drive to access specific files. It’s based on Windows Home Server 2003, so works perfectly and seamlessly on a Windows network environment. After I upgraded everything to Windows 10, once in a while I have issues with the old connector software, but it’s still usable, just that sometimes one of the computers loses connection with the server.

Unfortunately, HP stopped producing and supporting this line of products a long time ago and it seems that just now a company (Thecus) is coming with something based on Windows Server again. So, I may get another year of my HP EX487 and then I may consider this product:

Windows Storage Server

If anyone has used the product above, or know of any other Windows Home Server product, please let me know.
Thanks.

Reply With Quote
The following user says Thank You to dstrader for this post:


futures io Trading Community Off-Topic Tech Support > Disaster Strikes! Backup your data


Last Updated on January 24, 2016


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
 

Journal Challenge w/$1500 prizes from Topstep!

February
 

Identifying Setups & Targets Using Profile Charts w/Trevor & Tradovate

Feb 25
 

Battlestations! Show us your trading desk - $1,500 in prizes!

March
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts