Shrinking mdadm w/XFS to over-provision SSD - Tech Support | futures io social day trading
futures io futures trading


Shrinking mdadm w/XFS to over-provision SSD
Updated: Views / Replies:913 / 2
Created: by Big Mike Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Shrinking mdadm w/XFS to over-provision SSD

  #1 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,214 received

Shrinking mdadm w/XFS to over-provision SSD

I emailed this to a friend, but thought I would post it here as well as I need some help.


Big Mike
Given now that two drives out of four have failed in a year, I really need to re-provision both of these servers so that they are only using 80% of the SSD space (over-provision). But I cannot figure out a good way to do so.

I donít know of any utility that would let me shrink a mdadm array with an XFS filesystem. So this means I have to back everything up, then do a secure erase on the SSD to free up the blocks, then re-partition the drives so only 80% is used by mdadm raid, and then restore the backup to that now smaller XFS partition.

But I donít know how to do this really. I am not sure if a simple tar backup is good enough with all the symlinks everywhere, plus is there data on the file level that describes the total size of the partitions or mdadm array which upon restore would be incorrect?

Any ideas? I definitely do not want to re-create the servers. I just want to do a backup and restore at most. I am very uncomfortable trusting tar because Iíve had so many problems with it in the past with symlinks, and Iíve never tried to tar the entire / and expect it to restore correctly. I am more comfortable using something like CloneZilla to clone an entire partition, assuming that CloneZilla will let you restore to a smaller partition.

After I emailed him, I looked around some more and found some people using GParted and rsync, but this is without mdadm which won't work for me.

Utechu: Cloning Ubuntu to a smaller disk

I also found an article on CloneZilla about restoring a cloned partition to a smaller disk:

Clonezilla restore to smaller hard drive - Spiceworks

Clonezilla - Advanced Mode

The consensus was generally to use the -icds option, and it sounds like it would work. But since these are live production servers I have a lot of testing to do in a VM before I attempt it. Even if it would work, it's still a lot of work and I am unsure how mdadm will complicate things.

If anyone has advice let me know.

If you are unaware of SSD over provisioning, google it. Short version: I need to shrink mdadm and XFS (impossible, so will take backup+restore of some sort) so that only 80% of the drive is addressable by the OS, leaving 20% over-provisioned. To make things more interesting, the only way to gain this on an existing formatted drive would to be to do an SSD Secure Erase after the backup, so that each block is "owned" by the SSD and not addressable by the OS, freeing it up. So that means I had better have a good backup!

If anyone has direct experience with this or cares to do some VM simulations for me let me know. Debian 7 wheezy on kernel 3.12.0, mdadm 3.2.5, (2) disk RAID 1:

 
Code
root@phoenix:~# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[0] sdb[2]
      249926464 blocks super 1.2 [2/2] [UU]
Keep in mind that during all of this futures.io (formerly BMT) would be off-line, so I am looking for a compact/quick method. I can use a local server (NFS or SSH) to store the image at 1Gb link, so that will help.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #2 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,214 received

Saving this for later.

Technology of Today - Thoughts & Exploration: Using Clonezilla to Clone From a Larger Hard Drive to a Smaller Drive

Also I don't think I mentioned that the filesystem is XFS. Everything I am reading so far is only talking about raw file systems. The issue is when you add in mdadm it seems, not sure on this one...

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #3 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,214 received


@sam028 recommended


Quoting 
I would use xfsdump, that will dump all the file-system.
In the worst case, a simple dd if=/dev/... can do the job.
All these can be sent to stdout, which can be send on another host using ssh.

And I had looked at xfsdump even before I started this thread, but just afraid to use it on something like / but I guess I will take a look and see if I can make it work in a VM.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote

Reply



futures io > > > > Shrinking mdadm w/XFS to over-provision SSD

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
SSD (Solid State Disk) could improve trading speed? LukeGeniol Tech Support 28 March 26th, 2013 10:43 AM
installing a new SSD madLyfe Tech Support 7 November 22nd, 2012 01:46 AM
Obamacare Critics Prepare to Battle Over This Key Provision Quick Summary News and Current Events 0 July 7th, 2012 01:30 PM
Companies use popular provision to avoid income taxes kbit News and Current Events 0 April 9th, 2012 09:22 PM
SHRINKING ES TRADING RANGE tigertrader The Elite Circle 7 November 1st, 2010 05:59 PM


All times are GMT -4. The time now is 04:05 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-11 in 0.08 seconds with 19 queries on phoenix via your IP 54.226.113.250