headers
Miles Fidelman | 1 Nov 2011 01:38

possibly silly question (raid failover)

Hi Folks,

I've been exploring various ways to build a "poor man's high 
availability cluster."  Currently I'm running two nodes, using raid on 
each box, running DRBD across the boxes, and running Xen virtual 
machines on top of that.

I now have two brand new servers - for a total of four nodes - each with 
four large drives, and four gigE ports.

Between the configuration of the systems, and rack space limitations, 
I'm trying to use each server for both storage and processing - and been 
looking at various options for building a cluster file system across all 
16 drives, that supports VM migration/failover across all for nodes, and 
that's resistant to both single-drive failures, and to losing an entire 
server (and it's 4 drives), and maybe even losing two servers (8 drives).

The approach that looks most interesting is Sheepdog - but it's both 
tied to KVM rather than Xen, and a bit immature.

But it lead me to wonder if something like this might make sense:
- mount each drive using AoE
- run md RAID 10 across all 16 drives one one node
- mount the resulting md device using AoE
- if the node running the md device fails, use pacemaker/crm to 
auto-start an md device on another node, re-assemble and republish the array
- resulting in a 16-drive raid10 array that's accessible from all nodes

Or is this just silly and/or wrongheaded?
(Continue reading)

Permalink | Reply | 26 comments |
headers
NeilBrown | 1 Nov 2011 03:33
Picon
Gravatar

Re: [PATCH 1/3] conf_match(): Remove dead code

On Mon, 31 Oct 2011 14:53:52 +0100 Jes.Sorensen <at> redhat.com wrote:

> From: Jes Sorensen <Jes.Sorensen <at> redhat.com>
> 
> devname is always NULL, hence if () statement will always fail.
> 
> Signed-off-by: Jes Sorensen <Jes.Sorensen <at> redhat.com>
> ---
>  config.c |    9 ---------
>  1 files changed, 0 insertions(+), 9 deletions(-)
> 
> diff --git a/config.c b/config.c
> index c0a6baa..b57ba50 100644
> --- a/config.c
> +++ b/config.c
>  <at>  <at>  -1023,7 +1023,6  <at>  <at>  struct mddev_ident *conf_match(struct mdinfo *info, struct supertype *st)
>  {
>  	struct mddev_ident *array_list, *match;
>  	int verbose = 0;
> -	char *devname = NULL;
>  	array_list = conf_get_ident(NULL);
>  	match = NULL;
>  	for (; array_list; array_list = array_list->next) {
>  <at>  <at>  -1044,14 +1043,6  <at>  <at>  struct mddev_ident *conf_match(struct mdinfo *info, struct supertype *st)
>  					array_list->devname);
>  			continue;
>  		}
> -		if (array_list->devices && devname &&
> -		    !match_oneof(array_list->devices, devname)) {
> -			if (verbose >= 2 && array_list->devname)
(Continue reading)

Permalink | Reply | 1 comment |
headers
NeilBrown | 1 Nov 2011 04:57
Picon
Gravatar

Re: [PATCH 00/11] Memory/resource leaks and unchecked return fixes

On Mon, 31 Oct 2011 15:02:28 +0100 Jes.Sorensen <at> redhat.com wrote:

> From: Jes Sorensen <Jes.Sorensen <at> redhat.com>
> 
> Hi,
> 
> This is another pile of patches to fixup memory leaks and buffer
> overflows found in the coverity run. 

Thanks Jes,
  I have applied all of these.

NeilBrown

> 
> Cheers,
> Jes
> 
> 
> Jes Sorensen (11):
>   Fix memory leaks in reshape_array()
>   Fix memory leak
>   Fix memory leak
>   Fix memory leak of 'st3' in array_try_spare()
>   partition_try_spare() use closedir() to release DIR * returned by
>     opendir()
>   Fix memory leak
>   Add missing return in case of trying to grow sub-array
>   Avoid memory leak
>   policy_add(): Add missing va_end()
(Continue reading)

Permalink | Reply | 0 comments |
headers
NeilBrown | 1 Nov 2011 05:48
Picon
Gravatar

Re: [PATCH] kill-subarray: fix, IMSM cannot kill-subarray with unsupported metadata

On Mon, 31 Oct 2011 16:52:22 +0000 "Labun, Marcin" <Marcin.Labun <at> intel.com>
wrote:

> 
> 
> > -----Original Message-----
> > From: NeilBrown [mailto:neilb <at> suse.de]
> > Sent: Monday, October 31, 2011 1:32 AM
> > To: Labun, Marcin
> > Cc: linux-raid <at> vger.kernel.org; Williams, Dan J
> > Subject: Re: [PATCH] kill-subarray: fix, IMSM cannot kill-subarray with
> > unsupported metadata
> > 
> > On Fri, 28 Oct 2011 14:46:57 +0000 "Labun, Marcin"
> > <Marcin.Labun <at> intel.com>
> > wrote:
> > 
> > > Hi Neil,
> > > Please review modified patch in response for your comments.
> > > I have introduced two flags, one for activation blocking, second for
> > > container wide reshapes
> > > Blocking:
> > > - some arrays in container are blocked, the others can be activated
> > > and reshaped,
> > > - some arrays are blocked, others can be activated but container wide
> > reshaped is blocked for all.
> > >
> > > Thanks,
> > > Marcin Labun
> >
(Continue reading)

Permalink | Reply | 3 comments |
headers
NeilBrown | 1 Nov 2011 06:39
Picon
Gravatar

Re: [PANIC] : kernel BUG at drivers/md/raid5.c:2756!

On Mon, 31 Oct 2011 14:29:38 -0700 Manish Katiyar <mkatiyar <at> gmail.com> wrote:

> I was running following script (trying to reproduce an ext4 error
> reported in another thread) and the kernel dies with below error.
> 
> The place where it crashes is :-
> 2746 static void handle_parity_checks6(raid5_conf_t *conf, struct
> stripe_head *sh,
> 2747                                   struct stripe_head_state *s,
> 2748                                   int disks)
> 2749 {
> .....
> 2754         set_bit(STRIPE_HANDLE, &sh->state);
> 2755
> 2756         BUG_ON(s->failed > 2);   <============== !!!!
> 
> 
> 
> [ 9663.343974] md/raid:md11: Disk failure on loop3, disabling device.[
> 9663.343976] md/raid:md11: Operation continuing on 4 devices.[
> 9668.547289] ------------[ cut here ]------------[ 9668.547327] kernel
> BUG at drivers/md/raid5.c:2756![ 9668.547356] invalid opcode: 0000
> [#1] SMP [ 9668.547388] Modules linked in: parport_pc ppdev
> snd_hda_codec_hdmi snd_hda_codec_conexant aesni_intel cryptd aes_i586
> aes_generic nfsd exportfs btusb nfs bluetooth lockd fscache
> auth_rpcgss nfs_acl sunrpc binfmt_misc joydev snd_hda_intel
> snd_hda_codec fuse snd_hwdep thinkpad_acpi snd_pcm snd_seq_midi
> uvcvideo snd_rawmidi snd_seq_midi_event arc4 snd_seq videodev i915
> iwlagn mxm_wmi drm_kms_helper drm snd_timer psmouse snd_seq_device
> serio_raw mac80211 snd tpm_tis tpm nvram tpm_bios intel_ips cfg80211
(Continue reading)

Permalink | Reply | 6 comments |
headers
Manish Katiyar | 1 Nov 2011 07:09
Picon
Gravatar

Re: [PANIC] : kernel BUG at drivers/md/raid5.c:2756!

> I think you were quite unlucky to hit this and that you will find it hard to
> reproduce. :-(

Well, its always reproducible on my ubuntu when I run that script.

--

-- 
Thanks -
Manish
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 3 comments |
headers
Jes Sorensen | 1 Nov 2011 07:22
Picon
Favicon

Re: [PATCH 1/3] conf_match(): Remove dead code

On 11/01/11 03:33, NeilBrown wrote:
> Hi Jes,
>  thanks for these 3 patches.
>  In each case I chose to remove different code :-)
> 
>  In this case search_mdstat was nearly identical to conf_match, but uses the
>  bits you removed.  So I unified the two.
> 
>  For the 'Kill' fix, I removed a different test on 'force' so that the printf
>  that you removed could not actually be reached.

That is great! There are times where I am not quite sure what the
original intention of the code was, so I am just glad the patches is
proving to be of use :)

Cheers,
Jes
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 0 comments |
headers
NeilBrown | 1 Nov 2011 07:29
Picon
Gravatar

Re: [PANIC] : kernel BUG at drivers/md/raid5.c:2756!

On Mon, 31 Oct 2011 23:09:04 -0700 Manish Katiyar <mkatiyar <at> gmail.com> wrote:

> > I think you were quite unlucky to hit this and that you will find it hard to
> > reproduce. :-(
> 
> Well, its always reproducible on my ubuntu when I run that script.
> 

Excellent!!   That means you can reliably see if the patch fixes it. :-)

I guess I was thinking of 'real' devices which are slower than XOR.  You were
using loop back device on files in /tmp which are presumably always in memory.
I guess it would be a lot easier to hit in that case.

Thanks,
NeilBrown
Permalink | Reply | 2 comments |
headers
David Brown | 1 Nov 2011 10:14
Favicon

Re: possibly silly question (raid failover)

On 01/11/2011 01:38, Miles Fidelman wrote:
> Hi Folks,
>
> I've been exploring various ways to build a "poor man's high
> availability cluster." Currently I'm running two nodes, using raid on
> each box, running DRBD across the boxes, and running Xen virtual
> machines on top of that.
>
> I now have two brand new servers - for a total of four nodes - each with
> four large drives, and four gigE ports.
>
> Between the configuration of the systems, and rack space limitations,
> I'm trying to use each server for both storage and processing - and been
> looking at various options for building a cluster file system across all
> 16 drives, that supports VM migration/failover across all for nodes, and
> that's resistant to both single-drive failures, and to losing an entire
> server (and it's 4 drives), and maybe even losing two servers (8 drives).
>
> The approach that looks most interesting is Sheepdog - but it's both
> tied to KVM rather than Xen, and a bit immature.
>
> But it lead me to wonder if something like this might make sense:
> - mount each drive using AoE
> - run md RAID 10 across all 16 drives one one node
> - mount the resulting md device using AoE
> - if the node running the md device fails, use pacemaker/crm to
> auto-start an md device on another node, re-assemble and republish the
> array
> - resulting in a 16-drive raid10 array that's accessible from all nodes
>
(Continue reading)

Permalink | Reply | 20 comments |
headers
Johannes Truschnigg | 1 Nov 2011 10:26

Re: possibly silly question (raid failover)

Hi Miles,

On Mon, Oct 31, 2011 at 08:38:16PM -0400, Miles Fidelman wrote:
> Hi Folks,
> 
> I've been exploring various ways to build a "poor man's high
> availability cluster."  Currently I'm running two nodes, using raid
> on each box, running DRBD across the boxes, and running Xen virtual
> machines on top of that.
> [...]

while I do note that I don't answer your question at hand, I'm still inclined
to ask if you do know Ganeti (http://code.google.com/p/ganeti/) yet? It offers
pretty much everything you seem to want to have.

--

-- 
with best regards:
- Johannes Truschnigg ( johannes <at> truschnigg.info )

www:   http://johannes.truschnigg.info/
phone: +43 650 2 133337
xmpp:  johannes <at> truschnigg.info

Please do not bother me with HTML-eMail or attachments. Thank you.
Permalink | Reply | 2 comments |

Gmane