headers
Stephane LAPIE | 1 Nov 2011 04:42

Re: RFC: GEOM MULTIPATH rewrite

Hello,

First of all, many thanks. I am going to test your patch on 9.0-RC1, and
try to backport it to 8.2 (which is the main version I am currently
using at work, in the environment where I have a critical need for FC
multipath redundancy...)

Again, thanks for your efforts. I hope to be giving feedback soon.

Cheers,

On 11/01/2011 05:10 AM, Alexander Motin wrote:
> Hi.
> 
> Attempt to fix some GEOM MULTIPATH issues made me almost rewrite it. So
> I would like to present my results and request for testing and feedback.
> 
> The main changes:
>  - Improved locking and destruction process to fix crashes in many cases.
>  - Improved "automatic" configuration method to make it safe by reading
> metadata back from all specified paths after writing to one.
>  - Added provider size check to reduce chance of conflict with other
> GEOM classes.
>  - Added "manual" configuration method without using on-disk metadata.
>  - Added "add" and "remove" commands to manage paths manually.
>  - Failed paths no longer dropped from GEOM, but only marked as FAIL and
> excluded from I/O operations.
>  - Automatically restore failed paths when all others paths are marked
> as failed, for example, because of device-caused (not transport) errors.
>  - Added "fail" and "restore" commands to manually control FAIL flag.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Andrey V. Elsukov | 1 Nov 2011 08:23
Picon
Favicon

Re: gmirror failed with error 19.

On 28.10.2011 13:48, Sascha Klauder wrote:
>  I've got bitten by this as well when trying a source up-
> grade of a freshly installed 8.2-RELEASE system that had
> gmirror configured after installation according to the 
> procedure in the Handbook.
> 
>  The 9.0-RC1 kernel drops to the mountroot prompt when
> booting with
> 
>  GEOM_MIRROR: Device mirror/gm0 launched (2/2).
>  GEOM_PART: partition 1 has end offset beyond last LBA: 490350671 > 490350670
>  GEOM_PART: integrity check failed (mirror/gm0, MBR)

This is the main problem. Your MBR' slice is bigger than actual space
you have. The only way to fix this - recreate slice.
You can break your mirror, recreate the slice (NOTE: you must preserve
one sector for the gmirror's meta-data), then copy your data to the
newly created slice, then reboot from the new slice and recreate mirror.

--

-- 
WBR, Andrey V. Elsukov
Permalink | Reply | 3 comments |
headers
Pawel Jakub Dawidek | 1 Nov 2011 13:39
Picon
Favicon

Re: RFC: GEOM MULTIPATH rewrite

On Mon, Oct 31, 2011 at 10:10:14PM +0200, Alexander Motin wrote:
> Hi.
> 
> Attempt to fix some GEOM MULTIPATH issues made me almost rewrite it. So
> I would like to present my results and request for testing and feedback.
> 
> The main changes:
>  - Improved locking and destruction process to fix crashes in many cases.
>  - Improved "automatic" configuration method to make it safe by reading
> metadata back from all specified paths after writing to one.
>  - Added provider size check to reduce chance of conflict with other
> GEOM classes.
>  - Added "manual" configuration method without using on-disk metadata.
>  - Added "add" and "remove" commands to manage paths manually.
>  - Failed paths no longer dropped from GEOM, but only marked as FAIL and
> excluded from I/O operations.
>  - Automatically restore failed paths when all others paths are marked
> as failed, for example, because of device-caused (not transport) errors.
>  - Added "fail" and "restore" commands to manually control FAIL flag.
>  - GEOM is now destroyed on last provider disconnection. IMHO it is
> right to do if device was completely removed.
>  - Added optional Active/Active mode support. Unlike Active/Passive
> mode, load evenly distributed between all working paths. If supported by
> device, it allows to significantly improve performance, utilizing
> bandwidth of all paths. It is controlled by -A option during creation.
> Disabled by default now.
>  - Improved `status` and `list` commands output.
> 
> Latest patch can be found here:
> http://people.freebsd.org/~mav/gmultipath4.patch
(Continue reading)

Permalink | Reply | 2 comments |
headers
Alexander Motin | 1 Nov 2011 21:24
Picon
Favicon

Re: RFC: GEOM MULTIPATH rewrite

On 01.11.2011 19:50, Dennis Kögel wrote:
> Not sure if replying on-list or off-list makes more sense...

Replying on-list could share experience to other users.

> Anyway, some first impressions, on stable/9:
> 
> The lab environment here is a EMC VNX / Clariion SAN, which has two Storage Processors, connected to
different switches, connected to two isp(4)s on the test machine. So at any time, the machine sees four
paths, but only two are available (depending on which SP owns the LUN).
> 
> 580# camcontrol devlist
> <DGC VRAID 0531>                   at scbus0 target 0 lun 0 (da0,pass0)
> <DGC VRAID 0531>                   at scbus0 target 1 lun 0 (da1,pass1)
> <DGC VRAID 0531>                   at scbus1 target 0 lun 0 (da2,pass2)
> <DGC VRAID 0531>                   at scbus1 target 1 lun 0 (da3,pass3)
> <COMPAQ RAID 1(1VOLUME OK>         at scbus2 target 0 lun 0 (da4,pass4)
> <COMPAQ RAID 0  VOLUME OK>         at scbus2 target 1 lun 0 (da5,pass5)
> <hp DVD D  DS8D3SH HHE7>           at scbus4 target 0 lun 0 (cd0,pass6)
> 
> I miss the ability to "add" disks to automatic mode multipaths, but I (just now) realized this only makes
sense when gmultipath has some kind of path checking facility (like periodically trying to read sector 0
of each configured device, this is was Linux' devicemapper-multipathd does).

In automatic mode other paths supposed to be detected via metadata
reading. If in your case some paths are not readable, automatic mode
can't work as expected. By the way, could you describe how your
configuration supposed to work, like when other paths will start
working? Only when second storage processor itself detect that first one
is dead or it suppose to be controlled somehow? If booted with verbose
(Continue reading)

Permalink | Reply | 10 comments |
headers
Ivan Voras | 1 Nov 2011 23:23
Picon
Favicon
Gravatar

Re: RFC: GEOM MULTIPATH rewrite

On 31.10.2011. 21:10, Alexander Motin wrote:
> Hi.
>
> Attempt to fix some GEOM MULTIPATH issues made me almost rewrite it. So
> I would like to present my results and request for testing and feedback.

Good ideas, gmultipath robustness is a very good thing. Active/active 
support is also useful if ordering issues can be solved. There's a Linux 
paper linked at

http://en.wikipedia.org/wiki/Multipath_I/O

... but apparently it doesn't say much about this problem. Could a 
simple policy such as "block all IO iff an BIO_ORDERED request is being 
processed" work well enough?

I'd only add that you should be careful to be compatible with old 
gmultipath metadata (see e.g. recent user reports with 9.0 auto-updating 
gmirror metadata).

> The main changes:
>   - Improved locking and destruction process to fix crashes in many cases.
>   - Improved "automatic" configuration method to make it safe by reading
> metadata back from all specified paths after writing to one.
>   - Added provider size check to reduce chance of conflict with other
> GEOM classes.
>   - Added "manual" configuration method without using on-disk metadata.
>   - Added "add" and "remove" commands to manage paths manually.
>   - Failed paths no longer dropped from GEOM, but only marked as FAIL and
> excluded from I/O operations.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Sascha Klauder | 3 Nov 2011 12:50
Picon

Re: gmirror failed with error 19.

On Tue, 2011-11-01 11:23 +0400, Andrey V. Elsukov wrote:
> On 28.10.2011 13:48, Sascha Klauder wrote:
> >  GEOM_MIRROR: Device mirror/gm0 launched (2/2).
> >  GEOM_PART: partition 1 has end offset beyond last LBA: 490350671 > 490350670
> >  GEOM_PART: integrity check failed (mirror/gm0, MBR)
> This is the main problem. Your MBR' slice is bigger than actual space
> you have. The only way to fix this - recreate slice.

 I've partioned and labeled the disk with sysinstall(8) from
8.2-RELEASE media.  Does 9.0 use different disk geometry cal-
culation than 8.2 or is usage of gmirror the culprit?  Both
8.2 and 9.0 kernels report the disk having 490350672 sectors.

> You can break your mirror, recreate the slice (NOTE: you must preserve
> one sector for the gmirror's meta-data), then copy your data to the
> newly created slice, then reboot from the new slice and recreate mirror.

 I think I'd rather reinstall 9.0 from scratch, getting rid
of MBR/disklabel as well.

Cheers,
-sascha
Permalink | Reply | 1 comment |
headers
ae | 6 Nov 2011 20:12
Picon
Favicon

Re: kern/159091: [geom] GEOM fails to scan nested partitions to create subdevices

Synopsis: [geom] GEOM fails to scan nested partitions to create subdevices

State-Changed-From-To: open->closed
State-Changed-By: ae
State-Changed-When: Sun Nov 6 19:09:19 UTC 2011
State-Changed-Why: 
I'm sorry, but we do not support nested BSD schemes.
So, this is by design, not a bug. I close this PR.

http://www.freebsd.org/cgi/query-pr.cgi?pr=159091
Permalink | Reply | 0 comments |
headers
FreeBSD bugmaster | 7 Nov 2011 12:07
Picon
Favicon

Current problem reports assigned to freebsd-geom <at> FreeBSD.org

Note: to view an individual PR, use:
  http://www.freebsd.org/cgi/query-pr.cgi?pr=(number).

The following is a listing of current problems submitted by FreeBSD users.
These represent problem reports covering all versions including
experimental development code and obsolete releases.

S Tracker      Resp.      Description
--------------------------------------------------------------------------------
o kern/162147  geom       [geom] mirror GPT: GPT rejected -- may not be recovera
o kern/162010  geom       [geli] panic: Provider's error should be set (error=0)
o kern/161979  geom       [geom] glabel doesn't update after newfs, and glabel s
o kern/161752  geom       [geom] glabel(8) doesn't get gpt label change
o bin/161677   geom       gpart(8) Probably bug in gptboot
o kern/160562  geom       [geom][patch] Allow to insert new component to geom_ra
o kern/160409  geom       [geli] failed to attach provider
o kern/159595  geom       [geom] [panic] panic on gmirror unload in vbox [regres
p kern/158398  geom       [headers] [patch] <geom/geom.h> includes <sys/sbuf.h> 
o kern/158197  geom       [geom] geom_cache with size>1000 leads to panics
o kern/157879  geom       [libgeom] [regression] ABI change without version bump
o kern/157863  geom       [geli] kbdmux prevents geli passwords from being enter
o kern/157739  geom       [geom] GPT labels with geom_multipath
o kern/157724  geom       [geom] gpart(8) 'add' command must preserve gap for sc
o kern/157723  geom       [geom] GEOM should not process 'c' (raw) partitions fo
o kern/157108  geom       [gjournal] dumpon(8) fails on gjournal providers
o kern/155994  geom       [geom] Long "Suspend time" when reading large files fr
o kern/154226  geom       [geom] GEOM label does not change when you modify them
o kern/150858  geom       [geom] [geom_label] [patch] glabel(8) is not compatibl
o kern/150626  geom       [geom] [gjournal] gjournal(8) destroys label
o kern/150555  geom       [geom] gjournal unusable on GPT partitions
(Continue reading)

Permalink | Reply | 0 comments |
headers
FreeBSD bugmaster | 14 Nov 2011 12:07
Picon
Favicon

Current problem reports assigned to freebsd-geom <at> FreeBSD.org

Note: to view an individual PR, use:
  http://www.freebsd.org/cgi/query-pr.cgi?pr=(number).

The following is a listing of current problems submitted by FreeBSD users.
These represent problem reports covering all versions including
experimental development code and obsolete releases.

S Tracker      Resp.      Description
--------------------------------------------------------------------------------
o kern/162147  geom       [geom] mirror GPT: GPT rejected -- may not be recovera
o kern/162010  geom       [geli] panic: Provider's error should be set (error=0)
o kern/161979  geom       [geom] glabel doesn't update after newfs, and glabel s
o kern/161752  geom       [geom] glabel(8) doesn't get gpt label change
o bin/161677   geom       gpart(8) Probably bug in gptboot
o kern/160562  geom       [geom][patch] Allow to insert new component to geom_ra
o kern/160409  geom       [geli] failed to attach provider
o kern/159595  geom       [geom] [panic] panic on gmirror unload in vbox [regres
p kern/158398  geom       [headers] [patch] <geom/geom.h> includes <sys/sbuf.h> 
o kern/158197  geom       [geom] geom_cache with size>1000 leads to panics
o kern/157879  geom       [libgeom] [regression] ABI change without version bump
o kern/157863  geom       [geli] kbdmux prevents geli passwords from being enter
o kern/157739  geom       [geom] GPT labels with geom_multipath
o kern/157724  geom       [geom] gpart(8) 'add' command must preserve gap for sc
o kern/157723  geom       [geom] GEOM should not process 'c' (raw) partitions fo
o kern/157108  geom       [gjournal] dumpon(8) fails on gjournal providers
o kern/155994  geom       [geom] Long "Suspend time" when reading large files fr
o kern/154226  geom       [geom] GEOM label does not change when you modify them
o kern/150858  geom       [geom] [geom_label] [patch] glabel(8) is not compatibl
o kern/150626  geom       [geom] [gjournal] gjournal(8) destroys label
o kern/150555  geom       [geom] gjournal unusable on GPT partitions
(Continue reading)

Permalink | Reply | 0 comments |
headers
Anthony Chavez | 21 Nov 2011 09:54

gvinum + geli + gjournal

The subject describes the configuration that I would like.

# uname -a
FreeBSD metis.hexadecagram.org 8.2-STABLE FreeBSD 8.2-STABLE #0: Thu Nov 
10 16:1
3:27 MST 2011     root <at> metis.hexadecagram.org:/usr/obj/usr/src/sys/METIS 
  i386

All 3 drives are 4096-sector Seagate FreeAgent GoFlex Desk 3TB USB 2.0 
drives.  I partition them as follows:

# for d in /dev/da[345]; do
# 	gpart create -s gpt /dev/da3
# 	gpart add -t freebsd-vinum $d
# done

I then glabel them (which would be nice to have as they are USB drives):

# glabel extraid0 da3p1
# glabel extraid1 da4p1
# glabel extraid2 da5p1

Create the gvinum object:

# gvinum create external.gvinum

external.gvinum contains:

drive a device /dev/label/extraid0
drive b device /dev/label/extraid1
(Continue reading)

Permalink | Reply | 6 comments |

Gmane