headers
Oliver Francke | 24 May 2013 09:46
Picon
Favicon

qemu-1.5.0 savevm error -95 while writing vm with ceph-rbd as storage-backend

Hi,

with a running VM I encounter this strange behaviour, former 
qemu-versions don't show up such an error.
Perhaps this comes from the rbd-backend in qemu-1.5.0 in combination 
with ceph-0.56.6? Therefore my
crosspost.

Even if I have no real live-snapshot avail - they know of this 
restriction -, it's more work for the customers
to perform a shutdown before the wonna do some changes to their VM ;)

Any hints welcome,

Oliver.

--

-- 

Oliver Francke

filoo GmbH
Moltkestraße 25a
33330 Gütersloh
HRB4355 AG Gütersloh

Geschäftsführer: S.Grewing | J.Rehpöhler | C.Kunz

Folgen Sie uns auf Twitter: http://twitter.com/filoogmbh

--
(Continue reading)

Permalink | Reply | 3 comments |
headers
Sage Weil | 24 May 2013 04:42

libcephfs/Client api changes

Hi Matt!

I took a closer look at the libcephfs branch you guys have.  A copy is in 
wip-libcephfs in the main ceph.git, and I opened a pull request on github 
just to capture review comments.  Please take a look at 

	https://github.com/ceph/ceph/pull/318

Lots of small questions and issues, but overall the appraoch looks great.  
I also rebased/squashed teh changes into a separate set of patches that 
try to capture the logical changes (instead of the patchset evolution).. 
these are in wip-libcephfs-rebased.

Can you take a look?

I think the ideal path forward (at least from my perspective :) would be:

- diff the rebased branch with yours to make sure the things i dropped 
  aren't important.
- make sure teh 'reject' patches are unimportant, and drop them from the 
  series.  the objecter change would fall into this category..
- address the review comments, modifying the rebased patches as 
  appropriate
- fix up the patch changelogs to have proper authorship and (where 
  appropriate) any background/justification for the change.  (in several 
  cases I was guessing what the change was about)
- rebase onto current master branch
- submit a new github pull request
- profit!
(Continue reading)

Permalink | Reply | 0 comments |
headers
Milosz Tanski | 23 May 2013 23:55
Favicon

[PATCH 2/2] Enable fscache as an optional feature of ceph.

Enable fscache as an optional feature of ceph.

Adding support for fscache to the Ceph filesystem. This would bring it to on
par with some of the other network filesystems in Linux (like NFS, AFS, etc...)

This exploits the existing Ceph cache & lazyio capabilities.

Signed-off-by: Milosz Tanski <milosz <at> adfin.com>
---
 fs/ceph/Kconfig  |    9 ++++++
 fs/ceph/Makefile |    2 ++
 fs/ceph/addr.c   |   85 ++++++++++++++++++++++++++++++++++++++++--------------
 fs/ceph/caps.c   |   21 +++++++++++++-
 fs/ceph/file.c   |    9 ++++++
 fs/ceph/inode.c  |   25 ++++++++++++++--
 fs/ceph/super.c  |   25 ++++++++++++++--
 fs/ceph/super.h  |   12 ++++++++
 8 files changed, 162 insertions(+), 26 deletions(-)

diff --git a/fs/ceph/Kconfig b/fs/ceph/Kconfig
index 49bc782..ac9a2ef 100644
--- a/fs/ceph/Kconfig
+++ b/fs/ceph/Kconfig
 <at>  <at>  -16,3 +16,12  <at>  <at>  config CEPH_FS

   If unsure, say N.

+if CEPH_FS
+config CEPH_FSCACHE
+ bool "Enable Ceph client caching support"
(Continue reading)

Permalink | Reply | 0 comments |
headers
Milosz Tanski | 23 May 2013 23:54
Favicon

[PATCH 1/2] Fscache glue implementation for Ceph

Fscache glue implementation for Ceph

The facilities needed for fscache support support. This part of the work is
self contained in it's files and just getting Ceph ready.

Signed-off-by: Milosz Tanski <milosz <at> adfin.com>
---
 fs/ceph/cache.c |  334 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/ceph/cache.h |  115 +++++++++++++++++++
 2 files changed, 449 insertions(+)
 create mode 100644 fs/ceph/cache.c
 create mode 100644 fs/ceph/cache.h

diff --git a/fs/ceph/cache.c b/fs/ceph/cache.c
new file mode 100644
index 0000000..f320d0e
--- /dev/null
+++ b/fs/ceph/cache.c
 <at>  <at>  -0,0 +1,334  <at>  <at> 
+/*
+ * Ceph cache definitions.
+ *
+ *  Copyright (C) 2013 by Adfin Solutions, Inc. All Rights Reserved.
+ *  Written by Milosz Tanski (milosz <at> adfin.com)
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License version 2
+ *  as published by the Free Software Foundation.
+ *
+ *  This program is distributed in the hope that it will be useful,
(Continue reading)

Permalink | Reply | 0 comments |
headers
Milosz Tanski | 23 May 2013 23:48
Favicon

Fscache support for Ceph

This is my first at adding fscache support for the Ceph Linux module.

My motivation for doing this work was speedup our distributed database
that uses the Ceph filesystem as a backing store. By far more of the
workload that our application is doing is read only and latency is our
biggest challenge. Being able to cache frequently used blocks on the
SSD drives that our machines use dramatically speeds up our query
setup time when we're fetching multiple compressed indexes and then
navigating the block tree.

The branch containing the two patches is here:
https://bitbucket.org/adfin/linux-fs.git in the forceph branch.

If you want to review it in your browser here is the bitbucket url:
https://bitbucket.org/adfin/linux-fs/commits/branch/forceph

I've tested this both in mainline and in the branch that features
upcoming fscache changes. The patches are broken into two pieces.

01 - Setups the facility for fscache in it's independent files
02 - Enables fscache in the ceph filesystem and adds a new configuration option

The patches will follow in the new few emails as well.

Future wise; there's some new work being done to add write-back
caching to fscache & NFS. When that's done I'd like to integrated the
Ceph fscache implementation. From the benchmarks of the author of that
it seams like it has much the same benefit for write to NFS as bcache
does.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Dan Carpenter | 23 May 2013 20:36
Picon
Favicon

[patch] ceph: remove unneeded truncation

My static checker complains l->stripe_unit could still be zero after we
truncate it to 32 bits.  I don't see a reason to do the truncation so
I have removed it.  Both sides are u64 type.

Signed-off-by: Dan Carpenter <dan.carpenter <at> oracle.com>

diff --git a/fs/ceph/ioctl.c b/fs/ceph/ioctl.c
index e0b4ef3..98fe5e7 100644
--- a/fs/ceph/ioctl.c
+++ b/fs/ceph/ioctl.c
 <at>  <at>  -43,7 +43,7  <at>  <at>  static long __validate_layout(struct ceph_mds_client *mdsc,
 	if ((l->object_size & ~PAGE_MASK) ||
 	    (l->stripe_unit & ~PAGE_MASK) ||
 	    (l->stripe_unit != 0 &&
-	     ((unsigned)l->object_size % (unsigned)l->stripe_unit)))
+	     (l->object_size % l->stripe_unit)))
 		return -EINVAL;

 	/* make sure it's a valid data pool */
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 0 comments |
headers
Isaac Otsiabah | 23 May 2013 20:05
Picon
Favicon

how to pass mount options in ceph-deploy osd create or prepare commands


I see ceph-deploy mounts the ceph data drives as

/dev/sdc1 on /var/lib/ceph/osd/ceph-2 type xfs (rw,noatime)
/dev/sdd1 on /var/lib/ceph/osd/ceph-3 type xfs (rw,noatime)

how do i add a mount option (like -o inode64, ...etc ) to ceph-deploy in the

ceph-deploy osd create

or 

ceph-deploy osd prepare
ceph-deploy osd activate

commands  to use when it is mounting the ceph data drives. Thank you.

Isaac
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 0 comments |
headers
Jäger, Philipp | 23 May 2013 19:14

Ceph newbie questions

Hello, 

Problem1:

Ceph health says HEALTH_WARN clock skew detected on mon.b, mon.c

But:

root <at> hcmonko1:~# date && ssh hcmonko3 date && ssh hcmonko2 date
Thu May 23 18:19:38 CEST 2013
Thu May 23 18:19:38 CEST 2013
Thu May 23 18:19:38 CEST 2013

Seems to be ok, ntp configured. Restarted all the servers, still the same.

Problem 2:

Service ceph -a works seriell, not parallel.
Situation: every osd and mon is offline via service ceph -a stop. When I want to bring them online with
"service ceph -a start", it begins with mon a, then mon b, then osd0, osd1 etc.
But the procedure hangs at the point ceph-create keys for the first mon in the row.

root <at> hcosdko1:~# service ceph -a start
=== mon.a ===
Starting Ceph mon.a on hcmonko1...
Starting ceph-create-keys on hcmonko1...

I get to know via your irc-channel that ceph-create-keys needs the mon quorum to proceed. I think the ceph -a
start script is designed to not make it serial, rather it has to do it parallel for all mons+osd.
But in my scenario it hangs at this point: ceph is started on mon1, hangs at the keys because there is no quorum
(Continue reading)

Permalink | Reply | 0 comments |
headers
Isaac Otsiabah | 23 May 2013 19:22
Picon
Favicon

using ceph-deploy, how do i specify the cluster address?


Hello, before ceph-deply arrived, i used mkcephfs and specified the cluster address in ceph.conf as:

[osd.0]
        host = gluster3
        public addr = 192.168.0.102
        cluster addr = 192.168.1.102   <------------ My cluster address
        btrfs devs = /dev/sdc
        osd journal = /dev/sdb5

Now with ceph-deploy, how do i specify the cluster address since i do not want osd to osd communication to go
through the monitor address? Thank you very much.

Isaac
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 0 comments |
headers
Isaac Otsiabah | 23 May 2013 19:13
Picon
Favicon

ceph-deploy: how to change filesystem type from xfs to btrfs using --fs-type option


Hello, the link at http://ceph.com/docs/master/rados/deployment/ceph-deploy-osd/ say that you can use

--fs-type

option to change the default filesystem from xfs to either btrfs or ext4 but when i tried the following
syntax, they all failed:

1)

[root <at> g1 wkd]# ceph-deploy osd prepare --fs-type btrfs gluster3:sdd:/dev/sdb1
usage: ceph-deploy [-h] [-v | -q] [-n] [--overwrite-conf] [--cluster NAME]
                   COMMAND ...
ceph-deploy: error: unrecognized arguments: --fs-type

2)
[root <at> g1 wkd]# ceph-deploy --fs-type btrfs osd prepare  gluster3:sdd:/dev/sdb1
usage: ceph-deploy [-h] [-v | -q] [-n] [--overwrite-conf] [--cluster NAME]
                   COMMAND ...
ceph-deploy:
 error: argument COMMAND: invalid choice: 'btrfs' (choose from 
'purgedata', 'mds', 'forgetkeys', 'purge', 'admin', 'mon', 'install', 
'gatherkeys', 'new', 'disk', 'config', 'osd', 'uninstall')

3) 
[root <at> g1 wkd]# ceph-deploy osd create --fs-type btrfs gluster3:sdd:/dev/sdb1
usage: ceph-deploy [-h] [-v | -q] [-n] [--overwrite-conf] [--cluster NAME]
                   COMMAND ...
ceph-deploy: error: unrecognized arguments: --fs-type
(Continue reading)

Permalink | Reply | 0 comments |
headers
Alex Elder | 23 May 2013 14:11

[PATCH] rbd: flush dcache after zeroing page data

Neither zero_bio_chain() nor zero_pages() contains a call to flush
caches after zeroing a portion of a page.  This can cause problems
on architectures that have caches that allow virtual address
aliasing.

This resolves:
    http://tracker.ceph.com/issues/4777

Signed-off-by: Alex Elder <elder <at> inktank.com>
---
 drivers/block/rbd.c |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index 6e377a0..f860dd6 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
 <at>  <at>  -1124,6 +1124,7  <at>  <at>  static void zero_bio_chain(struct bio *chain, int
start_ofs)
 				buf = bvec_kmap_irq(bv, &flags);
 				memset(buf + remainder, 0,
 				       bv->bv_len - remainder);
+				flush_dcache_page(bv->bv_page);
 				bvec_kunmap_irq(buf, &flags);
 			}
 			pos += bv->bv_len;
 <at>  <at>  -1156,6 +1157,7  <at>  <at>  static void zero_pages(struct page **pages, u64
offset, u64 end)
 		local_irq_save(flags);
 		kaddr = kmap_atomic(*page);
(Continue reading)

Permalink | Reply | 2 comments |

Gmane