Pino Tsao | 27 May 10:29 2015

[PATCH RFC] New API: btrfs_convert

   Signed-off-by: Pino <caoj.fnst <at>>
 daemon/btrfs.c       | 29 +++++++++++++++++++++++++++++
 generator/ | 18 ++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/daemon/btrfs.c b/daemon/btrfs.c
index 39392f7..7f10792 100644
--- a/daemon/btrfs.c
+++ b/daemon/btrfs.c
 <at>  <at>  -38,6 +38,7  <at>  <at>  GUESTFSD_EXT_CMD(str_btrfsck, btrfsck);
 GUESTFSD_EXT_CMD(str_mkfs_btrfs, mkfs.btrfs);
 GUESTFSD_EXT_CMD(str_umount, umount);
 GUESTFSD_EXT_CMD(str_btrfsimage, btrfs-image);
+GUESTFSD_EXT_CMD(str_btrfsconvert, btrfs-convert);

 optgroup_btrfs_available (void)
 <at>  <at>  -2083,3 +2084,31  <at>  <at>  do_btrfs_image (char *const *sources, const char *image,

   return 0;
+do_btrfs_convert (const char *device, int rollback)
+  const size_t MAX_ARGS = 64;
+  const char *argv[MAX_ARGS];
+  size_t i = 0;
+  CLEANUP_FREE char *err = NULL;
(Continue reading)

NoxDaFox | 27 May 08:38 2015

Concurrent scanning of same disk


I am suffering of several weird errors which show randomly and make me suspect some concurrency issue.

Libguestfs version is 1.28.1, linux kernel 3.16, libvirt 1.2.9 and qemu 2.1.

What I'm trying to do is comparing the disk state at two different point of a guest execution.
Disk snapshots are taken through libvirt in different moments (I am aware of caching issue), from such snapshots, new disks are created using the "qemu-img convert" command with backing file pointing to the disk being executed.

Then, I spawn as many processes as disk images and in each process I create an guestfs instance and mount one of the disks.

The operation seems successful as I'm gathering the data I am looking for but I suffer from these random failures:

 * RuntimeError: file receive cancelled by daemon - On r = libguestfsmod.checksums_out (self._o, csumtype, directory, sumsfile)
 * RuntimeError: hivex_close: do_hivex_close: you must call 'hivex-open' first to initialize the hivex handle - On r = libguestfsmod.inspect_os (self._o)

The random nature of the errors make me think about some concurrency problem. The disk images are mounted read only, each process uses a new guestfs instance and mount a different disk image. Only shared data is the backing file.

Same code without backing image was running on a different server. I was able to compare whole disks image created from snapshots of a running one.

Most probably I am doing something wrong but I can't figure out what.
Libguestfs mailing list
Libguestfs <at>
Richard W.M. Jones | 26 May 14:34 2015

[PATCH] lib: Limit space and time used by 'qemu-img info' subprocess.

After fuzzing 'qemu-img info' I found that certain files can cause the
command to use lots of memory and time.  Modify the command
mini-library to allow us to place resource limits on subprocesses, and
use these to limit the amount of space and time used by 'qemu-img info'.
---           |  3 +++
 src/command.c          | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++
 src/guestfs-internal.h |  1 +
 src/info.c             | 22 ++++++++++++++++++++-
 4 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/ b/
index acd807b..6fea1e4 100644
--- a/
+++ b/
 <at>  <at>  -305,8 +305,10  <at>  <at>  AC_CHECK_HEADERS([\
     linux/raid/md_u.h \
     printf.h \
     sys/inotify.h \
+    sys/resource.h \
     sys/socket.h \
     sys/statvfs.h \
+    sys/time.h \
     sys/types.h \
     sys/un.h \
     sys/wait.h \
 <at>  <at>  -335,6 +337,7  <at>  <at>  AC_CHECK_FUNCS([\
     posix_fadvise \
     removexattr \
     setitimer \
+    setrlimit \
     setxattr \
     sigaction \
     statvfs \
diff --git a/src/command.c b/src/command.c
index 45ae5d6..993198a 100644
--- a/src/command.c
+++ b/src/command.c
 <at>  <at>  -77,6 +77,13  <at>  <at> 
 #include <sys/wait.h>
 #include <sys/select.h>

+#include <sys/time.h>
+#include <sys/resource.h>
 #include "guestfs.h"
 #include "guestfs-internal.h"

 <at>  <at>  -101,6 +108,12  <at>  <at>  struct buffering {
   void (*close_data) (struct command *cmd);

+struct child_rlimits {
+  struct child_rlimits *next;
+  int resource;
+  long limit;
 struct command
   guestfs_h *g;
 <at>  <at>  -139,6 +152,9  <at>  <at>  struct command
   cmd_child_callback child_callback;
   void *child_callback_data;

+  /* Optional child limits. */
+  struct child_rlimits *child_rlimits;
   /* Optional stdin forwarding to the child. */
   int infd;
 <at>  <at>  -329,6 +345,22  <at>  <at>  guestfs_int_cmd_set_child_callback (struct command *cmd,
   cmd->child_callback_data = data;

+/* Set up child rlimits, in case the process we are running could
+ * consume lots of space or time.
+ */
+guestfs_int_cmd_set_child_rlimit (struct command *cmd, int resource, long limit)
+  struct child_rlimits *p;
+  p = safe_malloc (cmd->g, sizeof *p);
+  p->resource = resource;
+  p->limit = limit;
+  p->next = cmd->child_rlimits;
+  cmd->child_rlimits = p;
 /* Finish off the command by either NULL-terminating the argv array or
  * adding a terminating \0 to the string, or die with an internal
  * error if no command has been added.
 <at>  <at>  -390,6 +422,10  <at>  <at>  run_command (struct command *cmd, bool get_stdin_fd, bool get_stdout_fd,
   int outfd[2] = { -1, -1 };
   int infd[2] = { -1, -1 };
   char status_string[80];
+  struct child_rlimits *child_rlimit;
+  struct rlimit rlimit;

   get_stdout_fd = get_stdout_fd || cmd->stdout_callback != NULL;
   get_stderr_fd = get_stderr_fd || cmd->capture_errors;
 <at>  <at>  -510,6 +546,23  <at>  <at>  run_command (struct command *cmd, bool get_stdin_fd, bool get_stdout_fd,
       _exit (EXIT_FAILURE);

+  for (child_rlimit = cmd->child_rlimits;
+       child_rlimit != NULL;
+       child_rlimit = child_rlimit->next) {
+    rlimit.rlim_cur = rlimit.rlim_max = child_rlimit->limit;
+    if (setrlimit (child_rlimit->resource, &rlimit) == -1) {
+      /* EPERM means we're trying to raise the limit (ie. the limit is
+       * already more restrictive than what we want), so ignore it.
+       */
+      if (errno != EPERM) {
+        perror ("setrlimit");
+        _exit (EXIT_FAILURE);
+      }
+    }
+  }
+#endif /* HAVE_SETRLIMIT */
   /* Run the command. */
   switch (cmd->style) {
diff --git a/src/guestfs-internal.h b/src/guestfs-internal.h
index 414a634..4f06c37 100644
--- a/src/guestfs-internal.h
+++ b/src/guestfs-internal.h
 <at>  <at>  -864,6 +864,7  <at>  <at>  extern void guestfs_int_cmd_set_stdout_callback (struct command *, cmd_stdout_ca
 extern void guestfs_int_cmd_set_stderr_to_stdout (struct command *);
+extern void guestfs_int_cmd_set_child_rlimit (struct command *, int resource, long limit);
 extern void guestfs_int_cmd_clear_capture_errors (struct command *);
 extern void guestfs_int_cmd_clear_close_files (struct command *);
 extern void guestfs_int_cmd_set_child_callback (struct command *, cmd_child_callback
child_callback, void *data);
diff --git a/src/info.c b/src/info.c
index bd4221c..bfd7860 100644
--- a/src/info.c
+++ b/src/info.c
 <at>  <at>  -31,6 +31,14  <at>  <at> 
 #include <assert.h>
 #include <string.h>

+#include <sys/time.h>
+#include <sys/resource.h>
 #include <yajl/yajl_tree.h>
 <at>  <at>  -269,7 +277,13  <at>  <at>  get_json_output (guestfs_h *g, const char *filename)
   guestfs_int_cmd_add_arg (cmd, "json");
   guestfs_int_cmd_add_arg (cmd, fdpath);
   guestfs_int_cmd_set_stdout_callback (cmd, parse_json, &tree,
-                                     CMD_STDOUT_FLAG_WHOLE_BUFFER);
+                                       CMD_STDOUT_FLAG_WHOLE_BUFFER);
+#ifdef RLIMIT_AS
+  guestfs_int_cmd_set_child_rlimit (cmd, RLIMIT_AS, 1000000000 /* 1GB */);
+#ifdef RLIMIT_CPU
+  guestfs_int_cmd_set_child_rlimit (cmd, RLIMIT_CPU, 10 /* seconds */);
   r = guestfs_int_cmd_run (cmd);
   close (fd);
   if (r == -1)
 <at>  <at>  -548,6 +562,12  <at>  <at>  old_parser_run_qemu_img_info (guestfs_h *g, const char *filename,
   guestfs_int_cmd_add_arg (cmd, "info");
   guestfs_int_cmd_add_arg (cmd, safe_filename);
   guestfs_int_cmd_set_stdout_callback (cmd, fn, data, 0);
+#ifdef RLIMIT_AS
+  guestfs_int_cmd_set_child_rlimit (cmd, RLIMIT_AS, 1000000000 /* 1GB */);
+#ifdef RLIMIT_CPU
+  guestfs_int_cmd_set_child_rlimit (cmd, RLIMIT_CPU, 10 /* seconds */);
   r = guestfs_int_cmd_run (cmd);
   if (r == -1)
     return -1;

Richard W.M. Jones | 26 May 12:38 2015

[PATCH 0/6] Update the way that API versions are generated for the man page.

The existing mechanism was clunky, slow and used ~ 10 MB of
local disk.

Richard W.M. Jones | 26 May 09:52 2015

[PATCH] api: Don't truncate /dev/stdout or /dev/stderr when used as FileOut.

In APIs such as guestfs_download, when the FileOut parameter exactly
matches "/dev/stdout" or "/dev/stderr", don't reopen the possibly
redirected output file with O_TRUNC (truncate).  Instead dup the file

This magic behaviour doesn't happen for /dev/fd/* (or any other output
file) allowing callers the choice of using /dev/stderr or /dev/fd/2
depending on whether or not they want truncation.

This works around an annoying virt-builder bug.  If you do:

  $ virt-builder fedora-21 --install no_such_package -v -x >& /tmp/log

then when the `--install' command fails, virt-builder will download
the log file using `guestfs_download (g, log, "/dev/stderr")'.  Since
this truncates the redirected /dev/stderr, the final log file is
truncated and corrupted.

With this patch the log file is no longer corrupted.
 customize/ |  7 +------
 src/proto.c                | 13 +++++++++++--
 2 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/customize/ b/customize/
index 5a7e209..d9547a0 100644
--- a/customize/
+++ b/customize/
 <at>  <at>  -45,12 +45,7  <at>  <at>  let run (g : Guestfs.guestfs) root (ops : ops) =

   (* Function to cat the log file, for debugging and error messages. *)
   let debug_logfile () =
-    try
-      (* XXX If stderr is redirected this actually truncates the
-       * redirection file, which is pretty annoying to say the
-       * least.
-       *)
-      g#download logfile "/dev/stderr"
+    try g#download logfile "/dev/stderr"
     with exn ->
       warning (f_"log file %s: %s (ignored)") logfile (Printexc.to_string exn) in

diff --git a/src/proto.c b/src/proto.c
index a019625..a46a382 100644
--- a/src/proto.c
+++ b/src/proto.c
 <at>  <at>  -756,9 +756,18  <at>  <at>  guestfs_int_recv_file (guestfs_h *g, const char *filename)

   g->user_cancel = 0;

-  fd = open (filename, O_WRONLY|O_CREAT|O_TRUNC|O_NOCTTY|O_CLOEXEC, 0666);
+  /* If downloading to /dev/stdout or /dev/stderr, dup the file
+   * descriptor instead of reopening the file, so that redirected
+   * stdout/stderr work properly.
+   */
+  if (STREQ (filename, "/dev/stdout"))
+    fd = dup (1);
+  else if (STREQ (filename, "/dev/stderr"))
+    fd = dup (2);
+  else
+    fd = open (filename, O_WRONLY|O_CREAT|O_TRUNC|O_NOCTTY|O_CLOEXEC, 0666);
   if (fd == -1) {
-    perrorf (g, "open: %s", filename);
+    perrorf (g, "%s", filename);
     goto cancel;


Rakesh Agrawal | 25 May 10:03 2015

Need Help

Hi All,


Can we use libguestfs to somehow uninstall softwares from VMware images?

The VMware image is of windows 2012 server.




DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

Libguestfs mailing list
Libguestfs <at>
Maros Zatko | 22 May 17:10 2015

[PATCH v5] inspector: recognize ppc64 and ppc64le archs (RHBZ#1211996)

Patch extracts MSB/LSB info from /bin/file output and passes it as separate
parameret from elf_arch. Then it is sent to impl_file_architecture and checked/

Modified magic_for_file to handle regex changes.

Now with tests for file_architecture, and error reporting for unknown

Fixes: RHBZ#1211996

Maros Zatko (1):
  inspector: recognize ppc64 and ppc64le archs (RHBZ#1211996)

 generator/ |  8 ++++++++
 src/filearch.c       | 28 +++++++++++++++++++---------
 2 files changed, 27 insertions(+), 9 deletions(-)


Richard W.M. Jones | 21 May 15:38 2015

On holiday until Tuesday 26th May

Subject says it all ...


Richard Jones, Virtualization Group, Red Hat
Read my programming and virtualization blog:
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
Richard W.M. Jones | 21 May 15:37 2015

supermin in Fedora Rawhide switched (again) to using dnf instead of yum

Previous attempt:

supermin needs to download packages (eg. RPMs) when preparing the

After a lot of work, 'dnf download' can now be used as a replacement
for the (deprecated) 'yumdownloader' program.

This only affects the 'supermin --prepare' phase, which means it only
affects people building libguestfs from source.  Ordinary use of
libguestfs, even in Rawhide, should be unaffected.

If you are building libguestfs from source on Fedora please try the
new supermin + dnf, and let me know if you find any problems.



Richard Jones, Virtualization Group, Red Hat
Read my programming and virtualization blog:
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
Bill Speirs | 21 May 06:10 2015

[nbdkit python]: unable to locate modules

I'm working on a python plugin for nbdkit that treats Google Cloud
Storage as a block device. I have a library,, that implements
simple commands: list, get, put, delete; source is here:

When I attempt to run nbdkit I get the following error:

nbdkit python script=/home/wspeirs/src/cldblkdev/
Traceback (most recent call last):
 File "/home/wspeirs/src/cldblkdev/", line 5, in <module>
   import gcs
 File "/home/wspeirs/src/cldblkdev/", line 2, in <module>
   import httplib2
 File "/usr/local/lib/python2.7/dist-packages/httplib2/",
line 930, in <module>
   class HTTPSConnectionWithTimeout(httplib.HTTPSConnection):
AttributeError: 'module' object has no attribute 'HTTPSConnection'
Error in sys.excepthook:
Traceback (most recent call last):
 File "/usr/lib/python2.7/dist-packages/", line
63, in apport_excepthook
   from apport.fileutils import likely_packaged, get_recent_crashes
 File "/usr/lib/python2.7/dist-packages/apport/", line 5, in <module>
   from import Report
 File "/usr/lib/python2.7/dist-packages/apport/", line 16, in <module>
   from xml.parsers.expat import ExpatError
 File "/usr/lib/python2.7/xml/parsers/", line 4, in <module>
   from pyexpat import *
ImportError: /usr/lib/python2.7/lib-dynload/
undefined symbol: _Py_ZeroStruct

Original exception was:
Traceback (most recent call last):
 File "/home/wspeirs/src/cldblkdev/", line 5, in <module>
   import gcs
 File "/home/wspeirs/src/cldblkdev/", line 2, in <module>
   import httplib2
 File "/usr/local/lib/python2.7/dist-packages/httplib2/",
line 930, in <module>
   class HTTPSConnectionWithTimeout(httplib.HTTPSConnection):
AttributeError: 'module' object has no attribute 'HTTPSConnection'
nbdkit: error: /home/wspeirs/src/cldblkdev/ error running
this script

Everything seems to work outside of nbdkit. I'm assuming this is some
path related issue... any thoughts on how to fix this?


Maros Zatko | 20 May 19:41 2015

[PATCH v3 0/3] RFE: support Windows drive letter in virt-ls

Fixes RHBZ#845234.

v3 changes: Drive letters works if inspection is enabled (-m is not given)
v2 changes: Ammended so it doesn't do inspection for every dir to list.

Maros Zatko (3):
  virt-ls: support drive letters on Windows
  virt-ls: update usage for win drive letters
  docs: amend virt-ls manpage with win drive letters

 cat/ls.c        | 41 +++++++++++++++++++++++++++++++++++++----
 cat/virt-ls.pod |  7 ++++++-
 2 files changed, 43 insertions(+), 5 deletions(-)