Pino Tsao | 27 May 10:29 2015

[PATCH RFC] New API: btrfs_convert

   Signed-off-by: Pino <caoj.fnst <at> cn.fujitsu.com>
---
 daemon/btrfs.c       | 29 +++++++++++++++++++++++++++++
 generator/actions.ml | 18 ++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/daemon/btrfs.c b/daemon/btrfs.c
index 39392f7..7f10792 100644
--- a/daemon/btrfs.c
+++ b/daemon/btrfs.c
 <at>  <at>  -38,6 +38,7  <at>  <at>  GUESTFSD_EXT_CMD(str_btrfsck, btrfsck);
 GUESTFSD_EXT_CMD(str_mkfs_btrfs, mkfs.btrfs);
 GUESTFSD_EXT_CMD(str_umount, umount);
 GUESTFSD_EXT_CMD(str_btrfsimage, btrfs-image);
+GUESTFSD_EXT_CMD(str_btrfsconvert, btrfs-convert);

 int
 optgroup_btrfs_available (void)
 <at>  <at>  -2083,3 +2084,31  <at>  <at>  do_btrfs_image (char *const *sources, const char *image,

   return 0;
 }
+
+int
+do_btrfs_convert (const char *device, int rollback)
+{
+  const size_t MAX_ARGS = 64;
+  const char *argv[MAX_ARGS];
+  size_t i = 0;
+  CLEANUP_FREE char *err = NULL;
(Continue reading)

NoxDaFox | 27 May 08:38 2015
Picon

Concurrent scanning of same disk

Greetings,

I am suffering of several weird errors which show randomly and make me suspect some concurrency issue.

Libguestfs version is 1.28.1, linux kernel 3.16, libvirt 1.2.9 and qemu 2.1.

What I'm trying to do is comparing the disk state at two different point of a guest execution.
Disk snapshots are taken through libvirt in different moments (I am aware of caching issue), from such snapshots, new disks are created using the "qemu-img convert" command with backing file pointing to the disk being executed.

Then, I spawn as many processes as disk images and in each process I create an guestfs instance and mount one of the disks.

The operation seems successful as I'm gathering the data I am looking for but I suffer from these random failures:

 * RuntimeError: file receive cancelled by daemon - On r = libguestfsmod.checksums_out (self._o, csumtype, directory, sumsfile)
 * RuntimeError: hivex_close: do_hivex_close: you must call 'hivex-open' first to initialize the hivex handle - On r = libguestfsmod.inspect_os (self._o)

The random nature of the errors make me think about some concurrency problem. The disk images are mounted read only, each process uses a new guestfs instance and mount a different disk image. Only shared data is the backing file.

Same code without backing image was running on a different server. I was able to compare whole disks image created from snapshots of a running one.

Most probably I am doing something wrong but I can't figure out what.
_______________________________________________
Libguestfs mailing list
Libguestfs <at> redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
Richard W.M. Jones | 26 May 14:34 2015

[PATCH] lib: Limit space and time used by 'qemu-img info' subprocess.

After fuzzing 'qemu-img info' I found that certain files can cause the
command to use lots of memory and time.  Modify the command
mini-library to allow us to place resource limits on subprocesses, and
use these to limit the amount of space and time used by 'qemu-img info'.
---
 configure.ac           |  3 +++
 src/command.c          | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++
 src/guestfs-internal.h |  1 +
 src/info.c             | 22 ++++++++++++++++++++-
 4 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index acd807b..6fea1e4 100644
--- a/configure.ac
+++ b/configure.ac
 <at>  <at>  -305,8 +305,10  <at>  <at>  AC_CHECK_HEADERS([\
     linux/raid/md_u.h \
     printf.h \
     sys/inotify.h \
+    sys/resource.h \
     sys/socket.h \
     sys/statvfs.h \
+    sys/time.h \
     sys/types.h \
     sys/un.h \
     sys/wait.h \
 <at>  <at>  -335,6 +337,7  <at>  <at>  AC_CHECK_FUNCS([\
     posix_fadvise \
     removexattr \
     setitimer \
+    setrlimit \
     setxattr \
     sigaction \
     statvfs \
diff --git a/src/command.c b/src/command.c
index 45ae5d6..993198a 100644
--- a/src/command.c
+++ b/src/command.c
 <at>  <at>  -77,6 +77,13  <at>  <at> 
 #include <sys/wait.h>
 #include <sys/select.h>

+#ifdef HAVE_SYS_TIME_H
+#include <sys/time.h>
+#endif
+#ifdef HAVE_SYS_RESOURCE_H
+#include <sys/resource.h>
+#endif
+
 #include "guestfs.h"
 #include "guestfs-internal.h"

 <at>  <at>  -101,6 +108,12  <at>  <at>  struct buffering {
   void (*close_data) (struct command *cmd);
 };

+struct child_rlimits {
+  struct child_rlimits *next;
+  int resource;
+  long limit;
+};
+
 struct command
 {
   guestfs_h *g;
 <at>  <at>  -139,6 +152,9  <at>  <at>  struct command
   cmd_child_callback child_callback;
   void *child_callback_data;

+  /* Optional child limits. */
+  struct child_rlimits *child_rlimits;
+
   /* Optional stdin forwarding to the child. */
   int infd;
 };
 <at>  <at>  -329,6 +345,22  <at>  <at>  guestfs_int_cmd_set_child_callback (struct command *cmd,
   cmd->child_callback_data = data;
 }

+/* Set up child rlimits, in case the process we are running could
+ * consume lots of space or time.
+ */
+void
+guestfs_int_cmd_set_child_rlimit (struct command *cmd, int resource, long limit)
+{
+  struct child_rlimits *p;
+
+  p = safe_malloc (cmd->g, sizeof *p);
+  p->resource = resource;
+  p->limit = limit;
+  p->next = cmd->child_rlimits;
+  cmd->child_rlimits = p;
+}
+
+
 /* Finish off the command by either NULL-terminating the argv array or
  * adding a terminating \0 to the string, or die with an internal
  * error if no command has been added.
 <at>  <at>  -390,6 +422,10  <at>  <at>  run_command (struct command *cmd, bool get_stdin_fd, bool get_stdout_fd,
   int outfd[2] = { -1, -1 };
   int infd[2] = { -1, -1 };
   char status_string[80];
+#ifdef HAVE_SETRLIMIT
+  struct child_rlimits *child_rlimit;
+  struct rlimit rlimit;
+#endif

   get_stdout_fd = get_stdout_fd || cmd->stdout_callback != NULL;
   get_stderr_fd = get_stderr_fd || cmd->capture_errors;
 <at>  <at>  -510,6 +546,23  <at>  <at>  run_command (struct command *cmd, bool get_stdin_fd, bool get_stdout_fd,
       _exit (EXIT_FAILURE);
   }

+#ifdef HAVE_SETRLIMIT
+  for (child_rlimit = cmd->child_rlimits;
+       child_rlimit != NULL;
+       child_rlimit = child_rlimit->next) {
+    rlimit.rlim_cur = rlimit.rlim_max = child_rlimit->limit;
+    if (setrlimit (child_rlimit->resource, &rlimit) == -1) {
+      /* EPERM means we're trying to raise the limit (ie. the limit is
+       * already more restrictive than what we want), so ignore it.
+       */
+      if (errno != EPERM) {
+        perror ("setrlimit");
+        _exit (EXIT_FAILURE);
+      }
+    }
+  }
+#endif /* HAVE_SETRLIMIT */
+
   /* Run the command. */
   switch (cmd->style) {
   case COMMAND_STYLE_EXECV:
diff --git a/src/guestfs-internal.h b/src/guestfs-internal.h
index 414a634..4f06c37 100644
--- a/src/guestfs-internal.h
+++ b/src/guestfs-internal.h
 <at>  <at>  -864,6 +864,7  <at>  <at>  extern void guestfs_int_cmd_set_stdout_callback (struct command *, cmd_stdout_ca
 #define CMD_STDOUT_FLAG_UNBUFFERED      1
 #define CMD_STDOUT_FLAG_WHOLE_BUFFER    2
 extern void guestfs_int_cmd_set_stderr_to_stdout (struct command *);
+extern void guestfs_int_cmd_set_child_rlimit (struct command *, int resource, long limit);
 extern void guestfs_int_cmd_clear_capture_errors (struct command *);
 extern void guestfs_int_cmd_clear_close_files (struct command *);
 extern void guestfs_int_cmd_set_child_callback (struct command *, cmd_child_callback
child_callback, void *data);
diff --git a/src/info.c b/src/info.c
index bd4221c..bfd7860 100644
--- a/src/info.c
+++ b/src/info.c
 <at>  <at>  -31,6 +31,14  <at>  <at> 
 #include <assert.h>
 #include <string.h>

+#ifdef HAVE_SYS_TIME_H
+#include <sys/time.h>
+#endif
+
+#ifdef HAVE_SYS_RESOURCE_H
+#include <sys/resource.h>
+#endif
+
 #if HAVE_YAJL
 #include <yajl/yajl_tree.h>
 #endif
 <at>  <at>  -269,7 +277,13  <at>  <at>  get_json_output (guestfs_h *g, const char *filename)
   guestfs_int_cmd_add_arg (cmd, "json");
   guestfs_int_cmd_add_arg (cmd, fdpath);
   guestfs_int_cmd_set_stdout_callback (cmd, parse_json, &tree,
-                                     CMD_STDOUT_FLAG_WHOLE_BUFFER);
+                                       CMD_STDOUT_FLAG_WHOLE_BUFFER);
+#ifdef RLIMIT_AS
+  guestfs_int_cmd_set_child_rlimit (cmd, RLIMIT_AS, 1000000000 /* 1GB */);
+#endif
+#ifdef RLIMIT_CPU
+  guestfs_int_cmd_set_child_rlimit (cmd, RLIMIT_CPU, 10 /* seconds */);
+#endif
   r = guestfs_int_cmd_run (cmd);
   close (fd);
   if (r == -1)
 <at>  <at>  -548,6 +562,12  <at>  <at>  old_parser_run_qemu_img_info (guestfs_h *g, const char *filename,
   guestfs_int_cmd_add_arg (cmd, "info");
   guestfs_int_cmd_add_arg (cmd, safe_filename);
   guestfs_int_cmd_set_stdout_callback (cmd, fn, data, 0);
+#ifdef RLIMIT_AS
+  guestfs_int_cmd_set_child_rlimit (cmd, RLIMIT_AS, 1000000000 /* 1GB */);
+#endif
+#ifdef RLIMIT_CPU
+  guestfs_int_cmd_set_child_rlimit (cmd, RLIMIT_CPU, 10 /* seconds */);
+#endif
   r = guestfs_int_cmd_run (cmd);
   if (r == -1)
     return -1;
--

-- 
2.3.1
Richard W.M. Jones | 26 May 12:38 2015
Picon

[PATCH 0/6] Update the way that API versions are generated for the man page.

The existing mechanism was clunky, slow and used ~ 10 MB of
local disk.

Rich.
Richard W.M. Jones | 26 May 09:52 2015
Picon

[PATCH] api: Don't truncate /dev/stdout or /dev/stderr when used as FileOut.

In APIs such as guestfs_download, when the FileOut parameter exactly
matches "/dev/stdout" or "/dev/stderr", don't reopen the possibly
redirected output file with O_TRUNC (truncate).  Instead dup the file
descriptor.

This magic behaviour doesn't happen for /dev/fd/* (or any other output
file) allowing callers the choice of using /dev/stderr or /dev/fd/2
depending on whether or not they want truncation.

This works around an annoying virt-builder bug.  If you do:

  $ virt-builder fedora-21 --install no_such_package -v -x >& /tmp/log

then when the `--install' command fails, virt-builder will download
the log file using `guestfs_download (g, log, "/dev/stderr")'.  Since
this truncates the redirected /dev/stderr, the final log file is
truncated and corrupted.

With this patch the log file is no longer corrupted.
---
 customize/customize_run.ml |  7 +------
 src/proto.c                | 13 +++++++++++--
 2 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/customize/customize_run.ml b/customize/customize_run.ml
index 5a7e209..d9547a0 100644
--- a/customize/customize_run.ml
+++ b/customize/customize_run.ml
 <at>  <at>  -45,12 +45,7  <at>  <at>  let run (g : Guestfs.guestfs) root (ops : ops) =

   (* Function to cat the log file, for debugging and error messages. *)
   let debug_logfile () =
-    try
-      (* XXX If stderr is redirected this actually truncates the
-       * redirection file, which is pretty annoying to say the
-       * least.
-       *)
-      g#download logfile "/dev/stderr"
+    try g#download logfile "/dev/stderr"
     with exn ->
       warning (f_"log file %s: %s (ignored)") logfile (Printexc.to_string exn) in

diff --git a/src/proto.c b/src/proto.c
index a019625..a46a382 100644
--- a/src/proto.c
+++ b/src/proto.c
 <at>  <at>  -756,9 +756,18  <at>  <at>  guestfs_int_recv_file (guestfs_h *g, const char *filename)

   g->user_cancel = 0;

-  fd = open (filename, O_WRONLY|O_CREAT|O_TRUNC|O_NOCTTY|O_CLOEXEC, 0666);
+  /* If downloading to /dev/stdout or /dev/stderr, dup the file
+   * descriptor instead of reopening the file, so that redirected
+   * stdout/stderr work properly.
+   */
+  if (STREQ (filename, "/dev/stdout"))
+    fd = dup (1);
+  else if (STREQ (filename, "/dev/stderr"))
+    fd = dup (2);
+  else
+    fd = open (filename, O_WRONLY|O_CREAT|O_TRUNC|O_NOCTTY|O_CLOEXEC, 0666);
   if (fd == -1) {
-    perrorf (g, "open: %s", filename);
+    perrorf (g, "%s", filename);
     goto cancel;
   }

--

-- 
2.3.1
Rakesh Agrawal | 25 May 10:03 2015

Need Help

Hi All,

 

Can we use libguestfs to somehow uninstall softwares from VMware images?

The VMware image is of windows 2012 server.

 

Thanks

Rakesh

DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

_______________________________________________
Libguestfs mailing list
Libguestfs <at> redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
Maros Zatko | 22 May 17:10 2015
Picon

[PATCH v5] inspector: recognize ppc64 and ppc64le archs (RHBZ#1211996)

Patch extracts MSB/LSB info from /bin/file output and passes it as separate
parameret from elf_arch. Then it is sent to impl_file_architecture and checked/

Modified magic_for_file to handle regex changes.

Now with tests for file_architecture, and error reporting for unknown
endianness.

Fixes: RHBZ#1211996

Maros Zatko (1):
  inspector: recognize ppc64 and ppc64le archs (RHBZ#1211996)

 generator/actions.ml |  8 ++++++++
 src/filearch.c       | 28 +++++++++++++++++++---------
 2 files changed, 27 insertions(+), 9 deletions(-)

--

-- 
1.9.3
Richard W.M. Jones | 21 May 15:38 2015
Picon

On holiday until Tuesday 26th May


Subject says it all ...

--

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
Richard W.M. Jones | 21 May 15:37 2015
Picon

supermin in Fedora Rawhide switched (again) to using dnf instead of yum


Previous attempt:
https://www.redhat.com/archives/libguestfs/2015-April/msg00013.html

supermin needs to download packages (eg. RPMs) when preparing the
appliance.

After a lot of work, 'dnf download' can now be used as a replacement
for the (deprecated) 'yumdownloader' program.

This only affects the 'supermin --prepare' phase, which means it only
affects people building libguestfs from source.  Ordinary use of
libguestfs, even in Rawhide, should be unaffected.

If you are building libguestfs from source on Fedora please try the
new supermin + dnf, and let me know if you find any problems.

Rich.

--

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
Bill Speirs | 21 May 06:10 2015
Picon

[nbdkit python]: unable to locate modules

I'm working on a python plugin for nbdkit that treats Google Cloud
Storage as a block device. I have a library, gcs.py, that implements
simple commands: list, get, put, delete; source is here:
https://github.com/wspeirs/cldblkdev/blob/master/gcs.py

When I attempt to run nbdkit I get the following error:

nbdkit python script=/home/wspeirs/src/cldblkdev/cldblkdev.py
Traceback (most recent call last):
 File "/home/wspeirs/src/cldblkdev/cldblkdev.py", line 5, in <module>
   import gcs
 File "/home/wspeirs/src/cldblkdev/gcs.py", line 2, in <module>
   import httplib2
 File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py",
line 930, in <module>
   class HTTPSConnectionWithTimeout(httplib.HTTPSConnection):
AttributeError: 'module' object has no attribute 'HTTPSConnection'
Error in sys.excepthook:
Traceback (most recent call last):
 File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line
63, in apport_excepthook
   from apport.fileutils import likely_packaged, get_recent_crashes
 File "/usr/lib/python2.7/dist-packages/apport/__init__.py", line 5, in <module>
   from apport.report import Report
 File "/usr/lib/python2.7/dist-packages/apport/report.py", line 16, in <module>
   from xml.parsers.expat import ExpatError
 File "/usr/lib/python2.7/xml/parsers/expat.py", line 4, in <module>
   from pyexpat import *
ImportError: /usr/lib/python2.7/lib-dynload/pyexpat.x86_64-linux-gnu.so:
undefined symbol: _Py_ZeroStruct

Original exception was:
Traceback (most recent call last):
 File "/home/wspeirs/src/cldblkdev/cldblkdev.py", line 5, in <module>
   import gcs
 File "/home/wspeirs/src/cldblkdev/gcs.py", line 2, in <module>
   import httplib2
 File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py",
line 930, in <module>
   class HTTPSConnectionWithTimeout(httplib.HTTPSConnection):
AttributeError: 'module' object has no attribute 'HTTPSConnection'
nbdkit: error: /home/wspeirs/src/cldblkdev/cldblkdev.py: error running
this script

Everything seems to work outside of nbdkit. I'm assuming this is some
path related issue... any thoughts on how to fix this?

Thanks...

Bill-
Maros Zatko | 20 May 19:41 2015
Picon

[PATCH v3 0/3] RFE: support Windows drive letter in virt-ls

Fixes RHBZ#845234.

v3 changes: Drive letters works if inspection is enabled (-m is not given)
v2 changes: Ammended so it doesn't do inspection for every dir to list.

Maros Zatko (3):
  virt-ls: support drive letters on Windows
  virt-ls: update usage for win drive letters
  docs: amend virt-ls manpage with win drive letters

 cat/ls.c        | 41 +++++++++++++++++++++++++++++++++++++----
 cat/virt-ls.pod |  7 ++++++-
 2 files changed, 43 insertions(+), 5 deletions(-)

--

-- 
1.9.3

Gmane