Re: [RFC] makedumpfile, crash: LZO compression support

> > Hello Hatayama-san,
> > 
> > Thank you for your work.
> > 
> >> Performance Comparison:
> >> 
> >>   Sample Data
> >> 
> >>     Ideally, I must have measured the performance for many enough
> >>     vmcores generated from machines that was actually running, but now
> >>     I don't have enough sample vmcores, I couldn't do so. So this
> >>     comparison doesn't answer question on I/O time improvement. This
> >>     is TODO for now.
> > 
> > I'll measure the performance for actual vmcores by makedumpfile.
> > Please wait for a while.

I measured the performance of makedumpfile for some vmcores.
Please see below.

Sample Data

  To simulate a working server, I captured VMCOREs while almost all pages 
  were alloceted and filled with random data. (See attached file "fill_random.c")

  I captured the VMCOREs of 5GB, 7.5GB and 10GB in the same condition.

How to measure

  I measured the total execution time and the size of output file.

Re: [PATCH] Fix junk values when run crash on a .gz file

The whole point of the "vmlinux" name check is to verify from
the compressed header itself whether it was generated from a file
that started with the name-string "vmlinux".  It's not interested 
in whatever the .gz creator named the compressed file.  If you want
to work around that restriction, use "crash -f ...".

Dave

----- Original Message -----
> Signed-off-by: Aruna Balakrishnaiah <aruna <at> linux.vnet.ibm.com>
> 
> ---
>  symbols.c |    4 ++--
>  1 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/symbols.c b/symbols.c
> index 0cd3a01..acd8ad5 100755
> --- a/symbols.c
> +++ b/symbols.c
>  <at>  <at>  -2995,10 +2995,10  <at>  <at>  is_compressed_kernel(char *file, char **tmp)
>  	type = 0;
>  
>  	if ((header[0] == 0x1f) && (header[1] == 0x8b) && (header[2] == 8))
>  	{
> -		if (!STRNEQ((char *)&header[10], "vmlinux") &&
> +		if (!STRNEQ(basename(file), "vmlinux") &&
>  		    !(st->flags & FORCE_DEBUGINFO)) {
>  			error(INFO, "%s: compressed file name does not "
> -			    "start with \"vmlinux\"\n", &header[10]);

[PATCH] Fix junk values when run crash on a .gz file

Signed-off-by: Aruna Balakrishnaiah <aruna <at> linux.vnet.ibm.com>

---
 symbols.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/symbols.c b/symbols.c
index 0cd3a01..acd8ad5 100755
--- a/symbols.c
+++ b/symbols.c
 <at>  <at>  -2995,10 +2995,10  <at>  <at>  is_compressed_kernel(char *file, char **tmp)
 	type = 0;

 	if ((header[0] == 0x1f) && (header[1] == 0x8b) && (header[2] == 8)) {
-		if (!STRNEQ((char *)&header[10], "vmlinux") && 
+		if (!STRNEQ(basename(file), "vmlinux") &&
 		    !(st->flags & FORCE_DEBUGINFO)) {
 			error(INFO, "%s: compressed file name does not "
-			    "start with \"vmlinux\"\n", &header[10]);
+			    "start with \"vmlinux\"\n", file);
 			error(CONT, 
 			    "Use \"-f %s\" on command line to override.\n\n",
 				file);

Re: [PATCH] Fix junk values when run crash on a .gz file

----- Original Message -----
> Hi Dave,
> 
> If a user by mistake issues the below command some junk characters are
> printed on the console. (crash vmlinux.gz or any .gz file) Please advise.
> 
> linux:/boot # crash vmlinux-3.0.10-0.7-default.gz
> 
> crash: ��{�UU����3ܣ <at> bjb
> R�I-& ��
>        
> �c�"O9�Y��X^f<3��86���v1RS�q��\`������#��^����k�s <at> ��}���=�׳d��g_�^{���Z{�����O-
> F�-W|y�7���f9�k���g���: compressed file name does
> not start with "vmlinux"
>        Use "-f vmlinux-3.0.10-0.7-default.gz" on command line to
>        override.
> 
> crash: vmlinux-3.0.10-0.7-default.gz: not a supported file format

Hmmm, I'm not sure why that's happening, although the name string in the
header does get clipped.  The closest example I have on hand is this one,
where the pre-compressed filename was vmlinux-2.6.36-0.16.rc3.git0.fc15.x86_64,
and the .gz filename is vmlinux-2.6.36-0.16.rc3.git0.fc15.x86_64.gz, and 
which works fine.  And if I change its name to "vmlinux.gz", that still
works as expected.  So I don't know how to reproduce what you are seeing.

Anyway, the file command shows the (possibly clipped) string in the header
that is being looked at:

Re: [PATCH] Fix junk values when run crash on a .gz file

----- Original Message -----
> Hi Dave,
> 
> If a user by mistake issues the below command some junk characters are
> printed on the console. (crash vmlinux.gz or any .gz file) Please advise.
> 
> linux:/boot # crash vmlinux-3.0.10-0.7-default.gz

And what do you mean by "If a user by mistake issues ..."?  What is the
mistake being made?

Dave

> 
> crash: ��{�UU����3ܣ <at> bjb
> R�I-& ��
>        
> �c�"O9�Y��X^f<3��86���v1RS�q��\`������#��^����k�s <at> ��}���=�׳d��g_�^{���Z{�����O-
> F�-W|y�7���f9�k���g���: compressed file name does
> not start with "vmlinux"
>        Use "-f vmlinux-3.0.10-0.7-default.gz" on command line to
>        override.
> 
> crash: vmlinux-3.0.10-0.7-default.gz: not a supported file format
> 
> Aruna
> 
> On Mon, 2011-12-05 at 08:28 -0500, Dave Anderson wrote:
> > The whole point of the "vmlinux" name check is to verify from

Re: [PATCH] Fix junk values when run crash on a .gz file

Hi Dave,

If a user by mistake issues the below command some junk characters are
printed on the console. (crash vmlinux.gz or any .gz file) Please
advise.

linux:/boot # crash vmlinux-3.0.10-0.7-default.gz 

crash: ��{�UU����3ܣ <at> bjb
R�I-& ��

�c�"O9�Y��X^f<3��86���v1RS�q��\`������#��^����k�s <at> ��}���=�׳d��g_�^{���Z{�����O-
F�-W|y�7���f9�k���g���: compressed file name does
not start with "vmlinux"
       Use "-f vmlinux-3.0.10-0.7-default.gz" on command line to override.

crash: vmlinux-3.0.10-0.7-default.gz: not a supported file format

Aruna

On Mon, 2011-12-05 at 08:28 -0500, Dave Anderson wrote:
> The whole point of the "vmlinux" name check is to verify from
> the compressed header itself whether it was generated from a file
> that started with the name-string "vmlinux".  It's not interested 
> in whatever the .gz creator named the compressed file.  If you want
> to work around that restriction, use "crash -f ...".
> 
> Dave
> 
> 

Re: [PATCH] Fix junk values when run crash on a .gz file

On Mon, 2011-12-05 at 09:46 -0500, Dave Anderson wrote:
> 
> ----- Original Message -----
> > Hi Dave,
> > 
> > If a user by mistake issues the below command some junk characters are
> > printed on the console. (crash vmlinux.gz or any .gz file) Please advise.
> > 
> > linux:/boot # crash vmlinux-3.0.10-0.7-default.gz
> > 
> > crash: ��{�UU����3ܣ <at> bjb
> > R�I-& ��
> >        
> > �c�"O9�Y��X^f<3��86���v1RS�q��\`������#��^����k�s <at> ��}���=�׳d��g_�^{���Z{�����O-
> > F�-W|y�7���f9�k���g���: compressed file name does
> > not start with "vmlinux"
> >        Use "-f vmlinux-3.0.10-0.7-default.gz" on command line to
> >        override.
> > 
> > crash: vmlinux-3.0.10-0.7-default.gz: not a supported file format
> 
> Hmmm, I'm not sure why that's happening, although the name string in the
> header does get clipped.  The closest example I have on hand is this one,
> where the pre-compressed filename was vmlinux-2.6.36-0.16.rc3.git0.fc15.x86_64,
> and the .gz filename is vmlinux-2.6.36-0.16.rc3.git0.fc15.x86_64.gz, and 
> which works fine.  And if I change its name to "vmlinux.gz", that still
> works as expected.  So I don't know how to reproduce what you are seeing.
> 
> Anyway, the file command shows the (possibly clipped) string in the header
> that is being looked at:

Re: [PATCH] Fix junk values when run crash on a .gz file

----- Original Message -----
> On Mon, 2011-12-05 at 09:46 -0500, Dave Anderson wrote:
> > 
> > ----- Original Message -----
> > > Hi Dave,
> > > 
> > > If a user by mistake issues the below command some junk characters are
> > > printed on the console. (crash vmlinux.gz or any .gz file) Please advise.
> > > 
> > > linux:/boot # crash vmlinux-3.0.10-0.7-default.gz
> > > 
> > > crash: ��{�UU����3ܣ <at> bjb
> > > R�I-& ��
> > >        
> > > �c�"O9�Y��X^f<3��86���v1RS�q��\`������#��^����k�s <at> ��}���=�׳d��g_�^{���Z{�����O-
> > > F�-W|y�7���f9�k���g���: compressed file name does
> > > not start with "vmlinux"
> > >        Use "-f vmlinux-3.0.10-0.7-default.gz" on command line to
> > >        override.
> > > 
> > > crash: vmlinux-3.0.10-0.7-default.gz: not a supported file format
> > 
> > Hmmm, I'm not sure why that's happening, although the name string in the
> > header does get clipped.  The closest example I have on hand is this one,
> > where the pre-compressed filename was vmlinux-2.6.36-0.16.rc3.git0.fc15.x86_64,
> > and the .gz filename is vmlinux-2.6.36-0.16.rc3.git0.fc15.x86_64.gz, and
> > which works fine.  And if I change its name to "vmlinux.gz", that still
> > works as expected.  So I don't know how to reproduce what you are
> > seeing.

Re: [PATCH] Fix junk values when run crash on a .gz file

----- Original Message -----
> 
> 
> If I look at my sample's gzip header, it looks like:
> 
> $ od -c vmlinux-2.6.36-0.16.rc3.git0.fc15.x86_64.gz | head
> 0000000 037 213  \b  \b   6 254 264   L  \0 003   v   m   l   i   n   u
> 0000020   x   -   2   .   6   .   3   6   -   0   .   1   6   .   r   c
> 0000040   3   .   g   i   t   0   .   f   c   1   5   .   x   8   6   _
> 0000060   6   4  \0 354 375  \v   | 024 325 375 300 177   O   n   $    <at> 
> 0000100   p 303 325 250 250   A 202   b 305   J 274 264   D    <at>  263 232
> 0000120 350 254 335   (   ^   P 274   U   , 202   V 333 376 254 356 002
> 0000140   Z 251   Y 227   ( 343 262 224 326   K 351 035   + 266 264 372
> 0000160 253   x  \a   T   H 270   d 301   k   D   E 224   V   n  \n 023
> 0000200 203 212 250 341   * 373 377   ~ 317 354   - 200 332 377 345   y
> 0000220 376 257 327 363   | 336   /   % 273   s   f 346 234 231   9   s
> $
> 
> where you can see the NULL-terminated filename string starting at
> byte offset 10.  What does yours look like?

I guess the name string can be stripped:

>From this google reference: http://www.dotnetperls.com/gzip-header

 Specification

 After researching this problem, I found the GZIP File Format Specification, 
 which outlines the contents of every valid GZIP file's headers. From the

Re: [PATCH] Fix junk values when run crash on a .gz file

----- Original Message -----
>
> So I guess the proper thing to do would be to check for the FNAME bit
> in the FLG byte, and if it's not set, I guess we could either accept
> the file without prejudice, or print a message indicating that the
> name has been stripped and that -f should be used.

I wasn't aware of "gzip -n" or "gzip --no-name" when I wrote the
original function.  How does the attached patch work for you?
Instead of either suggestion above, it will just print a NOTE if
you don't use "-f".

Dave

----- Original Message -----
>
> So I guess the proper thing to do would be to check for the FNAME bit
> in the FLG byte, and if it's not set, I guess we could either accept
> the file without prejudice, or print a message indicating that the
> name has been stripped and that -f should be used.

I wasn't aware of "gzip -n" or "gzip --no-name" when I wrote the
original function.  How does the attached patch work for you?
Instead of either suggestion above, it will just print a NOTE if
you don't use "-f".