Gabi Davar | 23 Jan 12:19 2015
Picon

Fairly Complete MSVC 2013 Solution/Project

Hi,

I've created vs2013 based solution for lzma DLL, lzma static and most
utils. It based on the works of Garen & Mārtiņš Možeiko.
It can be found at https://github.com/mindw/xz vs2013_520 branch.

The cmake branch has the build system ported to cmake (only for VS).
It enabled me to build xz for Python 2.7 use. (Intel C++ Compiler
targeted at VS2008).

Any interest?

-gabi

Dusty Mabe | 5 Jan 05:21 2015

Possible bug in xz/lzma compression


Hi,

I am noticing an error when compressing/decompressing using xz on Fedora 20. I 
was initially using xz-5.1.2-8alpha.fc20.x86_64 but then upgraded to 
xz-5.1.2-12alpha.fc20.x86_64 to see if that would fix things, but it didn't seem
to help.

I have a file at [1] that I am compressing/decompressing using xz. I first uncompress
it so that it is a plain text file and then compress it with xz. I then decompress it
back to plain text and compare with the original. I keep getting differences.. For example:

[dustymabe <at> localhost traces]$ cat /tmp/diffs 
--- /mnt/SEQ_LBM.t.gz.uncompressed      2015-01-04 17:14:33.178364955 -0500
+++ /mnt/SEQ_LBM.t.xz.uncompressed      2015-01-04 20:32:20.951230115 -0500
 <at>  <at>  -125953555,7 +125953555,7  <at>  <at> 
 r 0x7f0422a8
 w 0x7a7a0218
 r 0x6d480000
-r 0x7f0422a8
+p 0x7f0422a8
 w 0x7a7a00e0
 r 0x6d480008
 r 0x7f0422a8

Can someone please download the file at [1] (~194M) and run a similar test on it? 
The sums on the file are as follows for the gz compressed and uncompressed forms:

[dustymabe <at> localhost traces]$ sha256sum SEQ_LBM.t.gz /mnt/SEQ_LBM.t.gz.uncompressed 
017b3456df8e658e2e50ff3f33ca66ce11eb91b59abfd6c3d259502b53951608  SEQ_LBM.t.gz
(Continue reading)

Lasse Collin | 21 Dec 22:08 2014

XZ Utils 5.0.8 and 5.2.0

XZ Utils 5.0.8 and 5.2.0 are available at <http://tukaani.org/xz/>.
Here is an extract from the NEWS file:

5.0.8 (2014-12-21)

    * Fixed an old bug in xzgrep that affected OpenBSD and probably
      a few other operating systems too.

    * Updated French and German translations.

    * Added support for detecting the amount of RAM on AmigaOS/AROS.

    * Minor build system updates.

5.2.0 (2014-12-21)

    Since 5.1.4beta:

    * All fixes from 5.0.8

    * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset
      was used.

    * xzdiff: If mktemp isn't installed, mkdir will be used as
      a fallback to create a temporary directory. Installing mktemp
      is still recommended.

    * Updated French, German, Italian, Polish, and Vietnamese
      translations.

(Continue reading)

Kevin Wilson | 9 Nov 19:24 2014
Picon

How to compress a folder with xz ?

Hi, all,

The forum webpage give an error. So I hope you will forgive me
for sending to xz-devel, in the hope that someone may advice.

According to the man page of xz, it seems that
xz folderName or
xz folderName/
is enough.
However, when running:
xz /etc/
I get;
xz: /etc/: Is a directory, skipping

I saw that it is possible with the tar command, with soem command options,
but isn't it possible with xz ?

regards,
Kevin

Lasse Collin | 13 Oct 21:28 2014

Optimizing lzma_memcmplen for non-x86 processors

XZ Utils 5.1.4beta got a speed optimization for buffer comparisons
which improves encoding speed. It works on systems that support
unaligned memory access. The relevant code is in
src/liblzma/common/memcmplen.h:

    http://git.tukaani.org/?p=xz.git;a=blob;f=src/liblzma/common/memcmplen.h

Different architectures get the best performance with different code.
The code should be decent for x86-64 and maybe also for 32-bit x86 (at
least the SSE2 version). Those may still have some room left for
improvement and help is welcome to improve them. However, no one has
looked at how the code could be improved for non-x86 archs, so I'm
especially interested in finding people to help with that.

I have heard that the generic versions work on little endian 32-bit ARM
and 32-bit big endian PowerPC. On those the generic code is slightly
faster than the byte-by-byte buffer comparison, but perhaps
arch-specific code could do better. The method used for x86-64 could be
good for other 64-bit CPUs too if __builtin_ctzll maps to a fast
instruction.

Timing the speed of "xz -e" when compressing a fairly compressible file
(many source code tarballs are such) is a good way to test different
lzma_memcmplen implementations. The reason for using -e is that the
relative improvement tends to be bigger when that option is used. On
x86-64 I've seen even 25 % faster compression with some files compared
to the byte-by-byte method.

--

-- 
Lasse Collin  |  IRC: Larhzu  <at>  IRCnet & Freenode
(Continue reading)

Lasse Collin | 20 Sep 20:04 2014

XZ Utils 5.0.7

XZ Utils 5.0.7 is available at <http://tukaani.org/xz/>. Here is an 
extract from the NEWS file:

    * Fix regressions introduced in 5.0.6:

        - Fix building with non-GNU make.

        - Fix invalid Libs.private value in liblzma.pc which broke
          static linking against liblzma if the linker flags were
          taken from pkg-config.

--

-- 
Lasse Collin  |  IRC: Larhzu  <at>  IRCnet & Freenode

Lasse Collin | 14 Sep 21:46 2014

XZ Utils 5.0.6 and 5.1.4beta

XZ Utils 5.0.6 and 5.1.4beta are available at <http://tukaani.org/xz/>.
Here is an extract from the NEWS file:

5.0.6 (2014-09-14)

    * xzgrep now exits with status 0 if at least one file matched.

    * A few minor portability and build system fixes

5.1.4beta (2014-09-14)

    * All fixes from 5.0.6

    * liblzma: Fixed the use of presets in threaded encoder
      initialization.

    * xz --block-list and --block-size can now be used together
      in single-threaded mode. Previously the combination only
      worked in multi-threaded mode.

    * Added support for LZMA_IGNORE_CHECK to liblzma and made it
      available in xz as --ignore-check.

    * liblzma speed optimizations:

        - Initialization of a new LZMA1 or LZMA2 encoder has been
          optimized. (The speed of reinitializing an already-allocated
          encoder isn't affected.) This helps when compressing many
          small buffers with lzma_stream_buffer_encode() and other
          similar situations where an already-allocated encoder state
(Continue reading)

Florian Weimer | 31 Jul 16:40 2014
Picon

Disabling CRC/SHA-256 checks on decompression

Would it be possible to add a flag to disable these checks during 
decompression?  I have data format and lots of data encoded in it (RPMs, 
in case you wonder) which has its own integrity checking, and 
unfortunately, all the existing XZ streams have been built with SHA-256 
hashing.  Being able to disable hashing would result in a nice speed-up 
for me (based on preliminary tests using hand-crafted RPMs).

--

-- 
Florian Weimer / Red Hat Product Security

Pavel Raiskup | 11 Jun 16:22 2014
Picon

xzgrep should success if at least one file matches

Hi, in RHBZ, there was reported problem with xzgrep, we should exit 0 when
at lest one file contains matching string.  Grep behaves similarly.

Original bugreport:
https://bugzilla.redhat.com/show_bug.cgi?id=1108085

Patch is attached,
Pavel
Lasse Collin | 25 May 21:51 2014

Re: marking version 5.1 stable?

On 2014-05-23 Pavel Raiskup wrote:
> Looking at the http://tukaani.org/xz/ page for some time, I am curious
> whether we could "stabilize" the version 5.1.  Almost all
> distributions are shipping alpha/beta versions of xz* packages which
> is probably not what especially library users want.

Yes, the current situation isn't good.

> What are plans on this topic?  I checked the TODO file and didn't find
> what exactly we need to fix to mark xz 5.1 stable.

For the past year or more, the plan has been to just get the 5.2.0 out.
It's so horribly late that I don't plan to do anything except fix bugs
and possibly some do simple enhancements. The rest must wait past 5.2.0.

Somehow months just pass and I get little done (with xz or anything
else). Anyway, here are some things that I plan to do before 5.2.0:

  * Skim through some of the new code in case I can spot problems that
    should be fixed before 5.2.0.

  * Ensure that the new APIs look OK for long-term support (I like to
    keep API & ABI stable).

  * Check that the new xz features are correctly documented on
    the xz man page.

  * Once I'm sure I won't change any message strings, I need to ask
    for updated translations from the translators.

(Continue reading)

Lasse Collin | 8 Mar 12:43 2014

XZ for Java 1.5

XZ for Java 1.5 is available at <http://tukaani.org/xz/java.html> and
in the Maven Central (groupId = org.tukaani, artifactId = xz). Here is
an extract from the NEWS file:

    * Fix a wrong assertion in BCJ decoders.

    * Use a field instead of reallocating a temporary one-byte buffer
      in read() and write() implementations in several classes.

--

-- 
Lasse Collin  |  IRC: Larhzu  <at>  IRCnet & Freenode


Gmane