Lasse Collin | 29 Sep 14:53 2015

XZ Utils 5.2.2

XZ Utils 5.2.2 is available at <>. Here is an 
extract from the NEWS file:

  * Fixed bugs in QNX-specific code.

  * Omitted the use of pipe2() even if it is available to avoid
    portability issues with some old Linux and glibc combinations.

  * Updated German translation.

  * Added project files to build static and shared liblzma (not the
    whole XZ Utils) with Visual Studio 2013 update 2 or later.

  * Documented that threaded decompression hasn't been implemented
    yet. A 5.2.0 NEWS entry describing multi-threading support had
    incorrectly said "decompression" when it should have said

The 5.0.x branch is no longer maintained. As of 2015-09-29 there are no
known major bugs and it is fine to keep using 5.0.8 for now if you
don't want to upgrade to 5.2.x. The API and ABI in 5.2.x are backward
compatible with 5.0.x so it should be straightforward to upgrade from
5.0.x to 5.2.x.


Lasse Collin  |  IRC: Larhzu  <at>  IRCnet & Freenode

Adam Walling | 22 May 20:10 2015

minimal liblzma MSVC2013 solution and project

This is a minimal solution and two project files which allow liblzma to be built with MSVC2013. I did not attempt to include any tools or tests, or address any warnings. However, it is functional, so I would like to share this as an alternative to the 'Fairly Complete' solution / project.

This does not require any changes to the code itself, just adding the files within the /windows subdirectory is enough.

liblzma.vcxproj builds the static library
liblzma_dll.vcxproj builds a dll

Both projects have x86 and x86-64 platform configurations, as well as Debug, Release, and ReleaseMT configuration -- MT is the compiler switch to link to the CRT statically, so it will not have any other DLL dependencies.

I just put the files up on github alone rather than fork the entire repository:

I created these a while ago but put things on hold waiting for the other solution and projects to materialize, but it has been long enough now I figure I might as well share.



- Adam D. Walling
Hochhaus, Andy | 11 May 09:13 2015

xz 5.2.1 multi-threaded decompress problems (mutli-threaded compress works file)


I am using xz 5.2.1 (compiled from source) on Debian Jessie.

$ xz --version
xz (XZ Utils) 5.2.1
liblzma 5.2.1

When I run parallel compress I see all cores utilized (checked with htop).

xz -T 0 2275.dat

However, when I attempt to run parallel decompress I only see a single
core utilized.

xz -d -T 0 2275.dat.xz

The file I'm working with is sufficiently large (2.3G compressed ; 46G
uncompressed) that I would expect significant benefit from
decompression parallelism.

I noticed that a few other users in the phoronix forum [1] were seeing
similar behavior. Am I missing something obvious?



Lasse Collin | 26 Feb 18:12 2015

XZ Utils 5.2.1

XZ Utils 5.2.1 is available at <>. Here is an 
extract from the NEWS file:

  * Fixed a compression-ratio regression in fast mode of LZMA1 and
    LZMA2. The bug is present in 5.1.4beta and 5.2.0 releases.

  * Fixed a portability problem in xz that affected at least OpenBSD.

  * Fixed xzdiff to be compatible with FreeBSD's mktemp which differs
    from most other mktemp implementations.

  * Changed CPU core count detection to use cpuset_getaffinity() on


Lasse Collin  |  IRC: Larhzu  <at>  IRCnet & Freenode

Gabi Davar | 23 Jan 12:19 2015

Fairly Complete MSVC 2013 Solution/Project


I've created vs2013 based solution for lzma DLL, lzma static and most
utils. It based on the works of Garen & Mārtiņš Možeiko.
It can be found at vs2013_520 branch.

The cmake branch has the build system ported to cmake (only for VS).
It enabled me to build xz for Python 2.7 use. (Intel C++ Compiler
targeted at VS2008).

Any interest?


Dusty Mabe | 5 Jan 05:21 2015

Possible bug in xz/lzma compression


I am noticing an error when compressing/decompressing using xz on Fedora 20. I 
was initially using xz-5.1.2-8alpha.fc20.x86_64 but then upgraded to 
xz-5.1.2-12alpha.fc20.x86_64 to see if that would fix things, but it didn't seem
to help.

I have a file at [1] that I am compressing/decompressing using xz. I first uncompress
it so that it is a plain text file and then compress it with xz. I then decompress it
back to plain text and compare with the original. I keep getting differences.. For example:

[dustymabe <at> localhost traces]$ cat /tmp/diffs 
--- /mnt/SEQ_LBM.t.gz.uncompressed      2015-01-04 17:14:33.178364955 -0500
+++ /mnt/SEQ_LBM.t.xz.uncompressed      2015-01-04 20:32:20.951230115 -0500
 <at>  <at>  -125953555,7 +125953555,7  <at>  <at> 
 r 0x7f0422a8
 w 0x7a7a0218
 r 0x6d480000
-r 0x7f0422a8
+p 0x7f0422a8
 w 0x7a7a00e0
 r 0x6d480008
 r 0x7f0422a8

Can someone please download the file at [1] (~194M) and run a similar test on it? 
The sums on the file are as follows for the gz compressed and uncompressed forms:

[dustymabe <at> localhost traces]$ sha256sum SEQ_LBM.t.gz /mnt/SEQ_LBM.t.gz.uncompressed 
017b3456df8e658e2e50ff3f33ca66ce11eb91b59abfd6c3d259502b53951608  SEQ_LBM.t.gz
ca07c3604b104631bf377cac62537a40bd79be1c3971a23e7605843d5c588e72  /mnt/SEQ_LBM.t.gz.uncompressed

Please confirm if you see the behavior or not or possibly point me to a bug/commit
where this issue was already fixed.


[1] -
Lasse Collin | 21 Dec 22:08 2014

XZ Utils 5.0.8 and 5.2.0

XZ Utils 5.0.8 and 5.2.0 are available at <>.
Here is an extract from the NEWS file:

5.0.8 (2014-12-21)

    * Fixed an old bug in xzgrep that affected OpenBSD and probably
      a few other operating systems too.

    * Updated French and German translations.

    * Added support for detecting the amount of RAM on AmigaOS/AROS.

    * Minor build system updates.

5.2.0 (2014-12-21)

    Since 5.1.4beta:

    * All fixes from 5.0.8

    * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset
      was used.

    * xzdiff: If mktemp isn't installed, mkdir will be used as
      a fallback to create a temporary directory. Installing mktemp
      is still recommended.

    * Updated French, German, Italian, Polish, and Vietnamese

    Summary of fixes and new features added in the 5.1.x development

    * liblzma:

        - Added support for multi-threaded compression. See the
          lzma_mt structure, lzma_stream_encoder_mt(), and
          lzma_stream_encoder_mt_memusage() in <lzma/container.h>,
          lzma_get_progress() in <lzma/base.h>, and lzma_cputhreads()
          in <lzma/hardware.h> for details.

        - Made the uses of lzma_allocator const correct.

        - Added lzma_block_uncomp_encode() to create uncompressed
          .xz Blocks using LZMA2 uncompressed chunks.

        - Added support for LZMA_IGNORE_CHECK.

        - A few speed optimizations were made.

        - Added support for symbol versioning. It is enabled by default
          on GNU/Linux, other GNU-based systems, and FreeBSD.

        - liblzma (not the whole XZ Utils) should now be buildable
          with MSVC 2013 update 2 or later using windows/config.h.

    * xz:

        - Fixed a race condition in the signal handling. It was
          possible that e.g. the first SIGINT didn't make xz exit
          if reading or writing blocked and one had bad luck. The fix
          is non-trivial, so as of writing it is unknown if it will be
          backported to the v5.0 branch.

        - Multi-threaded decompression can be enabled with the
          --threads (-T) option.

        - New command line options in xz: --single-stream,
          --block-size=SIZE, --block-list=SIZES,
          --flush-timeout=TIMEOUT, and --ignore-check.

        - xz -lvv now shows the minimum xz version that is required to
          decompress the file. Currently it is 5.0.0 for all supported
          .xz files except files with empty LZMA2 streams require 5.0.2.

    * xzdiff and xzgrep now support .lzo files if lzop is installed.
      The .tzo suffix is also recognized as a shorthand for .tar.lzo.


Lasse Collin  |  IRC: Larhzu  <at>  IRCnet & Freenode

Kevin Wilson | 9 Nov 19:24 2014

How to compress a folder with xz ?

Hi, all,

The forum webpage give an error. So I hope you will forgive me
for sending to xz-devel, in the hope that someone may advice.

According to the man page of xz, it seems that
xz folderName or
xz folderName/
is enough.
However, when running:
xz /etc/
I get;
xz: /etc/: Is a directory, skipping

I saw that it is possible with the tar command, with soem command options,
but isn't it possible with xz ?


Lasse Collin | 13 Oct 21:28 2014

Optimizing lzma_memcmplen for non-x86 processors

XZ Utils 5.1.4beta got a speed optimization for buffer comparisons
which improves encoding speed. It works on systems that support
unaligned memory access. The relevant code is in

Different architectures get the best performance with different code.
The code should be decent for x86-64 and maybe also for 32-bit x86 (at
least the SSE2 version). Those may still have some room left for
improvement and help is welcome to improve them. However, no one has
looked at how the code could be improved for non-x86 archs, so I'm
especially interested in finding people to help with that.

I have heard that the generic versions work on little endian 32-bit ARM
and 32-bit big endian PowerPC. On those the generic code is slightly
faster than the byte-by-byte buffer comparison, but perhaps
arch-specific code could do better. The method used for x86-64 could be
good for other 64-bit CPUs too if __builtin_ctzll maps to a fast

Timing the speed of "xz -e" when compressing a fairly compressible file
(many source code tarballs are such) is a good way to test different
lzma_memcmplen implementations. The reason for using -e is that the
relative improvement tends to be bigger when that option is used. On
x86-64 I've seen even 25 % faster compression with some files compared
to the byte-by-byte method.


Lasse Collin  |  IRC: Larhzu  <at>  IRCnet & Freenode

Lasse Collin | 20 Sep 20:04 2014

XZ Utils 5.0.7

XZ Utils 5.0.7 is available at <>. Here is an 
extract from the NEWS file:

    * Fix regressions introduced in 5.0.6:

        - Fix building with non-GNU make.

        - Fix invalid Libs.private value in liblzma.pc which broke
          static linking against liblzma if the linker flags were
          taken from pkg-config.


Lasse Collin  |  IRC: Larhzu  <at>  IRCnet & Freenode

Lasse Collin | 14 Sep 21:46 2014

XZ Utils 5.0.6 and 5.1.4beta

XZ Utils 5.0.6 and 5.1.4beta are available at <>.
Here is an extract from the NEWS file:

5.0.6 (2014-09-14)

    * xzgrep now exits with status 0 if at least one file matched.

    * A few minor portability and build system fixes

5.1.4beta (2014-09-14)

    * All fixes from 5.0.6

    * liblzma: Fixed the use of presets in threaded encoder

    * xz --block-list and --block-size can now be used together
      in single-threaded mode. Previously the combination only
      worked in multi-threaded mode.

    * Added support for LZMA_IGNORE_CHECK to liblzma and made it
      available in xz as --ignore-check.

    * liblzma speed optimizations:

        - Initialization of a new LZMA1 or LZMA2 encoder has been
          optimized. (The speed of reinitializing an already-allocated
          encoder isn't affected.) This helps when compressing many
          small buffers with lzma_stream_buffer_encode() and other
          similar situations where an already-allocated encoder state
          isn't reused. This speed-up is visible in xz too if one
          compresses many small files one at a time instead running xz
          once and giving all files as command-line arguments.

        - Buffer comparisons are now much faster when unaligned access
          is allowed (configured with --enable-unaligned-access). This
          speeds up encoding significantly. There is arch-specific code
          for 32-bit and 64-bit x86 (32-bit needs SSE2 for the best
          results and there's no run-time CPU detection for now).
          For other archs there is only generic code which probably
          isn't as optimal as arch-specific solutions could be.

        - A few speed optimizations were made to the SHA-256 code.
          (Note that the builtin SHA-256 code isn't used on all
          operating systems.)

    * liblzma can now be built with MSVC 2013 update 2 or later
      using windows/config.h.

    * Vietnamese translation was added.


Lasse Collin  |  IRC: Larhzu  <at>  IRCnet & Freenode