Martok | 28 Apr 02:44 2016
Picon

Preserve file time on MinGW

Hello List,

xz when built for Windows with gcc on mingw32 or mingw-w64 does not preserve
timestamps. I have found that this is because MinGW only provides utime() with
filename as target, and this means that the file time is adjusted by
io_copy_attrs() and then immediately reset to 'now' when the dest_fd is closed
afterwards in io_close().

I have done a bit of digging and mingw actually has futime() (in <sys/utime.h>),
but for some reason exports it only as _futime(). Only utime() is wrapped with a
non-underscore-version.

Using something like this in io_copy_attrs works just fine and would fix the
issue...
	(void)_futime(pair->dest_fd, &buf);

Alternatively systems that need a file name could maybe first close the file and
then set the file time, but that would complicate things...

Regards,
Martok

Earnestly | 21 Apr 01:32 2016

[PATCH] Remove AC_CONFIG_AUX_DIR

When using ./autogen.sh or autoreconf -i (the prefered way going forward
apparently[0]) I get the following error:

    configure.ac:596: error: required file 'build-aux/ltmain.sh' not found

Removing the AUX_DIR lets ./configure complete successfully.

See also:
cloog https://groups.google.com/forum/?hl=en#!topic/cloog-development/28_M3Qk87-Q
check https://github.com/libcheck/check/pull/36
util-linux  http://marc.info/?l=util-linux-ng&m=146118708001749&w=2#1

0: https://www.gnu.org/software/automake/manual/html_node/Future-of-aclocal.html

Signed-off-by: Earnestly <zibeon@...>
---
 configure.ac | 1 -
 1 file changed, 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 3df4348..24c4fed 100644
--- a/configure.ac
+++ b/configure.ac
 <at>  <at>  -20,7 +20,6  <at>  <at>  AC_PREREQ([2.64])
 AC_INIT([XZ Utils], m4_esyscmd([/bin/sh build-aux/version.sh]),
 	[lasse.collin@...], [xz], [http://tukaani.org/xz/])
 AC_CONFIG_SRCDIR([src/liblzma/common/common.h])
-AC_CONFIG_AUX_DIR([build-aux])
 AC_CONFIG_MACRO_DIR([m4])
 AC_CONFIG_HEADER([config.h])
(Continue reading)

ty armour | 13 Apr 00:57 2016

(unknown)

unsubscribe xz-devel
ty armour | 12 Apr 19:33 2016

(unknown)

unsubscribe xz-devel
Matt Parlane | 12 Apr 03:05 2016
Picon

Only single core used when trying multi-thread compression

Hi all...

I have compiled xz from source (rev ac398c3), and I am trying to use
multithread compression by using -T 0 in the command line.

What I am finding is that when run straight from the command line, it
uses multiple cores as expected. However, when running via a shell
script via cron, only one core is utilised.

Any ideas? I'm running exactly the same command on the same file, and
I would really love to make full use of my system.

Thanks,

Matt Parlane

ty armour | 24 Jan 21:50 2016

Tutorial suggesitons

I have a theory on data compression, but I need tutorials on data compression in order to implement my theory.

Anyway if you want to, just post tutorials on every and any aspect of data execution and data compression.

I am building my own computers(like my own microchips) and this will help me out greatly to not only have a data compression software, but to also be able to save money and not need nearly as much ram.

Thanks for your time
-Ty
Rich Prohaska | 3 Nov 17:21 2015
Picon

valgrind conditional jump depends on uninitialized value

Hello,
Valgrind is reporting a uninitialized value problem when using xz library.

$ valgrind ./lzma-uninit-prepare
==21883== Memcheck, a memory error detector
==21883== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==21883== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==21883== Command: ./lzma-uninit-prepare
==21883==
lzma 5.2.2
1 148
==21883== Conditional jump or move depends on uninitialised value(s)
==21883==    at 0x4E4329D: lz_encoder_prepare (lz_encoder.c:231)
==21883==    by 0x4E43902: lzma_lz_encoder_init (lz_encoder.c:560)
==21883==    by 0x4E39D9B: lzma_raw_coder_init (filter_common.c:286)
==21883==    by 0x4E3B6AC: block_encode_normal (block_buffer_encoder.c:189)
==21883==    by 0x4E3B6AC: block_buffer_encode.part.1
(block_buffer_encoder.c:271)
==21883==    by 0x4E3B9A0: block_buffer_encode (block_buffer_encoder.c:322)
==21883==    by 0x4E3B9A0: lzma_block_buffer_encode (block_buffer_encoder.c:323)
==21883==    by 0x4E3CDE7: lzma_stream_buffer_encode
(stream_buffer_encoder.c:96)
==21883==    by 0x4E3C0C3: lzma_easy_buffer_encode (easy_buffer_encoder.c:25)
==21883==    by 0x400913: main (lzma-uninit-prepare.c:21)
==21883==
53

Cause:
mf->size is read is lz_encoder.c:226 before initialized. mf is
allocated in lz_encoder.c:532, buffer is iniitalized, size is NOT
initialized.  gcc 4.9 apparently compiles lz_encoder.c:231 into code
that uses old_size before the buffer != NULL check occurs.

Solution:
set mf->size = 0 after allocation in lz_encoder.c:532.

Reproduce
git clone http://git.tukaani.org/xz.git && git checkout v5.2.2
./configure --prefix=$HOME/usr/local/xz && make install
make and run my reproducer.

$ cat Makefile
XZBASE = $(HOME)/usr/local/xz
CPPFLAGS = -I$(XZBASE)/include
CFLAGS = -g -O0 -std=c99
LDFLAGS = -L$(XZBASE)/lib -llzma

lzma-uninit-prepare: lzma-uninit-prepare.c
        $(CC) $(CPPFLAGS) $(CFLAGS) -o $ <at>  $< $(LDFLAGS)

clean:
        rm -rf lzma-uninit-prepare

$ cat lzma-uninit-prepare.c
#include <stdio.h>
#include <assert.h>
#include <malloc.h>
#include <lzma.h>

int main(void) {
    printf("lzma %s\n", lzma_version_string());
    size_t src_len = 1;
    uint8_t *src = (uint8_t *) malloc(src_len);
    assert(src);
    for (int i = 0; i < src_len; i++)
        src[i] = 0;
    size_t compress_bound = lzma_stream_buffer_bound(src_len);
    printf("%lu %lu\n", src_len, compress_bound);

    size_t dest_len = 1 + compress_bound;
    uint8_t *dest = (uint8_t *) malloc(dest_len);
    assert(dest);

    size_t compress_size = 1;
    lzma_ret lr = lzma_easy_buffer_encode(2, LZMA_CHECK_NONE, NULL,
src, src_len, dest, &compress_size, dest_len);
    assert(lr == LZMA_OK);
    printf("%lu\n", compress_size);

    free(src);
    free(dest);

    return 0;
}

Ian S. Nelson | 23 Oct 15:58 2015
Picon

(unknown)

subscribe xz-devel
--
signature


Ian S. Nelson
PGP/GPG email preferred.
Public Key: 00D3D983
Fingerprint: 3EFD7B86B888D7E229B69E97576F1B9700D3D983

Lasse Collin | 29 Sep 14:53 2015

XZ Utils 5.2.2

XZ Utils 5.2.2 is available at <http://tukaani.org/xz/>. Here is an 
extract from the NEWS file:

  * Fixed bugs in QNX-specific code.

  * Omitted the use of pipe2() even if it is available to avoid
    portability issues with some old Linux and glibc combinations.

  * Updated German translation.

  * Added project files to build static and shared liblzma (not the
    whole XZ Utils) with Visual Studio 2013 update 2 or later.

  * Documented that threaded decompression hasn't been implemented
    yet. A 5.2.0 NEWS entry describing multi-threading support had
    incorrectly said "decompression" when it should have said
    "compression".

The 5.0.x branch is no longer maintained. As of 2015-09-29 there are no
known major bugs and it is fine to keep using 5.0.8 for now if you
don't want to upgrade to 5.2.x. The API and ABI in 5.2.x are backward
compatible with 5.0.x so it should be straightforward to upgrade from
5.0.x to 5.2.x.

--

-- 
Lasse Collin  |  IRC: Larhzu  <at>  IRCnet & Freenode

Adam Walling | 22 May 20:10 2015
Picon

minimal liblzma MSVC2013 solution and project

This is a minimal solution and two project files which allow liblzma to be built with MSVC2013. I did not attempt to include any tools or tests, or address any warnings. However, it is functional, so I would like to share this as an alternative to the 'Fairly Complete' solution / project.

This does not require any changes to the code itself, just adding the files within the /windows subdirectory is enough.

liblzma.vcxproj builds the static library
liblzma_dll.vcxproj builds a dll

Both projects have x86 and x86-64 platform configurations, as well as Debug, Release, and ReleaseMT configuration -- MT is the compiler switch to link to the CRT statically, so it will not have any other DLL dependencies.

I just put the files up on github alone rather than fork the entire repository:


I created these a while ago but put things on hold waiting for the other solution and projects to materialize, but it has been long enough now I figure I might as well share.

Thanks!

--

- Adam D. Walling
Hochhaus, Andy | 11 May 09:13 2015

xz 5.2.1 multi-threaded decompress problems (mutli-threaded compress works file)

Hello,

I am using xz 5.2.1 (compiled from source) on Debian Jessie.

$ xz --version
xz (XZ Utils) 5.2.1
liblzma 5.2.1

When I run parallel compress I see all cores utilized (checked with htop).

xz -T 0 2275.dat

However, when I attempt to run parallel decompress I only see a single
core utilized.

xz -d -T 0 2275.dat.xz

The file I'm working with is sufficiently large (2.3G compressed ; 46G
uncompressed) that I would expect significant benefit from
decompression parallelism.

I noticed that a few other users in the phoronix forum [1] were seeing
similar behavior. Am I missing something obvious?

Thanks,
-Andy

[1] http://www.phoronix.com/forums/forum/software/general-linux-open-source/47329-xz-5-2-adds-new-multi-threaded-options


Gmane