headers
Pavel Roskin | 1 Mar 2009 06:47
Picon

Re: [Orinoco-users] linux-firmware binary corruption with gitweb

On Sat, 2009-02-28 at 19:24 +0000, Dave wrote:
> I'm aware of at least a couple users of orinoco who have picked up
> corrupt firmware# from the linux-firmware tree*.
> 
> I've verified that the firmware in the repository itself is correct.
> 
> It appears that downloading the file using the blob/raw links from
> gitweb causes the corruption (0xc3 everywhere). At least it does with
> firefox.

I can confirm the problem with Firefox 3.0.6.  But it's not "0xc3
everywhere".  The corrupted file is a result of recoding from iso-8859-1
to utf-8.  The correct agere_sta_fw.bin is 65046 bytes long.  The
corrupted agere_sta_fw.bin is 89729 bytes long.

There is a way to recode the original binary with GNU recode:
recode utf8..iso8859-1 agere_sta_fw.bin

wget 1.11.4 also gets a corrupted file 89729 bytes long.

$ wget "http://git.kernel.org/?p=linux/kernel/git/dwmw2/linux-firmware.git;a=blob;f=agere_sta_fw.bin;h=bae000f5a7162f5a5b052a2f5b78016e95f825c5;hb=d4cfa9f14c55e9d62f053a542fac21744f22546b"
--2009-03-01 00:42:38--  http://git.kernel.org/?p=linux/kernel/git/dwmw2/linux-firmware.git;a=blob;f=agere_sta_fw.bin;h=bae000f5a7162f5a5b052a2f5b78016e95f825c5;hb=d4cfa9f14c55e9d62f053a542fac21744f22546b
Resolving git.kernel.org... 204.152.191.40, 149.20.20.136
Connecting to git.kernel.org|204.152.191.40|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/octet-stream]
Saving to: `index.html?p=linux%2Fkernel%2Fgit%2Fdwmw2%2Flinux-firmware.git;a=blob;f=agere_sta_fw.bin;h=bae000f5a7162f5a5b052a2f5b78016e95f825c5;hb=d4cfa9f14c55e9d62f053a542fac21744f22546b'

    [  <=>                                                  ] 89,729       237K/s   in 0.4s
(Continue reading)

Permalink | Reply | 1 comment |
headers
Dave | 3 Mar 2009 19:59

Re: [Orinoco-users] linux-firmware binary corruption with gitweb

Adding the git mailing list.

Pavel Roskin wrote:
> On Sat, 2009-02-28 at 19:24 +0000, Dave wrote:
>> I'm aware of at least a couple users of orinoco who have picked up
>> corrupt firmware# from the linux-firmware tree*.
>>
>> I've verified that the firmware in the repository itself is correct.
>>
>> It appears that downloading the file using the blob/raw links from
>> gitweb causes the corruption (0xc3 everywhere). At least it does with
>> firefox.
> 
> I can confirm the problem with Firefox 3.0.6.  But it's not "0xc3
> everywhere".  The corrupted file is a result of recoding from iso-8859-1
> to utf-8.  The correct agere_sta_fw.bin is 65046 bytes long.  The
> corrupted agere_sta_fw.bin is 89729 bytes long.
> 
> There is a way to recode the original binary with GNU recode:
> recode utf8..iso8859-1 agere_sta_fw.bin
> 
> wget 1.11.4 also gets a corrupted file 89729 bytes long.
> 
> curl 7.18.2 also get the corrupted file:
> 
> My strong impression is that the recoding takes place on the server.  I
> think the bug should be reported to the gitweb maintainers unless it a
> local breakage on the kernel.org site.

Thanks Pavel.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Jakub Narebski | 4 Mar 2009 01:26
Picon
Gravatar

Re: [Orinoco-users] linux-firmware binary corruption with gitweb

Dave <kilroyd <at> googlemail.com> writes:

> Adding the git mailing list.
> 
> Pavel Roskin wrote:
> > On Sat, 2009-02-28 at 19:24 +0000, Dave wrote:

>>> I'm aware of at least a couple users of orinoco who have picked up
>>> corrupt firmware# from the linux-firmware tree*.
>>>
>>> I've verified that the firmware in the repository itself is correct.
>>>
>>> It appears that downloading the file using the blob/raw links from
>>> gitweb causes the corruption (0xc3 everywhere). At least it does with
>>> firefox.
>> 
>> I can confirm the problem with Firefox 3.0.6.  But it's not "0xc3
>> everywhere".  The corrupted file is a result of recoding from iso-8859-1
>> to utf-8.  The correct agere_sta_fw.bin is 65046 bytes long.  The
>> corrupted agere_sta_fw.bin is 89729 bytes long.

[...]
>> My strong impression is that the recoding takes place on the server.  I
>> think the bug should be reported to the gitweb maintainers unless it a
>> local breakage on the kernel.org site.
> 
> Thanks Pavel.
> 
> I just did a quick scan of the gitweb README - is this an issue with the
> $mimetypes_file or $fallback_encoding configurations variables?
(Continue reading)

Permalink | Reply | 1 comment |
headers
Dave | 5 Mar 2009 00:52

Re: [Orinoco-users] linux-firmware binary corruption with gitweb

Jakub Narebski wrote:
> Dave <kilroyd <at> googlemail.com> writes:
>>> My strong impression is that the recoding takes place on the server.  I
>>> think the bug should be reported to the gitweb maintainers unless it a
>>> local breakage on the kernel.org site.
>> Thanks Pavel.
>>
>> I just did a quick scan of the gitweb README - is this an issue with the
>> $mimetypes_file or $fallback_encoding configurations variables?
> 
> First, what version of gitweb do you use? It should be in 'Generator'
> meta header, or (in older gitweb) in comments in HTML source at the
> top of the page.

Not sure where I'd find the meta header, but at the top of the HTML:

<!-- git web interface version 1.4.5-rc0.GIT-dirty, (C) 2005-2006, Kay
Sievers <kay.sievers <at> vrfy.org>, Christian Gierke -->
<!-- git core binaries version 1.6.1.1 -->

> Second, the file is actually sent to browser 'as is', using binmode :raw
> (or at least should be according to my understanding of Perl). And *.bin
> binary file gets application/octet-stream mimetype, and doesn't send any
> charset info. git.kernel.org should have modern enough gitweb to use this.
> Strange...

Dug around gitweb.perl in the main git repo. Then looked at the
git/warthog9/gitweb.git repo (after noting the Git Wiki says kernel.org
is running John Hawley's branch).
(Continue reading)

Permalink | Reply | 2 comments |
headers
Pavel Roskin | 5 Mar 2009 18:26
Picon

Re: linux-firmware binary corruption with gitweb

On Wed, 2009-03-04 at 23:52 +0000, Dave wrote:
>         binmode STDOUT, ':raw';
> -        print <$fd>;
> +        #print <$fd>;
> +        $output .= <$fd>;
>         binmode STDOUT, ':utf8'; # as set at the beginning of
> gitweb.cgi

Nice catch!

Looking at the gitweb repository from kernel.org, two instances of
circumventing binmode were introduced by this commit:

commit c79ae555fb3c89d91b4cafbfce306e695720507b
Author: John Hawley <warthog9@...>
Date:   Thu Dec 28 21:59:43 2006 -0800

    Last of the changes to deal with channeling the text through the caching
    engine.  Wow is this a total hack.

The original behavior was restored in git_snapshot() by the recent
commit c15229acd9bedf165f1eb05d99fa989d3b9f3e32, but git_blob_plain()
remains broken.

I don't see an easy fix.  We cannot manipulate the blob to counteract
the encoding, as it may not be valid utf-8, and therefore won't be
output in the utf-8 mode.

Maybe binmode should be raw everywhere, and adding to $output should
recode data to utf-8 from other encodings where needed, but it would be
(Continue reading)

Permalink | Reply | 0 comments |
headers
Jakub Narebski | 6 Mar 2009 01:03
Picon
Gravatar

Re: [Orinoco-users] linux-firmware binary corruption with gitweb

On Thu, 5 March 2009, Dave wrote:
> Jakub Narebski wrote:
>> Dave <kilroyd <at> googlemail.com> writes:

>>>> My strong impression is that the recoding takes place on the server.  I
>>>> think the bug should be reported to the gitweb maintainers unless it a
>>>> local breakage on the kernel.org site.

It is on server, but kernel.org runs modified version of gitweb, and
the bug is in the modifications.  See below.

CC-ed John 'Warthog9' Hawley, maintainer of gitweb on kernel.org

>>>>
>>> Thanks Pavel.
>>>
>>> I just did a quick scan of the gitweb README - is this an issue with the
>>> $mimetypes_file or $fallback_encoding configurations variables?
>> 
>> First, what version of gitweb do you use? It should be in 'Generator'
>> meta header, or (in older gitweb) in comments in HTML source at the
>> top of the page.
> 
> Not sure where I'd find the meta header,

<meta name="generator" content="gitweb/1.4.5-rc0.GIT-dirty git/1.6.1.1"/>

> but at the top of the HTML: 
> 
> <!-- git web interface version 1.4.5-rc0.GIT-dirty, (C) 2005-2006, Kay
(Continue reading)

Permalink | Reply | 0 comments |

Gmane