seth vidal | 6 Jul 19:26 2004

Re: One more bit

On Mon, 2004-06-28 at 14:17, Gustavo Niemeyer wrote:
> Hello folks,
> 
> I'm missing one more bit in the metadata format: "prerequires"
> information.
> 
> My suggestion is to include something like:
> 
> <rpm:requires>
> 	<rpm:entry ... pre="1">
> </rpm:requires>
> 

ok, since I've not heard any 'oh my god it's broken' responses from
folks, I'll assume that the prereq listing is working correctly.

We've saved some space by removing the namespace declaration from each
format tag. so any space we lose in pre="1" is in the noise.

Unless I hear an objection I'm going to bundle up these changes and make
a release. I'd like this to be a near-final release for the first
iteration of this format. 

Gustavo, if you have any more recommendations that will break
compatibility, send them to be this week. I'd like to see about getting
a createrepo rpm into FC3test1 so that, hopefully, the repository format
can be in fc3's release.

Thanks
-sv
(Continue reading)

seth vidal | 20 Jul 22:43 2004

new createrepo

Hi,
 I updated createrepo to 0.3.4:
available here:

http://linux.duke.edu/metadata/generate/

changes:
- re-enabled support for 'groups' file - it doesn't do any checking of
the groups file just puts it in the repodata dir and adds it to
repomd.xml so don't count on it being a standard, yet.
- added pre="1" for requirements needed before the file is installed -
this is needed for apt and some other pkg mgrs to get ordering right.
- moved the xmlns:rpm tag to the root tag of the file rather than
per-format tag. 

if this works okay, then great, if not, let me know.

thanks
-sv
Ville Skyttä | 23 Jul 20:20 2004
Picon
Picon

Anonymous CVS?

I suppose there's anoncvs access to the metadata stuff?
Documenting that in the README wouldn't be a bad idea.
seth vidal | 23 Jul 20:53 2004

Re: Anonymous CVS?

On Fri, 2004-07-23 at 21:20 +0300, Ville Skyttä wrote:
> I suppose there's anoncvs access to the metadata stuff?
> Documenting that in the README wouldn't be a bad idea.
> 

yes, good point.

anoncvs is here:

:pserver:anonymous <at> devel.linux.duke.edu:/cvsroot/metadata/cvs-root/metadata/cvs-root

that should work correctly

-sv
Ville Skyttä | 23 Jul 21:07 2004
Picon
Picon

Re: Anonymous CVS?

On Fri, 2004-07-23 at 21:53, seth vidal wrote:
> On Fri, 2004-07-23 at 21:20 +0300, Ville Skyttä wrote:
> > I suppose there's anoncvs access to the metadata stuff?
> > Documenting that in the README wouldn't be a bad idea.

> yes, good point.
> 
> anoncvs is here:
> 
> :pserver:anonymous <at> devel.linux.duke.edu:/cvsroot/metadata/cvs-root/metadata/cvs-root
> 
> that should work correctly

Not quite... I made some educated guesses getting "no such repository"
messages until I used:

  :pserver:anonymous <at> devel.linux.duke.edu:/cvsroot/metadata/cvs-root

...but now I get:

  no such user anonymous in CVSROOT/passwd
seth vidal | 23 Jul 22:17 2004

Re: Anonymous CVS?

On Fri, 2004-07-23 at 22:07 +0300, Ville Skyttä wrote:
> On Fri, 2004-07-23 at 21:53, seth vidal wrote:
> > On Fri, 2004-07-23 at 21:20 +0300, Ville Skyttä wrote:
> > > I suppose there's anoncvs access to the metadata stuff?
> > > Documenting that in the README wouldn't be a bad idea.
> 
> > yes, good point.
> > 
> > anoncvs is here:
> > 
> > :pserver:anonymous <at> devel.linux.duke.edu:/cvsroot/metadata/cvs-root/metadata/cvs-root
> > 
> > that should work correctly
> 
> Not quite... I made some educated guesses getting "no such repository"
> messages until I used:
> 
>   :pserver:anonymous <at> devel.linux.duke.edu:/cvsroot/metadata/cvs-root
> 
> ...but now I get:
> 
>   no such user anonymous in CVSROOT/passwd
> 

fixed, sorry. the anoncvs is replicated out on a regular basis but it
hadn't been synced yet.

should be fine as

:pserver:anonymous <at> devel.linux.duke.edu:/cvsroot/metadata/cvs-root
(Continue reading)

Ville Skyttä | 24 Jul 11:20 2004
Picon
Picon

createrepo: initial comments and a UTF-8 patch

I tried out createrepo yesterday somewhat, here's initial comments:

Issue 1:

It does not currently do a decent job in UTF-8'ifying content.  Not that
it would generate broken XML, but for example the UTF-8 "ä" in my
surname turns in to two "?"s, when it's already UTF-8 in a RPM header!

Patch attached.  This is a simplified version of what I use in fancix,
and the idea originates to decode() Skip Montanaro's query.py at
http://manatee.mojam.com/~skip/python/query.py

The other gotcha in the patch is that when adding content to a libxml2
tree, one does not need to XML escape it.  AFAIK that happens
automatically correctly at serialization time.  XML escaping would be
only needed when printing stuff directly somewhere outside of the libxml
objects; that is not currently done so I nuked xmlCleanString()
altogether.  While at it, I added explicit encodings to serialize()
calls.

With this patch applied, the output is improved quite a bit here.  Add
new encodings to the list in utf8String() if you like.

Issue 2:

The name "author" attribute in <changelog> is not a very good choice
IMO.  RPM defines it as the "name" of the changelog entry.  It is very
common that for RPMs the author attribute will contain stuff like "John
Doe &lt;john at doe dot com&gt; - 2.6.8-0.1", ie. it's not only the
author -> suggesting changing "author" to "name" unless it causes too
(Continue reading)

seth vidal | 24 Jul 16:23 2004

Re: createrepo: initial comments and a UTF-8 patch


> It does not currently do a decent job in UTF-8'ifying content.  Not that
> it would generate broken XML, but for example the UTF-8 "ä" in my
> surname turns in to two "?"s, when it's already UTF-8 in a RPM header!
> 
> Patch attached.  This is a simplified version of what I use in fancix,
> and the idea originates to decode() Skip Montanaro's query.py at
> http://manatee.mojam.com/~skip/python/query.py

my only concern is that it works with some of the suse and pld rpms.
When I tested them before it was very difficult to guess what encoding
they were in. I'll take a look again, thanks.

> The other gotcha in the patch is that when adding content to a libxml2
> tree, one does not need to XML escape it.  AFAIK that happens
> automatically correctly at serialization time.  XML escaping would be
> only needed when printing stuff directly somewhere outside of the libxml
> objects; that is not currently done so I nuked xmlCleanString()
> altogether.  While at it, I added explicit encodings to serialize()
> calls.

That only happens if you do serialize the whole thing, not just a node.
You can't serialize the whole thing b/c it would grow in memory use w/o
bound. That's why I use xmlCleanString().

> With this patch applied, the output is improved quite a bit here.  Add
> new encodings to the list in utf8String() if you like.
> 
> Issue 2:
> 
(Continue reading)

Ville Skyttä | 24 Jul 19:11 2004
Picon
Picon

Re: createrepo: initial comments and a UTF-8 patch

On Sat, 2004-07-24 at 17:23, seth vidal wrote:
> > It does not currently do a decent job in UTF-8'ifying content.  Not that
> > it would generate broken XML, but for example the UTF-8 "ä" in my
> > surname turns in to two "?"s, when it's already UTF-8 in a RPM header!
> > 
> > Patch attached.  This is a simplified version of what I use in fancix,
> > and the idea originates to decode() Skip Montanaro's query.py at
> > http://manatee.mojam.com/~skip/python/query.py
> 
> my only concern is that it works with some of the suse and pld rpms.
> When I tested them before it was very difficult to guess what encoding
> they were in. I'll take a look again, thanks.

I tested with some self-created nasty ones, as well as actual Conectiva
packages.  Could not find a PLD package with "bad" chars in any of the
fields in a quick search (but added ISO-8859-2 to the list of encodings
to guess anyway, thinking about PLD :)

Note that if the conversion is unsuccessful, it falls back to the old
"?" stuff.  And as said, the current code fails with stuff that is
already in UTF-8.  Try any recent package by yours truly...

> That only happens if you do serialize the whole thing, not just a node.
> You can't serialize the whole thing b/c it would grow in memory use w/o
> bound. That's why I use xmlCleanString().

Well, in my tests I could not find a case where libxml would not
automatically do the string cleanup, ie. where createrepo would produce
invalid XML without xmlCleanString() calls in place.  If you have a case
where the auto-escaping does not happen, let me know.
(Continue reading)

seth vidal | 26 Jul 08:25 2004

Re: createrepo: initial comments and a UTF-8 patch


> I tested with some self-created nasty ones, as well as actual Conectiva
> packages.  Could not find a PLD package with "bad" chars in any of the
> fields in a quick search (but added ISO-8859-2 to the list of encodings
> to guess anyway, thinking about PLD :)
> 
> Note that if the conversion is unsuccessful, it falls back to the old
> "?" stuff.  And as said, the current code fails with stuff that is
> already in UTF-8.  Try any recent package by yours truly...

Cool. I'll take a look at some of the packages I used to test with and
see if they explode or not :) the most important thing is that xmllint
--noout doesn't have problems when we're done.

Thanks.

> Well, in my tests I could not find a case where libxml would not
> automatically do the string cleanup, ie. where createrepo would produce
> invalid XML without xmlCleanString() calls in place.  If you have a case
> where the auto-escaping does not happen, let me know.

When I was working on it originally libxml2-python wouldn't play nicely.
If it does now, then great, but I'm afraid of it blowing up on older
versions. I'll see if anything gets odd on older systems.

Thanks.

> Where?  I see RPMTAG_CHANGELOG{TIME,NAME,TEXT}, no author.

You're right. Sorry about that - but I always took name == person who
(Continue reading)


Gmane