Kylie McClain | 23 Jul 11:55 2015

Re: Yet another checksum proposal


On 07/23/2015 05:12 AM, Niels Ole Salscheider wrote:
> Hi,
> 
> this sounds like a proposal that could be implemented quickly. It
> might only protect against broken downloads and upstream changing
> files but I would be glad if at least this annoying problem was
> solved.
> 
> Would this include unofficial repositories?

Yes, it would. Unofficial repository distfiles fetching was enabled on
distfiles.e.o a little while ago.

(that reminds me, maybe I should post the list of packages with bad
DOWNLOADS that the mirror daemon keeps spamming me with every day and
get others to do my work for me...)

> I thought a bit more about this problem after my last mail to the
> list and I think that another alternative would be to let paludis
> handle this: Imagine someone bumps a package with "git mv". He or
> she will still test the change locally (we all do, don't we?) and
> therefore use paludis to build it. Paludis would see that there is
> no checksum stored for (some of) the distfiles. It could then offer
> to add it to some defined file inside the repository, git commit
> the change (or amend the last commit) and push it to the local
> repository from which it synced before.

The concern I have there is that we get into the messy part of dealing
with automating SCM changes, which would result in making Paludis have
(Continue reading)

Kylie McClain | 21 Jul 23:14 2015

Yet another checksum proposal

Hi everyone, a week or so ago me and Kim were talking about the
distfiles mirror, and we began to drift to the topic of checksumming and
how we could prevent bad downloads and such. Due to to it being 5:30 AM
at the time, I believe I managed to come up with an idea of how to
possibly work checksumming into our system in a way that doesn't
interrupt our One True Workflow.

Essentially, we'd offload checksum generation to the mirror by adding a
manifest generation to run_accerso.sh. Every day when accerso is ran on
distfiles.e.o, it would do a few sanity checks on the fetched files,
delete bad ones, and create a list of checksums which would be fetched
at some point in time, perhaps sync, and then used during installation.

A few obvious issues with this is how it would work with locally added
packages (such as when bumping a package and testing it), and how it
puts trust in the mirror to generate good checksums. The first issue
could be taken care of by allowing fetching to continue with a warning
or something that the integrity can't be checked, and the second... the
second I don't believe is that much of an issue, because if the mirror
has a bad checksum, then that means bad fetches, which means we have a
problem anyway and someone needs to fix that package's bad DOWNLOADS.

Benedikt proposed for the sanity checks, we could just check the file
extension of what we've fetched against what the actual file type is;
ex. `for file in *.tar.gz;do [[ $(file -b --mime-type) ==
application/x-gzip ]] || rm -v "$file";done` or something similar.
That would be able to protect the checksum generation from at least
things like badly configured servers serving 404 pages without sending
an actual 404 status back to us.

(Continue reading)

Kylie McClain | 19 Jul 18:17 2015

The lists are back up

The lists are back! Yay!
Bernhard Frauendienst | 13 Jun 14:13 2015
Picon

Add debian to REMOTE_IDS

Hi everybody,

I propose to add debian to the list of remote databases (REMOTE_IDS).

This could be useful for clients that use the debian-maintained
information to do further processing. The project I'm having in mind
here is exherbo-pkg[1], which uses the debian watch file to find new
package versions.
While this is already possible for many packages which have the same
name on exherbo and debian, there are a number of packages which
currently don't satisfy this requirement. Adding a debian REMOTE_ID for
those packages would allow the tool to work with them as well.

If this proposal is accepted, debian.exlib should probably patched to
provide the remote id automatically.

What do you think?

Best Regards
Bernhard

[1] https://galileo.mailstation.de/pkg/exherbo-pkg.xml
Wulf C. Krueger | 7 Jun 17:04 2015

Install systemd units unconditionally


Hello,

I'd like to suggest installing systemd units unconditionally. At the
moment, the "systemd" option is mostly used to install them.

Sometimes, though, e. g. in PHP, it means something completely
different. In other cases, it's used to break cyclic dependencies and
in yet other cases, it's actually used to disable systemd integration.

It's a mess.

If we just install systemd units unconditionally...

we can use the "systemd" option to just mean "systemd integration"
(and a handful of cyclic dep breakers),

we don't hurt anyone - they're tiny text files,

we reduce confusion (e. g. PHP usually should have the systemd
option *disabled*),

we get rid of an option that's purely for runtime configuration,

we reduce the options clutter,

Plus: The systemd option defaults to on anyway so unless someone
globally sets "-systemd" (fool!) the units are installed anyway.

Please do respond even if it's just +1. Knowing you lazy bastards ;-),
(Continue reading)

Benedikt Morbach | 21 May 02:11 2015

dosbin, newsbin and heresbin are banned

As we merged sbin and bin with multiarch, 'dosbin', 'newsbin' and
'heresbin' are now banned in exheres-0. The fix is to replace them
with their 'bin' equivalents.

Please note that if you do find an exheres that still uses them then
it hasn't been adjusted to multiarch yet and might need some further
changes.

Cheers
Benedikt
Ciaran McCreesh | 4 Apr 13:54 2015

Paludis 2.4.0 Released

Paludis 2.4.0 has been released:

2.4.0:
    * Bug fixes.

    * We now use Ruby 2.2, unless --with-ruby-version is specified.

--

-- 
Ciaran McCreesh
_______________________________________________
Exherbo-dev mailing list
Exherbo-dev@...
http://lists.exherbo.org/mailman/listinfo/exherbo-dev
Saleem Abdulrasool | 6 Apr 02:25 2015

cross/multiarch is live!

Hello exherbo-dev,

We have merged the cross branch on arbor into master.  This means that cross is now live!  Owners of other repositories are encouraged to do the same.

Much like the case with multibuild/multilib, this does not mean that cross is complete.  It does however mean that everyone can more easily take advantage of this progress.

Not all packages have been adapted to cross/multiarch.  However, many people have been using the branches and a large number of packages either just worked or have been adjusted slightly to work with the requirements for cross.

We have taken steps to ensure that the changes are as transparent as possible, though, some of the biggest changes involve the toolchain packages which now require the target triple prefixing on the tools.  Other than that, it does require that you perform a migration.  We have instructions for that at http://exherbo.org/docs/multiarch.html.  Please follow the guide closely as otherwise you may render your system in an inoperable state.

We have some documentation to help with familiarising yourself with packaging with cross at http://exherbo.org/docs/cross.html.  As usual, if you find issues please try to fix them if you can, otherwise ask around on #exherbo or #exherbo-multiarch.

Finally, thank you to everyone who has helped make cross/multiarch a reality.  This would have been impossible without their help, and there are too many to list individually.  This is certainly a large change and opens a number of possibilities in the future.  

--
Saleem Abdulrasool
compnerd (at) compnerd (dot) org
_______________________________________________
Exherbo-dev mailing list
Exherbo-dev@...
http://lists.exherbo.org/mailman/listinfo/exherbo-dev
Wulf C. Krueger | 6 Apr 02:05 2015

The Great Merge II: Exherbo's multi-arch/cross implementation has been merged to master


Exherbo's multi-arch/cross implementation[1] has been merged to master.

You WILL have to migrate to this new setup in order to be able to
install new software and updates.

You'll find the migration guide at [2]. Make sure to follow it VERY
closely because getting things wrong does have the potential to break
your system. (You'll of course be able to access your data via a rescue
system if everything goes wrong. We're changing the base system but are
NOT touching user data.)

If you don't want to migrate right now, you shouldn't update or install
ANYTHING anymore until you are ready to do so.

The last commit prior to the migration has been tagged as "pre-cross" in
all official repositories so that you can stick with that if need be.
If you wish to stay on the pre-cross revision, you need to set
"sync_options = --revision=pre-cross" in the repository's configuration
file.

If you do migrate (which you should), you'll find that some packages
might not yet install. As always, you're fully expected to help with
migrating exheres to the new base system which basically involves
undoing our former multi-lib approach. You'll find more details at [3]
and you're, of course, encouraged to look at recent commits that mention
"multiarch" or "cross" for tips on converting other packages.

Furthermore, we have no control over 3rd-party/unofficial repositories.
While quite a few of them feature a "cross" branch right now, they might
not have been merged when the official repositories were.
Thus, you might have to add "sync_options = --branch=cross" (without the
quotation marks) to the respective repository's configuration until the
maintainer has merged the cross branch into master.

[1]: http://exherbo.org/docs/multiarch.txt
[2]: http://exherbo.org/docs/multiarch.html
[3]: http://exherbo.org/docs/cross.html

Best regards, Wulf

Wulf C. Krueger | 5 Apr 11:44 2015

Cross/multi-arch: Preparations for the merge


Hello,

this is the plan for today:

I'm currently doing one last migration of a VM to check the validity
of our migration guide.

If all is well, I'm going to tag all official repositories' last
"master" commit with "pre-cross".
Do me a favour and DO NOT commit to "master" anymore today before the
merge is done.

Once the tagging is done, we start merging all cross branches to master.

Did I miss anything?

--

-- 
Best regards, Wulf
Wulf C. Krueger | 4 Apr 11:50 2015

Cross/multi-arch: Get going! Merge date: TOMORROW, Sunday, April 5th 2015


Hello,

I'd like to remind everyone that we are going to merge all the cross
branches to their respective master branches

          TOMORROW, Sunday, April 5th 2015!

Whatever you're doing Exherbo-wise, please DROP IT for the moment and
help us get more packages ready!

Migrating exheres to the new base system isn't (most of the time)
complicated: It basically involves undoing our former multi-lib
approach. You'll find more details at [1] and you're, of course,
encouraged to look at recent commits that mention "multiarch" or
"cross" for tips on converting other packages.

You may, of course, test cross-compilation to something but what's
REALLY important is that the package works on a NATIVE system, e. g.
boost does work natively but fails at cross-compilation currently. For
now, that's ok.

If you want to convert packages in a third-party repository that
doesn't have a cross branch yet, please get in touch with me - I can
create cross branches for EVERY repository in Gerrit.

To get started:

Fetch a current stage
Migrate it to cross/multi-arch (cf. [2])
Start converting!

So, go for it!

(And always remember: My statistics will record your fabulous work for
generations to come who will sing the praise of the immortal heroes
who made cross work! ;-) )

(Questions? -> #exherbo or this list)

[1]: http://exherbo.org/docs/cross.html
[2]: http://exherbo.org/docs/multiarch.html

--

-- 
Best regards, Wulf

Gmane