Weekly Python Bug/Patch Summary

Re: Minimal GCC/Linux shared lib + EH bug example

----- Original Message -----
From: "Jason Merrill" <jason <at> redhat.com>

> IMO, it is unreasonable to expect C++ to work with RTLD_LOCAL unless the
> object so loaded is indeed self-contained (which precludes linking
against
> a common shared library, as in this case).  Too many aspects of the
> language depend on being able to merge duplicates coming from different
> sources.

I think there's an implicit assumption in your statement which should be
brought into the open: that there's an agreed-upon idea of what it means
for C++ to "work" with shared libraries. As you know, the language standard
doesn't define how share libs are supposed to work, and people have
different mental models. For example, on Windows, imports and exports are
explicitly declared. Nobody expects to share static variables in inline
functions across DLLs unless the function is explicitly exported. However,
exception-handling and RTTI /are/ expected to work.

> In this case, the problem comes from std::type_info; the runtime
> library expects to be able to compare type_info nodes by pointer
> equivalence.  Templates and static variables in inline functions would
also
> have trouble.

As I understand (guess) it, what happens is this:

1. lib1.so is loaded with RTLD_LOCAL. All of its symbols go into a new
"symbol space"
2. the loader notices the dependency on X.so, and loads any /new/ symbols

Re: Minimal GCC/Linux shared lib + EH bug example

>>>>> "David" == David Abrahams <david.abrahams <at> rcn.com> writes:

> I think there's an implicit assumption in your statement which should be
> brought into the open: that there's an agreed-upon idea of what it means
> for C++ to "work" with shared libraries. As you know, the language standard
> doesn't define how share libs are supposed to work, and people have
> different mental models. For example, on Windows, imports and exports are
> explicitly declared. Nobody expects to share static variables in inline
> functions across DLLs unless the function is explicitly exported. However,
> exception-handling and RTTI /are/ expected to work.

And on Windows, we don't rely on address equivalence.

> What I'd prefer to happen is that in step 4, the loader would use the
> existing definition for any loaded symbol which is defined in or used by
> lib2's immediate dependencies. That would nicely model the concept that
> lib2.so is sharing globally with X.so but not with lib1.so, and it seems
> like the "right" solution.

I noticed that the readme says that the test passes on Solaris.  Does it
provide these semantics?  How about SCO?  Anyone?

> However, for my application I'd be content if EH was just comparing the
> type_info::name() strings, as Martin von Loewis stated was the case in
> 2.95.x and again in 3.1:
> http://aspn.activestate.com/ASPN/Mail/Message/1191899 [This statement
> appears to be contradicted empirically, though: Ralf reports similar
> problems with GCC 3.1 - GNATS id 6629].

Yes, 3.1 still relies on pointer comparison.

Re: Minimal GCC/Linux shared lib + EH bug example

For the c++std-ext recipients, just added: previous messages in this
thread, which concerns the semantics of shared libraries w.r.t.
exception-handling and other C++ features which require "vague linkage",
can be found here:
http://gcc.gnu.org/ml/gcc/2002-05/msg00866.html
http://gcc.gnu.org/ml/gcc/2002-05/msg00869.html
http://gcc.gnu.org/ml/gcc/2002-05/msg00873.html

----- Original Message -----
From: "Jason Merrill" <jason <at> redhat.com>

> > What I'd prefer to happen is that in step 4, the loader would use the
> > existing definition for any loaded symbol which is defined in or used
by
> > lib2's immediate dependencies. That would nicely model the concept that
> > lib2.so is sharing globally with X.so but not with lib1.so, and it
seems
> > like the "right" solution.
>
> I noticed that the readme says that the test passes on Solaris.  Does it
> provide these semantics?  How about SCO?  Anyone?

The test as written doesn't really tell us the answer since it uses EH and
any implementation can make it a non-issue by comparing type_info::name()
strings instead of addresses. The test could easily be modified the so it
looks at the address of a class template's static data member, of course.

> > However, for my application I'd be content if EH was just comparing the
> > type_info::name() strings, as Martin von Loewis stated was the case in
> > 2.95.x and again in 3.1:

Re: Minimal GCC/Linux shared lib + EH bug example

>>>>> "David" == David Abrahams <david.abrahams <at> rcn.com> writes:

>> I noticed that the readme says that the test passes on Solaris.  Does it
>> provide these semantics?  How about SCO?  Anyone?

> The test as written doesn't really tell us the answer since it uses EH and
> any implementation can make it a non-issue by comparing type_info::name()
> strings instead of addresses.

I meant using gcc 3.0.4 on Solaris.

Jason

Re: Minimal GCC/Linux shared lib + EH bug example

Jason, I had to write all the following exposition to understand your
reply, but then it dawned on me what you meant 

I wrote, describing existing Linux/GCC semantics:
> >>> 4. lib2.so is loaded with RTLD_LOCAL. Because it's RTLD_LOCAL, the
loader
> >>> again creates a new "symbol space"; no duplicates are shared with
X.so.
<snip>

And then, describing my preferred semantics:
> >>> What I'd prefer to happen is that in step 4, the loader would use the
> >>> existing definition for any loaded symbol which is defined in or used
by
> >>> lib2's immediate dependencies. That would nicely model the concept
that
> >>> lib2.so is sharing globally with X.so but not with lib1.so, and it
seems
> >>> like the "right" solution.

Jason replied:
> >> I noticed that the readme says that the test passes on Solaris.  Does
it
> >> provide these semantics?  How about SCO?  Anyone?

Assuming by "these semantics", Jason meant my preferred semantics:
> > The test as written doesn't really tell us the answer since it uses EH
and
> > any implementation can make it a non-issue by comparing
type_info::name()

Re: Minimal GCC/Linux shared lib + EH bug example

> I find this testcase somewhat persuasive, as the offending dlopen call is
> not in the C++ code.  What do others think?

I agree with your other statement: RTLD_LOCAL and C++ don't really make
sense.

I think we're running down a slippery slope; once EH works, people
*will* wonder why things involving inlines and templates don't.

If, for example, you have *two* Python modules in C++, each of which
uses a nice package for managing global resources, and you can load
either module just fine, but loading both causes subtle runtime
problems, ...

We will have given people a bigger bazooka, but it will be aimed at
their own feet.

--

-- 
Mark Mitchell                mark <at> codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com

Adding Optik to the standard library

I think it's time to declare the work of the getopt-sig finished:
several competing proposals were put forward, but Optik appears to be
the only really complete, documented, field-tested (by someone other
than its author) library.  Not everyone on the getopt-sig will agree,
but I think that's the broad consensus.  Take this with a grain of salt,
though -- I'm biased.  

Anyways, I think further consensus is needed on how precisely to add
Optik to the standard library.  The only constraint I've heard from
Guido is to give it a less-cutesy name, which is fine by me.

First, background: Optik consists of three modules: optik.option,
optik.option_parser, and optik.errors, but that detail is hidden from
users -- Optik applications normally just do this:
  from optik import OptionParser
although there are a handful of other names that are occasionally useful
to import from the 'optik' package: Option, SUPPRESS_HELP,
OptionValueError, etc.  Optik's __init__.py file is here:
  http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/optik/optik/lib/__init__.py?rev=1.11&content-type=text/plain

It's only about 1100 lines, including docs and comments and blanks, so
they could easily be merged into one module if people think that's
preferable.

So the main issues are 1) where those names should be imported from
(interface), and 2) how the standard library should be rearranged to
make this interface unobtrusive and efficient (implementation).

I'm going to toss out ideas at random until I get bored.  Please provide
feedback and/or extra ideas on getopt-sig <at> python.org.

RE: [getopt-sig] Adding Optik to the standard library

[Greg Ward]
> 
> Anyways, I think further consensus is needed on how precisely to add
> Optik to the standard library.  The only constraint I've heard from
> Guido is to give it a less-cutesy name, which is fine by me.

Any chance we could wean you off your preference for underscores? <wink>

---
Patrick K. O'Brien
Orbtech

Re: [getopt-sig] Adding Optik to the standard library

I think it's better to pick a new name and leave the existing getopt
module alone.

I think keeping it a package is fine.  I prefer to have little or no
magic in __init__.py though (the email package's __init__.py is
borderline .

I think that "options" is a fine package name.  Yes, there are other
things that one could consider options.  No, I don't think that will
cause confusion.  After all "getopt" isn't much more specific. 

--Guido van Rossum (home page: http://www.python.org/~guido/)