headers
Neil Toronto | 10 Jan 2008 02:25

Batty idea of the week: no implementation inheritance

It occurred to me that every problem I've had with object-oriented 
programming stems from the fact that subclasses implicitly inherit 
implementation details from their parent classes. It also occurred to me 
that I've never solved a problem with implementation inheritance that 
couldn't be done via composition and delegation approximately as easily.

This kind of delegation would be similar to subclassing:

     class A:
         def foo(self):
             # do something cool

     class B:  # Not a subclass of A, but acts like one
         def __init__(self):
             self.a = A()

         def foo(self):
             # do something before
             ret = self.a.foo()
             # do something after
             return ret
         # or foo = A.foo for simple cases?

For lack of a better term (I'm probably just not well-versed enough in 
language theory) I'll call it reverse delegation. The big UI difference 
between this and subclassing is that here A retains control over its 
implementation and its writer doesn't have to worry about whether 
changing anything will break subclasses - because there aren't any.

There's also forward delegation for when a class doesn't want to care
(Continue reading)

Permalink | Reply | 0 comments |
headers
Aaron Watters | 10 Jan 2008 19:37
Picon

o(n**2) problem with marshal.dumps for large objects, with patch

Hi  folks.  Much to my surprise I found that one of
my applications seemed to be running slow as a result
of marshal.dumps.  I think the culprit is the w_more(...)
function, which grows the marshal buffer in 1k units.
This means that a marshal of size 100k will have 100
reallocations and string copies.  Other parts of Python
(and java etc) have a proportional reallocation strategy
which reallocates a new size based on the existing size.
This mean a 100k marshal requires just 5 or so
reallocations and string copies (n log n versus n**2
asymptotic performance).

I humbly submit the following patch (based on python 2.6a0
source).  I think it is a strict improvement on the existing
code, but I've been wrong before (twice ;)).
   -- Aaron Watters

PATCH FOLLOWS

*** marshal.c.original  2008-01-10 10:15:40.686838800 -0500
--- marshal.c   2008-01-10 11:32:01.838654000 -0500
***************
*** 61,75 ****
  static void
  w_more(int c, WFILE *p)
  {
        Py_ssize_t size, newsize;
        if (p->str == NULL)
                return; /* An error already occurred */
        size = PyString_Size(p->str);
!       newsize = size + 1024;
        if (_PyString_Resize(&p->str, newsize) != 0) {
                p->ptr = p->end = NULL;
        }
        else {
                p->ptr = PyString_AS_STRING((PyStringObject *)p->str) + size;
                p->end =
                        PyString_AS_STRING((PyStringObject *)p->str) + newsize;
--- 61,76 ----
  static void
  w_more(int c, WFILE *p)
  {
        Py_ssize_t size, newsize;
        if (p->str == NULL)
                return; /* An error already occurred */
        size = PyString_Size(p->str);
!       newsize = size + size + 1024;
!       /* printf("new size %d\n", newsize); */
        if (_PyString_Resize(&p->str, newsize) != 0) {
                p->ptr = p->end = NULL;
        }
        else {
                p->ptr = PyString_AS_STRING((PyStringObject *)p->str) + size;
                p->end =
                        PyString_AS_STRING((PyStringObject *)p->str) + newsize;

_______________________________________________
Python-ideas mailing list
Python-ideas@...
http://mail.python.org/mailman/listinfo/python-ideas
Permalink | Reply | 2 comments |
headers
Christian Heimes | 10 Jan 2008 20:04
Picon

Re: o(n**2) problem with marshal.dumps for large objects, with patch

Aaron Watters wrote:
> Hi  folks.  Much to my surprise I found that one of
> my applications seemed to be running slow as a result
> of marshal.dumps.  I think the culprit is the w_more(...)
> function, which grows the marshal buffer in 1k units.
> This means that a marshal of size 100k will have 100
> reallocations and string copies.  Other parts of Python
> (and java etc) have a proportional reallocation strategy
> which reallocates a new size based on the existing size.
> This mean a 100k marshal requires just 5 or so
> reallocations and string copies (n log n versus n**2
> asymptotic performance).
> 
> I humbly submit the following patch (based on python 2.6a0
> source).  I think it is a strict improvement on the existing
> code, but I've been wrong before (twice ;)).

Could you please submit the patch at http://bugs.python.org/ ? Patches
in mailing list posts have a tendency to get lost. An unified diff (diff
-u) is preferred.

Thanks

Christian
Permalink | Reply | 1 comment |
headers
Aaron Brady | 11 Jan 2008 09:54
Picon

Thread object

Hey Group:

I find myself using Threads quite a bit, and I so often write:
	bContinue= True
And in the thread:
	while bContinue:
And elsewhere:
	if ...: bContinue= False

Proposing for the standard library:

	th= threading.Thread( target= myfunction )

	def myfunction( selfth ):
		while selfth.bContinue:
			...

and

	th.bContinue= False.

Of course, there is a little more functionality that's very common as well.
Throw it in, touched up from above:

	def pausesleep( self, secs ):
		'''Pauses until 'stopped' is set, 'unpaused' is set
(currently
		neglects to account for remaining unelapsed seconds),
		or 'secs' seconds elapses.  Set 'stopped' and 'unpaused'
		with Stop(), Pause(), and Resume().  Also, Going() returns
		not stopped.isSet().'''

		self.stopped.wait( secs )
		self.unpaused.wait() #secs- timeout or sth.

Additional; further proposing:
	def Copy( self, runimm= None, name= None ),
	def StopAll(),
	def JoinAll(),
	def PauseAll(),
	def ResumeAll().

However quote the docs: 'group should be None; reserved for future extension
when a ThreadGroup class is implemented', so this may be under way.

Current implementation (read: working version) available and offered to
post.

Thank you and sincerely,
Aaron
Permalink | Reply | 0 comments |
headers
Facundo Batista | 11 Jan 2008 12:51
Picon

Re: o(n**2) problem with marshal.dumps for large objects, with patch

2008/1/10, Christian Heimes <lists@...>:

> Could you please submit the patch at http://bugs.python.org/ ? Patches
> in mailing list posts have a tendency to get lost. An unified diff (diff
> -u) is preferred.

Please, include also an small example that shows this improvement, so
it can actually be measured (some lines of Python code that takes long
at first, but small time after the patch).

Thank you!!

--

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
Permalink | Reply | 0 comments |
headers
Aaron Brady | 11 Jan 2008 08:43
Picon

Thread object

Hey Group:

I find myself using Threads quite a bit, and I so often write:
	bContinue= True
And in the thread:
	while bContinue:
And elsewhere:
	if ...: bContinue= False

Proposing for the standard library:

	th= threading.Thread( target= myfunction )

	def myfunction( selfth ):
		while selfth.bContinue:
			...

and

	th.bContinue= False.

Of course, there is a little more functionality that's very common as well.
Throw it in, touched up from above:

	def pausesleep( self, secs ):
		'''Pauses until 'stopped' is set, 'unpaused' is set
(currently
		neglects to account for remaining unelapsed seconds),
		or 'secs' seconds elapses.  Set 'stopped' and 'unpaused'
		with Stop(), Pause(), and Resume().  Also, Going() returns
		not stopped.isSet().'''

		self.stopped.wait( secs )
		self.unpaused.wait() #secs- timeout or sth.

Additional; further proposing:
	def Copy( self, runimm= None, name= None ),
	def StopAll(),
	def JoinAll(),
	def PauseAll(),
	def ResumeAll().

However quote the docs: 'group should be None; reserved for future extension
when a ThreadGroup class is implemented', so this may be under way.

Current implementation (read: working version) available and offered to
post.

Thank you and sincerely,
Aaron
Permalink | Reply | 0 comments |
headers
Aaron Brady | 12 Jan 2008 21:03
Picon

generic Ref class

For iteration and pass-by-reference semantics:

	class Ref:
		def __init__( self, val):
			self.val= val

To modify list during iteration:

	def refs( iterable ):
		return [ Ref( x ) for x in iterable ]

Then:

	for x in refs( listA ):
		x.val+= 1

Better than:

	for i, x in enumerate( listA ):
		listA[i]= x+ 1

Example is naturally trivial; imagine.  No bloat here; get over it.
Permalink | Reply | 6 comments |
headers
Adam Atlas | 12 Jan 2008 21:16

Re: generic Ref class


On 12 Jan 2008, at 15:03, Aaron Brady wrote:
> Then:
>
> 	for x in refs( listA ):
> 		x.val+= 1
>
> Better than:
>
> 	for i, x in enumerate( listA ):
> 		listA[i]= x+ 1

Don't these do different things? The latter modifies the original  
list, while the former, with your Ref class, apparently modifies (in a  
by-reference sense) a new list that is thrown away once the for loop  
is done.
Permalink | Reply | 2 comments |
headers
Aaron Brady | 12 Jan 2008 21:21
Picon

Re: generic Ref class

> -----Original Message-----
> From: python-ideas-bounces+castironpi=comcast.net@...
>
[mailto:python-ideas-bounces+castironpi=comcast.net@...]
On Behalf
> Of Adam Atlas
> 
> On 12 Jan 2008, at 15:03, Aaron Brady wrote:
> > Then:
> >
> > 	for x in refs( listA ):
> > 		x.val+= 1
> >
> > Better than:
> >
> > 	for i, x in enumerate( listA ):
> > 		listA[i]= x+ 1
> 
> Don't these do different things? The latter modifies the original
> list, while the former, with your Ref class, apparently modifies (in a
> by-reference sense) a new list that is thrown away once the for loop
> is done.

Ah yes.  Say:

	listA= refs( range( 20 ) ) #or your list

then:

	for x in listA:
		x.val+= 1

Slightly slower, but useful in addition to pass to functions too:

	def squareanint( intref ):
		intref.val**= 2

	a= 2
	squareanint( a )
	print a
Permalink | Reply | 1 comment |
headers
Aaron Brady | 12 Jan 2008 21:25
Picon

Re: generic Ref class

> -----Original Message-----
> From: python-ideas-bounces@... [mailto:python-ideas-
> bounces@...] On Behalf Of Aaron Brady
> 
> > -----Original Message-----
> > From: python-ideas-bounces+castironpi=comcast.net@...
> >
[mailto:python-ideas-bounces+castironpi=comcast.net@...] On
> Behalf
> > Of Adam Atlas
> >
> > On 12 Jan 2008, at 15:03, Aaron Brady wrote:
> > > Then:
> > >
> > > 	for x in refs( listA ):
> > > 		x.val+= 1
> > >
> > > Better than:
> > >
> > > 	for i, x in enumerate( listA ):
> > > 		listA[i]= x+ 1
> >
> > Don't these do different things? The latter modifies the original
> > list, while the former, with your Ref class, apparently modifies (in a
> > by-reference sense) a new list that is thrown away once the for loop
> > is done.
> 
> Ah yes.  Say:
> 
> 	listA= refs( range( 20 ) ) #or your list
> 
> then:
> 
> 	for x in listA:
> 		x.val+= 1
> 
> Slightly slower, but useful in addition to pass to functions too:
> 
> 	def squareanint( intref ):
> 		intref.val**= 2
> 
> 	a= 2
> 	squareanint( a )
> 	print a

And once again, I typo:

 	a= 2
 	squareanint( Ref( a ) )
 	print a
Permalink | Reply | 0 comments |

Gmane