Robert Bradshaw | 21 Mar 20:06 2012
Picon

Re: [Cython] Jenkins down

On Wed, Mar 21, 2012 at 8:58 AM, Stefan Behnel <stefan_ml@...> wrote:
> mark florisson, 21.03.2012 15:12:
>> On 21 March 2012 15:07, mark florisson wrote:
>>> On 21 March 2012 14:58, Stefan Behnel wrote:
>>>> it seems like the sage.math server is broken again, at least it's lacking
>>>> mounts when I try to log in. I'll restart Jenkins as soon as it's back working.
>>>
>>> Indeed, it's rather worrying and is actually a bit of a slowdown for
>>> development.
>
> Sure. However, we get to use it for free, and we've had a good gain from
> that for a long time now. It's just bad timing that this coincides with our
> release preparations.

I'd say that since I've started using it sage.math has averaged close
to 99% uptime, it's just bad timing at the moment.

>>> Dag and I discussed getting some funding from numfocus to
>>> use Jenkins in a cloud environment (along possibly with other
>>> projects, like sage, lxml, numpy, scipy, depending on interest of
>>> respective projects). At PyCon we saw one such company that provides
>>> these services, which is shiningpanda.com
>
> Their free OSS plan is one hour per day, which is way too short for us. Not
> sure how much we'd want to pay for their dedicated plan, and no idea what
> performance we'd get for the money.
>
> For our current setup, you can count way more than one hour of processing
> time for each single push and for each day's run of the CPython triggered
> integration tests. And then there's our set of separate developer branches
(Continue reading)

Robert Bradshaw | 21 Mar 21:24 2012
Picon

Re: [Cython] some pull requests for 0.16?

On Wed, Mar 21, 2012 at 6:47 AM, mark florisson
<markflorisson88@...> wrote:
> On 20 March 2012 18:51, Stefan Behnel <stefan_ml@...> wrote:
>> mark florisson, 20.03.2012 17:40:
>>> On 18 March 2012 11:58, Stefan Behnel wrote:
>>>> I put up two new pull requests on github:
>>>>
>>>> Implementation of PEP 380 (yield from):
>>>> https://github.com/cython/cython/pull/96
>>>>
>>>> Rewrite of dict iteration:
>>>> https://github.com/cython/cython/pull/95
>>>>
>>>> Given that the release of 0.16 has currently slowed down a bit, and given
>>>> that these are really nice features, could someone (and especially Mark, as
>>>> the responsible release manager) take a look at them and give an opinion
>>>> regarding their suitability for 0.16?
>>>
>>> Sorry about the slowdown, my BT internet connection at home got closed
>>> off, so working on a release is rather hard, hopefully that will be
>>> fixed within the next few days.

Thanks for the status update. Is there a separate release branch, or
are you working off the github master? Is there a list of
issues/blockers somewhere?

>>> Anyway, during the sprints Dag worked
>>> on the numpy attributes accesses to generate non-deprecated numpy
>>> code, I'm not sure how far along that is and if we can get that in as
>>> well, or have it wait until 0.16.1.
(Continue reading)

Robert Bradshaw | 21 Mar 21:56 2012
Picon

Re: [Cython] [cython-users] GSoC 2012

On Sat, Mar 10, 2012 at 10:44 PM, mark florisson
<markflorisson88@...> wrote:
> On 8 March 2012 14:27, Stefan Behnel <stefan_ml@...> wrote:
>> Hi,
>>
>> I noticed that people start rushing for the next season on Python's GSoC
>> mailing lists. Do we have any interested developers here, or general ideas
>> about suitable topics? I would expect that we'll do as in the last years
>> and participate under Python's umbrella.

Also, we'd like to see patches from anyone interesting in being a GSoC
student, as this will be a requirement as in past years.

> I will likely be submitting a proposal for the OpenCL support CEP.

OpenCL would be an interesting experiment, but I think still has
limited utility. Dag and I were talking the other day about the
challenge of generating the best possible code for evaluating array
expressions (think inlined memoryview arithmetic) taking into account
memory layout, blocking, etc. which Fortran does really well which
could be an interesting direction.

> As for other proposals, anyone can come up with something themselves or
> choose a suitable CEP. Some ideas:

Numbering items for clarity:

1.

> - fused cdef classes (probably not as an entire gsoc project)
(Continue reading)

Dag Sverre Seljebotn | 21 Mar 22:11 2012
Picon
Picon

Re: [Cython] [cython-users] GSoC 2012

On 03/21/2012 01:56 PM, Robert Bradshaw wrote:
> On Sat, Mar 10, 2012 at 10:44 PM, mark florisson
> <markflorisson88@...>  wrote:
>> On 8 March 2012 14:27, Stefan Behnel<stefan_ml@...>  wrote:
>>> Hi,
>>>
>>> I noticed that people start rushing for the next season on Python's GSoC
>>> mailing lists. Do we have any interested developers here, or general ideas
>>> about suitable topics? I would expect that we'll do as in the last years
>>> and participate under Python's umbrella.
>
> Also, we'd like to see patches from anyone interesting in being a GSoC
> student, as this will be a requirement as in past years.
>
>> I will likely be submitting a proposal for the OpenCL support CEP.
>
> OpenCL would be an interesting experiment, but I think still has
> limited utility. Dag and I were talking the other day about the
> challenge of generating the best possible code for evaluating array
> expressions (think inlined memoryview arithmetic) taking into account
> memory layout, blocking, etc. which Fortran does really well which
> could be an interesting direction.

Yes, a lot of water has run in the river since March 8 here. Anyone 
interested in reading up on current ideas on what Mark is thinking for 
his GSoC proposal should read up on the numpy-discussion thread "Looking 
for people interested in helping with Python compiler to LLVM".

Dag

(Continue reading)

Robert Bradshaw | 21 Mar 22:39 2012
Picon

Re: [Cython] [cython-users] GSoC 2012

On Wed, Mar 21, 2012 at 2:11 PM, Dag Sverre Seljebotn
<d.s.seljebotn <at> astro.uio.no> wrote:
> On 03/21/2012 01:56 PM, Robert Bradshaw wrote:
>>
>> On Sat, Mar 10, 2012 at 10:44 PM, mark florisson
>> <markflorisson88 <at> gmail.com>  wrote:
>>>
>>> On 8 March 2012 14:27, Stefan Behnel<stefan_ml <at> behnel.de>  wrote:
>>>>
>>>> Hi,
>>>>
>>>> I noticed that people start rushing for the next season on Python's GSoC
>>>> mailing lists. Do we have any interested developers here, or general
>>>> ideas
>>>> about suitable topics? I would expect that we'll do as in the last years
>>>> and participate under Python's umbrella.
>>
>>
>> Also, we'd like to see patches from anyone interesting in being a GSoC
>> student, as this will be a requirement as in past years.
>>
>>> I will likely be submitting a proposal for the OpenCL support CEP.
>>
>>
>> OpenCL would be an interesting experiment, but I think still has
>> limited utility. Dag and I were talking the other day about the
>> challenge of generating the best possible code for evaluating array
>> expressions (think inlined memoryview arithmetic) taking into account
>> memory layout, blocking, etc. which Fortran does really well which
>> could be an interesting direction.
(Continue reading)

Stefan Behnel | 22 Mar 08:18 2012
Picon

Re: [Cython] some pull requests for 0.16?

Robert Bradshaw, 21.03.2012 21:24:
> On Wed, Mar 21, 2012 at 6:47 AM, mark florisson wrote:
>> On 20 March 2012 18:51, Stefan Behnel wrote:
>>> mark florisson, 20.03.2012 17:40:
>>>> On 18 March 2012 11:58, Stefan Behnel wrote:
>>>>> I put up two new pull requests on github:
>>>>>
>>>>> Implementation of PEP 380 (yield from):
>>>>> https://github.com/cython/cython/pull/96
>>>>>
>>>>> Rewrite of dict iteration:
>>>>> https://github.com/cython/cython/pull/95
>>>>>
>>>>> Given that the release of 0.16 has currently slowed down a bit, and given
>>>>> that these are really nice features, could someone (and especially Mark, as
>>>>> the responsible release manager) take a look at them and give an opinion
>>>>> regarding their suitability for 0.16?
>>>>
>>>> Sorry about the slowdown, my BT internet connection at home got closed
>>>> off, so working on a release is rather hard, hopefully that will be
>>>> fixed within the next few days.
> 
> Thanks for the status update. Is there a separate release branch, or
> are you working off the github master?

Not currently. I also agree that we should try to get out the release soon.
What about this: I merge my pending pull requests, we test everything in
Sage (it's been normally tested in my dev branch already), and if that
doesn't break anything, we release another beta? If it does break anything,
it's either easy enough to fix, or we can create a release branch at that
(Continue reading)

Stefan Behnel | 22 Mar 08:34 2012
Picon

Re: [Cython] [cython-users] GSoC 2012

Robert Bradshaw, 21.03.2012 21:56:
> On Sat, Mar 10, 2012 at 10:44 PM, mark florisson wrote:
> 2.
>> - profile guided optimizations (using python's profilers and/or a
>> custom profiler that collects data such as types etc, which can be
>> used to specialize variables inside loops (or entire functions) with a
>> fallback to normal mode in case the type changes)
> 4.
>> - better type inference, that would be enabled by default and again
>> handle thing like reassignments of variables and fallbacks to the
>> default object type. With entry caching Cython could build a database
>> of types ((extension) classes, functions, variables) used in the
>> modules and functions that are compiled (also def functions), and
>> infer the types used and specialize on those. Maybe a switch should be
>> added to cython to handle circular dependencies, or maybe with the
>> distutils preprocessing it can run all the type inference first and
>> keep track of unresolved entries, and try to fill those in after
>> building the database. For bonus points the user can be allowed to
>> write plugins to aid the process.
> 
> 
> What I would most like to see is the common component in 2 and 4, i.e.
> the ability to generate optimized code for imperfectly-inferred types,
> with transparent fallback to generic code if conditions are not met at
> runtime (during as well as at entering the optimized code path).

Absolutely. The dict iteration change is pointing exactly in that
direction, but there are so many other places where the same thing applies.

Basically, it has separate C implementations for different cases all folded
(Continue reading)

mark florisson | 22 Mar 13:40 2012
Picon

Re: [Cython] some pull requests for 0.16?

On 22 March 2012 07:18, Stefan Behnel <stefan_ml@...> wrote:
> Robert Bradshaw, 21.03.2012 21:24:
>> On Wed, Mar 21, 2012 at 6:47 AM, mark florisson wrote:
>>> On 20 March 2012 18:51, Stefan Behnel wrote:
>>>> mark florisson, 20.03.2012 17:40:
>>>>> On 18 March 2012 11:58, Stefan Behnel wrote:
>>>>>> I put up two new pull requests on github:
>>>>>>
>>>>>> Implementation of PEP 380 (yield from):
>>>>>> https://github.com/cython/cython/pull/96
>>>>>>
>>>>>> Rewrite of dict iteration:
>>>>>> https://github.com/cython/cython/pull/95
>>>>>>
>>>>>> Given that the release of 0.16 has currently slowed down a bit, and given
>>>>>> that these are really nice features, could someone (and especially Mark, as
>>>>>> the responsible release manager) take a look at them and give an opinion
>>>>>> regarding their suitability for 0.16?
>>>>>
>>>>> Sorry about the slowdown, my BT internet connection at home got closed
>>>>> off, so working on a release is rather hard, hopefully that will be
>>>>> fixed within the next few days.
>>
>> Thanks for the status update. Is there a separate release branch, or
>> are you working off the github master?
>
> Not currently. I also agree that we should try to get out the release soon.
> What about this: I merge my pending pull requests, we test everything in
> Sage (it's been normally tested in my dev branch already), and if that
> doesn't break anything, we release another beta? If it does break anything,
(Continue reading)

mark florisson | 22 Mar 13:46 2012
Picon

Re: [Cython] Jenkins down

On 21 March 2012 13:58, Stefan Behnel <stefan_ml@...> wrote:
> Hi,
>
> it seems like the sage.math server is broken again, at least it's lacking
> mounts when I try to log in. I'll restart Jenkins as soon as it's back working.
>
> Stefan
> _______________________________________________
> cython-devel mailing list
> cython-devel@...
> http://mail.python.org/mailman/listinfo/cython-devel

Logging in through ssh seems to work, should I run any of the scripts
from your home directory Stefan? e.g. restart_dead_hudson.sh or
starthudson.sh?
William Stein | 22 Mar 14:11 2012
Picon

Re: [Cython] sage.math problems?

On Thu, Mar 22, 2012 at 9:04 AM, David Kirkby <david.kirkby@...> wrote:
> On 22 March 2012 12:33, William Stein <wstein@...> wrote:
>> The usb device (for /levi) never appeared after I remotely rebooted sage.math.
>> Physical access is thus probably required.  I've emailed some people who are in
>> Seattle, and we'll see.  (I'm thousands of miles away now, and will
>> only get further away
>> -- to Europe -- until Apr 2, when I'll be back in Seattle.)
>
> I believe any decent IT professional will tell you that you should not
> be using USB drives on servers.

Well I'm definitely currently not using USB drives on this server. :-(

If possible, can the stuff that was being done on /levi (jenkins for
Cython) be moved to /scratch, which is a reasonably fast NFS
filesystem.

 -- William

Gmane