Re: conf call minutes - Jan. 6th 2010

> I didn't noticed that I agreed to the Job/String duality.  The point
> about dealing with tens of thousands of jobs as Job objects being
> difficult may be true, but I'm not sure I like the idea of handling  
> them
> as String objects instead.  A better approach might be to offer some
> kind of aggregate object, like those silly opaque iterators from the C
> binding.
>
> I guess we need to look at use cases.  The only reason I can see that
> you'd want to wait for 10k jobs and actually look at the returned list
> of jobs, is if you have a list of jobs you launched, and you're  
> waiting
> for them to finish.  Since the job ids are opaque, you can't shortcut
> the job ids with math.  In order to keep a list of 10k jobs, even as
> String objects, you're going to have to take a clever approach.  If

Good point. From my knowledge, only scripting languages such as Python  
treat string as scalar data type. I still like the duality approach,  
but the reasoning is a little bit weak. We need to discuss that again.

> that you can find out the names of the jobs.  Heck, maybe job name
> should be part of the Job object directly.  In that kind of situation,
> an iterator would be exactly what you want, and it could be smart  
> enough
> not to keep all the Job objects alive at once.

Iterators are no concept in DRMAA so far, which is good. Mapping  
everything to C (in a nice way !) is already painful enough ....

Best,

Conference call - Feb 3rd - 20:00 UTC

Dear all,

the next DRMAA phone conference is scheduled on Feb 3rd at 20:00 UTC.

For time zone conversion, check this:

http://timeanddate.com/s/1jwg

The phone conference line is sponsored by Sun - aehm, by Oracle:

Phone number (toll-free from US): +001-866-545-5227
Access code: 5988285

Preliminary meeting agenda:

1. Meeting secretary for this meeting?
2. Introduction of DRMAAv2 Wiki (http://wikis.sun.com/display/DRMAAv2/Home 
)
3. Finalization of Job object / string duality discussion
4. Going forward with the open issues

Best regards,
Peter.

--
drmaa-wg mailing list
drmaa-wg@...
http://www.ogf.org/mailman/listinfo/drmaa-wg

Re: Conference call - Feb 3rd - 20:00 UTC

Attendees: Peter, Dan, Roger, Mariusz, Daniel G.

Meeting secretary: Daniel G.

2.) WIKI introduction 
-> make comments!
-> please ask for editing rights when you want 
   to work something out
-> in future calls we go through all comments
-> be aware of the red thread safety blocks (put 
   in here your implementation experiences/problems 
   related to multi-threading)

3.) Job object / String duality 

getJobs() -> returns Job object or String?

- performance is in theoretical complexity the same
1. Sequence of job objects (is in IDL an arbitrary construct), 
  language bindings can find a smart way?
2. Filter approach (implies filtering language? too much overhead).
  -> Agreed solution: Filtering with templates!

4.) Open Issues

File staging (like LSF/Condor/PBS) explicit

- DRMAA ignores security problems!
- File staging as an optional attribute in JobTemplate

OGF28 planning

Hi,

the OGF28 schedule is online:

http://www.ogf.org/gf/event_schedule/index.php?event_id=15

We have sessions on March 17th (Wed) and March 18th (Thur). Due to  
critical job activities, I will not make it for the public DRMAAv2  
presentation on Wednesday.

Who can take care of this ?

I will help with slides and all necessary information, so it is  
basically about celebrating a good show for the audience. For the  
internal / cross-group discussion on Thursday, I will be there.

Thanks,
Peter.

Re: OGF28 planning

It's looking like Daniel might be our only representative who can be 
there, unless we want to punt to one of our compatriots from SAGA.

Daniel

On 02/09/10 13:58, Peter Tröger wrote:
> Hi,
>
> the OGF28 schedule is online:
>
> http://www.ogf.org/gf/event_schedule/index.php?event_id=15
>
> We have sessions on March 17th (Wed) and March 18th (Thur). Due to
> critical job activities, I will not make it for the public DRMAAv2
> presentation on Wednesday.
>
> Who can take care of this ?
>
> I will help with slides and all necessary information, so it is
> basically about celebrating a good show for the audience. For the
> internal / cross-group discussion on Thursday, I will be there.
>
> Thanks,
> Peter.
> --
>    drmaa-wg mailing list
>    drmaa-wg@...
>    http://www.ogf.org/mailman/listinfo/drmaa-wg
>    

Re: OGF28 planning

Quoting [Daniel Templeton] (Feb 09 2010):
> 
> It's looking like Daniel might be our only representative who can be 
> there, unless we want to punt to one of our compatriots from SAGA.

Sure, we can help out and give a talk to convert all DRMAA customers
to SAGA 

But really: if needed we can certainly help out, but I guess Dan
would do a much better job at representing DRMAA than we could...

Anyway, just let us know, in case you become desparate 

Best, Andre.

> Daniel
> 
> On 02/09/10 13:58, Peter Tröger wrote:
> > Hi,
> >
> > the OGF28 schedule is online:
> >
> > http://www.ogf.org/gf/event_schedule/index.php?event_id=15
> >
> > We have sessions on March 17th (Wed) and March 18th (Thur). Due to
> > critical job activities, I will not make it for the public DRMAAv2
> > presentation on Wednesday.
> >
> > Who can take care of this ?
> >

Re: OGF28 planning

Hi,

>> It's looking like Daniel might be our only representative who can be 
>> there, unless we want to punt to one of our compatriots from SAGA.

I managed to move my job activity away, so I will be also there on Wednesday. 
It still would be nice to not only have SAGA people in the room 

Best,
Peter.

> 
> Sure, we can help out and give a talk to convert all DRMAA customers
> to SAGA 
> 
> But really: if needed we can certainly help out, but I guess Dan
> would do a much better job at representing DRMAA than we could...
> 
> Anyway, just let us know, in case you become desparate 
> 
> Best, Andre.
> 
> 
> 
>> Daniel
>> 
>> On 02/09/10 13:58, Peter Tröger wrote:
>>> Hi,
>>> 
>>> the OGF28 schedule is online:

Latest changes

Hi,

the advanced reservation proposal from Mariusz is now part of the DRMAAv2 spec draft:

http://wikis.sun.com/display/DRMAAv2/DRMAAv2+API

I only omitted the separate ReservationInfo interface - the machine specific part (such as used slot
count) became part of the (anyway existing) MachineMonitoring interface.

I also added the new file staging attribute, and (even more important) deleted the crappy DRMAAv1 file
transfer idea. So long, FileTransferMode enumeration ...

Best,
Peter.

Hi,

the advanced reservation proposal from Mariusz is now part of the DRMAAv2 spec draft:

http://wikis.sun.com/display/DRMAAv2/DRMAAv2+API

I only omitted the separate ReservationInfo interface - the machine specific part (such as used slot
count) became part of the (anyway existing) MachineMonitoring interface.

I also added the new file staging attribute, and (even more important) deleted the crappy DRMAAv1 file
transfer idea. So long, FileTransferMode enumeration ...

Work distribution

Dear core members,

in order to have any chance for some kind of finalized API until OGF28, we need to distribute the work now. I
need volunteers to independently take care of the following topics. We mainly talk about man page
crawling and text snippet writing:

--- snip ----

New job template attributes. 

This includes the continuation of DRM system comparison we already started in the Google spreadsheet. It
must also include the hunt for new job template placeholders, since this was one of the major wishes from
the survey.

File staging. 

We need the exhaustive description of how the new fileTransfers attribute should be used. Order of
activities and security might be topics here. Ideally, somebody with a GridFTP background can also
contribute. The initial idea was to copy from SAGA / LSF, so if you already know at least one of these
systems, please volunteer.

Advanced reservation. 

The API is in the IDL part, but we still need the detailed functions descriptions. Mariusz, could you take
care of this ?

Job states. 

There is a set of wishes regarding more job states and transitions. We also have a pending mapping to other
peoples job stage models.

Re: Work distribution

Hi Peter, all,

2010/2/15 Peter Tröger <peter <at> troeger.eu>:
> Dear core members,
>
> in order to have any chance for some kind of finalized API until OGF28, we need to distribute the work now. I
need volunteers to independently take care of the following topics. We mainly talk about man page
crawling and text snippet writing:
>
> --- snip ----
>
> New job template attributes.
>
> This includes the continuation of DRM system comparison we already started in the Google spreadsheet. It
must also include the hunt for new job template placeholders, since this was one of the major wishes from
the survey.
>
> File staging.
>
> We need the exhaustive description of how the new fileTransfers attribute should be used. Order of
activities and security might be topics here. Ideally, somebody with a GridFTP background can also
contribute. The initial idea was to copy from SAGA / LSF, so if you already know at least one of these
systems, please volunteer.
>
I did some "research" of the file staging capabilities  in the LSF,
Torque and the PBS Pro systems (you can find a short summary in the
Google Spreadsheet - the "File Staging" tab).

Some thoughts:
- most of the batch systems were designed to work with network shared