DRMAA F2F meeting in Potsdam

Hi,

as agreed on the list a while ago, we try to organize a last F2F meeting 
before the submission of DRMAAv2 at the end of June. I will host the 
meeting at HPI in Potsdam, here is the Doodle poll:

http://www.doodle.com/628iy242tt54r283

Please provide your opinion until Friday.

Thanks and best regards,
Peter.

Re: Conference call - June 1th - 19:00 UTC

Hi,

2011/5/31 Peter Tröger <peter <at> troeger.eu>:
> Dear all,
>
> the next DRMAA conf call is scheduled for June 1th, 19:00 UTC. We meet on
> Skype, please find me under my user name "potsdam_pit".
>
> Preliminary meeting agenda:
>
> 1. Meeting secretary for this meeting?
> 2. DRMAAv2 Draft 5 (see attachment)
>
> Best regards,
> Peter.
>

a new spreadsheet tab wich tries to summarize how different resource
limits are handled in GE/LSF/Torque:

https://spreadsheets.google.com/spreadsheet/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE&hl=en_US#gid=13

and the proposition of restructuring the section 5.6.25 ( text in
brackets [] == my comment):

5.6.26 resourceLimits  [not hardResourceLimits]

This attribute specifies the limits on resource utilization of the
job(s) on the execution host(s). The valid dictionary keys and their
value semantics are defined in Section 4.3.

Re: Conference call - June 1th - 19:00 UTC

Hi,

2011/5/31 Peter Tröger <peter <at> troeger.eu>:
> Dear all,
>
> the next DRMAA conf call is scheduled for June 1th, 19:00 UTC. We meet on
> Skype, please find me under my user name "potsdam_pit".
>
> Preliminary meeting agenda:
>
> 1. Meeting secretary for this meeting?
> 2. DRMAAv2 Draft 5 (see attachment)
>
> Best regards,
> Peter.
>

a new spreadsheet tab wich tries to summarize how different resource
limits are handled in GE/LSF/Torque:

https://spreadsheets.google.com/spreadsheet/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE&hl=en_US#gid=13

and the proposition of restructuring the section 5.6.25 ( text in
brackets [] == my comment):

5.6.26 resourceLimits  [not hardResourceLimits]

This attribute specifies the limits on resource utilization of the
job(s) on the execution host(s). The valid dictionary keys and their
value semantics are defined in Section 4.3.

Re: DRMAA F2F meeting in Potsdam

We got a result:

The DRMAA face-to-face meeting will take place on Monday, June 20th 2011 in Potsdam. Sorry Thijs ... 

I will be available from 8:30. If you need help with accommodation or traveling, just contact me directly.

Best regards,
Peter Tröger.

Am 01.06.2011 um 22:29 schrieb Peter Tröger:

> Hi,
> 
> as agreed on the list a while ago, we try to organize a last F2F meeting before the submission of DRMAAv2 at
the end of June. I will host the meeting at HPI in Potsdam, here is the Doodle poll:
> 
> http://www.doodle.com/628iy242tt54r283
> 
> Please provide your opinion until Friday.
> 
> Thanks and best regards,
> Peter.
> 

Conference call - June 8th - 19:00 UTC

Dear all,

the next DRMAA conf call is scheduled for June 8th, 19:00 UTC. We meet 
on Skype, please find me under my user name "potsdam_pit".

Preliminary meeting agenda:

1. Meeting secretary for this meeting?
2. DRMAAv2 Draft 5 (contd., see attachment)

Best regards,
Peter.

Dear all,

the next DRMAA conf call is scheduled for June 8th, 19:00 UTC. We meet 
on Skype, please find me under my user name "potsdam_pit".

Preliminary meeting agenda:

1. Meeting secretary for this meeting?
2. DRMAAv2 Draft 5 (contd., see attachment)

Best regards,
Peter.

Meeting minutes - Conference call - June 1th - 19:00 UTC

Participants: Peter, Daniel G. Mariusz, Roger

* JobInfo:slots
	* Line 545: Not necessary from Daniel's perspective
	* Exclusive complex: Possible to book a whole machine with only
	 one slot (e.g. memory amount) - difficult to detect
	* Better to make clear DRMAA semantics, JobInfo:slots should be
	in between the range from the job template
* Line 547:
	* Currently similar to MPI approach of host file
	* Might have large memory footprint (more slots)
	* Alternative with additional struct
	* Only necessary for reporting
	* Use case: Cluster monitoring, generate MPI machine file
	based on this information
	* Decision: Introduce new structure with machine name and
	slot count
	* Decision: Remove optional sentence
* Complete IDL list in draft 5 lacks DrmaaCapability structure
* Line 899: remove optional
* Line 933: Should be clarified that it is intended to fill
   out templates and structs
	* Set and get give impression that they are intended for
	DrmaaReflective interface attributes itself
* Line 735:
	* make it mandatory -> AR might not be implemented, but AR
	created outside should be supported then
	* InvalidValue as generic value feasible
* Research on resource limits
	* Line 244: DATA_SEG_SIZE has no use case, but we only take out

Re: Conference call - June 1th - 19:00 UTC

I like the proposal, makes sense to me.

Best regards,
Peter.

Am 01.06.11 23:16, schrieb Mariusz Mamoński:
> Hi,
> a new spreadsheet tab wich tries to summarize how different resource
> limits are handled in GE/LSF/Torque:
>
> https://spreadsheets.google.com/spreadsheet/ccc?key=0AqyvnBscJNqxcnJBSUs5dXRrU29EUVhGOGthc1lDTFE&hl=en_US#gid=13
>
> and the proposition of restructuring the section 5.6.25 ( text in
> brackets [] == my comment):
>
>
>
> 5.6.26 resourceLimits  [not hardResourceLimits]
>
> This attribute specifies the limits on resource utilization of the
> job(s) on the execution host(s). The valid dictionary keys and their
> value semantics are defined in Section 4.3.
>
> The CORE_FILE_SIZE, DATA_SEG_SIZE, FILE_SIZE, OPEN_FILES, STACK_SIZE,
> VIRTUAL_MEMORY limits SHOULD be implemented as the soft resource
> limits. An implementation MAY map them to an setrlimit call in the
> operating system. [I think the actual usecase for those resources is
> to increase the system default limit rather than actually limit the
> application]
>

Meeting minutes - conference call - June 8th - 19:00 UTC

Participants: Roger, Mariusz, Daniel G., Peter

- no conf call next week, F2F meeting in Potsdam on June 20th
- "Last call" immediately after successful F2F meeting
- Submission at the end of June, as planned

Line 553:
	- What is the advantage of knowing this ?
	- Mostly too late or too early, timing issues
	- Roger: Better to have this as additional information
		 on suspended state
	- Decision: Move text to another place as implementation
	suggestion
	- user should be somehow notified
	  if the job fails / remains suspendend due to
	resource unavailability
	- Decision: remove attribute

Line 932:
	- Decision: Use InvalidArgumentException
	
Line 963:
	- Does not help in finding the problem, just tells you
	which function you have used
	- Known cases: limit on number of parallel sessions, out
	of disk space for session information
	- Decision: Replace OutOfMemoryException by
	OutOfResourceException (memory, disk space), use this one
	instead of SessionManagementException

IDL issues

Hia folx,

I stumbled over a couple of issues in the IDL spec, and would
appreciate clarification / feedback.  Some are minor, some are major.
The bigger ones are likely caused by my limited understanding of the
background which led to the design, so please bear with me - I don't
want to reopen any discussions which have been closed for good...

  - the enum ResourceLimitType does not seem to be used anywhere

  - the enum JobTemplatePlaceholder does not seem to be used anywhere

  - I think the following is asymmetric.  There is likely a reason,
but I am not sure I understand it:

    - job     session: create (name, contact), open (name), close
(session), destroy (name)
    - reserv. session: create (name, contact), open (name), close
(session), destroy (name)
    - monit.  session: create (      contact),              close (session)

    From the above, it seems like create/close are pairs?  I would
naively expect the following
    pairs: open/close and create/destroy, as usual - what is the rationale?

    Why is the Monitoring session handled differently, i.e. has no
name/open/destroy?

  - jobInfo.exitStatus is a long.  Shouldn't that be an int?  Or is
that an IDL artifact?

DRMAA2 Draft 6, next steps, no conf call

Dear all,

after a very productive face-to-face meeting in Potsdam, we ended up 
with the new draft 6 of the DRMAAv2 spec. Please find attached the 
document. I would like to thank Mariusz, Daniel G. and Andre Merczy for 
investing their time and effort.

The good news is that we were able to clarify all pending functional 
issues. We are now in a sanity check phase, were the text itself gets 
some proof-reading to find inconsistencies.

Since at least three group members are now into reading and editing, I 
will drop the call for this week. If no serious (I mean *really* 
serious) things are found, we will wrap-up in a couple of days and 
perform the official "last call" for comments on the list.

Beside that, we started some initial debate on the C binding. Please 
understand that this discussion will go public only after the IDL spec 
was submitted, in order to avoid redundant efforts.

Best regards,
Peter.

Dear all,

after a very productive face-to-face meeting in Potsdam, we ended up 
with the new draft 6 of the DRMAAv2 spec. Please find attached the