Hi Paul and vmeetters,

thank you for submitting a revised version (-07) of your draft. Please find below my comments, preceded by the usual disclaimer about the length of my message .
I'll take some excerpts of the draft and let them be followed by my comments/observations. As a general comment, I personally think that the document mixes technical and non-technical requirements for the RPS and I find this a bit confusing. Technical requirements are those directly coupled to the functionality that the RPS will have to make available and would be easily described (or mapped) with a formal modeling approach like UML, in much the same way as with any other software requirements specification. Non-technical requirements are most of the times related to 'policies', which, as I have often claimed on IETF mailing list discussions, are always orthogonal to (and hence independent from)  technical details of a system. I'll try to make this clear in my detailed review, but for the sake of clarity I'll give you an example of what I'm saying right below:

- technical requirement:
**Requirement 07-02**: All tools in the RPS MUST be able to use both IPv4 and IPv6 addresses natively. - non-technical (policy-based) requirement:

**Requirement 07-07**: Both local and remote attendees SHOULD be able to easily contact a single entity who is available throughout the meeting if they find problems with any of the RPS tools, and to get fairly rapid response. This entity needs to be able to handle as RPS tool problems in the meeting rooms, or be able to quickly contact someone who can address those problems. My personal preference would be to have a clear distinction between the above classes of requirements. I think this would also make it easier, in the future, to edit a formal
software requirements specification for potential bidders.

This said, I also found a third category of requirements which in my view fall in a specific class which I would call something like "Requirements for the seamless integration of the future RPS system with the current IETF tools and meeting materials portal". To provide you an example of this third class of requirements, I'll copy/paste a couple below:
**Requirement 07-49**: The RPS MUST automatically convert PowerPoint presentations to PDF and make both available for distribution at the same time. NOTE: in the requirement below I assume that with the expression "make both available for distribution" you are referring to slides publication on the meeting materials page, right? **Requirement 07-50**: Presenters MUST be able to update their slides on the IETF site up to just before their presentation, if such update is allowed by the WG chairs. NOTE: this last example is in my view actually a mix of a "policy-based" and an "integration" requirement. So, let's get started with the details.
**Requirement 07-09**: The deployment of the tools here MUST take into account making the tools accessible to as many IETF attendees as possible. Such deployment is likely to include technical accommodations for those with visual and hearing disabilities.
 Agreed. But this formulation sounds a bit too generic. What kind of accommodations do you have in mind? As an example, for deaf people, you might be thinking of remote transcription services like "Remote CART" (Communication Access Real-time Translation) captioning, perhaps based on standards like XEP-0301 (Real-Time Text for XMPP -- http://www.xmpp.org/extensions/xep-0301.html). In any case, I would make this part more explicit.

**Requirement 07-10**: All remote attendees at regular IETF meetings and interim meetings who make contributions MUST register with the IETF Secretariat before contributing using any of the RPS tools. This is out of scope for the RPS. I think the basic idea behind section 2.1 is that the IETF secretariat should in the future envisage different registration options (local attendee, remote contributing attendee, remote non-contributing attendee) and the RPS should have access to such information when logging users into the system, so to be capable of associating them with a specific role. In summary, from the RPS's point of view, the only requirement should be that it be capable of supporting different user roles, once again configured through policies. You actually mention something like this when you state, at the end of the section, that "registration might happen in the IETF's Datatracker".

Coming to section 2.3 on Audio, I have already stated on this ML (http://www.ietf.org/mail-archive/web/vmeet/current/msg00548.html) that I strongly believe the priorities you identify in the document are not in the right order. I cannot think of a 'future' RPS which is not capable of reliably supporting audio from remote attendees (both those who have to give presentations and those who just want to make sporadic comments).  I know this brings in some technical obstacles, but I nonetheless insist on this point.
With respect to the specific requirements you identify:

**Requirement 07-20**: The RPS MUST enable relay of messages from IM to the mic to be able to happen as quickly as if the remote attendee was local. It is not clear to me what you mean by "as quickly as if the remote attendee was local".
From a technical standpoint, this should translate in something similar to what I was proposing in the already cited mail, i.e. a solution which automatically announces a request for attention coming, via the jabber room, from a remote: in a few words, if a remote writes a sentence in the chat and lets it be preceded by the [mic] prefix, the RPS should automatically play a predefined 'attention sound' inside the room.

**Requirement 07-21**: The person relaying from IM to the mic must be available throughout the WG meeting. To date, this has been done by WG volunteers in the room. In the future, it could be done the same way, or maybe could be facilitated by hiring people to attend meetings for the specific purpose of being IM-to-mic scribes, this part seems to be out of scope for the RPS
or maybe could be done with tools that allow copy-and-paste of text from IM to a speech synthesizer that reads it to the room. with respect to this part, you should obviously also look after moderation in this case (I copy-and-paste the sentence, but the sentence itself should be read to the room
at the right time, i.e. when the the user's turn to gain the floor arrives).

**Requirement 07-24**: A WG chair MUST be able to control the sound coming from any particular remote attendee. This control MUST allow reduction in volume, all the way to complete muting, of the remote speaker. This requirement, together with others that I will discuss, might be put in a general technical sub-category associated with floor control. With respect to this, I also remark
that if we start talking about moderation and floor control, we have to be careful when identifying roles, meaning that we should avoid making a direct logical link between
the WG chair (who is physically chairing the session) and the moderator (who is actually 'chairing' one or multiple floors/media): the two roles might easily be played by distinct
persons (this is, again, a matter of policies).

**Requirement 07-26**: The audio system used by the RPS MUST be able to integrate with systems commonly used in the venues used for IETF meetings. These venue systems typically include line-level audio outputs from mixers that combine all the mic inputs into a single stream. Some venue systems also allow for headphone level inputs from PCs to be mixed into the audio stream. Right. With reference to this part, you might want to have a look at what we are asking for when it comes to support sessions with Meetecho:

http://recordings.conf.meetecho.com/AudioMixerConfiguration.pdf

**Requirement 07-39**: The IETF Secretariat MUST be able to easily set up the individuals allowed to use the floor control system for a particular meeting and to change the settings at any time, including during the meeting.
Isn't this out of scope for the RPS (policy-related)?

**Requirement 07-40**: The chair SHOULD be able to monitor the sound levels of the audio being delivered to remote attendees to be sure that they can hear what is going on in the room. Beware that this calls for remote monitoring on the client side, which not as easy as one might expect. I would better state that the chair should be able to
'adjust' the sound of the audio of remotes, based on their feedback about the perceived quality.

**Requirement 07-43**: When video is allowed for remote attendees to give presentations (as described in Section 2.3.3), the audience in the room SHOULD be able to see the presenter speaking. Doesn't this simply translate in a requirement to have a secondary beamer in the room which is projecting the remote speaker's video (which would be out of scope for the RPS)?

The whole section 2.5 is in my view related to the third category of requirements I introduced above ("Requirements for the seamless integration of the future RPS system with the current IETF tools and meeting materials portal").
I think that a requirement for the RPS here would just be that of being able to show (in the meeting virtual room) the slides that have been uploaded to the meeting materials page. This might, e.g., be done through a conversion of ppt/pptx/pdf/odp/etc files to whatever the RPS uses internally (e.g. a sequence of images, a low-rate video, etc.).

**Requirement 07-53**: The slide stream MUST represent the slides as they are projected in the room, allowing the presenter to go back and forth, as well as to edit slides in real time. This makes it clear to the remote attendees which set of slides, and which slide number, is being currently shown. Real-time editing of the slides is a tough requirement, indeed. If you think of how this is typically done, the most likely option here would be to have the speaker make his presentation by using some sort of application sharing tool (e.g., by sharing his powerpoint application running on his laptop). I think we should relax this requirement, by associating a MUST to just showing slides and browsing back and forth through the presentation. Slide annotations might then be done by using a whiteboard or some similar tool. What is your feeling about this?

**Requirement 07-60**: A much-lower priority requirement is the ability for group-editing of graphics.
As before, this is a tough one. What do you mean by group editing of graphics? Are you thinking of a sort of shared "Paint", a whiteboard, or what?

**Requirement 07-61**: The RPS MUST support storage and distribution of recordings of the audio, video, and slide presentations for all WG meetings. I would also specify that it is highly desirable to have "synchronized" recordings of the meeting media.

**Requirement 07-64**: Users MUST be able to easily find the archives of the recordings and instant messaging transcripts of a particular WG or BoF session at a particular meeting. This is out of scope for the RPS (policies).

**Requirement 07-66**: The RPS MUST support recording and storage of recordings of the audio, video, and slide presentations of interim meetings as well as regular IETF meetings. Once again, from the RPS perspective, there should be no difference at all between recordings of a regular meeting and recordings of an interim.

**Requirement 07-67**: Given that interim meetings are often run without the help of the IETF Secretariat, making these recordings MUST be easy for WG chairs. Why should the chairs bother about the recordings? I think the RPS should be capable of making recordings in an automated fashion. The chairs
should just don't care about the inner workings.

**Requirement 07-74**: The RPS MUST allow a session to be moved from one room to another during the session This is needed because the Secretariat sometimes need to swap the rooms for WGs when it becomes clear that one is too full and another room has excess space. Apart from the unavoidable association between the mixer in the room and the virtual room in the RPS, this should have no other impact on the RPS itself.

**Requirement 07-79**: Remote attendees MUST be able to easily find all the material they need to effectively participate, including links to audio, video, instant messaging, slides, and so on. This material MUST be available well before the time of the meeting. The page with the meeting material SHOULD allow the remote attendee to easily perform a time conversion to and from the local time at the IETF meeting. Out of scope for the RPS (policies).

**Requirement 07-84**: The RPS SHOULD have a central location where the specifics about how remote participation is supported for every WG interim meeting. This will reduce the problems often seen where messages about how to participate in an interim meeting get buried in the WG mailing list. I don't understand this one, but looks like a policy-related issue.

Hope you'll find this feedback useful.

Cheers,

Simon


















-- _\\|//_ ( O-O ) ~~~~~~~~~~~~~~~~~~~~~~o00~~(_)~~00o~~~~~~~~~~~~~~~~~~~~~~~~ Simon Pietro Romano Universita' di Napoli Federico II Computer Science Department Phone: +39 081 7683823 -- Fax: +39 081 7684219 e-mail: spromano-7EJwN6hzp+g@public.gmane.org http://www.comics.unina.it/simonpietro.romano <<Molti mi dicono che lo scoraggiamento è l'alibi degli idioti. Ci rifletto un istante; e mi scoraggio>>. Magritte. oooO ~~~~~~~~~~~~~~~~~~~~~~( )~~ Oooo~~~~~~~~~~~~~~~~~~~~~~~~~ \ ( ( ) \_) ) / (_/

<div>
    Hi Paul and vmeetters,<br><br>
    thank you for submitting a revised version (-07) of your draft.
    Please find below my comments, preceded by the usual disclaimer
    about the length of my message .<br>
    I'll take some excerpts of the draft and let them be followed by my
    comments/observations. As a general comment, I personally think that
    the document mixes technical and non-technical requirements for the
    RPS and I find this a bit confusing. Technical requirements are
    those directly coupled to the functionality that the RPS will have
    to make available and would be easily described (or mapped) with a
    formal modeling approach like UML, in much the same way as with any
    other software requirements specification. Non-technical
    requirements are most of the times related to 'policies', which, as
    I have often claimed on IETF mailing list discussions, are always
    orthogonal to (and hence independent from)&nbsp; technical details of a
    system. I'll try to make this clear in my detailed review, but for
    the sake of clarity I'll give you an example of what I'm saying
    right below:<br><br>
    - technical requirement: <br>**Requirement 07-02**: All tools in the RPS MUST be able to use both
   IPv4 and IPv6 addresses natively.

    - non-technical (policy-based) requirement:<br><br>**Requirement 07-07**: Both local and remote attendees SHOULD be able
   to easily contact a single entity who is available throughout the
   meeting if they find problems with any of the RPS tools, and to get
   fairly rapid response.  This entity needs to be able to handle as RPS
   tool problems in the meeting rooms, or be able to quickly contact
   someone who can address those problems.

    My personal preference would be to have a clear distinction between
    the above classes of requirements. I think this would also make it
    easier, in the future, to edit a formal<br>
    software requirements specification for potential bidders.<br><br>
    This said, I also found a third category of requirements which in my
    view fall in a specific class which I would call something like
    "Requirements for the seamless integration of the future RPS system
    with the current IETF tools and meeting materials portal". To
    provide you an example of this third class of requirements, I'll
    copy/paste a couple below:<br>**Requirement 07-49**: The RPS MUST automatically convert PowerPoint
   presentations to PDF and make both available for distribution at the
   same time.

NOTE: in the requirement below I assume that with the expression "make both available for distribution" you are referring to slides publication on the 
meeting materials page, right?

**Requirement 07-50**: Presenters MUST be able to update their slides
   on the IETF site up to just before their presentation, if such update
   is allowed by the WG chairs.

    NOTE: this last example is in my view actually a mix of a "policy-based" and an "integration" requirement.

So, let's get started with the details.

    <br>**Requirement 07-09**: The deployment of the tools here MUST take
   into account making the tools accessible to as many IETF attendees as
   possible.  Such deployment is likely to include technical
   accommodations for those with visual and hearing disabilities.

    <br>
    &nbsp;Agreed. But this formulation sounds a bit too generic. What kind of
    accommodations do you have in mind? As an example, for deaf people,
    you might be thinking of remote transcription services like "Remote
    CART" (Communication Access Real-time Translation) captioning,
    perhaps based on standards like XEP-0301 (Real-Time Text for XMPP --
    <a class="moz-txt-link-freetext" href="http://www.xmpp.org/extensions/xep-0301.html">http://www.xmpp.org/extensions/xep-0301.html</a>). In any case, I would
    make this part more explicit.<br><br>**Requirement 07-10**: All remote attendees at regular IETF meetings
   and interim meetings who make contributions MUST register with the
   IETF Secretariat before contributing using any of the RPS tools.
    This is out of scope for the RPS. I think the basic idea behind
    section 2.1 is that the IETF secretariat should in the future
    envisage different registration options (local attendee, remote
    contributing attendee, remote non-contributing attendee) and the RPS
    should have access to such information when logging users into the
    system, so to be capable of associating them with a specific role.
    In summary, from the RPS's point of view, the only requirement
    should be that it be capable of supporting different user roles,
    once again configured through policies. You actually mention
    something like this when you state, at the end of the section, that
    "registration might happen in the IETF's Datatracker".<br><br>
    Coming to section 2.3 on Audio, I have already stated on this ML
    (<a class="moz-txt-link-freetext" href="http://www.ietf.org/mail-archive/web/vmeet/current/msg00548.html">http://www.ietf.org/mail-archive/web/vmeet/current/msg00548.html</a>)
    that I strongly believe the priorities you identify in the document
    are not in the right order. I cannot think of a 'future' RPS which
    is not capable of reliably supporting audio from remote attendees
    (both those who have to give presentations and those who just want
    to make sporadic comments).&nbsp; I know this brings in some technical
    obstacles, but I nonetheless insist on this point. <br>
    With respect to the specific requirements you identify:<br><br>**Requirement 07-20**: The RPS MUST enable relay of messages from IM
   to the mic to be able to happen as quickly as if the remote attendee
   was local.

    It is not clear to me what you mean by "as quickly as if the remote
    attendee was local".<br>
    From a technical standpoint, this should translate in something
    similar to what I was proposing in the already cited mail, i.e. a
    solution which automatically announces a request for attention
    coming, via the jabber room, from a remote: in a few words, if a
    remote writes a sentence in the chat and lets it be preceded by the
    [mic] prefix, the RPS should automatically play a predefined
    'attention sound' inside the room.<br><br>**Requirement 07-21**: The person relaying from IM to the mic must be
   available throughout the WG meeting.  To date, this has been done by
   WG volunteers in the room.  In the future, it could be done the same
   way, or maybe could be facilitated by hiring people to attend
   meetings for the specific purpose of being IM-to-mic scribes, 

    this part seems to be out of scope for the RPS<br>
   or maybe could be done with tools that allow copy-and-paste of text from
   IM to a speech synthesizer that reads it to the room.

    with respect to this part, you should obviously also look after
    moderation in this case (I copy-and-paste the sentence, but the
    sentence itself should be read to the room<br>
    at the right time, i.e. when the the user's turn to gain the floor
    arrives).<br><br>**Requirement 07-24**: A WG chair MUST be able to control the sound
   coming from any particular remote attendee.  This control MUST allow
   reduction in volume, all the way to complete muting, of the remote
   speaker.

    This requirement, together with others that I will discuss, might be
    put in a general technical sub-category associated with floor
    control. With respect to this, I also remark<br>
    that if we start talking about moderation and floor control, we have
    to be careful when identifying roles, meaning that we should avoid
    making a direct logical link between<br>
    the WG chair (who is physically chairing the session) and the
    moderator (who is actually 'chairing' one or multiple floors/media):
    the two roles might easily be played by distinct<br>
    persons (this is, again, a matter of policies).<br><br>**Requirement 07-26**: The audio system used by the RPS MUST be able
   to integrate with systems commonly used in the venues used for IETF
   meetings.  These venue systems typically include line-level audio
   outputs from mixers that combine all the mic inputs into a single
   stream.  Some venue systems also allow for headphone level inputs
   from PCs to be mixed into the audio stream.
    Right. With reference to this part, you might want to have a look at
    what we are asking for when it comes to support sessions with
    Meetecho:<br><br><a class="moz-txt-link-freetext" href="http://recordings.conf.meetecho.com/AudioMixerConfiguration.pdf">http://recordings.conf.meetecho.com/AudioMixerConfiguration.pdf</a><br><br> **Requirement 07-39**: The IETF Secretariat MUST be able to easily
   set up the individuals allowed to use the floor control system for a
   particular meeting and to change the settings at any time, including
   during the meeting.
    <br>
    Isn't this out of scope for the RPS (policy-related)?<br><br>**Requirement 07-40**: The chair SHOULD be able to monitor the sound
   levels of the audio being delivered to remote attendees to be sure
   that they can hear what is going on in the room.

    Beware that this calls for remote monitoring on the client side,
    which not as easy as one might expect. I would better state that the
    chair should be able to <br>
    'adjust' the sound of the audio of remotes, based on their feedback
    about the perceived quality.<br><br>**Requirement 07-43**: When video is allowed for remote attendees to
   give presentations (as described in <a href="http://tools.ietf.org/html/draft-ietf-genarea-rps-reqs-07#section-2.3.3">Section 2.3.3</a>), the audience in
   the room SHOULD be able to see the presenter speaking.

    Doesn't this simply translate in a requirement to have a secondary
    beamer in the room which is projecting the remote speaker's video
    (which would be out of scope for the RPS)?<br><br>
    The whole section 2.5 is in my view related to the third category of
    requirements I introduced above ("Requirements for the seamless
    integration of the future RPS system with the current IETF tools and
    meeting materials portal"). <br>
    I think that a requirement for the RPS here would just be that of
    being able to show (in the meeting virtual room) the slides that
    have been uploaded to the meeting materials page. This might, e.g.,
    be done through a conversion of ppt/pptx/pdf/odp/etc files to
    whatever the RPS uses internally (e.g. a sequence of images, a
    low-rate video, etc.).<br><br>**Requirement 07-53**: The slide stream MUST represent the slides as
   they are projected in the room, allowing the presenter to go back and
   forth, as well as to edit slides in real time.  This makes it clear
   to the remote attendees which set of slides, and which slide number,
   is being currently shown.

    Real-time editing of the slides is a tough requirement, indeed. If
    you think of how this is typically done, the most likely option here
    would be to have the speaker make his presentation by using some
    sort of application sharing tool (e.g., by sharing his powerpoint
    application running on his laptop). I think we should relax this
    requirement, by associating a MUST to just showing slides and
    browsing back and forth through the presentation. Slide annotations
    might then be done by using a whiteboard or some similar tool. What
    is your feeling about this?<br><br>**Requirement 07-60**: A much-lower priority requirement is the
   ability for group-editing of graphics.
    <br>
    As before, this is a tough one. What do you mean by group editing of
    graphics? Are you thinking of a sort of shared "Paint", a
    whiteboard, or what?<br><br> **Requirement 07-61**: The RPS MUST support storage and distribution
   of recordings of the audio, video, and slide presentations for all WG
   meetings.

    I would also specify that it is highly desirable to have
    "synchronized" recordings of the meeting media.<br><br> **Requirement 07-64**: Users MUST be able to easily find the archives
   of the recordings and instant messaging transcripts of a particular
   WG or BoF session at a particular meeting.

    This is out of scope for the RPS (policies).<br><br>**Requirement 07-66**: The RPS MUST support recording and storage of
   recordings of the audio, video, and slide presentations of interim
   meetings as well as regular IETF meetings.

    Once again, from the RPS perspective, there should be no difference
    at all between recordings of a regular meeting and recordings of an
    interim.<br><br> **Requirement 07-67**: Given that interim meetings are often run
   without the help of the IETF Secretariat, making these recordings
  MUST be easy for WG chairs.

    Why should the chairs bother about the recordings? I think the RPS
    should be capable of making recordings in an automated fashion. The
    chairs <br>
    should just don't care about the inner workings.<br><br> **Requirement 07-74**: The RPS MUST allow a session to be moved from
   one room to another during the session This is needed because the
   Secretariat sometimes need to swap the rooms for WGs when it becomes
   clear that one is too full and another room has excess space.

    Apart from the unavoidable association between the mixer in the room
    and the virtual room in the RPS, this should have no other impact on
    the RPS itself.<br><br> **Requirement 07-79**: Remote attendees MUST be able to easily find
   all the material they need to effectively participate, including
   links to audio, video, instant messaging, slides, and so on.  This
   material MUST be available well before the time of the meeting.  The
   page with the meeting material SHOULD allow the remote attendee to
   easily perform a time conversion to and from the local time at the
   IETF meeting.

    Out of scope for the RPS (policies).<br><br> **Requirement 07-84**: The RPS SHOULD have a central location where
   the specifics about how remote participation is supported for every
   WG interim meeting.  This will reduce the problems often seen where
   messages about how to participate in an interim meeting get buried in
   the WG mailing list.

    I don't understand this one, but looks like a policy-related issue.<br><br>
    Hope you'll find this feedback useful.<br><br>
    Cheers,<br><br>
    Simon<br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
    -- 
                            _\\|//_
                            ( O-O )
   ~~~~~~~~~~~~~~~~~~~~~~o00~~(_)~~00o~~~~~~~~~~~~~~~~~~~~~~~~
                    Simon Pietro Romano
              Universita' di Napoli Federico II
                 Computer Science Department 
        Phone: +39 081 7683823 -- Fax: +39 081 7684219
                e-mail: <a class="moz-txt-link-abbreviated" href="mailto:spromano@...">spromano@...</a>
          <a class="moz-txt-link-freetext" href="http://www.comics.unina.it/simonpietro.romano">http://www.comics.unina.it/simonpietro.romano</a>

    &lt;&lt;Molti mi dicono che lo scoraggiamento &egrave; l'alibi degli 
   idioti. Ci rifletto un istante; e mi scoraggio&gt;&gt;. Magritte.
                         oooO
   ~~~~~~~~~~~~~~~~~~~~~~(   )~~ Oooo~~~~~~~~~~~~~~~~~~~~~~~~~
                          \ (    (   )
                           \_)    ) /
                                 (_/

  </div>