Hollenbeck, Scott | 6 Jul 17:17 2006
Picon

EPP Implementation Test Matrix

Here's the first draft of a complete EPP test matrix.  I've tried to
document the results of testing between two client implementations and
three server implementations, with all of the software being developed
independently.  I'm sharing it here so that all can see what was done,
how I'd like to report the results, and to catch any errors or
omissions.  Each row should have at least one "X" in it to confirm that
the feature was tested and confirmed in at least two implementations.  I
believe we've met that requirement.

Please clue me in if I've missed anything.

-Scott-
----------
Key:
 D-C: DotRegistrar Client
 K-C: Key Systems Client
 A-S: Afilias Server
 N-S: NeuStar Server
 V-S: VeriSign Server

 "X": Feature implemented and tested by both client and server.
 "*": Feature implemented by server, but not tested by client.
 "-": Feature not implemented or not tested.

Client Feature                             Client-Server Support
------------------------------------------------------------------------
-
                                    | D-C | D-C | D-C | K-C | K-C | K-C
|
                                    | A-S | N-S | V-S | A-S | N-S | V-S
(Continue reading)

Klaus Malorny | 6 Jul 18:54 2006
Picon

Re: EPP Implementation Test Matrix

Hollenbeck, Scott wrote:
> Here's the first draft of a complete EPP test matrix.  I've tried to
> document the results of testing between two client implementations and
> three server implementations, with all of the software being developed
> independently.  I'm sharing it here so that all can see what was done,
> how I'd like to report the results, and to catch any errors or
> omissions.  Each row should have at least one "X" in it to confirm that
> the feature was tested and confirmed in at least two implementations.  I
> believe we've met that requirement.
> 
> Please clue me in if I've missed anything.
> 

Hi Scott,

I am not sure whether I understand why you send this to this list. We can't 
verify that anyway. For example:

>   <response>                        |  X  |  X  |  X  |  X  |  X  |  X
> |
>     OPTIONAL <value>                |  X  |  X  |  X  |  X  |  X  |  X

How can I verify that? I don't know what DotRegistrar and Key Systems get from 
Afilias and Neulevel, but I know what we get, and because of this, I wouldn't 
put an 'X' here. Your draft (draft-hollenbeck-epp-rfc3730bis-01, the latest I 
could find) says:

          Zero or more OPTIONAL <value> elements that identify a client-
          provided element (including XML tag and value) that caused a
          server error condition.
(Continue reading)

Hollenbeck, Scott | 6 Jul 19:45 2006
Picon

RE: EPP Implementation Test Matrix

> -----Original Message-----
> From: Klaus Malorny [mailto:Klaus.Malorny <at> knipp.de] 
> Sent: Thursday, July 06, 2006 12:55 PM
> To: Hollenbeck, Scott
> Cc: ietf-provreg <at> cafax.se
> Subject: Re: [ietf-provreg] EPP Implementation Test Matrix
> 
> Hollenbeck, Scott wrote:
> > Here's the first draft of a complete EPP test matrix.  I've tried to
> > document the results of testing between two client 
> implementations and
> > three server implementations, with all of the software 
> being developed
> > independently.  I'm sharing it here so that all can see 
> what was done,
> > how I'd like to report the results, and to catch any errors or
> > omissions.  Each row should have at least one "X" in it to 
> confirm that
> > the feature was tested and confirmed in at least two 
> implementations.  I
> > believe we've met that requirement.
> > 
> > Please clue me in if I've missed anything.
> > 
> 
> Hi Scott,
> 
> I am not sure whether I understand why you send this to this 
> list. We can't 
> verify that anyway. For example:
(Continue reading)

Klaus Malorny | 6 Jul 22:19 2006
Picon

Re: EPP Implementation Test Matrix

Hollenbeck, Scott wrote:

> This is precisely why I shared the matrix.  If your results differ, I
> need to know what others have seen for test results.  If there are
> differences of opinion we can talk about them to determine what really
> needs to be in the boxes.
> 

Hi Scott,

It is not a question of "differences". It is a question of interpretation of the 
term "implemented and tested". What about the "<hello>" message, for example? 
Six "X". Really? Does any of the three registries work with a different 
transport layer than RFC 3734? Not that I am aware of. So how could you have 
tested the <hello> command if RFC3734 does not even mention it? Comparing 
RFC3730 and RFC3734 shows that it is at least unclear (if not contradicting) 
whether or not the client may send a "<hello>" message at the beginning of the 
communication (and cannot be tested therefore). My personal interpretation of 
your RFC3734 is, however, that the "<hello>" element is not allowed in TCP-based 
communications. Since the "<hello>" message is not a command, it is also unclear 
in RFC3730 how a server should react if it receives a "<hello>" message at any 
point in time after the server has sent the "<greeting>" message. Also not testable.

>> Howsoever, I would be surprised if you would do anything but 
>> to ignore my 
>> objections -- as usual.
> 
> Ignore your "objections"?  Hardly.  There are multitudes of responses
> from me in the list archives (such as this one) to confirm that you're
> not being ignored when you participate in a productive dialogue.
(Continue reading)

Hollenbeck, Scott | 7 Jul 14:06 2006
Picon

RE: EPP Implementation Test Matrix

> -----Original Message-----
> From: owner-ietf-provreg <at> cafax.se 
> [mailto:owner-ietf-provreg <at> cafax.se] On Behalf Of Klaus Malorny
> Sent: Thursday, July 06, 2006 4:19 PM
> To: Hollenbeck, Scott
> Cc: ietf-provreg <at> cafax.se
> Subject: Re: [ietf-provreg] EPP Implementation Test Matrix
> 
> Hollenbeck, Scott wrote:
> 
> > This is precisely why I shared the matrix.  If your results 
> differ, I
> > need to know what others have seen for test results.  If there are
> > differences of opinion we can talk about them to determine 
> what really
> > needs to be in the boxes.
> > 
> 
> Hi Scott,
> 
> It is not a question of "differences". It is a question of 
> interpretation of the 
> term "implemented and tested". What about the "<hello>" 
> message, for example? 
> Six "X". Really? Does any of the three registries work with a 
> different 
> transport layer than RFC 3734? Not that I am aware of. So how 
> could you have 
> tested the <hello> command if RFC3734 does not even mention 
> it? Comparing 
(Continue reading)

Klaus Malorny | 7 Jul 15:07 2006
Picon

Re: EPP Implementation Test Matrix

Hollenbeck, Scott wrote:

> 
> Sorry, but I disagree.  Apparently other implementers do as well.
> There's absolutely nothing in 3730 that says a <hello> can't be used
> with a connection-oriented transport.  In fact, section 2.3 very clearly
> says that "An EPP client MAY request a <greeting> from an EPP server at
> any time by sending a <hello> to a server".  I don't know what's unclear
> about that.
> 

I know the statement in 2.3, but the figure 1 on page 4, titled "EPP Server 
State Machine" does not tell anything about an exchange of a <hello> or 
<greeting> message except for the startup. Since <hello> is not a command (it is 
not described in section 2.9), the state machine does not expect a <hello> 
message if in the "Waiting for Client Authentication" or "Waiting for Command" 
states. I hope you agree that there is a contradiction.

Also, in 3734, the server sends an unsolicited <greeting>. Should the server 
send a second <greeting> in case the client sends a <hello>? According to 
3730/2.3, it has to.

> Another old argument.  The definition I've consistently used [1] is
> described in Jim Gray and Andreas Reuter's "Transaction Processing:
> Concepts and Techniques" book.  If you disagree with it, take it up with
> them.
> 

I own that book and I clearly do not disagree with the authors of the book in 
any way. Instead I think that you are tweaking the definition until it fits for 
(Continue reading)

Hollenbeck, Scott | 7 Jul 15:22 2006
Picon

RE: EPP Implementation Test Matrix

> -----Original Message-----
> From: Klaus Malorny [mailto:Klaus.Malorny <at> knipp.de] 
> Sent: Friday, July 07, 2006 9:08 AM
> To: Hollenbeck, Scott
> Cc: ietf-provreg <at> cafax.se
> Subject: Re: [ietf-provreg] EPP Implementation Test Matrix
> 
> Hollenbeck, Scott wrote:
> 
> > 
> > Sorry, but I disagree.  Apparently other implementers do as well.
> > There's absolutely nothing in 3730 that says a <hello> can't be used
> > with a connection-oriented transport.  In fact, section 2.3 
> very clearly
> > says that "An EPP client MAY request a <greeting> from an 
> EPP server at
> > any time by sending a <hello> to a server".  I don't know 
> what's unclear
> > about that.
> > 
> 
> I know the statement in 2.3, but the figure 1 on page 4, 
> titled "EPP Server 
> State Machine" does not tell anything about an exchange of a 
> <hello> or 
> <greeting> message except for the startup. Since <hello> is 
> not a command (it is 
> not described in section 2.9), the state machine does not 
> expect a <hello> 
> message if in the "Waiting for Client Authentication" or 
(Continue reading)

Klaus Malorny | 7 Jul 15:51 2006
Picon

Re: EPP Implementation Test Matrix

Hollenbeck, Scott wrote:
> 
> Not a contradiction, an error.  The state machine diagram should be
> fixed.  It will be in the next iteration of the document.  Thanks for
> pointing out the error.
> 

See, I do this despite my opinion regarding EPP ;-)

>> Also, in 3734, the server sends an unsolicited <greeting>. 
>> Should the server 
>> send a second <greeting> in case the client sends a <hello>? 
>> According to 
>> 3730/2.3, it has to.
> 
> Correct about 3730, though the <greeting> isn't unsolicited.  It's sent
> in response to a client action.
> 

Maybe "unsolicited" is the wrong term, but not as a response to a <hello> 
message, but to the connect. So a <hello> message is not consumed by this.

> 
> There's another party to consider: the server operator.  It's important
> for them -- and I can guarantee that registrars WILL care if there are
> financial consequences for transactions repeatedly sent in error.
> 

Ah, come on. The registry is not responsible for the errors that are done by the 
registrar. If a registrar accidentally deletes a domain and has to restore it 
(Continue reading)

Hollenbeck, Scott | 7 Jul 16:14 2006
Picon

RE: EPP Implementation Test Matrix

> -----Original Message-----
> From: Klaus Malorny [mailto:Klaus.Malorny <at> knipp.de] 
> Sent: Friday, July 07, 2006 9:51 AM
> To: Hollenbeck, Scott
> Cc: ietf-provreg <at> cafax.se
> Subject: Re: [ietf-provreg] EPP Implementation Test Matrix
> 
> Hollenbeck, Scott wrote:

[snip]

> > There's another party to consider: the server operator.  
> It's important
> > for them -- and I can guarantee that registrars WILL care 
> if there are
> > financial consequences for transactions repeatedly sent in error.
> > 
> 
> Ah, come on. The registry is not responsible for the errors 
> that are done by the 
> registrar. If a registrar accidentally deletes a domain and 
> has to restore it 
> for big money, you are happy about this, aren't you?

We're continuing a walk through the weeds that has nothing to do with
the test matrix, so this will be the last I have to say about it.

A registry has to build and support servers that can handle whatever
registrars throw at them, such as hundreds or thousands of the exact
same command sent over a short period of time in an attempt to register
(Continue reading)

janusz | 7 Jul 17:51 2006

Re: EPP Implementation Test Matrix

Some registrars in their clients implementations don't rely on 
idempotency in EPP protocol and I can understand that the feature does 
not help them in any way. However there is a large group of registrars 
who explicitly or implicitly depend on the feature. Without idempotency 
enforced on the server side the ratio of registrars complains obout 
transform commands submitted "by mistake" would increase significantly. 
That would eventually lead to higher client implementation and 
maintenance cost. I would certainly not recommend removing idempotency 
requirement from EPP protocol. The feature IMO brings overall more value 
to EPP client than servers implementers and operators.

Janusz Sienkiewicz


Gmane