HTTP Draft-p1 and http scheme definition

#117: cache invalidation upon update

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/117>

I propose we close this issue with no action, as the specification now clearly says that when a cache has more
than one valid entry, the latest one wins -- making this a non-issue for GET.

<http://tools.ietf.org/html/draft-ietf-httpbis-p6-cache-14#section-2.2>:

> A cache MUST use the most recent response (as determined by the Date
>    header field) when more than one suitable response is stored.

Henrik?

Regards,

--
Mark Nottingham   http://www.mnot.net/

Re: NEW: #235: Cache Invalidation only happens upon successful responses

The editors talked about this in Prague, and we came to a place where we agreed specifying it for 2xx would be fine.

I changed my mind because my case below isn't the common case; that's a POST followed by a 201 with a Location.

Any objections to / thoughts about that?

Regards,

On 03/02/2011, at 5:08 PM, Mark Nottingham wrote:

> Reviving this thread...
> 
> I think that everyone agrees that 2xx should be considered successful, and that 1xx / 4xx / 5xx shouldn't.
The question is regarding 3xx.
> 
> The cache invalidation part of the spec explicitly includes the Location header, which is defined to work
on 201 as well as many 3xx responses.
> 
> The authoring case of:
> 
> GET /x --> 200 OK
> [ fill out form ]
> POST /y --> 201 Created 
> Location: /z
> 
> will work, in that anything previously at /y and /z will get invalidated.
> 
> The other case of:
> 
> GET /x --> 200 OK
> [ fill out form ]
> POST /y --> 303 See Other
> Location: /z
> 
> would not invalidate /y or /z. Depending on the semantics of /z, this isn't necessarily the end of the
world, but it may be suboptimal (e.g., if /z is a status page for all of the forms I've filled out, I'd expect
the one I just filled out to be on there). 
> 
> As such, I'm going to (with my hat off) argue for including both 2xx and 3xx as 'successful' in this context;
it's useful and simpler. Roy pointed out a potential downside that it'd invalidate a form I just filled
out, but as illustrated above, the form URI can be different from the form submission URI if this is a concern.
> 
> Thoughts?
> 
> 
> 
> 
> On 19/10/2010, at 11:41 AM, Mark Nottingham wrote:
> 
>> 
>> On 19/10/2010, at 11:37 AM, Roy T. Fielding wrote:
>> 
>>> On Oct 18, 2010, at 5:28 PM, Mark Nottingham wrote:
>>>> The question here, though, is whether /y should also be invalidated; since 2616 goes to pretty
extensive lengths to say that the URL indicated by Location is to be invalidated, I don't see why it
shouldn't be...
>>> 
>>> It does?  That sounds like a DoS attack vector.
>> 
>> 
>> 2616:
>> 
>>> In order to prevent denial of service attacks, an invalidation based on the URI in a Location or
Content-Location header MUST only be performed if the host part is the same as in the Request-URI.
>> 
>> --
>> Mark Nottingham   http://www.mnot.net/
>> 
>> 
>> 
>> 
> 
> --
> Mark Nottingham   http://www.mnot.net/
> 
> 
> 
> 

--
Mark Nottingham   http://www.mnot.net/

#282: Recommend minimum sizes for protocol elements

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/282>

We talked about this briefly in Prague. Since then I've put together a straw-man proposal:

Insert at end of p1 3.2:

> HTTP does not place a pre-defined limit on the length of header fields, either in isolation or as a set. A
server MUST be prepared to receive request headers of unbounded length and respond with the 413 (Request
Entity Too Large) status code if the received header(s) would be longer than the server wishes to handle
(see Section 8.4.14 of [Part2]).
> 
> A client that receives response headers that are longer than it wishes to handle can only treat it as a
server error.
> 
> Various ad-hoc limitations on header length are found in practice. It is RECOMMENDED that all HTTP
senders and recipients support messages whose combined headers have 20,000 or more octets.

Add section to p1 Security Considerations:

> 11.5 Protocol Element Size Overflows
> 
> Because HTTP uses mostly textual, character-delimited fields, attackers can overflow buffers in
implementations, and/or perform a Denial of Service against implementations that accept fields with
unlimited lengths.
> 
> To promote interoperability, this specification makes specific recommendations for size limits on
request-targets [ref] and blocks of header fields [ref]. These are minimum recommendations, chosen to
be supportable even by implementations with limited resources; it is expected that most
implementations will choose substantially higher limits.
> 
> This specification also provides a way for servers to reject messages that have request-targets that are
too long [ref] or request entities that are too large [ref].
> 
> Other fields (including but not limited to request methods, response status phrases, header
field-names, and body chunks) SHOULD be limited by implementations carefully, so as to not impede interoperability.

Thoughts? We can go further than this, of course, but IME request-target and headers are the big ones.

Cheers,

--
Mark Nottingham   http://www.mnot.net/

Re: HTTP Draft-p1 and http scheme definition

Considering that the registry entry is present, fully populated, and we're just asking them to update the
reference (see
<http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-14#section-10.2>), I don't see a
need to include the full template. 

On 02/05/2011, at 1:58 AM, Mykyta Yevstifeyev wrote:

> Greetings,
> 
> The first mention of http URI scheme in IETF document (RFC) appeared in RFC 1945 (HTTP 1.0 specification). 
Then, it remained unchanged (or was present with minor changes) in RFC 2068 and finally 2616.  RFC 2616
appeared long before RFC 4395, that outlined the registration procedures for URI schemes.  What I'm
pointing to?
> 
> The http scheme was never given a formal registration template, per RFC 4395.  Its definition of current
HTTP draft-p1 remains the same as in RFC 2616.  However, RFC 4395 requests the registration template for
permanent registrations, that is missing in this draft.
> 
> The same concerns https scheme, also defined in p1.
> 
> Thus I propose to add them (templates) for these URI schemes in order to suit the requirements of RFC 4395
for permanent registrations.  Any thoughts on it?
> 
> Mykyta Yevstifeyev

--
Mark Nottingham   http://www.mnot.net/

#186: Document HTTP's error-handling philosophy

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/186>

Straw-man proposal: replace "Requirements" sections (in all parts) with the following.

> Conformance and Error Handling
> 
> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT",
"RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].
> 
> This document defines conformance criteria for several roles in HTTP communication, including
Senders, Recipients, Clients, Servers, User-Agents, Origin Servers, Intermediaries, Proxies and
Gateways. See [ref to Terminology] for a definitions of these terms.
> 
> An implementation is considered conformant if it complies with all of the requirements associated with
its role(s). Note that SHOULD-level requirements are relevant here, unless one of the documented
exceptions is applicable.
> 
> This document also uses ABNF to define valid protocol elements. In addition to the prose requirements
placed upon them, Senders MUST NOT generate protocol elements that are invalid.
> 
> Unless noted otherwise, Recipients MAY take steps to recover a usable protocol element from an invalid
construct, and SHOULD NOT reject the message outright. However, HTTP does not define specific error
handling mechanisms, except in cases where it has direct impact on security. This is because different
uses of the protocol require different error handling strategies; for example, a Web browser may wish to
transparently recover from a response where the Content-Type header doesn't match the body, whereby in a
systems control protocol using HTTP, this type of error recovery could lead to dangerous consequences.

Note that this removes the "conditionally compliant" level of conformance; i.e., SHOULD is no longer
overloaded, and returns to is original RFC2119 semantic of identifying requirements that can be
violated for reasonable reasons (see also
<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/271>, which proposes that we try to enumerate
those reasons wherever possible).

That's a fairly big change. I'd argue that "conditional compliance" doesn't promote interop and should be
dropped. Thoughts?

--
Mark Nottingham   http://www.mnot.net/

Re: SHOULD-level requirements in p6-caching

Hi Willy,

Sorry, I thought I'd responded to this, but apparently not.

On 09/04/2011, at 4:05 PM, Willy Tarreau wrote:

> On Thu, Apr 07, 2011 at 06:19:51PM +1000, Mark Nottingham wrote:
>> In 3.2.1 (only-if-cached),
>> 
>>>      If it receives this
>>>      directive, a cache SHOULD either respond using a stored response
>>>      that is consistent with the other constraints of the request, or
>>>      respond with a 504 (Gateway Timeout) status code.
>> 
>> MUST?
> 
> Is there any reason for a 504 here ? I mean, 5xx are server-side errors
> which should theorically not be decided by the client. In my opinion here
> the client asks for an object and sets a condition to retrieve it, if the
> object cannot be fetched from the cache, the client's condition cannot be
> satisfied, so it should be a 4xx (possibly a 412 Precondition Failed).

Basically, because this is the way it was specified, and this is already used in production (e.g., when
Squid checks a peer for an object).

> It is important to ensure that clients cannot force servers to emit 5xx
> errors, because it is common to include status code checks when monitoring
> production systems. If a cache emits a 504, usually it's because it cannot
> reach the next hop so there is a connectivity issue that must be addressed.

I see your point, but can imagine strategies for workaround; e.g., if the logs are being monitored, make the
difference apparent in the logs; if the client is doing the monitoring, don't send only-if-cached ;)

Cheers,

--
Mark Nottingham   http://www.mnot.net/

Re: SHOULD-level requirements in p6-caching

On 08/04/2011, at 6:47 PM, Poul-Henning Kamp wrote:

> In message <87AB210C-782E-4475-BD4B-A552A549588E@...>, Mark
Nottingham wri
> tes:
> 
>>>> In 2.5, 
>>>> 
>>>>>  A cache that passes through requests with methods it does not
>>>>>  understand SHOULD invalidate the effective request URI (Section 4.3
> 
>>> First off, what does "not understand" mean here ?
>>> 
>>> Does that cover a cache which goes "Ohh, POST, I don't do those:
>>> pass it through" ?
>> 
>> POST is explicitly covered elsewhere in the section, so there's an 
>> overlap here; all caches are expected to do this (and more) for POST.
> 
> But that is not what the text says.
> 
> It is perfectly possible to write a cache that only "understands"
> GET and HEAD, and just pass everything else upstream.
> 
> If you want it to mean "methods not defined in this document which
> the cache does not understand", then the text should say that.

How about:

  A cache SHOULD invalidate the effective request URI when receiving requests with methods other than PUT,
DELETE or POST.

(delta discussion on SHOULD vs. MUST here, and issue #235).

[...]
>>> Third: do we really want to give script kiddies their own private
>>> standards-mandated cache-invalidation button ?
>> 
>> Please read the entire section; this is not new text, and has been 
>> present in HTTP for over a decade. In short, there is a mechanism to 
>> prevent this kind of attack.
> 
> There is ?  What would that be ?
> 
> All I have to do is send a "XYZZY / HTTP/1.1" request and the
> cache is forced to dump its copy.
> 
> As many sites use CMS systems with very systematic URLs, washing
> a cache that way, and bothering the backend server a lot is a
> Simple Matter Of Programming.
> 
> dDoS attack are a fact of life, and from a standards point of view
> we SHOULD consider clients hostile to cache/server-operation
> when judging such aspects of the standard.

In proxy and browser caches, a DOS is already partially protected against:

   However, a cache MUST NOT invalidate a URI from a Location or
   Content-Location header field if the host part of that URI differs
   from the host part in the effective request URI (Section 4.3 of
   [Part1]).  This helps prevent denial of service attacks.

Furthermore, the resolution to <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/235> will make
this protection stronger -- not only for those implementations, but also for gateway caches -- as it will
require the cache to only invalidate upon a successful response from the origin, rather than on any client request.

My reading of why this particular requirement was only SHOULD-level was to allow future methods to specify
that they don't invalidate. If that's the case still, we should say that. 

Likewise, if we want to allow caches to NOT invalidate requests -- i.e., we think that #235 doesn't offer
enough protection against DoS -- we can add that condition to the list too. My interest here is making the
SHOULDs describe their exceptions, rather than leaving them as open doors for interop failure. 

Cheers,

--
Mark Nottingham   http://www.mnot.net/

Re: HTTP Draft-p1 and http scheme definition

02.05.2011 4:54, Mark Nottingham wrote:
> Considering that the registry entry is present, fully populated, and we're just asking them to update the
reference
(see<http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-14#section-10.2>), I don't
see a need to include the full template.
This would be OK in the case if such template were already present 
somewhere.  However IANA registered these schemes before RFC 4395 and 
even RFC 2717 had appeared.  So I think adding the templates is 
necessary to remove this inaccuracy.
>
> On 02/05/2011, at 1:58 AM, Mykyta Yevstifeyev wrote:
>
>> Greetings,
>>
>> The first mention of http URI scheme in IETF document (RFC) appeared in RFC 1945 (HTTP 1.0
specification).  Then, it remained unchanged (or was present with minor changes) in RFC 2068 and finally
2616.  RFC 2616 appeared long before RFC 4395, that outlined the registration procedures for URI schemes. 
What I'm pointing to?
>>
>> The http scheme was never given a formal registration template, per RFC 4395.  Its definition of current
HTTP draft-p1 remains the same as in RFC 2616.  However, RFC 4395 requests the registration template for
permanent registrations, that is missing in this draft.
>>
>> The same concerns https scheme, also defined in p1.
>>
>> Thus I propose to add them (templates) for these URI schemes in order to suit the requirements of RFC 4395
for permanent registrations.  Any thoughts on it?
>>
>> Mykyta Yevstifeyev
> --
> Mark Nottingham   http://www.mnot.net/
>
>
>
>

Re: HTTP Draft-p1 and http scheme definition

The template is just a means to populate the registry; the definitive record is the registry itself.

On 02/05/2011, at 1:57 PM, Mykyta Yevstifeyev wrote:

> 02.05.2011 4:54, Mark Nottingham wrote:
>> Considering that the registry entry is present, fully populated, and we're just asking them to update
the reference
(see<http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-14#section-10.2>), I don't
see a need to include the full template.
> This would be OK in the case if such template were already present somewhere.  However IANA registered
these schemes before RFC 4395 and even RFC 2717 had appeared.  So I think adding the templates is necessary
to remove this inaccuracy.
>> 
>> On 02/05/2011, at 1:58 AM, Mykyta Yevstifeyev wrote:
>> 
>>> Greetings,
>>> 
>>> The first mention of http URI scheme in IETF document (RFC) appeared in RFC 1945 (HTTP 1.0
specification).  Then, it remained unchanged (or was present with minor changes) in RFC 2068 and finally
2616.  RFC 2616 appeared long before RFC 4395, that outlined the registration procedures for URI schemes. 
What I'm pointing to?
>>> 
>>> The http scheme was never given a formal registration template, per RFC 4395.  Its definition of current
HTTP draft-p1 remains the same as in RFC 2616.  However, RFC 4395 requests the registration template for
permanent registrations, that is missing in this draft.
>>> 
>>> The same concerns https scheme, also defined in p1.
>>> 
>>> Thus I propose to add them (templates) for these URI schemes in order to suit the requirements of RFC 4395
for permanent registrations.  Any thoughts on it?
>>> 
>>> Mykyta Yevstifeyev
>> --
>> Mark Nottingham   http://www.mnot.net/
>> 
>> 
>> 
>> 
> 

--
Mark Nottingham   http://www.mnot.net/