Internet Message Access Protocol Extension (imapext)

Neystadt, John | 1 May 2002 18:05

RE: I-D ACTION:draft-neystadt-imap-status-counters-00.txt

> Mark's suggestion of having a
> simple indicator
> to go look at the mailbox in more detail is reasonable for this
> application.

This is what the MUAs do today. Basic problem with that is to calculate message counters per message-context (MIME header) MUA must download message headers of whole mailbox. This can take a while on big mailboxes and represents considerable overhead for servers.

The idea of status-counters is to have server cache this information and be able to provide it to MUA in fraction of time and overhead.

Think for example about telephony MUA, that announces to subscriber "You have 301 new email messages". This must be down in less than 100ms after users picks up the phone and not after few secs as it must be now (after downloading and processing all message headers).

> If you need to do more than that then look to do
> it outside of
> IMAP - I thought SNAP was supposed to provide a way to get the same
> information as you are proposing adding to STATUS. Why duplicate SNAP
> inside of IMAP? Just let clients interested in that info use
> SNAP instead.

I rather would say they complement each other. SNAP is away for subscriber to get notifications when he is off-line. They can be sent to PUA, though Unix message service or thorugh SMS. When subscriber runs his mail program he wants to see counters which are most up-to-date.

John Neystadt
john <at> neystadt.org
http://www.neystadt.org/john/

headers

Neystadt, John | 1 May 2002 18:11

RE: [VPIM] Re: I-D ACTION:draft-neystadt-imap-status-counters-00. txt

> I have to agree with Mark's sentiment expressed below. The
> whole point of
> using IMAP for Pink Lemonade is there is a large existing
> base of servers
> and developers. If we set out to build "VPIM-IMAP", we might
> as well call
> it son/daughter of IMAP and be done with it.

First of all, some of us develop products who do enhancements for Pink Lemonde. For example SNAP protocol is such an enhancement that requires modification of the servers as well.

My extension does not mandate VPIM clients not to work with servers that does not implement it. Section 8.2 of the draft says:

> 8.2 Client Issues
>
>    Is imap server implementation does not support this command, its
>    logic can be achieved by the following algorithm:
>
>    For the given mailbox do:
>
>       1. a SELECT mailbox
>       2. b FETCH 1:* BODY[HEADER (Importance Message-Context)]
>       3. Calculate the counters from FETCH responses.

Servers that will not implement this, will burn more CPU and have worse response time, but other than that that will work.

John Neystadt
john <at> neystadt.org
http://www.neystadt.org/john/

headers

Mark Crispin | 1 May 2002 18:59

RE: I-D ACTION:draft-neystadt-imap-status-counters-00.txt

On Wed, 1 May 2002 19:05:47 +0300 , Neystadt, John wrote:
> This is what the MUAs do today. Basic problem with that is to calculate
> message counters per message-context (MIME header) MUA must download message
> headers of whole mailbox.

As I pointed out before, you do not need to do this.

You can do searches and partial header fetches to reduce greatly the amount of
downloading you must do.  You can also use keyword flags to avoid using the
headers at all.

Please to leverage existing mechanisms as much as possible rather than create
new mechanisms.  This includes design of extensions.  You're much more likely
to get extensions widely deployed if those extensions are built on top of
existing mechanisms, particularly if the extension is as narrowly focused as
this.

headers

Arnt Gulbrandsen | 1 May 2002 21:43

Re: [VPIM] Re: I-D ACTION:draft-neystadt-imap-status-counters-00. txt


"Neystadt, John" <John <at> comverse.com>
> My extension does not mandate VPIM clients not to work with servers that
> does not implement it. Section 8.2 of the draft says:
...

Why do you specify using FETCH? SEARCH seems better:

	a SELECT mailbox
	b SEARCH UNSEEN
	c SEARCH HEADER Importance: high
	d SEARCH HEADER Message-Context: blah

--Arnt

headers

Erev, Ari | 2 May 2002 10:11

RE: [VPIM] Re: I-D ACTION:draft-neystadt-imap-status-counters-00. txt

Arnt,

I think everybody (including John) agrees that _FUNCTIONALLY_ the current IMAP provides what is required for a client to get the 'counters' information - whether it is by using FETCH or SEARCH.

The main point in John proposal is not about functionality - it is about performance (and possibly scalability).
It allow the counters information (which is required very early in the session) to be available to the client with minimal round-trips and it allows the server to optimize for this task - if it selects to do so.

Let's assume a mailbox which has (only) 3 types of messages (message-contexts).
According to your example, I believe the client can get the counters information this way (12 round trips):

tag SEARCH UNSEEN HEADER Importance high HEADER Message-Context voice-message
tag SEARCH SEEN HEADER Importance high HEADER Message-Context voice-message
tag SEARCH UNSEEN HEADER Importance low HEADER Message-Context voice-message
tag SEARCH SEEN HEADER Importance low HEADER Message-Context voice-message

tag SEARCH UNSEEN HEADER Importance high HEADER Message-Context text-message
tag SEARCH SEEN HEADER Importance high HEADER Message-Context text-message
tag SEARCH UNSEEN HEADER Importance low HEADER Message-Context text-message
tag SEARCH SEEN HEADER Importance low HEADER Message-Context text-message

tag SEARCH UNSEEN HEADER Importance high HEADER Message-Context fax-message
tag SEARCH SEEN HEADER Importance high HEADER Message-Context fax-message
tag SEARCH UNSEEN HEADER Importance low HEADER Message-Context fax-message
tag SEARCH SEEN HEADER Importance low HEADER Message-Context fax-message

Can you suggest a better/quicker way to get this information?

Maybe there is an option to skip the last group of 4 by computing it somehow but anyway - it is obvious that with more "Message-Contexts" this scenario gets longer and longer - which does not seem to be a good way to achieve scalability AND short response time - which is the original requirement!

And so, John proposal suggests to get the required information in ONE command (STATUS).
This provides for short response time, and also allows for servers that want to optimize for this case to do so in whichever way they choose to do so.

Note that STATUS has always been designed to provide this "type" of information (message counters) - just not in the detailed manner it is now required when clients need to get the counters "per message-context". So, an enhancement to STATUS seems to me a logical way to achieve this.

Ari Erev

-----Original Message-----
From: Arnt Gulbrandsen [mailto:arnt <at> gulbrandsen.priv.no]
Sent: Wednesday, May 01, 2002 10:43 PM
To: Neystadt, John
Cc: 'eburger <at> snowshore.com'; ietf-imapext <at> imc.org; vpim <at> lists.neystadt.org
Subject: Re: [VPIM] Re: I-D ACTION:draft-neystadt-imap-status-counters-00. txt

"Neystadt, John" <John <at> comverse.com>
> My extension does not mandate VPIM clients not to work with servers that
> does not implement it. Section 8.2 of the draft says:
...

Why do you specify using FETCH? SEARCH seems better:

        a SELECT mailbox
        b SEARCH UNSEEN
        c SEARCH HEADER Importance: high
        d SEARCH HEADER Message-Context: blah

--Arnt

headers

Arnt Gulbrandsen | 2 May 2002 11:36

Re: [VPIM] Re: I-D ACTION:draft-neystadt-imap-status-counters-00. txt


"Erev, Ari" <Ari_Erev <at> icomverse.com>
> Let's assume a mailbox which has (only) 3 types of messages
> (message-contexts). 
> According to your example, I believe the client can get the counters
> information this way (12 round trips):

That's one scary-looking series of commands. However, it translates into
this equally scary prompt:

	Hello, this is your voice-mail system. You have 4 new important
	voice messages, 12 old important voice messages, 3 unimportant new
	voice messages, 4 unimportant old voice messages, 4 new important
	text messages, 12 old important text messages, 3 unimportant new
	text messages, 4 unimportant old text messages, 4 new important
	fax messages, 12 old important fax messages, 3 unimportant new fax
	messages and 4 unimportant old fax messages. Press 1 to...

Performancewise, this does not seem to be a problem. One search must
complete before the start of the sentence, and all 12 searches before the
word 'and'. And that prompt is a great April fool's joke ;)

Seriously, if the prompt contains n numbers, you should need about n
SEARCH commands.  If, as you say, the server selects to optimize for this
task, it should require as little processor time as e.g. SEARCH UNSEEN
does.

> Can you suggest a better/quicker way to get this information?

Yes, but you might still decide that a new extension is the best option.
I too think it might be. The current draft doesn't argue its case very
well.

As I see it, using SEARCH optimally, using the ANNOTATE extension, or a
combination of the two are all worth looking into.

ANNOTATE is a larger extension than STATUS-COUNTERS, but it looks to have
wide support, so that it'll be implemented by many servers. The message
context and importance you're using sound like annotations to my ears, so
ANNOTATE might be a good fit.

> Note that STATUS has always been designed to provide this "type" of
> information (message counters) - 

Agreed.

> just not in the detailed manner it is now
> required when clients need to get the counters "per message-context". So, an
> enhancement to STATUS seems to me a logical way to achieve this.

If so, it must not be VPIM-specific. Try to tweak the extension so that
it's useful for other clients, and provide a couple of examples showing
how non-VPIM clients might use it.

--Arnt

headers

Simon Josefsson | 2 May 2002 19:26

Re: I-D ACTION:draft-neystadt-imap-status-counters-00. txt


"Erev, Ari" <Ari_Erev <at> icomverse.com> writes:

> According to your example, I believe the client can get the counters
> information this way (12 round trips):
>
> tag SEARCH UNSEEN HEADER Importance high HEADER Message-Context
> voice-message
> tag SEARCH    SEEN HEADER Importance high HEADER Message-Context
> voice-message
> tag SEARCH UNSEEN HEADER Importance low HEADER Message-Context voice-message
> tag SEARCH     SEEN HEADER Importance low  HEADER Message-Context
> voice-message
>
> tag SEARCH UNSEEN HEADER Importance high HEADER Message-Context text-message
> tag SEARCH    SEEN HEADER Importance high HEADER Message-Context
> text-message
> tag SEARCH UNSEEN HEADER Importance low HEADER Message-Context text-message
> tag SEARCH    SEEN HEADER Importance low HEADER Message-Context text-message
>
> tag SEARCH UNSEEN HEADER Importance high HEADER Message-Context fax-message
> tag SEARCH    SEEN HEADER Importance high HEADER Message-Context fax-message
> tag SEARCH UNSEEN HEADER Importance low HEADER Message-Context fax-message
> tag SEARCH    SEEN HEADER Importance low HEADER Message-Context fax-message
>
> Can you suggest a better/quicker way to get this information? 

I see only one round trip.  Send all commands in one operation and
then wait for the responses.

headers

Alexey Melnikov | 3 May 2002 14:38

Re: I-D ACTION:draft-neystadt-imap-status-counters-00.txt

Let me through couple more rocks :

John Neystadt wrote:

> I think your proposal is very good for Unseen and may be Important (new flag
> or reuse \Flagged). BTW is \Flagged semantics is really "importance". If I
> make my server (by Sieve or other mechanism) turn \Flagged on all messages
> with "Importance: High" header, would that be ok.

What about "X-Priority: 1 (Highest)" and "Priority: HIGH" headers that are in
use by the MUA today?

Regards,
Alexey Melnikov
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Richmond, Surrey, UK
Phone: +44 20 8332 4508

I speak for myself only, not for my employer.
__________________________________________

headers

Alexey Melnikov | 3 May 2002 14:54

Re: [VPIM] Re: I-D ACTION:draft-neystadt-imap-status-counters-00. txt

Mark Crispin wrote:

> On Sun, 28 Apr 2002 20:54:15 -0600, Vaudreuil, Greg M (Greg) wrote:
> > Is there any objection to a new flag with explicit semantics of importance?
>
> I think that you will find a very strong lack of enthusiasm for adding a flag
> which duplicates the existing functionality of \Flagged and/or SEARCH HEADER.

I agree in general but not in this particular case.
I use Netscape Communicator 4.79 as a mail client. It allows to change priority
of the message (i.e. overwrite X-Priority header) and I use this feature quite a
bit. Unfortunately Netscape stores this value in the local mailbox cache and this
change is lost when I synchronize a mailbox. I would love to see the mail client
being able to store this on a IMAP server. [Netscape people, are you listening
?]

Regards,
Alexey Melnikov
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Richmond, Surrey, UK
Phone: +44 20 8332 4508

I speak for myself only, not for my employer.
__________________________________________

headers

Alexey Melnikov | 3 May 2002 15:34

Re: [VPIM] Re: I-D ACTION:draft-neystadt-imap-status-counters-00.txt

"Vaudreuil, Greg M (Greg)" wrote:

> The importance is a sender-indicated message attribute, not a recipient
> indication of a particular level of interest or importance they may wish to
> recall between sessions. We need a flag that is standardized by convention
> between a message deposit process that can set this flag and the many
> different clients that would expect the flag to be there if a message is
> tagged as important.
>
> I think use of a new flag makes plenty of sense for \important.  I don't
> think a flag per message context makes much sense tho.

If you decide to introduce a new flag, don't use \important, as the leading "\"
has special semantic in IMAP and server that don't support it will most likely
refuse to store it. I suggest you use $Important instead. This way you will be
able to store it on a IMAP server that allows for storing of arbitrary keywords.

Regards,
Alexey Melnikov
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Richmond, Surrey, UK
Phone: +44 20 8332 4508

I speak for myself only, not for my employer.
__________________________________________

Mon	Tue	Wed	Thu	Fri	Sat	Sun

		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31

21	June 2013
41	May 2013
21	April 2013
27	March 2013
8	February 2013
17	January 2013
10	December 2012
60	November 2012
6	October 2012
48	September 2012
113	August 2012
7	July 2012
16	March 2012
10	February 2012
2	July 2011
1	October 2010
15	July 2010
2	June 2010
2	March 2010
1	November 2009
2	September 2009
3	August 2009
52	July 2009
1	May 2009
5	March 2009
1	February 2009
10	November 2008
19	October 2008
2	August 2008
4	July 2008
30	June 2008
1	May 2008
12	April 2008
96	March 2008
39	February 2008
6	January 2008
30	December 2007
48	November 2007
6	October 2007
25	September 2007
61	August 2007
1	July 2007
15	June 2007
10	May 2007
68	April 2007
42	March 2007
1	February 2007
9	January 2007
9	December 2006
181	November 2006
26	October 2006
72	September 2006
63	August 2006
2	July 2006
7	June 2006
27	May 2006
43	April 2006
87	March 2006
39	February 2006
87	January 2006
69	December 2005
102	November 2005
55	October 2005
54	September 2005
35	August 2005
28	July 2005
87	June 2005
111	May 2005
18	April 2005
68	March 2005
100	February 2005
43	January 2005
18	December 2004
109	November 2004
74	October 2004
34	September 2004
56	August 2004
21	July 2004
28	June 2004
67	May 2004
35	April 2004
3	March 2004
29	January 2004
19	December 2003
31	November 2003
144	October 2003
57	September 2003
150	August 2003
17	July 2003
9	June 2003
40	May 2003
133	April 2003
24	March 2003
42	February 2003
31	January 2003
31	December 2002
36	November 2002
3	October 2002
25	September 2002
7	August 2002
5	July 2002
88	June 2002
22	May 2002
32	April 2002
30	March 2002
17	February 2002
14	January 2002
80	December 2001
19	November 2001
19	October 2001
3	September 2001
23	August 2001
72	July 2001
15	June 2001
9	May 2001
12	April 2001
34	March 2001
55	February 2001
26	January 2001
16	December 2000
63	November 2000
37	October 2000
9	September 2000
84	August 2000
9	July 2000
32	June 2000
21	May 2000
5	April 2000
26	March 2000
21	February 2000
1	January 2000
3	November 1999
26	October 1999
19	September 1999
32	August 1999
28	July 1999
30	June 1999
1	April 1999
7	March 1999
1	January 1999
2	December 1998
5	August 1998
1	May 1998
10	April 1998