Klaus Umbach | 2 Feb 2010 08:24
Picon
Favicon

Autorefresh with multiple accounts

Hello list,

I want to use autorefresh with multiple accounts. This "just doesn't work"
if I say "-a Personal" or "-a Work" autorefresh works fine, but when I use
"-a Work,Personal", it just uses Personal. What could I have doen wrong
here?

attached you find my .offlineimaprc.

Cheers
	Klaus
# Sample minimal config file.  Copy this to ~/.offlineimaprc and edit to
# suit to get started fast.

[general]
accounts = Work,Personal
ui = Noninteractive.Basic
#ui = MACHINE.MACHINEUI
holdconnectionopen = yes
metadata = ~/.offlineimap
pythonfile=~/.MyConfigs/offlineimap.py
fsync = false
autorefresh = 1

[mbnames]
enabled = yes
filename = /tmp/muttrc.mailboxes
header = "mailboxes " 
(Continue reading)

Dominic LoBue | 3 Feb 2010 09:43
Picon

Re: X-OfflineIMAP header and Exchange

On Fri, Jan 29, 2010 at 8:07 AM, Dave Mussulman <mussulma <at> gmail.com> wrote:
> Hi gang,
>
> I'd like to revisit the per-message dynamic header X-OfflineIMAP-xxxx
> issue because now I'm getting pressure from my Exchange admins about
> filling up the MAPI header tables.
>
> I found http://software.complete.org/software/issues/show/114 but had a
> few questions about the patch.
>
> The patch only seems to address the X-OfflineIMAP-* header when it's
> created.  Wouldn't a patch also need to modify wherever that header
> gets parsed/read/sync'd?  Or is just changing how it's created good
> enough?
>
> What's the impact of making this change on a large existing sync'd
> maildir?  Would my messages show up as new again?  Anything else to look
> out for before trying the patch?
>
> I'm running OfflineIMAP 6.1.2 because I like the IDLE feature.  The
> patch looks easy enough to apply; I just don't understand what
> OfflineIMAP does under the hood with that header.  My usage is pretty
> vanilla - one Remote Exchange store to a local Maildir.  No other
> instances or sync'ing.
>
> Thoughts?
>
> Dave
>
>
(Continue reading)

John Goerzen | 3 Feb 2010 15:52
Favicon

Re: X-OfflineIMAP header and Exchange

Dominic LoBue wrote:
> I am working on doing a rewrite of OfflineIMAP, and since John hasn't
> responded to your question yet I'll field it.

Hi Dominic,

I wasn't aware of that, but I have repeatedly tried to hand of 
"maintainer of OfflineIMAP" mantle to someone else.  What's the status 
of your project?  I would be happy to hand it off entirely to someone 
qualified and dedicated.

> Before I start responding to your questions though, let me begin by
> saying that I am still unsure myself of why exactly the X-OfflineIMAP
> header is constructed the way it is. I searched through the mailing

Primarily because it seemed easy and safe to start with.  Exchange is 
rather broken in what it's doing, but adapting to it shouldn't hurt 
anything.  That said, I work on this and many other projects on a 
volunteer basis and haven't had the time to work around this bug in a 
server that I never use myself.

I haven't had the chance to review the gmane links just now though.

If you develop a well-tested patch, I would be happy to apply it.

Thanks,

-- John

_______________________________________________
(Continue reading)

Dave Mussulman | 3 Feb 2010 18:46
Picon

Re: X-OfflineIMAP header and Exchange

On Wed, Feb 3, 2010 at 2:43 AM, Dominic LoBue <dom.lobue <at> gmail.com> wrote:
On Fri, Jan 29, 2010 at 8:07 AM, Dave Mussulman <mussulma <at> gmail.com> wrote:
> I found http://software.complete.org/software/issues/show/114 but had a
> few questions about the patch.
>
> The patch only seems to address the X-OfflineIMAP-* header when it's
> created.  Wouldn't a patch also need to modify wherever that header
> gets parsed/read/sync'd?  Or is just changing how it's created good
> enough?
>
> What's the impact of making this change on a large existing sync'd
> maildir?  Would my messages show up as new again?  Anything else to look
> out for before trying the patch?

My point: would you be willing to work with me to resolve this problem
once and for all? I'll be able to answer all your questions if you
agree.

Dominic, thanks.  I'm willing to work with you; if I didn't care, I wouldn't have mailed the list.  :)

 
If you do not want to work with me, then to answer your question on
what effect the patch would have on your existing mail, I see several
possible scenarios:
1. All emails using the X-OfflineIMAP header are duplicated.
2. All emails using the X-OfflineIMAP header are deleted.
3. Nothing happens.

I don't think there's a chance in hell scenario 3 will occur though.

I made the change on Monday, and "nothing happened".  I backed up my maildir first, but I didn't change the metadata dirs.  As far as I can tell, everything's working as before.  No dupes, no deletions, etc.  I made the change because I wasn't seeing the headers consistently anyway -- some older messages had them, most newer didn't, which I guessed goes back to Exchange's mailbox store LHS header store being at or above quota.

My only weird observation is I don't see the X-OfflineIMAP-* headers on my maildir copies of the mail, or when I IMAP in (with Thunderbird or mutt directly) but I do see them in Outlook when looking at the message headers.  My running hunch right now is OfflineIMAP wasn't getting those headers anyway, and was running alright without them.  (Although it was setting them on the messages it saved back to the server.)

I confirmed after my change that the new messages sync'd back have just the X-OfflineIMAP: LHS header, but again, I don't see that outside of Outlook.

If you're still interested in my data, let me know and we can take this off list.  Thanks.

Dave
_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap
Dominic LoBue | 4 Feb 2010 07:52
Picon

Re: X-OfflineIMAP header and Exchange

On Wed, Feb 3, 2010 at 9:46 AM, Dave Mussulman <mussulma <at> gmail.com> wrote:
> On Wed, Feb 3, 2010 at 2:43 AM, Dominic LoBue <dom.lobue <at> gmail.com> wrote:
>>
>> On Fri, Jan 29, 2010 at 8:07 AM, Dave Mussulman <mussulma <at> gmail.com>
>> wrote:
>> > I found http://software.complete.org/software/issues/show/114 but had a
>> > few questions about the patch.
>> >
>> > The patch only seems to address the X-OfflineIMAP-* header when it's
>> > created.  Wouldn't a patch also need to modify wherever that header
>> > gets parsed/read/sync'd?  Or is just changing how it's created good
>> > enough?
>> >
>> > What's the impact of making this change on a large existing sync'd
>> > maildir?  Would my messages show up as new again?  Anything else to look
>> > out for before trying the patch?
>>
>> My point: would you be willing to work with me to resolve this problem
>> once and for all? I'll be able to answer all your questions if you
>> agree.
>
> Dominic, thanks.  I'm willing to work with you; if I didn't care, I wouldn't
> have mailed the list.  :)
>
>
>>
>> If you do not want to work with me, then to answer your question on
>> what effect the patch would have on your existing mail, I see several
>> possible scenarios:
>> 1. All emails using the X-OfflineIMAP header are duplicated.
>> 2. All emails using the X-OfflineIMAP header are deleted.
>> 3. Nothing happens.
>>
>> I don't think there's a chance in hell scenario 3 will occur though.
>
> I made the change on Monday, and "nothing happened".  I backed up my maildir
> first, but I didn't change the metadata dirs.  As far as I can tell,
> everything's working as before.  No dupes, no deletions, etc.  I made the
> change because I wasn't seeing the headers consistently anyway -- some older
> messages had them, most newer didn't, which I guessed goes back to
> Exchange's mailbox store LHS header store being at or above quota.
>
> My only weird observation is I don't see the X-OfflineIMAP-* headers on my
> maildir copies of the mail, or when I IMAP in (with Thunderbird or mutt
> directly) but I do see them in Outlook when looking at the message headers.
> My running hunch right now is OfflineIMAP wasn't getting those headers
> anyway, and was running alright without them.  (Although it was setting them
> on the messages it saved back to the server.)
>
> I confirmed after my change that the new messages sync'd back have just the
> X-OfflineIMAP: LHS header, but again, I don't see that outside of Outlook.
>
> If you're still interested in my data, let me know and we can take this off
> list.  Thanks.
>
> Dave
>

Dave,

I'm glad to hear that it is working; and yes, I definitely would like
to see your metadata.

As to why your local messages are unmodified, this makes sense
actually. Since the strings that make up  the X-OfflineIMAP header are
generated by calculating a hash of the email itself, putting the
header string into the local copy of the email would change what that
string would come out to the next time the generator is run. Its a
mathematical chicken and the egg dilemma.

This means that either that string is generated from scratch every
time OfflineIMAP needs it, or it is stored in the metadata.

When you email the metadata I highly recommend sending it just to me
instead of the rest of the list.

Thanks!

--

-- 
Dominic LoBue

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap

Dominic LoBue | 4 Feb 2010 08:46
Picon

Re: X-OfflineIMAP header and Exchange

On Wed, Feb 3, 2010 at 6:52 AM, John Goerzen <jgoerzen <at> complete.org> wrote:
> Dominic LoBue wrote:
>>
>> I am working on doing a rewrite of OfflineIMAP, and since John hasn't
>> responded to your question yet I'll field it.
>
> Hi Dominic,
>
> I wasn't aware of that, but I have repeatedly tried to hand of "maintainer
> of OfflineIMAP" mantle to someone else.  What's the status of your project?
>  I would be happy to hand it off entirely to someone qualified and
> dedicated.
>

John,

I actually sent you an email about this a month or two back, but I
never heard back from you.

The current status is I am in the low-level design stage. You know,
"measure twice, cut once" and all that. I already have the high-level
concept in my head of how I want to approach the problem, what I'm
specifically working on is figuring out how best to implement my ideas
with the "lessons learned" already in OfflineIMAP. I also have
implemented IDLE in imaplib, twisted's IMAP4Client, and another IMAP
library that I like.

My goals in the rewrite are to make OfflineIMAP safer (no lost
emails), more robust, inclusion of all "lessons learned" already in
OfflineIMAP, and able to recover from failure and unexpected data.

I plan to do this in a number of ways:
-Make sync process more granular: per-message syncing instead of per-folder.
-Switch metadata format from a bunch of regular text files to a single
bsddb database.
-Use worker design pattern.

There's more, but that's just off the top of my head. I also have a
bunch of other improvements and features I plan to add: NTLM
authentication, support for keyring module so passwords aren't stored
in plaintext (http://pypi.python.org/pypi/keyring/0.2), inotify, IDLE,
support for moving mail between folders (should save a lot of
bandwidth for people who move mail between folders).

Other features I want to add, but are a bit on the far side of realistic:
-SASL support (libgsasl)
-mail archival feature (mail older than X days is permanently moved
off the server and moved into size efficient container)
-provide a maildir API for mail clients (works with "only sync x days
worth of mail" feature so that if a client wants to read a message
that is older than x days, OfflineIMAP downloads it).

>> Before I start responding to your questions though, let me begin by
>> saying that I am still unsure myself of why exactly the X-OfflineIMAP
>> header is constructed the way it is. I searched through the mailing
>
> Primarily because it seemed easy and safe to start with.  Exchange is rather
> broken in what it's doing, but adapting to it shouldn't hurt anything.  That
> said, I work on this and many other projects on a volunteer basis and
> haven't had the time to work around this bug in a server that I never use
> myself.
>
> I haven't had the chance to review the gmane links just now though.
>
> If you develop a well-tested patch, I would be happy to apply it.
>
> Thanks,
>
> -- John
>

Considering how much work I have cut out for myself in the rewrite, I
don't plan on backporting any of the new features.

--

-- 
Dominic LoBue

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap

Christoph Höger | 4 Feb 2010 15:40
Picon

Re: X-OfflineIMAP header and Exchange

Hi Dominic,

since John wants to orphan offlineimap for some time now, how about
making your rewrite offlineimap 7?

A fork would be a bad idea IMO. This way you get the userbase and some
more helpers (you could e.g. use my dynamic plugin patch that John still
did not find the time to apply). Also a switch to python 3 seems at the
time.

regards

Christoph

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap
John Goerzen | 4 Feb 2010 17:42
Favicon

Re: X-OfflineIMAP header and Exchange

Christoph Höger wrote:
> Hi Dominic,
> 
> since John wants to orphan offlineimap for some time now, how about
> making your rewrite offlineimap 7?
> 
> A fork would be a bad idea IMO. This way you get the userbase and some
> more helpers (you could e.g. use my dynamic plugin patch that John still
> did not find the time to apply). Also a switch to python 3 seems at the
> time.

This is something of a question of semantics.  Obviously at this point
Dominic doesn't have code that is a full replacement for OfflineIMAP, so
calling it "OfflineIMAP 7" would be rather confusing.

If/when he does produce a finished product, that has the features and
stability of what we have now and a migration path, then I'd be happy to
"bless" it as OfflineIMAP 7 and hand the maintainer mantle over to him.

-- John

> 
> regards
> 
> Christoph
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> OfflineIMAP-project mailing list
> OfflineIMAP-project <at> lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project
> 
> OfflineIMAP homepage: http://software.complete.org/offlineimap

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap

Gmane