zydronium/imapsync
1
0
Fork 0
mirror of https://github.com/imapsync/imapsync.git synced 2025-07-25 11:38:15 +02:00
Code Issues Projects Releases Packages Wiki Activity

1.727

Browse source
This commit is contained in:
Nick Bebout 2016-09-19 10:17:24 -05:00
parent 3eaac56812
commit 137242e609
114 changed files with 10852 additions and 8980 deletions
Download patch file Download diff file Expand all files Collapse all files

20
FAQ.d/.htaccess
View file

@ -1,20 +0,0 @@
# $Id: .htaccess,v 1.1 2016/01/18 18:13:48 gilles Exp gilles $
AddDescription "<b>Domino</b>." FAQ.Domino.txt
AddDescription "<b>Dovecot</b>." FAQ.Dovecot.txt
AddDescription "<b>Duplicated</b> messages issues." FAQ.Duplicates.txt
AddDescription "<b>Exchange 20xx</b> and <b>Office365</b>." FAQ.Exchange.txt
AddDescription "<b>Changing folders names</b>." FAQ.Folders_Mapping.txt
AddDescription "<b>Flags</b>." FAQ.Flags.txt
AddDescription "<b>Gmail</b> accounts." FAQ.Gmail.txt
AddDescription "<b>ISP</b>." FAQ.ISP.txt
AddDescription "<b>Massive/bulk migrations</b>." FAQ.Massive.txt
AddDescription "<b>Selecting messages</b>." FAQ.Messages_selection.txt
AddDescription "<b>Oracle-UCS</b>." FAQ.Oracle-UCS.txt
AddDescription "<b>Security</b>." FAQ.Security.txt
AddDescription "<b>SmarterMail</b>." FAQ.SmarterMail.txt
AddDescription "<b>Various imap server</b> softwares." FAQ.Various_Server_Softwares.txt
AddDescription "<b>XOAUTH2</b> (<b>Gmail</b>)." FAQ.XOAUTH2.txt

1
FAQ.d/.htaccess Symbolic link
View file

@ -0,0 +1 @@
htaccess.txt

23
FAQ.d/FAQ.Archiving.txt Normal file
View file

@ -0,0 +1,23 @@
#!/bin/cat
$Id: FAQ.Archiving.txt,v 1.2 2016/05/09 13:03:14 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
============================================
Imapsync tips about archiving messages.
============================================
=======================================================================
Q. How to move emails from one IMAP folder to another either on the
same IMAP server or a different one?
For example, all messages older than 1 year, move from INBOX to Archive.
R1. Solution:
imapsync ... --minage 365 --folder Inbox --f1f2 INBOX=Archive --delete --noexpungeaftereach
Add option --noexpungeaftereach if speed is a concern.
Add option --delete if you want to move messages, instead of just copy/sync them.
R2. See also file FAQ.Folders_Mapping.txt

25
FAQ.d/FAQ.Connection.txt
View file

@ -1,25 +0,0 @@
#!/bin/cat
$Id: FAQ.Connection.txt,v 1.2 2016/03/16 13:33:51 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
============================================
Imapsync tips about connection issues.
============================================
=======================================================================
Q.How to test a ssl imap connection without imapsync?
R1.Use openssl command like the following,
an example with imap.gmail.com server:
openssl s_client -connect imap.gmail.com:993
The previous command is an interactive connection, hit ctrl-c
to finish it. If you want to finish it gently, then use:
{ sleep 1; echo "a LOGOUT"; } | openssl s_client -connect imap.gmail.com:993
=======================================================================

49
FAQ.d/FAQ.Contacts_Calendars.txt Normal file
View file

@ -0,0 +1,49 @@
#!/bin/cat
$Id: FAQ.Contacts_Calendars.txt,v 1.3 2016/05/09 13:37:21 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
=========================================================
Imapsync issues about syncing Contacts & Calendars.
=========================================================
=======================================================================
Q. Can I copy or sync Calendar or Contacts with imapsync?
R. No, Imapsync can't migrate Contacts and Calendars.
It's because most IMAP servers don't get contacts and calendars
events via IMAP. In other words, messages synced by imapsync from
Calendars or Contacts folders are not used by email servers to set
or get the contacts or calendars.
No way via IMAP, no way via imapsync.
But see next question.
=======================================================================
Q. How can I copy or synchronize Calendars or Contacts?
R1. It can't be done with imapsync. See previous question for explanations.
R2. It can be done, depending on the email server softwares used.
a) From Exchange to Exchange, export contacts and calendar to
PST format files on host1 and import them on host2.
b) From Gmail to Gmail, export and import calendars in ical format,
extension for those files is .ics.
Contacts can be copied using a csv file. See the help page
http://support.google.com/mail/bin/topic.py?hl=en&topic=1669027
c) Etc. Search the web. There's also specific tools and paid services.
There's no silver bullet to migrate Calendars and Contacts,
if you find one, tell me!
Also consider asking experts at Sumatra company
https://twitter.com/sumatra_dev
http://sumatra.com/
d) I plan to make tools for that but so far nothing has began
more than this discussion:
http://www.linux-france.org/prj/imapsync_list/msg01797.html
http://www.linux-france.org/prj/imapsync_list/msg01811.html
=======================================================================

19
FAQ.d/FAQ.Dates.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Dates.txt,v 1.2 2016/03/07 02:08:12 gilles Exp gilles $
$Id: FAQ.Dates.txt,v 1.3 2016/07/27 13:14:30 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -25,21 +25,26 @@ but not with:
A thing to keep in mind, imapsync does not touch any byte of messages
unless told to do so by option --regexmess or --pipemess
Messages on both parts should be identical.
Messages on both parts should be exactly identical.
Now I explain the whole picture about dates of messages.
There are several different dates for any message.
First, there is the "Date:" header. Most of the time, this date is set
by the MUA (MUA means Mail User Agent) it can be Outlook, Thunderbird,
a webmail or Mutt. The Date header is usually the date the message was written
First, there is the "Date:" header, which belongs to the message itself.
You can see this "Date:" header line by asking View->Message Source
in most email client software (Ctrl-u in Thunderbird).
Most of the time, this date is set by the MUA, MUA stands for "Mail User Agent".
MUA can be Outlook, Thunderbird, a webmail or Mutt etc.
The Date header is usually the date the message was written
or sent the first time, at the writer side.
This Date: header line is never changed by any transfer or copy.
This "Date:" header line is never changed by any transfer or copy.
If an email reader uses the "Date:" header for displaying the date of a
message then no problem should arise.
Another date is the internal date. In IMAP, the internal date is handled
by the server and normally it corresponds to the arrival date in the mailbox.
by the server and normally it corresponds to the arrival date in the mailbox.
This date is not part of the email message itself, it's a metadata kept
by the imap server.
The IMAP protocol allows the internal date to be set by a email client
when a message is delivered by the imap APPEND command.
Imapsync synchronizes internal dates by default, internal dates on host2

47
FAQ.d/FAQ.Duplicates.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Duplicates.txt,v 1.9 2016/03/08 21:17:21 gilles Exp gilles $
$Id: FAQ.Duplicates.txt,v 1.10 2016/04/17 19:06:39 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -20,21 +20,50 @@ R0.
Normally and by default, imapsync doesn't generate duplicates.
So if it does generate duplicates it means a problem occurs
with message identification. It happens sometimes with IMAP
servers changing the "Message-Id" line or "Received:" in
the header part of messages.
servers changing the "Message-Id" header line or one or more
of the "Received:" header lines in the header part of messages.
Imapsync uses "Message-Id" header line and "Received:" header
lines to identify messages on both sides.
R1.
You can use option --useuid, with it, imapsync won't use
header lines to compare messages in folders.
Keep in mind it uses a local cache.
A first solution is to use option --useuid.
With option --useuid, imapsync doesn't use header lines
to identify and compare messages in folders.
Instead of some headers, --useuid tell imapsync to use
the imap UIDs given by imap servers on bith sides.
To avoid duplicates on next runs, imapsync uses a local cache
where it keeps UIDs already transfered.
imapsync ... --useuid
There is an issue when --useuid is not used the first time.
A big issue with --useuid is that it doesn't generate duplicates if
used from the first time but it does generate duplicates after a previous
run without --useuid (because it uses a different method to identify
the messages). A solution? it depends. --delete2 solves this problem
if you are permitted to use it.
run without --useuid (because then uses a different method to identify
the messages).
A solution? Two solutions.
The easiest is --delete2
if you are permitted to use it. Option --delete2 removes messages on host2
that are not on host1 so with --delete2 you go for resyncing all
messages again but all previously transferred messages are deleted,
and also messages previously there without imapsync.
So --useuid --delete2 is easy to remove duplicates but not for
all context.
A second solution, better if R2 works (see R2 below), is to build
the cache before using --useuid
First sync:
imapsync ... --useheader "Message-Id" --addheader --usecache
Next syncs:
imapsync ... --useuid
imapsync ... --useuid
...
R2.
Best way if you can follow it.

35
FAQ.d/FAQ.Emptying.txt Normal file
View file

@ -0,0 +1,35 @@
#!/bin/cat
$Id: FAQ.Emptying.txt,v 1.4 2016/04/24 00:07:47 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
============================================
Imapsync tip to empty an account.
============================================
=======================================================================
Q. How to delete all emails of all folders of an account with imapsync?
R. Use the same account on both sides with option --delete
--noexpungeaftereach
Option --noexpungeaftereach is just to speed up the deletion.
Example:
./imapsync \
--host1 test.lamiral.info --user1 test --password1 secret \
--host2 test.lamiral.info --user2 test --password2 secret \
--delete --noexpungeaftereach
=======================================================================
Q. How to delete all folders of an account with imapsync?
R. Use an already empty account as source and sync this account
to the account you want to clear, with --delete2folders
Example:
./imapsync ... --delete2folders --foldersizes
=======================================================================

21
FAQ.d/FAQ.Exchange.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Exchange.txt,v 1.20 2016/03/03 15:44:17 gilles Exp gilles $
$Id: FAQ.Exchange.txt,v 1.23 2016/08/10 01:29:37 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -91,7 +91,6 @@ In case you don't have it you can still use
--regexmess 's,(.{9900}),$1\r\n,g'
instead of --maxlinelengthcmd 'reformime -r7'
=======================================================================
Q. How to sync from XXX to Office365
@ -143,7 +142,7 @@ R3. Try with a classic email client like Thunderbird and the same
=======================================================================
Q. Exchange fails with "User is authenticated but not connected".
R. "The message User is authenticated but not connected is due to a
R1. "The message User is authenticated but not connected is due to a
bug in the Exchange server's IMAP implementation. If the client
presents a valid user name but an invalid password, the server
accepts the login, but subsequent commands fail with the
@ -152,6 +151,10 @@ http://unix.stackexchange.com/questions/164823/user-is-authenticated-but-not-con
Thanks to James Abbottsmith for this link and explanation at
https://github.com/imapsync/imapsync/issues/32#issuecomment-153561647
R2. Miguel Alameda reported understanding and solving this issue
like this, the context was admin/authuser:
"The admin user had not permission in the target mailbox."
=======================================================================
Q. Exchange fails with "BAD Command received in Invalid state".
@ -276,7 +279,7 @@ R. It looks like limiting 4 messages per second is enough to never
imapsync ... --maxmessagespersecond 4
======================================================================
Q. How to migrate from or to Exchange 2007/2010 with an
Q. How to migrate from or to Exchange 2007/2010/2013 with an
admin/authuser account?
R. The tricks comes from Michele Marcionelli and Benjamin Priestman:
@ -326,9 +329,6 @@ The command that I used for imapsync is:
--user2 "windows-domain\admin-user\aaaa" ^
--password2 admin-password ...
======================================================================
Q. How to migrate from or to Office 365 with an admin/authuser account?
@ -336,8 +336,7 @@ Note from Yago Torres Fernandez:
(a working command using admin/authuser on host2 Office 365)
imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated@domain.com ^
--password2 XXXX --ssl2 --exclude "Shared Folders" ^
--regextrans2 "s#Trash$#Papelera#" --regextrans2 "s#^Deleted Items$#Papelera#"
--password2 XXXX --ssl2 ^
but previous in Office365 you must do something like that, using powershell:
@ -351,8 +350,6 @@ Same behavior with the --authuser2 option.
See also:
http://www.linux-france.org/prj/imapsync_list/msg02203.html
======================================================================
Q. How to migrate from or to Exchange 2003 with an admin/authuser
account?
@ -387,8 +384,6 @@ In case you are not aware:
omitted in our case
- Use --debugimap when testing initial connectivity, if necessary
=======================================================================
Q. Couldn't create folder [trash] "Mailbox already exists".

12
FAQ.d/FAQ.Folders_Selection.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Folders_Selection.txt,v 1.3 2016/02/07 22:01:56 gilles Exp gilles $
$Id: FAQ.Folders_Selection.txt,v 1.4 2016/06/07 22:19:04 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -9,6 +9,16 @@ This documentation is also at http://imapsync.lamiral.info/#doc
By default, Imapsync syncs all folders, one by one, in alphanumeric order.
The IMAP protocol has a specific way to code folder names,
when these names use non-ascii 7bit characters.
Imapsync uses the same code as IMAP.
In order to well specify them on the command line you will be
helped by the complete folder list printed by imapsync at each run.
The left column is what you have to use without the first enclosing
square brackets, the right column is the human utf8 view.
======================================================================
Q. How can I sync only one folder?

148
FAQ.d/FAQ.Gmail.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Gmail.txt,v 1.21 2016/03/17 20:14:14 gilles Exp gilles $
$Id: FAQ.Gmail.txt,v 1.25 2016/07/22 00:18:35 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -11,6 +11,8 @@ Questions anwswered in this FAQ are:
Q. Can I use imapsync to transfer from or to Gmail accounts?
Q. How many days does it take to transfer X GB?
Q. How to synchronize from Gmail to Gmail?
Q. How to synchronize from XXX to Gmail?
@ -50,8 +52,18 @@ it has to be allowed in the Gmail configuration part:
-> Enable IMAP
=======================================================================
Q. How to synchronize from Gmail to Gmail?
Q. How many days does it take to transfer X GB?
R. Basically it takes X days to transfer X GB per account.
Gmail has usage limits per day
http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
From the previous link,
it's 2X days to upload X GB to Gmail
it's X/2 days to download X BG from Gmail
but that's the theory.
=======================================================================
Q. How to synchronize from Gmail to Gmail?
R. Use the following example:
@ -64,7 +76,7 @@ R. Use the following example:
--ssl2 \
--user2 account2@gmail.com \
--password2 gmailsecret2 \
--exitwhenover 500000000 \
--maxbytespersecond 10000 \
--automap \
--exclude "\[Gmail\]$"
@ -74,17 +86,18 @@ Explanations:
--ssl1 --ssl2 are mandatory since Gmail only supports
imap ssl connections.
--exitwhenover 500000000 ( ~500 MB ) option is here to avoid
locking or errors when imap transfers exceed maximum limit.
--maxbytespersecond 10000 ( 10 kBytes/s ) option is here to avoid
locking or errors when imap transfers exceed Gmail maximum limit.
See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
--exitwhenover is not mandatory in the sense Gmail may allow you
to use an upper value than 500 MB without disconnections; I don't
know the hard value, it seems to vary, so just have some tries
and report me what you discover in case you detect something
reliable.
--maxbytespersecond 10000 is not mandatory in the sense Gmail may
allow you to use an upper value than 1 GBytes per 24h without
disconnections.
--automap is not mandatory but it's a feature to automatically
map folder names based on the Gmail user configuration itself, par account.
It will save manual folder names changes or the use of --regextrans2
or --f1f2 to map folder names.
For example, imap folder "[Gmail]/Sent Mail" may be mapped as one of
E-mails enviados
Enviada
@ -97,7 +110,7 @@ Skickat
Verzonden
etc.
on both sides, host1 or host2, maybe differently, sometimes in
incomprehensible alphabets, a headache for imap sysadmins.
incomprehensible alphabets, a headache for imap sysadmins.
See a listing here:
http://stackoverflow.com/questions/2185391/localized-gmail-imap-folders/2185548#2185548
@ -112,34 +125,32 @@ Q. How to synchronize from XXX to Gmail?
R. Use the following example:
imapsync --host1 mail.oldhost.com \
imapsync --host1 mail.oldhost.com \
--user1 my_email@oldhost.com \
--password1 password \
--password1 password \
--host2 imap.gmail.com \
--user2 my_email@gmail.com \
--password2 password \
--ssl2 \
--exitwhenover 500000000 \
--maxbytespersecond 10000 \
--maxsize 25000000 \
--automap \
--expunge1 \
--automap \
--expunge1 \
--addheader \
--exclude "\[Gmail\]$" \
--regextrans2 "s/[ ]+/_/g" \
--regextrans2 "s/[\^]/_/g" \
--regextrans2 "s/['\"\\\\]/_/g"
--exclude "\[Gmail\]$" \
--regextrans2 "s/[ ]+/_/g" \
--regextrans2 "s/['\^\"\\\\]/_/g"
Explanations:
--ssl2 is mandatory since Gmail only supports imap ssl connections.
--exitwhenover 500000000 option is here to avoid locking when
transfers exceed maximum limit.
--maxbytespersecond 10000 ( 10 kBytes/s ) option is here to avoid
locking or errors when imap transfers exceed Gmail maximum limit.
See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
--exitwhenover is not mandatory in the sense you may be able to
use an upper value than 500 MB without disconnections; I don't
know the hard value, it seems to vary, so just have some tries
and report me what you discover in case you detect something.
--maxbytespersecond 10000 is not mandatory in the sense Gmail may
allow you to use an upper value than 1 GBytes per 24h without
disconnections.
--maxsize 25000000 is mandatory since Gmail limits messages size
up to 25 MB. This value increases over time, it was 10 MB some
@ -147,7 +158,7 @@ years ago so you can try higher values. The Gmail page about
this limit is https://support.google.com/mail/answer/6584
--automap is optional but it will save manual folder names
changes or the use of --regextrans2 to map folder names.
changes or the use of --regextrans2 or --f1f2 to map folder names.
--expunge1 is optional. It deletes messages marked \Deleted on host1.
Imapsync syncs messages with all their flags, Gmail takes the messages
@ -203,7 +214,7 @@ R. Use this example:
--user2 tata \
--password2 tatasecret \
--ssl1 \
--exitwhenover 2500000000 \
--maxbytespersecond 20000 \
--useheader="X-Gmail-Received" \
--useheader "Message-Id" \
--automap \
@ -219,14 +230,12 @@ Explanations:
--ssl1 is mandatory since Gmail only supports imap ssl connections.
--exitwhenover 2500000000 (2.5 GB) option is here to avoid
locking when transfers exceed maximum limit.
--maxbytespersecond 20000 ( 20 kBytes/s ) option is here to avoid
locking or errors when imap transfers exceed Gmail maximum limit.
See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
--exitwhenover is not mandatory in the sense you may be able to
use an upper value than 2.5 GB without disconnections; I don't
know the hard value, it seems to vary, so just have some tries
and report me what you discover in case you detect something.
--maxbytespersecond 20000 is not mandatory in the sense Gmail may
allow you to use an upper value than 2 GBytes per 24h without
disconnections.
--useheader="X-Gmail-Received" --useheader "Message-Id" are not
mandatory. I use them because I found (several years ago, it may
@ -245,10 +254,6 @@ get rid of this "[Gmail]" part with that.
You can select folders exported to imap within the gmail preferences,
for example you may unselect all "System labels".
--exitwhenover option is here to avoid locking when transfers
exceed maximum limit.
See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
--skipcrossduplicates is optional but it can save Gigabytes of hard
disk memory. Within imap protocol, Gmail presents Gmail labels as
folders, so a message labeled "Work" "ProjectX" "Urgent" ends up
@ -270,37 +275,24 @@ label CanWait and only it.
--skipcrossduplicates, will only put in "[Gmail]/All Mail"
the messages that are not labeled at all.
=======================================================================
Q. Can I use the Extension of the SEARCH command: X-GM-RAW described at
https://support.google.com/mail/answer/7190?hl=en
https://developers.google.com/gmail/imap_extensions#extension_of_the_search_command_x-gm-raw
R. Sure. Example, to search only emails with attachment and in unread state:
On Unix:
imapsync ... --search 'X-GM-RAW "has:attachment in:unread"'
On Windows:
imapsync.exe ... --search "X-GM-RAW ""has:attachment in:unread"""
=======================================================================
Q. How to avoid the [IMAP] prefix on Gmail side?
How to stop creating folder with this prefix?
Any switch we can use? e.g. [IMAP]/Archive
R. No switch in imapsync since [IMAP]/ prefix is done by Gmail,
it might be configurable within Gmail parameters configuration.
=======================================================================
Q. I can't authenticate with Gmail via IMAP
and Gmail says "Please log in via your web browser"
R0. My current settings on two Gmail account are made
with 0) 1) 2)
0) Login via a browser to the Gmail account concerned.
1) The 2-step verification is OFF
https://myaccount.google.com/security?hl=en&pli=1&nlr=1#signin
2) Access for less secure apps is turned ON
https://www.google.com/settings/security/lesssecureapps
https://support.google.com/accounts/answer/6010255?hl=en
R1. See Coert Grobbelaar solution:
https://security.stackexchange.com/questions/86404/how-do-i-interact-with-google-to-import-email-via-imapsync
https://web.archive.org/web/20150906230041/http://security.stackexchange.com/questions/86404/how-do-i-interact-with-google-to-import-email-via-imapsync
R2. I had the same issue one time (mars 2015) logging to Gmail
with imapsync.
@ -309,9 +301,6 @@ so I logged for this account via a web browser,
it asked me to receive a code via a mobile, I said yes,
I entered the code and everything went ok.
R3. Use https://www.google.com/settings/security/lesssecureapps
(thanks to Flavio Zarur)
See https://support.google.com/accounts/answer/6010255?hl=en
=======================================================================
Q. Can not open imap connection on [imap.gmail.com]:
@ -353,6 +342,29 @@ https://support.google.com/mail/answer/78755?hl=en
To really remove messages in folder "[Gmail]/All Mail", they have to
be moved to the "Trash" folder and be deleted from "Trash".
=======================================================================
Q. Can I use the Extension of the SEARCH command: X-GM-RAW described at
https://support.google.com/mail/answer/7190?hl=en
https://developers.google.com/gmail/imap_extensions#extension_of_the_search_command_x-gm-raw
R. Sure. Example, to search only emails with attachment and in unread state:
On Unix:
imapsync ... --search 'X-GM-RAW "has:attachment in:unread"'
On Windows:
imapsync.exe ... --search "X-GM-RAW ""has:attachment in:unread"""
=======================================================================
Q. How to avoid the [IMAP] prefix on Gmail side?
How to stop creating folder with this prefix?
Any switch we can use? e.g. [IMAP]/Archive
R. No switch in imapsync since [IMAP]/ prefix is done by Gmail,
it might be configurable within Gmail parameters configuration.
=======================================================================
Q. Does imapsync have the capability to do 2 stage authentication?

2
FAQ.d/FAQ.Massive.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Massive.txt,v 1.8 2016/02/07 17:21:40 gilles Exp gilles $
$Id: FAQ.Massive.txt,v 1.9 2016/03/25 19:59:45 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc

30
FAQ.d/FAQ.Memory.txt Normal file
View file

@ -0,0 +1,30 @@
#!/bin/cat
$Id: FAQ.Memory.txt,v 1.3 2016/05/09 13:01:22 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
============================================
Imapsync tips about memory issues.
============================================
=======================================================================
Q. I got an "Out of memory" error message then imapsync crashes.
How to fix that?
R0. Add option --debugmemory to debug memory issues.
R1. It's probably a very big message on the host1 account.
Before imapsync release 1.688 memory usage is at least 5 times
the biggest message size. then 2 times with release 1.688.
So a 700 MB message will crunch 3.5 GB of memory.
Starting with 1.688 it will crunch 1.4 GB of memory.
R2. Usually "Out of memory" errors are related to old days,
to old Mail::IMAPClient module releases, before 3.26.
At that time memory usage was around 17 times the biggest message.
Look at imapsync output first lines to get the Mail::IMAPClient
release used. Then upgrade Mail::IMAPClient Perl module if needed.
=======================================================================

164
FAQ.d/FAQ.Messages_Selection.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Messages_Selection.txt,v 1.5 2016/02/05 08:22:26 gilles Exp gilles $
$Id: FAQ.Messages_Selection.txt,v 1.6 2016/04/18 12:45:20 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -65,12 +65,27 @@ Full explanation:
past|----maxage+++++minage---->now (intersection)
past|++++minage-----maxage++++>now (union)
C.1 By default,
option --maxage is implemented as a --search SENTSINCE
RFC 3501 says: SENTSINCE <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is within or later than the specified date.
Option --minage is implemented as a --search SENTBEFORE
RFC 3501 says: SENTBEFORE <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is earlier than the specified date.
If --noabletosearch is on then --minage and --maxage deal
with the internal dates given by a FETCH imap command but
not the Date: header. Internal date is the arrival date
in the mailbox.
=======================================================================
Q. I want to sync messages based on their UID.
R. First have in mind that UIDs are uniq only per folder, so work this
R. First have in mind that UIDs are unique only per folder, so work this
way only with one folder at a time, with --folder option.
To show UIDs, there is the --debugLIST parameter.
@ -91,7 +106,148 @@ UIDs 20000 20002 20004:
imapsync ... --search1 'NOT OR OR UID 20000 UID 20002 UID 20004' --folder INBOX
If you search n UIDs then you have to put n-1 OR in the search line.
That's IMAP.
If you search n UIDs then you have to put n-1 OR in the search line,
that's IMAP.
=======================================================================
Q. What is the selection criteria available with --search option?
R. The list of search criteria are listed below, an excerpt from RFC3501.
http://www.faqs.org/rfcs/rfc3501.html
6.4.4. SEARCH Command
...
ALL
All messages in the mailbox; the default initial key for
ANDing.
ANSWERED
Messages with the \Answered flag set.
BCC <string>
Messages that contain the specified string in the envelope
structure's BCC field.
BEFORE <date>
Messages whose internal date (disregarding time and timezone)
is earlier than the specified date.
BODY <string>
Messages that contain the specified string in the body of the
message.
CC <string>
Messages that contain the specified string in the envelope
structure's CC field.
DELETED
Messages with the \Deleted flag set.
DRAFT
Messages with the \Draft flag set.
FLAGGED
Messages with the \Flagged flag set.
FROM <string>
Messages that contain the specified string in the envelope
structure's FROM field.
HEADER <field-name> <string>
Messages that have a header with the specified field-name (as
defined in [RFC-2822]) and that contains the specified string
in the text of the header (what comes after the colon). If the
string to search is zero-length, this matches all messages that
have a header line with the specified field-name regardless of
the contents.
KEYWORD <flag>
Messages with the specified keyword flag set.
LARGER <n>
Messages with an [RFC-2822] size larger than the specified
number of octets.
NEW
Messages that have the \Recent flag set but not the \Seen flag.
This is functionally equivalent to "(RECENT UNSEEN)".
NOT <search-key>
Messages that do not match the specified search key.
OLD
Messages that do not have the \Recent flag set. This is
functionally equivalent to "NOT RECENT" (as opposed to "NOT
NEW").
ON <date>
Messages whose internal date (disregarding time and timezone)
is within the specified date.
OR <search-key1> <search-key2>
Messages that match either search key.
RECENT
Messages that have the \Recent flag set.
SEEN
Messages that have the \Seen flag set.
SENTBEFORE <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is earlier than the specified date.
SENTON <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is within the specified date.
SENTSINCE <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is within or later than the specified date.
SINCE <date>
Messages whose internal date (disregarding time and timezone)
is within or later than the specified date.
SMALLER <n>
Messages with an [RFC-2822] size smaller than the specified
number of octets.
SUBJECT <string>
Messages that contain the specified string in the envelope
structure's SUBJECT field.
TEXT <string>
Messages that contain the specified string in the header or
body of the message.
TO <string>
Messages that contain the specified string in the envelope
structure's TO field.
UID <sequence set>
Messages with unique identifiers corresponding to the specified
unique identifier set. Sequence set ranges are permitted.
UNANSWERED
Messages that do not have the \Answered flag set.
UNDELETED
Messages that do not have the \Deleted flag set.
UNDRAFT
Messages that do not have the \Draft flag set.
UNFLAGGED
Messages that do not have the \Flagged flag set.
UNKEYWORD <flag>
Messages that do not have the specified keyword flag set.
UNSEEN
Messages that do not have the \Seen flag set.
=======================================================================

44
FAQ.d/FAQ.Passwords_on_Windows.txt Normal file
View file

@ -0,0 +1,44 @@
#!/bin/cat
$Id: FAQ.Passwords_on_Windows.txt,v 1.2 2016/05/10 19:19:16 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
======================================================
Imapsync issues with passwords on Windows.
======================================================
http://www.robvanderwoude.com/escapechars.php
http://stackoverflow.com/questions/3288552/how-can-i-escape-an-exclamation-mark-in-cmd-scripts
=======================================================================
Q. On Windows, some passwords contain $ characters. Login fails.
R1. Enclose passwords between ""
imapsync ... --password1 "zzz$zz$$z"
R2. Prefix each $ character with a ^ since ^ is the escape character
on Windows
imapsync ... --password1 zzz^$zz^$^$z
For a password that is exactly the 8 characters string $%&<>|^"
you have to enter
imapsync ... --password1 "$%%&<>|^"^"
=======================================================================
Q. On Windows, some passwords begin with an equal = character.
Login fails. What can I do?
R. Use twice equals == characters instead; For example, if =secret
is the password then use:
imapsync ... --password1 ==secret
or even
imapsync ... --password1 "==secret"

32
FAQ.d/FAQ.Security.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: FAQ.Security.txt,v 1.6 2016/02/07 21:49:52 gilles Exp gilles $
# $Id: FAQ.Security.txt,v 1.9 2016/08/19 17:53:21 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -64,32 +64,40 @@ b OK 73 good day (Success)
The client part to type is "a LOGIN ..." and "b LOGOUT" without
the double-quotes.
=======================================================================
Q.How to test a ssl imap connection without imapsync?
R1.Use openssl command like the following,
an example with imap.gmail.com server:
openssl s_client -crlf -connect imap.gmail.com:993
The previous command is an interactive connection, hit ctrl-c
to finish it. If you want to finish it gently, then use:
{ sleep 2; echo "a logout"; sleep 1; } | openssl s_client -crlf -connect imap.gmail.com:993
======================================================================
Q. Imapsync used to use SSL_VERIFY_PEER now it uses SSL_VERIFY_NONE.
How can I change this back to the more secure SSL_VERIFY_PEER?
R1. In function "sub set_ssl", replace
IO::Socket::SSL::SSL_VERIFY_NONE()
by
IO::Socket::SSL::SSL_VERIFY_PEER()
R2. After imapsync 1.673, for example
R2. After imapsync 1.673,
to set SSL_verify_mode to SSL_VERIFY_PEER on host1
and SSL_verify_mode to SSL_VERIFY_NONE on host2
imapsync ... --ssl1 --ssl2 \
--sslargs1 SSL_verify_mode=1 \
--sslargs2 SSL_verify_mode=0 \
--sslargs2 SSL_verify_mode=0
See perldoc IO::Socket::SSL for all possibilities, also at
http://search.cpan.org/perldoc?IO%3A%3ASocket%3A%3ASSL
C1. Don't do this manual change in function "sub set_tls",
it won't work by principle, because tls is done AFTER
the application level connexion is established.
It might be possible you need an extra:
--sslargs1 SSL_ca_file=/etc/ssl/certs/ca-certificates.crt
to help the ssl software verifying the server certificate.
The file ca-certificates.crt may be elsewhere on your system, even
named differently.
=======================================================================
Q. How can I manually test a login via ssl?

30
FAQ.d/FAQ.Use_addheader.txt Normal file
View file

@ -0,0 +1,30 @@
#!/bin/cat
$Id: FAQ.Use_addheader.txt,v 1.1 2016/06/01 12:25:56 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
============================================
Imapsync --addheader suggestion.
============================================
=======================================================================
Q. What means this log message:
"Host1 Sent/1234 ignored (no wanted headers so we ignore this message.
To solve this: use --addheader)"
R. In order to sync messages from one account to another Imapsync has
to identify messages on both sides, and compare them.
Imapsync can use two completely different methods to identify
messages on each account.
The first one, the default one, uses some headers of messages,
Real world syncs with various imap servers have bring to use
two headers "Message-Id:" and "Received:". It works for most
messages except for special folders like Sent or Draft where
messages in those folders don't have "Message-Id:" nor
"Received:" headers.
Here comes -addheader option. Option --addheader adds
a "Message-Id" header consisting of the imap UID of the message
on the host1 folder, like "Message-Id: 12345@imapsync".
This way, messages are well identified on both sides.
=======================================================================

35
FAQ.d/FAQ.User_Concurrent_Access.txt Normal file
View file

@ -0,0 +1,35 @@
#!/bin/cat
$Id: FAQ.User_Concurrent_Access.txt,v 1.1 2016/07/22 00:00:08 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================
Imapsync and user concurrent access.
=======================================
=======================================================================
Q. What happens if a user access the mailbox during the transfer process?
R. Think all potential problems with a sync between
a master/source/host1 and a slave/destination/host2
What happens when a user access syncing accounts depends
on what he does, where he does and when:
* Reading: no problem anywhere, anytime.
* Writing at source/host1:
* adding folder => will need a resync after
* deleting folder => deletion not synced unless --delete2folders
* deleting message => deletion not synced unless --delete2
* adding message => will need a resync
* message flag change => will need a resync
* Writing at destination/host2:
* adding folder => no problem, unless --delete2folders is used
* deleting folder => might reappear by the sync
* adding message => no problem, unless --delete2 is used
* deleting message => might reappear by the sync
* message flag change => might be canceled by the sync
=======================================================================

63
FAQ.d/FAQ.XOAUTH2.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.XOAUTH2.txt,v 1.9 2016/01/28 14:34:15 gilles Exp gilles $
$Id: FAQ.XOAUTH2.txt,v 1.11 2016/07/27 23:08:19 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -14,8 +14,64 @@ Q. Is XOAUTH2 authentication available with imapsync?
R. Yes, but XOAUTH2 has been really tested on Unix systems,
less profund on Windows but it should work.
Two file formats are available from Gmail: json and pk12.
json is easier to manage than pk12.
=======================================================================
Q. How to use XOAUTH2 to globally authenticate gmail users?
Q. Imapsync XOAUTH2 fails with the following message, how to fix that?
{
"error": "unauthorized_client",
"error_description": "Unauthorized client or scope in request."
}
R. In order to work you also have to allow the service https://mail.google.com/
in the Google client API manager for OAUTH2.
"Select OAuth 2.0 scopes:"
=======================================================================
Q. How to use XOAUTH2 via a json file to globally authenticate gmail users?
R. Unless you use an imapsync binary like imapsync.exe or imapsync_bin_Darwin,
Perl modules needed for xoauth2 are:
Crypt::OpenSSL::RSA
JSON
JSON::WebToken
LWP
HTML::Entities
Encode::Byte
A easy way to install or upgrade Perl modules is to use cpanm command,
also called cpanminus. On Linux it is something like
sudo cpanm JSON::WebToken JSON Crypt::OpenSSL::RSA LWP HTML::Entities Encode::Byte
The json file patch code and explanation comes from Secretion at
https://github.com/imapsync/imapsync/pull/68
Here is a complete example for Gmail. It is a little stupid
since it is the same account as source and destination but
it's just to get the picture for xoauth2 authentication.
All xoauth2 config is given via the --password1 parameter.
It has the form:
--password1 secret.xoauth2.json
where secret.xoauth2.json is the json file given by Gmail.
imapsync \
--host1 imap.gmail.com --ssl1 --user1 gilles.lamiral@gmail.com \
--password1 secret.xoauth2.json --authmech1 XOAUTH2 \
--host2 imap.gmail.com --ssl2 --user2 gilles.lamiral@gmail.com \
--password2 secret.xoauth2.json --authmech2 XOAUTH2 \
--justlogin --debug
Use your own xoauth2 values.
=======================================================================
Q. How to use XOAUTH2 via pk12 file to globally authenticate gmail users?
R. First, consider the XOAUTH2 feature at a prototype level.
@ -115,7 +171,8 @@ R2. In case you still have to use XOAUTH, here is the method:
--password2 secret2 \
--authmech2 XOAUTH
Google Apps is a paid service, but you can try it for 30 days without any cost.
Google Apps is a paid service, but you can try it for 30 days without any cost,
or you could try, time goes on on free trial offers.
Some notes about configuring the Google Apps XOAUTH:

31
FAQ.d/htaccess.txt Normal file
View file

@ -0,0 +1,31 @@
# $Id: htaccess.txt,v 1.10 2016/07/22 00:20:13 gilles Exp gilles $
AddDescription "<b>Back to Imapsync main page</b>." ..
AddDescription "<b>Archiving tips</b>." FAQ.Archiving.txt
AddDescription "<b>Connection issues</b>." FAQ.Connection.txt
AddDescription "<b>Contacts & Calendars issues</b>." FAQ.Contacts_Calendars.txt
AddDescription "<b>Dates issues</b>." FAQ.Dates.txt
AddDescription "<b>Domino</b>." FAQ.Domino.txt
AddDescription "<b>Dovecot</b>." FAQ.Dovecot.txt
AddDescription "<b>Duplicated</b> messages issues." FAQ.Duplicates.txt
AddDescription "<b>Emptying</b> an account." FAQ.Emptying.txt
AddDescription "<b>Exchange 20xx</b> and <b>Office365</b>." FAQ.Exchange.txt
AddDescription "<b>Changing folders names</b>." FAQ.Folders_Mapping.txt
AddDescription "<b>Selecting folders</b>." FAQ.Folders_Selection.txt
AddDescription "<b>Flags</b>." FAQ.Flags.txt
AddDescription "<b>Gmail</b> accounts." FAQ.Gmail.txt
AddDescription "<b>ISP tips</b>." FAQ.ISP.txt
AddDescription "<b>Massive/bulk migrations</b>." FAQ.Massive.txt
AddDescription "<b>Memory issues</b>." FAQ.Memory.txt
AddDescription "<b>Password & special characters on Windows</b>." FAQ.Passwords_on_Windows.txt
AddDescription "<b>Selecting messages</b>." FAQ.Messages_Selection.txt
AddDescription "<b>Oracle-UCS</b>." FAQ.Oracle-UCS.txt
AddDescription "<b>Security</b>." FAQ.Security.txt
AddDescription "<b>SmarterMail</b>." FAQ.SmarterMail.txt
AddDescription "<b>User concurrent access</b>." FAQ.User_Concurrent_Access.txt
AddDescription "<b>Why use --addheader?</b>." FAQ.Use_addheader.txt
AddDescription "<b>Various imap server</b> softwares." FAQ.Various_Server_Softwares.txt
AddDescription "<b>XOAUTH2</b> (<b>Gmail</b>)." FAQ.XOAUTH2.txt
AddDescription "<b>Where this Description column comes from</b>." htaccess.txt