This commit is contained in:
Nick Bebout 2019-07-02 18:17:46 -05:00
parent eafd200ec0
commit 62531f58cd
300 changed files with 9212 additions and 88495 deletions

View file

@ -1,11 +1,14 @@
#!/bin/cat
$Id: FAQ.Admin_Authentication.txt,v 1.2 2017/01/06 14:11:13 gilles Exp gilles $
$Id: FAQ.Admin_Authentication.txt,v 1.6 2018/09/09 10:58:16 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Admin_Authentication.txt
==============================================
Imapsync tips about admin authentication.
==============================================
=======================================================================
Imapsync tips about imap admin authentication.
=======================================================================
It can be useful to authenticate without knowing each user password.
Using an admin account to authenticate is not a standard feature
@ -17,16 +20,21 @@ if an admin account is actually configured and
finally check an authentication via this admin account for
a standard user.
Known imap server software supporting admin authentication:
Known imap server software supporting imap admin authentication:
* Exchange 2003/2007/2010/2013. See the file FAQ.Exchange.txt
* Office365. See the file FAQ.Exchange.txt
* Gmail. See the file FAQ.XOAUTH2.txt
* Dovecot. See the file FAQ.Dovecot.txt
* Zimbra. See the file FAQ.Zimbra.txt
* UW-imap. See the file FAQ.General.txt
* Cyrus-imap. See the file FAQ.General.txt
* Sun Java Enterprise System/SunOne/iPlanet. See the file FAQ.General.txt
Known imap server software NOT supporting imap admin authentication:
* Domino Lotus Notes.
* SmartCloud Notes.
=======================================================================
=======================================================================

View file

@ -1,11 +1,13 @@
#!/bin/cat
$Id: FAQ.Archiving.txt,v 1.7 2018/04/24 00:16:21 gilles Exp gilles $
$Id: FAQ.Archiving.txt,v 1.13 2018/12/09 21:26:25 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Archiving.txt
============================================
Imapsync issues and tips about archiving.
============================================
=======================================================================
Imapsync issues and tips about archiving
=======================================================================
Questions answered in this FAQ are:
@ -28,10 +30,10 @@ Q. Can I archive different accounts on the same destination account,
R. Yes. Use --subfolder2
--subfolder2 str : Move whole host1 folders hierarchy under the
host2 folder str.
It is done it by adding two --regextrans2 options before
all others. Add --debug to see what's really going on.
--subfolder2 str : Syncs the whole host1 folders hierarchy under the
host2 given folder str.
(It does it internally by adding two --regextrans2
options before all others.)
Example:
@ -48,17 +50,19 @@ in the right place, ie, the subfolder "foo".
=======================================================================
Q. How to move emails from one IMAP folder to another either on the
same IMAP server or a different one?
For example, all messages older than 1 year, move from INBOX to Archive.
For example, move all messages older than 1 year from INBOX to Archive.
R1. Solution:
imapsync ... --minage 365 --folder Inbox --f1f2 INBOX=Archive --delete --noexpungeaftereach
imapsync ... --minage 365 --folder Inbox --f1f2 INBOX=Archive --delete1 --noexpungeaftereach
Add option --noexpungeaftereach if speed is a concern.
Add option --delete if you want to move messages, instead of just copy/sync them.
Add option --delete1 if you want to move messages, instead of just copy/sync them.
R2. See also file FAQ.Folders_Mapping.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Archiving.txt
=======================================================================
Q. Can imapsync be used to maintain a local offline copy of a
mailbox from host1, eg for backup purposes, so that if the
@ -71,14 +75,16 @@ R2. Imapsync can't but rick-sanders-imap-tools can!
See https://github.com/andrewnimmo/rick-sanders-imap-tools
Back up and restore IMAP accounts with imapdump.pl and dumptoIMAP.pl
It looks like imapdump.pl has options to avoid duplicates and be
in an update mode. There are options -u -U
[-u] Don't dump messages already dumped
[-U] Don't dump message if it already exists in the dump directory
I wonder the difference, I've never used Rick's tools.
Remember that doing backups without trying the restore process
is more dangerous than doing no backups at all. No backups makes
people act in a safer way, well, usually...
=======================================================================
=======================================================================
=======================================================================

View file

@ -1,25 +1,37 @@
#!/bin/cat
$Id: FAQ.Authentication_failure.txt,v 1.5 2017/09/04 11:03:57 gilles Exp gilles $
$Id: FAQ.Authentication_failure.txt,v 1.9 2019/01/11 13:13:08 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Admin_Authentication.txt
=======================================================================
Imapsync authentication issues
=======================================================================
Questions answered in this FAQ are:
Q. Imapsync fails with the following error, what can I do?
Host1 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 NO [AUTHENTICATIONFAILED] Authentication failed
Q. Imapsync fails with the following error, what can I do?
Host2 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 BAD Invalid characters in atom
=======================================================================
Q. Imapsync fails with the following error, what can I do?
Host1 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 NO [AUTHENTICATIONFAILED] Authentication failed
One over four imapsync syncs ends up quickly with the error message
"Authentication failed" or "NO LOGIN failed" or similar.
Authentication failure is the primary failure with imapsync
and since nothing tangible can be done without authentication,
this stage must succeed to go further.
R. One over four imapsync syncs ends up quickly with the error message
"Authentication failed" or "NO LOGIN failed" or similar.
Authentication failure is the primary failure with imapsync
and since nothing tangible can be done without authentication,
this stage must succeed to go further.
Here are some advices to get you pass this difficult stage of authentication:
* Triple check each credendial parameter, there are three parameters at each side:
* Triple check each credential parameter, there are three parameters at each side:
* triple check --host1
* triple check --user1
* triple check --password1
@ -40,15 +52,44 @@ Here are some advices to get you pass this difficult stage of authentication:
Option --showpasswords shows passwords again when the IMAP dialog
is dumped by --debugimap option. Search for a line like
"Sending: 2 LOGIN test1 secret1" (secret1 is the password here)
* It is sometimes very hard to quote correctly unusual characters,
especially on Windows. See
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
The quicker trick might be to change the password temporarilly
The quickest trick may be to change the password temporally
with easy characters like the classical alphabet, a long
string will still ensure strong security.
string still ensure strong security.
* Sometimes some servers announce they support LOGIN but it
actually fails because the authentication mechanism
working is something else like CRAM-MD5 or PLAIN.
So:
* Try --authmech1 CRAM-MD5 (or --authmech2 CRAM-MD5)
* Try --authmech1 PLAIN (or --authmech2 PLAIN)
* If you want to play manually the IMAP protocol on your server,
here is an example with the command telnet, user test1
and password secret1 on the host test.lamiral.info.
telnet test.lamiral.info 143
c1 LOGIN test1 "secret1"
c2 LOGOUT
Same example using an ssl conneection:
telnet-ssl -z ssl test.lamiral.info 993
c1 LOGIN test1 "secret1"
c2 LOGOUT
=======================================================================
Q. Imapsync fails with the following error, what can I do?
Host2 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 BAD Invalid characters in atom
R. It might be a Dovecot imap server. The password string might
contain special characters that Dovecot doesn't like. Change them.
=======================================================================
=======================================================================

View file

@ -1,23 +1,25 @@
#!/bin/cat
$Id: FAQ.Contacts_Calendars.txt,v 1.7 2018/03/07 04:26:04 gilles Exp gilles $
$Id: FAQ.Contacts_Calendars.txt,v 1.11 2018/09/26 22:25:32 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Contacts_Calendars.txt
=========================================================
Imapsync issues about syncing Contacts & Calendars.
=========================================================
=======================================================================
Imapsync issues about syncing Contacts & Calendars.
=======================================================================
Questions answered in this FAQ are:
Q. Can I copy or sync Calendar or Contacts with imapsync?
Q. Can I copy or sync Calendars or Contacts with imapsync?
Q. Can I copy or sync Chat messages with imapsync?
Q. How can I avoid copying Calendar or Contacts folders?
Q. How can I copy or synchronize Calendars or Contacts?
Now the questions again with their answers.
=======================================================================
Q. Can I copy or sync Calendar or Contacts with imapsync?
Q. Can I copy or sync Calendars or Contacts with imapsync?
R. No, Imapsync can't migrate Contacts and Calendars.
It's because most IMAP servers don't get contacts and calendars
@ -26,7 +28,14 @@ R. No, Imapsync can't migrate Contacts and Calendars.
or get the contacts or calendars.
No way via IMAP, no way via imapsync.
So it's a good idea to avoid syncing contacts and calendars.
But see next question.
But see the question below
"How can I copy or synchronize Calendars or Contacts?".
=======================================================================
Q. Can I copy or sync Chat messages with imapsync?
R. No, Imapsync can't migrate Chat messages (those in GSuite for example)
The explanation is the same as with Calendars or Contacts.
=======================================================================
Q. How can I avoid copying Calendar or Contacts folders?
@ -43,10 +52,11 @@ R1. You can avoid synchronizing Calendar or Contacts folders with
Q. How can I copy or synchronize Calendars or Contacts?
R1. Synchronizing Calendars or Contacts can't be done with imapsync.
See the first question for detailed explanations.
For detailed explanations, see the question above
"Can I copy or sync Calendars or Contacts with imapsync?".
R2. Synchronizing Calendars or Contacts can be done, not
with imapsync, depending on the email server softwares used.
with imapsync, depending on the imap server software tool used.
a) From Exchange to Exchange, export contacts and calendar to
PST format files on host1 and import them on host2.

View file

@ -1,11 +1,13 @@
#!/bin/cat
$Id: FAQ.Dates.txt,v 1.7 2018/03/07 04:12:37 gilles Exp gilles $
$Id: FAQ.Dates.txt,v 1.9 2018/11/04 09:27:17 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Dates.txt
===============================
Imapsync tips about dates.
===============================
=======================================================================
Imapsync tips about dates.
=======================================================================
Questions answered in this FAQ are:
@ -32,32 +34,35 @@ but not with:
- Zimbra
- Gmail
A thing to keep in mind, imapsync does not touch any byte of messages
unless told to do so by option --regexmess or --pipemess
Messages on both parts should be exactly identical.
A first thing to keep in mind.
Imapsync does not touch any byte of messages unless told to
do so by special options like --regexmess or --pipemess.
Messages on both parts should be exactly identical because
imapsync does a verbatim sync of all messages.
Now I explain the whole picture about dates of messages.
There are several different dates for any message.
There are several different dates attached to any message.
First, there is the "Date:" header, which belongs to the message itself.
You can see this "Date:" header line by asking View->Message Source
You can see this "Date:" header line by asking "View->Message Source"
in most email client software (Ctrl-u in Thunderbird).
Most of the time, this date is set by the MUA, MUA stands for "Mail User Agent".
Most of the time, this "Date:" header date is set by the MUA,
MUA stands for "Mail User Agent".
MUA can be Outlook, Thunderbird, a webmail or Mutt etc.
The Date header is usually the date the message was written
The "Date:" header is usually the date the message was written
or sent the first time, at the writer side.
This "Date:" header line is never changed by any transfer or copy.
If an email reader uses the "Date:" header for displaying the date of a
message then no problem should arise.
Another date is the internal date. In IMAP, the internal date is handled
by the server and normally it corresponds to the arrival date in the mailbox.
by the imap server and normally it corresponds to the arrival date in the mailbox.
This date is not part of the email message itself, it's a metadata kept
by the imap server.
The IMAP protocol allows the internal date to be set by a email client
when a message is delivered by the imap APPEND command.
Imapsync synchronizes internal dates by default, internal dates on host2
should then be the same as the internal dates on host1.
Imapsync synchronizes these internal dates, by default, so internal dates
on host2 should then be the same as the internal dates on host1.
If an email reader uses the internal date for displaying the date of a
message then the sync date problem only occurs when the host2 server software
ignores the internal date given by imapsync during the APPEND imap
@ -73,14 +78,14 @@ has done its best to keep all the dates synchronized. Bad luck.
Solutions:
a) Use a another email client or configure it in order it sorts messages
by sent date, the "Date:" header line.
a) Use another email client or configure it in order that it sorts messages
by sent dates, the "Date:" header line.
b) Use a imap server that respects the imap RFC and accepts
the internal date set by imapsync.
the internal date set by imapsync.
c) Try to understand why the email client software shows another date
than the "Date:" header line.
than the "Date:" header line.
=======================================================================
Q. Is there a way to set any message with

View file

@ -1,11 +1,14 @@
#!/bin/cat
# $Id: FAQ.Domino.txt,v 1.9 2018/03/13 03:22:55 gilles Exp gilles $
# $Id: FAQ.Domino.txt,v 1.10 2018/05/24 11:34:30 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Domino.txt
=============================
Imapsync tips for Domino.
=============================
=======================================================================
Imapsync tips for Domino.
=======================================================================
=======================================================================

View file

@ -1,14 +1,17 @@
#!/bin/cat
$Id: FAQ.Dovecot.txt,v 1.5 2017/01/06 14:21:06 gilles Exp gilles $
$Id: FAQ.Dovecot.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Dovecot.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips for Dovecot. Specific issues and solutions.
=======================================================================
======================================================================
=======================================================================
Q. How to migrate to Dovecot with an admin/MasterUser account?
R. Dovecot uses the same syntax as uw-imap
@ -18,4 +21,5 @@ R. Dovecot uses the same syntax as uw-imap
To setup a Dovecot MasterUser see
http://wiki2.dovecot.org/Authentication/MasterUsers
======================================================================
=======================================================================
=======================================================================

View file

@ -1,18 +1,14 @@
#!/bin/cat
$Id: FAQ.Duplicates.txt,v 1.14 2018/05/02 12:21:54 gilles Exp gilles $
$Id: FAQ.Duplicates.txt,v 1.18 2019/02/14 17:22:10 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
=======================================================================
Imapsync tips about duplicated messages issues.
Imapsync tips about duplicated messages issues.
=======================================================================
=======================================================================
Q. How can I remove duplicates in a lone account?
R. Just run imapsync on the same account with option --delete2duplicates,
ie, with host1 == host2 and user1 == user2
=======================================================================
Q. How can I know if imapsync will generate duplicates on a second run?
@ -37,7 +33,7 @@ Messages skipped : 1555
=======================================================================
Q: Multiple copies, duplicates, when I run imapsync twice ore more.
R0.
R0. First, some explanations to understand the issue.
Normally and by default, imapsync doesn't generate duplicates.
So if it does generate duplicates it means a problem occurs
with message identification. It happens sometimes with IMAP
@ -46,8 +42,39 @@ of the "Received:" header lines in the header part of messages.
By default, Imapsync uses "Message-Id" header line and
"Received:" header lines to identify messages on both sides.
R1.
A first solution is to use option --useuid.
R1. This solution is R3 simplified.
A quick practical solution is to change the way imapsync
identify messages that works most of the time. But since
you're reading this because you encountered duplicates issue,
let's check this solution in a safe way.
First use the same commmand with additionnal options:
imapsync ... --useheader "Message-Id" --dry
The previous command does nothing real but it will show you
if imapsync handles duplicates in a better way.
The criterium is to search at the end of the sync for a line
like this one:
Messages skipped : 1555
where 1555 is an example but reflects mostly the number
of all messages already transferred.
If you end with:
Messages skipped : 0
don't go on, it means imapsync is still suffering to
identify messages.
If you end with many messages skipped then it's very
good and now you can safely resync the mailboxe
and get rid of the dupplicates messages on host2 with:
imapsync ... --useheader "Message-Id" --delete2duplicates
End of the problem!
R2.
A second solution is to use option --useuid.
With option --useuid, imapsync doesn't use header lines
to identify and compare messages in folders.
Instead of some headers, --useuid tell imapsync to use
@ -63,7 +90,7 @@ used from the first time but it does generate duplicates after a previous
run without --useuid (because it then uses a different method to identify
the messages).
A solution? Two solutions.
A solution? Two solutions.
The easiest is --delete2 if you are permitted to use it.
Option --delete2 removes messages on host2
@ -74,7 +101,7 @@ So --useuid --delete2 is easy to remove duplicates but not for
all contexts. The host2 account must be considered as a strict
replication of the host1 account, ie, not active.
A second solution, better if R2 works (see R2 below), is to build
A second solution, better if R3 works (see R3 below), is to build
the cache before using --useuid
First sync:
@ -87,7 +114,7 @@ Next syncs:
imapsync ... --useuid
...
R2.
R3.
Best way if you can follow it.
Multiple copies of the emails on the destination server. Some IMAP
servers (Domino for example) change some headers for each message
@ -161,5 +188,11 @@ R2. With option --useuid imapsync doesn't use headers to identify
messages on both sides but it uses their imap uid identifier.
In that case duplicates on host1 are also transferred on host2.
=======================================================================
Q. How can I remove duplicates in a lone account?
R. Just run imapsync on the same account with option --delete2duplicates,
ie, with host1 == host2, user1 == user2, password1 == password2
=======================================================================
=======================================================================

View file

@ -1,10 +1,12 @@
#!/bin/cat
$Id: FAQ.Emptying.txt,v 1.5 2017/01/06 14:11:13 gilles Exp gilles $
$Id: FAQ.Emptying.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/
=======================================================================
Imapsync tip to empty an account.
Imapsync tip to empty an account.
=======================================================================
=======================================================================

View file

@ -1,11 +1,14 @@
$Id: FAQ.Exchange.txt,v 1.38 2018/03/21 16:26:23 gilles Exp gilles $
$Id: FAQ.Exchange.txt,v 1.49 2018/12/03 23:09:28 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt
=================================================================================
Imapsync tips for Exchange 20xx and Office365. Specific issues and solutions.
=================================================================================
=======================================================================
Imapsync tips for Exchange 20xx and Office365.
=======================================================================
Questions answered in this FAQ are:
@ -13,14 +16,26 @@ Q. Can I use imapsync to transfer from or to Exchange or Office365 accounts?
Q. How to sync from XXX to Exchange 2010/2013/2016
Q. How to sync from Office365 to XXX?
Q. How to sync from XXX to Office365
Q. For Office365 I have double and triple checked the username and
password spelling but I still get a "LOGIN failed". Any clue?
Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that? Happens with Office365 or Exchange 2016.
Q. Office365 throttles the sync and says:
"Request is throttled. Suggested Backoff Time: 299961 milliseconds".
What can I do with that?
Q. Office365 refuses to create the folder named "Files" with the error
"NO Folder name is reserved". What happens?
Q. Office365 users complain that a folder named "Files" contains
messages with no sender.
Q. Exchange fails with "User is authenticated but not connected".
Q. Exchange fails with "BAD Command received in Invalid state".
@ -41,9 +56,11 @@ Q. Exchange and Office365 have throttle mechanisms to limit any huge
usage. Sometimes imapsync transfers are too stressful for servers.
How to deal with that?
Q. How to migrate from or to Exchange 2007/2010 with an
Q. How to migrate from or to Exchange 2007/2010/2013 with an
admin/authuser account?
Q. How to migrate from or to Exchange 2016 with an admin/authuser account?
Q. How to migrate from or to Office 365 with an admin/authuser account?
Q. How to migrate from or to Exchange 2003 with an admin/authuser
@ -115,6 +132,18 @@ In case you don't have it you can still use
--regexmess 's,(.{9900}),$1\r\n,g'
instead of --maxlinelengthcmd 'reformime -r7'
=======================================================================
Q. How to sync from Office365 to XXX?
R. On Windows, use:
imapsync.exe ... --office1
On Unix, use:
imapsync ... --office1
=======================================================================
Q. How to sync from XXX to Office365
@ -136,7 +165,7 @@ On Windows, use:
or use (before imapsync release 1.836):
imapsync.exe ... ^
--maxsize 45000000 ^
--maxmessagespersecond 4 ^
--maxmessagespersecond 2 ^
--regexflag "s/\\Flagged//g" ^
--disarmreadreceipts ^
--regexmess "s,(.{10500}),$1\r\n,g"
@ -150,7 +179,7 @@ or use (before imapsync release 1.836):
imapsync ... \
--maxsize 45000000 \
--maxlinelength 10500 \
--maxmessagespersecond 4 \
--maxmessagespersecond 2 \
--regexflag 's/\\Flagged//g' \
--disarmreadreceipts \
--maxlinelengthcmd 'reformime -r7'
@ -193,9 +222,13 @@ R2. Miguel Alameda reported understanding and solving this issue
=======================================================================
Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that?
What can I do with that? Happens with Office365 or Exchange 2016
R. Office 365 supports send/receive max message sizes of up to 150MB
R0. It happens with Office365 and Exchange 2016.
See R1 for Office365
See R2 for Exchange 2016
R1. Office365 supports send/receive max message sizes of up to 150MB
but you need to make changes in your tenant(s) to support it.
The following PowerShell command will increase the message sizes that
@ -216,14 +249,112 @@ values on the migration acounts in the source and target tenants.
Thanks to Sean McDougall, Ian Thomas & Matt Wilks from Toronto
for this FAQ item.
R2. With Exchange 2016
Thanks to Torsten Bergemann from Germany:
In order to have exchange accept big e-mails via imap, several size
limitations have to be removed on several levels of the exchange
software: For the organization (within exchange admin panel),
for each connector (with exchange admin panel), for each Postfach
(using exchange management shell).
For details and instructions, check
https://technet.microsoft.com/en-us/library/bb124345(v=exchg.160).aspx
https://technet.microsoft.com/en-us/library/hh529949(v=exchg.160).aspx
Unfortunately, the second link does not give instructions to remove the limit from imap4 service.
To solve it for Exchange 2013/2016, add the line
<add key="MaxReceiveSize" value="1024000000" />
to the file
C:\Program Files\Microsoft\Exchange Server\V14 [or V15]\ClientAccess\PopImap\Microsoft.Exchange.Imap4.exe.config
Please note that this change may be lost each time you install an
cumulative update. After the change, restart the services exchange
transport role, exchange information store and exchange imap4 Backend.
(Source: https://social.technet.microsoft.com/Forums/de-DE/2c07cf43-fe0e-4d22-b061-676a67c6d58d/exchange-2013-max-mailgre-per-imap?forum=exchange_serverde)
This change made our Exchange installation accept bigger e-mails
in order to remove the imapsync Exchange error message
"NO Maximum size of appendable message has been exceeded"
=======================================================================
Q. Office365 throttles the sync and says:
"Request is throttled. Suggested Backoff Time: 299961 milliseconds".
What can I do with that?
R. To solve the throttles issues from 0365, there's two solutions:
R1. Call Microsoft Office365 and ask them to remove the limits on your
mailboxes. That's not a joke, they do it for 90 days usually,
sometimes only after you reach the second technician you call,
the first one been not enough competent to understand what you're
talking about.
R2. Play with options --maxbytespersecond or --maxmessagespersecond
or --exitwhenover
imapsync ... --maxbytespersecond 100_000
imapsync ... --maxmessagespersecond 2
imapsync ... --exitwhenover 1_000_000_000
I don't know the upper value that avoid the default throttling from 0365
and I guess it changes over time.
=======================================================================
Q. Office365 refuses to create the folder named "Files" with the error
"NO Folder name is reserved". What happens?
R. The folder Files is a standard folder in Office365. It should not
be synced in IMAP. See the next question.
=======================================================================
Q. Office365 users complain that a folder named "Files" contains
messages with no sender.
R0. To fix this, add --exclude Files
imapsync ... --exclude Files
R1. This folder "Files" seems to be a standard folder in Exchange Online,
but it is not. The folder contains all attachments in every email
that is in the mailbox, but without any headers.
This causes some confusion for users as these appear in their
search results as duplicate lines but without the sender details
or even the message body.
This folder seems to be usually hidden so IMAP clients cant see it,
but for some reason sometimes it becomes visible.
R2. It looks to be a common problem with Exchange Online.
Im not sure what causes the folder to appear.
More info here:
https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_exchon-mso_o365b/exclude-the-exchange-online-system-folder-called/2adbdf84-db4a-4c7f-ac29-738757980a0d
https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_dep365-mso_o365b/no-sender-emails-in-files-folder/534bae8f-a7d7-4f5f-8ed7-5bad0d5fa23f
(This question/answer is taken quasi verbatim from Perttu Aaltonen)
=======================================================================
Q. Exchange fails with "BAD Command received in Invalid state".
R. This message might happens when authenticating without ssl nor tls.
Add --tls1 or --ssl1 if this error message comes from host1.
Add --tls2 or --ssl2 if this error message comes from host2.
R1. This message might happen when authenticating without ssl nor tls,
wich is rare now since imapsync tries to do ssl then tls by default.
Add --tls1 or else --ssl1 if this error message comes from host1.
Add --tls2 or else --ssl2 if this error message comes from host2.
R2. This message might also happen when authenticating with ssl and tls.
Don't use --tlsX and --sslX on the same side X (1 or 2), since it
asks for a double encryption, an encryption inside an encryption.
It won't work, you can't do a direct double encryption, few
servers allow that.
(Never add --tlsX and --sslX on the same side X (1 or 2), it won't work)
=======================================================================
Q. From XXX to Exchange 2013 or Office365, read receipts are all
resent again after a sync. Even for old messages. How can I fix that?
@ -271,6 +402,9 @@ R1. "SSL read or write error" happens sometimes, it isn't related to
R2. Another solution is to remove --tls or --ssl options for Exchange
and accept a clear text sync.
R3. See also the FAQ FAQ.SSL_errors.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
=======================================================================
Q. From XXX to Exchange 2010/2013 or Office365 I get this error message
sometimes: "BAD Command Argument Error 11". What does it mean?
@ -400,7 +534,6 @@ As it seems you need to use the mailNickname (e.g. alias) defined
for user2, not the username itself.
http://blogs.technet.com/b/exchange/archive/2004/03/31/105275.aspx
A little note from John Becker.
After you pointed out that I had issues with authorization, I realized that
as an admin user I don't have all the rights on the mailboxes.
@ -412,6 +545,32 @@ The command that I used for imapsync is:
--user2 "windows-domain\admin-user\aaaa" ^
--password2 admin-password ...
======================================================================
Q. How to migrate from or to Exchange 2016 with an admin/authuser account?
R. This might work:
imapsync ... --user2 user2 --authuser2 admin2 --password2 adminpassword2 ...
Note from
https://github.com/imapsync/imapsync/issues/136#issuecomment-397020248
The key is to not use the built-in Administrator account which
Microsoft seems to have blocked access to in imap.
I'm doing a sync to Exchange 2016 accounts and this works:
imapsync ... --ssl2 --user2 account@domain.com ^
--authuser2 accountwfullaccess@domain.com ^
--password2 "accountwfullaccesspw"
The admin account you use needs to have full access to the account(s)
you want to sync which you can setup in the gui or via powershell
and again - don't use administrator@domain.com - as that will never
work because IMAP LOGIN is disabled for that account apparently
under all circumstances.
======================================================================
Q. How to migrate from or to Office 365 with an admin/authuser account?
@ -421,7 +580,7 @@ Note from Yago Torres Fernandez:
imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated@domain.com ^
--password2 XXXX --ssl2 ^
but previous in Office365 you must do something like that, using powershell:
but previously in Office365 you must do something like that, using powershell:
Add-MailboxPermission -identity user_to_be_migrated@domain.com -user user_admin@domain.com -accessrights fullaccess -inheritancetype all

View file

@ -1,10 +1,12 @@
#!/bin/cat
$Id: FAQ.Flags.txt,v 1.17 2017/07/27 15:39:57 gilles Exp gilles $
$Id: FAQ.Flags.txt,v 1.20 2019/02/14 16:01:41 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt
=======================================================================
Imapsync tips about flags.
Imapsync tips about flags.
=======================================================================
Questions answered here are:
@ -270,8 +272,7 @@ Two solutions:
Q. Flags are resynced at each run for already synced/copied messages,
how can I avoid this feature?
R. It is not possible to avoid this feature by an option for now.
If you really need this then ask for it and I might code it.
R. Use the option --noresyncflags
=======================================================================

View file

@ -1,18 +1,24 @@
#!/bin/cat
$Id: FAQ.Folders_Mapping.txt,v 1.15 2017/11/13 00:01:14 gilles Exp gilles $
$Id: FAQ.Folders_Mapping.txt,v 1.17 2018/11/20 10:26:18 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips about changing folders names.
Imapsync tips about changing folders names.
=======================================================================
Folders names are by default reproduced identical except for
the prefix and the separator which are automatically adapted
for host2.
Before using --regextrans2 you should consider using --automap
and --f1f2 because they are simpler to understand and use.
This document gives most examples with the powerful but complex
option --regextrans2. Before using --regextrans2 you should
consider using --automap and --f1f2 because they are simpler
to understand and use.
--automap : guesses folders mapping, for folders like
"Sent", "Junk", "Drafts", "All", "Archive", "Flagged".
@ -20,10 +26,11 @@ and --f1f2 because they are simpler to understand and use.
--f1f2 str1=str2 : Force folder str1 to be synced to str2,
--f1f2 overrides --automap and --regextrans2.
--subfolder2 str : Move whole host1 folders hierarchy under this
host2 folder str .
It does it by adding two --regextrans2 options before
all others. Add --debug to see what's really going on.
--subfolder2 str : Syncs the whole host1 folders hierarchy under the
host2 folder named str.
(It does it internally by adding two
--regextrans2 options before all others.
Add --debug to see what's really going on.)

View file

@ -1,10 +1,13 @@
#!/bin/cat
$Id: FAQ.Folders_Selection.txt,v 1.6 2017/07/20 11:02:11 gilles Exp gilles $
$Id: FAQ.Folders_Selection.txt,v 1.7 2018/05/24 11:34:30 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Selection.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips to select folders.
Imapsync tips to select folders.
=======================================================================
By default, Imapsync syncs all folders, one by one, in alphanumeric order.

View file

@ -1,7 +1,9 @@
#!/bin/cat
$Id: FAQ.Folders_Sizes.txt,v 1.1 2017/09/29 11:10:53 gilles Exp gilles $
$Id: FAQ.Folders_Sizes.txt,v 1.2 2018/05/24 11:34:30 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Sizes.txt
=======================================================================
Imapsync tips to know folders sizes.

View file

@ -1,19 +1,19 @@
#!/bin/cat
# $Id: FAQ.General.txt,v 1.238 2018/03/07 04:26:04 gilles Exp gilles $
# $Id: FAQ.General.txt,v 1.240 2018/11/26 11:49:56 gilles Exp gilles $
=======================================================================
General FAQ for imapsync
=======================================================================
This document is also available at
https://imapsync.lamiral.info/#doc
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.General.txt
Questions answered in this FAQ are:
Q. Do I need to create IMAP mailboxes at the destination platform?
Q. Am I forced to publish the IMAP service on internet since the two
Q. Am I forced to publish the IMAP service on the Internet since the two
environment are not in the same location or same LAN?
Q. What are the most important differences between the Unix shell syntax
@ -120,12 +120,12 @@ R. Yes!
with the standard IMAP protocol. So you have to create them first.
=======================================================================
Q. Am I forced to publish the IMAP service on internet since the two
Q. Am I forced to publish the IMAP service on the Internet since the two
environment are not in the same location or same LAN?
R. The host where you run imapsync has to contact both servers via imap.
You are not obliged to publish the imap service on internet if the host
where you run imapsync can contact both imap servers via imap.
You are not obliged to publish the imap service on the Internet if
the host running imapsync can contact both imap servers via imap.
Use their names or their IP addresses.
=======================================================================

View file

@ -1,7 +1,9 @@
#!/bin/cat
$Id: FAQ.Gmail.txt,v 1.37 2018/03/23 19:07:37 gilles Exp gilles $
$Id: FAQ.Gmail.txt,v 1.50 2018/12/25 11:10:30 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
=======================================================================
Imapsync tips for Gmail accounts.
@ -11,6 +13,8 @@ Questions answered in this FAQ are:
Q. Can I use imapsync to transfer from or to Gmail accounts?
Q. Is there special configuration tips to facilitate a Gmail transfer?
Q. How many days does it take to transfer X GB?
Q. How to synchronize from Gmail to Gmail?
@ -25,6 +29,8 @@ Q. I can't authenticate with Gmail via IMAP
Q. Can not open imap connection on [imap.gmail.com]
Unable to connect to imap.gmail.com
Q. Can I transfer "Chat" messages with imapsync?
Q. Can I safely use --useuid for Gmail transfers?
Q. Gmail does not really delete messages in folder [Gmail]/All Mail
@ -59,6 +65,16 @@ it has to be allowed in the Gmail configuration part:
-> IMAP Access
-> Enable IMAP
=======================================================================
Q. Is there special configuration tips to facilitate a Gmail transfer?
R. Turned off "Conversation views".
With "Conversation views" turned off, Gmail shows the exact message
count for each folder and it matches what imapsync shows.
With "Conversation views" on, a single conversation can have
multiple messages and it doesn't match what imapsync presents.
(Thanks to Karthik.V for this input)
=======================================================================
Q. How many days does it take to transfer X GB?
@ -82,46 +98,56 @@ R. Basically it takes X days to transfer X GB per account.
This is either a disconnection with
"BYE Session expired, please login again"
or a very small rate, less than 1 Kib/s
https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
=======================================================================
Q. How to synchronize from Gmail to Gmail?
R. Use the following example:
./imapsync \
imapsync \
--user1 account1@gmail.com \
--password1 gmailsecret1 \
--user2 account2@gmail.com \
--password2 gmailsecret2 \
--gmail12
--gmail1 --gmail2
Or, replacing what does --gmail12:
Or, replacing what does --gmail1 --gmail2 (values are from imapsync 1.882):
./imapsync \
imapsync \
--host1 imap.gmail.com \
--ssl1 \
--user1 account1@gmail.com \
--password1 gmailsecret1 \
--host2 imap.gmail.com \
--ssl2 \
--user2 account2@gmail.com \
--password2 gmailsecret2 \
--maxbytespersecond 10000 \
--maxbytespersecond 20_000 \
--maxbytesafter 1_000_000_000 \
--maxsleep 2 \
--automap \
--exclude "\[Gmail\]$"
With --gmail1 --gmail2 you can override the parameters
that are activated by them, for example:
imapsync ... --gmail1 --gmail2 --maxbytespersecond 50_000
Explanations:
--ssl1 --ssl2 are mandatory since Gmail only supports
imap ssl connections.
--maxbytespersecond 10000 ( 10 kBytes/s ) option is here to avoid
--maxbytespersecond 20_000 ( 20 kBytes/s ) option is here to avoid
locking or errors when imap transfers exceed Gmail maximum limit.
See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
--maxbytespersecond 10000 is not mandatory in the sense Gmail may
--maxbytespersecond 20_000 is not mandatory in the sense Gmail may
allow you to use an upper value than 1 GBytes per 24h without
disconnections.
disconnections. If you count well, it should be 10_000, not 20_000.
--maxbytesafter 1_000_000_000 option is there because
option --maxbytespersecond 20000 is unnecessarily slow if the Gmail
account size is less than 1 GB.
--maxsleep 2 is to avoid sleeping more than 2 seconds when
the --maxbytespersecond value implies a long sleep between 2 copies.
--automap is not mandatory but it's a feature to automatically
map folder names based on the Gmail user configuration itself, par account.
@ -146,9 +172,12 @@ http://stackoverflow.com/questions/2185391/localized-gmail-imap-folders/2185548#
--exclude "\[Gmail\]$" is just there to avoid a warning error
when selecting this not used folder.
=======================================================================
Be aware that --gmail1 --gmail2 is a special case, it's not what
--gmail1 does plus what --gmail2 does when they are not invoked
together.
https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
=======================================================================
Q. How to synchronize from XXX to Gmail?
@ -159,9 +188,9 @@ imapsync --host1 mail.oldhost.com \
--password1 password \
--user2 my_email@gmail.com \
--password2 password \
--gmail2
--gmail2 --maxsize 35_651_584
Or, replacing what does --gmail2:
Or, replacing what does --gmail2 (values are from imapsync 1.882):
imapsync --host1 mail.oldhost.com \
--user1 my_email@oldhost.com \
@ -169,28 +198,40 @@ imapsync --host1 mail.oldhost.com \
--host2 imap.gmail.com \
--user2 my_email@gmail.com \
--password2 password \
--ssl2 \
--maxbytespersecond 10000 \
--maxsize 25000000 \
--maxbytespersecond 20_000 \
--maxbytesafter 1_000_000_000 \
--maxsleep 2 \
--maxsize 35_651_584 \
--automap \
--expunge1 \
--addheader \
--exclude "\[Gmail\]$" \
--regextrans2 "s/[ ]+/_/g" \
--regextrans2 "s/['\^\"\\\\]/_/g"
--regextrans2 "s/['\^\"\\\\]/_/g"
With --gmail2 you can override the parameters
that are activated by them, for example:
imapsync ... --gmail2 --maxbytespersecond 50_000 --noautomap --maxsleep 0
Explanations:
--ssl2 is mandatory since Gmail only supports imap ssl connections.
--maxbytespersecond 10000 ( 10 kBytes/s ) option is here to avoid
--maxbytespersecond 20_000 ( 20 kBytes/s ) option is here to avoid
locking or errors when imap transfers exceed Gmail maximum limit.
See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
--maxbytespersecond 10000 is not mandatory in the sense Gmail may
--maxbytespersecond 20_000 is not mandatory in the sense Gmail may
allow you to use an upper value than 1 GBytes per 24h without
disconnections.
disconnections. If you count well, it should be 10_000, not 20_000.
--maxsize 25000000 is mandatory since Gmail limits messages size
--maxbytesafter 1_000_000_000 option is there because
option --maxbytespersecond 20000 is unnecessarily slow if the Gmail
account size is less than 1 GB. Counting well, it should
be 500_000_000, not 1_000_000_000, but 1GB is usually ok.
--maxsleep 2 is to avoid sleeping more than 2 seconds when
the --maxbytespersecond value implies a long sleep between 2 copies.
--maxsize 25_000_000 is mandatory since Gmail limits messages size
up to 25 MB. This value increases over time, it was 10 MB some
years ago so you can try higher values. The Gmail page about
this limit is https://support.google.com/mail/answer/6584
@ -239,29 +280,29 @@ You can select folders exported to imap within the gmail preferences.
Select or unselect some "System labels", depending on your needs.
https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
=======================================================================
Q. How to synchronize from Gmail to XXX?
R. Use this example:
./imapsync \
imapsync \
--user1 gilles.lamiral@gmail.com \
--password1 gmailsecret \
--host2 localhost \
--user2 tata \
--gmail1
Or, replacing what does --gmail1:
Or, replacing what does --gmail1 (values are from imapsync 1.882):
./imapsync \
imapsync \
--host1 imap.gmail.com \
--user1 gilles.lamiral@gmail.com \
--password1 gmailsecret \
--host2 localhost \
--user2 tata \
--password2 tatasecret \
--ssl1 \
--maxbytespersecond 20000 \
--maxbytespersecond 20_000 \
--useheader="X-Gmail-Received" \
--useheader "Message-Id" \
--automap \
@ -269,15 +310,17 @@ Or, replacing what does --gmail1:
--skipcrossduplicates \
--folderlast "[Gmail]/All Mail"
With --gmail1 you can override the parameters
that are activated by them, for example:
imapsync ... --gmail1 --maxbytespersecond 50_000 --noskipcrossduplicates --noautomap
Explanations:
--ssl1 is mandatory since Gmail only supports imap ssl connections.
--maxbytespersecond 20000 ( 20 kBytes/s ) option is here to avoid
--maxbytespersecond 20_000 ( 20 kBytes/s ) option is here to avoid
locking or errors when imap transfers exceed Gmail maximum limit.
See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
--maxbytespersecond 20000 is not mandatory in the sense Gmail may
--maxbytespersecond 20_000 is not mandatory in the sense Gmail may
allow you to use an upper value than 2 GBytes per 24h without
disconnections.
@ -302,11 +345,11 @@ for example you may unselect all "System labels".
disk memory. Within imap protocol, Gmail presents Gmail labels as
folders, so a message labeled "Work" "ProjectX" "Urgent" ends up
in three different imap folders "Work" "ProjectX" and "Urgent"
after an imap sync. --skipcrossduplicates prevent this behavior.
after an imap sync. Option --skipcrossduplicates prevent this behavior.
An issue with --skipcrossduplicates is that the first label synced
by imapsync goes to its corresponding folder and other labels are
ignored. This way, at least you can choose what labels have the
by imapsync goes to its corresponding folder but other labels are
then ignored. You can choose what labels have the
priority by using the --folderfirst option. For example
--folderfirst "Work" will sync messages labeled "Work" before
messages labeled "CanWait" or "Urgent". By default imapsync
@ -320,15 +363,15 @@ label CanWait and only it.
the messages that are not labeled at all.
https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
=======================================================================
Q. I can't authenticate with Gmail via IMAP
and Gmail says "Please log in via your web browser"
R0. My current settings on two Gmail account are made
with 0) 1) 2)
R0. My current settings on two Gmail accounts are made
with three steps 0) 1) 2)
0) Login via a browser to the Gmail account concerned.
0) Login via a web browser to the Gmail account concerned.
1) The 2-step verification is OFF
https://myaccount.google.com/security?hl=en&pli=1&nlr=1#signin
@ -343,9 +386,10 @@ https://web.archive.org/web/20150906230041/http://security.stackexchange.com/que
R2. I had the same issue one time (mars 2015) logging to Gmail
with imapsync.
The Gmail imap message error said "Please log in via your web browser"
so I logged for this account via a web browser,
so I logged for this account via a web browser. Then,
it asked me to receive a code via a mobile, I said yes,
I entered the code and everything went ok.
I entered the code and everything went ok on next syncs
with imapsync.
=======================================================================
@ -380,6 +424,15 @@ Or go to http://ping.eu/nslookup/ to get the resolution.
Thanks to Chris Nolan to report, understand and fix this issue!
=======================================================================
Q. Can I transfer "Chat" messages with imapsync?
R. No.
Even if you think you can because the Chat folder is seen in IMAP,
with Chat messages in them, they won't be well transferred.
Chat messages are not well presented and not well transferred
using the IMAP protocol.
=======================================================================
Q. Can I safely use --useuid for Gmail transfers?
@ -395,16 +448,32 @@ ends up with no duplicates on Gmail but a waste of bandwith and time,
which is the opposite goal of --usecache implied by --useuid.
=======================================================================
Q. Gmail does not really delete messages in folder [Gmail]/All Mail
What happens? What can I do?
=======================================================================
Q. Gmail does not really delete messages in folder [Gmail]/All Mail
What happens? What can I do?
R. It's true and explained in Gmail documentation at
https://support.google.com/mail/answer/78755?hl=en
https://support.google.com/mail/answer/7401?hl=en
To really remove messages in folder "[Gmail]/All Mail", they have to
be moved to the "Trash" folder and be deleted from "Trash".
Quoting the link above (25 december 2018, happy christmas by the way!)
...
Empty your Trash
If you don't want a message to stay in your Trash for 30 days,
you can permanently delete it.
* On your computer, go to Gmail.
* On the left side of the page, scroll down,
then click More and then Trash.
* Check the box next to messages you want to permanently delete,
then click Delete forever.
* Note: The Delete forever option is only available in the Trash.
You wont be able to delete items forever from the Search result view.
To delete all messages in your trash, click Empty Trash now.
=======================================================================
Q. Can I use the Extension of the SEARCH command: X-GM-RAW described at
https://support.google.com/mail/answer/7190?hl=en
@ -441,18 +510,21 @@ So you have to follow the Google recommendation and generate an
application-specific password or normal authentication
or use XOAUTH or XOAUTH2.
=======================================================================
Q. How to use XOAUTH2 to globally authenticate gmail users?
R. Yes, but really tested on Unix systems, not sure on Windows. See:
http://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt
=======================================================================
Q. How to use XOAUTH to globally authenticate gmail users?
R0. XOAUTH is considered obsolete and superseded by XOAUTH2
See http://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt
=======================================================================
Q. How to use a Gmail account to backup several different imap accounts?
@ -465,6 +537,7 @@ is no supplementary label created on the multi-archive Gmail
destination account. No labels all over the place and all original
xxx sub-folders show up nested within xxx/xxx.
=======================================================================
Q. How to migrate email from gmail to google apps?

View file

@ -1,7 +1,9 @@
#!/bin/cat
$Id: FAQ.ISP.txt,v 1.8 2017/06/21 18:09:00 gilles Exp gilles $
$Id: FAQ.ISP.txt,v 1.9 2018/05/24 11:34:30 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.ISP.txt
=======================================================================
Imapsync tips for ISP. Specific issues and solutions.

View file

@ -1,7 +1,10 @@
#!/bin/cat
$Id: FAQ.Massive.txt,v 1.17 2017/12/20 03:38:07 gilles Exp gilles $
$Id: FAQ.Massive.txt,v 1.24 2018/12/30 21:28:03 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Massive.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips for massive/bulk migrations.
@ -9,6 +12,8 @@ This documentation is also at http://imapsync.lamiral.info/#doc
Questions answered here are:
Q. How long will take the whole migration?
Q. I need to migrate hundred accounts, how can I do?
Q. I have to migrate 500k users using 400 TB of disk space.
@ -16,18 +21,30 @@ Q. I have to migrate 500k users using 400 TB of disk space.
Q. How to determine where is the bottleneck in an imapsync process?
Q. Can I run more instances of imapsync in parallel on a Windows host?
Q. Can I run several instances of imapsync in parallel on a Windows host?
Q. I run multiple imapsync applications at the same time then get a
warning "imapsync.pid already exists, overwriting it".
Is this a potential problem when trying to sync multiple
IMAP account in parallel?
=======================================================================
Q. How long will take the whole migration?
R. Consider the mean imapsync transfer rate to be around 340 Kbytes/s,
ie, 2.8 Mbps, no matter the local link bandwidth.
It's a mean, measured upon various different syncs, coming from
the online interface /X.
With 1 TB to transfer and one sync at a time it will end in 35 days.
At 10 transfers at a time, 1 TB will take 3.5 days.
Double the time because best scenario never happens.
Triple because, well, real world is like that.
=======================================================================
Q. I need to migrate hundred accounts, how can I do?
R. If you have many mailboxes to migrate think about a little
R. If you have many mailboxes to migrate, think about a little
script program. Write a file called file.txt (for example)
containing hosts, users and passwords on both sides.
The separator used in this example is ";"
@ -118,27 +135,39 @@ How many syncs can we run in parallel? here comes measurements.
Since adding this way is not so easy, just look at the overall
network rate of the imapsync host.
On Linux, nload is good candidate to measure this overall
network rate, every 6 seconds, on eth0 interface, values in Kbytes:
On Linux and FreeBSD, nload is a good candidate to measure this overall
network rate, every 6 seconds (-t 6000), on eth0 or em0 interface,
values in Kbytes (-u K):
nload -t 6000 eth0 -u K
nload -t 6000 eth0 -u K # Linux
nload -t 6000 em0 -u K # FreeBSD
Another good network tool is dstat:
During nload, press the h to see the display commands available,
every single feature is useful! Press h again and try each one.
On Linux only, another very good network tool is dstat:
dstat -n -N eth0 6
dstat -n -N eth0 6 # Linux only (in 2018)
A excellent tool for this purpose is iftop, the following
command will monitor imap and imaps connexions
on interfce eth0, only them, and sum them up:
iftop -i eth0 -f 'port imap or port imaps' -B
iftop -i eth0 -f 'port imap or port imaps' -B # Linux
iftop -i em0 -f 'port imap or port imaps' -B # FreeBSD
On Windows, get the overall network rate with the classical
task manager (Ctrl-Alt-Sup), there is a network tab in it.
Don't hesitate to send me free good tools to measure the
overall transfer rate (the best would be one to sum up only
imap traffic but that's not mandatory at all).
On Windows 8.1 Windows 10 Windows 2012 R2 Windows 2016,
get the overall network rate with the classical
task manager (Ctrl-Shift-Esc), there is a Performance tab
in it where resides a Network grap monitor.
On Windows 7, get the overall network rate with the classical
task manager (Ctrl-Shift-Esc), there is a Network tab in it.
I'm looking for a free and simple tool on Windows that could
sum up only the imap traffic.
2) Launch new parallel runs, one by one, as long as the total
transfer rate increase.
@ -190,7 +219,7 @@ Possible bottlenecks:
Usually available bandwidth is NOT a bottleneck.
Meanwhile, it can be a bottleneck on small Internet connexions.
Imapsync downloads messages from host1 and upload messages to host2,
consider this in case the connexion are asymetric.
consider this in case the connexion are asymmetric.
- I/O on disks.
I/O are a classical bottleneck, almost always forgotten.
@ -225,7 +254,7 @@ Other possible bottlenecks:
=======================================================================
Q. Can I run more instances of imapsync in parallel on a Windows host?
Q. Can I run several instances of imapsync in parallel on a Windows host?
R. Yes!

View file

@ -1,7 +1,10 @@
#!/bin/cat
$Id: FAQ.Memory.txt,v 1.4 2017/01/06 14:11:13 gilles Exp gilles $
$Id: FAQ.Memory.txt,v 1.8 2018/08/31 13:29:53 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Memory.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips about memory issues.
@ -12,7 +15,27 @@ This documentation is also at http://imapsync.lamiral.info/#doc
Q. I got an "Out of memory" error message then imapsync crashes.
How to fix that?
R0. Add option --debugmemory to debug memory issues.
R0. Add option --debugmemory to debug memory issues.
On Windows this option uses a DOS command similar to:
tasklist /NH /FO CSV | findstr imapsync
On Unix it uses:
ps -o vsz -p PID_of_imapsync # value is in KB
Similar to:
ps -ax -o vsz,comm | grep imapsync
Use those commands in a DOS window or a Unix shell in order
to monitor a imapsync run you know it will crash.
The goal is to detect the memory limit, when and how it happens.
* Is the "final" amount near the total RAM available or
just a small part of it?
* Does the rise of memory consumption happens suddenly
or along the sync?
R1. It's probably a very big message on the host1 account.
Before imapsync release 1.688 memory usage is at least 5 times
@ -20,8 +43,23 @@ R1. It's probably a very big message on the host1 account.
So a 700 MB message will crunch 3.5 GB of memory.
Starting with 1.688 it will crunch 1.4 GB of memory.
R2. To check if the problem is a big message crunching to much
memory, add --maxsize 50_000_000, this will limit the message
size treated to 50 MB. If the sync succeed while limiting
messages sizes it's probably is a big message crunching to much
memory issue. Possible solutions:
* Try on another host
* Buy memory
* Build imapsync.exe binary 64bits
R2. Usually "Out of memory" errors are related to old days,
R3. If R2 shows that the memory issue is not a big message crunching
too much memory, then it can be imapsync using too much memory
to identify messages (a very big folder served by a
very talkative imap server). A possible solution is to use
the option --useuid. If it still fails, I have no clue.
If you have time, drop me a note.
R4. Usually "Out of memory" errors are related to old days,
to old Mail::IMAPClient module releases, before 3.26.
At that time memory usage was around 17 times the biggest message.
Look at imapsync output first lines to get the Mail::IMAPClient

View file

@ -1,7 +1,10 @@
#!/bin/cat
$Id: FAQ.Messages_Selection.txt,v 1.13 2018/03/07 04:12:37 gilles Exp gilles $
$Id: FAQ.Messages_Selection.txt,v 1.14 2018/05/24 11:34:30 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips to select messages.

View file

@ -1,49 +1,140 @@
#!/bin/cat
$Id: FAQ.Migration_Plan.txt,v 1.2 2018/03/07 04:28:00 gilles Exp gilles $
$Id: FAQ.Migration_Plan.txt,v 1.5 2019/01/11 13:13:55 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Migration_Plan.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=====================================================================
Imapsync. Suggestions for a good, low impact on users,
well executed email migration plan.
=====================================================================
A good migration plan:
There is two main different scenarios depending on the response to the
following question:
* Create the new mailboxes on the destination server.
* Decrease the TTL of the MX, as well as the imap hostname resolution,
to 5 minutes (or even less). See FAQ.TTL.txt to understand why.
* Presync all the mailboxes from the old to the new server, using different name
that the ones used by the imap software clients (use their IP for example).
Presyncs can usefully be done with --delete2 but never use --delete2 once users
started playing with their new account on host2.
Will the imap software tools used by the users use the exact same
credentials triplet for both imap servers, the old server host1 and
the new server host2?
* Decide a migration day/hour
* repeat the presyncs (--delete2) daily until the migration hour.
This repeated process will show how long should take the last sync.
The credentials triplet is hostname/username/password.
If the answer is yes, ie, clients email tools use the exact same
triplet credentials, then it is possible to perform a migration
without changing anything on the users side. This may be a very time
saving option. But it's a rare condition so I'll describe this
scenario later in this document.
=====================================================================
Classical scenario, credentials triplets are different on both sides
=====================================================================
* Decrease the TTL of the MX, to 5 minutes (or even less). See
FAQ.TTL.txt to understand why it's an advantage. If you can't
decrease the TTL, the migration will span a little more but that's
ok, the situation is not that bad.
* Create the new mailboxes on the destination server host2. If the
users are already playing with the new mailboxes on host2, don't
follow this scenario.
* Presync all the mailboxes from the old server host1 to the new
server host2. Presyncs can usefully be done with --delete2 in
order to get an exact sync. But never use the option --delete2 once
users have started to play with their new account on host2, their
play will be lost on the next presync, or when the MX is changed,
since INBOX will start to receive new messages that are not on
host1.
* Decide a migration day/hour.
* Repeat the presyncs (with the --delete2 options) daily until the
migration hour. This repeated process will show how long should
take the last sync.
* At the migration hour, cut access to the users to the old server
(by changing the imap hostname to a non-imap server for example).
* Change also the MX (for the new messages arriving in the new imap server).
* Wait 5 minutes.
* Do a last sync exactly like the presyncs, not using the imap hostname.
* Shutdown the old imap server.
host1, if you can. Or tell them to not use it anymore.
Now a fork in the process, depending on the response to the following question:
Will the imap software clients use the exact same
triplet credentials as the old server? hostname/username/password
* Do a last presync exactly like previous ones.
* Change the MX, the new messages should start to arrive in the new
imap server host2.
* Wait the TTL value, aka 5 minutes. Now, new messages should
not arrive to the old server host1.
* Tell the users that the old imap server host1 is down and no
longer available.
* Do a postsync. A postsync is a sync with the following options
--maxage 1 --delete1 --folder INBOX
This postsync will move the last new messages arrived on host1 to
host2 during the TTL interval and delete them on host1. Do not use
the option --delete2 in a postsync.
* Give access to new accounts to the users with their new credential
triplet hostname/username/password. If the way to contact users is
email then you should give this long before shutting down the old
server.
If the answer is yes (clients use the exact same triplet credentials):
* Change the user imap hostname resolution from the old to the new imap server
* Migration done.
If the answer is no (clients use a different triplet credentials):
* Tell the users the old imap server is down and no longer available.
* Give access to new accounts to the users with their new credential triplet
hostname/username/password (the most similar the best for them).
If the way to contact users is email then you should give this long before
shutting down the old server.
* Migration Done
* In case there are still messages arriving at the old imap server
host1, you can perform more postsyncs, ie, syncs every day
with the options:
--maxage 1 --delete1 --folder INBOX.
=====================================================================
Lucky scenario, credentials triplets are the same on both sides
=====================================================================
* Decrease the TTL of the MX, as well as the imap hostname resolution,
to 5 minutes (or even less). See FAQ.TTL.txt to understand why.
* Create the new mailboxes on the destination server host2.
* Presync all the mailboxes from the old host1 to the new server host2,
using different names that the ones used by the imap software
clients (use their IP for example).
Presyncs have to be done with --delete2 but never use --delete2
once users have started playing with their new account on host2.
* Decide a migration day/hour.
* repeat the presyncs (with the --delete2 options) daily until the
migration hour. This repeated process will show how long should
take the last sync.
* At the migration hour, cut access to the users to the old server.
You can do this by changing the imap host1 hostname to a non-imap
server for example, or by changing their password on host1.
* Do a last sync exactly like the presyncs, not using the imap hostname.
* Change also the MX resolution, the new messages should start
to arrive in the new imap server very soon.
* Wait the TTL value, aka 5 minutes. Now, new messages should
not arrive to the old server host1.
* Do a postsync. A postsync is a sync with the following options
--maxage 1 --delete1 --folder INBOX
This postsync will move the last new messages arrived on host1 to
host2 during the TTL interval and delete them on host1. Do not use
the option --delete2 in a postsync.
* Shutdown the old imap server.
* Change the user imap hostname resolution from the old IP of host1
to the IP of the new imap server host2.
* Migration done.
=======================================================================
=======================================================================

View file

@ -1,10 +1,13 @@
#!/bin/cat
$Id: FAQ.OnlineUI.txt,v 1.5 2018/05/02 12:22:13 gilles Exp gilles $
$Id: FAQ.OnlineUI.txt,v 1.19 2019/01/28 10:59:16 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.OnlineUI.txt
=====================================================================
Imapsync tips about the online visual user interface
Imapsync tips about the online visual user interfaces
https://i005.lamiral.info/X/
https://imapsync.lamiral.info/X/
=====================================================================
@ -27,28 +30,38 @@ R0. Well, I don't know if asking the provider whether his online
R1. Some figures
Date of this report: 8 March 2018.
The online imapsync service /X started 9 January 2017 (422 days of service).
In average, /X has 20 users per day lunching 6 different migrations, from
one launch to many many (hundreds).
The total volume /X transferred is 17 TiB in 48000 email imap migrations.
Date of this report: 26 December 2018.
The online imapsync service /X started 9 January 2017
(716 days of service).
In average, /X has 20 users per day lunching in mean 6
different migrations, from one launch to many (hundreds).
The total volume /X transferred is around 45 TiB in more
than 111 thousands email imap migrations,
157 millions email messages.
R2. Pros & Cons
The online imapsync service /X runs on https only, with a
letsencrypt certificate, a certificate overall rated "A" at
https://www.ssllabs.com/ssltest/analyze.html?d=imapsync.lamiral.info
letsencrypt certificate, a certificate overall rated "A+" at
https://www.ssllabs.com/ssltest/analyze.html?d=i005.lamiral.info
Because of the https usage, what the users enter in their browser,
the imap logins and passwords, can't be eavesdropped on the network.
Imapsync itself cares about encryption for the imap sessions,
if possible: It tries SSL first on port 993, then TLS if the
servers announces TLS, then no encryption. What is done with
an imap server is independent of what is done with the other.
Imapsync itself takes care about encryption for the imap sessions,
if possible: It tries SSL first on port 993, then TLS on port 143
if the servers announces TLS, then no encryption at all.
What is done with the source imap server host1 is independent
of what is done with the destination imap server host2.
At the date of 8 March 2018, there is no security problem detected
or reported to me (Gilles LAMIRAL) so far.
At the date of 26 December 2018, there is no security problem
detected or reported to me (Gilles LAMIRAL), so far.
Feel free to attack the service and feel free to report any
hole encountered. Have in mind I can watch what you try
from the server side.
As the owner of the service, it could have been 48 000 pairs of
credentials collected and nearly 17 terabytes of email messages.
@ -60,49 +73,62 @@ because too many imap servers are crappy configured regarding
certificates.
This default behavior is chosen like this because users of /X
wants their emails transferred, instead of not trasferred because
of an incompetent imap server sysadmin.
Anyway, this part, checking imap ssl/tls certificates, could be
improved from my side by including well known certificates
directly in imapsync.
want their emails transferred, instead of being not transferred
because of an incompetent imap server sysadmin.
I admint that this part, checking imap ssl/tls certificates,
could be improved from my side by including well known
certificates directly in imapsync.
If the imap servers don't honor ssl nor tls, then logins, passwords
and everything will go clear text during the imap transfers.
That's not good at all but what "comforts" me is that if the
imap servers do only clear text transfer, then it is also true
for all imap sessions the owner of the accounts encounter,
imap servers do only clear text transfers, then it's also true
for all imap sessions the accounts' owner encounters,
imapsync is just one of them.
Last point, who could be sure no cracker cracked the online host and
currently sniffs the credentials? No one, I'm not sure myself, even
if I do take care of that possibility.
Last point, who could be sure that no cracker cracked the online
hosts and that he isn't currently sniffing the credentials?
No one, I'm not sure myself, even if I do take care of that
possibility. So changing the imap accounts passwords after
a sync is a safe and recommended practice!
=====================================================================
Q. Will I have any issues with browser timing out? What happens
if the browser connection is closed for whatever reason?
R. It should be ok
R. It stops the imapsync process, ie, the sync is ended right away.
Further comments on this behavior.
When using the /X interface there are three connections.
One connection is the Browser-WebServer connection, the
two others are the WebServer-ImapServers connections (imapsync stuff).
One connection is the Browser/WebServer connection,
the two others connections are the WebServer/ImapServers
connections (imapsync stuff).
If the Browser-WebServer connection is timeout (but it shouldn't
because of the log refresh), the imapsync sync might continue
anyway. To see if it continues or not, just do a sync again and the
interface will tell you that a sync is already going on, if the
"Sync!" button is gray/inactive then just reload the page (F5 or
similar), and reenter the credentials.
If the Browser/WebServer connection is timeout or ended,
the imapsync sync is also ended immediately by the remote
Apache https server. Technically, Apache sends a TERM signal,
to the imapsync process then wait some seconds before
sending a KILL signal if it is still alive.
By the way, on the /X you can try to do several parallel runs on the same
mailbox even if there is no timeout, open a new tab/windows with /X
and start a same sync, it's safe, the /X will say, if any, that there
is already a current sync.
You can relaunch a sync again with "Sync!" button, at any time.
If the "Sync!" button is gray/inactive then just reload
the page (F5 or similar), and reenter the credentials.
If the interface tells you that a sync is already going on,
it may be that a sync is running from another browser or place.
You can stop this sync with the "Abort!" button from any /X
tab/window, even from another browser or place. To doing this with
success, you have to give the same account parameters, same
credentials, or imapsync will ignore the demand.
tab/window, even from another browser or place. To be able
to abort with success, you have to give the same account
parameters, same credentials, or imapsync will ignore the demand.
In other words, you can try safely to launch several parallel
runs between the same mailboxes. Open a new tab/windows with /X,
and start a same sync, it's safe, the /X will say, if any, that
there is already a current sync running.
=====================================================================
=====================================================================

View file

@ -1,7 +1,10 @@
#!/bin/cat
$Id: FAQ.Oracle-UCS.txt,v 1.4 2017/01/06 14:11:13 gilles Exp gilles $
$Id: FAQ.Oracle-UCS.txt,v 1.5 2018/05/24 11:34:30 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Oracle-UCS.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips for Oracle-UCS. Specific issues.

View file

@ -1,10 +1,13 @@
#!/bin/cat
$Id: FAQ.POP3.txt,v 1.2 2018/02/02 20:32:19 gilles Exp gilles $
$Id: FAQ.POP3.txt,v 1.3 2018/05/24 11:34:30 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.POP3.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips about POP3.
Imapsync tips about POP3.
=======================================================================
Questions answered here are:
@ -55,6 +58,5 @@ See also Rick Sanders discussion about this issue:
https://www.emailquestions.com/threads/how-to-prevent-duplicate-emails-after-server-migration-for-pop3-users-with-leave-mail-on-server.8109/
=======================================================================
=======================================================================
=======================================================================

View file

@ -1,7 +1,10 @@
#!/bin/cat
$Id: FAQ.Passwords_on_Unix.txt,v 1.1 2017/01/06 14:11:13 gilles Exp gilles $
$Id: FAQ.Passwords_on_Unix.txt,v 1.3 2019/01/10 13:27:28 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync issues with passwords on Unix.
@ -11,8 +14,14 @@ This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Q. On Unix, some passwords contain some *(),;& characters. Login fails.
R. Use double-quotes within single-quotes in order to enclose the
password within double-quotes in the imap LOGIN command:
R1. Enclose the password within single-quotes in the imapsync
command line:
imapsync ... --password1 'passw*(),;&rd'
R2. If R1 fails, with very old imapsync or old Mail::IMAPClient Perl module,
try also using double-quotes within single-quotes. It will enclose
the password within double-quotes in the imap LOGIN command:
imapsync ... --password1 '"passw*(),;&rd"'

View file

@ -1,12 +1,16 @@
#!/bin/cat
$Id: FAQ.Passwords_on_Windows.txt,v 1.3 2017/01/06 14:11:13 gilles Exp gilles $
$Id: FAQ.Passwords_on_Windows.txt,v 1.5 2018/06/11 22:35:10 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync issues with passwords on Windows.
=======================================================================
Related documentation:
http://www.robvanderwoude.com/escapechars.php
http://stackoverflow.com/questions/3288552/how-can-i-escape-an-exclamation-mark-in-cmd-scripts
@ -15,7 +19,17 @@ In case you're brave and relentless, understand and try this:
http://www.dostips.com/forum/viewtopic.php?f=3&t=1733
=======================================================================
Q. On Windows, some passwords contain $ characters. Login fails.
Q. On Windows, some passwords contain $ or other special characters
like one more more of the nine characters $%&<>|^"!
Login fails.
R0. Change the password so that only normal characters remain.
It's often a quick and good solution. If you care about
strong password policy, just make it longer.
If you can't avoid special characters use no problematic
characters such as underscore _ or minus - or comma , or dot.
If you can't change passwords then read on and good luck!
R1. Enclose passwords between ""
@ -31,6 +45,9 @@ you have to enter
imapsync ... --password1 "$%%&<>|^"^"
The exclamation mark ! does not have to be escaped sometimes,
so have a try.
=======================================================================
Q. On Windows, some passwords begin with an equal = character.
Login fails. What can I do?

View file

@ -1,18 +1,21 @@
#!/bin/cat
$Id: FAQ.Release_Checklist.txt,v 1.6 2017/09/16 20:21:39 gilles Exp gilles $
$Id: FAQ.Release_Checklist.txt,v 1.10 2019/02/14 15:52:44 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Release_Checklist.txt
===================================
Imapsync developper notes
===================================
=======================================================================
Imapsync developer notes
=======================================================================
Checklist before release a new release:
- Add a new section in S/news.shtml reading "rlog imapsync"
- Generate the README
- Run a spell checker on the README
- Read the README again slowly. Fix all issues, all
- Read the README again slowly. Fix all issues, all.
- Read the OPTIONS section of README and read it very slowly
- Read slowly README_Windows.txt
- Read slowly the TUTORIAL_Unix file in html
@ -21,7 +24,9 @@ Checklist before release a new release:
- Review the general FAQ.d/FAQ.General.txt
- make dist
- make publish
- verify VERSION is uploaded
=======================================================================
=======================================================================

View file

@ -1,11 +1,14 @@
#!/bin/cat
$Id: FAQ.Reporting_Bugs.txt,v 1.3 2018/04/10 00:17:18 gilles Exp gilles $
$Id: FAQ.Reporting_Bugs.txt,v 1.4 2018/05/24 11:34:30 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Reporting_Bugs.txt
===================================================
Imapsync bugs reports
===================================================
=======================================================================
Imapsync bugs reports
=======================================================================
=======================================================================
Q. How can I report bugs or problems I encountered with Imapsync?
@ -25,5 +28,6 @@ than 1MB. Don't zip it, it will slow my response.
Thanks!
=======================================================================
=======================================================================

View file

@ -1,11 +1,14 @@
#!/bin/cat
$Id: FAQ.SSL_errors.txt,v 1.5 2018/04/25 17:58:12 gilles Exp gilles $
$Id: FAQ.SSL_errors.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
=====================================================
Imapsync SSL errors
=====================================================
=======================================================================
Imapsync SSL errors
=======================================================================
Questions answered in this FAQ are:
@ -74,7 +77,7 @@ R. Use:
=======================================================================
=======================================================================
=======================================================================

View file

@ -1,7 +1,10 @@
#!/bin/cat
# $Id: FAQ.Security.txt,v 1.16 2017/04/26 21:50:00 gilles Exp gilles $
# $Id: FAQ.Security.txt,v 1.19 2018/09/13 17:53:45 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Security.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips about security. Issues and solutions.
@ -88,14 +91,18 @@ a) Use native --ssl1 and/or --ssl2 options
b) Use stunnel
http://www.stunnel.org/
Use stunnel3 command since stunnel now usually calls
stunnel4 or stunnel5 and the command line options syntax
has changed (option "-c" not recognized for example).
Assuming there is an imaps (993) server on imap.foo.org,
on your localhost machine (or bar machine), run:
stunnel -c -d imap -r imap.foo.org:imaps
stunnel3 -c -d imap -r imap.foo.org:imaps -f
or using numbers instead of names:
stunnel -c -d 143 -r imap.foo.org:993
stunnel3 -c -d 143 -r imap.foo.org:993 -f
then use imapsync on localhost (or bar machine) imap (143) port.
If the local port 143 is already taken then use a free one,
@ -104,7 +111,7 @@ b) Use stunnel
c) Other example for accessing gmail with no local root access
to open port 143
stunnel -f -P '' -c -d 9993 -r imap.gmail.com:993
stunnel3 -P '' -c -d 9993 -r imap.gmail.com:993 -f
Then, to access gmail as host2 use:
@ -193,24 +200,6 @@ to help the ssl software verifying the server certificate.
The file ca-certificates.crt may be elsewhere on your system, even
named differently.
=======================================================================
Q. How can I manually test a login via ssl?
R. Use ncat or telnet-ssl like in this example:
ncat --ssl -C imap.gmail.com 993
* OK Gimap ready for requests from 78.196.254.58 q1mb175739668wix
a LOGIN "gilles.lamiral@gmail.com" "secret"
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE ... ESEARCH
a OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success)
b LOGOUT
* BYE LOGOUT Requested
b OK 73 good day (Success)
The client part to type is "a LOGIN ..." and "b LOGOUT" without
the double-quotes.
=======================================================================
Q: How to have an imaps server?

View file

@ -1,7 +1,10 @@
#!/bin/cat
$Id: FAQ.SmarterMail.txt,v 1.11 2017/09/05 15:11:20 gilles Exp gilles $
$Id: FAQ.SmarterMail.txt,v 1.12 2018/05/24 11:34:30 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.SmarterMail.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips for SmarterMail. Specific issues and solutions.

View file

@ -1,7 +1,10 @@
#!/bin/cat
$Id: FAQ.TTL.txt,v 1.1 2017/05/03 22:27:45 gilles Exp gilles $
$Id: FAQ.TTL.txt,v 1.2 2018/05/24 11:34:30 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.TTL.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=====================================================================
Imapsync tips about TTL when changing name resolution of hosts
@ -30,3 +33,5 @@ the migration done, no problem to set back the TTL to 24h or more. If
you can't decrease TTL under 4h or even 24h, it's ok anyway, imapsync
can sync the new messages dropped in the old server.
=======================================================================
=======================================================================

View file

@ -1,10 +1,13 @@
#!/bin/cat
$Id: FAQ.Use_addheader.txt,v 1.4 2017/01/25 23:54:02 gilles Exp gilles $
$Id: FAQ.Use_addheader.txt,v 1.6 2018/12/10 20:12:05 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Use_addheader.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync --addheader suggestion.
Imapsync --addheader suggestion.
=======================================================================
=======================================================================
@ -12,18 +15,21 @@ Q. What means this log message:
"Host1 Sent/1234 ignored (no wanted headers so we ignore this message.
To solve this: use --addheader)"
R. In order to sync messages from one account to another Imapsync has
R. In order to sync messages from one account to another, Imapsync has
to identify messages on both sides, and compare them.
Imapsync can use two completely different methods to identify
messages on each account.
The first one, the default one, uses some headers of messages,
Real world syncs with various imap servers have bring to use
The first one, the default one, uses some headers of messages.
Real world syncs with various imap servers have bring me to use
two headers "Message-Id:" and "Received:". It works for most
messages except for special folders like Sent or Draft where
messages in those folders don't have "Message-Id:" nor
"Received:" headers.
"Received:" headers.
Here comes --addheader option. When a message has no "Message-Id:"
nor "Received:", option --addheader adds a "Message-Id" header
nor "Received:" headers, option --addheader adds a "Message-Id" header
consisting of the imap UID of the message on the host1 folder,
like "Message-Id: 12345@imapsync".
This way, messages are well identified on both sides,

View file

@ -1,11 +1,14 @@
$Id: FAQ.Use_cache.txt,v 1.6 2018/03/07 04:12:37 gilles Exp gilles $
$Id: FAQ.Use_cache.txt,v 1.8 2018/10/22 17:30:45 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt
============================================
Imapsync --usecache option
============================================
=======================================================================
Imapsync --usecache option
=======================================================================
Questions answered in this FAQ are:
@ -89,11 +92,19 @@ filesystem devoted to the imapsync cache.
# Mount this brand new filesystem
mkdir -p /var/tmp/cachedir
mount -o loop /var/tmp/fscache /var/tmp/cachedir
df -i /var/tmp/cachedir # count inodes available
# Tell imapsync to use it
imapsync ... --tmpdir /var/tmp/cachedir/
# When the migration is over and the cache becomes really useless
umount /var/tmp/cachedir/
rm /var/tmp/fscache
=======================================================================
=======================================================================

View file

@ -1,7 +1,10 @@
#!/bin/cat
$Id: FAQ.User_Concurrent_Access.txt,v 1.4 2017/12/19 14:55:36 gilles Exp gilles $
$Id: FAQ.User_Concurrent_Access.txt,v 1.5 2018/05/24 11:34:30 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.User_Concurrent_Access.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync and user concurrent access.

View file

@ -1,283 +0,0 @@
#!/bin/cat
$Id: FAQ.Various_Server_Softwares.txt,v 1.8 2017/09/05 15:11:20 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips for various imap server softwares.
=======================================================================
=======================================================================
Q. From Zimbra to XXX
imapsync ... \
--exclude "Conversation Action Settings" \
--exclude "Quick Step Settings" \
--exclude "News Feed"
=======================================================================
Q. From or to HMailServer version 4.4.1.
R. You have to add prefix and separator manually because 4.4.1 doesn't
honor the NAMESPACE imap command.
Example for host1:
imapsync ... \
--prefix1 "" --sep1 .
No specific option for HMailServer 5.3.3 since NAMESPACE is supported.
Maybe --subscribe_all will help you to see all migrated folders.
=======================================================================
Q. Synchronizing from Kerio Connect to XXX
R. No special options required.
See also:
http://www.linux-france.org/prj/imapsync_list/msg01756.html
http://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-1/
http://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-2/
=======================================================================
Q. from Microsoft's Exchange 2007 to Google Apps for your Domain
(GAFYD)
R. Take a look at:
http://mark.ossdl.de/2009/02/migrating-from-exchange-2007-to-google-apps-mail/
=======================================================================
Q. Migrating from or to Parallels Plex Server
R. It depends on the OS
Parallells Plesk Panel for Windows requires --sep2 / --prefix2 ""
Parallells Plesk Panel for Linux works with default parameters.
=======================================================================
Q. I'm migrating from WU to Cyrus, and the mail folders are under
/home/user/mail but the tool copies everything in /home/user, how
can i avoid that?
Two solutions:
R. Use
imapsync ... --include '^mail'
R. or (better)
imapsync ... --subscribed --subscribe
=======================================================================
Q. I'm migrating from WU to Cyrus, and the mail folders are under
/home/user/mail directory. When imapsync creates the folders in
the new cyrus imap server, it makes a folder "mail" and below that
folder puts all the mail folders the user have in /home/user/mail,
i would like to have all those folders directly under INBOX.
R. Use
imapsync ... --regextrans2 's/^mail/INBOX/' --dry
look at the simulation and if all transformations seem
good then remove the --dry option.
=======================================================================
Q. Migrating from Groupwise to Cyrus
R. By Jamie Neil:
I eventually managed to get the mail to migrate without errors using the
following options:
--sep1 /
- doesn't report separator so has to be set explicitly.
--nosyncacls
- doesn't support ACLs.
--skipheader '^Content-Type'
- MIME separator IDs seem to change every time a mail is accessed so
this is required to stop duplicates.
--maxage 3650
- some messages just don't seem to want to transfer and produce the
perl errors I mentioned before. This prevents the errors, but the
bad messages don't transfer.
Even though the mail migrated OK, there are a couple of gotchas with
Groupwise IMAP:
1) Some of the GW folders are not real folders and are not available
to IMAP, the main problem one being "Sent Items". I could find no way
of coping the contents of these folders. The nearest I got was to
create a "real" folder and copy/move the sent items into it, but
imapsync still didn't see the messages (I think because there is
something funny about the reported dates/sizes).
It think this problem has been rectified in GW6.5.
2) The "skipheader '^Content-Type'" directive is required to stop
duplicate messages being created. GW seems to generate this field on
the fly for messages that have MIME separators and so it's different
every time.
3) Version 6.0.1 of the Groupwise Internet Connector sucks. I was
getting server aborts when I pushed it a bit hard! I eventually had to
upgrade to 6.0.4 which seems to be a lot more stable.
=======================================================================
Q. Migrating from iPlanet Messaging Server
5.2 Patch 2 (built Jul 14 2004)) to Groupwise 7.0
I encounter many errors like this:
"Error trying to append string: 17847 BAD APPEND"
R. GroupWise 7 seems buggy. Apply GroupWise 7 support pack 1
=======================================================================
Q. Migrating from David Tobit V10 (DvISE Mail Access Server MA-...)
R. Use the following options:
imapsync ... --prefix1 "" --sep1 / --idatefromheader ^
--nofoldersizes --useuid --nocheckmessageexists
=======================================================================
Q. Migrating from David Tobit V8
("* OK IMAP4rev1 DvISE Mail Access Server MA-8.10a (0126)")
First try above V10 solution since improvments have been made
to support Tobit.
R. Use the following options :
imapsync ... --prefix1 INBOX. --sep1 / --subscribe --subscribed
=======================================================================
Q. Migrating from Tobit David Server 6
("DvISE Mail Access Server MA-6.60a (0118)")
First try above V10 solution since improvments have been made
to support Tobit.
R. Look at the discussion:
http://www.linux-france.org/prj/imapsync_list/msg00582.html
http://www.linux-france.org/prj/imapsync_list/threads.html#00582
patch saved in ./patches/imapsync-1.337_tobit_V6.patch
=======================================================================
Q. I need to migrate 1250 mailboxes, passwords are in a MySQL Database.
Can you tell me if your script suits my needs?
R. Mailboxes must exist before running imapsync.
You have to extract users logins and passwords in a csv file.
See the "HUGE MIGRATION" section in the README file.
======================================================================
Q: From MailEnable 1.75
R: --sep1 "/" --prefix1 ""
Q: From MailEnable 2.2
R: --sep1 "." --prefix1 ""
Q: To MailEnable
R: --sep2 / --prefix2 "" --addheader --messageidnodomain --syncflagsaftercopy
======================================================================
Q. From GMX IMAP4 StreamProxy
R. Use:
--prefix1 INBOX and --sep1 .
======================================================================
Q. From Courier to Archiveopteryx
R. You can read http://www.archiveopteryx.org/migration/imapsync
Default values might be fine now with latest imapsync.
======================================================================
Q. To Sun Java(tm) System Messaging Server 6.2-7.05
Q. To Communigate Pro - Solaris version
R. See and run patches/imapsync_1.267_jari
======================================================================
Q. From Softalk Workgroup Mail 7.6.4
R. Old Softalk releases don't support the IMAP SEARCH command.
Here are the options to get it working.
imapsync ... --sep1 '.' --prefix1 '' \
--noabletosearch1 --nocheckmessageexists --addheader
(Thanks to Andrew Tucker)
======================================================================
Q. From or to QQMail IMAP4Server
R. imapsync ... --noabletosearch1
======================================================================
Q. From FirstClass to XXX
http://www.firstclass.com/
R. Migrating from FirstClass is not easy because FirstClass, strangely,
does not show all messages via IMAP. To make it show all messages,
a trick, painful to follow by hand, is moving emails
out and back in, for each folder. May be it can be done by a script.
FirstClass releases prior to release 12 do not shows the "Sent"
folder in IMAP but FirstClass release 12 shows it.
I advice you to upgrade to FirstClass release 12 before leaving it
with imapsync or another imap tool.
Here is a command line used to migrate from FirtClass release 12:
imapsync ... \
--tmpdir /var/tmp --usecache \
--useheader Message-ID \
--idatefromheader \
--addheader \
--regextrans2 "s,(/|^) +,\$1,g" --regextrans2 "s, +(/|$),\$1,g" \
--regextrans2 "s/[\^]/_/g" \
--regextrans2 "s/['\"\\\\]/_/g" \
--regextrans2 "s,&AC8-,-,g" \
--regextrans2 "s,&APg-,oe,g"
On Windows:
imapsync.exe ... ^
--automap ^
--usecache ^
--useheader Message-ID ^
--idatefromheader ^
--addheader ^
--regextrans2 "s,(/|^) +,$1,g" ^
--regextrans2 "s, +(/|$),$1,g" ^
--regextrans2 "s/[\^]/_/g" ^
--regextrans2 "s/['\\]/_/g" ^
--regextrans2 "s,^&AC8-,-,g" ^
--regextrans2 "s,^&APg-,oe,g"
Special thanks to Kristian Wind and Joey Alexander for helping me
writing this FAQ item.
See also this worth reading discussion in a Zimbra forum:
http://www.zimbra.com/forums/migration/20349-help-needed-migrating-firstclass.html
======================================================================
Q. From XXX to FTGate
R. Do NOT use --usecache since new UIDs are not given by FTGate and also
badly guessed by imapsync. UIDEXPUNGE does not work so use also
--expunge2 when using --delete2
imapsync ... \
--sep2 / --prefix2 "" \
--useheader Message-Id \
=======================================================================
=======================================================================

View file

@ -1,7 +1,10 @@
$Id: FAQ.Virus.txt,v 1.4 2018/03/21 16:18:53 gilles Exp gilles $
$Id: FAQ.Virus.txt,v 1.5 2018/05/24 11:34:30 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Virus.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync and virus scanners

View file

@ -1,7 +1,10 @@
#!/bin/cat
$Id: FAQ.XOAUTH2.txt,v 1.12 2017/01/06 14:11:13 gilles Exp gilles $
$Id: FAQ.XOAUTH2.txt,v 1.15 2018/08/28 21:48:53 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips to use XOAUTH2 authentication (Gmail) and old XOAUTH
@ -70,6 +73,29 @@ imapsync \
Use your own xoauth2 values.
The secret.xoauth2.json looks like:
{
"type": "service_account",
"project_id": "your-project-name",
"private_key_id": "1cfb..............................bd7fbe",
"private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiGziM...ZV5ACKPHuOfp8A46I=\n-----END PRIVATE KEY-----\n",
"client_email": "jsonfile@your-project-name.iam.gserviceaccount.com",
"client_id": "105................689",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/jsonfile%40your-project-name.iam.gserviceaccount.com"
}
You get this json file by a link like:
https://console.developers.google.com/apis/credentials?project=your-project-name
See also:
https://developers.google.com/gmail/imap/xoauth2-protocol
https://developers.google.com/identity/protocols/OAuth2
=======================================================================
Q. How to use XOAUTH2 via pk12 file to globally authenticate gmail users?

View file

@ -1,7 +1,10 @@
$Id: FAQ.Yahoo.txt,v 1.7 2017/12/18 10:45:00 gilles Exp gilles $
$Id: FAQ.Yahoo.txt,v 1.8 2018/05/24 11:34:30 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Yahoo.txt
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips for Yahoo.

View file

@ -1,13 +1,15 @@
# $Id: htaccess.txt,v 1.20 2018/03/17 16:12:54 gilles Exp gilles $
# $Id: htaccess.txt,v 1.23 2019/02/16 22:42:25 gilles Exp gilles $
AddDescription "<b>Back to Imapsync main page</b>." ..
AddDescription "<b>Authenticate via an admin account</b>." FAQ.Admin_Authentication.txt
AddDescription "<b>Back</b> to Imapsync main page." ..
AddDescription "<b>APPEND errors</b>." FAQ.Admin_Authentication.txt
AddDescription "<b>Authenticate via an admin account</b>." FAQ.APPEND_errors.txt
AddDescription "<b>Authentication failures</b>." FAQ.Authentication_failure.txt
AddDescription "<b>Archiving tips</b>." FAQ.Archiving.txt
AddDescription "<b>Archiving</b> tips." FAQ.Archiving.txt
AddDescription "<b>Big mailboxes</b> tips." FAQ.Big_Mailbox.txt
AddDescription "<b>Connection issues</b>." FAQ.Connection.txt
AddDescription "<b>Contacts & Calendars issues</b>." FAQ.Contacts_Calendars.txt
AddDescription "<b>Contacts & Calendars & Chat issues</b>." FAQ.Contacts_Calendars.txt
AddDescription "<b>Dates issues</b>." FAQ.Dates.txt
AddDescription "<b>Domino</b>." FAQ.Domino.txt
AddDescription "<b>Dovecot</b>." FAQ.Dovecot.txt
@ -43,8 +45,9 @@ AddDescription "<b>Option --usecache and inodes</b>." FAQ.Use_cache
AddDescription "<b>User concurrent access</b>." FAQ.User_Concurrent_Access.txt
AddDescription "<b>Virus scanners on Imapsync</b>." FAQ.Virus.txt
AddDescription "<b>Why use --addheader?</b>." FAQ.Use_addheader.txt
AddDescription "<b>Various imap server</b> softwares." FAQ.Various_Server_Softwares.txt
AddDescription "<b>Various imap software servers</b>." FAQ.Various_Software_Servers.txt
AddDescription "<b>XOAUTH2</b> (<b>Gmail</b>)." FAQ.XOAUTH2.txt
AddDescription "<b>Yahoo</b>." FAQ.Yahoo.txt
AddDescription "<b>Zimbra</b>." FAQ.Zimbra.txt
AddDescription "<b>Where this Description column comes from</b>." htaccess.txt