This commit is contained in:
Nick Bebout 2018-05-07 09:04:23 -05:00
parent 65f8552a91
commit 9a927be251
292 changed files with 53540 additions and 39537 deletions

View file

@ -1,12 +1,27 @@
#!/bin/cat
$Id: FAQ.Archiving.txt,v 1.6 2017/07/13 14:44:07 gilles Exp gilles $
$Id: FAQ.Archiving.txt,v 1.7 2018/04/24 00:16:21 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
============================================
Imapsync tips about archiving messages.
Imapsync issues and tips about archiving.
============================================
Questions answered in this FAQ are:
Q. Can I archive different accounts on the same destination account,
each account on a separate folder?
Q. How to move emails from one IMAP folder to another either on the
same IMAP server or a different one?
For example, all messages older than 1 year, move from INBOX to Archive.
Q. Can imapsync be used to maintain a local offline copy of a
mailbox from host1, eg for backup purposes, so that if the
server fails, then the mailbox could be reinstated,
ideally without requiring a second IMAP host?
=======================================================================
Q. Can I archive different accounts on the same destination account,
each account on a separate folder?
@ -18,7 +33,6 @@ R. Yes. Use --subfolder2
It is done it by adding two --regextrans2 options before
all others. Add --debug to see what's really going on.
Example:
imapsync ... --user1 foo --subfolder2 foo
@ -46,4 +60,25 @@ Add option --delete if you want to move messages, instead of just copy/sync them
R2. See also file FAQ.Folders_Mapping.txt
=======================================================================
Q. Can imapsync be used to maintain a local offline copy of a
mailbox from host1, eg for backup purposes, so that if the
server fails, then the mailbox could be reinstated,
ideally without requiring a second IMAP host?
R1. No. Imapsync plays with imap servers only.
R2. Imapsync can't but rick-sanders-imap-tools can!
See https://github.com/andrewnimmo/rick-sanders-imap-tools
Back up and restore IMAP accounts with imapdump.pl and dumptoIMAP.pl
=======================================================================
=======================================================================
=======================================================================

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Contacts_Calendars.txt,v 1.5 2017/01/06 14:11:13 gilles Exp gilles $
$Id: FAQ.Contacts_Calendars.txt,v 1.7 2018/03/07 04:26:04 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -7,12 +7,15 @@ This documentation is also at http://imapsync.lamiral.info/#doc
Imapsync issues about syncing Contacts & Calendars.
=========================================================
Questions anwswered in this FAQ are:
Questions answered in this FAQ are:
Q. Can I copy or sync Calendar or Contacts with imapsync?
Q. How can I avoid copying Calendar or Contacts folders?
Q. How can I copy or synchronize Calendars or Contacts?
Now the questions again with their answers.
=======================================================================
Q. Can I copy or sync Calendar or Contacts with imapsync?

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Dates.txt,v 1.6 2017/08/29 05:34:55 gilles Exp gilles $
$Id: FAQ.Dates.txt,v 1.7 2018/03/07 04:12:37 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -7,7 +7,7 @@ This documentation is also at http://imapsync.lamiral.info/#doc
Imapsync tips about dates.
===============================
Questions anwswered in this FAQ are:
Questions answered in this FAQ are:
Q. We have found that the time and date displayed have been changed to
the time at which the file was synchronized. What happened? Any fix?

View file

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: FAQ.Domino.txt,v 1.8 2017/03/02 12:54:15 gilles Exp gilles $
# $Id: FAQ.Domino.txt,v 1.9 2018/03/13 03:22:55 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -11,6 +11,13 @@ This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Q. From Domino Notes to XXX
R1. Use --domino1, like this:
imapsync ... --domino1
R2. Or use the following options:
On Windows use:
imapsync.exe ... --sep1 "\\" --prefix1 "" --messageidnodomain
@ -22,6 +29,13 @@ On Unix use:
=======================================================================
Q. From XXX to Domino Notes
R1. Use --domino2, like this:
imapsync ... --domino2
R2. Or follow this:
Domino doesn't accept INBOX subfolders.
On Windows:
@ -73,6 +87,5 @@ For Domino 853FP6 on Linux, we used this command on Unix:
--regexmess 's{\A(.*?(?! ^$))^Date:(.*?)$}{$1Migratedbyus:$2\nx-MailDate:$2}gxms'
=======================================================================
=======================================================================

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Duplicates.txt,v 1.12 2017/08/31 03:06:45 gilles Exp gilles $
$Id: FAQ.Duplicates.txt,v 1.14 2018/05/02 12:21:54 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -8,7 +8,7 @@ This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
=======================================================================
Q. How can I remove duplicates in an lone account?
Q. How can I remove duplicates in a lone account?
R. Just run imapsync on the same account with option --delete2duplicates,
ie, with host1 == host2 and user1 == user2
@ -43,35 +43,36 @@ So if it does generate duplicates it means a problem occurs
with message identification. It happens sometimes with IMAP
servers changing the "Message-Id" header line or one or more
of the "Received:" header lines in the header part of messages.
Imapsync uses "Message-Id" header line and "Received:" header
lines to identify messages on both sides.
By default, Imapsync uses "Message-Id" header line and
"Received:" header lines to identify messages on both sides.
R1.
A first solution is to use option --useuid.
With option --useuid, imapsync doesn't use header lines
to identify and compare messages in folders.
Instead of some headers, --useuid tell imapsync to use
the imap UIDs given by imap servers on bith sides.
the imap UIDs given by imap servers on both sides.
To avoid duplicates on next runs, imapsync uses a local cache
where it keeps UIDs already transfered.
where it keeps UIDs already transfered.
imapsync ... --useuid
There is an issue when --useuid is not used the first time.
A big issue with --useuid is that it doesn't generate duplicates if
used from the first time but it does generate duplicates after a previous
run without --useuid (because then uses a different method to identify
run without --useuid (because it then uses a different method to identify
the messages).
A solution? Two solutions.
The easiest is --delete2
if you are permitted to use it. Option --delete2 removes messages on host2
that are not on host1 so with --delete2 you go for resyncing all
The easiest is --delete2 if you are permitted to use it.
Option --delete2 removes messages on host2
that are not on host1. So, with --delete2 you go for resyncing all
messages again but all previously transferred messages are deleted,
and also messages previously there without imapsync.
but also messages previously there without imapsync.
So --useuid --delete2 is easy to remove duplicates but not for
all context.
all contexts. The host2 account must be considered as a strict
replication of the host1 account, ie, not active.
A second solution, better if R2 works (see R2 below), is to build
the cache before using --useuid
@ -120,8 +121,8 @@ can be deleted with option --delete2duplicates
imapsync ... --useheader "Message-Id" --delete2duplicates
Another good way toward a solution is to isolate two or three messages
in a BUG folder and send me the --debug output by email at
gilles.lamiral@laposte.net
in a BUG folder and send me the --debug output by email to
gilles@lamiral.info
imapsync ... --debug --folder BUG
@ -133,10 +134,10 @@ Remark. (Trick found by Tomasz Kaczmarski)
Option --useheader "Message-Id" asks the server to send only header
lines beginning with "Message-Id". Some (buggy) servers send the whole
header (all lines) instead of the "Message-Id" line. In that case, a
trick to keep the --useheader filtering behaviour is to use
trick to keep the --useheader filtering behavior is to use
--skipheader with a negative lookahead pattern:
imapsync ... --skipheader "^(?!Message-Id)"
imapsync ... --skipheader "^(?!Message-Id)"
Read it as "skip every header except Message-Id".

View file

@ -1,5 +1,5 @@
$Id: FAQ.Exchange.txt,v 1.34 2017/07/18 21:27:18 gilles Exp gilles $
$Id: FAQ.Exchange.txt,v 1.38 2018/03/21 16:26:23 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -7,7 +7,7 @@ This documentation is also at http://imapsync.lamiral.info/#doc
Imapsync tips for Exchange 20xx and Office365. Specific issues and solutions.
=================================================================================
Questions anwswered in this FAQ are:
Questions answered in this FAQ are:
Q. Can I use imapsync to transfer from or to Exchange or Office365 accounts?
@ -58,12 +58,18 @@ Q. How to sync from any to Exchange2007?
Q. How to sync from Microsoft Exchange 2000 IMAP4rev1 server?
Now the questions again with their answers.
=======================================================================
Q. Can I use imapsync to transfer from or to Exchange or Office365 accounts?
R. Yes. But IMAP access to a Exchange or Office365 account is not always
allowed by default so it has to be allowed in the server configuration
part:
part.
Exchange: https://technet.microsoft.com/en-us/library/bb124489(v=exchg.150).aspx
Office365: https://support.microsoft.com/en-us/help/2416434/how-to-enable-or-disable-pop3-imap-mapi-outlook-web-app-or-exchange-ac
=======================================================================
Q. How to sync from XXX to Exchange 2010/2013/2016
@ -71,11 +77,17 @@ Q. How to sync from XXX to Exchange 2010/2013/2016
R0. IMAP is not enable by default on Exchange, see how to enable it:
https://technet.microsoft.com/en-us/library/bb124489(v=exchg.150).aspx
R1. Following is a command line resume that solves most encountered
issues when migrating to Exchange. To fully understand or change
the details you have to read the next Q/R sections.
On Windows:
On Windows, use:
imapsync.exe ... --exchange2
or use (before imapsync release 1.836):
imapsync.exe ... ^
--maxsize 10000000 ^
@ -84,7 +96,11 @@ On Windows:
--disarmreadreceipts ^
--regexmess "s,(.{9900}),$1\r\n,g"
On Unix:
On Unix, use:
imapsync ... --exchange2
or use (before imapsync release 1.836):
imapsync ... \
--maxsize 10000000 \
@ -102,14 +118,22 @@ instead of --maxlinelengthcmd 'reformime -r7'
=======================================================================
Q. How to sync from XXX to Office365
R0. IMAP is not enable by default on Office365, see how to enable it:
https://support.microsoft.com/en-us/help/2416434/how-to-enable-or-disable-pop3-imap-mapi-outlook-web-app-or-exchange-ac
https://technet.microsoft.com/en-us/library/gg298947(v=exchg.150).aspx
https://support.context.io/hc/en-us/articles/213586243-Enabling-IMAP-for-Office365
R. Here is a command line resume that solves most encountered issues when
migrating to Office365. It's similar with Exchange except for some
values. To understand or change the details you have to read
next Q/R sections.
On Windows:
On Windows, use:
imapsync.exe ... --office2
or use (before imapsync release 1.836):
imapsync.exe ... ^
--maxsize 45000000 ^
--maxmessagespersecond 4 ^
@ -117,7 +141,11 @@ On Windows:
--disarmreadreceipts ^
--regexmess "s,(.{10500}),$1\r\n,g"
On Unix:
On Unix, use:
imapsync ... --office2
or use (before imapsync release 1.836):
imapsync ... \
--maxsize 45000000 \

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Folders_Mapping.txt,v 1.13 2017/05/24 19:18:41 gilles Exp gilles $
$Id: FAQ.Folders_Mapping.txt,v 1.15 2017/11/13 00:01:14 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -12,7 +12,19 @@ the prefix and the separator which are automatically adapted
for host2.
Before using --regextrans2 you should consider using --automap
and -f1f2 because they are simpler to understand and use.
and --f1f2 because they are simpler to understand and use.
--automap : guesses folders mapping, for folders like
"Sent", "Junk", "Drafts", "All", "Archive", "Flagged".
--f1f2 str1=str2 : Force folder str1 to be synced to str2,
--f1f2 overrides --automap and --regextrans2.
--subfolder2 str : Move whole host1 folders hierarchy under this
host2 folder str .
It does it by adding two --regextrans2 options before
all others. Add --debug to see what's really going on.
Things to know and understand before playing with --regextrans2

View file

@ -0,0 +1,24 @@
#!/bin/cat
$Id: FAQ.Folders_Sizes.txt,v 1.1 2017/09/29 11:10:53 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips to know folders sizes.
=======================================================================
=======================================================================
Q. How can I know the size of all folders before launching a sync?
R. Use --justfoldersizes option.
With --justfoldersizes option, imapsync displays the folder size
part of a run and then exits without doing any transfer:
imapsync ... --justfoldersizes
=======================================================================
=======================================================================

View file

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: FAQ.General.txt,v 1.233 2017/09/07 22:08:42 gilles Exp gilles $
# $Id: FAQ.General.txt,v 1.238 2018/03/07 04:26:04 gilles Exp gilles $
=======================================================================
General FAQ for imapsync
@ -9,7 +9,7 @@ This document is also available at
https://imapsync.lamiral.info/#doc
https://imapsync.lamiral.info/FAQ.d/FAQ.General.txt
Questions anwswered in this FAQ are:
Questions answered in this FAQ are:
Q. Do I need to create IMAP mailboxes at the destination platform?
@ -55,8 +55,6 @@ Q. How can I use imapsync with Mail::IMAPClient 2.2.9 perl module?
Q. How to verify imapsync.exe I got is the right file bit per bit?
Q. Can I use imapsync to migrate emails from pop3 server to imap server?
Q. Folders are not created on host2. What happens?
Q. I am interested in creating a local clone of the IMAP on a LAN
@ -112,6 +110,7 @@ Q: How can I write an .rpm with imapsync
Q. Where I can read up on the various IMAP RFCs?
Now the questions again with their answers.
=======================================================================
Q. Do I need to create IMAP mailboxes at the destination platform?
@ -225,14 +224,12 @@ users who bought the support get my help first.
=======================================================================
Q. Where I can find old imapsync releases?
R. https://imapsync.lamiral.info/dist/old_releases/
R. Search the internet or ask the author.
=======================================================================
Q. Where I can find free open and gratis imapsync releases?
R. https://imapsync.lamiral.info/dist/
Github has a copy at
https://github.com/imapsync/imapsync
R. Search the internet or ask the author.
Q. Is is legal to find imapsync gratis (or not) elsewhere?
@ -297,12 +294,6 @@ R. Add also --noexpungeaftereach if you use --delete.
imapsync ... --delete --noexpunge --noexpungeaftereach --expunge2
R. Add also --nocheckmessageexists
--nocheckmessageexists is on by default since release 1.520.
Since transfer can be long on a huge mailbox imapsync checks
a message exist before copying it, but it takes time and
cpu on the host1 server.
Notes about --useuid
Case where building the cache first is necessary (to avoid multiples transfers)
@ -436,19 +427,6 @@ R. Use md5sum to check integrity of the file.
Then compare the checksum with the one given by the author.
=======================================================================
Q. Can I use imapsync to migrate emails from pop3 server to imap server?
R1. No, but you can migrate emails from a pop3 server to an imap server
with the command line tool pop2imap:
http://www.linux-france.org/prj/pop2imap/
R2. Yes, sometimes, because many pop3 servers runs in parallel
with an imap server on exactly the same mailboxes. They serve
the same INBOX (imap serves INBOX and several other folders,
pop3 serves only INBOX).
So have a try with imapsync on the same host1.
=======================================================================
Q. Folders are not created on host2. What happens?
@ -774,7 +752,7 @@ R. Here:
RFC 3501 - INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1
http://www.faqs.org/rfcs/rfc3501.html
RFC2683 - IMAP4 Implementation Recommendations
RFC 2683 - IMAP4 Implementation Recommendations
http://www.faqs.org/rfcs/rfc2683.html
RFC 2595 - Using TLS with IMAP, POP3 and ACAP
@ -786,7 +764,7 @@ http://www.faqs.org/rfcs/rfc2822.html
RFC 2342 - IMAP4 Namespace
http://www.faqs.org/rfcs/rfc2342.html
RFC2180 - IMAP4 Multi-Accessed Mailbox Practice
RFC 2180 - IMAP4 Multi-Accessed Mailbox Practice
http://www.faqs.org/rfcs/rfc2180.html
RFC 4549 - Synchronization Operations for Disconnected IMAP4 Clients

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Gmail.txt,v 1.33 2017/09/05 15:11:20 gilles Exp gilles $
$Id: FAQ.Gmail.txt,v 1.37 2018/03/23 19:07:37 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -7,7 +7,7 @@ This documentation is also at http://imapsync.lamiral.info/#doc
Imapsync tips for Gmail accounts.
=======================================================================
Questions anwswered in this FAQ are:
Questions answered in this FAQ are:
Q. Can I use imapsync to transfer from or to Gmail accounts?
@ -46,6 +46,9 @@ Q. How to use a Gmail account to backup several different imap accounts?
Q. How to migrate email from gmail to google apps?
Now the questions again with their answers.
=======================================================================
Q. Can I use imapsync to transfer from or to Gmail accounts?
@ -84,6 +87,15 @@ Q. How to synchronize from Gmail to Gmail?
R. Use the following example:
./imapsync \
--user1 account1@gmail.com \
--password1 gmailsecret1 \
--user2 account2@gmail.com \
--password2 gmailsecret2 \
--gmail12
Or, replacing what does --gmail12:
./imapsync \
--host1 imap.gmail.com \
--ssl1 \
@ -142,6 +154,15 @@ Q. How to synchronize from XXX to Gmail?
R. Use the following example:
imapsync --host1 mail.oldhost.com \
--user1 my_email@oldhost.com \
--password1 password \
--user2 my_email@gmail.com \
--password2 password \
--gmail2
Or, replacing what does --gmail2:
imapsync --host1 mail.oldhost.com \
--user1 my_email@oldhost.com \
--password1 password \
@ -223,6 +244,15 @@ Q. How to synchronize from Gmail to XXX?
R. Use this example:
./imapsync \
--user1 gilles.lamiral@gmail.com \
--password1 gmailsecret \
--host2 localhost \
--user2 tata \
--gmail1
Or, replacing what does --gmail1:
./imapsync \
--host1 imap.gmail.com \
--user1 gilles.lamiral@gmail.com \
@ -237,9 +267,6 @@ R. Use this example:
--automap \
--regextrans2 "s,\[Gmail\].,," \
--skipcrossduplicates \
--folderfirst "Work" \
--folderfirst "Friends" \
--folderlast "CanWait" \
--folderlast "[Gmail]/All Mail"
@ -391,7 +418,8 @@ On Unix:
On Windows:
imapsync.exe ... --search "X-GM-RAW ""has:attachment in:unread"""
perl imapsync ... --search "X-GM-RAW ""has:attachment in:unread"""
=======================================================================
Q. How to avoid the [IMAP] prefix on Gmail side?

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Massive.txt,v 1.15 2017/06/17 13:45:15 gilles Exp gilles $
$Id: FAQ.Massive.txt,v 1.17 2017/12/20 03:38:07 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -123,10 +123,17 @@ How many syncs can we run in parallel? here comes measurements.
nload -t 6000 eth0 -u K
Another excellent network tool is dstat:
Another good network tool is dstat:
dstat -n -N eth0 6
A excellent tool for this purpose is iftop, the following
command will monitor imap and imaps connexions
on interfce eth0, only them, and sum them up:
iftop -i eth0 -f 'port imap or port imaps' -B
On Windows, get the overall network rate with the classical
task manager (Ctrl-Alt-Sup), there is a network tab in it.
Don't hesitate to send me free good tools to measure the
@ -164,14 +171,14 @@ Same thing if they both decrease!
R2. Isolating and overcoming bottlenecks
On any process involving several mechanisms, among all elements taking
part on the process there is always a bottleneck. No one knows in
part on the process, there is always a bottleneck. No one knows in
advance what is the first bottleneck. The first bottleneck has to be
determined, by measurements, not by guesses. Once this first
bottleneck is known and overcome then the next bottleneck has to be
determined and overcome too, if needed. Repeat the process of looking
for the next bottleneck and its elimination until you estimate the
transfer rates, money costs and final dates are good enough to proceed
the whole huge migration.
transfer rates, money costs, time spent on this, and final dates
are good enough to proceed the whole huge migration.
Possible bottlenecks:
@ -180,7 +187,7 @@ Possible bottlenecks:
For example Gmail, Office365, Exchange have throttle limits.
- Bandwidth.
Usually available bandwidth is not a bottleneck.
Usually available bandwidth is NOT a bottleneck.
Meanwhile, it can be a bottleneck on small Internet connexions.
Imapsync downloads messages from host1 and upload messages to host2,
consider this in case the connexion are asymetric.
@ -189,6 +196,11 @@ Possible bottlenecks:
I/O are a classical bottleneck, almost always forgotten.
Unlike CPU and RAM, Input/Output performances don't improve
very much as time goes on so it's often a bottleneck.
To measure and overcome an I/O disk bottleneck, you need
usually a direct access to host1 and host2.
An I/O bottleneck where imapsync runs is possible if
--usecache or --useuid is used or with very big messages.
- RAM memory.
On all sides, monitor that your systems don't swap on disk,

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Messages_Selection.txt,v 1.12 2017/09/05 15:11:20 gilles Exp gilles $
$Id: FAQ.Messages_Selection.txt,v 1.13 2018/03/07 04:12:37 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -7,7 +7,7 @@ This documentation is also at http://imapsync.lamiral.info/#doc
Imapsync tips to select messages.
=======================================================================
Questions anwswered in this FAQ are:
Questions answered in this FAQ are:
Q. What messages imapsync syncs by default?

View file

@ -0,0 +1,49 @@
#!/bin/cat
$Id: FAQ.Migration_Plan.txt,v 1.2 2018/03/07 04:28:00 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
=====================================================================
Imapsync. Suggestions for a good, low impact on users,
well executed email migration plan.
=====================================================================
A good migration plan:
* Create the new mailboxes on the destination server.
* Decrease the TTL of the MX, as well as the imap hostname resolution,
to 5 minutes (or even less). See FAQ.TTL.txt to understand why.
* Presync all the mailboxes from the old to the new server, using different name
that the ones used by the imap software clients (use their IP for example).
Presyncs can usefully be done with --delete2 but never use --delete2 once users
started playing with their new account on host2.
* Decide a migration day/hour
* repeat the presyncs (--delete2) daily until the migration hour.
This repeated process will show how long should take the last sync.
* At the migration hour, cut access to the users to the old server
(by changing the imap hostname to a non-imap server for example).
* Change also the MX (for the new messages arriving in the new imap server).
* Wait 5 minutes.
* Do a last sync exactly like the presyncs, not using the imap hostname.
* Shutdown the old imap server.
Now a fork in the process, depending on the response to the following question:
Will the imap software clients use the exact same
triplet credentials as the old server? hostname/username/password
If the answer is yes (clients use the exact same triplet credentials):
* Change the user imap hostname resolution from the old to the new imap server
* Migration done.
If the answer is no (clients use a different triplet credentials):
* Tell the users the old imap server is down and no longer available.
* Give access to new accounts to the users with their new credential triplet
hostname/username/password (the most similar the best for them).
If the way to contact users is email then you should give this long before
shutting down the old server.
* Migration Done

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.OnlineUI.txt,v 1.1 2017/05/04 23:24:55 gilles Exp gilles $
$Id: FAQ.OnlineUI.txt,v 1.5 2018/05/02 12:22:13 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -8,24 +8,93 @@ This documentation is also at http://imapsync.lamiral.info/#doc
https://imapsync.lamiral.info/X/
=====================================================================
Q .Will I have any issues with browser timing out? What happens
if the connection is closed for whatever reason?
Questions answered in this FAQ are:
Q. How secure is the online visual user interface /X?
Q. Will I have any issues with browser timing out? What happens
if the browser connection is closed for whatever reason?
Now the questions again with their answers.
=====================================================================
Q. How secure is the online visual user interface /X?
R0. Well, I don't know if asking the provider whether his online
service is secure or not would be of any interest.
Let's do it anyway, you'll be the judge.
R1. Some figures
Date of this report: 8 March 2018.
The online imapsync service /X started 9 January 2017 (422 days of service).
In average, /X has 20 users per day lunching 6 different migrations, from
one launch to many many (hundreds).
The total volume /X transferred is 17 TiB in 48000 email imap migrations.
R2. Pros & Cons
The online imapsync service /X runs on https only, with a
letsencrypt certificate, a certificate overall rated "A" at
https://www.ssllabs.com/ssltest/analyze.html?d=imapsync.lamiral.info
Because of the https usage, what the users enter in their browser,
the imap logins and passwords, can't be eavesdropped on the network.
Imapsync itself cares about encryption for the imap sessions,
if possible: It tries SSL first on port 993, then TLS if the
servers announces TLS, then no encryption. What is done with
an imap server is independent of what is done with the other.
At the date of 8 March 2018, there is no security problem detected
or reported to me (Gilles LAMIRAL) so far.
As the owner of the service, it could have been 48 000 pairs of
credentials collected and nearly 17 terabytes of email messages.
I haven't kept them but I can't prove I haven't. It's just trust,
like nearly every online service in the universe.
The imap server certificates are not checked (by default)
because too many imap servers are crappy configured regarding
certificates.
This default behavior is chosen like this because users of /X
wants their emails transferred, instead of not trasferred because
of an incompetent imap server sysadmin.
Anyway, this part, checking imap ssl/tls certificates, could be
improved from my side by including well known certificates
directly in imapsync.
If the imap servers don't honor ssl nor tls, then logins, passwords
and everything will go clear text during the imap transfers.
That's not good at all but what "comforts" me is that if the
imap servers do only clear text transfer, then it is also true
for all imap sessions the owner of the accounts encounter,
imapsync is just one of them.
Last point, who could be sure no cracker cracked the online host and
currently sniffs the credentials? No one, I'm not sure myself, even
if I do take care of that possibility.
=====================================================================
Q. Will I have any issues with browser timing out? What happens
if the browser connection is closed for whatever reason?
R. It should be ok
With the /X interface there is two connections (three connections in
fact but let simplify the picture), 1 is the Browser-WebServer
connection, 2 is the WebServer-ImapServers connections (imapsync
stuff).
When using the /X interface there are three connections.
One connection is the Browser-WebServer connection, the
two others are the WebServer-ImapServers connections (imapsync stuff).
If the Browser-WebServer connection is timeout (but it shouldn't
because of the log refresh), the imapsync sync might continue
anyway. To see if it continues or not, just do a sync again and the
interface will tell you that a sync is already going on, if the
"Sync!" button is grey/inactive then just reload the page (F5 or
similar).
"Sync!" button is gray/inactive then just reload the page (F5 or
similar), and reenter the credentials.
Anyway, on the /X you can try to do several parallel runs on the same
By the way, on the /X you can try to do several parallel runs on the same
mailbox even if there is no timeout, open a new tab/windows with /X
and start a same sync, it's safe, the /X will say, if any, that there
is already a current sync.
@ -34,3 +103,6 @@ You can stop this sync with the "Abort!" button from any /X
tab/window, even from another browser or place. To doing this with
success, you have to give the same account parameters, same
credentials, or imapsync will ignore the demand.
=====================================================================
=====================================================================

60
FAQ.d/FAQ.POP3.txt Normal file
View file

@ -0,0 +1,60 @@
#!/bin/cat
$Id: FAQ.POP3.txt,v 1.2 2018/02/02 20:32:19 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
=======================================================================
Imapsync tips about POP3.
=======================================================================
Questions answered here are:
Q. Can I use imapsync to migrate emails from pop3 server to imap server?
Q. How can I handle those terrible POP-leaving-a-copy users?
POP-leaving-a-copy clients will download a new copy of all
their emails after a migration, which is a big pain and
causes hundreds of support calls.
=======================================================================
Q. Can I use imapsync to migrate emails from pop3 server to imap server?
R1. No, but you can migrate emails from a pop3 server to an imap server
with the command line tool pop2imap:
http://www.linux-france.org/prj/pop2imap/
R2. Yes, sometimes, because many pop3 servers runs in parallel
with an imap server on exactly the same mailboxes. They serve
the same INBOX (imap serves INBOX and several other folders,
pop3 serves only INBOX).
So have a try with imapsync on the same host1.
=======================================================================
Q. How can I handle those terrible POP-leaving-a-copy users?
POP-leaving-a-copy clients will download a new copy of all
their emails after a migration, which is a big pain and
causes hundreds of support calls.
R. A discussion about this issue
http://www.linux-france.org/prj/imapsync_list/msg02622.html
http://www.linux-france.org/prj/imapsync_list/msg02623.html
My reply was not very good
http://www.linux-france.org/prj/imapsync_list/msg02624.html
It's because I wrote pop2imap and it uses a stateless
mechanism to avoid duplicates, I wasn't aware of UIDL.
Here is a better response to handle and fix this problem, I hope:
If the pop mailboxes are known and known to be regularly fetched, then
all the INBOX messages, or at least the "old" ones, can be moved in an
subfolder INBOX_back. It requires to have "access" to the POP
sessions. The POP server log file is usually enough to know which
account opened a pop connection, and when.
See also Rick Sanders discussion about this issue:
https://www.emailquestions.com/threads/how-to-prevent-duplicate-emails-after-server-migration-for-pop3-users-with-leave-mail-on-server.8109/
=======================================================================
=======================================================================

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Release_Checklist.txt,v 1.5 2017/09/07 12:38:45 gilles Exp gilles $
$Id: FAQ.Release_Checklist.txt,v 1.6 2017/09/16 20:21:39 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -19,4 +19,9 @@ Checklist before release a new release:
- Review the newsletter by running:
m4 -P W/ml_announce.in
- Review the general FAQ.d/FAQ.General.txt
- Review the general FAQ.d/FAQ.General.txt
- make dist
- make publish
- verify VERSION is uploaded

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Reporting_Bugs.txt,v 1.2 2017/09/07 12:10:49 gilles Exp gilles $
$Id: FAQ.Reporting_Bugs.txt,v 1.3 2018/04/10 00:17:18 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -13,7 +13,7 @@ Q. How can I report bugs or problems I encountered with Imapsync?
R. Help me to help you: follow the following guidelines.
Report any bug or feature request directly to the author by
email at <gilles.lamiral@laposte.net>
email at <gilles@lamiral.info>
Put a useful title with word "imapsync" in it: my spam filters
won't filter it.

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.SSL_errors.txt,v 1.1 2017/05/24 19:16:40 gilles Exp gilles $
$Id: FAQ.SSL_errors.txt,v 1.5 2018/04/25 17:58:12 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -7,23 +7,26 @@ This documentation is also at http://imapsync.lamiral.info/#doc
Imapsync SSL errors
=====================================================
Questions anwswered in this FAQ are:
Questions answered in this FAQ are:
Q. What are the errors
DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
or
DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error
Q. What can I do to avoid those "SSL read/write errors"?
Q. SSL connect attempt failed SSL
routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
=======================================================================
Q. What are the errors
DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
or
DEBUG: .../IO/Socket/SSL.pm:1088: local error: SSL read error
R. Like they claim, those errors are SSL errors. SSL is not directly
R1.Like they claim, those errors are SSL errors. SSL is not directly
done by imapsync but by an underlying Perl module called
IO::Socket::SSL. Those errors arise sometimes and sometimes
they form a serie that ends with imapsync auto-abortion.
@ -35,6 +38,9 @@ R. Like they claim, those errors are SSL errors. SSL is not directly
=======================================================================
Q. What can I do to avoid those "SSL read/write errors"?
R0. Windows users: upgrade to imapsync.exe release 1.836 (or next ones)
Those errors don't appear with recent releases, post 1.836
R1. Remove all ssl/tls encryption
imapsync ... --nossl1 --notls1 --nossl2 --notls2
@ -48,9 +54,28 @@ R3. Run imapsync on a Linux machine, a VM is ok, there are less
SSL errors on Unix.
R4. Use https://imapsync.lamiral.info/X/
It's a Linux host so R3 applies there.
It's a Linux host so response R3 applies there.
R5. Set up a ssltunnel proxy to the host.
Read the file FAQ.Security.txt
Read the file FAQ.Security.txt for an example to set up
a ssltunnel proxy.
=======================================================================
Q. SSL connect attempt failed SSL
routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
R. Use:
imapsync ... --sslargs1 'SSL_cipher_list=DEFAULT'
or
imapsync ... --sslargs2 'SSL_cipher_list=DEFAULT'
depending on where the error occurs, host1 or host2 or both.
=======================================================================
=======================================================================
=======================================================================
=======================================================================

View file

@ -1,5 +1,5 @@
$Id: FAQ.Use_cache.txt,v 1.4 2017/02/14 17:35:22 gilles Exp gilles $
$Id: FAQ.Use_cache.txt,v 1.6 2018/03/07 04:12:37 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -7,13 +7,13 @@ This documentation is also at http://imapsync.lamiral.info/#doc
Imapsync --usecache option
============================================
Questions anwswered in this FAQ are:
Questions answered in this FAQ are:
Q. On Windows, with --useuid or --usecache a problem occurs with long
nested folder names. The error message is:
"No such file or directory; The filename or extension is too long"
Q. Inode problem with --usecache on Linux
Q. Inode problem with --usecache or --useuid on Linux
Questions and their answers:
@ -68,7 +68,7 @@ Fixing this long path problem directly in imapsync is in the TODO file
for a very long time.
=======================================================================
Q. Inode problem with --usecache on Linux
Q. Inode problem with --usecache or --useuid on Linux
R. You may run out of inodes using --usecache, especially with large
migration. Option --usecache creates a empty file per email message

View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.User_Concurrent_Access.txt,v 1.2 2017/01/06 14:11:13 gilles Exp gilles $
$Id: FAQ.User_Concurrent_Access.txt,v 1.4 2017/12/19 14:55:36 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -14,23 +14,30 @@ R. Think all potential problems with a sync between
a master/source/host1 and a slave/destination/host2
What happens when a user access syncing accounts depends
on what he does, where he does and when:
on what he does, where he does and when.
* Reading: no problem anywhere, anytime.
As a summary, reading is safe, writing/deleting is not.
Details:
* Reading: no problem anywhere, at source/host1 or at destination/host2,
anytime. Reading is safe.
* Writing at source/host1:
* adding folder => will need a resync after
* deleting folder => deletion not synced unless --delete2folders
* deleting message => deletion not synced unless --delete2
* adding message => will need a resync
* adding folder => will need a resync after
* deleting folder => folders deletions are not synced unless --delete2folders
* deleting message => messages deletions are not synced unless --delete2
* adding message => will need a resync
* moving message => will need a resync. Moving is like adding + deleting
so --delete2 is needed to sync the deletion.
* message flag change => will need a resync
* Writing at destination/host2:
* adding folder => no problem, unless --delete2folders is used
* deleting folder => might reappear by the sync
* adding message => no problem, unless --delete2 is used
* deleting message => might reappear by the sync
* message flag change => might be canceled by the sync
* adding folder => no problem, unless --delete2folders is used
* deleting folder => will reappear by a new sync
* adding message => no problem, unless --delete2 is used on next sync.
* deleting message => might reappear by a new sync.
* message flag change => will be reseted by a new sync.
=======================================================================
=======================================================================

View file

@ -1,5 +1,5 @@
$Id: FAQ.Virus.txt,v 1.2 2017/07/03 19:30:19 gilles Exp gilles $
$Id: FAQ.Virus.txt,v 1.4 2018/03/21 16:18:53 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -11,7 +11,7 @@ This documentation is also at http://imapsync.lamiral.info/#doc
Q. My virus scanner claims imapsync.exe is a virus/malware/trojan etc.
What the hell?
R. Yes, I found the same. Two antivirus, Baidu and Jiangmin, report that
R1. Yes, I found the same. Two antivirus, Baidu and Jiangmin, report that
imapsync might have a trojan in it.
All others virus scanner say imapsync.exe is ok.
I've done this test on imapsync.exe release 1.727 on the two following
@ -26,9 +26,18 @@ R2. Explanation:
https://imapsync.lamiral.info/#NUMBERS
Any other explanation is welcome!
R3. There is no virus alerts reported for the Perl script imapsync
itself. The binary imapsync.exe is just a compiled version of
the script imapsync, with perl interpreter itself and all modules
needed by imapsync in order to make imapsync.exe a standalone software.
needed by imapsync in order to make imapsync.exe a standalone software.
R4. imapsync.exe release 1.836 is reported as totally clean by both
https://www.virustotal.com/
https://www.virustotal.com/#/file/2a33eb93198da77677a4013a5dfb4868cb6aa8f4513e2102a55884b258d19043/detection
and https://metadefender.opswat.com/
https://metadefender.opswat.com/results#!/file/ZTE4MDMxN1MxZTk4aGhxdEdCSlpxVTJoOVlN/regular/information

View file

@ -1,5 +1,5 @@
$Id: FAQ.Yahoo.txt,v 1.6 2017/06/17 14:42:01 gilles Exp gilles $
$Id: FAQ.Yahoo.txt,v 1.7 2017/12/18 10:45:00 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -20,10 +20,28 @@ R. Use --host1 imap.mail.yahoo.com --ssl1
--ssl1 \
--host2 XXX \
--user2 billy \
--password2 secret
--password2 secret \
--noabletosearch --errorsmax 10000
SSL is mandatory for yahoo since November 2011.
Why increasing --errorsmax to 10000?
Because during the sync you may encounter several lines like this one:
- msg Inbox/12600 {0} S[36012] F[] I[27-Nov-2015 07:08:35 +0000] could not be fetched: 190 NO [UNAVAILABLE] UID FETCH Server error while fetching messages
- msg Inbox/12600 skipped.
I don't know why it happens, why Yahoo server says
"NO [UNAVAILABLE] UID FETCH Server error while fetching messages"
Usually another run syncs the message successfully.
Why --noabletosearch option?
Yahoo IMAP does not honor well the SEARCH imap command (Dec 2017).
So don't use the --search option.
You can use --minage or/and --maxage but then add also --noabletosearch
Yahoo bandwidth per connexion is relatively low, 50 KiB/s to 500 KiB/s max, usually 100 KiB/s.
Consider using --useuid to speed up large accounts.
You also need to go to Yahoo, security and enable
"Allow less secure apps to login",
otherwise the login will not work.

View file

@ -1,5 +1,5 @@
# $Id: htaccess.txt,v 1.18 2017/09/03 03:14:48 gilles Exp gilles $
# $Id: htaccess.txt,v 1.20 2018/03/17 16:12:54 gilles Exp gilles $
AddDescription "<b>Back to Imapsync main page</b>." ..
@ -16,22 +16,32 @@ AddDescription "<b>Emptying</b> an account." FAQ.Emptying.
AddDescription "<b>Exchange 20xx</b> and <b>Office365</b>." FAQ.Exchange.txt
AddDescription "<b>Changing folders names</b>." FAQ.Folders_Mapping.txt
AddDescription "<b>Selecting folders</b>." FAQ.Folders_Selection.txt
AddDescription "<b>Folders sizes with Imapsync</b>." FAQ.Folders_Sizes.txt
AddDescription "<b>Flags</b>." FAQ.Flags.txt
AddDescription "<b>General</b> and <b>pot-pourri</b> issues" FAQ.General.txt
AddDescription "<b>Gmail</b> accounts." FAQ.Gmail.txt
AddDescription "<b>ISP tips</b>." FAQ.ISP.txt
AddDescription "<b>Massive/bulk migrations</b>." FAQ.Massive.txt
AddDescription "<b>Memory issues</b>." FAQ.Memory.txt
AddDescription "<b>Migration Plan</b>." FAQ.Migration_Plan.txt
AddDescription "<b>The Online UI service Q&R</b>." FAQ.OnlineUI.txt
AddDescription "<b>Password & special characters on Windows</b>." FAQ.Passwords_on_Windows.txt
AddDescription "<b>Password & special characters on Unix</b>." FAQ.Passwords_on_Unix.txt
AddDescription "<b>POP3 issues</b>." FAQ.POP3.txt
AddDescription "<b>Selecting messages</b>." FAQ.Messages_Selection.txt
AddDescription "<b>Oracle-UCS</b>." FAQ.Oracle-UCS.txt
AddDescription "<b>Checklist before release a new release</b>." FAQ.Release_Checklist.txt
AddDescription "<b>Guidelines to report bugs</b>." FAQ.Reporting_Bugs.txt
AddDescription "<b>Security</b>." FAQ.Security.txt
AddDescription "<b>SSL errors</b>." FAQ.SSL_errors.txt
AddDescription "<b>SmarterMail</b>." FAQ.SmarterMail.txt
AddDescription "<b>TTL (Time To Live) delay in DNS configuration</b>." FAQ.TTL.txt
AddDescription "<b>Option --usecache and inodes</b>." FAQ.Use_cache.txt
AddDescription "<b>User concurrent access</b>." FAQ.User_Concurrent_Access.txt
AddDescription "<b>Virus scanners on Imapsync</b>." FAQ.Virus.txt
AddDescription "<b>Why use --addheader?</b>." FAQ.Use_addheader.txt
AddDescription "<b>Various imap server</b> softwares." FAQ.Various_Server_Softwares.txt
AddDescription "<b>XOAUTH2</b> (<b>Gmail</b>)." FAQ.XOAUTH2.txt

4
FAQ.d/memo Normal file
View file

@ -0,0 +1,4 @@
( cd .. && make upload_FAQ )