diff --git a/CREDITS b/CREDITS index fe82a83..c5f44fb 100644 --- a/CREDITS +++ b/CREDITS @@ -1,5 +1,5 @@ #!/bin/cat -# $Id: CREDITS,v 1.191 2018/04/08 17:27:58 gilles Exp gilles $ +# $Id: CREDITS,v 1.192 2018/10/04 10:20:47 gilles Exp gilles $ If you want to make a donation to me, imapsync author, Gilles LAMIRAL, use any of the following ways: @@ -29,6 +29,9 @@ Sean found the solution and wrote the FAQ item in FAQ.Exchange.txt "NO Maximum size of appendable message has been exceeded" and Ian & Matt reported it to me. +Tomasz Makara +Gave FAQ.Massive.txt fixes about network monitoring on Windows any release. + Joe Pruett Bugfix about delete_message_on_host1() not using --noexpungeaftereach diff --git a/ChangeLog b/ChangeLog index 0d937e6..8776928 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,17 +1,209 @@ RCS file: RCS/imapsync,v Working file: imapsync -head: 1.882 +head: 1.920 branch: locks: strict - gilles: 1.882 + gilles: 1.920 access list: symbolic names: keyword substitution: kv -total revisions: 882; selected revisions: 882 +total revisions: 920; selected revisions: 920 description: ---------------------------- -revision 1.882 locked by: gilles; +revision 1.920 locked by: gilles; +date: 2019/02/09 15:45:31; author: gilles; state: Exp; lines: +37 -33 +Better "Extra arguments found" error message, added "or some misspelling options" +---------------------------- +revision 1.919 +date: 2019/02/09 14:20:41; author: gilles; state: Exp; lines: +74 -32 +Added authmech X-MASTERAUTH. To be used for Kerio with --authmech1 X-MASTERAUTH or --authmech2 X-MASTERAUTH +Patch from https://github.com/imapsync/imapsync/pull/53/files +---------------------------- +revision 1.918 +date: 2019/02/07 12:16:28; author: gilles; state: Exp; lines: +79 -44 +Fixed test since macosx.polarhome.com can not do ipv6 these days. +Allow --tests --testslive for cover analyse +---------------------------- +revision 1.917 +date: 2019/02/06 12:13:44; author: gilles; state: Exp; lines: +279 -110 +Added --addheader description in the README part. +Bugfix --debugssl 0 was impossible and equivalent to --debugssl 1 +Deglobalized $justfolders now $sync->{ justfolders } +Deglobalized %h2_folders_of_md5 it's now $sync->{ h2_folders_of_md5 } +Bugfix. Now split the deletion of messages in several chunks (--split) +Added stat "Messages found in host1 not in host2" +Added stat "Messages found in host2 not in host1" +Removed stat "Total bytes duplicate hostX" +Removed stat "Total bytes error" +Added stat "Load end is" +Added comments on final stat about messages in 1 not in 2 +Added comments on final stat about messages in 2 not in 1 +Added comment on final stat about unidentified messages +1264 unit tests. +---------------------------- +revision 1.916 +date: 2019/01/05 14:45:10; author: gilles; state: Exp; lines: +550 -470 +Added missing "Entering tests_..." and "Leaving tests_..." in tests_* subroutines. +Some perlcritic fixes. +---------------------------- +revision 1.915 +date: 2019/01/04 12:39:56; author: gilles; state: Exp; lines: +12 -15 +Remove the unit test $imap->new() $imap->connect() because IO::Socket::IP kills us on this. +---------------------------- +revision 1.914 +date: 2019/01/03 04:38:10; author: gilles; state: Exp; lines: +10 -10 +unit test revised for match_a_pid_number() +---------------------------- +revision 1.913 +date: 2019/01/03 04:31:11; author: gilles; state: Exp; lines: +9 -8 +A pid can be up to 99999 on FreeBSD +---------------------------- +revision 1.912 +date: 2019/01/03 04:18:54; author: gilles; state: Exp; lines: +14 -14 +Added prints in sub remove_pidfile_not_running() in order to debug bad behavior on freeBSD +---------------------------- +revision 1.911 +date: 2019/01/03 02:52:43; author: gilles; state: Exp; lines: +29 -11 +Added --sigprint HUP in order to allow some asynchronous prints in CGI mode and avoid timeouts from browser +---------------------------- +revision 1.910 +date: 2018/12/25 18:20:48; author: gilles; state: Exp; lines: +77 -38 +Added --addheader suggestion at the final stats. +Added a counter of crossduplicates on host2, presented on final stats. +---------------------------- +revision 1.909 +date: 2018/12/09 21:23:54; author: gilles; state: Exp; lines: +8 -8 +Typo. loose lose +---------------------------- +revision 1.908 +date: 2018/11/20 13:22:07; author: gilles; state: Exp; lines: +80 -39 +Played with IO::Prompt and IO::Prompter to remove perl critic. Failed... +Deglobalized $passfile1 and $passfile2 +---------------------------- +revision 1.907 +date: 2018/11/16 10:08:11; author: gilles; state: Exp; lines: +142 -135 +Localized %SIG +Fixed Host1 Host2 prints to be more homogene +---------------------------- +revision 1.906 +date: 2018/11/08 18:37:18; author: gilles; state: Exp; lines: +120 -47 +Added tests_operators_and_exclam_precedence() +Added tests_teelaunch() +Added tests_logfileprepa() +---------------------------- +revision 1.905 +date: 2018/10/22 17:28:02; author: gilles; state: Exp; lines: +10 -9 +Added IO::Socket::IP in the list of modules versions. +---------------------------- +revision 1.904 +date: 2018/10/19 11:57:24; author: gilles; state: Exp; lines: +157 -59 +1238 unit tests +Refactored appendlimit() to use Mail::IMAPClient::capability like the API. No hack from internal $imap->{ APPENDLIMIT } +With --office1 option, folder "Files" is excluded because it's a "special" folder. Like --exclude "^Files$". Unless --noexclude is used. +With --office2 option, folder "Files" on host1 is renamed like --f1f2 "Files=Files_renamed_by_imapsync". Unless --nof1f2 is used. +---------------------------- +revision 1.903 +date: 2018/10/06 09:16:57; author: gilles; state: Exp; lines: +9 -9 +--noskipcrossduplicates with --gmail12 +---------------------------- +revision 1.902 +date: 2018/10/05 19:50:23; author: gilles; state: Exp; lines: +12 -10 +Added --folderlast '[Gmail]/All Mail' within --gmail12 +---------------------------- +revision 1.901 +date: 2018/10/03 23:49:51; author: gilles; state: Exp; lines: +815 -412 +Removed --maxsize 25_000_000 from --gmail2 because now it's get from APPENDLIMIT in CAPABILITY (and it is currently 35651584, greater than 25_000_000) +Moved opening { in all sub to next line. +---------------------------- +revision 1.900 +date: 2018/10/03 23:17:11; author: gilles; state: Exp; lines: +16 -16 +Deglobalized $maxsize +---------------------------- +revision 1.899 +date: 2018/10/03 22:50:23; author: gilles; state: Exp; lines: +332 -14 +Added automatic --maxsize from CAPABILITY APPENDLIMIT=xxxx (Gmail) +---------------------------- +revision 1.898 +date: 2018/09/28 14:12:21; author: gilles; state: Exp; lines: +89 -29 +Added --maxsize 35_651_584 with --gmail2 +(Should me replaced by automatic APPEND_LIMIT in CAPABILITY). Soon. +Added --exclude '^Files$' with --office2 +---------------------------- +revision 1.897 +date: 2018/09/19 14:54:26; author: gilles; state: Exp; lines: +96 -74 +Deglobalized $delete1, $delete2, $delete2duplicates, $expunge2, $uidexpunge2 +Started sub tests_uidexpunge_or_expunge() sub uidexpunge_or_expunge() +---------------------------- +revision 1.896 +date: 2018/09/19 11:51:17; author: gilles; state: Exp; lines: +81 -60 +Deletions on host1 are done by batch during a resync. +---------------------------- +revision 1.895 +date: 2018/09/13 17:38:53; author: gilles; state: Exp; lines: +97 -87 +Refactor. Deglobarized variable + * $exitwhenover, + * $total_bytes_skipped + * $nb_msg_skipped +---------------------------- +revision 1.894 +date: 2018/09/11 13:59:54; author: gilles; state: Exp; lines: +19 -14 +Commented test for @regexflag = ( 's/\\Flagged/X/g' ) since \F is new and was a bug anyway. +---------------------------- +revision 1.893 +date: 2018/09/11 09:09:34; author: gilles; state: Exp; lines: +89 -75 +Bugfix. --exchange1 and --exchange2 were not options! No one complained... +Added inline explanations for --office1 --office2 --exchange1 --exchange2 (what they set and how to unset) +Added --noregexmess to avoid the regexmess setting with --office2 and --exchange2 (wrap long lines >= 10500 char) +Added --noregexflag to avoid the regexflag set by exchange2 (remove \Flagged flag) +Bugfix. The regex to remove flag \Flagged was wrong. It worked because of another filter run after... +---------------------------- +revision 1.892 +date: 2018/09/04 04:04:34; author: gilles; state: Exp; lines: +9 -9 +--skipcrossduplicates activated with --gmail1 --gmail2 +---------------------------- +revision 1.891 +date: 2018/09/04 03:28:09; author: gilles; state: Exp; lines: +175 -123 +Enhancement. Added option --synclabels to sync Gmail labels. It should speed up Gmail to Gmail syncs. + --synclabels applies only during a copy of a message for now. resyncing labels will be coded later. +---------------------------- +revision 1.890 +date: 2018/09/01 21:45:41; author: gilles; state: Exp; lines: +117 -114 +Refactoring. Replaced $imap1 and $imap2 by $sync->{imap1} and $sync->{imap2} +---------------------------- +revision 1.889 +date: 2018/08/31 01:02:06; author: gilles; state: Exp; lines: +19 -7 +Applied https://github.com/imapsync/imapsync/issues/95 +Thanks jh1995! +---------------------------- +revision 1.888 +date: 2018/08/29 11:02:59; author: gilles; state: Exp; lines: +9 -9 +Gives CGI.pm release. +Calls memory_consumption_of_pids_win32() with Cygwin (https://github.com/imapsync/imapsync/pull/22) +---------------------------- +revision 1.887 +date: 2018/08/23 19:10:44; author: gilles; state: Exp; lines: +11 -10 +--maxsleep override $MAX_SLEEP in gmail*() subroutines. +Default $sync->{ sigignore } to empty list. +---------------------------- +revision 1.886 +date: 2018/08/06 15:02:51; author: gilles; state: Exp; lines: +28 -11 +Added mygetppid because USR1 no available on Windows. +Removed --nosubscribed since it is not coded and useless anyway. +---------------------------- +revision 1.885 +date: 2018/07/07 04:14:26; author: gilles; state: Exp; lines: +8 -8 +Sheband uses env now. Fed up to change freebsd /usr/local/bin/perl +---------------------------- +revision 1.884 +date: 2018/07/07 03:40:54; author: gilles; state: Exp; lines: +18 -15 +Added PPID info where PID is printed. +---------------------------- +revision 1.883 +date: 2018/07/03 03:55:03; author: gilles; state: Exp; lines: +32 -8 +Added freebsd support for loadavg and cpu_cores +---------------------------- +revision 1.882 date: 2018/05/05 21:10:43; author: gilles; state: Exp; lines: +8 -8 typo ---------------------------- diff --git a/FAQ.d/FAQ.Admin_Authentication.txt b/FAQ.d/FAQ.Admin_Authentication.txt index baacae5..1fee661 100644 --- a/FAQ.d/FAQ.Admin_Authentication.txt +++ b/FAQ.d/FAQ.Admin_Authentication.txt @@ -1,11 +1,14 @@ #!/bin/cat -$Id: FAQ.Admin_Authentication.txt,v 1.2 2017/01/06 14:11:13 gilles Exp gilles $ +$Id: FAQ.Admin_Authentication.txt,v 1.6 2018/09/09 10:58:16 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Admin_Authentication.txt -============================================== - Imapsync tips about admin authentication. -============================================== + +======================================================================= + Imapsync tips about imap admin authentication. +======================================================================= It can be useful to authenticate without knowing each user password. Using an admin account to authenticate is not a standard feature @@ -17,16 +20,21 @@ if an admin account is actually configured and finally check an authentication via this admin account for a standard user. -Known imap server software supporting admin authentication: +Known imap server software supporting imap admin authentication: * Exchange 2003/2007/2010/2013. See the file FAQ.Exchange.txt * Office365. See the file FAQ.Exchange.txt * Gmail. See the file FAQ.XOAUTH2.txt * Dovecot. See the file FAQ.Dovecot.txt + * Zimbra. See the file FAQ.Zimbra.txt * UW-imap. See the file FAQ.General.txt * Cyrus-imap. See the file FAQ.General.txt * Sun Java Enterprise System/SunOne/iPlanet. See the file FAQ.General.txt - +Known imap server software NOT supporting imap admin authentication: + + * Domino Lotus Notes. + * SmartCloud Notes. + ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.Archiving.txt b/FAQ.d/FAQ.Archiving.txt index 67909c6..d551a04 100644 --- a/FAQ.d/FAQ.Archiving.txt +++ b/FAQ.d/FAQ.Archiving.txt @@ -1,11 +1,13 @@ #!/bin/cat -$Id: FAQ.Archiving.txt,v 1.7 2018/04/24 00:16:21 gilles Exp gilles $ +$Id: FAQ.Archiving.txt,v 1.13 2018/12/09 21:26:25 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Archiving.txt -============================================ - Imapsync issues and tips about archiving. -============================================ +======================================================================= + Imapsync issues and tips about archiving +======================================================================= Questions answered in this FAQ are: @@ -28,10 +30,10 @@ Q. Can I archive different accounts on the same destination account, R. Yes. Use --subfolder2 ---subfolder2 str : Move whole host1 folders hierarchy under the - host2 folder str. - It is done it by adding two --regextrans2 options before - all others. Add --debug to see what's really going on. + --subfolder2 str : Syncs the whole host1 folders hierarchy under the + host2 given folder str. + (It does it internally by adding two --regextrans2 + options before all others.) Example: @@ -48,17 +50,19 @@ in the right place, ie, the subfolder "foo". ======================================================================= Q. How to move emails from one IMAP folder to another either on the same IMAP server or a different one? - For example, all messages older than 1 year, move from INBOX to Archive. + For example, move all messages older than 1 year from INBOX to Archive. R1. Solution: - imapsync ... --minage 365 --folder Inbox --f1f2 INBOX=Archive --delete --noexpungeaftereach + imapsync ... --minage 365 --folder Inbox --f1f2 INBOX=Archive --delete1 --noexpungeaftereach Add option --noexpungeaftereach if speed is a concern. -Add option --delete if you want to move messages, instead of just copy/sync them. +Add option --delete1 if you want to move messages, instead of just copy/sync them. R2. See also file FAQ.Folders_Mapping.txt + +https://imapsync.lamiral.info/FAQ.d/FAQ.Archiving.txt ======================================================================= Q. Can imapsync be used to maintain a local offline copy of a mailbox from host1, eg for backup purposes, so that if the @@ -71,14 +75,16 @@ R2. Imapsync can't but rick-sanders-imap-tools can! See https://github.com/andrewnimmo/rick-sanders-imap-tools Back up and restore IMAP accounts with imapdump.pl and dumptoIMAP.pl + It looks like imapdump.pl has options to avoid duplicates and be + in an update mode. There are options -u -U + [-u] Don't dump messages already dumped + [-U] Don't dump message if it already exists in the dump directory + I wonder the difference, I've never used Rick's tools. + + Remember that doing backups without trying the restore process + is more dangerous than doing no backups at all. No backups makes + people act in a safer way, well, usually... - - - - - - -======================================================================= ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.Authentication_failure.txt b/FAQ.d/FAQ.Authentication_failure.txt index ce8a72b..03faec1 100644 --- a/FAQ.d/FAQ.Authentication_failure.txt +++ b/FAQ.d/FAQ.Authentication_failure.txt @@ -1,25 +1,37 @@ #!/bin/cat -$Id: FAQ.Authentication_failure.txt,v 1.5 2017/09/04 11:03:57 gilles Exp gilles $ +$Id: FAQ.Authentication_failure.txt,v 1.9 2019/01/11 13:13:08 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Admin_Authentication.txt ======================================================================= Imapsync authentication issues ======================================================================= +Questions answered in this FAQ are: +Q. Imapsync fails with the following error, what can I do? +Host1 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 NO [AUTHENTICATIONFAILED] Authentication failed + +Q. Imapsync fails with the following error, what can I do? + Host2 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 BAD Invalid characters in atom + + +======================================================================= +Q. Imapsync fails with the following error, what can I do? Host1 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 NO [AUTHENTICATIONFAILED] Authentication failed -One over four imapsync syncs ends up quickly with the error message -"Authentication failed" or "NO LOGIN failed" or similar. -Authentication failure is the primary failure with imapsync -and since nothing tangible can be done without authentication, -this stage must succeed to go further. +R. One over four imapsync syncs ends up quickly with the error message + "Authentication failed" or "NO LOGIN failed" or similar. + Authentication failure is the primary failure with imapsync + and since nothing tangible can be done without authentication, + this stage must succeed to go further. Here are some advices to get you pass this difficult stage of authentication: - * Triple check each credendial parameter, there are three parameters at each side: + * Triple check each credential parameter, there are three parameters at each side: * triple check --host1 * triple check --user1 * triple check --password1 @@ -40,15 +52,44 @@ Here are some advices to get you pass this difficult stage of authentication: Option --showpasswords shows passwords again when the IMAP dialog is dumped by --debugimap option. Search for a line like "Sending: 2 LOGIN test1 secret1" (secret1 is the password here) - - + + * It is sometimes very hard to quote correctly unusual characters, especially on Windows. See https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt - The quicker trick might be to change the password temporarilly + The quickest trick may be to change the password temporally with easy characters like the classical alphabet, a long - string will still ensure strong security. - + string still ensure strong security. + + * Sometimes some servers announce they support LOGIN but it + actually fails because the authentication mechanism + working is something else like CRAM-MD5 or PLAIN. + So: + * Try --authmech1 CRAM-MD5 (or --authmech2 CRAM-MD5) + * Try --authmech1 PLAIN (or --authmech2 PLAIN) + + * If you want to play manually the IMAP protocol on your server, + here is an example with the command telnet, user test1 + and password secret1 on the host test.lamiral.info. + + telnet test.lamiral.info 143 + c1 LOGIN test1 "secret1" + c2 LOGOUT + + Same example using an ssl conneection: + telnet-ssl -z ssl test.lamiral.info 993 + c1 LOGIN test1 "secret1" + c2 LOGOUT + +======================================================================= +Q. Imapsync fails with the following error, what can I do? + Host2 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2 BAD Invalid characters in atom + +R. It might be a Dovecot imap server. The password string might + contain special characters that Dovecot doesn't like. Change them. + + ======================================================================= + ======================================================================= \ No newline at end of file diff --git a/FAQ.d/FAQ.Contacts_Calendars.txt b/FAQ.d/FAQ.Contacts_Calendars.txt index ea398e2..1709919 100644 --- a/FAQ.d/FAQ.Contacts_Calendars.txt +++ b/FAQ.d/FAQ.Contacts_Calendars.txt @@ -1,23 +1,25 @@ #!/bin/cat -$Id: FAQ.Contacts_Calendars.txt,v 1.7 2018/03/07 04:26:04 gilles Exp gilles $ +$Id: FAQ.Contacts_Calendars.txt,v 1.11 2018/09/26 22:25:32 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Contacts_Calendars.txt -========================================================= - Imapsync issues about syncing Contacts & Calendars. -========================================================= +======================================================================= + Imapsync issues about syncing Contacts & Calendars. +======================================================================= Questions answered in this FAQ are: -Q. Can I copy or sync Calendar or Contacts with imapsync? +Q. Can I copy or sync Calendars or Contacts with imapsync? +Q. Can I copy or sync Chat messages with imapsync? Q. How can I avoid copying Calendar or Contacts folders? Q. How can I copy or synchronize Calendars or Contacts? - Now the questions again with their answers. ======================================================================= -Q. Can I copy or sync Calendar or Contacts with imapsync? +Q. Can I copy or sync Calendars or Contacts with imapsync? R. No, Imapsync can't migrate Contacts and Calendars. It's because most IMAP servers don't get contacts and calendars @@ -26,7 +28,14 @@ R. No, Imapsync can't migrate Contacts and Calendars. or get the contacts or calendars. No way via IMAP, no way via imapsync. So it's a good idea to avoid syncing contacts and calendars. - But see next question. + But see the question below + "How can I copy or synchronize Calendars or Contacts?". + +======================================================================= +Q. Can I copy or sync Chat messages with imapsync? + +R. No, Imapsync can't migrate Chat messages (those in GSuite for example) + The explanation is the same as with Calendars or Contacts. ======================================================================= Q. How can I avoid copying Calendar or Contacts folders? @@ -43,10 +52,11 @@ R1. You can avoid synchronizing Calendar or Contacts folders with Q. How can I copy or synchronize Calendars or Contacts? R1. Synchronizing Calendars or Contacts can't be done with imapsync. - See the first question for detailed explanations. + For detailed explanations, see the question above + "Can I copy or sync Calendars or Contacts with imapsync?". R2. Synchronizing Calendars or Contacts can be done, not - with imapsync, depending on the email server softwares used. + with imapsync, depending on the imap server software tool used. a) From Exchange to Exchange, export contacts and calendar to PST format files on host1 and import them on host2. diff --git a/FAQ.d/FAQ.Dates.txt b/FAQ.d/FAQ.Dates.txt index 6d2ed2e..49d01b4 100644 --- a/FAQ.d/FAQ.Dates.txt +++ b/FAQ.d/FAQ.Dates.txt @@ -1,11 +1,13 @@ #!/bin/cat -$Id: FAQ.Dates.txt,v 1.7 2018/03/07 04:12:37 gilles Exp gilles $ +$Id: FAQ.Dates.txt,v 1.9 2018/11/04 09:27:17 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Dates.txt -=============================== - Imapsync tips about dates. -=============================== +======================================================================= + Imapsync tips about dates. +======================================================================= Questions answered in this FAQ are: @@ -32,32 +34,35 @@ but not with: - Zimbra - Gmail -A thing to keep in mind, imapsync does not touch any byte of messages -unless told to do so by option --regexmess or --pipemess -Messages on both parts should be exactly identical. +A first thing to keep in mind. +Imapsync does not touch any byte of messages unless told to +do so by special options like --regexmess or --pipemess. +Messages on both parts should be exactly identical because +imapsync does a verbatim sync of all messages. Now I explain the whole picture about dates of messages. -There are several different dates for any message. +There are several different dates attached to any message. First, there is the "Date:" header, which belongs to the message itself. -You can see this "Date:" header line by asking View->Message Source +You can see this "Date:" header line by asking "View->Message Source" in most email client software (Ctrl-u in Thunderbird). -Most of the time, this date is set by the MUA, MUA stands for "Mail User Agent". +Most of the time, this "Date:" header date is set by the MUA, +MUA stands for "Mail User Agent". MUA can be Outlook, Thunderbird, a webmail or Mutt etc. -The Date header is usually the date the message was written +The "Date:" header is usually the date the message was written or sent the first time, at the writer side. This "Date:" header line is never changed by any transfer or copy. If an email reader uses the "Date:" header for displaying the date of a message then no problem should arise. Another date is the internal date. In IMAP, the internal date is handled -by the server and normally it corresponds to the arrival date in the mailbox. +by the imap server and normally it corresponds to the arrival date in the mailbox. This date is not part of the email message itself, it's a metadata kept by the imap server. The IMAP protocol allows the internal date to be set by a email client when a message is delivered by the imap APPEND command. -Imapsync synchronizes internal dates by default, internal dates on host2 -should then be the same as the internal dates on host1. +Imapsync synchronizes these internal dates, by default, so internal dates +on host2 should then be the same as the internal dates on host1. If an email reader uses the internal date for displaying the date of a message then the sync date problem only occurs when the host2 server software ignores the internal date given by imapsync during the APPEND imap @@ -73,14 +78,14 @@ has done its best to keep all the dates synchronized. Bad luck. Solutions: -a) Use a another email client or configure it in order it sorts messages -by sent date, the "Date:" header line. +a) Use another email client or configure it in order that it sorts messages +by sent dates, the "Date:" header line. b) Use a imap server that respects the imap RFC and accepts -the internal date set by imapsync. +the internal date set by imapsync. c) Try to understand why the email client software shows another date - than the "Date:" header line. + than the "Date:" header line. ======================================================================= Q. Is there a way to set any message with diff --git a/FAQ.d/FAQ.Domino.txt b/FAQ.d/FAQ.Domino.txt index 3194145..9ecdc82 100644 --- a/FAQ.d/FAQ.Domino.txt +++ b/FAQ.d/FAQ.Domino.txt @@ -1,11 +1,14 @@ #!/bin/cat -# $Id: FAQ.Domino.txt,v 1.9 2018/03/13 03:22:55 gilles Exp gilles $ +# $Id: FAQ.Domino.txt,v 1.10 2018/05/24 11:34:30 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Domino.txt -============================= - Imapsync tips for Domino. -============================= + +======================================================================= + Imapsync tips for Domino. +======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.Dovecot.txt b/FAQ.d/FAQ.Dovecot.txt index f4d1600..e269229 100644 --- a/FAQ.d/FAQ.Dovecot.txt +++ b/FAQ.d/FAQ.Dovecot.txt @@ -1,14 +1,17 @@ #!/bin/cat -$Id: FAQ.Dovecot.txt,v 1.5 2017/01/06 14:21:06 gilles Exp gilles $ +$Id: FAQ.Dovecot.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $ + +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Dovecot.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync tips for Dovecot. Specific issues and solutions. ======================================================================= -====================================================================== +======================================================================= Q. How to migrate to Dovecot with an admin/MasterUser account? R. Dovecot uses the same syntax as uw-imap @@ -18,4 +21,5 @@ R. Dovecot uses the same syntax as uw-imap To setup a Dovecot MasterUser see http://wiki2.dovecot.org/Authentication/MasterUsers -====================================================================== +======================================================================= +======================================================================= \ No newline at end of file diff --git a/FAQ.d/FAQ.Duplicates.txt b/FAQ.d/FAQ.Duplicates.txt index d257371..5d69b7d 100644 --- a/FAQ.d/FAQ.Duplicates.txt +++ b/FAQ.d/FAQ.Duplicates.txt @@ -1,18 +1,14 @@ #!/bin/cat -$Id: FAQ.Duplicates.txt,v 1.14 2018/05/02 12:21:54 gilles Exp gilles $ +$Id: FAQ.Duplicates.txt,v 1.18 2019/02/14 17:22:10 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt ======================================================================= - Imapsync tips about duplicated messages issues. + Imapsync tips about duplicated messages issues. ======================================================================= -======================================================================= -Q. How can I remove duplicates in a lone account? - -R. Just run imapsync on the same account with option --delete2duplicates, - ie, with host1 == host2 and user1 == user2 - ======================================================================= Q. How can I know if imapsync will generate duplicates on a second run? @@ -37,7 +33,7 @@ Messages skipped : 1555 ======================================================================= Q: Multiple copies, duplicates, when I run imapsync twice ore more. -R0. +R0. First, some explanations to understand the issue. Normally and by default, imapsync doesn't generate duplicates. So if it does generate duplicates it means a problem occurs with message identification. It happens sometimes with IMAP @@ -46,8 +42,39 @@ of the "Received:" header lines in the header part of messages. By default, Imapsync uses "Message-Id" header line and "Received:" header lines to identify messages on both sides. -R1. -A first solution is to use option --useuid. +R1. This solution is R3 simplified. +A quick practical solution is to change the way imapsync +identify messages that works most of the time. But since +you're reading this because you encountered duplicates issue, +let's check this solution in a safe way. + +First use the same commmand with additionnal options: + + imapsync ... --useheader "Message-Id" --dry + +The previous command does nothing real but it will show you +if imapsync handles duplicates in a better way. +The criterium is to search at the end of the sync for a line +like this one: +Messages skipped : 1555 +where 1555 is an example but reflects mostly the number +of all messages already transferred. + +If you end with: +Messages skipped : 0 +don't go on, it means imapsync is still suffering to +identify messages. + +If you end with many messages skipped then it's very +good and now you can safely resync the mailboxe +and get rid of the dupplicates messages on host2 with: + + imapsync ... --useheader "Message-Id" --delete2duplicates + +End of the problem! + +R2. +A second solution is to use option --useuid. With option --useuid, imapsync doesn't use header lines to identify and compare messages in folders. Instead of some headers, --useuid tell imapsync to use @@ -63,7 +90,7 @@ used from the first time but it does generate duplicates after a previous run without --useuid (because it then uses a different method to identify the messages). -A solution? Two solutions. +A solution? Two solutions. The easiest is --delete2 if you are permitted to use it. Option --delete2 removes messages on host2 @@ -74,7 +101,7 @@ So --useuid --delete2 is easy to remove duplicates but not for all contexts. The host2 account must be considered as a strict replication of the host1 account, ie, not active. -A second solution, better if R2 works (see R2 below), is to build +A second solution, better if R3 works (see R3 below), is to build the cache before using --useuid First sync: @@ -87,7 +114,7 @@ Next syncs: imapsync ... --useuid ... -R2. +R3. Best way if you can follow it. Multiple copies of the emails on the destination server. Some IMAP servers (Domino for example) change some headers for each message @@ -161,5 +188,11 @@ R2. With option --useuid imapsync doesn't use headers to identify messages on both sides but it uses their imap uid identifier. In that case duplicates on host1 are also transferred on host2. +======================================================================= +Q. How can I remove duplicates in a lone account? + +R. Just run imapsync on the same account with option --delete2duplicates, + ie, with host1 == host2, user1 == user2, password1 == password2 + ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.Emptying.txt b/FAQ.d/FAQ.Emptying.txt index d5ae282..9ae1e85 100644 --- a/FAQ.d/FAQ.Emptying.txt +++ b/FAQ.d/FAQ.Emptying.txt @@ -1,10 +1,12 @@ #!/bin/cat -$Id: FAQ.Emptying.txt,v 1.5 2017/01/06 14:11:13 gilles Exp gilles $ +$Id: FAQ.Emptying.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/ ======================================================================= - Imapsync tip to empty an account. + Imapsync tip to empty an account. ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.Exchange.txt b/FAQ.d/FAQ.Exchange.txt index f980be4..aa78fe5 100644 --- a/FAQ.d/FAQ.Exchange.txt +++ b/FAQ.d/FAQ.Exchange.txt @@ -1,11 +1,14 @@ -$Id: FAQ.Exchange.txt,v 1.38 2018/03/21 16:26:23 gilles Exp gilles $ +$Id: FAQ.Exchange.txt,v 1.49 2018/12/03 23:09:28 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt -================================================================================= - Imapsync tips for Exchange 20xx and Office365. Specific issues and solutions. -================================================================================= + +======================================================================= + Imapsync tips for Exchange 20xx and Office365. +======================================================================= Questions answered in this FAQ are: @@ -13,14 +16,26 @@ Q. Can I use imapsync to transfer from or to Exchange or Office365 accounts? Q. How to sync from XXX to Exchange 2010/2013/2016 +Q. How to sync from Office365 to XXX? + Q. How to sync from XXX to Office365 Q. For Office365 I have double and triple checked the username and password spelling but I still get a "LOGIN failed". Any clue? Q. I see "NO Maximum size of appendable message has been exceeded" + What can I do with that? Happens with Office365 or Exchange 2016. + +Q. Office365 throttles the sync and says: + "Request is throttled. Suggested Backoff Time: 299961 milliseconds". What can I do with that? +Q. Office365 refuses to create the folder named "Files" with the error + "NO Folder name is reserved". What happens? + +Q. Office365 users complain that a folder named "Files" contains + messages with no sender. + Q. Exchange fails with "User is authenticated but not connected". Q. Exchange fails with "BAD Command received in Invalid state". @@ -41,9 +56,11 @@ Q. Exchange and Office365 have throttle mechanisms to limit any huge usage. Sometimes imapsync transfers are too stressful for servers. How to deal with that? -Q. How to migrate from or to Exchange 2007/2010 with an +Q. How to migrate from or to Exchange 2007/2010/2013 with an admin/authuser account? +Q. How to migrate from or to Exchange 2016 with an admin/authuser account? + Q. How to migrate from or to Office 365 with an admin/authuser account? Q. How to migrate from or to Exchange 2003 with an admin/authuser @@ -115,6 +132,18 @@ In case you don't have it you can still use --regexmess 's,(.{9900}),$1\r\n,g' instead of --maxlinelengthcmd 'reformime -r7' +======================================================================= +Q. How to sync from Office365 to XXX? + +R. On Windows, use: + + imapsync.exe ... --office1 + +On Unix, use: + + imapsync ... --office1 + + ======================================================================= Q. How to sync from XXX to Office365 @@ -136,7 +165,7 @@ On Windows, use: or use (before imapsync release 1.836): imapsync.exe ... ^ --maxsize 45000000 ^ - --maxmessagespersecond 4 ^ + --maxmessagespersecond 2 ^ --regexflag "s/\\Flagged//g" ^ --disarmreadreceipts ^ --regexmess "s,(.{10500}),$1\r\n,g" @@ -150,7 +179,7 @@ or use (before imapsync release 1.836): imapsync ... \ --maxsize 45000000 \ --maxlinelength 10500 \ - --maxmessagespersecond 4 \ + --maxmessagespersecond 2 \ --regexflag 's/\\Flagged//g' \ --disarmreadreceipts \ --maxlinelengthcmd 'reformime -r7' @@ -193,9 +222,13 @@ R2. Miguel Alameda reported understanding and solving this issue ======================================================================= Q. I see "NO Maximum size of appendable message has been exceeded" - What can I do with that? + What can I do with that? Happens with Office365 or Exchange 2016 -R. Office 365 supports send/receive max message sizes of up to 150MB +R0. It happens with Office365 and Exchange 2016. + See R1 for Office365 + See R2 for Exchange 2016 + +R1. Office365 supports send/receive max message sizes of up to 150MB but you need to make changes in your tenant(s) to support it. The following PowerShell command will increase the message sizes that @@ -216,14 +249,112 @@ values on the migration acounts in the source and target tenants. Thanks to Sean McDougall, Ian Thomas & Matt Wilks from Toronto for this FAQ item. +R2. With Exchange 2016 + +Thanks to Torsten Bergemann from Germany: + +In order to have exchange accept big e-mails via imap, several size +limitations have to be removed on several levels of the exchange +software: For the organization (within exchange admin panel), +for each connector (with exchange admin panel), for each Postfach +(using exchange management shell). + +For details and instructions, check +https://technet.microsoft.com/en-us/library/bb124345(v=exchg.160).aspx +https://technet.microsoft.com/en-us/library/hh529949(v=exchg.160).aspx + +Unfortunately, the second link does not give instructions to remove the limit from imap4 service. + +To solve it for Exchange 2013/2016, add the line + + + +to the file +C:\Program Files\Microsoft\Exchange Server\V14 [or V15]\ClientAccess\PopImap\Microsoft.Exchange.Imap4.exe.config + +Please note that this change may be lost each time you install an +cumulative update. After the change, restart the services exchange +transport role, exchange information store and exchange imap4 Backend. + +(Source: https://social.technet.microsoft.com/Forums/de-DE/2c07cf43-fe0e-4d22-b061-676a67c6d58d/exchange-2013-max-mailgre-per-imap?forum=exchange_serverde) + +This change made our Exchange installation accept bigger e-mails +in order to remove the imapsync Exchange error message +"NO Maximum size of appendable message has been exceeded" + + +======================================================================= +Q. Office365 throttles the sync and says: + "Request is throttled. Suggested Backoff Time: 299961 milliseconds". + What can I do with that? + +R. To solve the throttles issues from 0365, there's two solutions: + +R1. Call Microsoft Office365 and ask them to remove the limits on your + mailboxes. That's not a joke, they do it for 90 days usually, + sometimes only after you reach the second technician you call, + the first one been not enough competent to understand what you're + talking about. + +R2. Play with options --maxbytespersecond or --maxmessagespersecond + or --exitwhenover + + imapsync ... --maxbytespersecond 100_000 + + imapsync ... --maxmessagespersecond 2 + + imapsync ... --exitwhenover 1_000_000_000 + +I don't know the upper value that avoid the default throttling from 0365 +and I guess it changes over time. + +======================================================================= +Q. Office365 refuses to create the folder named "Files" with the error + "NO Folder name is reserved". What happens? + +R. The folder Files is a standard folder in Office365. It should not + be synced in IMAP. See the next question. + +======================================================================= +Q. Office365 users complain that a folder named "Files" contains + messages with no sender. + +R0. To fix this, add --exclude Files + + imapsync ... --exclude Files + +R1. This folder "Files" seems to be a standard folder in Exchange Online, + but it is not. The folder contains all attachments in every email + that is in the mailbox, but without any headers. + This causes some confusion for users as these appear in their + search results as duplicate lines but without the sender details + or even the message body. + + This folder seems to be usually hidden so IMAP clients can’t see it, + but for some reason sometimes it becomes visible. + +R2. It looks to be a common problem with Exchange Online. + I’m not sure what causes the folder to appear. + + More info here: +https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_exchon-mso_o365b/exclude-the-exchange-online-system-folder-called/2adbdf84-db4a-4c7f-ac29-738757980a0d +https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_dep365-mso_o365b/no-sender-emails-in-files-folder/534bae8f-a7d7-4f5f-8ed7-5bad0d5fa23f + + (This question/answer is taken quasi verbatim from Perttu Aaltonen) ======================================================================= Q. Exchange fails with "BAD Command received in Invalid state". -R. This message might happens when authenticating without ssl nor tls. - Add --tls1 or --ssl1 if this error message comes from host1. - Add --tls2 or --ssl2 if this error message comes from host2. +R1. This message might happen when authenticating without ssl nor tls, + wich is rare now since imapsync tries to do ssl then tls by default. + Add --tls1 or else --ssl1 if this error message comes from host1. + Add --tls2 or else --ssl2 if this error message comes from host2. + +R2. This message might also happen when authenticating with ssl and tls. + Don't use --tlsX and --sslX on the same side X (1 or 2), since it + asks for a double encryption, an encryption inside an encryption. + It won't work, you can't do a direct double encryption, few + servers allow that. -(Never add --tlsX and --sslX on the same side X (1 or 2), it won't work) ======================================================================= Q. From XXX to Exchange 2013 or Office365, read receipts are all resent again after a sync. Even for old messages. How can I fix that? @@ -271,6 +402,9 @@ R1. "SSL read or write error" happens sometimes, it isn't related to R2. Another solution is to remove --tls or --ssl options for Exchange and accept a clear text sync. +R3. See also the FAQ FAQ.SSL_errors.txt + https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt + ======================================================================= Q. From XXX to Exchange 2010/2013 or Office365 I get this error message sometimes: "BAD Command Argument Error 11". What does it mean? @@ -400,7 +534,6 @@ As it seems you need to use the mailNickname (e.g. alias) defined for user2, not the username itself. http://blogs.technet.com/b/exchange/archive/2004/03/31/105275.aspx - A little note from John Becker. After you pointed out that I had issues with authorization, I realized that as an admin user I don't have all the rights on the mailboxes. @@ -412,6 +545,32 @@ The command that I used for imapsync is: --user2 "windows-domain\admin-user\aaaa" ^ --password2 admin-password ... + + +====================================================================== +Q. How to migrate from or to Exchange 2016 with an admin/authuser account? + +R. This might work: + + imapsync ... --user2 user2 --authuser2 admin2 --password2 adminpassword2 ... + +Note from +https://github.com/imapsync/imapsync/issues/136#issuecomment-397020248 + +The key is to not use the built-in Administrator account which +Microsoft seems to have blocked access to in imap. +I'm doing a sync to Exchange 2016 accounts and this works: + + imapsync ... --ssl2 --user2 account@domain.com ^ + --authuser2 accountwfullaccess@domain.com ^ + --password2 "accountwfullaccesspw" + +The admin account you use needs to have full access to the account(s) +you want to sync which you can setup in the gui or via powershell +and again - don't use administrator@domain.com - as that will never +work because IMAP LOGIN is disabled for that account apparently +under all circumstances. + ====================================================================== Q. How to migrate from or to Office 365 with an admin/authuser account? @@ -421,7 +580,7 @@ Note from Yago Torres Fernandez: imapsync ... --authuser2 user_admin@domain.com --user2 user_to_be_migrated@domain.com ^ --password2 XXXX --ssl2 ^ -but previous in Office365 you must do something like that, using powershell: +but previously in Office365 you must do something like that, using powershell: Add-MailboxPermission -identity user_to_be_migrated@domain.com -user user_admin@domain.com -accessrights fullaccess -inheritancetype all diff --git a/FAQ.d/FAQ.Flags.txt b/FAQ.d/FAQ.Flags.txt index 581486a..e9760ff 100644 --- a/FAQ.d/FAQ.Flags.txt +++ b/FAQ.d/FAQ.Flags.txt @@ -1,10 +1,12 @@ #!/bin/cat -$Id: FAQ.Flags.txt,v 1.17 2017/07/27 15:39:57 gilles Exp gilles $ +$Id: FAQ.Flags.txt,v 1.20 2019/02/14 16:01:41 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt ======================================================================= - Imapsync tips about flags. + Imapsync tips about flags. ======================================================================= Questions answered here are: @@ -270,8 +272,7 @@ Two solutions: Q. Flags are resynced at each run for already synced/copied messages, how can I avoid this feature? -R. It is not possible to avoid this feature by an option for now. - If you really need this then ask for it and I might code it. +R. Use the option --noresyncflags ======================================================================= diff --git a/FAQ.d/FAQ.Folders_Mapping.txt b/FAQ.d/FAQ.Folders_Mapping.txt index 339df52..96fc0c7 100644 --- a/FAQ.d/FAQ.Folders_Mapping.txt +++ b/FAQ.d/FAQ.Folders_Mapping.txt @@ -1,18 +1,24 @@ #!/bin/cat -$Id: FAQ.Folders_Mapping.txt,v 1.15 2017/11/13 00:01:14 gilles Exp gilles $ +$Id: FAQ.Folders_Mapping.txt,v 1.17 2018/11/20 10:26:18 gilles Exp gilles $ + +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt + -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= - Imapsync tips about changing folders names. + Imapsync tips about changing folders names. ======================================================================= Folders names are by default reproduced identical except for the prefix and the separator which are automatically adapted for host2. -Before using --regextrans2 you should consider using --automap -and --f1f2 because they are simpler to understand and use. +This document gives most examples with the powerful but complex +option --regextrans2. Before using --regextrans2 you should +consider using --automap and --f1f2 because they are simpler +to understand and use. --automap : guesses folders mapping, for folders like "Sent", "Junk", "Drafts", "All", "Archive", "Flagged". @@ -20,10 +26,11 @@ and --f1f2 because they are simpler to understand and use. --f1f2 str1=str2 : Force folder str1 to be synced to str2, --f1f2 overrides --automap and --regextrans2. - --subfolder2 str : Move whole host1 folders hierarchy under this - host2 folder str . - It does it by adding two --regextrans2 options before - all others. Add --debug to see what's really going on. + --subfolder2 str : Syncs the whole host1 folders hierarchy under the + host2 folder named str. + (It does it internally by adding two + --regextrans2 options before all others. + Add --debug to see what's really going on.) diff --git a/FAQ.d/FAQ.Folders_Selection.txt b/FAQ.d/FAQ.Folders_Selection.txt index 43dfc00..86e98d9 100644 --- a/FAQ.d/FAQ.Folders_Selection.txt +++ b/FAQ.d/FAQ.Folders_Selection.txt @@ -1,10 +1,13 @@ #!/bin/cat -$Id: FAQ.Folders_Selection.txt,v 1.6 2017/07/20 11:02:11 gilles Exp gilles $ +$Id: FAQ.Folders_Selection.txt,v 1.7 2018/05/24 11:34:30 gilles Exp gilles $ + +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Selection.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= - Imapsync tips to select folders. + Imapsync tips to select folders. ======================================================================= By default, Imapsync syncs all folders, one by one, in alphanumeric order. diff --git a/FAQ.d/FAQ.Folders_Sizes.txt b/FAQ.d/FAQ.Folders_Sizes.txt index 3d96e77..da5ee44 100644 --- a/FAQ.d/FAQ.Folders_Sizes.txt +++ b/FAQ.d/FAQ.Folders_Sizes.txt @@ -1,7 +1,9 @@ #!/bin/cat -$Id: FAQ.Folders_Sizes.txt,v 1.1 2017/09/29 11:10:53 gilles Exp gilles $ +$Id: FAQ.Folders_Sizes.txt,v 1.2 2018/05/24 11:34:30 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Sizes.txt ======================================================================= Imapsync tips to know folders sizes. diff --git a/FAQ.d/FAQ.General.txt b/FAQ.d/FAQ.General.txt index f5a7338..4417c6f 100644 --- a/FAQ.d/FAQ.General.txt +++ b/FAQ.d/FAQ.General.txt @@ -1,19 +1,19 @@ #!/bin/cat -# $Id: FAQ.General.txt,v 1.238 2018/03/07 04:26:04 gilles Exp gilles $ +# $Id: FAQ.General.txt,v 1.240 2018/11/26 11:49:56 gilles Exp gilles $ ======================================================================= General FAQ for imapsync ======================================================================= -This document is also available at -https://imapsync.lamiral.info/#doc +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/FAQ.General.txt Questions answered in this FAQ are: Q. Do I need to create IMAP mailboxes at the destination platform? -Q. Am I forced to publish the IMAP service on internet since the two +Q. Am I forced to publish the IMAP service on the Internet since the two environment are not in the same location or same LAN? Q. What are the most important differences between the Unix shell syntax @@ -120,12 +120,12 @@ R. Yes! with the standard IMAP protocol. So you have to create them first. ======================================================================= -Q. Am I forced to publish the IMAP service on internet since the two +Q. Am I forced to publish the IMAP service on the Internet since the two environment are not in the same location or same LAN? R. The host where you run imapsync has to contact both servers via imap. - You are not obliged to publish the imap service on internet if the host - where you run imapsync can contact both imap servers via imap. + You are not obliged to publish the imap service on the Internet if + the host running imapsync can contact both imap servers via imap. Use their names or their IP addresses. ======================================================================= diff --git a/FAQ.d/FAQ.Gmail.txt b/FAQ.d/FAQ.Gmail.txt index 7b33771..9990d45 100644 --- a/FAQ.d/FAQ.Gmail.txt +++ b/FAQ.d/FAQ.Gmail.txt @@ -1,7 +1,9 @@ #!/bin/cat -$Id: FAQ.Gmail.txt,v 1.37 2018/03/23 19:07:37 gilles Exp gilles $ +$Id: FAQ.Gmail.txt,v 1.50 2018/12/25 11:10:30 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt ======================================================================= Imapsync tips for Gmail accounts. @@ -11,6 +13,8 @@ Questions answered in this FAQ are: Q. Can I use imapsync to transfer from or to Gmail accounts? +Q. Is there special configuration tips to facilitate a Gmail transfer? + Q. How many days does it take to transfer X GB? Q. How to synchronize from Gmail to Gmail? @@ -25,6 +29,8 @@ Q. I can't authenticate with Gmail via IMAP Q. Can not open imap connection on [imap.gmail.com] Unable to connect to imap.gmail.com +Q. Can I transfer "Chat" messages with imapsync? + Q. Can I safely use --useuid for Gmail transfers? Q. Gmail does not really delete messages in folder [Gmail]/All Mail @@ -59,6 +65,16 @@ it has to be allowed in the Gmail configuration part: -> IMAP Access -> Enable IMAP +======================================================================= +Q. Is there special configuration tips to facilitate a Gmail transfer? + +R. Turned off "Conversation views". +With "Conversation views" turned off, Gmail shows the exact message +count for each folder and it matches what imapsync shows. +With "Conversation views" on, a single conversation can have +multiple messages and it doesn't match what imapsync presents. +(Thanks to Karthik.V for this input) + ======================================================================= Q. How many days does it take to transfer X GB? @@ -82,46 +98,56 @@ R. Basically it takes X days to transfer X GB per account. This is either a disconnection with "BYE Session expired, please login again" or a very small rate, less than 1 Kib/s + +https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt ======================================================================= Q. How to synchronize from Gmail to Gmail? R. Use the following example: -./imapsync \ +imapsync \ --user1 account1@gmail.com \ --password1 gmailsecret1 \ --user2 account2@gmail.com \ --password2 gmailsecret2 \ - --gmail12 + --gmail1 --gmail2 -Or, replacing what does --gmail12: +Or, replacing what does --gmail1 --gmail2 (values are from imapsync 1.882): -./imapsync \ +imapsync \ --host1 imap.gmail.com \ - --ssl1 \ --user1 account1@gmail.com \ --password1 gmailsecret1 \ --host2 imap.gmail.com \ - --ssl2 \ --user2 account2@gmail.com \ --password2 gmailsecret2 \ - --maxbytespersecond 10000 \ + --maxbytespersecond 20_000 \ + --maxbytesafter 1_000_000_000 \ + --maxsleep 2 \ --automap \ --exclude "\[Gmail\]$" +With --gmail1 --gmail2 you can override the parameters +that are activated by them, for example: + +imapsync ... --gmail1 --gmail2 --maxbytespersecond 50_000 Explanations: ---ssl1 --ssl2 are mandatory since Gmail only supports -imap ssl connections. ---maxbytespersecond 10000 ( 10 kBytes/s ) option is here to avoid +--maxbytespersecond 20_000 ( 20 kBytes/s ) option is here to avoid locking or errors when imap transfers exceed Gmail maximum limit. See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518 ---maxbytespersecond 10000 is not mandatory in the sense Gmail may +--maxbytespersecond 20_000 is not mandatory in the sense Gmail may allow you to use an upper value than 1 GBytes per 24h without -disconnections. +disconnections. If you count well, it should be 10_000, not 20_000. +--maxbytesafter 1_000_000_000 option is there because +option --maxbytespersecond 20000 is unnecessarily slow if the Gmail +account size is less than 1 GB. + +--maxsleep 2 is to avoid sleeping more than 2 seconds when +the --maxbytespersecond value implies a long sleep between 2 copies. --automap is not mandatory but it's a feature to automatically map folder names based on the Gmail user configuration itself, par account. @@ -146,9 +172,12 @@ http://stackoverflow.com/questions/2185391/localized-gmail-imap-folders/2185548# --exclude "\[Gmail\]$" is just there to avoid a warning error when selecting this not used folder. -======================================================================= +Be aware that --gmail1 --gmail2 is a special case, it's not what +--gmail1 does plus what --gmail2 does when they are not invoked +together. +https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt ======================================================================= Q. How to synchronize from XXX to Gmail? @@ -159,9 +188,9 @@ imapsync --host1 mail.oldhost.com \ --password1 password \ --user2 my_email@gmail.com \ --password2 password \ - --gmail2 + --gmail2 --maxsize 35_651_584 -Or, replacing what does --gmail2: +Or, replacing what does --gmail2 (values are from imapsync 1.882): imapsync --host1 mail.oldhost.com \ --user1 my_email@oldhost.com \ @@ -169,28 +198,40 @@ imapsync --host1 mail.oldhost.com \ --host2 imap.gmail.com \ --user2 my_email@gmail.com \ --password2 password \ - --ssl2 \ - --maxbytespersecond 10000 \ - --maxsize 25000000 \ + --maxbytespersecond 20_000 \ + --maxbytesafter 1_000_000_000 \ + --maxsleep 2 \ + --maxsize 35_651_584 \ --automap \ --expunge1 \ --addheader \ --exclude "\[Gmail\]$" \ --regextrans2 "s/[ ]+/_/g" \ - --regextrans2 "s/['\^\"\\\\]/_/g" + --regextrans2 "s/['\^\"\\\\]/_/g" + +With --gmail2 you can override the parameters +that are activated by them, for example: + +imapsync ... --gmail2 --maxbytespersecond 50_000 --noautomap --maxsleep 0 Explanations: ---ssl2 is mandatory since Gmail only supports imap ssl connections. - ---maxbytespersecond 10000 ( 10 kBytes/s ) option is here to avoid +--maxbytespersecond 20_000 ( 20 kBytes/s ) option is here to avoid locking or errors when imap transfers exceed Gmail maximum limit. See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518 ---maxbytespersecond 10000 is not mandatory in the sense Gmail may +--maxbytespersecond 20_000 is not mandatory in the sense Gmail may allow you to use an upper value than 1 GBytes per 24h without -disconnections. +disconnections. If you count well, it should be 10_000, not 20_000. ---maxsize 25000000 is mandatory since Gmail limits messages size +--maxbytesafter 1_000_000_000 option is there because +option --maxbytespersecond 20000 is unnecessarily slow if the Gmail +account size is less than 1 GB. Counting well, it should +be 500_000_000, not 1_000_000_000, but 1GB is usually ok. + +--maxsleep 2 is to avoid sleeping more than 2 seconds when +the --maxbytespersecond value implies a long sleep between 2 copies. + +--maxsize 25_000_000 is mandatory since Gmail limits messages size up to 25 MB. This value increases over time, it was 10 MB some years ago so you can try higher values. The Gmail page about this limit is https://support.google.com/mail/answer/6584 @@ -239,29 +280,29 @@ You can select folders exported to imap within the gmail preferences. Select or unselect some "System labels", depending on your needs. +https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt ======================================================================= Q. How to synchronize from Gmail to XXX? R. Use this example: -./imapsync \ +imapsync \ --user1 gilles.lamiral@gmail.com \ --password1 gmailsecret \ --host2 localhost \ --user2 tata \ --gmail1 -Or, replacing what does --gmail1: +Or, replacing what does --gmail1 (values are from imapsync 1.882): -./imapsync \ +imapsync \ --host1 imap.gmail.com \ --user1 gilles.lamiral@gmail.com \ --password1 gmailsecret \ --host2 localhost \ --user2 tata \ --password2 tatasecret \ - --ssl1 \ - --maxbytespersecond 20000 \ + --maxbytespersecond 20_000 \ --useheader="X-Gmail-Received" \ --useheader "Message-Id" \ --automap \ @@ -269,15 +310,17 @@ Or, replacing what does --gmail1: --skipcrossduplicates \ --folderlast "[Gmail]/All Mail" +With --gmail1 you can override the parameters +that are activated by them, for example: + +imapsync ... --gmail1 --maxbytespersecond 50_000 --noskipcrossduplicates --noautomap Explanations: ---ssl1 is mandatory since Gmail only supports imap ssl connections. - ---maxbytespersecond 20000 ( 20 kBytes/s ) option is here to avoid +--maxbytespersecond 20_000 ( 20 kBytes/s ) option is here to avoid locking or errors when imap transfers exceed Gmail maximum limit. See http://support.google.com/a/bin/answer.py?hl=en&answer=1071518 ---maxbytespersecond 20000 is not mandatory in the sense Gmail may +--maxbytespersecond 20_000 is not mandatory in the sense Gmail may allow you to use an upper value than 2 GBytes per 24h without disconnections. @@ -302,11 +345,11 @@ for example you may unselect all "System labels". disk memory. Within imap protocol, Gmail presents Gmail labels as folders, so a message labeled "Work" "ProjectX" "Urgent" ends up in three different imap folders "Work" "ProjectX" and "Urgent" -after an imap sync. --skipcrossduplicates prevent this behavior. +after an imap sync. Option --skipcrossduplicates prevent this behavior. An issue with --skipcrossduplicates is that the first label synced -by imapsync goes to its corresponding folder and other labels are -ignored. This way, at least you can choose what labels have the +by imapsync goes to its corresponding folder but other labels are +then ignored. You can choose what labels have the priority by using the --folderfirst option. For example --folderfirst "Work" will sync messages labeled "Work" before messages labeled "CanWait" or "Urgent". By default imapsync @@ -320,15 +363,15 @@ label CanWait and only it. the messages that are not labeled at all. - +https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt ======================================================================= Q. I can't authenticate with Gmail via IMAP and Gmail says "Please log in via your web browser" -R0. My current settings on two Gmail account are made -with 0) 1) 2) +R0. My current settings on two Gmail accounts are made +with three steps 0) 1) 2) -0) Login via a browser to the Gmail account concerned. +0) Login via a web browser to the Gmail account concerned. 1) The 2-step verification is OFF https://myaccount.google.com/security?hl=en&pli=1&nlr=1#signin @@ -343,9 +386,10 @@ https://web.archive.org/web/20150906230041/http://security.stackexchange.com/que R2. I had the same issue one time (mars 2015) logging to Gmail with imapsync. The Gmail imap message error said "Please log in via your web browser" -so I logged for this account via a web browser, +so I logged for this account via a web browser. Then, it asked me to receive a code via a mobile, I said yes, -I entered the code and everything went ok. +I entered the code and everything went ok on next syncs +with imapsync. ======================================================================= @@ -380,6 +424,15 @@ Or go to http://ping.eu/nslookup/ to get the resolution. Thanks to Chris Nolan to report, understand and fix this issue! +======================================================================= +Q. Can I transfer "Chat" messages with imapsync? + +R. No. + Even if you think you can because the Chat folder is seen in IMAP, + with Chat messages in them, they won't be well transferred. + Chat messages are not well presented and not well transferred + using the IMAP protocol. + ======================================================================= Q. Can I safely use --useuid for Gmail transfers? @@ -395,16 +448,32 @@ ends up with no duplicates on Gmail but a waste of bandwith and time, which is the opposite goal of --usecache implied by --useuid. -======================================================================= -Q. Gmail does not really delete messages in folder [Gmail]/All Mail - What happens? What can I do? +======================================================================= +Q. Gmail does not really delete messages in folder [Gmail]/All Mail + What happens? What can I do? R. It's true and explained in Gmail documentation at -https://support.google.com/mail/answer/78755?hl=en +https://support.google.com/mail/answer/7401?hl=en To really remove messages in folder "[Gmail]/All Mail", they have to be moved to the "Trash" folder and be deleted from "Trash". +Quoting the link above (25 december 2018, happy christmas by the way!) +... +Empty your Trash +If you don't want a message to stay in your Trash for 30 days, +you can permanently delete it. +* On your computer, go to Gmail. +* On the left side of the page, scroll down, + then click More and then Trash. +* Check the box next to messages you want to permanently delete, + then click Delete forever. + +* Note: The Delete forever option is only available in the Trash. + You wont be able to delete items forever from the Search result view. + To delete all messages in your trash, click Empty Trash now. + + ======================================================================= Q. Can I use the Extension of the SEARCH command: X-GM-RAW described at https://support.google.com/mail/answer/7190?hl=en @@ -441,18 +510,21 @@ So you have to follow the Google recommendation and generate an application-specific password or normal authentication or use XOAUTH or XOAUTH2. + ======================================================================= Q. How to use XOAUTH2 to globally authenticate gmail users? R. Yes, but really tested on Unix systems, not sure on Windows. See: http://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt + ======================================================================= Q. How to use XOAUTH to globally authenticate gmail users? R0. XOAUTH is considered obsolete and superseded by XOAUTH2 See http://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt + ======================================================================= Q. How to use a Gmail account to backup several different imap accounts? @@ -465,6 +537,7 @@ is no supplementary label created on the multi-archive Gmail destination account. No labels all over the place and all original xxx sub-folders show up nested within xxx/xxx. + ======================================================================= Q. How to migrate email from gmail to google apps? diff --git a/FAQ.d/FAQ.ISP.txt b/FAQ.d/FAQ.ISP.txt index 8ecd7c4..501d820 100644 --- a/FAQ.d/FAQ.ISP.txt +++ b/FAQ.d/FAQ.ISP.txt @@ -1,7 +1,9 @@ #!/bin/cat -$Id: FAQ.ISP.txt,v 1.8 2017/06/21 18:09:00 gilles Exp gilles $ +$Id: FAQ.ISP.txt,v 1.9 2018/05/24 11:34:30 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.ISP.txt ======================================================================= Imapsync tips for ISP. Specific issues and solutions. diff --git a/FAQ.d/FAQ.Massive.txt b/FAQ.d/FAQ.Massive.txt index c3a5276..40a2d2b 100644 --- a/FAQ.d/FAQ.Massive.txt +++ b/FAQ.d/FAQ.Massive.txt @@ -1,7 +1,10 @@ #!/bin/cat -$Id: FAQ.Massive.txt,v 1.17 2017/12/20 03:38:07 gilles Exp gilles $ +$Id: FAQ.Massive.txt,v 1.24 2018/12/30 21:28:03 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Massive.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync tips for massive/bulk migrations. @@ -9,6 +12,8 @@ This documentation is also at http://imapsync.lamiral.info/#doc Questions answered here are: +Q. How long will take the whole migration? + Q. I need to migrate hundred accounts, how can I do? Q. I have to migrate 500k users using 400 TB of disk space. @@ -16,18 +21,30 @@ Q. I have to migrate 500k users using 400 TB of disk space. Q. How to determine where is the bottleneck in an imapsync process? -Q. Can I run more instances of imapsync in parallel on a Windows host? +Q. Can I run several instances of imapsync in parallel on a Windows host? Q. I run multiple imapsync applications at the same time then get a warning "imapsync.pid already exists, overwriting it". Is this a potential problem when trying to sync multiple IMAP account in parallel? +======================================================================= +Q. How long will take the whole migration? +R. Consider the mean imapsync transfer rate to be around 340 Kbytes/s, + ie, 2.8 Mbps, no matter the local link bandwidth. + It's a mean, measured upon various different syncs, coming from + the online interface /X. + + With 1 TB to transfer and one sync at a time it will end in 35 days. + At 10 transfers at a time, 1 TB will take 3.5 days. + Double the time because best scenario never happens. + Triple because, well, real world is like that. + ======================================================================= Q. I need to migrate hundred accounts, how can I do? -R. If you have many mailboxes to migrate think about a little +R. If you have many mailboxes to migrate, think about a little script program. Write a file called file.txt (for example) containing hosts, users and passwords on both sides. The separator used in this example is ";" @@ -118,27 +135,39 @@ How many syncs can we run in parallel? here comes measurements. Since adding this way is not so easy, just look at the overall network rate of the imapsync host. - On Linux, nload is good candidate to measure this overall - network rate, every 6 seconds, on eth0 interface, values in Kbytes: + On Linux and FreeBSD, nload is a good candidate to measure this overall + network rate, every 6 seconds (-t 6000), on eth0 or em0 interface, + values in Kbytes (-u K): - nload -t 6000 eth0 -u K + nload -t 6000 eth0 -u K # Linux + nload -t 6000 em0 -u K # FreeBSD - Another good network tool is dstat: + During nload, press the h to see the display commands available, + every single feature is useful! Press h again and try each one. + + On Linux only, another very good network tool is dstat: - dstat -n -N eth0 6 + dstat -n -N eth0 6 # Linux only (in 2018) A excellent tool for this purpose is iftop, the following command will monitor imap and imaps connexions on interfce eth0, only them, and sum them up: - iftop -i eth0 -f 'port imap or port imaps' -B + iftop -i eth0 -f 'port imap or port imaps' -B # Linux + iftop -i em0 -f 'port imap or port imaps' -B # FreeBSD - On Windows, get the overall network rate with the classical - task manager (Ctrl-Alt-Sup), there is a network tab in it. - Don't hesitate to send me free good tools to measure the - overall transfer rate (the best would be one to sum up only - imap traffic but that's not mandatory at all). + + On Windows 8.1 Windows 10 Windows 2012 R2 Windows 2016, + get the overall network rate with the classical + task manager (Ctrl-Shift-Esc), there is a Performance tab + in it where resides a Network grap monitor. + + On Windows 7, get the overall network rate with the classical + task manager (Ctrl-Shift-Esc), there is a Network tab in it. + + I'm looking for a free and simple tool on Windows that could + sum up only the imap traffic. 2) Launch new parallel runs, one by one, as long as the total transfer rate increase. @@ -190,7 +219,7 @@ Possible bottlenecks: Usually available bandwidth is NOT a bottleneck. Meanwhile, it can be a bottleneck on small Internet connexions. Imapsync downloads messages from host1 and upload messages to host2, - consider this in case the connexion are asymetric. + consider this in case the connexion are asymmetric. - I/O on disks. I/O are a classical bottleneck, almost always forgotten. @@ -225,7 +254,7 @@ Other possible bottlenecks: ======================================================================= -Q. Can I run more instances of imapsync in parallel on a Windows host? +Q. Can I run several instances of imapsync in parallel on a Windows host? R. Yes! diff --git a/FAQ.d/FAQ.Memory.txt b/FAQ.d/FAQ.Memory.txt index f6ebf6d..0bfc464 100644 --- a/FAQ.d/FAQ.Memory.txt +++ b/FAQ.d/FAQ.Memory.txt @@ -1,7 +1,10 @@ #!/bin/cat -$Id: FAQ.Memory.txt,v 1.4 2017/01/06 14:11:13 gilles Exp gilles $ +$Id: FAQ.Memory.txt,v 1.8 2018/08/31 13:29:53 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Memory.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync tips about memory issues. @@ -12,7 +15,27 @@ This documentation is also at http://imapsync.lamiral.info/#doc Q. I got an "Out of memory" error message then imapsync crashes. How to fix that? -R0. Add option --debugmemory to debug memory issues. +R0. Add option --debugmemory to debug memory issues. + On Windows this option uses a DOS command similar to: + + tasklist /NH /FO CSV | findstr imapsync + + On Unix it uses: + + ps -o vsz -p PID_of_imapsync # value is in KB + + Similar to: + + ps -ax -o vsz,comm | grep imapsync + + Use those commands in a DOS window or a Unix shell in order + to monitor a imapsync run you know it will crash. + The goal is to detect the memory limit, when and how it happens. + + * Is the "final" amount near the total RAM available or + just a small part of it? + * Does the rise of memory consumption happens suddenly + or along the sync? R1. It's probably a very big message on the host1 account. Before imapsync release 1.688 memory usage is at least 5 times @@ -20,8 +43,23 @@ R1. It's probably a very big message on the host1 account. So a 700 MB message will crunch 3.5 GB of memory. Starting with 1.688 it will crunch 1.4 GB of memory. +R2. To check if the problem is a big message crunching to much + memory, add --maxsize 50_000_000, this will limit the message + size treated to 50 MB. If the sync succeed while limiting + messages sizes it's probably is a big message crunching to much + memory issue. Possible solutions: + * Try on another host + * Buy memory + * Build imapsync.exe binary 64bits -R2. Usually "Out of memory" errors are related to old days, +R3. If R2 shows that the memory issue is not a big message crunching + too much memory, then it can be imapsync using too much memory + to identify messages (a very big folder served by a + very talkative imap server). A possible solution is to use + the option --useuid. If it still fails, I have no clue. + If you have time, drop me a note. + +R4. Usually "Out of memory" errors are related to old days, to old Mail::IMAPClient module releases, before 3.26. At that time memory usage was around 17 times the biggest message. Look at imapsync output first lines to get the Mail::IMAPClient diff --git a/FAQ.d/FAQ.Messages_Selection.txt b/FAQ.d/FAQ.Messages_Selection.txt index d900a53..9b59653 100644 --- a/FAQ.d/FAQ.Messages_Selection.txt +++ b/FAQ.d/FAQ.Messages_Selection.txt @@ -1,7 +1,10 @@ #!/bin/cat -$Id: FAQ.Messages_Selection.txt,v 1.13 2018/03/07 04:12:37 gilles Exp gilles $ +$Id: FAQ.Messages_Selection.txt,v 1.14 2018/05/24 11:34:30 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync tips to select messages. diff --git a/FAQ.d/FAQ.Migration_Plan.txt b/FAQ.d/FAQ.Migration_Plan.txt index aa4a176..aa82985 100644 --- a/FAQ.d/FAQ.Migration_Plan.txt +++ b/FAQ.d/FAQ.Migration_Plan.txt @@ -1,49 +1,140 @@ #!/bin/cat -$Id: FAQ.Migration_Plan.txt,v 1.2 2018/03/07 04:28:00 gilles Exp gilles $ +$Id: FAQ.Migration_Plan.txt,v 1.5 2019/01/11 13:13:55 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Migration_Plan.txt -This documentation is also at http://imapsync.lamiral.info/#doc ===================================================================== Imapsync. Suggestions for a good, low impact on users, well executed email migration plan. ===================================================================== -A good migration plan: +There is two main different scenarios depending on the response to the +following question: - * Create the new mailboxes on the destination server. - * Decrease the TTL of the MX, as well as the imap hostname resolution, - to 5 minutes (or even less). See FAQ.TTL.txt to understand why. - * Presync all the mailboxes from the old to the new server, using different name - that the ones used by the imap software clients (use their IP for example). - Presyncs can usefully be done with --delete2 but never use --delete2 once users - started playing with their new account on host2. +Will the imap software tools used by the users use the exact same +credentials triplet for both imap servers, the old server host1 and +the new server host2? - * Decide a migration day/hour - * repeat the presyncs (--delete2) daily until the migration hour. - This repeated process will show how long should take the last sync. +The credentials triplet is hostname/username/password. + +If the answer is yes, ie, clients email tools use the exact same +triplet credentials, then it is possible to perform a migration +without changing anything on the users side. This may be a very time +saving option. But it's a rare condition so I'll describe this +scenario later in this document. + + +===================================================================== +Classical scenario, credentials triplets are different on both sides +===================================================================== + + * Decrease the TTL of the MX, to 5 minutes (or even less). See + FAQ.TTL.txt to understand why it's an advantage. If you can't + decrease the TTL, the migration will span a little more but that's + ok, the situation is not that bad. + + * Create the new mailboxes on the destination server host2. If the + users are already playing with the new mailboxes on host2, don't + follow this scenario. + + * Presync all the mailboxes from the old server host1 to the new + server host2. Presyncs can usefully be done with --delete2 in + order to get an exact sync. But never use the option --delete2 once + users have started to play with their new account on host2, their + play will be lost on the next presync, or when the MX is changed, + since INBOX will start to receive new messages that are not on + host1. + + * Decide a migration day/hour. + + * Repeat the presyncs (with the --delete2 options) daily until the + migration hour. This repeated process will show how long should + take the last sync. * At the migration hour, cut access to the users to the old server - (by changing the imap hostname to a non-imap server for example). - * Change also the MX (for the new messages arriving in the new imap server). - * Wait 5 minutes. - * Do a last sync exactly like the presyncs, not using the imap hostname. - * Shutdown the old imap server. + host1, if you can. Or tell them to not use it anymore. - Now a fork in the process, depending on the response to the following question: - Will the imap software clients use the exact same - triplet credentials as the old server? hostname/username/password + * Do a last presync exactly like previous ones. + + * Change the MX, the new messages should start to arrive in the new + imap server host2. + + * Wait the TTL value, aka 5 minutes. Now, new messages should + not arrive to the old server host1. + + * Tell the users that the old imap server host1 is down and no + longer available. + + * Do a postsync. A postsync is a sync with the following options + --maxage 1 --delete1 --folder INBOX + + This postsync will move the last new messages arrived on host1 to + host2 during the TTL interval and delete them on host1. Do not use + the option --delete2 in a postsync. + + * Give access to new accounts to the users with their new credential + triplet hostname/username/password. If the way to contact users is + email then you should give this long before shutting down the old + server. -If the answer is yes (clients use the exact same triplet credentials): - * Change the user imap hostname resolution from the old to the new imap server * Migration done. -If the answer is no (clients use a different triplet credentials): - * Tell the users the old imap server is down and no longer available. - * Give access to new accounts to the users with their new credential triplet - hostname/username/password (the most similar the best for them). - If the way to contact users is email then you should give this long before - shutting down the old server. - * Migration Done + * In case there are still messages arriving at the old imap server + host1, you can perform more postsyncs, ie, syncs every day + with the options: + --maxage 1 --delete1 --folder INBOX. + + +===================================================================== +Lucky scenario, credentials triplets are the same on both sides +===================================================================== + * Decrease the TTL of the MX, as well as the imap hostname resolution, + to 5 minutes (or even less). See FAQ.TTL.txt to understand why. + + * Create the new mailboxes on the destination server host2. + + * Presync all the mailboxes from the old host1 to the new server host2, + using different names that the ones used by the imap software + clients (use their IP for example). + Presyncs have to be done with --delete2 but never use --delete2 + once users have started playing with their new account on host2. + + * Decide a migration day/hour. + + * repeat the presyncs (with the --delete2 options) daily until the + migration hour. This repeated process will show how long should + take the last sync. + + * At the migration hour, cut access to the users to the old server. + You can do this by changing the imap host1 hostname to a non-imap + server for example, or by changing their password on host1. + + * Do a last sync exactly like the presyncs, not using the imap hostname. + + * Change also the MX resolution, the new messages should start + to arrive in the new imap server very soon. + + * Wait the TTL value, aka 5 minutes. Now, new messages should + not arrive to the old server host1. + + * Do a postsync. A postsync is a sync with the following options + --maxage 1 --delete1 --folder INBOX + + This postsync will move the last new messages arrived on host1 to + host2 during the TTL interval and delete them on host1. Do not use + the option --delete2 in a postsync. + + * Shutdown the old imap server. + + * Change the user imap hostname resolution from the old IP of host1 + to the IP of the new imap server host2. + + * Migration done. + + ======================================================================= + ======================================================================= - \ No newline at end of file diff --git a/FAQ.d/FAQ.OnlineUI.txt b/FAQ.d/FAQ.OnlineUI.txt index 1a5fad9..44125c5 100644 --- a/FAQ.d/FAQ.OnlineUI.txt +++ b/FAQ.d/FAQ.OnlineUI.txt @@ -1,10 +1,13 @@ #!/bin/cat -$Id: FAQ.OnlineUI.txt,v 1.5 2018/05/02 12:22:13 gilles Exp gilles $ +$Id: FAQ.OnlineUI.txt,v 1.19 2019/01/28 10:59:16 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.OnlineUI.txt ===================================================================== - Imapsync tips about the online visual user interface + Imapsync tips about the online visual user interfaces + https://i005.lamiral.info/X/ https://imapsync.lamiral.info/X/ ===================================================================== @@ -27,28 +30,38 @@ R0. Well, I don't know if asking the provider whether his online R1. Some figures -Date of this report: 8 March 2018. -The online imapsync service /X started 9 January 2017 (422 days of service). -In average, /X has 20 users per day lunching 6 different migrations, from -one launch to many many (hundreds). -The total volume /X transferred is 17 TiB in 48000 email imap migrations. +Date of this report: 26 December 2018. + +The online imapsync service /X started 9 January 2017 +(716 days of service). + +In average, /X has 20 users per day lunching in mean 6 +different migrations, from one launch to many (hundreds). + +The total volume /X transferred is around 45 TiB in more +than 111 thousands email imap migrations, +157 millions email messages. R2. Pros & Cons The online imapsync service /X runs on https only, with a -letsencrypt certificate, a certificate overall rated "A" at -https://www.ssllabs.com/ssltest/analyze.html?d=imapsync.lamiral.info +letsencrypt certificate, a certificate overall rated "A+" at +https://www.ssllabs.com/ssltest/analyze.html?d=i005.lamiral.info Because of the https usage, what the users enter in their browser, the imap logins and passwords, can't be eavesdropped on the network. -Imapsync itself cares about encryption for the imap sessions, -if possible: It tries SSL first on port 993, then TLS if the -servers announces TLS, then no encryption. What is done with -an imap server is independent of what is done with the other. +Imapsync itself takes care about encryption for the imap sessions, +if possible: It tries SSL first on port 993, then TLS on port 143 +if the servers announces TLS, then no encryption at all. +What is done with the source imap server host1 is independent +of what is done with the destination imap server host2. -At the date of 8 March 2018, there is no security problem detected -or reported to me (Gilles LAMIRAL) so far. +At the date of 26 December 2018, there is no security problem +detected or reported to me (Gilles LAMIRAL), so far. +Feel free to attack the service and feel free to report any +hole encountered. Have in mind I can watch what you try +from the server side. As the owner of the service, it could have been 48 000 pairs of credentials collected and nearly 17 terabytes of email messages. @@ -60,49 +73,62 @@ because too many imap servers are crappy configured regarding certificates. This default behavior is chosen like this because users of /X -wants their emails transferred, instead of not trasferred because -of an incompetent imap server sysadmin. -Anyway, this part, checking imap ssl/tls certificates, could be -improved from my side by including well known certificates -directly in imapsync. +want their emails transferred, instead of being not transferred +because of an incompetent imap server sysadmin. + +I admint that this part, checking imap ssl/tls certificates, +could be improved from my side by including well known +certificates directly in imapsync. If the imap servers don't honor ssl nor tls, then logins, passwords and everything will go clear text during the imap transfers. That's not good at all but what "comforts" me is that if the -imap servers do only clear text transfer, then it is also true -for all imap sessions the owner of the accounts encounter, +imap servers do only clear text transfers, then it's also true +for all imap sessions the accounts' owner encounters, imapsync is just one of them. -Last point, who could be sure no cracker cracked the online host and -currently sniffs the credentials? No one, I'm not sure myself, even -if I do take care of that possibility. +Last point, who could be sure that no cracker cracked the online +hosts and that he isn't currently sniffing the credentials? + +No one, I'm not sure myself, even if I do take care of that +possibility. So changing the imap accounts passwords after +a sync is a safe and recommended practice! ===================================================================== Q. Will I have any issues with browser timing out? What happens if the browser connection is closed for whatever reason? -R. It should be ok +R. It stops the imapsync process, ie, the sync is ended right away. + +Further comments on this behavior. When using the /X interface there are three connections. -One connection is the Browser-WebServer connection, the -two others are the WebServer-ImapServers connections (imapsync stuff). +One connection is the Browser/WebServer connection, +the two others connections are the WebServer/ImapServers +connections (imapsync stuff). -If the Browser-WebServer connection is timeout (but it shouldn't -because of the log refresh), the imapsync sync might continue -anyway. To see if it continues or not, just do a sync again and the -interface will tell you that a sync is already going on, if the -"Sync!" button is gray/inactive then just reload the page (F5 or -similar), and reenter the credentials. +If the Browser/WebServer connection is timeout or ended, +the imapsync sync is also ended immediately by the remote +Apache https server. Technically, Apache sends a TERM signal, +to the imapsync process then wait some seconds before +sending a KILL signal if it is still alive. -By the way, on the /X you can try to do several parallel runs on the same -mailbox even if there is no timeout, open a new tab/windows with /X -and start a same sync, it's safe, the /X will say, if any, that there -is already a current sync. +You can relaunch a sync again with "Sync!" button, at any time. +If the "Sync!" button is gray/inactive then just reload +the page (F5 or similar), and reenter the credentials. +If the interface tells you that a sync is already going on, +it may be that a sync is running from another browser or place. You can stop this sync with the "Abort!" button from any /X -tab/window, even from another browser or place. To doing this with -success, you have to give the same account parameters, same -credentials, or imapsync will ignore the demand. +tab/window, even from another browser or place. To be able +to abort with success, you have to give the same account +parameters, same credentials, or imapsync will ignore the demand. + +In other words, you can try safely to launch several parallel +runs between the same mailboxes. Open a new tab/windows with /X, +and start a same sync, it's safe, the /X will say, if any, that +there is already a current sync running. + ===================================================================== ===================================================================== diff --git a/FAQ.d/FAQ.Oracle-UCS.txt b/FAQ.d/FAQ.Oracle-UCS.txt index b855ada..af621d9 100644 --- a/FAQ.d/FAQ.Oracle-UCS.txt +++ b/FAQ.d/FAQ.Oracle-UCS.txt @@ -1,7 +1,10 @@ #!/bin/cat -$Id: FAQ.Oracle-UCS.txt,v 1.4 2017/01/06 14:11:13 gilles Exp gilles $ +$Id: FAQ.Oracle-UCS.txt,v 1.5 2018/05/24 11:34:30 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Oracle-UCS.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync tips for Oracle-UCS. Specific issues. diff --git a/FAQ.d/FAQ.POP3.txt b/FAQ.d/FAQ.POP3.txt index 2a02793..d7af159 100644 --- a/FAQ.d/FAQ.POP3.txt +++ b/FAQ.d/FAQ.POP3.txt @@ -1,10 +1,13 @@ #!/bin/cat -$Id: FAQ.POP3.txt,v 1.2 2018/02/02 20:32:19 gilles Exp gilles $ +$Id: FAQ.POP3.txt,v 1.3 2018/05/24 11:34:30 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.POP3.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= - Imapsync tips about POP3. + Imapsync tips about POP3. ======================================================================= Questions answered here are: @@ -55,6 +58,5 @@ See also Rick Sanders discussion about this issue: https://www.emailquestions.com/threads/how-to-prevent-duplicate-emails-after-server-migration-for-pop3-users-with-leave-mail-on-server.8109/ - ======================================================================= -======================================================================= \ No newline at end of file +======================================================================= diff --git a/FAQ.d/FAQ.Passwords_on_Unix.txt b/FAQ.d/FAQ.Passwords_on_Unix.txt index a67a161..28396ad 100644 --- a/FAQ.d/FAQ.Passwords_on_Unix.txt +++ b/FAQ.d/FAQ.Passwords_on_Unix.txt @@ -1,7 +1,10 @@ #!/bin/cat -$Id: FAQ.Passwords_on_Unix.txt,v 1.1 2017/01/06 14:11:13 gilles Exp gilles $ +$Id: FAQ.Passwords_on_Unix.txt,v 1.3 2019/01/10 13:27:28 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync issues with passwords on Unix. @@ -11,8 +14,14 @@ This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Q. On Unix, some passwords contain some *(),;& characters. Login fails. -R. Use double-quotes within single-quotes in order to enclose the - password within double-quotes in the imap LOGIN command: +R1. Enclose the password within single-quotes in the imapsync + command line: + + imapsync ... --password1 'passw*(),;&rd' + +R2. If R1 fails, with very old imapsync or old Mail::IMAPClient Perl module, + try also using double-quotes within single-quotes. It will enclose + the password within double-quotes in the imap LOGIN command: imapsync ... --password1 '"passw*(),;&rd"' diff --git a/FAQ.d/FAQ.Passwords_on_Windows.txt b/FAQ.d/FAQ.Passwords_on_Windows.txt index e536457..d940d75 100644 --- a/FAQ.d/FAQ.Passwords_on_Windows.txt +++ b/FAQ.d/FAQ.Passwords_on_Windows.txt @@ -1,12 +1,16 @@ #!/bin/cat -$Id: FAQ.Passwords_on_Windows.txt,v 1.3 2017/01/06 14:11:13 gilles Exp gilles $ +$Id: FAQ.Passwords_on_Windows.txt,v 1.5 2018/06/11 22:35:10 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync issues with passwords on Windows. ======================================================================= +Related documentation: http://www.robvanderwoude.com/escapechars.php http://stackoverflow.com/questions/3288552/how-can-i-escape-an-exclamation-mark-in-cmd-scripts @@ -15,7 +19,17 @@ In case you're brave and relentless, understand and try this: http://www.dostips.com/forum/viewtopic.php?f=3&t=1733 ======================================================================= -Q. On Windows, some passwords contain $ characters. Login fails. +Q. On Windows, some passwords contain $ or other special characters + like one more more of the nine characters $%&<>|^"! + Login fails. + + +R0. Change the password so that only normal characters remain. + It's often a quick and good solution. If you care about + strong password policy, just make it longer. + If you can't avoid special characters use no problematic + characters such as underscore _ or minus - or comma , or dot. + If you can't change passwords then read on and good luck! R1. Enclose passwords between "" @@ -31,6 +45,9 @@ you have to enter imapsync ... --password1 "$%%&<>|^"^" +The exclamation mark ! does not have to be escaped sometimes, +so have a try. + ======================================================================= Q. On Windows, some passwords begin with an equal = character. Login fails. What can I do? diff --git a/FAQ.d/FAQ.Release_Checklist.txt b/FAQ.d/FAQ.Release_Checklist.txt index b5ae915..6528ecc 100644 --- a/FAQ.d/FAQ.Release_Checklist.txt +++ b/FAQ.d/FAQ.Release_Checklist.txt @@ -1,18 +1,21 @@ #!/bin/cat -$Id: FAQ.Release_Checklist.txt,v 1.6 2017/09/16 20:21:39 gilles Exp gilles $ +$Id: FAQ.Release_Checklist.txt,v 1.10 2019/02/14 15:52:44 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This documentation is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Release_Checklist.txt -=================================== - Imapsync developper notes -=================================== +======================================================================= + Imapsync developer notes +======================================================================= Checklist before release a new release: +- Add a new section in S/news.shtml reading "rlog imapsync" - Generate the README - Run a spell checker on the README -- Read the README again slowly. Fix all issues, all +- Read the README again slowly. Fix all issues, all. - Read the OPTIONS section of README and read it very slowly - Read slowly README_Windows.txt - Read slowly the TUTORIAL_Unix file in html @@ -21,7 +24,9 @@ Checklist before release a new release: - Review the general FAQ.d/FAQ.General.txt - - make dist - make publish - verify VERSION is uploaded + +======================================================================= +======================================================================= diff --git a/FAQ.d/FAQ.Reporting_Bugs.txt b/FAQ.d/FAQ.Reporting_Bugs.txt index 10d7887..fdf6655 100644 --- a/FAQ.d/FAQ.Reporting_Bugs.txt +++ b/FAQ.d/FAQ.Reporting_Bugs.txt @@ -1,11 +1,14 @@ #!/bin/cat -$Id: FAQ.Reporting_Bugs.txt,v 1.3 2018/04/10 00:17:18 gilles Exp gilles $ +$Id: FAQ.Reporting_Bugs.txt,v 1.4 2018/05/24 11:34:30 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Reporting_Bugs.txt -=================================================== - Imapsync bugs reports -=================================================== + +======================================================================= + Imapsync bugs reports +======================================================================= ======================================================================= Q. How can I report bugs or problems I encountered with Imapsync? @@ -25,5 +28,6 @@ than 1MB. Don't zip it, it will slow my response. Thanks! ======================================================================= +======================================================================= diff --git a/FAQ.d/FAQ.SSL_errors.txt b/FAQ.d/FAQ.SSL_errors.txt index 1602ea3..272941e 100644 --- a/FAQ.d/FAQ.SSL_errors.txt +++ b/FAQ.d/FAQ.SSL_errors.txt @@ -1,11 +1,14 @@ #!/bin/cat -$Id: FAQ.SSL_errors.txt,v 1.5 2018/04/25 17:58:12 gilles Exp gilles $ +$Id: FAQ.SSL_errors.txt,v 1.6 2018/05/24 11:34:30 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt -===================================================== - Imapsync SSL errors -===================================================== + +======================================================================= + Imapsync SSL errors +======================================================================= Questions answered in this FAQ are: @@ -74,7 +77,7 @@ R. Use: ======================================================================= ======================================================================= -======================================================================= + diff --git a/FAQ.d/FAQ.Security.txt b/FAQ.d/FAQ.Security.txt index 2d8da17..71a3b4b 100644 --- a/FAQ.d/FAQ.Security.txt +++ b/FAQ.d/FAQ.Security.txt @@ -1,7 +1,10 @@ #!/bin/cat -# $Id: FAQ.Security.txt,v 1.16 2017/04/26 21:50:00 gilles Exp gilles $ +# $Id: FAQ.Security.txt,v 1.19 2018/09/13 17:53:45 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Security.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync tips about security. Issues and solutions. @@ -88,14 +91,18 @@ a) Use native --ssl1 and/or --ssl2 options b) Use stunnel http://www.stunnel.org/ + Use stunnel3 command since stunnel now usually calls + stunnel4 or stunnel5 and the command line options syntax + has changed (option "-c" not recognized for example). + Assuming there is an imaps (993) server on imap.foo.org, on your localhost machine (or bar machine), run: - stunnel -c -d imap -r imap.foo.org:imaps + stunnel3 -c -d imap -r imap.foo.org:imaps -f or using numbers instead of names: - stunnel -c -d 143 -r imap.foo.org:993 + stunnel3 -c -d 143 -r imap.foo.org:993 -f then use imapsync on localhost (or bar machine) imap (143) port. If the local port 143 is already taken then use a free one, @@ -104,7 +111,7 @@ b) Use stunnel c) Other example for accessing gmail with no local root access to open port 143 - stunnel -f -P '' -c -d 9993 -r imap.gmail.com:993 + stunnel3 -P '' -c -d 9993 -r imap.gmail.com:993 -f Then, to access gmail as host2 use: @@ -193,24 +200,6 @@ to help the ssl software verifying the server certificate. The file ca-certificates.crt may be elsewhere on your system, even named differently. -======================================================================= -Q. How can I manually test a login via ssl? - -R. Use ncat or telnet-ssl like in this example: - -ncat --ssl -C imap.gmail.com 993 -* OK Gimap ready for requests from 78.196.254.58 q1mb175739668wix -a LOGIN "gilles.lamiral@gmail.com" "secret" -* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE ... ESEARCH -a OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success) -b LOGOUT -* BYE LOGOUT Requested -b OK 73 good day (Success) - -The client part to type is "a LOGIN ..." and "b LOGOUT" without -the double-quotes. - - ======================================================================= Q: How to have an imaps server? diff --git a/FAQ.d/FAQ.SmarterMail.txt b/FAQ.d/FAQ.SmarterMail.txt index 8cb5033..b2ddd05 100644 --- a/FAQ.d/FAQ.SmarterMail.txt +++ b/FAQ.d/FAQ.SmarterMail.txt @@ -1,7 +1,10 @@ #!/bin/cat -$Id: FAQ.SmarterMail.txt,v 1.11 2017/09/05 15:11:20 gilles Exp gilles $ +$Id: FAQ.SmarterMail.txt,v 1.12 2018/05/24 11:34:30 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.SmarterMail.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync tips for SmarterMail. Specific issues and solutions. diff --git a/FAQ.d/FAQ.TTL.txt b/FAQ.d/FAQ.TTL.txt index db769c3..1ef8928 100644 --- a/FAQ.d/FAQ.TTL.txt +++ b/FAQ.d/FAQ.TTL.txt @@ -1,7 +1,10 @@ #!/bin/cat -$Id: FAQ.TTL.txt,v 1.1 2017/05/03 22:27:45 gilles Exp gilles $ +$Id: FAQ.TTL.txt,v 1.2 2018/05/24 11:34:30 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.TTL.txt -This documentation is also at http://imapsync.lamiral.info/#doc ===================================================================== Imapsync tips about TTL when changing name resolution of hosts @@ -30,3 +33,5 @@ the migration done, no problem to set back the TTL to 24h or more. If you can't decrease TTL under 4h or even 24h, it's ok anyway, imapsync can sync the new messages dropped in the old server. +======================================================================= +======================================================================= diff --git a/FAQ.d/FAQ.Use_addheader.txt b/FAQ.d/FAQ.Use_addheader.txt index 144a2ef..57ab04d 100644 --- a/FAQ.d/FAQ.Use_addheader.txt +++ b/FAQ.d/FAQ.Use_addheader.txt @@ -1,10 +1,13 @@ #!/bin/cat -$Id: FAQ.Use_addheader.txt,v 1.4 2017/01/25 23:54:02 gilles Exp gilles $ +$Id: FAQ.Use_addheader.txt,v 1.6 2018/12/10 20:12:05 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Use_addheader.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= - Imapsync --addheader suggestion. + Imapsync --addheader suggestion. ======================================================================= ======================================================================= @@ -12,18 +15,21 @@ Q. What means this log message: "Host1 Sent/1234 ignored (no wanted headers so we ignore this message. To solve this: use --addheader)" -R. In order to sync messages from one account to another Imapsync has +R. In order to sync messages from one account to another, Imapsync has to identify messages on both sides, and compare them. + Imapsync can use two completely different methods to identify messages on each account. - The first one, the default one, uses some headers of messages, - Real world syncs with various imap servers have bring to use + + The first one, the default one, uses some headers of messages. + Real world syncs with various imap servers have bring me to use two headers "Message-Id:" and "Received:". It works for most messages except for special folders like Sent or Draft where messages in those folders don't have "Message-Id:" nor - "Received:" headers. + "Received:" headers. + Here comes --addheader option. When a message has no "Message-Id:" - nor "Received:", option --addheader adds a "Message-Id" header + nor "Received:" headers, option --addheader adds a "Message-Id" header consisting of the imap UID of the message on the host1 folder, like "Message-Id: 12345@imapsync". This way, messages are well identified on both sides, diff --git a/FAQ.d/FAQ.Use_cache.txt b/FAQ.d/FAQ.Use_cache.txt index 05edd38..64c029d 100644 --- a/FAQ.d/FAQ.Use_cache.txt +++ b/FAQ.d/FAQ.Use_cache.txt @@ -1,11 +1,14 @@ -$Id: FAQ.Use_cache.txt,v 1.6 2018/03/07 04:12:37 gilles Exp gilles $ +$Id: FAQ.Use_cache.txt,v 1.8 2018/10/22 17:30:45 gilles Exp gilles $ -This documentation is also at http://imapsync.lamiral.info/#doc +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt -============================================ - Imapsync --usecache option -============================================ + +======================================================================= + Imapsync --usecache option +======================================================================= Questions answered in this FAQ are: @@ -89,11 +92,19 @@ filesystem devoted to the imapsync cache. # Mount this brand new filesystem + mkdir -p /var/tmp/cachedir mount -o loop /var/tmp/fscache /var/tmp/cachedir + df -i /var/tmp/cachedir # count inodes available # Tell imapsync to use it imapsync ... --tmpdir /var/tmp/cachedir/ - + +# When the migration is over and the cache becomes really useless + + umount /var/tmp/cachedir/ + rm /var/tmp/fscache + + ======================================================================= ======================================================================= diff --git a/FAQ.d/FAQ.User_Concurrent_Access.txt b/FAQ.d/FAQ.User_Concurrent_Access.txt index a7bff9c..18e504c 100644 --- a/FAQ.d/FAQ.User_Concurrent_Access.txt +++ b/FAQ.d/FAQ.User_Concurrent_Access.txt @@ -1,7 +1,10 @@ #!/bin/cat -$Id: FAQ.User_Concurrent_Access.txt,v 1.4 2017/12/19 14:55:36 gilles Exp gilles $ +$Id: FAQ.User_Concurrent_Access.txt,v 1.5 2018/05/24 11:34:30 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.User_Concurrent_Access.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync and user concurrent access. diff --git a/FAQ.d/FAQ.Various_Server_Softwares.txt b/FAQ.d/FAQ.Various_Server_Softwares.txt deleted file mode 100644 index 99bca37..0000000 --- a/FAQ.d/FAQ.Various_Server_Softwares.txt +++ /dev/null @@ -1,283 +0,0 @@ -#!/bin/cat -$Id: FAQ.Various_Server_Softwares.txt,v 1.8 2017/09/05 15:11:20 gilles Exp gilles $ - -This documentation is also at http://imapsync.lamiral.info/#doc - -======================================================================= - Imapsync tips for various imap server softwares. -======================================================================= - - - -======================================================================= -Q. From Zimbra to XXX - -imapsync ... \ - --exclude "Conversation Action Settings" \ - --exclude "Quick Step Settings" \ - --exclude "News Feed" - -======================================================================= -Q. From or to HMailServer version 4.4.1. - -R. You have to add prefix and separator manually because 4.4.1 doesn't -honor the NAMESPACE imap command. - -Example for host1: - -imapsync ... \ - --prefix1 "" --sep1 . - -No specific option for HMailServer 5.3.3 since NAMESPACE is supported. - -Maybe --subscribe_all will help you to see all migrated folders. - - -======================================================================= -Q. Synchronizing from Kerio Connect to XXX - -R. No special options required. -See also: -http://www.linux-france.org/prj/imapsync_list/msg01756.html -http://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-1/ -http://www.safetynet-it.com/it-support/mac-kerio-server-to-microsoft-exchange-2010-migration-2/ - - -======================================================================= -Q. from Microsoft's Exchange 2007 to Google Apps for your Domain - (GAFYD) - -R. Take a look at: -http://mark.ossdl.de/2009/02/migrating-from-exchange-2007-to-google-apps-mail/ - - -======================================================================= -Q. Migrating from or to Parallels Plex Server - -R. It depends on the OS - -Parallells Plesk Panel for Windows requires --sep2 / --prefix2 "" -Parallells Plesk Panel for Linux works with default parameters. - -======================================================================= -Q. I'm migrating from WU to Cyrus, and the mail folders are under - /home/user/mail but the tool copies everything in /home/user, how - can i avoid that? - -Two solutions: - -R. Use - imapsync ... --include '^mail' - -R. or (better) - imapsync ... --subscribed --subscribe - - -======================================================================= -Q. I'm migrating from WU to Cyrus, and the mail folders are under - /home/user/mail directory. When imapsync creates the folders in - the new cyrus imap server, it makes a folder "mail" and below that - folder puts all the mail folders the user have in /home/user/mail, - i would like to have all those folders directly under INBOX. - -R. Use - imapsync ... --regextrans2 's/^mail/INBOX/' --dry - look at the simulation and if all transformations seem - good then remove the --dry option. - - -======================================================================= -Q. Migrating from Groupwise to Cyrus - -R. By Jamie Neil: - -I eventually managed to get the mail to migrate without errors using the -following options: - ---sep1 / - - doesn't report separator so has to be set explicitly. - ---nosyncacls - - doesn't support ACLs. - ---skipheader '^Content-Type' - - MIME separator IDs seem to change every time a mail is accessed so - this is required to stop duplicates. - ---maxage 3650 - - some messages just don't seem to want to transfer and produce the - perl errors I mentioned before. This prevents the errors, but the - bad messages don't transfer. - -Even though the mail migrated OK, there are a couple of gotchas with -Groupwise IMAP: - -1) Some of the GW folders are not real folders and are not available -to IMAP, the main problem one being "Sent Items". I could find no way -of coping the contents of these folders. The nearest I got was to -create a "real" folder and copy/move the sent items into it, but -imapsync still didn't see the messages (I think because there is -something funny about the reported dates/sizes). - -It think this problem has been rectified in GW6.5. - -2) The "skipheader '^Content-Type'" directive is required to stop -duplicate messages being created. GW seems to generate this field on -the fly for messages that have MIME separators and so it's different -every time. - -3) Version 6.0.1 of the Groupwise Internet Connector sucks. I was -getting server aborts when I pushed it a bit hard! I eventually had to -upgrade to 6.0.4 which seems to be a lot more stable. - - -======================================================================= -Q. Migrating from iPlanet Messaging Server -5.2 Patch 2 (built Jul 14 2004)) to Groupwise 7.0 -I encounter many errors like this: -"Error trying to append string: 17847 BAD APPEND" - -R. GroupWise 7 seems buggy. Apply GroupWise 7 support pack 1 - -======================================================================= -Q. Migrating from David Tobit V10 (DvISE Mail Access Server MA-...) - -R. Use the following options: - -imapsync ... --prefix1 "" --sep1 / --idatefromheader ^ - --nofoldersizes --useuid --nocheckmessageexists - -======================================================================= -Q. Migrating from David Tobit V8 - ("* OK IMAP4rev1 DvISE Mail Access Server MA-8.10a (0126)") - -First try above V10 solution since improvments have been made -to support Tobit. - -R. Use the following options : -imapsync ... --prefix1 INBOX. --sep1 / --subscribe --subscribed - -======================================================================= -Q. Migrating from Tobit David Server 6 - ("DvISE Mail Access Server MA-6.60a (0118)") - -First try above V10 solution since improvments have been made -to support Tobit. - -R. Look at the discussion: -http://www.linux-france.org/prj/imapsync_list/msg00582.html -http://www.linux-france.org/prj/imapsync_list/threads.html#00582 -patch saved in ./patches/imapsync-1.337_tobit_V6.patch - -======================================================================= -Q. I need to migrate 1250 mailboxes, passwords are in a MySQL Database. -Can you tell me if your script suits my needs? - -R. Mailboxes must exist before running imapsync. -You have to extract users logins and passwords in a csv file. -See the "HUGE MIGRATION" section in the README file. - - -====================================================================== -Q: From MailEnable 1.75 -R: --sep1 "/" --prefix1 "" - -Q: From MailEnable 2.2 -R: --sep1 "." --prefix1 "" - -Q: To MailEnable -R: --sep2 / --prefix2 "" --addheader --messageidnodomain --syncflagsaftercopy - -====================================================================== -Q. From GMX IMAP4 StreamProxy -R. Use: - --prefix1 INBOX and --sep1 . - -====================================================================== -Q. From Courier to Archiveopteryx -R. You can read http://www.archiveopteryx.org/migration/imapsync - Default values might be fine now with latest imapsync. - -====================================================================== -Q. To Sun Java(tm) System Messaging Server 6.2-7.05 -Q. To Communigate Pro - Solaris version - -R. See and run patches/imapsync_1.267_jari - - -====================================================================== -Q. From Softalk Workgroup Mail 7.6.4 - -R. Old Softalk releases don't support the IMAP SEARCH command. - Here are the options to get it working. - -imapsync ... --sep1 '.' --prefix1 '' \ - --noabletosearch1 --nocheckmessageexists --addheader - -(Thanks to Andrew Tucker) - -====================================================================== -Q. From or to QQMail IMAP4Server - -R. imapsync ... --noabletosearch1 - -====================================================================== -Q. From FirstClass to XXX - http://www.firstclass.com/ - -R. Migrating from FirstClass is not easy because FirstClass, strangely, - does not show all messages via IMAP. To make it show all messages, - a trick, painful to follow by hand, is moving emails - out and back in, for each folder. May be it can be done by a script. - - FirstClass releases prior to release 12 do not shows the "Sent" - folder in IMAP but FirstClass release 12 shows it. - I advice you to upgrade to FirstClass release 12 before leaving it - with imapsync or another imap tool. - -Here is a command line used to migrate from FirtClass release 12: - - imapsync ... \ - --tmpdir /var/tmp --usecache \ - --useheader Message-ID \ - --idatefromheader \ - --addheader \ - --regextrans2 "s,(/|^) +,\$1,g" --regextrans2 "s, +(/|$),\$1,g" \ - --regextrans2 "s/[\^]/_/g" \ - --regextrans2 "s/['\"\\\\]/_/g" \ - --regextrans2 "s,&AC8-,-,g" \ - --regextrans2 "s,&APg-,oe,g" - -On Windows: - imapsync.exe ... ^ - --automap ^ - --usecache ^ - --useheader Message-ID ^ - --idatefromheader ^ - --addheader ^ - --regextrans2 "s,(/|^) +,$1,g" ^ - --regextrans2 "s, +(/|$),$1,g" ^ - --regextrans2 "s/[\^]/_/g" ^ - --regextrans2 "s/['\\]/_/g" ^ - --regextrans2 "s,^&AC8-,-,g" ^ - --regextrans2 "s,^&APg-,oe,g" - - -Special thanks to Kristian Wind and Joey Alexander for helping me -writing this FAQ item. -See also this worth reading discussion in a Zimbra forum: -http://www.zimbra.com/forums/migration/20349-help-needed-migrating-firstclass.html - -====================================================================== -Q. From XXX to FTGate - -R. Do NOT use --usecache since new UIDs are not given by FTGate and also - badly guessed by imapsync. UIDEXPUNGE does not work so use also - --expunge2 when using --delete2 - - imapsync ... \ - --sep2 / --prefix2 "" \ - --useheader Message-Id \ - -======================================================================= -======================================================================= \ No newline at end of file diff --git a/FAQ.d/FAQ.Virus.txt b/FAQ.d/FAQ.Virus.txt index c69fc0a..ece627d 100644 --- a/FAQ.d/FAQ.Virus.txt +++ b/FAQ.d/FAQ.Virus.txt @@ -1,7 +1,10 @@ -$Id: FAQ.Virus.txt,v 1.4 2018/03/21 16:18:53 gilles Exp gilles $ +$Id: FAQ.Virus.txt,v 1.5 2018/05/24 11:34:30 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Virus.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync and virus scanners diff --git a/FAQ.d/FAQ.XOAUTH2.txt b/FAQ.d/FAQ.XOAUTH2.txt index 92e76a6..300415b 100644 --- a/FAQ.d/FAQ.XOAUTH2.txt +++ b/FAQ.d/FAQ.XOAUTH2.txt @@ -1,7 +1,10 @@ #!/bin/cat -$Id: FAQ.XOAUTH2.txt,v 1.12 2017/01/06 14:11:13 gilles Exp gilles $ +$Id: FAQ.XOAUTH2.txt,v 1.15 2018/08/28 21:48:53 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.XOAUTH2.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync tips to use XOAUTH2 authentication (Gmail) and old XOAUTH @@ -70,6 +73,29 @@ imapsync \ Use your own xoauth2 values. +The secret.xoauth2.json looks like: + +{ + "type": "service_account", + "project_id": "your-project-name", + "private_key_id": "1cfb..............................bd7fbe", + "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiGziM...ZV5ACKPHuOfp8A46I=\n-----END PRIVATE KEY-----\n", + "client_email": "jsonfile@your-project-name.iam.gserviceaccount.com", + "client_id": "105................689", + "auth_uri": "https://accounts.google.com/o/oauth2/auth", + "token_uri": "https://accounts.google.com/o/oauth2/token", + "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", + "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/jsonfile%40your-project-name.iam.gserviceaccount.com" +} + +You get this json file by a link like: +https://console.developers.google.com/apis/credentials?project=your-project-name + +See also: +https://developers.google.com/gmail/imap/xoauth2-protocol +https://developers.google.com/identity/protocols/OAuth2 + + ======================================================================= Q. How to use XOAUTH2 via pk12 file to globally authenticate gmail users? diff --git a/FAQ.d/FAQ.Yahoo.txt b/FAQ.d/FAQ.Yahoo.txt index be84bbc..ffd44bf 100644 --- a/FAQ.d/FAQ.Yahoo.txt +++ b/FAQ.d/FAQ.Yahoo.txt @@ -1,7 +1,10 @@ -$Id: FAQ.Yahoo.txt,v 1.7 2017/12/18 10:45:00 gilles Exp gilles $ +$Id: FAQ.Yahoo.txt,v 1.8 2018/05/24 11:34:30 gilles Exp gilles $ + +This document is also available online at +https://imapsync.lamiral.info/FAQ.d/ +https://imapsync.lamiral.info/FAQ.d/FAQ.Yahoo.txt -This documentation is also at http://imapsync.lamiral.info/#doc ======================================================================= Imapsync tips for Yahoo. diff --git a/FAQ.d/htaccess.txt b/FAQ.d/htaccess.txt index d863a81..647f785 100644 --- a/FAQ.d/htaccess.txt +++ b/FAQ.d/htaccess.txt @@ -1,13 +1,15 @@ -# $Id: htaccess.txt,v 1.20 2018/03/17 16:12:54 gilles Exp gilles $ +# $Id: htaccess.txt,v 1.23 2019/02/16 22:42:25 gilles Exp gilles $ -AddDescription "Back to Imapsync main page." .. -AddDescription "Authenticate via an admin account." FAQ.Admin_Authentication.txt +AddDescription "Back to Imapsync main page." .. +AddDescription "APPEND errors." FAQ.Admin_Authentication.txt +AddDescription "Authenticate via an admin account." FAQ.APPEND_errors.txt AddDescription "Authentication failures." FAQ.Authentication_failure.txt -AddDescription "Archiving tips." FAQ.Archiving.txt +AddDescription "Archiving tips." FAQ.Archiving.txt +AddDescription "Big mailboxes tips." FAQ.Big_Mailbox.txt AddDescription "Connection issues." FAQ.Connection.txt -AddDescription "Contacts & Calendars issues." FAQ.Contacts_Calendars.txt +AddDescription "Contacts & Calendars & Chat issues." FAQ.Contacts_Calendars.txt AddDescription "Dates issues." FAQ.Dates.txt AddDescription "Domino." FAQ.Domino.txt AddDescription "Dovecot." FAQ.Dovecot.txt @@ -43,8 +45,9 @@ AddDescription "Option --usecache and inodes." FAQ.Use_cache AddDescription "User concurrent access." FAQ.User_Concurrent_Access.txt AddDescription "Virus scanners on Imapsync." FAQ.Virus.txt AddDescription "Why use --addheader?." FAQ.Use_addheader.txt -AddDescription "Various imap server softwares." FAQ.Various_Server_Softwares.txt +AddDescription "Various imap software servers." FAQ.Various_Software_Servers.txt AddDescription "XOAUTH2 (Gmail)." FAQ.XOAUTH2.txt AddDescription "Yahoo." FAQ.Yahoo.txt +AddDescription "Zimbra." FAQ.Zimbra.txt AddDescription "Where this Description column comes from." htaccess.txt diff --git a/INSTALL.d/INSTALL.ANY.txt b/INSTALL.d/INSTALL.ANY.txt index 08ceaf9..2cd4841 100644 --- a/INSTALL.d/INSTALL.ANY.txt +++ b/INSTALL.d/INSTALL.ANY.txt @@ -1,26 +1,31 @@ -# $Id: INSTALL.ANY.txt,v 1.57 2018/02/11 13:35:14 gilles Exp gilles $ +# $Id: INSTALL.ANY.txt,v 1.58 2018/05/24 11:43:31 gilles Exp gilles $ # # This is the main INSTALL file for imapsync. # imapsync : IMAP sync and migrate tool. -INTRODUCTION -============ +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.ANY.txt + +======================================================================= + INTRODUCTION +======================================================================= imapsync works fine under any operating system with Perl and Perl modules (listed below). imapsync.exe works fine standalone under Windows XP, Vista, Seven, 20XX, either 32 or 64bit. -==================================== -== Installing imapsync on WINDOWS == -==================================== +======================================================================= + Installing imapsync on WINDOWS +======================================================================= Read the file README_Windows.txt Also available at https://imapsync.lamiral.info/README_Windows.txt -================================= -== Installing imapsync on Unix == -================================= +======================================================================= + Installing imapsync on Unix +======================================================================= There are specific INSTALL files in the imapsync directory INSTALL.d/ Those documents are also available at https://imapsync.lamiral.info/INSTALL.d/ @@ -37,18 +42,18 @@ Those documents are also available at https://imapsync.lamiral.info/INSTALL.d/ If you are not on one of these systems then read the section below called "Installing imapsync on other Unixes". -===================================== -== Installing imapsync on Mac OS X == -===================================== +======================================================================= + Installing imapsync on Mac OS X +======================================================================= Easy. Read the file INSTALL.d/INSTALL.Darwin.txt This document is also available at https://imapsync.lamiral.info/INSTALL.d/INSTALL.Darwin.txt -===================================== -== Installing imapsync on FreeBSD == -===================================== +======================================================================= + Installing imapsync on FreeBSD +======================================================================= Easy. Read the file INSTALL.d/INSTALL.FreeBSD.txt @@ -56,9 +61,9 @@ This document is also available at https://imapsync.lamiral.info/INSTALL.d/INSTALL.FreeBSD.txt -=================================== -== Installing imapsync on CentOS == -=================================== +======================================================================= + Installing imapsync on CentOS +======================================================================= Easy. Read the file INSTALL.d/INSTALL.Centos.txt @@ -66,9 +71,9 @@ This document is also available at https://imapsync.lamiral.info/INSTALL.d/INSTALL.Centos.txt -=================================== -== Installing imapsync on CPanel == -=================================== +======================================================================= + Installing imapsync on CPanel +======================================================================= Easy. Read the file INSTALL.d/INSTALL.CPanel.txt @@ -76,45 +81,45 @@ This document is also available at https://imapsync.lamiral.info/INSTALL.d/INSTALL.CPanel.txt -========================================== -== Installing imapsync on Debian 6 or 7 == -========================================== +======================================================================= + Installing imapsync on Debian 6 or 7 +======================================================================= Not so easy. See the file INSTALL.d/INSTALL.Debian.txt This document is also available at https://imapsync.lamiral.info/INSTALL.d/INSTALL.Debian.txt -============================================ -== Installing imapsync on Ubuntu 12 or 14 == -============================================ +======================================================================= + Installing imapsync on Ubuntu 12 or 14 +======================================================================= Not so easy. See the file INSTALL.d/INSTALL.Ubuntu.txt This document is also available at https://imapsync.lamiral.info/INSTALL.d/INSTALL.Ubuntu.txt -================================== -= Installing imapsync on AWS EC2 = -================================== +======================================================================= + Installing imapsync on AWS EC2 +======================================================================= Not so easy. See the file INSTALL.d/INSTALL.AWS_EC2.txt This document is also available at https://imapsync.lamiral.info/INSTALL.d/INSTALL.AWS_EC2.txt -====================================== -= Installing imapsync online like /X = -====================================== +======================================================================= + Installing imapsync online like /X +======================================================================= Follow the document INSTALL.d/INSTALL.OnlineUI.txt This document is also available at https://imapsync.lamiral.info/INSTALL.d/INSTALL.OnlineUI.txt -========================================= -== Installing imapsync on other Unixes == -========================================= +======================================================================= + Installing imapsync on other Unixes +======================================================================= Purchase imapsync at @@ -172,3 +177,5 @@ Now imapsync should work on your system. To check it run: ./imapsync --testslive +======================================================================= +======================================================================= diff --git a/INSTALL.d/INSTALL.AWS_EC2.txt b/INSTALL.d/INSTALL.AWS_EC2.txt index 8722b2c..6304dd3 100644 --- a/INSTALL.d/INSTALL.AWS_EC2.txt +++ b/INSTALL.d/INSTALL.AWS_EC2.txt @@ -1,9 +1,13 @@ #!/bin/cat -# $Id: INSTALL.AWS_EC2.txt,v 1.4 2017/11/19 07:50:24 gilles Exp gilles $ +# $Id: INSTALL.AWS_EC2.txt,v 1.5 2018/05/24 11:43:31 gilles Exp gilles $ -================================== -= Installing imapsync on AWS EC2 = -================================== +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.AWS_EC2.txt + +======================================================================= + Installing imapsync on AWS EC2 +======================================================================= AWS: Amazon Web Service @@ -56,3 +60,5 @@ chmod +x imapsync ./imapsync --testslive +======================================================================= +======================================================================= diff --git a/INSTALL.d/INSTALL.CPanel.txt b/INSTALL.d/INSTALL.CPanel.txt index 2d2b7de..6dbfa09 100644 --- a/INSTALL.d/INSTALL.CPanel.txt +++ b/INSTALL.d/INSTALL.CPanel.txt @@ -1,13 +1,13 @@ #!/bin/cat -# $Id: INSTALL.CPanel.txt,v 1.5 2016/11/07 10:40:53 gilles Exp gilles $ +# $Id: INSTALL.CPanel.txt,v 1.6 2018/05/24 11:43:31 gilles Exp gilles $ -================================= -= Installing imapsync on CPanel = -================================= +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.CPanel.txt -============ -== CPanel == -============ +======================================================================= + Installing imapsync on CPanel +======================================================================= This procedure has been tested on cPanel 11.36 (64 bits). It should be the same for any Cpanel distribution based on Centos. @@ -24,4 +24,7 @@ Then install imapsync and its dependencies: After installing imapsync, it should be able to work on your system. A good test that shows also the basic example: - imapsync \ No newline at end of file + imapsync + +======================================================================= +======================================================================= diff --git a/INSTALL.d/INSTALL.Centos.txt b/INSTALL.d/INSTALL.Centos.txt index 05a9a34..2a0f878 100644 --- a/INSTALL.d/INSTALL.Centos.txt +++ b/INSTALL.d/INSTALL.Centos.txt @@ -1,15 +1,19 @@ #!/bin/cat -# $Id: INSTALL.Centos.txt,v 1.7 2018/04/24 00:15:32 gilles Exp gilles $ +# $Id: INSTALL.Centos.txt,v 1.10 2019/02/05 18:20:42 gilles Exp gilles $ -================================= -= Installing imapsync on CentOS = -================================= +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.Centos.txt + +======================================================================= + Installing imapsync on CentOS +======================================================================= There is one section for Centos 7 and one for Centos 6. -============== -== Centos 7 == -============== +======================================================================= + Centos 7 +======================================================================= First install access to the Epel repository via yum: @@ -32,9 +36,9 @@ Unit tests: imapsync --tests -================================== -== Centos 7 and latest imapsync == -================================== +======================================================================= + Centos 7 and latest imapsync +======================================================================= At the time of this writing (April 2018), the epel repository contains imapsync release 1.727, which is not the latest available release. @@ -61,10 +65,7 @@ Then: perl-Test-Fatal \ perl-Test-Mock-Guard \ perl-Test-Requires \ - perl-Test-NoWarnings \ - perl-Test-Deep \ - perl-Test-Warn \ - perl-Try-Tiny + perl-Test-Deep wget https://imapsync.lamiral.info/imapsync @@ -73,10 +74,16 @@ Then: Now you have the latest imapsync on Centos 7. +If you want to install an online imapsync service +like /X then you also need a recent CGI.pm, +add with cpanm: -============== -== Centos 6 == -============== + cpanm CGI + + +======================================================================= + Centos 6 +======================================================================= This section has been tested with imapsync release 1.670 @@ -102,4 +109,40 @@ Unit tests: imapsync --tests +======================================================================= + Centos 6 and latest imapsync +======================================================================= +Install the epel imapsync 1.670 release via yum like mentioned above. + +It will install many needed packages, Perl itself and Perl modules +for release 1.670. The latest imapsync needs more Perl modules. + +Then: + + yum install perl-App-cpanminus \ + perl-Dist-CheckConflicts \ + perl-HTML-Parser \ + perl-libwww-perl \ + perl-Module-Implementation \ + perl-Module-ScanDeps \ + perl-Package-Stash \ + perl-Package-Stash-XS \ + perl-PAR-Packer \ + perl-Regexp-Common \ + perl-Sys-MemInfo \ + perl-Test-Fatal \ + perl-Test-Mock-Guard \ + perl-Test-Requires \ + perl-Test-Deep + + wget https://imapsync.lamiral.info/imapsync + + chmod +x imapsync + ./imapsync --testslive + +Now you have the latest imapsync on Centos 6, +do you? + +======================================================================= +======================================================================= diff --git a/INSTALL.d/INSTALL.Darwin.txt b/INSTALL.d/INSTALL.Darwin.txt index 52a5a67..c92912d 100644 --- a/INSTALL.d/INSTALL.Darwin.txt +++ b/INSTALL.d/INSTALL.Darwin.txt @@ -1,9 +1,13 @@ #!/bin/cat -# $Id: INSTALL.Darwin.txt,v 1.17 2017/11/30 02:10:27 gilles Exp gilles $ +# $Id: INSTALL.Darwin.txt,v 1.19 2018/12/06 10:07:58 gilles Exp gilles $ -=================================================== -= Installing imapsync binary on Darwin / Mac OS X = -=================================================== +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.Darwin.txt + +======================================================================= + Installing imapsync binary on Darwin / Mac OS X +======================================================================= There is a standalone imapsync binary for Mac OS X called "imapsync_bin_Darwin" (without the quotes), @@ -52,10 +56,23 @@ you have to use: --host1 test1.lamiral.info --user1 test1 --password1 secret1 \ --host2 test2.lamiral.info --user2 test2 --password2 secret2 +======================================================================= + Installing imapsync script on Darwin / Mac OS X with brew +======================================================================= -=================================================== -= Installing imapsync script on Darwin / Mac OS X = -=================================================== +In case you have brew installed on your system ( see https://brew.sh/ ), +install imapsync with the command: + + brew install imapsync + +Do a real test with imap server test.lamiral.info: + + imapsync --testslive + + +======================================================================= + Installing imapsync script on Darwin / Mac OS X the way I do +======================================================================= This part is only for advanced Unix users, or brave users. @@ -113,9 +130,9 @@ Now imapsync should work on your system. ./imapsync --testslive -================================================= -= Building imapsync binary on Darwin / Mac OS X = -================================================= +======================================================================= + Building imapsync binary on Darwin / Mac OS X +======================================================================= cpanm Module::ScanDeps cpanm PAR::Packer @@ -128,10 +145,12 @@ pp -o imapsync.bin imapsync ./imapsync.bin --module -========================================== -== Installing imapsync Mac OS X Leopard == -========================================== +======================================================================= + Installing imapsync Mac OS X Leopard +======================================================================= Mac OS X Leopard has perl 5.8.8 and an old openssl version. +======================================================================= +======================================================================= diff --git a/INSTALL.d/INSTALL.Debian.txt b/INSTALL.d/INSTALL.Debian.txt index 0fa211f..30ff18e 100644 --- a/INSTALL.d/INSTALL.Debian.txt +++ b/INSTALL.d/INSTALL.Debian.txt @@ -1,5 +1,10 @@ #!/bin/cat -# $Id: INSTALL.Debian.txt,v 1.21 2018/05/06 15:03:04 gilles Exp gilles $ +# $Id: INSTALL.Debian.txt,v 1.25 2018/11/20 10:28:33 gilles Exp gilles $ + +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.Debian.txt + There is three install sections in this document, one for Debian 9 Stretch @@ -18,13 +23,13 @@ See also a more detailed story at https://lists.debian.org/debian-user/2016/11/msg00849.html -=========================================== -= Installing imapsync on Debian 9 Stretch = -=========================================== +======================================================================= + Installing imapsync on Debian 9 Stretch +======================================================================= Here is the command to install imapsync dependencies. The repositories are the classical ones, ie, a -sources.list file with: +/etc/apt/sources.list file with: deb http://deb.debian.org/debian stretch main contrib @@ -65,9 +70,9 @@ After installing the dependencies, imapsync should be working, go to the section "After installing the dependencies" below. -========================================== -= Installing imapsync on Debian 8 Jessie = -========================================== +======================================================================= + Installing imapsync on Debian 8 Jessie +======================================================================= Here are the commands to install imapsync dependencies. You need root priviledge to run them. @@ -134,9 +139,9 @@ After installing the dependencies, imapsync should be working, go to the section "After installing the dependencies" below. -========================================== -= Installing imapsync on Debian 7 Wheezy = -========================================== +======================================================================= + Installing imapsync on Debian 7 Wheezy +======================================================================= Here are the two commands to install imapsync dependencies. You need root priviledge to run them. @@ -197,9 +202,9 @@ the module JSON::WebToken with the command: After installing the dependencies, imapsync should be working. -===================================== -= After installing the dependencies = -====================================== +======================================================================= + After installing the dependencies +======================================================================= You don't have to be root to test and use imapsync. @@ -230,3 +235,5 @@ on the system: Now go to read http://imapsync.lamiral.info/#doc start with the tutorial. +======================================================================= +======================================================================= diff --git a/INSTALL.d/INSTALL.Docker.txt b/INSTALL.d/INSTALL.Docker.txt index 6ef7895..2b2d59e 100644 --- a/INSTALL.d/INSTALL.Docker.txt +++ b/INSTALL.d/INSTALL.Docker.txt @@ -1,21 +1,34 @@ #!/bin/cat -# $Id: INSTALL.Docker.txt,v 1.2 2017/10/22 14:21:46 gilles Exp gilles $ +# $Id: INSTALL.Docker.txt,v 1.4 2018/09/13 17:50:06 gilles Exp gilles $ -======================================= -= Installing imapsync docker image -======================================= +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.Docker.txt -== Installation == +======================================================================= + Installing imapsync docker image +======================================================================= - docker pull gilleslamiral/imapsync +========================== +== Installation & usage == +========================== -== Usage == +That's all you have to run: docker run gilleslamiral/imapsync imapsync +========================= +== Installation alone == +========================== + +To install or update the imapsync image, run: + + docker pull gilleslamiral/imapsync == Docker hub location == https://hub.docker.com/r/gilleslamiral/imapsync/ +======================================================================= +======================================================================= diff --git a/INSTALL.d/INSTALL.Docker_build.txt b/INSTALL.d/INSTALL.Docker_build.txt index 391438e..248c76c 100644 --- a/INSTALL.d/INSTALL.Docker_build.txt +++ b/INSTALL.d/INSTALL.Docker_build.txt @@ -1,9 +1,13 @@ #!/bin/cat -# $Id: INSTALL.Docker_build.txt,v 1.7 2018/04/10 00:16:18 gilles Exp gilles $ +# $Id: INSTALL.Docker_build.txt,v 1.8 2018/05/24 11:43:31 gilles Exp gilles $ -================================================= -= Building an imapsync docker image from Debian = -================================================= +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.Docker_build.txt + +======================================================================= + Building an imapsync docker image from Debian +======================================================================= # Dockerfile for building a docker imapsync image @@ -50,3 +54,6 @@ USER nobody CMD ["/usr/bin/imapsync"] # End of Dockerfile + +======================================================================= +======================================================================= diff --git a/INSTALL.d/INSTALL.FreeBSD.txt b/INSTALL.d/INSTALL.FreeBSD.txt index c6dec1a..67e9796 100644 --- a/INSTALL.d/INSTALL.FreeBSD.txt +++ b/INSTALL.d/INSTALL.FreeBSD.txt @@ -1,9 +1,13 @@ #!/bin/cat -# $Id: INSTALL.FreeBSD.txt,v 1.8 2018/04/10 00:16:52 gilles Exp gilles $ +# $Id: INSTALL.FreeBSD.txt,v 1.10 2018/07/03 04:29:11 gilles Exp gilles $ -================================== -= Installing imapsync on FreeBSD = -================================== +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.FreeBSD.txt + +======================================================================= + Installing imapsync on FreeBSD +======================================================================= Thanks to Kurt Jaeger pi@FreeBSD.org a imapsync package is available in FreeBSD http://portsmon.freebsd.org/portoverview.py?category=mail&portname=imapsync @@ -16,4 +20,16 @@ or if you're already root on the console: pkg install imapsync Tested 7 March 2015 on FreeBSD 10.1-RELEASE - +Tested 2 July 2018 on FreeBSD 11.2-RELEASE + +======================================================================= +======================================================================= + +In order to install and be able to run latest imapsync online /X, +install also: +p5-Regexp-Common +p5-Test-MockObject +p5-CGI + +Call to FreeBSD developer: It's also the new dependencies to add +for next the imapsync package 1.882 diff --git a/INSTALL.d/INSTALL.OnlineUI.txt b/INSTALL.d/INSTALL.OnlineUI.txt index 8204f55..de17bff 100644 --- a/INSTALL.d/INSTALL.OnlineUI.txt +++ b/INSTALL.d/INSTALL.OnlineUI.txt @@ -1,78 +1,148 @@ #!/bin/cat -# $Id: INSTALL.OnlineUI.txt,v 1.14 2018/04/24 00:14:51 gilles Exp gilles $ +# $Id: INSTALL.OnlineUI.txt,v 1.27 2018/11/08 14:51:09 gilles Exp gilles $ -============================== -= Installing imapsync online = -============================== +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.OnlineUI.txt + +======================================================================= + Installing imapsync online +======================================================================= Please consider this as relatively new and experimental. -I add I'm beginning to be confident with /X since the /X service -is up and running quite well since January 2017. +However, to a certain extent, I'm now confident with /X +since the /X service is up and running quite well since January 2017. +======================================================================= + Hardware consideration + + +RAM used per imapsync process, mean value: 230 MB. +Average_bandwidth_rate: 345 KiB/s ~ 2.8 Mbps. +Load mean: 0.8 on a CPU 4 cores "Intel(R) i5-2320 3.00GHz K8-class" + + + +======================================================================= + Installation + You have to be a little familiar with what a CGI script is -and how to activate a CGI script on an the Apache (or any other) -HTTP server. +and how to activate a CGI script on the Apache +HTTP server, or any other HTTP server. Linux is also a +preferred platform. -The web visual user interface frontend is compounded in three -files, a html5 file, a css file, and a javascript file: +I haven't tested yet this visual interface on Windows nor Mac, +feedback is welcome from successes or failures on those platforms. +Like imapsync is written, it should fail on Windows because of +some hard coded Unix paths. + +The web visual user interface frontend /X is compounded in four +files: a html5 file, a css file, a javascript file and +a logo image: + +* https://i005.lamiral.info/X/imapsync_form.html +* https://i005.lamiral.info/X/imapsync_form.css +* https://i005.lamiral.info/X/imapsync_form.js +* https://i005.lamiral.info/X/logo_imapsync_Xn.png -* https://imapsync.lamiral.info/X/imapsync_form.html -* https://imapsync.lamiral.info/X/imapsync_form.js -* https://imapsync.lamiral.info/X/imapsync_form.css You can do a "view source" to see the html file as it is written, and a "save" to get it locally. -The two other files called imapsync_form.js and -imapsync_form.css can be saved the same way. +The three other files can be saved the same way or +with wget. -Those three files can be put anywhere on a web server, +Those four files can be put anywhere on a web server, as long as they stand in the same directory. If you want to put them in different directories, just change the content of imapsync_form.html to reflect the change, ie, change the two lines referencing imapsync_form.css and imapsync_form.js href="imapsync_form.css" (near the beginning of imapsync_form.html) src="imapsync_form.js" (near the end of imapsync_form.html) +I let you changing the image logo as an exercise. The actual imap syncing work is done by imapsync acting as a CGI, the visual interface is only there to give imapsync the parameters needed for the sync. -In order to make imapsync work as a cgi, there is two conditions. -First it has to work by itself on the web host. If imapsync -doesn't work by itself, on a command line, then it won't -work as a cgi. +Use at least Perl module CGI.pm release 4.08 (2014-10-18) +to avoid the bug "Undefined subroutine CGI::multi_param". +You can use the command cpanm to upgrade CGI.pm to its last version, +it's the easiest way. -This imapsync_form.html file in action calls the CGI location +Print the CGI.pm release with: + + perl -MCGI -e 'print "$CGI::VERSION\n"' + +To check and fix the Perl modules dependencies, run: + + cd + wget -N https://imapsync.lamiral.info/prerequisites_imapsync + sh prerequisites_imapsync + +In order to make imapsync work as a cgi, there is two conditions. +First, imapsync has to work by itself on the web host. +If imapsync doesn't work by itself, as a command line, +then it won't work as a cgi. +Second, imapsync has to work by itself on the web host using +the Unix user running the web server. +Detailed examples to verify that will be provided further +in this document. + +The imapsync_form.html file in action calls the CGI location /cgi-bin/imapsync -which has to be imapsync itself. +which has to be imapsync itself, the file script (not the directory). The very latest and relatively stable imapsync is at https://imapsync.lamiral.info/imapsync It is the program file used verbatim for the service given at -https://imapsync.lamiral.info/X/ +https://i005.lamiral.info/X/ -Copy imapsync_form.* on a directory that is visible by your HTTP server. +Copy the three files imapsync_form.* on a directory that is exported +by your HTTP server. Copy the imapsync script on the cgi-bin/ directory allowing CGIs and you'll have your own imapsync visual interface -and service. - -I haven't tested yet this visual interface on Windows nor Mac, -feedback is welcome from successes or failures on those platforms. +and service. The cgi-bin/ directory is usually outside the +hierarchy exported to anybody by the HTTP server. -Example on a Debian server with Apache: + +Concrete example on a Debian server with Apache: Imapsync script place on the server disk: /usr/lib/cgi-bin/imapsync +Get and install latest imapsync: + + cd + wget -N -O /usr/lib/cgi-bin/imapsync https://imapsync.lamiral.info/imapsync + chmod +x /usr/lib/cgi-bin/imapsync + + Check it like this: + + cd + /usr/lib/cgi-bin/imapsync --testslive + +Assuming that the Unix account running Apache is www-data, +check that it will work under Apache with this command: + + su -s /bin/sh -c 'SERVER_SOFTWARE=foo /usr/lib/cgi-bin/imapsync' www-data + + wget -nv -S -O- http://localhost/cgi-bin/imapsync?testslive=1 + +It should print something like: +Status: 200 OK to sync IMAP boxes. Load on bar is ... +... + + This classical /cgi-bin directory is already configured in the Apache configuration file /etc/apache2/sites-available/default-ssl or /etc/apache2/sites-available/default -The configuration file contains somewhere, -maybe in comments for now, with some # characters +The configuration file contains the following section +somewhere, maybe in comments for now, ie, with +some # characters at the beginning to make them ignored: ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ @@ -91,25 +161,44 @@ The UI front-end file place on the server disk is but it can be placed it anywhere on disk, the important thing is that it has to be served by the web server. -The imapsync working directory in cgi mode is + mkdir /var/www/html/X/ + cd /var/www/html/X/ + wget -N \ + https://i005.lamiral.info/X/imapsync_form.html \ + https://i005.lamiral.info/X/imapsync_form.css \ + https://i005.lamiral.info/X/imapsync_form.js \ + https://i005.lamiral.info/X/logo_imapsync_Xn.png + ln -s imapsync_form.html index.html + +The imapsync process working directory in cgi mode is /var/tmp/imapsync_cgi/ it is not configurable unless changing it in imapsync directly, it is hard-coded in imapsync. In this directory will go the log files and the pid files. -Use at least CGI.pm release 4.08 (2014-10-18) -to avoid the bug "Undefined subroutine CGI::multi_param" -You can use the command cpanm to upgrade CGI.pm to its last version, -it's the easiest way. - - Example on a Centos 7 server with httpd Apache: mkdir /var/www/html/X/ - cp imapsync_form.html imapsync_form.js imapsync_form.css /var/www/html/X/ + cd /var/www/html/X/ + wget -N \ + https://i005.lamiral.info/X/imapsync_form.html \ + https://i005.lamiral.info/X/imapsync_form.css \ + https://i005.lamiral.info/X/imapsync_form.js \ + https://i005.lamiral.info/X/logo_imapsync_Xn.png + ln -s imapsync_form.html index.html + + cd + wget -N https://imapsync.lamiral.info/imapsync + chmod +x imapsync + # some basic tests + ./imapsync + ./imapsync --testslive cp imapsync /var/www/cgi-bin/ + su -s /bin/sh -c 'SERVER_SOFTWARE=foo /var/www/cgi-bin/imapsync' apache + wget -nv -S -O- http://localhost/cgi-bin/imapsync?testslive=1 + Check http://yourhost/X/imapsync_form.html @@ -118,11 +207,16 @@ or the safer That's all for installing a /X service on Centos 7. -====== mod_perl ====== +======================================================================= +======================================================================= + +====== mod_perl failure ====== + +This part is for mod_perl experts only. The script imapsync doesn't work under Modperl::Registry -nor under ModPerl::PerlRun so read on if you think you -are better than me. +nor under ModPerl::PerlRun. So read on if you think you +are better than me. I've try the standard way, telling how any cgi perl script can be run under mod_perl perlrun, but it fails with imapsync. @@ -166,5 +260,5 @@ Any hint welcome! curl http://localhost/perl-run/imapsync - -====================== +======================================================================= +======================================================================= diff --git a/INSTALL.d/INSTALL.Ubuntu.txt b/INSTALL.d/INSTALL.Ubuntu.txt index 4c7dbd7..b0ceed4 100644 --- a/INSTALL.d/INSTALL.Ubuntu.txt +++ b/INSTALL.d/INSTALL.Ubuntu.txt @@ -1,21 +1,25 @@ #!/bin/cat -# $Id: INSTALL.Ubuntu.txt,v 1.12 2018/05/06 15:03:29 gilles Exp gilles $ +# $Id: INSTALL.Ubuntu.txt,v 1.15 2018/11/20 10:28:05 gilles Exp gilles $ -================================================= -= Installing imapsync on Ubuntu 16.04 or higher = -================================================= +This documentation is also located online at +https://imapsync.lamiral.info/INSTALL.d/ +https://imapsync.lamiral.info/INSTALL.d/INSTALL.Ubuntu.txt + +======================================================================= + Installing imapsync on Ubuntu 16.04 or higher +======================================================================= Here is the command to install imapsync dependencies, you need root privilege to run them. This command installs standard Ubuntu packages: - sudo apt-get install \ -libauthen-ntlm-perl \ -libclass-load-perl \ -libcrypt-ssleay-perl \ -libdata-uniqid-perl \ -libdigest-hmac-perl \ + sudo apt-get install \ +libauthen-ntlm-perl \ +libclass-load-perl \ +libcrypt-ssleay-perl \ +libdata-uniqid-perl \ +libdigest-hmac-perl \ libdist-checkconflicts-perl \ libfile-copy-recursive-perl \ libio-compress-perl \ @@ -24,15 +28,16 @@ libio-socket-ssl-perl \ libio-tee-perl \ libmail-imapclient-perl \ libmodule-scandeps-perl \ +libnet-dbus-perl \ libnet-ssleay-perl \ libpar-packer-perl \ libreadonly-perl \ libregexp-common-perl \ -libssl-dev \ libsys-meminfo-perl \ libterm-readkey-perl \ libtest-fatal-perl \ libtest-mock-guard-perl \ +libtest-mockobject-perl \ libtest-pod-perl \ libtest-requires-perl \ libtest-simple-perl \ @@ -87,3 +92,5 @@ You can use imapsync. Now go to read http://imapsync.lamiral.info/#doc start with the tutorial. +======================================================================= +======================================================================= diff --git a/INSTALL.d/prerequisites_imapsync b/INSTALL.d/prerequisites_imapsync index 48dc40b..9e6869d 100755 --- a/INSTALL.d/prerequisites_imapsync +++ b/INSTALL.d/prerequisites_imapsync @@ -1,6 +1,6 @@ #!/bin/sh -# $Id: prerequisites_imapsync,v 1.25 2018/04/24 00:14:23 gilles Exp gilles $ +# $Id: prerequisites_imapsync,v 1.29 2018/09/28 14:16:44 gilles Exp gilles $ MODULES_MANDATORY=' App::cpanminus @@ -45,14 +45,16 @@ Test::MockObject Test::More Test::Pod Test::Requires -Test::NoWarnings Test::Deep -Test::Warn Try::Tiny Unicode::String URI::Escape ' +MODULES_FOR_CODERS=' +Smart::Comments +' + test_perl() { # First we need perl @@ -62,7 +64,7 @@ test_perl() { echo Ok: Found Perl $perl_version return 0 else - echo Failure: Perl is not here. You have to install Perl first. + echo 'Failure: Perl is not here. You have to install Perl first.' return 1 fi } @@ -70,17 +72,31 @@ test_perl() { test_make() { # Second we need make to build some Perl modules - if make -v > /dev/null 2>&1 ; then + if echo foo: | make -f - foo ; then make_version=`make -v |head -1` echo Ok: Found make $make_version return 0 else - echo Failure: make is not here. You have to install the make command. + echo 'Failure: make is not here. You have to install the "make" command.' return 1 fi } +test_cpanm() { +# Second we need make to build some Perl modules + # redirect "cpanm ... < /dev/null" is there for macos buid via ssh + # no clue why it's necessary + if cpanm -h > /dev/null 2>&1 < /dev/null; then + cpanm_version=`cpanm --version < /dev/null | head -1` + echo Ok: Found cpanm $cpanm_version + return 0 + else + echo 'Failure: cpanm is not here. You have to install the "cpanm" command.' + return 1 + fi +} + test_module() { @@ -109,16 +125,27 @@ search_modules_any() { echo "All needed modules are already installed" return } + + # Debian, Ubuntu & Co apt-file -h > /dev/null 2>&1 && { search_modules_apt "$@" return } + + # Centos & Co yum -h > /dev/null 2>&1 && { search_modules_yum "$@" return } + + # ArchLinux & Co + pacman -h > /dev/null 2>&1 && { + search_modules_pacman "$@" + return + } + - # no yum, no apt-get + # no yum, no apt-get, no pacman { search_modules_cpanm "$@" return @@ -132,6 +159,31 @@ cpanm $@ EOD } +search_modules_pacman() { + echo + echo Searching pacman packages names + #echo pacman -Fy + #pacman -Fy + echo pacman -S --noconfirm --needed pkgfile + pacman -S --noconfirm --needed pkgfile + echo pkgfile --update + pkgfile --update + for M in "$@" ; do + echo "==== Searching pacman package name for $M" + F=`echo $M|tr -s ":" "/"`.pm + + # Not very good "pacman -Fs" + #echo pacman -Fs "$F" + #echo + #pacman -Fs "$F" + #echo + # Better! pkgfile --reg + echo "pkgfile --reg $F | grep perl-" + echo + pkgfile --reg "$F" | grep perl- + echo + done +} search_modules_yum() { echo @@ -181,6 +233,8 @@ test_unix() { uname -a } + + test_unix #exit test_perl || exit @@ -193,4 +247,5 @@ EXIT=$? search_modules_any $LIST_TO_INSTALL test "$1" = "MODULES_MANDATORY" && search_modules_any $MODULES_MANDATORY +test_cpanm exit $EXIT diff --git a/Makefile b/Makefile index 7178858..bfd08eb 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ -# $Id: Makefile,v 1.269 2018/05/06 15:09:09 gilles Exp gilles $ +# $Id: Makefile,v 1.289 2019/02/17 15:25:37 gilles Exp gilles $ .PHONY: help usage all doc @@ -35,6 +35,7 @@ usage: @echo "make upload_X # upload online UI" @echo "make upload_latest # upload latest imapsync and binaries (dev)" @echo "make upload_cgi # upload latest imapsync online, after local and remote --tests success." + @echo "make upload_cgi_memo # upload cgi_memo stat_patterns.txt to /X servers." @echo "make valid_index # check index.shtml for good syntax" @echo "make upload_ks" @echo "make imapsync.exe" @@ -56,29 +57,29 @@ usage: PREFIX ?= /usr -DIST_PATH ::= dist2/ -VERSION ::= $(shell perl ./imapsync --version 2>/dev/null || cat VERSION) -VERSION_PREVIOUS ::= $(shell perl ./$(DIST_PATH)/imapsync --version 2>/dev/null || echo ERROR) -VERSION_EXE ::= $(shell cat ./VERSION_EXE) +DIST_PATH := dist2/ +VERSION := $(shell perl ./imapsync --version 2>/dev/null || cat VERSION) +VERSION_PREVIOUS := $(shell perl ./$(DIST_PATH)/imapsync --version 2>/dev/null || echo ERROR) +VERSION_EXE := $(shell cat ./VERSION_EXE) -IMAPClient_3xx ::= ./W/Mail-IMAPClient-3.39/lib -IMAPClient ::= $(IMAPClient_3xx) +IMAPClient_3xx := ./W/Mail-IMAPClient-3.40/lib +IMAPClient := $(IMAPClient_3xx) -DIST_NAME ::= imapsync-$(VERSION) -DIST_FILE ::= $(DIST_NAME).tgz -DEB_FILE ::= $(DIST_NAME).deb +DIST_NAME := imapsync-$(VERSION) +DIST_FILE := $(DIST_NAME).tgz +DEB_FILE := $(DIST_NAME).deb -HELLO ::= $(shell date;uname -a) +HELLO := $(shell date;uname -a) -HOSTNAME ::= $(shell hostname -s) -ARCH ::= $(shell uname -m) -KERNEL ::= $(shell uname -s) -BIN_NAME ::= imapsync_bin_$(KERNEL)_$(ARCH) -DISTRO_NAME ::= $(shell lsb_release -i -s || echo Unknown) -DISTRO_RELEASE ::= $(shell lsb_release -r -s || echo 0.0) -DISTRO_CODE ::= $(shell lsb_release -c -s || echo Unknown) -DISTRO ::= $(DISTRO_NAME)_$(DISTRO_RELEASE)_$(DISTRO_CODE) +HOSTNAME := $(shell hostname -s) +ARCH := $(shell uname -m) +KERNEL := $(shell uname -s) +BIN_NAME := imapsync_bin_$(KERNEL)_$(ARCH) +DISTRO_NAME := $(shell lsb_release -i -s || echo Unknown) +DISTRO_RELEASE := $(shell lsb_release -r -s || echo 0.0) +DISTRO_CODE := $(shell lsb_release -c -s || echo Unknown) +DISTRO := $(DISTRO_NAME)_$(DISTRO_RELEASE)_$(DISTRO_CODE) hello: @echo "$(VERSION)" @@ -120,7 +121,7 @@ doc/GOOD_PRACTICES.html: doc/GOOD_PRACTICES.t2t txt2tags -i doc/GOOD_PRACTICES.t2t -t html --toc -o doc/GOOD_PRACTICES.html ./W/tools/validate_html4 doc/GOOD_PRACTICES.html ./W/tools/validate doc/GOOD_PRACTICES.html - + doc/TUTORIAL_Unix.html: doc/TUTORIAL_Unix.t2t txt2tags -i doc/TUTORIAL_Unix.t2t -t html --toc -o doc/TUTORIAL_Unix.html @@ -157,10 +158,10 @@ clean_bak: .PHONY: install dist man -man: imapsync.1 +man: W/imapsync.1 clean_man: - rm -f imapsync.1 + rm -f W/imapsync.1 W/imapsync.1: imapsync pod2man imapsync > W/imapsync.1 @@ -180,15 +181,21 @@ install: testp W/imapsync.1 # Local goals ############### -.PHONY: prereq test tests unitests testp testf test3xx testv3 perlcritic allcritic crit compok dev cover tidy +.PHONY: prereq test tests unitests testp testf test3xx testv3 perlcritic allcritic crit compok dev cover tidy nytprof -dev: test crit bin +dev: test crit cover nytprof bin docker: ssh ks3 'cd docker/imapsync && . memo' +nytprof: + sh tests.sh ll_nytprof + nytprofhtml + cover: - perl -MDevel::Cover ./imapsync --tests + perl -c ./imapsync + perl -MDevel::Cover ./imapsync --tests --testslive + cover tidy: W/imapsync.tdy @@ -418,7 +425,24 @@ exe: imapsync scp Admin@c:'C:/msys/1.0/home/Admin/imapsync/imapsync.exe' . (date "+%s"| tr "\n" " "; echo -n "END " $(VERSION) ": "; date) >> W/.BUILD_EXE_TIME +zzz: + unix2dos W/build_exe.bat W/install_module_one.bat + scp imapsync W/build_exe.bat W/install_module_one.bat pc_HP_DV7_p24:'Desktop/imapsync_build' +# ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat' +# ./W/check_win64err build_exe.bat +imapsync_64bit.exe: imapsync + (date "+%s"| tr "\n" " "; echo -n "BEGIN 64bit " $(VERSION) ": "; date) >> W/.BUILD_EXE_TIME + ssh 'pc HP DV7'@p24 'perl -V' + unix2dos W/build_exe.bat W/test_exe.bat W/install_modules.bat W/install_module_one.bat + scp imapsync W/build_exe.bat W/install_modules.bat W/install_module_one.bat \ + W/test_exe_tests.bat W/test_exe.bat \ + pc_HP_DV7_p24:'Desktop/imapsync_build' + ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat' + ./W/check_win64err build_exe.bat + rm -f imapsync_64bit.exe + scp pc_HP_DV7_p24:'Desktop/imapsync_build/imapsync.exe' imapsync_64bit.exe + (date "+%s"| tr "\n" " "; echo -n "END 64bit " $(VERSION) ": "; date) >> W/.BUILD_EXE_TIME zip: dosify_bat @@ -428,45 +452,46 @@ zip: dosify_bat cp -av W/build_exe.bat W/install_modules.bat W/test_cook_exe.bat W/test_cook_src.bat imapsync ../prepa_zip/imapsync_$(VERSION_EXE)/Cook/ for f in README ; do cp -av $$f ../prepa_zip/imapsync_$(VERSION_EXE)/$$f.txt ; done cp -av FAQ.d/*.txt ../prepa_zip/imapsync_$(VERSION_EXE)/FAQ.d/ - cp -av imapsync.exe README_Windows.txt ../prepa_zip/imapsync_$(VERSION_EXE)/ + cp -av imapsync.exe imapsync_64bit.exe README_Windows.txt ../prepa_zip/imapsync_$(VERSION_EXE)/ unix2dos ../prepa_zip/imapsync_$(VERSION_EXE)/*.txt cd ../prepa_zip/ && rm -f ./imapsync_$(VERSION_EXE).zip && zip -r ./imapsync_$(VERSION_EXE).zip ./imapsync_$(VERSION_EXE)/ scp ../prepa_zip/imapsync_$(VERSION_EXE).zip Admin@c:'C:/msys/1.0/home/Admin/' + scp ../prepa_zip/imapsync_$(VERSION_EXE).zip pc_HP_DV7_p24:'Desktop/' cp ../prepa_zip/imapsync_$(VERSION_EXE).zip /fe/imapsync/ # C:\Users\mansour\Desktop\imapsync -.PHONY: mac bin win lin +.PHONY: mac bin win lin win64 mac: imapsync_bin_Darwin imapsync_bin_Darwin: imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync rcsdiff imapsync - rsync -p -e 'ssh -p 995' imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync gilleslamira@gate.polarhome.com: - ssh -p 995 gilleslamira@gate.polarhome.com 'sh build_mac.sh' - rsync -P -e 'ssh -p 995' gilleslamira@gate.polarhome.com:imapsync_bin_Darwin . + rsync -p -e 'ssh -4 -p 995' imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync gilleslamira@gate.polarhome.com: + ssh -4 -p 995 gilleslamira@gate.polarhome.com 'sh -x build_mac.sh' + rsync -P -e 'ssh -4 -p 995' gilleslamira@gate.polarhome.com:imapsync_bin_Darwin . mactests: - rsync -p -e 'ssh -p 995' imapsync gilleslamira@gate.polarhome.com: - ssh -p 995 gilleslamira@gate.polarhome.com '. .bash_profile; perl imapsync --tests' + rsync -p -e 'ssh -4 -p 995' imapsync gilleslamira@gate.polarhome.com: + ssh -4 -p 995 gilleslamira@gate.polarhome.com '. .bash_profile; perl imapsync --tests' mactestsdebug: - rsync -p -e 'ssh -p 995' imapsync gilleslamira@gate.polarhome.com: - ssh -p 995 gilleslamira@gate.polarhome.com '. .bash_profile; perl imapsync --testsdebug --debug' + rsync -p -e 'ssh -4 -p 995' imapsync gilleslamira@gate.polarhome.com: + ssh -4 -p 995 gilleslamira@gate.polarhome.com '. .bash_profile; perl imapsync --testsdebug --debug' mactestslive: - rsync -p -e 'ssh -p 995' imapsync gilleslamira@gate.polarhome.com: - ssh -p 995 gilleslamira@gate.polarhome.com '. .bash_profile; perl imapsync --testslive' + rsync -p -e 'ssh -4 -p 995' imapsync gilleslamira@gate.polarhome.com: + ssh -4 -p 995 gilleslamira@gate.polarhome.com '. .bash_profile; perl imapsync --testslive' -bin: lin mac win +bin: lin mac win win64 lin: $(BIN_NAME) win: imapsync.exe - +win64: imapsync_64bit.exe $(BIN_NAME): imapsync rcsdiff imapsync @@ -481,7 +506,7 @@ $(BIN_NAME): imapsync ./$(BIN_NAME) --tests ./$(BIN_NAME) --testslive ./$(BIN_NAME) --justbanner - + lfo: upload_lfo @@ -492,7 +517,7 @@ tarball: cidone echo making tarball ../prepa_dist/$(DIST_FILE) mkdir -p dist mkdir -p ../prepa_dist/$(DIST_NAME) - rsync -aCvH --delete --omit-dir-times --exclude $(DIST_PATH) --exclude imapsync.exe ./ ../prepa_dist/$(DIST_NAME)/ + rsync -aCvH --delete --delete-excluded --omit-dir-times --exclude $(DIST_PATH) --exclude-from=W/rsync_exclude_dist.txt ./ ../prepa_dist/$(DIST_NAME)/ cd ../prepa_dist && tar czfv $(DIST_FILE) $(DIST_NAME) cd ../prepa_dist && md5sum $(DIST_FILE) > $(DIST_FILE).md5.txt cd ../prepa_dist && md5sum -c $(DIST_FILE).md5.txt @@ -545,7 +570,7 @@ README_dist: sh W/tools/gen_README_dist > $(DIST_PATH)/README.txt unix2dos $(DIST_PATH)/README.txt -.PHONY: publish upload_ks ks valid_index biz ks2tests ks2tests_gilles ks2tests_root +.PHONY: publish upload_ks ks valid_index biz ks2tests_gilles ks2tests_root biz: S/imapsync_sold_by_country.txt @@ -588,6 +613,10 @@ ks2tests_root: rsync -aP imapsync root@ks.lamiral.info: ssh root@ks.lamiral.info './imapsync --tests' +i005tests_root: + rsync -P imapsync root@i005.lamiral.info: + ssh root@i005.lamiral.info './imapsync --tests' + ks2testslive: rsync -aP imapsync gilles@ks.lamiral.info:public_html/imapsync/imapsync ssh gilles@ks.lamiral.info 'public_html/imapsync/imapsync --testslive' @@ -608,12 +637,9 @@ centos: PUBLIC = ./ChangeLog ./NOLIMIT ./LICENSE ./CREDITS ./FAQ \ ./index.shtml ./INSTALL ./README_Windows.txt \ ./VERSION ./VERSION_EXE ./imapsync \ -./README ./TODO +./README ./TODO ./vnstat - -PUBLIC_doc = ./doc/TUTORIAL_Unix.html ./doc/GOOD_PRACTICES.html - ml: rcsdiff W/ml_announce.in m4 -P W/ml_announce.in | mutt -H- @@ -635,12 +661,12 @@ valid_index: W/.valid.index.shtml cl: ./W/checklink.txt ./W/checklink.txt: index.shtml - checklink -b -q http://lamiral.info/~gilles/imapsync/ |tee ./W/checklink.txt + checklink --broken --quiet http://lamiral.info/~gilles/imapsync/ |tee ./W/checklink.txt cle: ./W/checklinkext.txt ./W/checklinkext.txt: S/news.shtml S/external.shtml S/imapservers.shtml S/template_xhtml1.shtml - checklink -b -q \ + checklink --broken --quiet \ http://lamiral.info/~gilles/imapsync/S/template_xhtml1.shtml \ http://lamiral.info/~gilles/imapsync/S/news.shtml \ http://lamiral.info/~gilles/imapsync/S/external.shtml \ @@ -655,11 +681,12 @@ W/.valid.index.shtml: index.shtml .PHONY: upload_index upload_FAQ ci_imapsync upload_bin upload_bin: - rsync -a imapsync.exe imapsync_bin_Darwin imapsync_bin_Linux_i686 imapsync ../imapsync_website/ + rsync -a imapsync.exe imapsync_64bit.exe imapsync_bin_Darwin imapsync_bin_Linux_i686 imapsync ../imapsync_website/ rsync -aHvz --delete ../imapsync_website/ root@ks.lamiral.info:/var/www/imapsync/ upload_index: W/.valid.index.shtml clean_permissions - rcsdiff index.shtml README_Windows.txt S/style.css S/*.shtml FAQ.d/*.txt LICENSE CREDITS TODO examples/*.bat index.shtml INSTALL.d/*.txt + rcsdiff index.shtml README_Windows.txt S/style.css S/*.shtml FAQ.d/*.txt LICENSE CREDITS TODO examples/*.bat examples/*.sh index.shtml INSTALL.d/*.txt + rm -f examples/LOG_imapsync/* rsync -avH index.shtml README_Windows.txt FAQ INSTALL NOLIMIT LICENSE CREDITS TODO S/robots.txt S/favicon.ico ../imapsync_website/ rsync -aHv --delete ./W/ks.htaccess ../imapsync_website/.htaccess rsync -aHv --delete S/ ../imapsync_website/S/ @@ -678,9 +705,15 @@ upload_latest: unitests ci_imapsync bin rsync -aHvz --delete ../imapsync_website/ root@ks.lamiral.info:/var/www/imapsync/ ssh root@ks.lamiral.info 'apachectl configtest && /etc/init.d/apache2 reload' -upload_cgi: unitests ks2tests ci_imapsync +upload_cgi: unitests ks2tests i005tests_root ci_imapsync rsync -a imapsync ../imapsync_website/ rsync -aHvz --delete ../imapsync_website/ root@ks.lamiral.info:/var/www/imapsync/ + rsync -P imapsync root@i005.lamiral.info:/home/www/apache24/cgi-bin/imapsync + +upload_cgi_memo: + rsync -av X/cgi_memo X/stat_patterns.txt root@ks2:/var/tmp/imapsync_cgi/ + rsync -av X/cgi_memo X/stat_patterns.txt root@i005:/var/tmp/imapsync_cgi/ + upload_X: ./W/tools/validate_xml_html5 X/imapsync_form.html @@ -688,6 +721,7 @@ upload_X: rcsdiff X/imapsync_form.js rsync -av --delete X/ ../imapsync_website/X/ rsync -aHvz --delete ../imapsync_website/ root@ks.lamiral.info:/var/www/imapsync/ + rsync -av --delete X/ root@i005:/home/www/apache24/data/X/ upload_FAQ: rcsdiff FAQ.d/*.txt LICENSE CREDITS TODO INSTALL.d/*.txt @@ -698,6 +732,9 @@ upload_FAQ: rsync -aHvz --delete ../imapsync_website/ root@ks.lamiral.info:/var/www/imapsync/ +upload_ks_W_memo: + rsync -av W/memo gilles@ks.lamiral.info:public_html/imapsync/W/memo + upload_ks: ci tarball rsync -aHv $(PUBLIC) ../imapsync_website/ rsync -aHv --delete ./W/ks.htaccess ../imapsync_website/.htaccess @@ -708,4 +745,6 @@ upload_ks: ci tarball rsync -aHv --delete ./FAQ.d/ ../imapsync_website/FAQ.d/ rsync -avH --delete ./doc/ ../imapsync_website/doc/ rsync -aHvz --delete ../imapsync_website/ root@ks.lamiral.info:/var/www/imapsync/ - ssh root@ks.lamiral.info 'apachectl configtest && /etc/init.d/apache2 reload' \ No newline at end of file + ssh root@ks.lamiral.info 'apachectl configtest && /etc/init.d/apache2 reload' + + diff --git a/README b/README index 4085053..fda3ab2 100644 --- a/README +++ b/README @@ -5,7 +5,7 @@ NAME VERSION - This documentation refers to Imapsync $Revision: 1.882 $ + This documentation refers to Imapsync $Revision: 1.920 $ USAGE @@ -84,19 +84,19 @@ OPTIONS reg means regular expression cmd means command - --dry : Makes imapsync doing nothing for real, just print what + --dry : Makes imapsync doing nothing for real, just print what would be done without --dry. OPTIONS/credentials --host1 str : Source or "from" imap server. Mandatory. - --port1 int : Port to connect on host1. + --port1 int : Port to connect on host1. Optional since default port is 143 or 993 if --ssl1 --user1 str : User to login on host1. Mandatory. --password1 str : Password for the user1. --host2 str : "destination" imap server. Mandatory. - --port2 int : Port to connect on host2. + --port2 int : Port to connect on host2. Optional since default port is 143 or 993 if --ssl2 --user2 str : User to login on host2. Mandatory. --password2 str : Password for the user2. @@ -110,6 +110,9 @@ OPTIONS the password on the command line like --password1 does. --passfile2 str : Password file for the user2. Contains the password. + You can also pass the passwords in the environment variables + IMAPSYNC_PASSWORD1 and IMAPSYNC_PASSWORD2 + OPTIONS/encryption --nossl1 : Do not use a SSL connection on host1. @@ -191,10 +194,11 @@ OPTIONS --f1f2 str1=str2 : Force folder str1 to be synced to str2, --f1f2 overrides --automap and --regextrans2. - --subfolder2 str : Move whole host1 folders hierarchy under this - host2 folder str . - It does it by adding two --regextrans2 options before - all others. Add --debug to see what's really going on. + --subfolder2 str : Syncs the whole host1 folders hierarchy under the + host2 folder named str. + (It does it internally by adding two + --regextrans2 options before all others. + Add --debug to see what's really going on.) --subscribed : Transfers subscribed folders. --subscribe : Subscribe to the folders transferred on the @@ -224,7 +228,7 @@ OPTIONS --nofoldersizes : Do not calculate the size of each folder at the beginning of the sync. Default is to calculate them. - --nofoldersizesatend: Do not calculate the size of each folder at the + --nofoldersizesatend: Do not calculate the size of each folder at the end of the sync. Default is to calculate them. --justfoldersizes : Exit after having printed the initial folder sizes. @@ -273,7 +277,7 @@ OPTIONS --resyncflags : Resync flags for already transferred messages. On by default. --noresyncflags : Do not resync flags for already transferred messages. - May be useful when a user has already started to play + May be useful when a user has already started to play with its host2 account. OPTIONS/deletions @@ -342,7 +346,7 @@ OPTIONS command. Applied on both sides. For a complete of what can be search see https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt - + --search1 str : Same as --search but for selecting host1 messages only. --search2 str : Same as --search but for selecting host2 messages only. --search CRIT equals --search1 CRIT --search2 CRIT @@ -367,6 +371,10 @@ OPTIONS --syncacls : Synchronizes acls (Access Control Lists). --nosyncacls : Does not synchronize acls. This is the default. Acls in IMAP are not standardized, be careful. + --addheader : When a message has no headers to be identified, + --addheader adds a "Message-Id" header, + like "Message-Id: 12345@imapsync", where 12345 + is the imap UID of the message on the host1 folder. OPTIONS/debugging @@ -391,28 +399,28 @@ OPTIONS --gmail1 : sets --host1 to Gmail and options from FAQ.Gmail.txt --gmail2 : sets --host2 to Gmail and options from FAQ.Gmail.txt - + --office1 : sets --host1 to Office365 options from FAQ.Exchange.txt --office2 : sets --host2 to Office365 options from FAQ.Exchange.txt --exchange1 : sets options from FAQ.Exchange.txt, account1 part --exchange2 : sets options from FAQ.Exchange.txt, account2 part - + --domino1 : sets options from FAQ.Domino.txt, account1 part --domino2 : sets options from FAQ.Domino.txt, account2 part OPTIONS/behavior --maxmessagespersecond int : limits the number of messages transferred per second. - + --maxbytespersecond int : limits the average transfer rate per second. - --maxbytesafter int : starts --maxbytespersecond limitation only after + --maxbytesafter int : starts --maxbytespersecond limitation only after --maxbytesafter amount of data transferred. - + --maxsleep int : do not sleep more than int seconds. On by default, 2 seconds max, like --maxsleep 2 - --abort : terminates a previous call still running. + --abort : terminates a previous call still running. It uses the pidfile to know what process to abort. --exitwhenover int : Stop syncing when total bytes transferred reached. @@ -542,13 +550,13 @@ HUGE MIGRATION INSTALL Imapsync works under any Unix with perl. - + Imapsync works under most Windows (2000, XP, Vista, Seven, Eight, Ten and all Server releases 2000, 2003, 2008 and R2, 2012 and R2) as a standalone binary software called imapsync.exe, - usually launched from a batch file in order to avoid always typing + usually launched from a batch file in order to avoid always typing the options. - + Imapsync works under OS X as a standalone binary software called imapsync_bin_Darwin @@ -584,7 +592,7 @@ SIMILAR SOFTWARE imap_tools : https://github.com/andrewnimmo/rick-sanders-imap-tools offlineimap : https://github.com/nicolas33/offlineimap - Doveadm-Sync : http://wiki2.dovecot.org/Tools/Doveadm/Sync + Doveadm-Sync : http://wiki2.dovecot.org/Tools/Doveadm/Sync ( Dovecot sync tool ) mbsync : http://isync.sourceforge.net/ mailsync : http://mailsync.sourceforge.net/ diff --git a/README.md b/README.md deleted file mode 100644 index d1f2a56..0000000 --- a/README.md +++ /dev/null @@ -1,636 +0,0 @@ - -# Imapsync payment - -Help the imapsync author, Gilles, to maintain free and gratis imapsync and its online services; -My business is that I earn my living from imapsync buyers and from the word of mouth of others - -In case you're using imapsync in a professional context, -then consider buying imapsync at https://imapsync.lamiral.info/#buy_all, -you will get also a regular invoice for your company. - -If 60 EUR is too much, then consider a smaller donation: - -[![paypal](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TUENPW59U9LL2) - -Thanks in any case, even no money, I'm ok with that! - -```` -NAME - - imapsync - Email IMAP tool for syncing, copying and migrating email - mailboxes between two imap servers, one way, and without duplicates. - -VERSION - - This documentation refers to Imapsync $Revision: 1.882 $ - -USAGE - - To synchronize the source imap account - "test1" on server "test1.lamiral.info" with password "secret1" - to the destination imap account - "test2" on server "test2.lamiral.info" with password "secret2" - do: - - imapsync \ - --host1 test1.lamiral.info --user1 test1 --password1 secret1 \ - --host2 test2.lamiral.info --user2 test2 --password2 secret2 - -DESCRIPTION - - We sometimes need to transfer mailboxes from one imap server to one - another. - - Imapsync command is a tool allowing incremental and recursive imap - transfers from one mailbox to another. - - By default all folders are transferred, recursively, meaning the whole - folder hierarchy is taken, all messages in them, and all messages flags - (\Seen \Answered \Flagged etc.) are synced too. - - Imapsync reduces the amount of data transferred by not transferring a - given message if it resides already on both sides. - - Same specific headers and the transfer is done only once. By default, - the identification headers are "Message-Id:" and "Received:" lines but - this choice can be changed with the --useheader option. - - All flags are preserved, unread messages will stay unread, read ones - will stay read, deleted will stay deleted. - - You can stop the transfer at any time and restart it later, imapsync - works well with bad connections and interruptions, by design. - - You can decide to delete the messages from the source mailbox after a - successful transfer, it can be a good feature when migrating live - mailboxes since messages will be only on one side. - - In that case, use the --delete1 option. Option --delete1 implies also - option --expunge1 so all messages marked deleted on host1 will be really - deleted. - - You can also decide to remove empty folders once all of their messages - have been transferred. Add --delete1emptyfolders to obtain this - behavior. - - A different scenario is synchronizing a mailbox B from another mailbox A - in case you just want to keep a "live" copy of A in B. - - For this, option --delete2 has to be used, it deletes messages in host2 - folder B that are not in host1 folder A. If you also need to destroy - host2 folders that are not in host1 then use --delete2folders. See also - --delete2foldersonly and --delete2foldersbutnot. - - Imapsync is not adequate for maintaining two active imap accounts in - synchronization when the user plays independently on both sides. Use - offlineimap (written by John Goerzen) or mbsync (written by Michael R. - Elkins) for a 2 ways synchronization. - -OPTIONS - - usage: imapsync [options] - - Mandatory options are the six values, three on each sides, needed to log - in into the IMAP servers, ie, a host, a username, and a password, two - times. - - Conventions used: - - str means string - int means integer - reg means regular expression - cmd means command - - --dry : Makes imapsync doing nothing for real, just print what - would be done without --dry. - - OPTIONS/credentials - - --host1 str : Source or "from" imap server. Mandatory. - --port1 int : Port to connect on host1. - Optional since default port is 143 or 993 if --ssl1 - --user1 str : User to login on host1. Mandatory. - --password1 str : Password for the user1. - - --host2 str : "destination" imap server. Mandatory. - --port2 int : Port to connect on host2. - Optional since default port is 143 or 993 if --ssl2 - --user2 str : User to login on host2. Mandatory. - --password2 str : Password for the user2. - - --showpasswords : Shows passwords on output instead of "MASKED". - Useful to restart a complete run by just reading the log, - or to debug passwords. It's not a secure practice. - - --passfile1 str : Password file for the user1. It must contain the - password on the first line. This option avoids to show - the password on the command line like --password1 does. - --passfile2 str : Password file for the user2. Contains the password. - - OPTIONS/encryption - - --nossl1 : Do not use a SSL connection on host1. - --ssl1 : Use a SSL connection on host1. On by default if possible. - - --nossl2 : Do not use a SSL connection on host2. - --ssl2 : Use a SSL connection on host2. On by default if possible. - - --notls1 : Do not use a TLS connection on host1. - --tls1 : Use a TLS connection on host1. On by default if possible. - - --notls2 : Do not use a TLS connection on host2. - --tls2 : Use a TLS connection on host2. On by default if possible. - - --debugssl int : SSL debug mode from 0 to 4. - - --sslargs1 str : Pass any ssl parameter for host1 ssl or tls connection. Example: - --sslargs1 SSL_verify_mode=1 --sslargs1 SSL_version=SSLv3 - See all possibilities in the new() method of IO::Socket::SSL - http://search.cpan.org/perldoc?IO::Socket::SSL#Description_Of_Methods - --sslargs2 str : Pass any ssl parameter for host2 ssl or tls connection. - See --sslargs1 - - --timeout1 int : Connection timeout in seconds for host1. - Default is 120 and 0 means no timeout at all. - --timeout2 int : Connection timeout in seconds for host2. - Default is 120 and 0 means no timeout at all. - - OPTIONS/authentication - - --authmech1 str : Auth mechanism to use with host1: - PLAIN, LOGIN, CRAM-MD5 etc. Use UPPERCASE. - --authmech2 str : Auth mechanism to use with host2. See --authmech1 - - --authuser1 str : User to auth with on host1 (admin user). - Avoid using --authmech1 SOMETHING with --authuser1. - --authuser2 str : User to auth with on host2 (admin user). - --proxyauth1 : Use proxyauth on host1. Requires --authuser1. - Required by Sun/iPlanet/Netscape IMAP servers to - be able to use an administrative user. - --proxyauth2 : Use proxyauth on host2. Requires --authuser2. - - --authmd51 : Use MD5 authentication for host1. - --authmd52 : Use MD5 authentication for host2. - --domain1 str : Domain on host1 (NTLM authentication). - --domain2 str : Domain on host2 (NTLM authentication). - - OPTIONS/folders - - --folder str : Sync this folder. - --folder str : and this one, etc. - --folderrec str : Sync this folder recursively. - --folderrec str : and this one, etc. - - --folderfirst str : Sync this folder first. --folderfirst "Work" - --folderfirst str : then this one, etc. - --folderlast str : Sync this folder last. --folderlast "[Gmail]/All Mail" - --folderlast str : then this one, etc. - - --nomixfolders : Do not merge folders when host1 is case-sensitive - while host2 is not (like Exchange). Only the first - similar folder is synced (ex: with Sent SENT sent - on host1 only Sent will be synced to host2). - - --skipemptyfolders : Empty host1 folders are not created on host2. - - --include reg : Sync folders matching this regular expression - --include reg : or this one, etc. - If both --include --exclude options are used, then - include is done before. - --exclude reg : Skips folders matching this regular expression - Several folders to avoid: - --exclude 'fold1|fold2|f3' skips fold1, fold2 and f3. - --exclude reg : or this one, etc. - - --automap : guesses folders mapping, for folders well known as - "Sent", "Junk", "Drafts", "All", "Archive", "Flagged". - - --f1f2 str1=str2 : Force folder str1 to be synced to str2, - --f1f2 overrides --automap and --regextrans2. - - --subfolder2 str : Move whole host1 folders hierarchy under this - host2 folder str . - It does it by adding two --regextrans2 options before - all others. Add --debug to see what's really going on. - - --subscribed : Transfers subscribed folders. - --subscribe : Subscribe to the folders transferred on the - host2 that are subscribed on host1. On by default. - --subscribeall : Subscribe to the folders transferred on the - host2 even if they are not subscribed on host1. - - --prefix1 str : Remove prefix str to all destination folders, - usually INBOX. or INBOX/ or an empty string "". - imapsync guesses the prefix if host1 imap server - does not have NAMESPACE capability. This option - should not be used, most of the time. - --prefix2 str : Add prefix to all host2 folders. See --prefix1 - --sep1 str : Host1 separator in case NAMESPACE is not supported. - --sep2 str : Host2 separator in case NAMESPACE is not supported. - - --regextrans2 reg : Apply the whole regex to each destination folders. - --regextrans2 reg : and this one. etc. - When you play with the --regextrans2 option, first - add also the safe options --dry --justfolders - Then, when happy, remove --dry, remove --justfolders. - Have in mind that --regextrans2 is applied after prefix - and separator inversion. For examples see - https://imapsync.lamiral.info/FAQ.d/FAQ.Folders_Mapping.txt - - OPTIONS/folders sizes - - --nofoldersizes : Do not calculate the size of each folder at the - beginning of the sync. Default is to calculate them. - --nofoldersizesatend: Do not calculate the size of each folder at the - end of the sync. Default is to calculate them. - --justfoldersizes : Exit after having printed the initial folder sizes. - - OPTIONS/tmp - - --tmpdir str : Where to store temporary files and subdirectories. - Will be created if it doesn't exist. - Default is system specific, Unix is /tmp but - /tmp is often too small and deleted at reboot. - --tmpdir /var/tmp should be better. - --pidfile str : The file where imapsync pid is written, - it can be dirname/filename. - Default name is imapsync.pid in tmpdir. - --pidfilelocking : Abort if pidfile already exists. Useful to avoid - concurrent transfers on the same mailbox. - - OPTIONS/log - - --nolog : Turn off logging on file - --logfile str : Change the default log filename (can be dirname/filename). - --logdir str : Change the default log directory. Default is LOG_imapsync/ - - OPTIONS/messages - - --skipmess reg : Skips messages matching the regex. - Example: 'm/[\x80-ff]/' # to avoid 8bits messages. - --skipmess is applied before --regexmess - --skipmess reg : or this one, etc. - - --pipemess cmd : Apply this cmd command to each message content - before the copy. - --pipemess cmd : and this one, etc. - - --disarmreadreceipts : Disarms read receipts (host2 Exchange issue) - - --regexmess reg : Apply the whole regex to each message before transfer. - Example: 's/\000/ /g' # to replace null by space. - --regexmess reg : and this one, etc. - - OPTIONS/flags - - --regexflag reg : Apply the whole regex to each flags list. - Example: 's/"Junk"//g' # to remove "Junk" flag. - --regexflag reg : then this one, etc. - - --resyncflags : Resync flags for already transferred messages. - On by default. - --noresyncflags : Do not resync flags for already transferred messages. - May be useful when a user has already started to play - with its host2 account. - - OPTIONS/deletions - - --delete1 : Deletes messages on host1 server after a successful - transfer. Option --delete1 has the following behavior: - it marks messages as deleted with the IMAP flag - \Deleted, then messages are really deleted with an - EXPUNGE IMAP command. If expunging after each message - slows down too much the sync then use - --noexpungeaftereach to speed up. - --expunge1 : Expunge messages on host1 just before syncing a folder. - Expunge is done per folder. - Expunge aims is to really delete messages marked deleted. - An expunge is also done after each message copied - if option --delete1 is set. - --noexpunge1 : Do not expunge messages on host1. - --delete1emptyfolders : Deletes empty folders on host1, INBOX excepted. - Useful with --delete1 since what remains on host1 - is only what failed to be synced. - - --delete2 : Delete messages in host2 that are not in - host1 server. Useful for backup or pre-sync. - --delete2duplicates : Delete messages in host2 that are duplicates. - Works only without --useuid since duplicates are - detected with an header part of each message. - - --delete2folders : Delete folders in host2 that are not in host1 server. - For safety, first try it like this (it is safe): - --delete2folders --dry --justfolders --nofoldersizes - --delete2foldersonly reg : Deleted only folders matching regex. - Example: --delete2foldersonly "/^Junk$|^INBOX.Junk$/" - --delete2foldersbutnot reg : Do not delete folders matching regex. - Example: --delete2foldersbutnot "/Tasks$|Contacts$|Foo$/" - - --expunge2 : Expunge messages on host2 after messages transfer. - --uidexpunge2 : uidexpunge messages on the host2 account - that are not on the host1 account, requires --delete2 - - OPTIONS/dates - - --syncinternaldates : Sets the internal dates on host2 same as host1. - Turned on by default. Internal date is the date - a message arrived on a host (mtime). - --idatefromheader : Sets the internal dates on host2 same as the - "Date:" headers. - If you encounter problems with dates see also - https://imapsync.lamiral.info/FAQ.d/FAQ.Dates.txt - - OPTIONS/message selection - - --maxsize int : Skip messages larger (or equal) than int bytes - --minsize int : Skip messages smaller (or equal) than int bytes - --maxage int : Skip messages older than int days. - final stats (skipped) don't count older messages - see also --minage - --minage int : Skip messages newer than int days. - final stats (skipped) don't count newer messages - You can do (+ are the messages selected): - past|----maxage+++++++++++++++>now - past|+++++++++++++++minage---->now - past|----maxage+++++minage---->now (intersection) - past|++++minage-----maxage++++>now (union) - - --search str : Selects only messages returned by this IMAP SEARCH - command. Applied on both sides. - For a complete of what can be search see - https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt - - --search1 str : Same as --search but for selecting host1 messages only. - --search2 str : Same as --search but for selecting host2 messages only. - --search CRIT equals --search1 CRIT --search2 CRIT - - --maxlinelength int : skip messages with a line length longer than int bytes. - RFC 2822 says it must be no more than 1000 bytes. - - - --useheader str : Use this header to compare messages on both sides. - Ex: Message-ID or Subject or Date. - --useheader str and this one, etc. - - --usecache : Use cache to speed up the sync. - --nousecache : Do not use cache. Caveat: --useuid --nousecache creates - duplicates on multiple runs. - --useuid : Use UIDs instead of headers as a criterium to recognize - messages. Option --usecache is then implied unless - --nousecache is used. - - OPTIONS/miscellaneous - - --syncacls : Synchronizes acls (Access Control Lists). - --nosyncacls : Does not synchronize acls. This is the default. - Acls in IMAP are not standardized, be careful. - - OPTIONS/debugging - - --debug : Debug mode. - --debugfolders : Debug mode for the folders part only. - --debugcontent : Debug content of the messages transferred. Huge output. - --debugflags : Debug mode for flags. - --debugimap1 : IMAP debug mode for host1. Very verbose. - --debugimap2 : IMAP debug mode for host2. Very verbose. - --debugimap : IMAP debug mode for host1 and host2. Twice very verbose. - --debugmemory : Debug mode showing memory consumption after each copy. - - --errorsmax int : Exit when int number of errors is reached. Default is 50. - - --tests : Run local non-regression tests. Exit code 0 means all ok. - --testslive : Run a live test with test1.lamiral.info imap server. - Useful to check the basics. Needs internet connection. - --testslive6 : Run a live test with ks2ipv6.lamiral.info imap server. - Useful to check the ipv6 connectivity. Needs internet. - - OPTIONS/specific - - --gmail1 : sets --host1 to Gmail and options from FAQ.Gmail.txt - --gmail2 : sets --host2 to Gmail and options from FAQ.Gmail.txt - - --office1 : sets --host1 to Office365 options from FAQ.Exchange.txt - --office2 : sets --host2 to Office365 options from FAQ.Exchange.txt - - --exchange1 : sets options from FAQ.Exchange.txt, account1 part - --exchange2 : sets options from FAQ.Exchange.txt, account2 part - - --domino1 : sets options from FAQ.Domino.txt, account1 part - --domino2 : sets options from FAQ.Domino.txt, account2 part - - OPTIONS/behavior - - --maxmessagespersecond int : limits the number of messages transferred per second. - - --maxbytespersecond int : limits the average transfer rate per second. - --maxbytesafter int : starts --maxbytespersecond limitation only after - --maxbytesafter amount of data transferred. - - --maxsleep int : do not sleep more than int seconds. - On by default, 2 seconds max, like --maxsleep 2 - - --abort : terminates a previous call still running. - It uses the pidfile to know what process to abort. - - --exitwhenover int : Stop syncing when total bytes transferred reached. - - --version : Print only software version. - --noreleasecheck : Do not check for new imapsync release (a http request). - --releasecheck : Check for new imapsync release (a http request). - --noid : Do not send/receive ID command to imap servers. - --justconnect : Just connect to both servers and print useful - information. Need only --host1 and --host2 options. - --justlogin : Just login to both host1 and host2 with users - credentials, then exit. - --justfolders : Do only things about folders (ignore messages). - - --help : print this help. - - Example: to synchronize imap account "test1" on "test1.lamiral.info" - to imap account "test2" on "test2.lamiral.info" - with test1 password "secret1" - and test2 password "secret2" - - imapsync \ - --host1 test1.lamiral.info --user1 test1 --password1 secret1 \ - --host2 test2.lamiral.info --user2 test2 --password2 secret2 - -SECURITY - - You can use --passfile1 instead of --password1 to give the password - since it is safer. With --password1 option, any user on your host can - see the password by using the 'ps auxwwww' command. Using a variable - (like $PASSWORD1) is also dangerous because of the 'ps auxwwwwe' - command. So, saving the password in a well protected file (600 or - rw-------) is the best solution. - - Imapsync activates ssl or tls encryption by default, if possible. What - detailed behavior is under this "if possible"? Imapsync activates ssl if - the well known port imaps port (993) is open on the imap servers. If the - imaps port is closed then it open a normal (clear) connection on port - 143 but it looks for TLS support in the CAPABILITY list of the servers. - If TLS is supported then imapsync goes to encryption. - - If the automatic ssl/tls detection fails then imapsync will not protect - against sniffing activities on the network, especially for passwords. - - If you want to force ssl or tls just use --ssl1 --ssl2 or --tls1 --tls2 - - See also the document FAQ.Security.txt in the FAQ.d/ directory or at - https://imapsync.lamiral.info/FAQ.d/FAQ.Security.txt - -EXIT STATUS - - Imapsync will exit with a 0 status (return code) if everything went - good. Otherwise, it exits with a non-zero status. - -LICENSE AND COPYRIGHT - - Imapsync is free, open, public but not always gratis software cover by - the NOLIMIT Public License. See the LICENSE file included in the - distribution or just read this simple sentence as it IS the licence - text: - - "No limits to do anything with this work and this license." - - In case it is not long enough, I repeat: - - "No limits to do anything with this work and this license." - - https://imapsync.lamiral.info/LICENSE - -AUTHOR - - Gilles LAMIRAL - - Feedback good or bad is very often welcome. - - Gilles LAMIRAL earns his living by writing, installing, configuring and - teaching free, open and often gratis software. Imapsync used to be - "always gratis" but now it is only "often gratis" because imapsync is - sold by its author, a good way to maintain and support free open public - software over decades. - -BUGS AND LIMITATIONS - - See https://imapsync.lamiral.info/FAQ.d/FAQ.Reporting_Bugs.txt - -IMAP SERVERS supported - - See https://imapsync.lamiral.info/S/imapservers.shtml - -HUGE MIGRATION - - Pay special attention to options --subscribed --subscribe --delete1 - --delete1emptyfolders --delete2 --delete2folders --maxage --minage - --maxsize --useuid --usecache - - If you have many mailboxes to migrate think about a little shell - program. Write a file called file.txt (for example) containing users and - passwords. The separator used in this example is ';' - - The file.txt file contains: - - user001_1;password001_1;user001_2;password001_2 - user002_1;password002_1;user002_2;password002_2 - user003_1;password003_1;user003_2;password003_2 - user004_1;password004_1;user004_2;password004_2 - user005_1;password005_1;user005_2;password005_2 ... - - On Unix the shell program can be: - - { while IFS=';' read u1 p1 u2 p2; do - imapsync --host1 imap.side1.org --user1 "$u1" --password1 "$p1" \ - --host2 imap.side2.org --user2 "$u2" --password2 "$p2" ... - done ; } < file.txt - - On Windows the batch program can be: - - FOR /F "tokens=1,2,3,4 delims=; eol=#" %%G IN (file.txt) DO imapsync ^ - --host1 imap.side1.org --user1 %%G --password1 %%H ^ - --host2 imap.side2.org --user2 %%I --password2 %%J ... - - The ... have to be replaced by nothing or any imapsync option. Welcome - in shell or batch programming ! - - You will find already written scripts at - https://imapsync.lamiral.info/examples/ - -INSTALL - - Imapsync works under any Unix with perl. - - Imapsync works under most Windows (2000, XP, Vista, Seven, Eight, Ten - and all Server releases 2000, 2003, 2008 and R2, 2012 and R2) - as a standalone binary software called imapsync.exe, - usually launched from a batch file in order to avoid always typing - the options. - - Imapsync works under OS X as a standalone binary - software called imapsync_bin_Darwin - - Purchase latest imapsync at - https://imapsync.lamiral.info/ - - You'll receive a link to a compressed tarball called imapsync-x.xx.tgz - where x.xx is the version number. Untar the tarball where - you want (on Unix): - - tar xzvf imapsync-x.xx.tgz - - Go into the directory imapsync-x.xx and read the INSTALL file. - As mentioned at https://imapsync.lamiral.info/#install - the INSTALL file can also be found at - https://imapsync.lamiral.info/INSTALL.d/INSTALL.ANY.txt - It is now split in several files for each system - https://imapsync.lamiral.info/INSTALL.d/ - -CONFIGURATION - - There is no specific configuration file for imapsync, everything is - specified by the command line parameters and the default behavior. - -HACKING - - Feel free to hack imapsync as the NOLIMIT license permits it. - -SIMILAR SOFTWARE - - See also https://imapsync.lamiral.info/S/external.shtml - for a better up to date list. - - imap_tools : https://github.com/andrewnimmo/rick-sanders-imap-tools - offlineimap : https://github.com/nicolas33/offlineimap - Doveadm-Sync : http://wiki2.dovecot.org/Tools/Doveadm/Sync - ( Dovecot sync tool ) - mbsync : http://isync.sourceforge.net/ - mailsync : http://mailsync.sourceforge.net/ - mailutil : http://www.washington.edu/imap/ - part of the UW IMAP tookit. - imaprepl : http://www.bl0rg.net/software/ - http://freecode.com/projects/imap-repl/ - imapcopy : http://www.ardiehl.de/imapcopy/ - migrationtool : http://sourceforge.net/projects/migrationtool/ - imapmigrate : http://sourceforge.net/projects/cyrus-utils/ - wonko_imapsync: http://wonko.com/article/554 - see also file W/tools/wonko_ruby_imapsync - exchange-away : http://exchange-away.sourceforge.net/ - pop2imap : http://www.linux-france.org/prj/pop2imap/ - - Feedback (good or bad) will often be welcome. - -HISTORY - - I wrote imapsync because an enterprise (basystemes) paid me to install a - new imap server without losing huge old mailboxes located in a far away - remote imap server, accessible by a low-bandwidth often broken link. The - tool imapcp (written in python) could not help me because I had to - verify every mailbox was well transferred, and then delete it after a - good transfer. Imapsync started its life as a patch of the - copy_folder.pl script. The script copy_folder.pl comes from the - Mail-IMAPClient-2.1.3 perl module tarball source (more precisely in the - examples/ directory of the Mail-IMAPClient tarball). So many happened - since then that I wonder if it remains any lines of the original - copy_folder.pl in imapsync source code. - -```` diff --git a/README_Windows.txt b/README_Windows.txt index 83d6056..6865bea 100644 --- a/README_Windows.txt +++ b/README_Windows.txt @@ -1,4 +1,4 @@ -# $Id: README_Windows.txt,v 1.11 2018/05/05 22:46:01 gilles Exp gilles $ +# $Id: README_Windows.txt,v 1.13 2019/02/14 15:50:09 gilles Exp gilles $ # # This is the README_Windows.txt file for imapsync # imapsync : IMAP sync and migrate tool. @@ -42,14 +42,15 @@ In the folder extracted imapsync_1.xxx you see 6 files and 2 directories: * Cook/ is the directory to build imapsync.exe from its source, for the B) way and expert users. -You can copy or rename imapsync_example.bat as you wish as long as -its extension remains ".bat". On Windows systems .bat extension -means "I'm a batch script". Same thing for sync_loop_windows.bat. +You can copy or rename the file imapsync_example.bat as you wish, +as long as its extension remains ".bat". +On Windows systems .bat extension means "I'm a batch script". +Same thing for sync_loop_windows.bat. The batch scripts have to stay in the same directory than imapsync.exe because of the way they call imapsync.exe, they use ".\imapsync.exe", so let them be in the same directory. Or change the path .\ to whatever you want if you understand what -you're doing. +you're doing (you have to use a pathname from the script point of view). For the rest of this documentation I assume you copied imapsync_example.bat to a file named imapsync_stuff.bat @@ -72,21 +73,25 @@ A.5) Run the batch file To run imapsync with your values just double-click on the batch file imapsync_stuff.bat -You do not need to have administrator privileges to run imapsync. +There is no need to have administrator privileges to run imapsync. -A.6) Loop on A.5) A.6) edit, run, edit, run etc. +A.6) Loop on A.4) A.5) edit, run, edit, run etc. Loop the process of editing and running imapsync until you solve all issues and all values suit your needs. A.7) Look the sync running. You can abort it at any time with a - quick double ctrl-c, hit ctrl-c twice within one second. - (a single ctrl-c will reconnect to both imap servers) + quick double ctrl-c, hit ctrl-c twice within one second. + (a single ctrl-c will reconnect to both imap servers). + You can also abort the sync by closing the DOS window. + +A.8) When the sync is finished you can find the whole log file +of the output in the folder named "LOG_imapsync", +the logfile name is based on the launching date, +hour, minute, second, milliseconds and the user2 parameter. + +There is one logfile per run. -A.8) When the sync is finished you can find the whole log of the output -in the folder named "LOG_imapsync", the logfile name is based -on the launching date, hour, minute, second, miliseconds and the -user2 parameter. There is one logfile per run. The logfile name is printed at the end of the imapsync run. If you do not want logging to a file then use option --nolog diff --git a/S/external.shtml b/S/external.shtml index a009794..4323cc3 100644 --- a/S/external.shtml +++ b/S/external.shtml @@ -31,11 +31,11 @@

External online IMAP migration services (back to menu) @@ -67,16 +71,16 @@ I don't think they use Imapsync. Prices are given par mailbox and may be outdated (December 2011).

@@ -107,7 +111,7 @@ alt="Viewable With Any Browser" /> This document last modified on -($Id: external.shtml,v 1.24 2018/05/05 22:51:15 gilles Exp gilles $)
+($Id: external.shtml,v 1.28 2019/02/17 14:42:26 gilles Exp gilles $)
Top of the page

diff --git a/S/imapservers.shtml b/S/imapservers.shtml index 0a67c6b..a91cd14 100644 --- a/S/imapservers.shtml +++ b/S/imapservers.shtml @@ -27,7 +27,7 @@ -

Imapsync list of 75 imap server software applications supported (and the broken ones) (back to menu) +

Imapsync list of 77 imap server software applications supported (and the broken ones) (back to menu)

To know whether your IMAP server is a widespread choice, @@ -35,7 +35,7 @@ take a look at http://openemailsurvey.org/

Let's start with the long reported success stories list: -74 different imap server software tools supported!
+77 different imap server software tools supported!
[host1] means "source server" and [host2] means "destination server":

@@ -55,7 +55,7 @@ Example:

You can use option --justconnect to get those lines. -Examples:

+Example:

 imapsync --host1 test1.lamiral.info \
          --host2 test2.lamiral.info \
@@ -86,8 +86,8 @@ imapsync --host1 test1.lamiral.info \
 	
  • Cyrus IMAP 1.5, 1.6, 
 	2.1, 2.1.15, 2.1.16, 2.1.18 
 	2.2.1, 2.2.2-BETA, 2.2.3, 2.2.6, 2.2.10, 2.2.12, 2.2.13,
-	2.3-alpha (OSI Approved), 2.3.1, 2.3.7, 2.3.16
-	(http://cyrusimap.web.cmu.edu/)   
+	2.3-alpha (OSI Approved), 2.3.1, 2.3.7, 2.3.16, v2.4.16 [host1]
+	(https://www.cyrusimap.org/)
 	
    • 
 	
  • David Tobit V8.    (http://de.tobit.com/)   
    • 
 	
  • David.fx Mail Access Server MA-12.00a (0188) [host1]     
    • 
@@ -128,7 +128,8 @@ imapsync --host1 test1.lamiral.info \
         
  • iPlanet Messaging server 4.15, 5.1, 5.2    
 	(http://en.wikipedia.org/wiki/Oracle_Communications_Messaging_Server)   
    • 
         
  • iRedMail, iRedMail imap server software is Dovecot. http://www.iredmail.org/
    • 
-	
  • Kerio 7.2.0P1 [host1]    (http://www.kerio.com/)   
    • 
+	
  • Kerio 7.2.0P1 [host1]    (https://www.kerio.com/)   
    • 
+        
  • Kolab. Kolab groupware uses Cyrus.(https://kolab.org/)
    • 
         
  • Kopano (https://kopano.com/) (successor of Zarafa)  
    • 
 	
  • Mail2World IMAP4 Server 2.5 [host1] (http://www.mail2world.com/)
    • 
 	
  • MailEnable 4.23 [host1][host2], 4.26 [host1][host2], 5 [host1]    
@@ -138,7 +139,7 @@ imapsync --host1 test1.lamiral.info \
 	(http://www.altn.com/)   
    • 
 	
  • Mercury 4.1 (Windows server 2000 platform)    (http://www.pmail.com/)   
    • 
 
-	
  • Mirapoint server 4.1.9-GA [host1]    (http://www.mirapoint.com/)   
    • 
+	
  • Mirapoint server 4.1.9-GA [host1]    (https://www.icewarp.com/mirapoint/)   
    • 
 	
  • Netscape Mail Server 3.6 (Wintel)       
    • 
 	
  • Netscape Messaging Server 4.15 Patch 7       
    • 
 	
  • Office 365 [host1] [host2] (http://office365.com)  
    • 
@@ -154,7 +155,7 @@ imapsync --host1 test1.lamiral.info \
 	
  • Rockliffe Mailsite 5.3.11, 4.5.6    (http://www.mailsite.com/)   
    • 
    	
  • RackSpace hoster secure.emailsrvr.com:993 [host1] http://www.rackspace.com/)
    • 
 	
  • QQMail IMAP4Server [host1] [host2] (See FAQ) https://en.mail.qq.com/ 
    • 
-	
  • Samsung Contact IMAP server 8.5.0       
    • 
+	
  • Samsung Contact IMAP server 8.5.0 
    • 
 	
  • Scalix v10.1, 10.0.1.3, 11.0.0.431, 11.4.5, 11.4.6 (http://www.scalix.com/)   
    • 
         
  • Sendmail Mail Store IMAP4rev1 (5.5.6/mstore-5-5-build-1874 [host1].
    • 
 	
  • SmarterMail, Smarter Mail 5.0 Enterprise, Smarter Mail 5.5 [host1], SmarterMail Professional 10.2 [host1],
@@ -167,7 +168,7 @@ imapsync --host1 test1.lamiral.info \
 	
  • UW-imap servers (imap-2000b) rijkkramer IMAP4rev1 2000.287
 	(RedHat uses UW like 2003.338rh), v12.264 Solaris 5.7 (OSI Approved) 
 	(http://www.washington.edu/imap/)   
    • 
-	
  • VMS, Imap part of TCP/IP suite of VMS 7.3.2    (http://h71000.www7.hp.com/openvms/)   
    • 
+	
  • VMS, Imap part of TCP/IP suite of VMS 7.3.2    (https://www.vmssoftware.com/)   
    • 
         
  • Xeams/Synametrics (http://www.xeams.com/) 
    • 
 	
  • Yahoo [host1]    (http://www.yahoo.com/) 
    • 
 	
  • Zarafa 6,40,0,20653 [host1] (http://www.zarafa.com/) 
    • 
@@ -224,7 +225,7 @@ alt="Viewable With Any Browser" />
 
 
 This document last modified on 
-($Id: imapservers.shtml,v 1.36 2018/04/10 00:26:50 gilles Exp gilles $)
    
+($Id: imapservers.shtml,v 1.38 2019/02/17 14:34:25 gilles Exp gilles $)
    
 Top of the page
 
    
 
diff --git a/S/imapsync_sold_by_country.txt b/S/imapsync_sold_by_country.txt
index f7b7f4f..babc065 100644
--- a/S/imapsync_sold_by_country.txt
+++ b/S/imapsync_sold_by_country.txt
@@ -1,94 +1,97 @@
-1445 	         Etats-Unis______________ 23.15 %	23 %    1
-1206 	         Allemagne_______________ 19.32 %	42 %    2
-560  	         Royaume-Uni_____________ 8.97 %	51 %    3
-379  	         Italie__________________ 6.07 %	58 %    4
-303  	         France__________________ 4.85 %	62 %    5
-261  	         Canada__________________ 4.18 %	67 %    6
-252  	         Pays-Bas________________ 4.04 %	71 %    7
-251  	         Suisse__________________ 4.02 %	75 %    8
-207  	         Australie_______________ 3.32 %	78 %    9
-142  	         Autriche________________ 2.27 %	80 %    10
-131  	         Espagne_________________ 2.10 %	82 %    11
-102  	         Belgique________________ 1.63 %	84 %    12
-94   	         Suede___________________ 1.51 %	85 %    13
-79   	         Danemark________________ 1.27 %	87 %    14
-61   	         Bresil__________________ 0.98 %	88 %    15
-55   	         Pologne_________________ 0.88 %	89 %    16
-47   	         Norvege_________________ 0.75 %	89 %    17
-46   	         Republique_tcheque______ 0.74 %	90 %    18
-44   	         Finlande________________ 0.70 %	91 %    19
-33   	         Hongrie_________________ 0.53 %	91 %    20
-31   	         Russie__________________ 0.50 %	92 %    21
-29   	         Nouvelle-Zelande________ 0.46 %	92 %    22
-27   	         Japon___________________ 0.43 %	93 %    23
-25   	         ________________________ 0.40 %	93 %    24
-24   	         Portugal________________ 0.38 %	93 %    25
-23   	         Irlande_________________ 0.37 %	94 %    26
-23   	         Grece___________________ 0.37 %	94 %    27
-22   	         Afrique_du_Sud__________ 0.35 %	95 %    28
-20   	         Hong-Kong_______________ 0.32 %	95 %    29
-19   	         Inde____________________ 0.30 %	95 %    30
-17   	         Slovaquie_______________ 0.27 %	95 %    31
-17   	         Mexique_________________ 0.27 %	96 %    32
-17   	         Argentine_______________ 0.27 %	96 %    33
-16   	         Roumanie________________ 0.26 %	96 %    34
-16   	         Malaisie________________ 0.26 %	97 %    35
-16   	         Chili___________________ 0.26 %	97 %    36
-15   	         Chine___________________ 0.24 %	97 %    37
-14   	         Slovenie________________ 0.22 %	97 %    38
-14   	         Singapour_______________ 0.22 %	97 %    39
-14   	         Luxembourg______________ 0.22 %	98 %    40
-12   	         Israel__________________ 0.19 %	98 %    41
-10   	         Emirats_Arabes_Unis_____ 0.16 %	98 %    42
-9    	         Lettonie________________ 0.14 %	98 %    43
-7    	         Croatie_________________ 0.11 %	98 %    44
-6    	         Thailande_______________ 0.10 %	98 %    45
-6    	         Malte___________________ 0.10 %	98 %    46
-6    	         Islande_________________ 0.10 %	99 %    47
-6    	         Estonie_________________ 0.10 %	99 %    48
-5    	         Lituanie________________ 0.08 %	99 %    49
-5    	         Egypte__________________ 0.08 %	99 %    50
-4    	         Turquie_________________ 0.06 %	99 %    51
-4    	         Serbie__________________ 0.06 %	99 %    52
-4    	         Ireland_________________ 0.06 %	99 %    53
-4    	         Indonesie_______________ 0.06 %	99 %    54
-4    	         Chypre__________________ 0.06 %	99 %    55
-4    	         Bulgarie________________ 0.06 %	99 %    56
-3    	         Venezuela_______________ 0.05 %	99 %    57
-3    	         Uruguay_________________ 0.05 %	99 %    58
-3    	         Philippines_____________ 0.05 %	99 %    59
-2    	         Vietnam_________________ 0.03 %	99 %    60
-2    	         Ukraine_________________ 0.03 %	99 %    61
-2    	         Perou___________________ 0.03 %	99 %    62
-2    	         Nouvelle-Caledonie______ 0.03 %	99 %    63
-2    	         Costa_Rica______________ 0.03 %	100 %    64
-2    	         Antilles_neerlandaises__ 0.03 %	100 %    65
-1    	         Trinite-et-Tobago_______ 0.02 %	100 %    66
-1    	         Tanzanie________________ 0.02 %	100 %    67
-1    	         Taiwan__________________ 0.02 %	100 %    68
-1    	         Senegal_________________ 0.02 %	100 %    69
-1    	Saint_Christophe-Nevis-Anguilla__ 0.02 %	100 %    70
-1    	       Republique_d'Azerbaidjan__ 0.02 %	100 %    71
-1    	         Qatar___________________ 0.02 %	100 %    72
-1    	         Panama__________________ 0.02 %	100 %    73
-1    	         Nigeria_________________ 0.02 %	100 %    74
-1    	         Namibie_________________ 0.02 %	100 %    75
-1    	         Mongolie________________ 0.02 %	100 %    76
-1    	         Monaco__________________ 0.02 %	100 %    77
-1    	         Moldavie________________ 0.02 %	100 %    78
-1    	         Maroc___________________ 0.02 %	100 %    79
-1    	         Maldives________________ 0.02 %	100 %    80
-1    	         Koweit__________________ 0.02 %	100 %    81
-1    	         Jordanie________________ 0.02 %	100 %    82
-1    	         Jamaique________________ 0.02 %	100 %    83
-1    	      Iles_Vierges_britanniques__ 0.02 %	100 %    84
-1    	         Grenade_________________ 0.02 %	100 %    85
-1    	         Coree_du_Sud____________ 0.02 %	100 %    86
-1    	         Colombie________________ 0.02 %	100 %    87
-1    	         Cameroun________________ 0.02 %	100 %    88
-1    	         Burkina_Faso____________ 0.02 %	100 %    89
-1    	         Bosnie-Herzegovine______ 0.02 %	100 %    90
-1    	         Bahrein_________________ 0.02 %	100 %    91
-1    	         Arabie_Saoudite_________ 0.02 %	100 %    92
-1    	         Albanie_________________ 0.02 %	100 %    93
-TOTAL = 6242 sales 294171 EUR over 93 countries on Sun May 6 01:20:26 CEST 2018
+1    	         Albanie_________________ 0.01 %	100 %   0 %    96
+1    	         Arabie_Saoudite_________ 0.01 %	100 %   0 %    95
+1    	         Bahrein_________________ 0.01 %	100 %   0 %    94
+1    	         Bolivie_________________ 0.01 %	100 %   0 %    93
+1    	         Bosnie-Herzegovine______ 0.01 %	100 %   0 %    92
+1    	         Burkina_Faso____________ 0.01 %	100 %   0 %    91
+1    	         Cameroun________________ 0.01 %	100 %   0 %    90
+1    	         Colombie________________ 0.01 %	100 %   0 %    89
+1    	         Coree_du_Sud____________ 0.01 %	100 %   0 %    88
+1    	         Grenade_________________ 0.01 %	100 %   0 %    87
+1    	         Iles_Feroe______________ 0.01 %	100 %   0 %    86
+1    	      Iles_Vierges_britanniques__ 0.01 %	100 %   0 %    85
+1    	         Jamaique________________ 0.01 %	100 %   0 %    84
+1    	         Jordanie________________ 0.01 %	100 %   0 %    83
+1    	         Koweit__________________ 0.01 %	100 %   0 %    82
+1    	         Maldives________________ 0.01 %	100 %   0 %    81
+1    	         Maroc___________________ 0.01 %	100 %   0 %    80
+1    	         Moldavie________________ 0.01 %	100 %   0 %    79
+1    	         Monaco__________________ 0.01 %	100 %   0 %    78
+1    	         Mongolie________________ 0.01 %	100 %   0 %    77
+1    	         Namibie_________________ 0.01 %	100 %   0 %    76
+1    	         Panama__________________ 0.01 %	100 %   0 %    75
+1    	         Qatar___________________ 0.01 %	100 %   0 %    74
+1    	       Republique_d'Azerbaidjan__ 0.01 %	100 %   0 %    73
+1    	Saint_Christophe-Nevis-Anguilla__ 0.01 %	100 %   0 %    72
+1    	         Senegal_________________ 0.01 %	100 %   0 %    71
+1    	         Taiwan__________________ 0.01 %	100 %   0 %    70
+1    	         Tanzanie________________ 0.01 %	100 %   0 %    69
+1    	         Trinite-et-Tobago_______ 0.01 %	100 %   0 %    68
+2    	         Antilles_neerlandaises__ 0.03 %	100 %   0 %    67
+2    	         Costa_Rica______________ 0.03 %	100 %   0 %    66
+2    	         Nouvelle-Caledonie______ 0.03 %	100 %   0 %    65
+2    	         Perou___________________ 0.03 %	99 %   1 %    64
+2    	         Russie,_Federation______ 0.03 %	99 %   1 %    63
+2    	         Ukraine_________________ 0.03 %	99 %   1 %    62
+3    	         Nigeria_________________ 0.04 %	99 %   1 %    61
+3    	         Uruguay_________________ 0.04 %	99 %   1 %    60
+3    	         Venezuela_______________ 0.04 %	99 %   1 %    59
+4    	         Bulgarie________________ 0.06 %	99 %   1 %    58
+4    	         Indonesie_______________ 0.06 %	99 %   1 %    57
+4    	         Philippines_____________ 0.06 %	99 %   1 %    56
+4    	         Serbie__________________ 0.06 %	99 %   1 %    55
+4    	         Turquie_________________ 0.06 %	99 %   1 %    54
+4    	         Vietnam_________________ 0.06 %	99 %   1 %    53
+5    	         Chypre__________________ 0.07 %	99 %   1 %    52
+5    	         Egypte__________________ 0.07 %	99 %   1 %    51
+5    	         Ireland_________________ 0.07 %	99 %   1 %    50
+5    	         Lituanie________________ 0.07 %	99 %   1 %    49
+6    	         Malte___________________ 0.09 %	99 %   1 %    48
+6    	         Thailande_______________ 0.09 %	99 %   1 %    47
+7    	         Estonie_________________ 0.10 %	98 %   2 %    46
+7    	         Islande_________________ 0.10 %	98 %   2 %    45
+8    	         Croatie_________________ 0.12 %	98 %   2 %    44
+10   	         Emirats_Arabes_Unis_____ 0.15 %	98 %   2 %    43
+10   	         Lettonie________________ 0.15 %	98 %   2 %    42
+13   	         Israel__________________ 0.19 %	98 %   2 %    41
+14   	         Luxembourg______________ 0.21 %	98 %   2 %    40
+14   	         Singapour_______________ 0.21 %	97 %   3 %    39
+15   	         Chine___________________ 0.22 %	97 %   3 %    38
+16   	         Chili___________________ 0.24 %	97 %   3 %    37
+16   	         Malaisie________________ 0.24 %	97 %   3 %    36
+17   	         Argentine_______________ 0.25 %	96 %   4 %    35
+17   	         Slovenie________________ 0.25 %	96 %   4 %    34
+18   	         Mexique_________________ 0.27 %	96 %   4 %    33
+18   	         Slovaquie_______________ 0.27 %	96 %   4 %    32
+19   	         Inde____________________ 0.28 %	95 %   5 %    31
+21   	         Roumanie________________ 0.31 %	95 %   5 %    30
+23   	         Irlande_________________ 0.34 %	95 %   5 %    29
+24   	         Afrique_du_Sud__________ 0.36 %	94 %   6 %    28
+24   	         Grece___________________ 0.36 %	94 %   6 %    27
+24   	         Hong-Kong_______________ 0.36 %	94 %   6 %    26
+26   	         ________________________ 0.39 %	93 %   7 %    25
+30   	         Portugal________________ 0.45 %	93 %   7 %    24
+31   	         Japon___________________ 0.46 %	93 %   7 %    23
+31   	         Nouvelle-Zelande________ 0.46 %	92 %   8 %    22
+31   	         Russie__________________ 0.46 %	92 %   8 %    21
+33   	         Hongrie_________________ 0.49 %	91 %   9 %    20
+46   	         Finlande________________ 0.69 %	91 %   9 %    19
+48   	         Norvege_________________ 0.72 %	90 %   10 %    18
+50   	         Republique_tcheque______ 0.75 %	89 %   11 %    17
+64   	         Bresil__________________ 0.96 %	89 %   11 %    16
+71   	         Pologne_________________ 1.06 %	88 %   12 %    15
+89   	         Danemark________________ 1.33 %	87 %   13 %    14
+103  	         Suede___________________ 1.54 %	85 %   15 %    13
+111  	         Belgique________________ 1.66 %	84 %   16 %    12
+144  	         Espagne_________________ 2.15 %	82 %   18 %    11
+160  	         Autriche________________ 2.39 %	80 %   20 %    10
+221  	         Australie_______________ 3.30 %	77 %   23 %    9
+268  	         Suisse__________________ 4.00 %	74 %   26 %    8
+274  	         Canada__________________ 4.09 %	70 %   30 %    7
+280  	         Pays-Bas________________ 4.18 %	66 %   34 %    6
+326  	         France__________________ 4.87 %	62 %   38 %    5
+403  	         Italie__________________ 6.02 %	57 %   43 %    4
+609  	         Royaume-Uni_____________ 9.09 %	51 %   49 %    3
+1302 	         Allemagne_______________ 19.44 %	42 %   58 %    2
+1507 	         Etats-Unis______________ 22.50 %	22 %   78 %    1
+TOTAL = 6699 sales 316542 EUR over 96 countries on Thu Feb 14 18:27:39 CET 2019
diff --git a/S/news.shtml b/S/news.shtml
index f07d9c5..52d54dc 100644
--- a/S/news.shtml
+++ b/S/news.shtml
@@ -39,16 +39,25 @@ next and previous releases (back to menu)
 
 
 
+ 
+ 
      
+
    • 1.920
      • 
+
    • Enhancement: Script sync_parallel_unix.sh to parallelize massive syncs. 
+It uses the powerful GNU parallel command.
+
      • 
+
+
    • Enhancement: Added option --synclabels to sync Gmail labels. It should speed up Gmail to Gmail syncs. 
+  The option --synclabels applies only during a copy of a message for now. Resyncing Gmail labels will be coded later.
+
      • 
+
    • Enhancement: Option --synclabels is activated with --gmail1 --gmail2 used together.
      • 
+
+
    • Portability: The imapsync script Shebang (first line of imapsync) uses env now,
+ instead of direct perl path. 
+ I'm not sure it's a good change that 
+ #!/usr/bin/perl becomes #!/usr/bin/env perl
      • 
+
+
    • Enhancement: Final stats:
+        
        
+        
      • Added a counter of crossduplicates on host2 ( a crossduplicate is a message found accross different folders )
        • 
+        
      • Added counter of "Messages found in host1 not in host2"
        • 
+        
      • Added counter of "Messages found in host2 not in host1"
        • 
+        
      • Added stat "Load end is"
        • 
+        
      • Added comment on final stat about messages in host1 not in host2
        • 
+        
      • Added comment on final stat about messages in host2 not in host1
        • 
+        
      • Added comment on final stat about unidentified messages and an --addheader suggestion
        • 
+        
      • Removed counters of "Total bytes duplicate host1" and "Total bytes duplicate host2"
        • 
+        
      • Removed counter of "Total bytes error"
        • 
+        
      
+
      • 
+
+
    • Enhancement: Added authmech X-MASTERAUTH. To be used for Kerio with --authmech1 X-MASTERAUTH or --authmech2 X-MASTERAUTH ( Patch from https://github.com/imapsync/imapsync/pull/53/files )
      • 
+
+
    • Enhancement: Added --sigprint HUP (signal HUP by default) in order to allow some asynchronous prints (with, for example, a kill -HUP IMAPSYNC_PID) in CGI mode and avoid timeouts from browser
      • 
+
    • Enhancement: --skipcrossduplicates activated with --gmail1 alone
      • 
+
    • Enhancement: --skipcrossduplicates is NOT activated with --gmail2 alone or with --gmail1 --gmail2 together used.
      • 
+
    • Enhancement: Speed. Deletions on host1 are done by batch during a resync.
      • 
+
    • Enhancement: Added automatic --maxsize from "CAPABILITY  APPENDLIMIT=xxxx" (Gmail)
      • 
+
    • Enhancement: Added --folderlast '[Gmail]/All Mail' with --gmail1 --gmail2
      • 
+
+
    • Enhancement: With --office1 option, folder "Files" is excluded because it's a "special" folder. Like --exclude "^Files$". 
+                        Unless --noexclude is used.
      • 
+
    • Enhancement: With --office2 option, folder "Files" on host1 is renamed like --f1f2 "Files=Files_renamed_by_imapsync". 
+                        Unless --nof1f2 is used.
      • 
+
    • Enhancement: Added --exclude '^Files$' with --office2 option.
      • 
+
    • Enhancement: Added inline explanations for the options --office1 --office2 --exchange1 --exchange2 (what they set and how to avoid what they set).
      • 
+
    • Enhancement: Added --noregexmess to avoid the regexmess setting with --office2 and --exchange2 (wrap long lines >= 10500 char).
      • 
+
    • Enhancement: Added --noregexflag to avoid the regexflag set by --exchange2 (remove \Flagged flag).
      • 
+
+
    • Enhancement: Added IO::Socket::IP and CGI in the list of the modules versions.
      • 
+
+
+
+
    • Usability: Added PPID info where PID is printed. Can be useful in some contextes, CGI, embeded, debugging etc.
      • 
+
+
+
+
    • Bug fix: Now split the deletion of messages in several chunks (using --split1)
      • 
+
    • Bug fix:  --debugssl 0 was impossible and equivalent to --debugssl 1
      • 
+
+
    • Bug fix: Removed --nosubscribed usage since it is not coded and useless anyway.
      • 
+
    • Bug fix: Option --maxsleep override variable $MAX_SLEEP in gmail*() subroutines.
      • 
+
    • Bug fix: Do not create host2 folder when no messages are found on this folder on host1 
+       (See https://github.com/imapsync/imapsync/issues/95)
      • 
+
    • Bug fix: The regex to remove the flag \Flagged was wrong. It worked because of another filter run after.
      • 
+
    • Bug fix: The options --exchange1 and --exchange2 were not options! No one complained...
      • 
+
    • Bug fix: A pid can be up to 99999 on FreeBSD
      • 
+
+
    • Refactoring: Deglobarised some global variables
      • 
+
    • Refactoring: Refactored sub appendlimit() to use Mail::IMAPClient::capability like the API. 
+                        No hack from internal $imap->{ APPENDLIMIT }
      • 
+
    • Refactoring: Fixed some perl critics.
      • 
+
+
+
    • Portability: Added freebsd support for loadavg and cpu_cores.
      • 
+
    • Portability: Calls memory_consumption_of_pids_win32() within Cygwin environment (https://github.com/imapsync/imapsync/pull/22)
      • 
+
+
+
    • Dependency removed: None.
      • 
+
    • Dependency added: None.
      • 
+
+
    • Roadmap:  Proxy mode abandoned. Will be another lighter memory tool (since less memory consumption is the goal).
      • 
+
+
    
+
+ 
 
      
 
    • 1.882 (2018/05/06) On the road to a proxy mode! (not done yet)
      • 
 
    • Enhancement: Added option --nocheckfoldersexist ( Default is like --checkfoldersexist )
      • 
 
    • Enhancement: Added option --noresyncflags     : Do not resync flags for already transfered messages.
                         May be useful when a user has already started to play with its host2 account.
      • 
 
    • Enhancement: Option --abort now creates a log too, in order to see if imapsync ended by signal comes from option --abort or something else.
      • 
-
    • Enhancement: Added option --sigexit; For example --sigexit QUIT --sigexit TERM to exit when receiving QUIT or TERM signals.
      
-Added option --sigreconnect. For example --sigreconnect INT to reconnect to both imap servers when receiving INT signal.
      
-Added option --sigignore. For example --sigignore TERM to "ignore" the TERM signal. In fact print stats during the sync.
      
+
    • Enhancement: Added option --sigexit; For example --sigexit QUIT --sigexit TERM to exit when receiving QUIT or TERM signals.
      
+Added option --sigreconnect. For example --sigreconnect INT to reconnect to both imap servers when receiving INT signal.
      
+Added option --sigignore. For example --sigignore TERM to "ignore" the TERM signal. In fact print stats during the sync.
      
 Default is like:  --sigexit QUIT --sigexit TERM --sigreconnect INT 
 
      • 
-
    • Enhancement: Moved the hostname printing at first line. The goal is for clarity when in proxy mode.
      
-Added free RAM info in the banner.
      
+
    • Enhancement: Moved the hostname printing at first line. The goal is for clarity when in proxy mode.
      
+Added free RAM info in the banner.
      
 
      • 
 
    • Enhancement: When using sleeping behavior, a signal USR1 toggles sleep from 2s to 0s (back to 2s with on next USR1 signal). Unix only.
      • 
 
@@ -583,15 +685,15 @@ by ignoring PERMANENTFLAGS (Exchange tests)
 
 
      
 
@@ -601,7 +703,7 @@ by ignoring PERMANENTFLAGS (Exchange tests)
 
 
 This document last modified on 
-($Id: news.shtml,v 1.54 2018/05/06 15:03:59 gilles Exp gilles $)
      
+($Id: news.shtml,v 1.59 2019/02/17 14:34:36 gilles Exp gilles $)
      
 Top of the page
 
      
 
diff --git a/S/paypal.shtml b/S/paypal.shtml
index 26e84cd..1b2bc58 100644
--- a/S/paypal.shtml
+++ b/S/paypal.shtml
@@ -15,20 +15,11 @@
 
 
 
-
 
 
 
 
 This document last modified on 
-($Id: paypal.shtml,v 1.12 2016/12/20 10:06:54 gilles Exp gilles $)
+($Id: paypal.shtml,v 1.13 2018/08/21 12:07:50 gilles Exp gilles $)
 
      
 
 
diff --git a/S/paypal_return.shtml b/S/paypal_return.shtml
index b31dc35..2eaecaf 100644
--- a/S/paypal_return.shtml
+++ b/S/paypal_return.shtml
@@ -1,12 +1,12 @@
 
 
+
 
 
 imapsync download
-
 
 
-
+
 
 
 
@@ -15,6 +15,7 @@
 
 
 
+
 
-
-
-
-
      imapsync support
      
-
-
      I thank you for buying Imapsync support!
      
-
-
      The payment has been made and the transaction has been completed.
      
-A receipt for your purchase has been emailed to you.
      
-You may log into your account at www.paypal.com 
-to view details of this transaction. 
-
      
-
-
      You will receive an invoice soon.
      
-
-
      Now you can contact me (Gilles LAMIRAL) by email or phone
      
-
        
-
      • Email address: gilles.lamiral@@laposte.net.
        • 
-
      • Professionnal phone number: +33 951 84 42 42 (in France) I can call you back for free in many countries.
        • 
-
      • Mobile phone number: +33 620 79 76 06 (in France).
        • 
-
      
-
-
      I thank you again for buying imapsync support,
-I wish you successful imap transfers and I will help you until you reach this goal!
      
-
-
      imapsync homepage
      
-
-
      Gilles LAMIRAL
      
-gilles.lamiral@@laposte.net
      
-
-
      
-
      
-    Valid XHTML 1.0 Strict
-
-
-    CSS Valide !
-
-
      
-
-
-This document last modified on 
      
-($Id: paypal_return_support.shtml,v 1.5 2011/08/03 00:57:39 gilles Exp gilles $)
-
      
-
-
-
-
-
-
-
-
-
-
-@
-
-
-1.5
-log
-@*** empty log message ***
-@
-text
-@d8 1
-a8 1
-
-d17 1
-a34 1
-
-a35 3
-
-
-
-d48 1
-a48 1
-
      Now you can contact me (Gilles LAMIRAL) by email or phone
      
-d56 1
-a56 1
-I wish you successful imap transfers and I will help you until you reach this goal!
      
-d58 1
-a58 1
-
      imapsync homepage
      
-d78 1
-a78 1
-($Id: paypal_return_support.shtml,v 1.4 2011/07/29 02:21:08 gilles Exp gilles $)
-@
-
-
-1.4
-log
-@30 -> .90
-@
-text
-@d81 1
-a81 1
-($Id: paypal_return_support.shtml,v 1.3 2011/05/20 12:32:25 gilles Exp gilles $)
-d93 2
-a94 2
-if (.90) {
-  google_conversion_value = .90;
-d102 1
-a102 1
-
-@
-
-
-1.3
-log
-@typo
-@
-text
-@d81 1
-a81 1
-($Id: paypal_return_support.shtml,v 1.2 2011/04/19 13:09:12 gilles Exp gilles $)
-d93 2
-a94 2
-if (30) {
-  google_conversion_value = 30;
-d102 1
-a102 1
-
-@
-
-
-1.2
-log
-@typo
-@
-text
-@d54 1
-a54 1
-
    • Professionnal phone number: +33 9 51 84 42 42 (in France) I can call you back for free in many countries.
      • 
-d81 1
-a81 1
-($Id: paypal_return_support.shtml,v 1.1 2011/04/19 13:08:26 gilles Exp gilles $)
-@
-
-
-1.1
-log
-@Initial revision
-@
-text
-@d54 1
-a54 1
-
    • Professionnal phone numner: +33 9 51 84 42 42 (in France) I can call you back for free in many countries.
      • 
-d81 1
-a81 1
-($Id: paypal_return.shtml,v 1.2 2011/03/24 01:21:27 gilles Exp gilles $)
-@
diff --git a/X/imapsync_form.html b/X/imapsync_form.html
index 313ec1a..2d64fad 100644
--- a/X/imapsync_form.html
+++ b/X/imapsync_form.html
@@ -1,5 +1,5 @@
 
-
+
 
 
  
@@ -16,11 +16,8 @@
         
         
 
-    
-    
+    
+