zydronium/imapsync
1
0
Fork 0
mirror of https://github.com/imapsync/imapsync.git synced 2025-06-04 19:57:21 +02:00
Code Issues Projects Releases Packages Wiki Activity

2.178

Browse source
This commit is contained in:
Nick Bebout 2022-02-15 12:29:45 -06:00
parent f98c9946e5
commit 2f815205a9
72 changed files with 4567 additions and 1469 deletions
Download patch file Download diff file Expand all files Collapse all files

178
ChangeLog
View file

@ -1,17 +1,187 @@
RCS file: RCS/imapsync,v
Working file: imapsync
head: 2.140
head: 2.178
branch:
locks: strict
gilles: 2.140
gilles: 2.178
access list:
symbolic names:
keyword substitution: kv
total revisions: 1040; selected revisions: 1040
total revisions: 1078; selected revisions: 1078
description:
----------------------------
revision 2.140 locked by: gilles;
revision 2.178 locked by: gilles;
date: 2022/01/12 21:28:37; author: gilles; state: Exp; lines: +75 -13
Added tests_compress_ssl()
----------------------------
revision 2.177
date: 2021/12/23 20:38:44; author: gilles; state: Exp; lines: +8 -8
Bugfix. 1849 -> 1848 unit tests
----------------------------
revision 2.176
date: 2021/12/23 20:19:04; author: gilles; state: Exp; lines: +35 -23
Bugfix. memory consumption on Mac was not relevant. Use RSS instead of VSZ, on Mac only. Linux stays with VSZ.
----------------------------
revision 2.175
date: 2021/12/21 17:46:14; author: gilles; state: Exp; lines: +23 -21
Bugix. Darwin, fixed loadavg tests, no more W/t/loadavg.out needed.
----------------------------
revision 2.174
date: 2021/12/14 15:51:58; author: gilles; state: Exp; lines: +62 -17
Added advice for:
ERR_APPEND_SIZE
ERR_CONNECTION_FAILURE_HOST1
ERR_CONNECTION_FAILURE_HOST2
ERR_AUTHENTICATION_FAILURE_USER1
ERR_AUTHENTICATION_FAILURE_USER2
----------------------------
revision 2.173
date: 2021/12/09 18:30:34; author: gilles; state: Exp; lines: +61 -40
Bugfix. Made reconnect mechanism works with --oauthdirect1 --oauthdirect2 --oauthaccesstoken1 --oauthaccesstoken2
----------------------------
revision 2.172
date: 2021/12/07 18:19:42; author: gilles; state: Exp; lines: +27 -17
Added some tests to tests_get_options_from_string() and tests_get_options_extra()
I can not have the unknown options back with Getopt::Long::GetOptionsFromString(), just the unknown values, ie, caca but not --caca
----------------------------
revision 2.171
date: 2021/12/07 17:50:12; author: gilles; state: Exp; lines: +14 -8
Added --ssl1 --ssl2 --tls1 --tls2 --compress1 --compress2 to get_options_from_string()
----------------------------
revision 2.170
date: 2021/12/07 17:19:44; author: gilles; state: Exp; lines: +12 -8
Added --keepalive1 --keepalive2 --reconnectretry1 --reconnectretry2 in get_options_from_string()
----------------------------
revision 2.169
date: 2021/12/07 17:13:44; author: gilles; state: Exp; lines: +101 -36
Added --keepalive1 and --keepalive2 options. On by default.
See https://metacpan.org/pod/Mail::IMAPClient#Keepalive
Use --nokeepalive1 and --nokeepalive2 to disable Keepalive.
----------------------------
revision 2.168
date: 2021/12/05 23:27:27; author: gilles; state: Exp; lines: +29 -11
Added --debugflags --errorsmax --folder --timeout in options from file
----------------------------
revision 2.167
date: 2021/12/02 16:59:51; author: gilles; state: Exp; lines: +9 -9
Moved the call to extra options after the chdir of the CGI context.
----------------------------
revision 2.166
date: 2021/12/02 15:23:37; author: gilles; state: Exp; lines: +107 -9
Now read extra options on the first line of the file ./options_extra.txt if it is present and readable.
----------------------------
revision 2.165
date: 2021/11/30 20:27:38; author: gilles; state: Exp; lines: +30 -11
cpu_time rounded with 2 decimals.
Added 4 tests when no compression is on.
----------------------------
revision 2.164
date: 2021/11/24 21:26:46; author: gilles; state: Exp; lines: +12 -11
Compression no more on by default. Just to see.
----------------------------
revision 2.163
date: 2021/11/20 20:11:46; author: gilles; state: Exp; lines: +8 -7
Bugfix. imapsync --version was buggy in docker context.
----------------------------
revision 2.162
date: 2021/11/20 10:16:33; author: gilles; state: Exp; lines: +14 -12
Docker. Current working directory changed from /var/tmp to /var/tmp/uid_$EFFECTIVE_USER_ID
----------------------------
revision 2.161
date: 2021/11/19 18:17:48; author: gilles; state: Exp; lines: +8 -8
Bugfix. inline doc mentioned always --nocompress1
----------------------------
revision 2.160
date: 2021/11/19 14:40:58; author: gilles; state: Exp; lines: +10 -7
Added inline documentation about --nocompress1 and --nocompress2
----------------------------
revision 2.159
date: 2021/11/19 14:29:31; author: gilles; state: Exp; lines: +113 -12
Dependency. Added use Compress::Zlib
Enhancement. Added compression in imap. On by default no matter the capability announced. Use --nocompress1 and --nocompress2 to turn it off.
----------------------------
revision 2.158
date: 2021/11/13 20:01:35; author: gilles; state: Exp; lines: +13 -7
Added README help for --truncmess
----------------------------
revision 2.157
date: 2021/10/23 21:51:43; author: gilles; state: Exp; lines: +61 -16
Refactor. Deglobalized $debugcontent. Added debugcontent() function.
1800 unit tests.
----------------------------
revision 2.156
date: 2021/10/12 13:36:36; author: gilles; state: Exp; lines: +72 -71
Added comment for ERR_FLAGS "Many STORE errors with FLAGS. Retry with the option --noresyncflags"
Refactored errors_incr() and errors_listing()
Bugfix. errors_incr() existed with CATCH_ALL
----------------------------
revision 2.155
date: 2021/10/01 07:46:55; author: gilles; state: Exp; lines: +9 -8
Like --useheader X-Gmail-Received --useheader Message-Id when --gmail1 --gmail2
----------------------------
revision 2.154
date: 2021/09/29 20:38:04; author: gilles; state: Exp; lines: +8 -8
Bugfix. Added 'E&AwE-le&AwE-ments envoye&AwE-s' in possible_special
----------------------------
revision 2.153
date: 2021/09/29 13:56:15; author: gilles; state: Exp; lines: +34 -12
Does not checknoabletosearch if --justfolders
When creating folder check if selectable and consider ok if selectable
----------------------------
revision 2.152
date: 2021/09/29 12:03:12; author: gilles; state: Exp; lines: +362 -58
Added inline advice on error ERR_OVERQUOTA
Added inline advice on error ERR_TRANSFER_EXCEEDED
Renamed function bytes_display_string() bytes_display_string_bin()
bytes_display_string_bin converts bytes to human KiB MiB GiB TiB PiB
Added function bytes_display_string_dec() to convert bytes to human decimal KB MB GB TB PB (1000 base)
----------------------------
revision 2.151
date: 2021/09/08 10:34:25; author: gilles; state: Exp; lines: +23 -18
Bugfix. load average on macosx was sometimes buggy. locale float 3,14 instead of 3.14
----------------------------
revision 2.150
date: 2021/08/08 12:10:51; author: gilles; state: Exp; lines: +9 -10
Refactor. Deglobalized warn_release variable
----------------------------
revision 2.149
date: 2021/07/31 11:24:24; author: gilles; state: Exp; lines: +7 -26
Removed --fast option.
----------------------------
revision 2.148
date: 2021/07/22 14:21:09; author: gilles; state: Exp; lines: +10 -10
Bugfix. abort didn't do charset=UTF-8; same for heavy load 503 Service Unavailable
----------------------------
revision 2.147
date: 2021/07/22 13:13:50; author: gilles; state: Exp; lines: +12 -12
Bugfix. The maximum number a pid can be is 2^22, not 2^32.
----------------------------
revision 2.146
date: 2021/07/22 12:48:21; author: gilles; state: Exp; lines: +8 -8
1742 unit tests.
----------------------------
revision 2.145
date: 2021/07/22 12:45:31; author: gilles; state: Exp; lines: +16 -13
Bugfix. Up the maximum number a pid can be. From 99999 to 2^32. See https://unix.stackexchange.com/questions/16883/what-is-the-maximum-value-of-the-process-id
----------------------------
revision 2.144
date: 2021/07/17 18:07:59; author: gilles; state: Exp; lines: +13 -10
Just an experimental loaddelay forced to 0.
----------------------------
revision 2.143
date: 2021/07/12 19:45:44; author: gilles; state: Exp; lines: +111 -15
Bugfix. Under docker context --version was buggy, too verbose.
----------------------------
revision 2.142
date: 2021/07/12 14:04:01; author: gilles; state: Exp; lines: +17 -19
Removed that hard limit --maxsize 35_651_584 set by --gmail2 since CAPABILITY is used and good.
----------------------------
revision 2.141
date: 2021/07/06 02:59:19; author: gilles; state: Exp; lines: +8 -8
cpu_time param
----------------------------
revision 2.140
date: 2021/07/05 20:14:18; author: gilles; state: Exp; lines: +14 -10
Fixed perlcrit 4 severity 4 violations.
----------------------------

54
FAQ.d/FAQ.Archiving.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Archiving.txt,v 1.25 2021/05/04 16:56:59 gilles Exp gilles $
$Id: FAQ.Archiving.txt,v 1.26 2022/01/11 12:55:53 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -14,12 +14,12 @@ Questions answered in this FAQ are:
Q. How can I archive different accounts on the same destination account,
each account on a separate folder?
Q. How can I restore a complete account archived in a subfolder, like
with the option --subfolder2?
Q. How can I restore a complete account backuped in a subfolder, like
a backup done with the option --subfolder2?
Q. How to move emails from one IMAP folder to another either on the
same IMAP server or a different one?
For example, all messages older than 1 year, move from INBOX to Archive.
Q. How to move emails from one IMAP folder to another, either on the
same IMAP server or on a different one? Example:
How to move all messages older than 1 year, from INBOX to Archive?
Q. Can imapsync be used to maintain and restore a local offline copy
of a mailbox, eg for backup purposes, using Mbox or Maildir
@ -47,8 +47,12 @@ R. Use --subfolder2
Examples:
To backup the account Foo in a subfolder called FooArchiv:
imapsync ... --user1 Foo --subfolder2 FooArchiv
To backup the account Bar in a subfolder called BarArchiv:
imapsync ... --user1 Bar --subfolder2 BarArchiv
In case you need a very strict sync, add options
@ -58,14 +62,20 @@ It will delete on account2 what is not on account1 but only in the right place,
ie, the subfolder "FooArchiv".
Examples:
To mirror the account Foo in a subfolder called FooArchiv:
imapsync ... --user1 Foo --subfolder2 FooArchiv --delete2 --delete2foldersonly "FooArchiv"
To mirror the account Bar in a subfolder called BarArchiv:
imapsync ... --user1 Bar --subfolder2 BarArchiv --delete2 --delete2foldersonly "BarArchiv"
See how to restore those with the next Q/R.
=======================================================================
Q. How can I restore a complete account archived in a subfolder, like
with the option --subfolder2?
Q. How can I restore a complete account backuped in a subfolder, like
a backup done with the option --subfolder2?
R. Use --subfolder1
@ -81,26 +91,29 @@ R. Use --subfolder1
Examples:
To restore the whole folder FooArchiv hierarchy into the Foo account:
imapsync ... --user2 Foo --subfolder1 FooArchiv
imapsync ... --user2 Bar --subfolder1 BarArchiv
To restore the whole folder BarArchiv hierarchy into the Bar account:
imapsync ... --user2 Bar --subfolder1 BarArchiv
Read the previous FAQ item "How can I archive different accounts on the same
destination account, each account on a separate folder?"
=======================================================================
Q. How to move emails from one IMAP folder to another either on the
same IMAP server or a different one?
For example, all messages older than 1 year, move from INBOX to Archive.
Q. How to move emails from one IMAP folder to another, either on the
same IMAP server or on a different one? Example:
How to move all messages older than 1 year, from INBOX to Archive?
R1. Solution:
imapsync ... --minage 365 --folder Inbox --f1f2 INBOX=Archive --delete1 --noexpungeaftereach
Add option --noexpungeaftereach if speed is a concern.
Add option --delete1 if you want to move messages, instead of just copy/sync them.
Use the option --noexpungeaftereach if speed is a concern.
Use the option --delete1 if you want to move messages, instead of just copy/sync them.
R2. See also file FAQ.Folders_Mapping.txt
@ -145,20 +158,23 @@ I have never used any of this tools so asking me for support with them
won't go far away.
=======================================================================
Q. How to archive a whole year in a subfolder?
Q. How to backup a whole year in a subfolder?
R. Example:
R. To backup the whole year 2018 in a subfolder called ARCHIVE_2018:
imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \
--folder2 ARCHIVE_2018
--subfolder2 ARCHIVE_2018
=======================================================================
Q. How to archive a whole year in subfolders but without subsubfolders?
R. Example:
R. Instead of the solution with --subfolder2, use:
imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \
--sep2 _ --regextrans2 's{(.*)}{Archive_$1_2018}'
For each folder "FFFF" in the source account, it will create a destination
folder called "Archive_FFFF_2018", at the top level hierarchy.
=======================================================================
=======================================================================

11
FAQ.d/FAQ.Authentication_failure.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Authentication_failure.txt,v 1.13 2021/03/02 19:24:22 gilles Exp gilles $
$Id: FAQ.Authentication_failure.txt,v 1.14 2021/11/05 11:35:07 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -26,14 +26,16 @@ Host1 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2
R. One over four imapsync syncs ends up quickly with the error message
"Authentication failed" or "NO LOGIN failed" or similar.
"Authentication failed" or "NO LOGIN failed" or a similar message.
Authentication failure is the primary failure with imapsync
and since nothing tangible can be done without authentication,
this stage must succeed to go further.
Here are some advices to get you pass this difficult stage of authentication:
* Triple check each credential parameter, there are three parameters at each side:
* Triple check each credential parameter. It's not always a password problem,
it can be the wrong host or the wrong user.
There are three parameters at each side:
* triple check --host1
* triple check --user1
* triple check --password1
@ -53,8 +55,7 @@ Here are some advices to get you pass this difficult stage of authentication:
instead of the string MASKED. It helps for debugging quoting issues.
Option --showpasswords shows passwords again when the IMAP dialog
is dumped by --debugimap option. Search for a line like
"Sending: 2 LOGIN test1 secret1" (secret1 is the password here)
"Sending: 2 LOGIN test1 secret1" (secret1 is the password here).
* It is sometimes very hard to quote correctly unusual characters,
especially on Windows (and Mac sometimes). See

21
FAQ.d/FAQ.Contacts_Calendars.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Contacts_Calendars.txt,v 1.14 2020/11/02 16:04:09 gilles Exp gilles $
$Id: FAQ.Contacts_Calendars.txt,v 1.15 2021/09/06 14:32:19 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -11,21 +11,20 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Contacts_Calendars.txt
Questions answered in this FAQ are:
Q. Can I copy or sync Calendars or Contacts with imapsync?
Q. Can I copy or sync Chat messages with imapsync?
Q. How can I avoid copying Calendar or Contacts folders?
Q. Can I copy or sync Contacts, Calendars, Tasks or Chat messages with imapsync?
Q. How can I copy or synchronize Calendars or Contacts?
Q. How can I avoid copying Calendar or Contacts folders?
Now the questions again with their answers.
=======================================================================
Q. Can I copy or sync Calendars or Contacts with imapsync?
Q. Can I copy or sync Contacts, Calendars, Tasks or Chat messages with imapsync?
R. No, Imapsync can't migrate Contacts and Calendars.
R. No, Imapsync can't migrate Contacts, Calendars, Tasks nor Chat messages.
It's because most IMAP servers don't get contacts and calendars
events via IMAP. In other words, messages synced by imapsync from
Calendars or Contacts folders are not used by email servers to set
or get the contacts or calendars.
Contacts/Calendars/Tasks/Chat folders are not used by email servers to set
or get the contacts, calendars, tasks or chat messages.
No way via IMAP, no way via imapsync.
So it's a good idea to avoid syncing contacts and calendars when
@ -35,12 +34,6 @@ R. No, Imapsync can't migrate Contacts and Calendars.
But see also the question below
"How can I copy or synchronize Calendars or Contacts?".
=======================================================================
Q. Can I copy or sync Chat messages with imapsync?
R. No, Imapsync can't migrate Chat messages (those in GSuite for example).
The explanation is the same as with Calendars or Contacts.
=======================================================================
Q. How can I copy or synchronize Calendars or Contacts?

73
FAQ.d/FAQ.Docker.txt
View file

@ -1,13 +1,13 @@
#!/bin/cat
# $Id: FAQ.Docker.txt,v 1.8 2021/05/04 16:57:09 gilles Exp gilles $
# $Id: FAQ.Docker.txt,v 1.10 2021/07/12 21:22:14 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Docker.txt
===========================================================================
Installing and using imapsync docker image
===========================================================================
======================================================================
Installing and using imapsync docker image
======================================================================
Questions answered in this FAQ are:
@ -23,27 +23,29 @@ Q. Where is the imapsync Docker hub location?
Q. How can I mask the passwords on the command line without mounting
inside the container?
===========================================================================
Q. Why imapsync isn't logging to a file in Docker context.
How can I allow it?
Q. Any tips for the Docker Mailcow distribution?
Q. How can I build my own image?
Q. What do you do to update the docker hub image?
Now the questions again with their answers.
===========================================================================
======================================================================
Q. How can I install and use the imapsync Docker image on my amd64 system?
R. Install Docker on your system. Once Docker is installed on your system,
all you have to do in order to run imapsync is the command line:
all you have to do in order install and run imapsync is the command line:
docker run gilleslamiral/imapsync imapsync <usual imapsync arguments>
===========================================================================
======================================================================
Q. How can I install and use the imapsync Docker image on my i386 system?
R. You can't for now. It will throw this error when running it:
R1. You can't do it directly for now. It will throw this error when running it:
docker run gilleslamiral/imapsync imapsync
standard_init_linux.go:211: exec user process caused "exec format error"
@ -51,21 +53,24 @@ R. You can't for now. It will throw this error when running it:
I plan to make the image multi-plateform. Drop me a note if you are
eager to have it.
===========================================================================
R2. Build the image on a i386 system with the exact same Dockerfile provided
and run it on your i386 system.
======================================================================
Q. How can I install or update only the imapsync Docker image on my system?
R. To install or update the imapsync image, run:
docker pull gilleslamiral/imapsync
===========================================================================
======================================================================
Q. Where is the imapsync Docker hub location?
R. Here:
https://hub.docker.com/r/gilleslamiral/imapsync/
===========================================================================
======================================================================
Q. How can I mask the passwords on the command line without mounting
anything inside the container?
@ -84,7 +89,7 @@ See:
https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file
https://stackoverflow.com/a/30494145/491175
===========================================================================
======================================================================
Q. Why imapsync isn't logging to a file in Docker context.
How can I allow it?
@ -93,14 +98,14 @@ R. In Docker context, writing the log is disabled by default because you
inside the docker and no longer available after the sync, unless
a special mount is done before.
Starting with imapsync release 2.113 the logging can be set on
Starting with imapsync release 2.113 the logging can be turned on
by using the option --log. You can add access to the logfile with a mount
or similar. See https://docs.docker.com/storage/volumes/"
See also
https://github.com/imapsync/imapsync/issues/283
===========================================================================
======================================================================
Q. Any tips for the Docker Mailcow distribution?
R. With the Mailcow distribution, imapsync is running inside a Docker
@ -116,6 +121,40 @@ See the original Mathilde's comment:
https://github.com/imapsync/imapsync/issues/201#issuecomment-559500077
======================================================================
Q. How can I build my own image?
===========================================================================
===========================================================================
R. Use the following command to build the image:
docker build -t gilleslamiral/imapsync .
with the Dockerfile in the current directory.
The latest Dockerfile can be found at
https://imapsync.lamiral.info/INSTALL.d/Dockerfile
or
https://hub.docker.com/r/gilleslamiral/imapsync
or
https://imapsync.lamiral.info/INSTALL.d/INSTALL.Docker_build.txt
======================================================================
Q. What do you do to update the docker hub image?
R. I do in my imapsync directory:
make docker_build
make docker_upload_docker_hub
Then I go to
https://hub.docker.com/r/gilleslamiral/imapsync
I edit and update the docker "Readme" with the last Dockerfile I used.
I also update INSTALL.Docker_build.txt with the last Dockerfile
I also update the last Dockerfile on the imapsync site with:
make upload_index
======================================================================
======================================================================

9
FAQ.d/FAQ.Dovecot.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Dovecot.txt,v 1.12 2021/04/30 13:02:06 gilles Exp gilles $
$Id: FAQ.Dovecot.txt,v 1.13 2021/12/03 12:51:23 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -13,6 +13,8 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Dovecot.txt
Questions answered in this FAQ are:
Q. Is there imapsync specific options to deal with Dovecot?
Q. Subfolders are not created by imapsync and the error is
"Mailbox doesn't allow inferior mailboxes"
@ -28,6 +30,11 @@ Q. Imapsync encounters timeouts and Dovecot log says
Now the questions again with their answers.
=======================================================================
Q. Is there imapsync specific options to deal with Dovecot?
R. Dovecot is ok with imapsync, nothing special is needed, just the
basic option, the credentials, are needed.
=======================================================================
Q. How to migrate to Dovecot with an admin/MasterUser account?

18
FAQ.d/FAQ.Duplicates.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Duplicates.txt,v 1.24 2021/07/04 22:59:56 gilles Exp gilles $
$Id: FAQ.Duplicates.txt,v 1.25 2021/10/06 20:21:13 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -12,6 +12,10 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
Questions answered in this FAQ are:
Q. Without imapsync, I made several copies that partially failed and it
ended with many duplicates/triplicates messages or more. Can I clean
up the account with imapsync and how?
Q. How does imapsync identify messages and duplicates?
Q. How can I know if imapsync will generate duplicates on a second run?
@ -30,6 +34,14 @@ Q. How can I remove duplicates in a lone account?
Now the questions again with their answers.
=======================================================================
Q. Without imapsync, I made several copies that partially failed and it
ended with many duplicates/triplicates messages or more. Can I clean
up the account with imapsync and how?
R. Yes.
See the Q/R "How can I remove duplicates in a lone account?" below.
=======================================================================
Q. How does imapsync identify messages and duplicates?
@ -260,6 +272,8 @@ R. In order to remove duplicates in a lone account, just run imapsync
on the same account as source and destination, plus the
option --delete2duplicates, ie, with
host1 == host2, user1 == user2, password1 == password2
imapsync ... --delete2duplicates
=======================================================================
=======================================================================

46
FAQ.d/FAQ.Emptying.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Emptying.txt,v 1.11 2020/10/13 16:24:43 gilles Exp gilles $
$Id: FAQ.Emptying.txt,v 1.12 2021/09/09 11:53:53 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -13,10 +13,18 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Emptying.txt
Questions answered in this FAQ are:
Q. How to delete all emails of all folders of an account with imapsync?
Q. How to delete all folders of an account with imapsync?
Q. How to delete some folders of an account with imapsync?
Q. How to revert a sync that went to the wrong account?
Q. How to delete emails older than 90 days with imapsync?
Q. How to delete emails matching a certain criterium with imapsync?
Now the questions again with their answers.
=======================================================================
@ -27,11 +35,11 @@ R. Use the same account on both sides with option --delete1
Option --noexpungeaftereach is just to speed up the deletions.
Example (real example):
Example:
imapsync \
--host1 test.lamiral.info --user1 empty --password1 secret \
--host2 test.lamiral.info --user2 empty --password2 secret \
--host1 test.lamiral.info --user1 foo --password1 secret \
--host2 test.lamiral.info --user2 foo --password2 secret \
--delete1 --noexpungeaftereach
=======================================================================
@ -90,5 +98,35 @@ R. If all the wrong emails have some common characteristic that is not
If the wrongly emails messages synced have no special common
characteristic then take an handkerchief and cry.
=======================================================================
Q. How to delete emails older than 90 days with imapsync?
R. Use the same account on both sides with option --delete1
--noexpungeaftereach --minage 90
Example:
imapsync \
--host1 test.lamiral.info --user1 foo --password1 secret \
--host2 test.lamiral.info --user2 foo --password2 secret \
--delete1 --noexpungeaftereach --minage 90
=======================================================================
Q. How to delete emails matching a certain criterium with imapsync?
R. Use the --search option to select the messages you want to delete
and then use the same account on both sides with the suplementary
options --delete1 --noexpungeaftereach
To select the messages read the document FAQ.Messages_Selection.txt
at https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
Example:
imapsync \
--host1 test.lamiral.info --user1 foo --password1 secret \
--host2 test.lamiral.info --user2 foo --password2 secret \
--delete1 --noexpungeaftereach --search "SENTBEFORE 31-Dec-2010"
=======================================================================
=======================================================================

42
FAQ.d/FAQ.Exchange.txt
View file

@ -1,5 +1,5 @@
$Id: FAQ.Exchange.txt,v 1.75 2021/05/25 10:23:33 gilles Exp gilles $
$Id: FAQ.Exchange.txt,v 1.79 2021/12/17 11:59:48 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -31,6 +31,9 @@ Q. Exchange fails with "User is authenticated but not connected".
Q. Exchange fails with "BAD Command received in Invalid state".
Q. Exchange fails with "NO Server Unavailable. 15" and imapsync says
"could not append ... NO Server Unavailable. 15"
Q. From XXX to Exchange 2013, read receipts are all resent again
after a sync. Even for old messages. How can I fix that?
@ -201,6 +204,14 @@ R2. Miguel Alameda reported understanding and solving this issue
like this, the context was admin/authuser:
"The admin user had not permission in the target mailbox."
R3. Jason from magiclamp.net adds this tip (2021_11_23).
"In GoDaddy Exchange admin center, under recipients, you can
double click on a mailbox, and in the "mailbox delegation" scrolling
down to the bottom it shows a input field named "Full Access".
Add the admin account there."
======================================================================
Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that?
@ -298,6 +309,35 @@ R2. This message might also happen when authenticating with ssl and tls.
It won't work, you can't do a direct double encryption, few
servers allow that.
======================================================================
Q. Exchange fails with "NO Server Unavailable. 15" and imapsync says
"could not append ... NO Server Unavailable. 15"
R1. The problem could be a mailbox quota limitation.
In the Exchange log file, if you see many
"Microsoft.Exchange.Data.Storage.QuotaExceededException",
it looks like "Quota Exceeded" could be the issue.
The solution is to increase the quota for this mailbox.
Quoting
https://docs.microsoft.com/en-us/exchange/recipients/user-mailboxes/storage-quotas?view=exchserver-2019
In the Exchange admin center (EAC), navigate to Recipients > Mailboxes.
In the list of user mailboxes, click the mailbox that you want to change the storage quotas for,
and then click Edit icon, then change "Prohibit send and receive at (GB)", then save your changes.
Or use the Exchange Management Shell, aka PowerShell:
Set-Mailbox -Identity "Joe Healy" -IssueWarningQuota 499gb -ProhibitSendQuota 500gb -ProhibitSendReceiveQuota 500gb -UseDatabaseQuotaDefaults $false
R2. A user fixed it by enabling IMAP logging on Exchange which is
quite crazy as a solution. But why not, here it is:
https://www.mantisbt.org/forums/viewtopic.php?p=68835&sid=8b1ecd2d976e85218fbf33fcd38c209d#p68835
by TerranceM
Set-ImapSettings -Server <server-name> -ProtocolLogEnabled $true
======================================================================
Q. From XXX to Exchange 2013, read receipts are all resent again
after a sync. Even for old messages. How can I fix that?

80
FAQ.d/FAQ.Fun_Things.txt Normal file
View file

@ -0,0 +1,80 @@
#!/bin/cat
$Id: FAQ.Fun_Things.txt,v 1.2 2021/10/15 13:37:05 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Fun_Things.txt
=======================================================================
Imapsync fun things to do with
=======================================================================
Questions answered in this FAQ are:
Q. How can I have a daily digest of my mailbox?
Now the questions again with their answers.
=======================================================================
Q. How can I have a daily digest of my mailbox?
R. Use the same account as source and destination (not mandatory)
imapsync \
--host1 imap.example.com --user1 foo --password1 secret \
--host2 imap.example.com --user2 foo --password2 secret \
--folder INBOX --maxage 1 --dry --nodry1 --debugcontent \
--truncmess 1000 --f1f2 INBOX=Unused
Some explanations on the options suggested:
--folder INBOX to only focus on the folder INBOX
--maxage 1 to select messages less old than 1 day
--dry --nodry1 to avoid really syncing the messages
--debugcontent to display the content
--truncmess 1000 to display only the first 1000 bytes
--f1f2 INBOX=Unused to force imapsync to do something.
Since it is the same mailbox as source and destination,
imapsync usually doesn't sync the messages as they are
already at the destination. The folder Unused has to
exist, otherwise imapsync won't display the messages.
If you pipe this command with a filter like egrep, it becomes like
magic:
imapsync \
--host1 imap.example.com --user1 foo --password1 secret \
--host2 imap.example.com --user2 foo --password2 secret \
--folder INBOX --maxage 1 --dry --nodry1 --debugcontent \
--truncmess 1000 --f1f2 INBOX=Unused \
| egrep 'From:|To:|Subject:|Date:|=====|msg '
Output example:
$RCSfile: FAQ.Fun_Things.txt,v $ $Revision: 1.2 $ $Date: 2021/10/15 13:37:05 $
msg INBOX/22359 copying to INBOX.Fake (not really since --dry mode) ETA: vendredi 15 octobre 2021-10-15 15:31:38 +0200 CEST 57 s 38/38 msgs left
================================================================================
Subject: test: lei7eeCh4Fulahei
To: <tata@localhost.info>
Date: Mon, 11 Oct 2021 11:56:05 +0200 (CEST)
From: gilles@localhost.info (Gilles LAMIRAL)
================================================================================
msg INBOX/22360 copying to INBOX.Fake (not really since --dry mode) ETA: vendredi 15 octobre 2021-10-15 15:31:38 +0200 CEST 57 s 37/38 msgs left
================================================================================
Subject: test: yahShee3Ohvu9mai
To: <tata@localhost.info>
Date: Mon, 11 Oct 2021 11:56:05 +0200 (CEST)
From: gilles@localhost.info (Gilles LAMIRAL)
================================================================================
Thanks to John Fedewa for this trick!
=======================================================================
=======================================================================

5
FAQ.d/FAQ.Gmail.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Gmail.txt,v 1.80 2021/07/06 02:30:11 gilles Exp gilles $
$Id: FAQ.Gmail.txt,v 1.81 2021/10/06 20:21:26 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -236,7 +236,7 @@ imapsync \
--password2 gmailsecret2 \
--gmail1 --gmail2
Or, replacing what does --gmail1 --gmail2 (values are from imapsync 1.970):
Or, replacing what does --gmail1 --gmail2 (values are from imapsync 2.155):
imapsync \
--host1 imap.gmail.com \
@ -251,6 +251,7 @@ imapsync \
--maxbytesafter 1_000_000_000 \
--automap \
--maxsleep 2 \
--useheader X-Gmail-Received --useheader Message-Id \
--synclabels \
--resynclabels \
--exclude "\[Gmail\]$" \

206
FAQ.d/FAQ.ISP.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.ISP.txt,v 1.9 2018/05/24 11:34:30 gilles Exp gilles $
$Id: FAQ.ISP.txt,v 1.13 2021/12/06 20:02:39 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -9,36 +9,24 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.ISP.txt
Imapsync tips for ISP. Specific issues and solutions.
=======================================================================
* IMAPSync - usage scenario with ISP - by Flávio Zarur Lucarelli.
* IMAPSync - usage scenario with ISP - by Flavio Zarur Lucarelli (https://www.lucanet.com.br)
I thought Id write a quick step by step on my attempts to learn the
imapsync features that matter the most, so it works as we expected in
the cenario in which we use it, which is to migrate customers from
I thought I had to write a quick step by step on my attempts to learn
the imapsync features that matter the most, so it works as we expected
in the scenario in which we use it, which is to migrate customers from
their old ISP to our ISP/email hosting. Thanks to the master Gilles
Lamiral for all his help and hard work.
First of all, remember to use --dry to test things first always and
check the log file to see what would actually happen.
Initially, I used a method where I'd do an exact sync of source to
destination, deleting messages which were in destination, but not
source. I call this "Method 2", below. In this cenario, customer
shouldn't be using the destination account yet. Then, after I switch MX,
I'd do a final sync based on date. The big advantage of this is, you
get an exact sync.
Easier, however, is method 1, where I sync based on message ID, this
way, I can use the same syntax before and after MX change, with no
worries. Only disadvantage, you might not get an exact sync, there
might be some difference in terms of total emails in folders, due to
duplicates, emails that had same message ID in source server.
* Method 1 - sync based on message ID, can use same syntax before and
after MX change
This type of copy keeps destination exactly like source (except, it doesn't
copy duplicates), so use it ONLY before changing your MX to the new server,
as it will delete whatever is in destination which isn't in source.
imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd \
--addheader
--subscribeall --delete2 --delete2folders --addheader
Note: add header adds message ID when it doesn't exist.
@ -46,57 +34,141 @@ This syntax can also be used to sync different source accounts to one
same destination account, simply execute it as many times as desired,
switching source user (user1).
* Method 2 - exact sync source do destination, then sync based on date
after MX change
If you have a list of accounts to sync, use this:
https://imapsync.lamiral.info/examples/sync_loop_unix.sh
My first goal is to have an exact sync of an account from
current/source host to the new/destination host and be able to sync
several times. The --useuid parameter is very important for that
purpose. This is what I use:
Adjust the parameters according to the original syntax which I
explained above.
imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \
--useuid --delete2 --delete2folders
The Sync loop Unix script now outputs an error log, which can be
useful.
This makes it so imap.myisp.com (destination) is an exact copy of the
account at imap.gmail.com (source). This is not a problem, since the
user is not using the new host yet. ]You can check Imapsync log files
and surely you will see the final difference should be 0. Check also
for any possible errors in the log (search for "error").
We also have shell scripts to check for errors, such as:
The second goal is to lower the TTL (ex: 5 min) for the host
associated with the MX record, in the domain's DNS server. Let's say
customer has a host mail which his MX points to, with a high TTL
(usually 1 hour). Lower it to 5 min so that, when you change the MX,
it propagates faster.
- Accounts over quota
grep -R "EXIT_OVERQUOTA" *.txt
- Login error
grep -R "Error login" *.txt
- Check just critical errors
egrep 'EXIT_AUTHENTICATION_FAILURE|EXIT_CONNECTION_FAILURE|EXIT_TLS_FAILURE|EXIT_OVERQUOTA' *.txt
- Check all sorts of errors (can be too much info)
grep 'Exiting with return value' *.txt | grep -v 'Exiting with return value 0'
Also, I like to check log file sizes and compare account sizes on
source and destination.
--
Back to it.
After you do a full sync, check that everything is consistant and
let's proceed to the final stage.
Lower the TTL (ex: 5 min) for the host associated with the MX record,
in the domain's DNS server. Let's say customer has a host mail which
his MX points to, with a high TTL (usually 1 hour). Lower it to 5 min
so that, when you change the MX, it propagates faster.
When comes time to switch over to the new host, do a final sync with
above syntax, before changing the MX. Then, change the MX and tell
your users to start using exclusively the new host.
A few hours after the MX change, we will run Imapsync again. We have
to start preserving emails users move or flag in the new host, which
to start preserving emails users move or flags in the new host, which
they started using, so we can't do an exact sync anymore.
The best solution for me was to Sync any new emails (maxage:1) from
source (that could arrive in source even after MX change, due to
cache) and delete such emails from source server. This way, customer's
mailbox is still intact on the source server, except new emails, which
get synced to new server and deleted from source.
One simple solution is the same syntax as before, but without
--delete2 and --delete2folders.
imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \
--folder INBOX --useuid --noexpungeaftereach --skipemptyfolders --maxage 1 --delete1
So:
I personally prefer to keep a copy of users box intact in source, but
if that's not an issue for you, you can remove --folder INBOX and even
--maxage, but then, all emails in source will be deleted. You can use
--maxage 1 with --delete1, however, for all folder (without specifying
--folder INBOX), so only any new email that arrives at source is
copied to destination and deleted from source.
imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd \
--subscribeall --addheader
My next goal was to automate the process, so I followed this advice:
http://imapsync.lamiral.info/examples/sync_loop_unix.sh
The only issue with that syntax is, it would simply copy what is in
source that is not in destination. For example, user might have moved
emails to some sub-folder in host2. However, if you do this once, one
hour after MX change, for example, it's comprehensible that
destination is not exactly as the source anymore.
What we've been doing, especially in the case of huge migrations,
since our destination server auto-archives emails in sub-folders and
changes the hierarchy, is syncing just the last day or few days of
email and just of INBOX.
So we do the sync initially explained in this doc, before the MX
change. Then we auto-archive accounts and after auto-archiving, we
change MX and do a final sync using maxage.
Something like this:
imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd \
--subscribeall --addheader --maxage 1
If your last full sync was 2 days ago, then use --maxage 2
Goal is to get all the new emails (2 last days) from source.
Another alternative for syncing after the MX change, which Lamiral
suggests, involves deleting emails from source from the last day.
I prefer the method explained above. Anyway, here is this other cenario...
Something like this:
imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \
--folder INBOX --noexpungeaftereach --skipemptyfolders --maxage 1 --delete1
This way, any new email that arrived at source is copied to
destination and deleted from source. Without the --delete1 option,
messages moved on host2 by the user are coming back from host1, and
flags changed on host2 are also reset to what they are on host1.
However, as I wrote before, if you run it just once without --delete1,
perhaps not that bad.
*** Other tips for ISPs
1) Sync entire account into 1 folder of another account
imapsync --host1 xxx --user1 user1@domain.com --password1 secret1 --ssl1 \
--host2 yyy --user2 user2@domain.com --password2 secret2 --ssl2 \
--addheader --subfolder2 "otheraccountfolder" --delete2 --delete2foldersonly /otheraccountfolder/
Above, user must not be using such destination folder in destination
account until you finish syncing.
I've used it without --delete2 and --delete2foldersonly, as it caused
problems for us, of that folder being unsubscribed in destination...
Don't think it should be a problem as this copy is done usually
without using really using the folder in host2 until you instruct him
to.
Another nice scenario, sync just one folder hierarchy of an account to
another folder of another account.
--subfolder1 test --subfolder2 test2 --delete2 --delete2foldersonly /test2/ --addheader
Also, very useful is to be move emails older than one year, from an
account to another one, to offload the source account.
imapsync ... --subfolder2 "account_user1" --addheader --minage 365 --delete1
So everything older than 365 days is deleted from source. You can even
specify same account for source and recipient, if you want to "delete
older than x days" for the account itself, without moving old emails
to another account.
2) Translate folder names
I also ended up requiring a regex to translate folder names. On the
old server (Gmail), Sent items were in a folder called [Gmail]/E-mails
@ -110,17 +182,21 @@ So this was added:
--regextrans2 "s,Lixeira,Trash," \
--regextrans2 "s,Rascunhos,Drafts,"
*** Other cenarios
Option --automap is also good to do most of the translate job.
- Sync entire account into 1 folder of another account
3) Folders to exclude, to speed up syncs, etc.
imapsync --host1 xxx --user1 user1@domain.com --password1 secret1 --ssl1 \
--host2 yyy --user2 user2@domain.com --password2 secret2 --ssl2 \
--useuid --subfolder2 otheraccountfolder --delete2 --delete2foldersonly /otheraccountfolder/
--exclude "^Backups" --exclude "(?i)spam" --exclude "(?i)junk" --exclude "(?i)trash" --exclude "(?i)lixeira" --exclude "(?i)Deleted Items"
Above is based on message UID, advantage of no dupes, however, user
must not be using such destination folder in destination account until
you finish syncing.
4) Delete older than 1 year from an account.
Use the account itself as source and destination and use:
--minage 365 --delete1
5) Check size of folders in an account. Use Imapsync with same account as source and destination, with this parameter:
--justfoldersizes
=======================================================================
=======================================================================

41
FAQ.d/FAQ.Logfile.txt Normal file
View file

@ -0,0 +1,41 @@
$Id: FAQ.Logfile.txt,v 1.2 2021/07/23 12:27:31 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Logfile.txt
======================================================================
Imapsync log file
======================================================================
Questions answered in this FAQ are:
Q. I have these error "could not be fetched", like thos one:
Err 1/1: - msg INBOX/17219 {0} S[36592] F[] I[16-Jul-2021 12:04:15 +0000] could not be fetched
What can I do about it?
Now the questions again with their answers.
======================================================================
Q. I have these error "could not be fetched", like thos one:
Err 1/1: - msg INBOX/17219 {0} S[36592] F[] I[16-Jul-2021 12:04:15 +0000] could not be fetched
What can I do about it?
R. Explanation of a log line:
- msg INBOX/17219 {0} S[36592] F[] I[16-Jul-2021 12:04:15 +0000]
INBOX = folder
17219 = UID
{0} = got 0 bytes
S[36592] = should have got 36592 bytes (RFC size announced)
F[] = Flags, none here
I[16-Jul-2021 12:04:15 +0000] = Internal date
Usually this error doesn't appear on next runs, at least not for the
same messages. So rerun imapsync until all messages are synced.
======================================================================
======================================================================

8
FAQ.d/FAQ.Massive.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Massive.txt,v 1.31 2020/10/15 01:53:52 gilles Exp gilles $
$Id: FAQ.Massive.txt,v 1.32 2021/11/01 11:30:48 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -56,9 +56,9 @@ R2. To estimate the presync period, consider the mean imapsync transfer
rate to be around 340 Kbytes/s, ie, 2.8 Mbps, no matter the local
link bandwidth.
It's a mean, measured upon various different syncs, coming from
the online service /X where the network card flow rate is 100 Mbps
and the provider bandwidth is also 100 Mbps. Max seen is a transfer
at 21 Mbps.
the online service /X where the network card flow rate is 200 Mbps
(Rx 100 Mps, Tx 100 Mps) and the provider bandwidth is also 200 Mbps.
The maximum global rate seen is 22 MiB/s (176 Mbps).
At 340 Kbytes/s,
1 TB to transfer and one sync at a time will end in 35 days (1024^3/340/3600/24).

146
FAQ.d/FAQ.Messages_Too_Big.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Messages_Too_Big.txt,v 1.5 2021/06/22 15:31:39 gilles Exp gilles $
$Id: FAQ.Messages_Too_Big.txt,v 1.8 2021/12/16 11:44:26 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -11,76 +11,128 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Too_Big.txt
Questions answered in this FAQ are:
Q. What does imapsync with big messages?
Q. Imapsync reports errors like these ones:
"could not append...Maximum size of appendable message has been exceeded"
"could not append...maximum message size exceeded"
"The most frequent error is ERR_APPEND_SIZE"
What does it mean?
Q. Does imapsync limits big messages?
Q. What are the well known servers messages size limits?
Q. What can I do to transfer messages bigger than what allows the
imap destination server?
Q. The solution --truncmess xxxx doesn't work with Gmail.
What can I do?
Now the questions again with their answers.
=======================================================================
Q. What does imapsync with big messages?
Q. Imapsync reports errors like these ones:
"could not append...Maximum size of appendable message has been exceeded"
"could not append...maximum message size exceeded"
"The most frequent error is ERR_APPEND_SIZE"
What does it mean?
R. It means the message couldn't be transferred to the destination
account because it is too big, imapsync tries to APPEND it to the
server (that's IMAP jargon) but the server refuses it by saying
either "NO. Maximum size of appendable message has been exceeded"
or "NO. maximum message size exceeded".
=======================================================================
Q. Does imapsync limits big messages?
R. No. Imapsync synchronizes messages no matter their sizes.
R. By default, imapsync checks if the host2 imap server announces a
message size limit with "APPENDLIMIT=xxxx" in its CAPABILITY response.
Then imapsync takes this value to set automatically the --maxsize
option, unless --maxsize is already set on the command line and has a
smaller value than the APPENDLIMIT value.
If the host2 imap server doesn't announce any message size limit then
imapsync syncs all messages no matter their sizes.
When a message is copied to the destination server, the destination
server accepts it or not but it does it as a whole, it takes the full
message or nothing. Nothing triggers an error in imapsync.
server accepts it or not. It does it as a whole, it takes the full
message or nothing. The destination server can refuse a too big
message, it then triggers an APPEND error message in imapsync.
* Gmail announces APPENDLIMIT=35651584 (June 2021).
* Office365 announces nothing.
* Dovecot announces nothing.
* Usually imap servers announce nothing.
So far, December 2021, the biggest message transferred by imapsync on
the online /X service is 1.51 GiB. Who can top that?
=======================================================================
Q. What are the well known servers messages size limits?
* Gmail message size limit is 35 MB.
Gmail announces APPENDLIMIT=35651584 (June 2021).
* Office365 message size limit is 45 MB by default
and it can be upped by configuration. See
https://imapsync.lamiral.info/FAQ.d/FAQ.Office365.txt
Q. I see "NO Maximum size of appendable message has been exceeded"
* Exchange message size limit is 10 MB by default
and it can be upped by configuration. See
https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt
Q. I see "NO Maximum size of appendable message has been exceeded"
* Dovecot message size limit is variable. It is defined
by the sysadmin. For example, quota_max_mail_size = 100M
https://doc.dovecot.org/configuration_manual/quota/
* Usually, any IMAP server has a message size limit, defined
by the sysadmin. You discover it sooner or later
=======================================================================
Q. What can I do to transfer messages bigger than what allows the
imap destination server?
R1. A solution is to truncate the message to the maximum size allowed
before copying it to host2. That's a little dirty but it's ok if you
prefer having a truncated message than no message at all. The
attachments after the truncation won't be available but everything
before should be ok. It may also depend on the software tool reading
the email message.
R1. A solution is ask imapsync to truncate the message to the maximum
size allowed before copying it to host2 by using the option
--truncmess.
For example, Gmail currently limits messages to 35651584 bytes,
aka 34 MiB (June 2021),
That's a little dirty but it's ok if you prefer having a truncated
message than no message at all. The attachments after the truncation
won't be available but everything before should be ok. It may also
depend on the software tool reading the email message.
Starting with imapsync release 1.938 option --truncmess xxxx truncates
messages bigger than the given size xxxx, size given in bytes.
imapsync --truncmess 35651584 --appendlimit 1000000000000
To truncate messages to 10 MB, use:
The --appendlimit option is only needed imap servers that offer the
APPENDLIMIT capability, for example, Gmail. Using it doesn't hurt
anyway.
Prior to imapsync release 1.938, instead of --truncmess:
On Linux, use:
imapsync ... --pipemess 'perl -0ne "print substr \$_,0,35651584" '
On windows, use (I'm not sure of, not tested yet):
imapsync ... --pipemess "perl -0ne 'print substr $_,0,35651584' "
imapsync ... --truncmess 10_000_000
R2. A not-so-dirty solution would be to transform the big messages,
replace the attachments with links to the attachments stored elsewhere.
Well, it's not done and it's too much complicated compared to a simpler
solution which is to allow bigger messages on the destination account.
replace the attachments with links to the attachments stored
elsewhere. Well, it's not done and it's too much complicated compared
to a simpler solution which is to allow bigger messages on the
destination account.
As time goes on, disk space is cheaper and cheaper. Don't make the
new user account, the destination account, worst than his previous,
buy disks. Monitor and rule your resources, not the users.
new user account, the destination account, worst than his previous
account: buy disks.
Monitor and rule your hardware resources, serve your users,
not the contrary.
=======================================================================
Q. The solution --truncmess xxxx doesn't work with Gmail.
What can I do?
R. Gmail currently limits messages size up to 35651584 bytes, aka 34
MiB (June 2021) but imapsync knows it so it avoid to transfer these
big messages. Solution, use:
imapsync ... --truncmess 35_651_584 --appendlimit 1_000_000_000_000
Explanation. By default, imapsync checks if the host2 imap server
announces a message size limit with "APPENDLIMIT=xxxx" in its
CAPABILITY response, like Gmail does. Imapsync takes this APPENDLIMIT
value to set automatically the --maxsize option, unless --maxsize is
already set on the command line and has a smaller value than the
APPENDLIMIT value.
1_000_000_000_000 is one TB, I guess you don't have any message of
that size yet.
=======================================================================
=======================================================================

124
FAQ.d/FAQ.Migration_Plan.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Migration_Plan.txt,v 1.7 2021/03/02 22:31:36 gilles Exp gilles $
$Id: FAQ.Migration_Plan.txt,v 1.9 2021/12/19 10:00:32 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -8,23 +8,23 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Migration_Plan.txt
=====================================================================
Imapsync. Suggestions for a good, low impact on users,
well executed email migration plan.
well-executed email migration plan.
=====================================================================
There is two main different scenarios depending on the response to the
following question:
There are two main different scenarios. Choosing which one fits your
context depends on the response to the following question:
Will the imap software tools used by the users use the exact same
Will the imap software tools used by the users use the same
credentials triplet for both imap servers, the old server host1 and
the new server host2?
The credentials triplet is hostname/username/password.
If the answer is yes, ie, clients email tools use the exact same
triplet credentials, then it is possible to perform a migration
without changing anything on the users side. This may be a very time
saving option. But it's a rare condition so I'll describe this
scenario later in this document.
If the answer is yes, ie, clients' email tools use the same triplet
credentials, then it is possible to perform a migration without
changing anything on the users' side. This may be a very time-saving
option. But it's a rare condition so I'll describe this scenario later
in this document.
=====================================================================
@ -40,104 +40,124 @@ Classical scenario, credentials triplets are different on both sides
users are already playing with the new mailboxes on host2, don't
follow this scenario.
* Presync all the mailboxes from the old server host1 to the new
server host2. If the imap server names are going to change their IP
resolution then don't use those names, use names that will always
* Pre-synchronize all the mailboxes from the old server host1 to the
new server host2. If an imap server name is going to change its IP
address, then don't use this name, use a name that will always
match the same imap servers, or use their IP addresses.
Presyncs can usefully be done with --delete2 in order to get an
exact sync. But never use the option --delete2 once users have
started to play with their new account on host2, their play will be
lost on the next presync, or when the MX is changed, since INBOX
will start to receive new messages that are not on host1.
Pre-synchronizations can usefully be done with --delete2 to get an
exact synchronization. But never use the option --delete2 once the
users have started to play with their new account on host2, their
play will be lost on the next synchronization. Don't use --delete2
either when the MX is changed since INBOX will start to receive new
messages that are not on host1 and then removing them is not a good
idea.
* Decide a migration day/hour.
* Repeat the presyncs (with the --delete2 options) daily until the
migration hour. This repeated process will show how long should
take the last sync.
* Repeat the pre-synchronizations (with the --delete2 options) daily
until the migration hour. This repeated process will show how long
should take the last synchronization.
* At the migration hour, cut access to the users to the old server
host1, if you can. Or tell them to not use it anymore.
* Do a last presync exactly like previous ones.
* Do the last pre-synchronization exactly like the previous ones.
* Change the MX, the new messages should start to arrive in the new
imap server host2.
* Wait the TTL value, aka 5 minutes. Now, new messages should
not arrive to the old server host1.
* Wait for the TTL value, aka 5 minutes. Now, new messages should
not arrive at the old server host1.
* Tell the users that the old imap server host1 is down and no
longer available.
* Do a postsync. A postsync is a sync with the following options
--maxage 1 --delete1 --folder INBOX
* Do a post-synchronization. A post-synchronization is a run with the
following options: --folder INBOX --delete1 --maxage 1
This postsync will move the last new messages arrived on host1 to
host2 during the TTL interval and delete them on host1. Do not use
the option --delete2 in a postsync.
This post-synchronization will copy the messages arrived in the
last day (--maxage 1) in the folder INBOX (--folder INBOX) on the
source account, to the destination account. It will also delete
them on host1 (--delete1). It's --delete1, it's not --delete2.
Remember, do not use the option --delete2 in a post
synchronization, as users won't appreciate seeing their newly
arrived messages disappear because of you.
* Give access to new accounts to the users with their new credential
triplet hostname/username/password. If the way to contact users is
email then you should give this long before shutting down the old
server.
by email then you should give them the new credentials long before
shutting down the old server.
* Migration done.
* In case there are still messages arriving at the old imap server
host1, you can perform more postsyncs, ie, syncs every day
with the options:
--maxage 1 --delete1 --folder INBOX
host1, you can perform more post-synchronizations, ie, runs every
day with the options: --maxage 1 --delete1 --folder INBOX
* Increase the TTL of the MX back to its previous value, usually
24 hours, 86400 seconds. You don't want all your email system
to break down completely when your DNS are not available
temporarily, keeping dns values in cache for a 24h is a savvy
practice.
=====================================================================
Lucky scenario, credentials triplets are the same on both sides
=====================================================================
* Decrease the TTL of the MX, as well as the imap hostname resolution,
to 5 minutes (or even less). See FAQ.TTL.txt to understand why.
to 5 minutes (or even less). The document FAQ.TTL.txt explains why.
* Create the new mailboxes on the destination server host2.
* Presync all the mailboxes from the old host1 to the new server host2,
using different names that the ones used by the imap software
clients (use their IP for example).
Presyncs have to be done with --delete2 but never use --delete2
once users have started playing with their new account on host2.
* Pre-synchronize all the mailboxes from the old host1 to the new
server host2, using different names than the ones used by the imap
software clients (use their IP for example). Presyncs have to be
done with --delete2 but never use --delete2 once users have started
playing with their new account on host2.
* Decide a migration day/hour.
* repeat the presyncs (with the --delete2 options) daily until the
migration hour. This repeated process will show how long should
take the last sync.
* Repeat the pre-synchronizations (the runs with the --delete2
options) daily until the migration hour. This repeated process will
show how long should take the last sync.
* At the migration hour, cut access to the users to the old server.
You can do this by changing the imap host1 hostname to a non-imap
server for example, or by changing their password on host1.
* Do a last sync exactly like the presyncs, not using the imap hostname.
* Do the last run exactly like the pre-synchronizations.
* Change also the MX resolution, the new messages should start
to arrive in the new imap server very soon.
* Wait the TTL value, aka 5 minutes. Now, new messages should
not arrive to the old server host1.
* Wait for the TTL value, aka 5 minutes. Now, new messages should
not arrive at the old server host1.
* Do a postsync. A postsync is a sync with the following options
--maxage 1 --delete1 --folder INBOX
* Do a post-synchronization. A post-synchronization is a run with the
following options: --folder INBOX --delete1 --maxage 1
This postsync will move the last new messages arrived on host1 to
host2 during the TTL interval and delete them on host1. Do not use
the option --delete2 in a postsync.
This post-synchronization will copy the messages arrived in the
last day (--maxage 1) in the folder INBOX (--folder INBOX) on the
source account, to the destination account. It will also delete
them on host1 (--delete1). It's --delete1, it's not --delete2.
Remember, do not use the option --delete2 in a post
synchronization, as users won't appreciate seeing their newly
arrived messages disappear because of you.
* Shutdown the old imap server.
* Shut down the old imap server.
* Change the user imap hostname resolution from the old IP of host1
to the IP of the new imap server host2.
* Migration done.
* Increase the TTL of the MX back to its previous value, usually
24 hours, 86400 seconds. You don't want all your email system
to break down completely when your DNS are not available
temporarily, keeping dns values in cache for a 24h is a savvy
practice.
=======================================================================
=======================================================================

236
FAQ.d/FAQ.Office365.txt
View file

@ -1,5 +1,5 @@
$Id: FAQ.Office365.txt,v 1.23 2021/07/06 02:30:01 gilles Exp gilles $
$Id: FAQ.Office365.txt,v 1.33 2022/01/05 11:17:21 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -14,14 +14,24 @@ Questions answered in this FAQ are:
Q. Can I use imapsync to transfer from or to Office365 accounts?
Q. Does IMAPSync support OAUTH2 authentication in Office 365?
Q. Can imapsync work for users that are administrators for
an Office 365 domain? (quick answer: no)
Q. Does imapsync support OAUTH2 authentication for Office365 accounts?
Q. How to sync from Office365 to XXX?
Q. How to sync from XXX to Office365
Q. For Office365 I have double and triple-checked the username and
password spelling but I still get a "LOGIN failed". Any clue?
password spelling but I still get a "LOGIN failed". Any clue?
Q. How can I access an Office365 shared mailbox?
Q. How can I use a shared account as a backup account for several
mailboxes and so avoid spending too many dollars in backup accounts?
Q. Office365 fails with "User is authenticated but not connected".
Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that?
@ -33,6 +43,8 @@ Q. Office365 throttles the sync and says:
"Request is throttled. Suggested Backoff Time: 299961 milliseconds".
What can I do with that?
Q. What are the receive and sending limits of Office365?
Q. The sync fails with many "Trying command when NOT connected!".
What can I do?
@ -68,20 +80,101 @@ Now the questions again with their answers.
======================================================================
Q. Can I use imapsync to transfer from or to Office365 accounts?
R. Yes. But IMAP access to Office365 account is not always allowed by default
so it has to be allowed in the server configuration part.
R0. Yes. But IMAP access to Office365 account is not always allowed
by default so it has to be allowed in the server configuration part.
https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
R1. Imapsync doesn't support OAUTH2 authentication for Office365
accounts yet. That's not totally true, if you have an access token
to access your mailbox then you can authenticate with imapsync this
way:
imapsync ... --oauthaccesstoken1 tokenfile
where "tokenfile" is a file containing the access token.
R2. Enable double-step authentication and configure it but after use
an "app password" with imapsync. Details:
a) Go to
https://account.microsoft.com/security
b) Click on "Advanced Security Options"
Turn "Two-step verification" on. Follow the steps and finish".
c) Then now "App passwords" is available.
Click on "Create a new app password".
Use this password to authenticate with imapsync.
d) Delete this app password when the job with imapsync is finished.
R3. Also, check a license is assigned to that account in Office365.
R4. Microsoft introduced something called "security defaults" which is enabled
by default for new tenants. One of the rules blocks IMAP access as of imapsync.
The funny thing is that you can't disable a single rule of this
security package without buying additional licenses.
Switching the whole thing off allows the IMAP login.
Also, disable double-step authentication on the Azure/Active Directory portal.
See here:
https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults
Thanks to Stephan Buhre for this R4 answer.
R5. Are there special characters in the password?
https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
R6. Triple check the hostname then. Try all of these:
* outlook.office365.com
* imap-mail.outlook.com
* imap.outlook.com
======================================================================
Q. Can imapsync work for users that are administrators for
an Office 365 domain? (quick answer: no)
R. I doesn't seem possible to use imap for administrators, so imapsync
won't be able to work for an administrator's mailbox.
See:
https://github.com/imapsync/imapsync/issues/310#issuecomment-1002396218
https://exhaust.lewiscollard.com/post/146866104/office365-to-migadu-migration/
Solution: as explained in the article above, use Davmail as a proxy to
access this mailbox. http://davmail.sourceforge.net/
Thanks to Lewis Collard for this report and solution.
======================================================================
Q. Does imapsync support OAUTH2 authentication for Office365 accounts?
R. No. But I plan to add it during the year 2021.
R. Yes but partially. Imapsync won't help you getting an access token
but if you have one then you can use it with imapsync this way:
Office365 as source:
imapsync ... --oauthaccesstoken1 tokenfile1
Office365 as destination:
imapsync ... --oauthaccesstoken2 tokenfile2
======================================================================
Q. How to sync from Office365 to XXX?
R0. IMAP is not enabled by default on Office365, see how to enable it:
https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
If IMAP cannot be enabled or if it doesn't work well, you can try DavMail
(http://davmail.sourceforge.net/) which acts as an IMAP gateway
through Outlook Web Access.
R. On Windows, use:
imapsync.exe ... --office1
@ -141,42 +234,65 @@ Use it like this:
It often does some miracles on messages.
======================================================================
Q. For Office365 I have double and triple-checked the username and
password spelling but I still get a "LOGIN failed". Any clue?
Q. For Office365 I have double and triple-checked the username and
password spelling but I still get a "LOGIN failed". Any clue?
R0. Imapsync doesn't support OAUTH2 authentication for Office365 accounts yet.
R. Go to "Q. Can I use imapsync to transfer from or to Office365 accounts?"
R1. Disable double-step authentication, also known as 2-factor,
2-step authentication on the Azure/Active Directory portal.
======================================================================
Q. How can I access an Office365 shared mailbox?
R2. Also, check a license is assigned to that account in Office365.
R. First, create a shared mailbox, for example shared@example.com.
Then give full permissions to a licensed account user@example.com.
Now with imapsync, use the licensed user login with the syntax:
R3. Microsoft introduced something called "security defaults" which is enabled
by default for new tenants. One of the rules blocks IMAP access as of imapsync.
The funny thing is that you can't disable single rules of this security
package without buying additional licenses.
Switching the whole thing off allows the login.0
Also, disable double-step authentication on the Azure/Active Directory portal.
See here:
https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults
Thanks to Stephan Buhre for this R3 answer.
user@example.com\shared@example.com
R4. Are there special characters in the password?
https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
and the password for user@example.com
R5. Triple check the hostname then. Try all of these:
* outlook.office365.com
* imap-mail.outlook.com
* imap.outlook.com
Caveat: Character \ is a special character, so
use double-quotes around the user login, like:
imapsync ... --user1 "user@example.com\shared@example.com"
Sources:
https://adam-hand.com/2017/07/25/connect-a-shared-mailbox-from-o365-to-outlook-via-imap/
https://www.arclab.com/en/kb/email/imap-settings-shared-mailbox-office-exchange-online.html
https://social.technet.microsoft.com/Forums/en-US/336e02ee-6767-4810-90a0-1352bd7cc9e9/office-365-how-to-access-a-shared-mailbox-using-imap-client?forum=onlineservicesexchange
======================================================================
Q. How can I use a shared account as a backup account for several
mailboxes and so avoid spending too many dollars in backup accounts?
R. Use the option --subfolder2 backup_foo
imapsync ... --user2 sharedloginsyntax --subfolder2 backup_foo
To restore, do the reverse with:
imapsync ... --user1 sharedloginsyntax --subfolder1 backup_foo
======================================================================
Q. Office365 fails with "User is authenticated but not connected".
R1. "The message User is authenticated but not connected is due to a
bug in the Office365 server's IMAP implementation. If the client
presents a valid user name but an invalid password, the server
accepts the login, but subsequent commands fail with the
aforementioned error message." Source:
https://unix.stackexchange.com/questions/164823/user-is-authenticated-but-not-connected-after-changing-my-exchange-password
Thanks to James Abbottsmith for this link and explanation at
https://github.com/imapsync/imapsync/issues/32#issuecomment-153561647
R2. Miguel Alameda reported understanding and solving this issue
like this, the context was admin/authuser:
"The admin user had not permission in the target mailbox."
======================================================================
Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that?
R. Office365 supports send/receive max message sizes of up to 150MB
but you need to make changes in your tenant(s) to support it.
@ -204,14 +320,15 @@ Q. The imap connection to Office365 is not working very efficiently,
R. Yes. Try DavMail
http://davmail.sourceforge.net/
I don't use it myself but a user, Yannick Palanque, reported great results using it.
I don't use it myself but a user, Yannick Palanque, reported
great results using it.
======================================================================
Q. The sync fails with many "Trying command when NOT connected!".
What can I do?
R. The --debugimap option can show you more details, especially some
message like:
R. The --debugimap option can show you more details, especially
messages like this one:
"BAD Request is throttled. Suggested Backoff Time: 178755 milliseconds"
In that case, see the next faq item and its fixes.
@ -249,6 +366,29 @@ R2. Play with options --maxbytespersecond or --maxmessagespersecond
I don't know the upper value that avoids the default throttling from 0365
and I guess it changes over time.
======================================================================
Q. What are the receive and sending limits of Office365?
R1. 3600 messages per hour (22 July 2021). Maybe 1200 as imapsync can
be viewed as a single sender. Tell me what you experienced,
I'll update this point.
https://docs.microsoft.com/en-us/office365/servicedescriptions/exchange-online-service-description/exchange-online-limits#receiving-and-sending-limits
In that case, imapsync can adapt to this with (1 message/second = 3600 messages per hour)
imapsync ... --maxmessagespersecond 1
or
imapsync ... --maxmessagespersecond 0.33
R2. I also found "Microsoft theoretically allows for about 300MB of
throughput per user per hour." at
https://www.systools.in/blog/microsoft-office-365-throttling-policy/
In that case, imapsync can adapt to this with (83333 bytes/second = 300 MBytes/hour)
imapsync ... --maxbytespersecond 83333
======================================================================
Q. Office365 refuses to create the folder named "Files" with the error
"NO Folder name is reserved". What happens?
@ -277,12 +417,13 @@ R0. To fix this, add --exclude Files
The host2 account ends up with a folder named
"Files_renamed_by_imapsync", but no complaining.
R1. This folder "Files" seems to be a standard folder in Exchange Online,
but it is not. This folder contains all attachments in every email
that is in the mailbox but without any headers.
This causes some confusion for users as these appear in their search
results as duplicate lines but without the sender details or even
the message body.
R1. This folder "Files" seems to be a standard folder in Exchange Online,
but it is not. This folder contains all attachments in every email
that is in the mailbox but without any headers.
This causes some confusion for users as these appear in their search
results as duplicate lines but without the sender details or even
the message body.
This folder seems to be usually hidden so IMAP clients can’t see it,
but for some reason sometimes it becomes visible.
@ -350,8 +491,8 @@ R3. See also the FAQ FAQ.SSL_errors.txt
Q. From XXX Office365 I get this error message sometimes:
"BAD Command Argument Error 11". What does it mean?
R. This error message comes from the Office365 IMAP server when it
encounters any problem. Most of the time it is one of the following:
R. This error message comes from the Office365 IMAP server when it
encounters any problem. Most of the time it is one of the following:
* Some messages are bigger than the size limit.
45 MB by default on Office365.
@ -365,7 +506,7 @@ R. This error message comes from the Office365 IMAP server when it
* Quota reached. The whole account is full.
It can be upped by configuration.
* You use --synclabels --resynclabels from a previous command line
* You use --synclabels --resynclabels from a previous command line
related to syncing from Gmail to Gmail. Remove them.
* Some messages have some lines too long. Use option --maxlinelength
@ -437,7 +578,8 @@ but previously in Office365, you must do something like the following, using Pow
Note from Betsy Lawlor: You can use global modern authentication with two factor
on Exchange Online (M365) but you must have "AllowBasicAuthImap" on the admin account you are using to migrate the mail.
on Exchange Online (M365) but you must have "AllowBasicAuthImap" on
the admin account you are using to migrate the mail.
Remark: PLAIN authentication is the only way to go with --authuser1 for now.
So don't use --authmech1 SOMETHING with --authuser1 admin_user,
@ -453,7 +595,7 @@ so if two factors authentication is enabled you will no be able to sync
the mail (it will not connect to host2).
OWA (I think this is the two-factor authentication) needs to be disabled
globally and enabled individually on the users you want to.
One account can have the rights to access other mailboxes no matter if admin or
One account can have the right to access other mailboxes no matter if admin or
any other account but needs to have OWA disabled to be able to connect
to the office365 server.
@ -473,8 +615,8 @@ Date: Mon, 1 Jun 2015 17:53:54 +0000
Q. Couldn't create folder [trash] "Mailbox already exists".
R. Some servers take care of character cases in folder names,
some servers do not, like Exchange. Since not respecting the case can merge
two different folders into one then imapsync respects the case.
some servers do not, like Exchange. Since not respecting the case
can merge two different folders into one then imapsync respects the case.
For example, if a host1 server has a folder name called "trash"
and the host2 server already has a folder "Trash" or "TRASH"

102
FAQ.d/FAQ.OnlineUI.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.OnlineUI.txt,v 1.29 2021/05/05 21:38:32 gilles Exp gilles $
$Id: FAQ.OnlineUI.txt,v 1.31 2021/12/24 21:02:52 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -38,47 +38,52 @@ Let's do it anyway, you'll be the judge.
R1. Some figures
Date of this report: 22 April 2021.
Date of this report: 24 December 2021. Happy Christmas!
The online imapsync service /X started 9 January 2017
(1564 days of service).
The online imapsync service /X started 9 January 2017,
5 years ago (1810 days of service).
On average, /X has 51 users per day, each user lunches on average
6 account migrations, from just one launch to many (hundreds).
6 account migrations, from just 1 launch to many (hundreds).
The total volume /X transferred so far is around 210 TiB coming
from nearly 450 000 imap account migrations and 700 million email messages.
The total volume /X transferred so far is around 240 TiB coming
from nearly 520 000 imap account migrations and 810 million email messages.
R2. Pros & Cons
The online imapsync service /X runs on HTTPS only, with a
letsencrypt certificate, a certificate overall rated "A+" at
The online imapsync service /X runs on HTTPS only, with a letsencrypt
certificate, an up to date certificate overall rated "A+" at
https://www.ssllabs.com/ssltest/analyze.html?d=imapsync.lamiral.info
Because of the HTTPS usage, what the users enter in their browser,
the imap logins and passwords, can't be eavesdropped on the network.
Because of the HTTPS usage, what the users enter in their browser, the
imap logins and passwords, can't be eavesdropped on the network.
Imapsync itself takes care of encryption for the imap sessions,
if possible. First, imapsync tries to use SSL on port 993,
then TLS on port 143 if the servers announce TLS, then no encryption at all.
Imapsync itself takes care of encryption for the imap sessions, if
possible. First, imapsync tries to use SSL on port 993, then TLS on
port 143 if the servers announce TLS, then no encryption at all.
Concerning encryption, what is done with the source imap server host1
is independent of what is done with the destination imap server host2.
On the date of 22 April 2021, there is no security problem
detected or reported to me (Gilles LAMIRAL), so far.
Feel free to attack the service and feel free to report any
hole encountered. Have in mind I can watch what you do
from the server-side and take measures if the service suffers from
your acts. Drop me a note before will be fair play.
On the date of 24 December 2021, there is no security problem detected
or reported to me (Gilles LAMIRAL), so far.
As the owner of the service, it could have been 450 000 pairs of
credentials collected and nearly 210 terabytes of email messages.
I haven't kept them but I can't prove I haven't. It's just trust,
like nearly every online service in the universe.
Concerning the Log4j threats, the imapsync server doesn't use Log4j at
all, so it should be safe about this issue.
The imap server certificates are not checked for authenticity
(by default) because too many imap servers are crappy configured
regarding certified certificates.
Feel free to attack the service and feel free to report any hole
encountered. Have in mind that I can watch what you do from the
server-side, and then take measures if the service suffers from your
acts. Drop me a note before will be fair play and I will let you act
as harsh as you can.
As the owner of the service, it could have been 520 000 pairs of
credentials collected and nearly 240 terabytes of email messages.
That's massive. I haven't kept them but I can't prove I haven't. It's
just trust, like nearly every online service in the universe.
Concerning imapsync transfers, the imap server certificates are not
checked for authenticity by default because too many imap servers are
crappy configured regarding their certified certificates.
This default behavior is chosen like this because users of /X
want their emails transferred, instead of being not transferred
@ -88,35 +93,38 @@ I admit that this part, checking imap ssl/tls certificates,
could be improved from my side by including well known
certificates directly in imapsync.
If the imap servers don't honor ssl nor tls, then logins, passwords
and everything will go clear text during the imap transfers.
That's not good at all but what "comforts" me is that if an
imap server does only clear text transfers, then it's also true
for all the imap sessions the account owner encounters,
imapsync is just one of them; the imap servers are then always
unsecured for any imap software client.
If an imap server doesn't honor ssl nor tls, then logins, passwords
and everything will go clear text during the imap transfers. That's
not good at all! But what "comforts" me is that if an imap server does
only clear text transfers, then it's also true for all the imap
sessions the account owner encounters, imapsync is just one of them;
the imap servers are then always unsecured for any imap software
client.
Last point, who could be sure that no cracker cracked the online hosts
and that he isn't currently sniffing the credentials?
Last point, who could be sure that no cracker cracked the online
hosts and that he isn't currently sniffing the credentials?
No one, I'm not sure myself, even if I do take care of that
No one! I'm not sure myself, even if I do take care of that
possibility. So changing the imap accounts passwords after
the sync is a safe and recommended practice!
the sync is a safe and recommended practice! You can even
change the password just after imapsync has started its job,
just after a successful login. This way you know the previously
given password is no longer working just after imapsync has
started its job.
=====================================================================
Q. Does the online service store any sensitive information
like my passwords?
No.
The online passwords are kept by your browser. The goal is
that the next time you run a sync from your browser, all the
parameters are already there without having to re-enter them.
The save is done when you click on the "Sync or resync" green
button. So, if you don't want your browser to keep your password,
enter a wrong one or empty the password field and click
on the "Sync or resync" green button, the wrong password
will be saved in your browser and no sync will be done,
The online passwords are kept by your browser. The goal is that the
next time you run a sync from your browser, all the parameters are
already there without having to re-enter them. The save is done when
you click on the "Sync or resync" green button. So, if you don't want
your browser to keep your password, enter a wrong one or empty the
password field and click on the "Sync or resync" green button, the
wrong password will be saved in your browser and no sync will be done,
since a good login is mandatory to do anything on your mailboxes.
=====================================================================

17
FAQ.d/FAQ.Release_Checklist.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
$Id: FAQ.Release_Checklist.txt,v 1.20 2021/07/06 02:29:51 gilles Exp gilles $
$Id: FAQ.Release_Checklist.txt,v 1.22 2022/01/14 12:46:38 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -12,6 +12,16 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Release_Checklist.txt
Checklist before release a new release:
- Make "imapsync --tests" pass on Linux
- Make "imapsync --tests" pass on Mac
- Make "imapsync --tests" pass on Windows 32 bits
- Make "imapsync --tests" pass on Windows 64 bits
- Make "imapsync --testslive" pass on Linux
- Make "imapsync --testslive" pass on Mac
- Make "imapsync --testslive" pass on Windows 32 bits
- Make "imapsync --testslive" pass on Windows 64 bits
- Add a new section in S/news.shtml reading "rlog imapsync"
- In case of a new dependency, add it to all the INSTALL/* files.
- Generate the README
@ -20,6 +30,11 @@ Checklist before release a new release:
- Read the OPTIONS section of README and read it very slowly
- Read slowly README_Windows.txt
- Read slowly the TUTORIAL_Unix file in html
- Make "rcsdiff imapsync" give nothing relevant and exit 0
- make ci pass.
- Review the newsletter by running:
m4 -P W/ml_announce.in.txt

28
FAQ.d/FAQ.SSL_errors.txt
View file

@ -1,14 +1,14 @@
#!/bin/cat
$Id: FAQ.SSL_errors.txt,v 1.15 2021/06/22 15:35:10 gilles Exp gilles $
$Id: FAQ.SSL_errors.txt,v 1.17 2021/08/01 12:49:00 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
=======================================================================
======================================================================
Imapsync SSL errors
=======================================================================
======================================================================
Questions answered in this FAQ are:
@ -25,11 +25,14 @@ Q. SSL connect attempt failed SSL
SSL connect attempt failed SSL
SSL routines:ssl_choose_client_version:unsupported protocol
Q. fatal SSL error: SSL connect attempt failed with unknown error
SSL wants a read first
Q. How to see the certificate and identify problems in it?
Now the questions again with their answers.
=======================================================================
======================================================================
Q. What are the errors
DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
or
@ -45,11 +48,11 @@ it's often Exchange or Office365. I don't know what exactly happens.
Those errors happen more often on Windows than on Linux.
=======================================================================
======================================================================
Q. What can I do to avoid those "SSL read/write errors"?
R0. Windows users: upgrade to imapsync.exe release 1.836 (or next ones)
Those errors don't appear with recent releases, post 1.836
Those errors appear less often with imapsync releases post 1.836
R1. Remove all ssl/tls encryption
@ -70,7 +73,7 @@ R5. Set up a ssltunnel proxy to the host.
Read the file FAQ.Security.txt for an example to set up
a ssltunnel proxy.
=======================================================================
======================================================================
Q. SSL connect attempt failed SSL
routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
or
@ -145,8 +148,13 @@ when they receive a TLS version 1.1 request.
In this case setting the version
to 'SSLv23:!SSLv2:!SSLv3:!TLSv1_1:!TLSv1_2' might help.
======================================================================
Q. fatal SSL error: SSL connect attempt failed with unknown error
SSL wants a read first
=======================================================================
R. If you're using --ssl1 or --ssl2, try instead --tls1 or --tls2
======================================================================
Q. How to see the certificate and identify problems in it?
R. Use the command openssl like this:
@ -221,5 +229,5 @@ ggEBANuPNbYLPMZ4vPa9NBoHAUdIXqpi0eqdXMXd2sT+qRmqxS5ihr999BHOROcr
...
=======================================================================
=======================================================================
======================================================================
======================================================================

209
FAQ.d/FAQ.Security.txt
View file

@ -1,45 +1,122 @@
#!/bin/cat
# $Id: FAQ.Security.txt,v 1.23 2021/06/22 15:32:10 gilles Exp gilles $
# $Id: FAQ.Security.txt,v 1.26 2021/07/11 12:11:38 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Security.txt
=======================================================================
Imapsync tips about security. Issues and solutions.
=======================================================================
SSL: going to encryption before the imap session start.
It is on port 993.
Questions answered in this FAQ are:
TLS: going to encryption after the imap session start but before
the credential are sent. It is on port 143.
Q. Is running this program a secure method of transferring emails?
Are there any security concerns?
Q. Does imapsync refer to SSL security protocols with --ssl1 and --ssl2
and does it refer to TLS security protocols with the --tls1 and --tls2
options?
Short answer: No.
Q. I noticed that the online UI has no option for TLS/SSL.
Is this secure?
Is this more secure than using the .bat file on my computer?
Q. Are transferred emails/attachments stored on any other
server/location aside from my originating/destination server(s)?
Q. Other than changing passwords on the originating/destination email
accounts once the relevant emails have been moved,
are there any other security tips I should know?
Q. I need to transfer mail from an imap server to another imap server.
Which ports need to be open on the firewall to make this possible?
Q. Does imapsync support the IMAP command STARTTLS?
Q. Does imapsync support IMAP over SSL/TLS (IMAPS)?
Q. How can I test an ssl/tls imap connection without imapsync?
Q. How can I manually test login using --ssl?
Q. Imapsync used to use SSL_VERIFY_PEER now it uses SSL_VERIFY_NONE.
How can I change this back to the more secure SSL_VERIFY_PEER?
Q: How can I have an imaps server?
Now the questions again with their answers.
=======================================================================
Q. Is running this program a secure method of transferring emails?
Are there any security concerns?
R. Well, it depends. Use encryption, secure the access to the host running
imapsync and everything shall be safe.
R. Well, it depends. Use encryption and secure access to the host running
imapsync then everything shall be safe.
=======================================================================
Q. I noticed that the online UI has no option for TLS/SSL
Is this secure? Is this more secure that using the .bat file
on my computer?
Q. Does imapsync refer to SSL security protocols with --ssl1 and --ssl2
and does it refer to TLS security protocols with the --tls1 and --tls2
options?
Short answer: No.
R1. The online UI does TLS/SSL imap connections if the imap
servers support TLS/SSL. If you are concerned by security then
using the .bat file or .sh on your computer should be more secure
since you can examine and secure it by yourself, no matter high is
your paranoid spirit. The online UI security is mine, I am
concerned by security, not to the upmost high level possible
but I won't give you direct access to the host to discover my level.
With a good guy spirit, feel free to try to break the online UI
security and report me any security issue you encounter, I'll do my
best to fix them as soon as possible. Drop me a note before
starting because I may detect a sort of abuse and ban
definitively your IPs.
R. No.
Imapsync behaviour:
--ssl: Goes to encryption before the imap session start.
The connection is on port 993.
Can use the TLS or SSL security protocols.
Fails if encryption cannot be established.
--tls: Goes to encryption after the imap session start but before the
credential are sent.
The connection is on port 143.
Can use the TLS or SSL securityprotocols.
Fails if encryption cannot be established.
Explanation:
To force transferring emails over an encrypted connection, you can use
the imapsync parameters --ssl1 and --ssl2 or the parameters --tls1 and
--tls2, they all force the connection to be encrypted by the security
protocols series SSL/TLS.
In a security context, SSL refers to all deprecated Secure Sockets
Layer protocols. TLS refers to the SSL successors, Transport Layer
Security protocols. But TLS 1.0 and TLS 1.1 are also deprecated.
Current endorsed versions of TLS are only TLS 1.2 and TLS 1.3 (July 2021).
The whole story is detailed here:
https://en.wikipedia.org/wiki/Transport_Layer_Security
Options --ssl1 and --ssl2 are a little more paranoid than --tls1 and
--tls2 because they verify that the hostname of the certificate is the
same as the one used by imapsync. Other than that, the security
behavior is the same.
=======================================================================
Q. I noticed that the online UI has no option for TLS/SSL.
Is this secure?
Is this more secure than using the .bat file on my computer?
R1. The online UI does TLS/SSL imap connections if the imap servers
support TLS/SSL.
If you are concerned about security then using the .bat file or .sh on
your computer should be more secure since you can examine and secure
it by yourself, no matter high is your paranoid spirit compared to
mine.
The online UI security is mine, I am concerned by security, not to the
utmost high level possible but I won't give you direct access to the
host to discover my level. With a good guy spirit, feel free to try
to break the online UI security and report me any security issue you
encounter, I'll do my best to fix them as soon as possible. Drop me a
note before starting because I may detect a sort of abuse and ban your
IPs.
=======================================================================
Q. Are transferred emails/attachments stored on any other
@ -48,15 +125,15 @@ Q. Are transferred emails/attachments stored on any other
R. No!
=======================================================================
Q. Other than changing passwords on the originating/destination email
accounts once the relevant emails have been moved,
Q. Other than changing passwords on the originating/destination email
accounts once the relevant emails have been moved,
are there any other security tips I should know?
R. Secure the host where imapsync is running since credentials are
on it.
R. Yes. Secure the host where imapsync is running since credentials
are on it.
=======================================================================
Q. I need to transfer mail from an imap server to an other imap server.
Q. I need to transfer mail from an imap server to another imap server.
Which ports need to be open on the firewall to make this possible?
R. It depends. Open either:
@ -64,29 +141,31 @@ R. It depends. Open either:
* port 993 in ssl mode (--ssl1 or --ssl2)
=======================================================================
Q. Does imapsync support IMAP TLS?
Q. Does imapsync support the IMAP command STARTTLS?
R1. Yes.
Use --tls1 and/or --tls2 options:
--tls1 tells imapsync to use tls on host1.
--tls2 tells imapsync to use tls on host2.
Use --tls1 and --tls2 options:
R2. Since imapsync release 1.755 TLS mode is activated automatically
if the server announce it supports it, STARTTLS inside the response
to CAPABILITY, and if neither --notls nor --ssl is explicitely
mentioned on the command line options.
--tls1 tells imapsync to use STARTTLS on host1.
--tls2 tells imapsync to use STARTTLS on host2.
R2. Since imapsync release 1.755 STARTTLS mode is activated
automatically when the server announces that it supports it by
listing STARTTLS inside the response to the CAPABILITY command.
If either --notls or --ssl are explicitly mentioned on the
command-line options then STARTTLS won't be done.
=======================================================================
Q. Does imapsync support IMAP over SSL (IMAPS)?
Q. Does imapsync support IMAP over SSL/TLS (IMAPS)?
R. Yes natively since release 1.161.
Still, there are 2 ways, at least, to use ssl:
a) Use native --ssl1 and/or --ssl2 options
--ssl1 tells imapsync to use ssl on host1.
--ssl2 tells imapsync to use ssl on host2.
--ssl1 tells imapsync to use ssl/tls on host1.
--ssl2 tells imapsync to use ssl/tls on host2.
b) Use stunnel
@ -108,18 +187,18 @@ b) Use stunnel
If the local port 143 is already taken then use a free one,
like 10143 for example.
c) Other example for accessing gmail with no local root access
to open port 143
c) Another example for accessing Gmail with no local root access
to open port 143:
stunnel3 -P '' -c -d 9993 -r imap.gmail.com:993 -f
Then, to access gmail as host2 use:
Then, to access Gmail as host2 use:
imapsync ... --host2 localhost --port2 9993 --nossl2
=======================================================================
Q.How can I test an ssl imap connection without imapsync?
Q.How can I test an ssl/tls imap connection without imapsync?
R1. Use either ncat or telnet-ssl or openssl commands like in the
following examples with imap.gmail.com server:
@ -133,37 +212,39 @@ R1. Use either ncat or telnet-ssl or openssl commands like in the
The previous commands are interactive, hit ctrl-c
to finish them. If you want to finish automatically, then use:
{ sleep 2; echo "a logout"; sleep 1; } | ncat --ssl -C imap.gmail.com 993
{ sleep 2; echo "A LOGOUT"; sleep 1; } | ncat --ssl -C imap.gmail.com 993
=======================================================================
Q. How can I manually test a login via ssl?
Q. How can I manually test login using --ssl?
R. Use either ncat or telnet-ssl or openssl commands like in the
following examples with imap.gmail.com server:
ncat --ssl -C imap.gmail.com 993
telnet-ssl -z ssl imap.gmail.com 993
openssl s_client -crlf -connect imap.gmail.com:993
Tipical dialog for an imap LOGIN command:
Typical dialog for an imap LOGIN command:
* OK Gimap ready for requests from 78.196.254.58 q1mb175739668wix
a LOGIN "gilles.lamiral@gmail.com" "secret"
A1 LOGIN "gilles.lamiral@gmail.com" "secret"
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE ... ESEARCH
a OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success)
b LOGOUT
A1 OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success)
A2 LOGOUT
* BYE LOGOUT Requested
b OK 73 good day (Success)
A2 OK 73 good day (Success)
The client part you have to type is
a LOGIN ...
b LOGOUT
A1 LOGIN ...
A2 LOGOUT
while replacing ... by your credentials values,
other lines are the server responses.
=======================================================================
Q.How can I test an tls imap connection without imapsync?
Q. How can I test a STARTTLS imap connection without imapsync?
R1. Use openssl command like the following example with
an outlook.office365.com server:
@ -177,13 +258,12 @@ to finish them. If you want to finish automatically, then use:
Replace outlook.office365.com with your imap server name.
======================================================================
Q. Imapsync used to use SSL_VERIFY_PEER now it uses SSL_VERIFY_NONE.
How can I change this back to the more secure SSL_VERIFY_PEER?
R2. After imapsync 1.673,
R. After imapsync 1.673,
to set SSL_verify_mode to SSL_VERIFY_PEER on host1
and SSL_verify_mode to SSL_VERIFY_NONE on host2
@ -191,18 +271,20 @@ R2. After imapsync 1.673,
--sslargs1 SSL_verify_mode=1 \
--sslargs2 SSL_verify_mode=0
See perldoc IO::Socket::SSL for all possibilities, also at
See "perldoc IO::Socket::SSL" for all possibilities, also at
http://search.cpan.org/perldoc?IO%3A%3ASocket%3A%3ASSL
It might be possible you need an extra:
--sslargs1 SSL_ca_file=/etc/ssl/certs/ca-certificates.crt
It might be possible you need an extra option
--sslargs1 SSL_ca_file=/etc/ssl/certs/ca-certificates.crt
to help the ssl software verifying the server certificate.
The file ca-certificates.crt may be elsewhere on your system, even
named differently.
The imap server certificates are not checked for authenticity
(by default) because too many imap servers are crappy configured
regarding certified certificates.
by imapsync by default because too many imap servers are crappy
configured regarding certified certificates.
This default behavior is chosen like this because users
want their emails transferred, instead of being not transferred
@ -211,12 +293,13 @@ because of an incompetent imap server sysadmin.
I admit that this part, checking imap ssl/tls certificates,
could be improved from my side by including well known
certificates directly in imapsync.
Drop me a note to encourage me.
Drop me a note to encourage me, I'm lazy.
=======================================================================
Q: How to have an imaps server?
Q: How can I have an imaps server?
R. Three solutions.
R.
a) Install one
b) or use stunnel :

31
FAQ.d/FAQ.Yahoo.txt
View file

@ -1,5 +1,5 @@
$Id: FAQ.Yahoo.txt,v 1.12 2021/05/04 11:04:51 gilles Exp gilles $
$Id: FAQ.Yahoo.txt,v 1.13 2021/12/03 12:38:58 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
@ -18,24 +18,6 @@ Q. Synchronizing from XXX to Yahoo
Now the questions again with their answers.
Some explanation about the options given later:
https://imapsync.lamiral.info/README
...
--noabletosearch : Makes --minage and --maxage options use the internal
dates given by a FETCH imap command instead of the
"Date:" header. Internal date is the arrival date
in the mailbox.
--noabletosearch equals --noabletosearch1 --noabletosearch2
--regexmess reg : Apply the whole regex to each message before transfer.
Example: 's/\000/ /g' # to replace null by space.
--regexmess reg : and this one, etc.
I'm not sure anymore about the need of --noabletosearch with Yahoo, maybe
they fixed the issue about the standard imap searching command SEARCH
they didn't honor before.
=======================================================================
Q. Synchronizing from Yahoo to XXX
@ -49,8 +31,7 @@ Example:
--password1 billysecreta \
--host2 XXX \
--user2 Billy@other.com \
--password2 billysecretb \
--noabletosearch
--password2 billysecretb
You also need to go to Yahoo, security and enable
@ -90,7 +71,7 @@ R. Example:
--host2 imap.mail.yahoo.com \
--user2 Billy@yahoo.com \
--password2 billysecretb \
--noabletosearch --regexmess "s{(?<![\n])\z}{\r\n}gxms"
--regexmess "s{(?<![\n])\z}{\r\n}gxms"
You wonder why there is this ugly regex, what it does and where does it
come from?
@ -101,5 +82,11 @@ when copying some messages to Yahoo.
It adds a final \r\n on a message if a final \r\n is missing.
See https://github.com/imapsync/imapsync/issues/209
https://imapsync.lamiral.info/README
...
--regexmess reg : Apply the whole regex to each message before transfer.
Example: 's/\000/ /g' # to replace null by space.
--regexmess reg : and this one, etc.
=======================================================================
=======================================================================

12
FAQ.d/htaccess.txt
View file

@ -1,5 +1,5 @@
# $Id: htaccess.txt,v 1.31 2021/05/29 08:09:11 gilles Exp gilles $
# $Id: htaccess.txt,v 1.32 2021/07/23 12:38:19 gilles Exp gilles $
AddDescription "<b>Back</b> to Imapsync main page." ..
@ -40,13 +40,15 @@ AddDescription "<b>Flags tips and issues</b>." FAQ.Flag
AddDescription "<b>General Data Protection Regulation</b> (GDPR)." FAQ.GDPR.txt
AddDescription "<b>General</b> and <b>pot-pourri</b> issues" FAQ.General.txt
AddDescription "<b>Gmail</b> accounts." FAQ.Gmail.txt
AddDescription "<b>GoDaddy</b> accounts." FAQ.GoDaddy.txt
AddDescription "<b>IceWarp</b> accounts." FAQ.IceWarp.txt
AddDescription "<b>ISP tips</b>." FAQ.ISP.txt
AddDescription "<b>Kerio</b> accounts." FAQ.Kerio.txt
AddDescription "<b>MailEnable</b> accounts." FAQ.MailEnable.txt
AddDescription "<b>Massive/bulk migrations</b>." FAQ.Massive.txt
AddDescription "<b>Memory issues</b>." FAQ.Memory.txt
AddDescription "<b>Migration Plan</b>." FAQ.Migration_Plan.txt
AddDescription "<b>Massive and bulk</b> migrations." FAQ.Massive.txt
AddDescription "<b>Log file, content, errors and tips</b>." FAQ.Logfile.txt
AddDescription "<b>Memory</b> issues." FAQ.Memory.txt
AddDescription "Migration <b>Plan</b>." FAQ.Migration_Plan.txt
AddDescription "<b>Office365</b> accounts." FAQ.Office365.txt
AddDescription "<b>The Online UI service Q&R</b>." FAQ.OnlineUI.txt
AddDescription "<b>Password & special characters on Windows</b>." FAQ.Passwords_on_Windows.txt
@ -56,7 +58,7 @@ AddDescription "<b>POP3 issues</b>." FAQ.POP3
AddDescription "<b>Principles & design decisions</b>." FAQ.Principles.txt
AddDescription "<b>Selecting messages</b>." FAQ.Messages_Selection.txt
AddDescription "<b>How to transfer too big messages</b>." FAQ.Messages_Too_Big.txt
AddDescription "<b>Why the imapsync web site looks so old?</b>" FAQ.Old_Style_Web_Design.txt
AddDescription "Why the imapsync web site <b>looks so old?</b>" FAQ.Old_Style_Web_Design.txt
AddDescription "<b>UCS</b> accounts." FAQ.UCS.txt
AddDescription "<b>Checklist before release a new release</b>." FAQ.Release_Checklist.txt

33
INSTALL.d/Dockerfile Executable file → Normal file
View file

@ -1,6 +1,6 @@
## Dockerfile for building a docker imapsync image
# $Id: Dockerfile,v 1.30 2020/03/26 07:15:04 gilles Exp gilles $
# $Id: Dockerfile,v 1.38 2021/11/29 10:20:56 gilles Exp gilles $
# I use the following command to build the image:
#
# docker build -t gilleslamiral/imapsync .
@ -10,9 +10,9 @@
# I thank you very much
# I like thanks
# I like stars
# I also like (and need) money
# I also need money to keep on doing this stuff
# Number of imapsync images pulled so far (2021_05_29): 368881
# Number of imapsync images pulled so far (2021_11_29): 608326
# Command used:
# curl -s https://hub.docker.com/v2/repositories/gilleslamiral/imapsync/ | jq '.pull_count'
@ -24,16 +24,13 @@ LABEL maintainer="Gilles LAMIRAL <gilles@lamiral.info>" \
description="Imapsync" \
documentation="https://imapsync.lamiral.info/#doc"
# Put a copy of the Dockerfile in the image itself
# It can help future maintenance, isn't it?
# I put a copy of the Dockerfile in the image itself
# It can help maintenance, isn't it?
# Also put optionally my local and usually more recent imapsync on /, for testing purpose
COPY Dockerfile /
COPY Dockerfile imapsyn[c] prerequisites_imapsyn[c] /
# Also put my local imapsync on /usr/bin/ for testing purpose
# renamed imapsync_local since would be erased by the wget
COPY imapsync /usr/bin/imapsync_local
COPY prerequisites_imapsync /usr/bin/prerequisites_imapsync_local
RUN set -xe && \
apt-get update \
@ -44,7 +41,8 @@ RUN set -xe && \
libdata-uniqid-perl \
libencode-imaputf7-perl \
libfile-copy-recursive-perl \
libfile-tail-perl \
libfile-tail-perl \
libio-compress-perl \
libio-socket-ssl-perl \
libio-socket-inet6-perl \
libio-tee-perl \
@ -70,16 +68,17 @@ RUN set -xe && \
lsof \
ncat \
openssl \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN set -xe \
&& cd /usr/bin/ \
&& pwd \
&& wget -N https://imapsync.lamiral.info/imapsync \
https://imapsync.lamiral.info/prerequisites_imapsync \
https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/python/oauth2.py
&& mv imapsync oauth2.py /usr/bin/ \
&& chmod +x /usr/bin/imapsync /usr/bin/imapsync_local /usr/bin/oauth2.py # just_a_comment_to_force_update 2019_11_26_12_58_34
&& wget -N --no-check-certificate https://imapsync.lamiral.info/imapsync \
https://imapsync.lamiral.info/prerequisites_imapsync \
https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/python/oauth2.py \
&& chmod +x imapsync oauth2.py \
&& /usr/bin/imapsync --testslive && /usr/bin/imapsync --tests # just_a_comment_to_force_update 2021_11_29_11_20_56
USER nobody:nogroup

51
INSTALL.d/INSTALL.Centos.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: INSTALL.Centos.txt,v 1.25 2021/02/02 17:08:46 gilles Exp gilles $
# $Id: INSTALL.Centos.txt,v 1.29 2021/12/17 14:52:20 gilles Exp gilles $
This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/
@ -9,37 +9,43 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.Centos.txt
Installing imapsync on CentOS
=======================================================================
There is one section for Centos 8,
There is one section for AlmaLinux and Centos 8,
one for Centos 7,
and one for Centos 6.
=======================================================================
Centos 8
AlmaLinux and Centos 8
=======================================================================
How to install imapsync on AlmaLinux and Centos 8:
How to install the latest imapsync on Centos 8
dnf install --enablerepo=powertools imapsync
imapsync --testslive
dnf install wget perl-App-cpanminus gcc
# then
cpanm lazy Test::MockObject
wget https://imapsync.lamiral.info/imapsync
perl -Mlazy imapsync
=======================================================================
AlmaLinux and Centos 8 and latest imapsync
=======================================================================
At the time of this writing (December 2021), the powertools repository
contains imapsync release 1.977, which is not the latest available
imapsync release. (1.977 date is 2019/12/23 while 2.174 is 2021/12/14)
In order to install the latest imapsync I suggest the following process:
Install the powertools imapsync 1.977 release via dnf like mentioned above.
It will install the needed packages, Perl itself and Perl modules
for release 1.977.
The latest imapsync 2.174 needs no more Perl modules.
wget -N https://imapsync.lamiral.info/imapsync
chmod +x imapsync
./imapsync --testslive
./imapsync --tests
If you encounter the following error during the run of "perl -Mlazy imapsync"
...
Can't locate object method "new" via package "Archive::Tar" at ...
FAIL install IO-String-1.08
fix it with:
wget http://mirror.centos.org/centos/8/PowerTools/x86_64/os/Packages/perl-IO-String-1.08-31.el8.noarch.rpm
yum install perl-IO-String-1.08-31.el8.noarch.rpm -y
See also https://github.com/imapsync/imapsync/issues/211
You may get an even more recent imapsync release this way.
=======================================================================
Centos 7
@ -72,7 +78,7 @@ Unit tests:
At the time of this writing (March 2020), the epel7 repository still
contains imapsync release 1.727, which is not the latest available
imapsync release. (1.727 date is 2016/08/19 while 1.979 is 2020/01/14)
imapsync release. (1.727 date is 2016/08/19 while 2.174 is 2021/12/14)
In order to install the latest imapsync I suggest the following process:
@ -102,7 +108,8 @@ Then:
perl-Test-NoWarnings \
perl-Test-Simple \
perl-Test-Warn \
perl-Sub-Uplevel
perl-Sub-Uplevel \
ca-certificates
Also, run the following commands:

51
INSTALL.d/INSTALL.Darwin.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: INSTALL.Darwin.txt,v 1.34 2021/06/15 10:02:37 gilles Exp gilles $
# $Id: INSTALL.Darwin.txt,v 1.36 2021/12/24 21:08:13 gilles Exp gilles $
This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/
@ -23,18 +23,18 @@ A) Installing imapsync binary on Darwin / Mac OS X
There is a standalone imapsync binary for Mac OS X
called "imapsync_bin_Darwin" (without the quotes),
available in the compressed tarball called imapsync-1.977.tgz
where 1.977 is the imapsync version number.
available in the compressed tarball called imapsync-2.140.tgz
where 2.140 is the imapsync version number.
In case you haven't buy it yet or forgot where the download link is,
check https://imapsync.lamiral.info/dist/
Download the tarball imapsync-1.977.tgz, not the binary imapsync_bin_Darwin
Download the tarball imapsync-2.140.tgz, not the binary imapsync_bin_Darwin
because the binary is also in the tarball and there are other useful files
in the tarball. The binary present in /dist/ is just there for a fast upgrade
when you need one.
I suppose this tarball imapsync-1.977.tgz is downloaded under your $HOME directory,
I suppose this tarball imapsync-2.140.tgz is downloaded under your $HOME directory,
let say /Users/gilles/, but you can put it anywhere. In real, your $HOME directory
is not /Users/gilles/, it maybe /Users/john/ or /Users/zoey/ where John or Zoey
is you login name.
@ -45,19 +45,19 @@ Untar the tarball:
cd
pwd
tar xzvf imapsync-1.977.tgz
tar xzvf imapsync-2.140.tgz
In case the previous command fails, it means the tarball file called imapsync-1.977.tgz
In case the previous command fails, it means the tarball file called imapsync-2.140.tgz
is not in your $HOME directory, you may have downloaded it elsewhere on the file system.
A way to find it is the command:
find / | grep imapsync
Now that the tarball is extracted, it created a directory called imapsync-1.977/
Go into the directory imapsync-1.977 with the command:
Now that the tarball is extracted, it created a directory called imapsync-2.140/
Go into the directory imapsync-2.140 with the command:
cd imapsync-1.977
cd imapsync-2.140
First let's have a simple run to see if imapsync_bin_Darwin works.
@ -178,18 +178,19 @@ Install the "make" command in case it is not already installed.
First let us install cpanminus locally in ~/perl5
wget --no-check-certificate -O- http://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib
curl -L https://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib
Then take this install into account in the current environment
eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`
perl -I ~/perl5/lib/perl5 -Mlocal::lib
perl -I ~/perl5/lib/perl5 -Mlocal::lib # just to see the variables
eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib` # doing the variables assigments
If you want to have always this setting in your environment then run the commands
echo 'eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`' >> ~/.profile
echo 'export MANPATH=$HOME/perl5/man:$MANPATH' >> ~/.profile
cat ~/.profile
. ~/.profile
Now let's update the standard CPAN Perl module
@ -202,18 +203,26 @@ helps to verify what is needed to install on your system
sh prerequisites_imapsync
cpanm Authen::NTLM
cpanm File::Copy::Recursive IO::Tee
cpanm Mail::IMAPClient
cpanm Readonly
cpanm IO::Tee
cpanm Mail::IMAPClient # File::Copy::Recursive?
cpanm Unicode::String
cpanm Sys::MemInfo
cpanm File::Tail
cpanm Test::MockObject
cpanm Readonly
cpanm Data::Uniqid
cpanm JSON::WebToken
We're ready to install and test the latest imapsync
wget -c http://imapsync.lamiral.info/imapsync
curl -L http://imapsync.lamiral.info/imapsync > imapsync
chmod +x imapsync
./imapsync
./imapsync --modules
./imapsync --testslive
./imapsync --tests
You can rerun "sh prerequisites_imapsync"
to verify everything is ok:
@ -223,10 +232,6 @@ to verify everything is ok:
When everything is ok the script execution ends with this sentence
"All needed modules are already installed"
Now imapsync should work on your system.
./imapsync --testslive
=======================================================================
E) Building imapsync binary on Darwin / Mac OS X
=======================================================================
@ -234,7 +239,7 @@ E) Building imapsync binary on Darwin / Mac OS X
cpanm Module::ScanDeps
cpanm PAR::Packer
pp -o imapsync.bin imapsync
pp -x -u -o imapsync.bin imapsync
./imapsync.bin
./imapsync.bin --testslive

127
INSTALL.d/INSTALL.Debian.txt
View file

@ -1,24 +1,26 @@
#!/bin/cat
# $Id: INSTALL.Debian.txt,v 1.32 2020/12/11 09:27:29 gilles Exp gilles $
# $Id: INSTALL.Debian.txt,v 1.35 2021/12/17 20:30:33 gilles Exp gilles $
This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/
https://imapsync.lamiral.info/INSTALL.d/INSTALL.Debian.txt
There is three install sections in this document,
1) one for Debian 9 Stretch and Debian 10 Buster
2) one for Debian 8 Jessie
3) one for Debian 7 Wheezy
There is four install sections in this document,
1) one for Debian 9 Stretch, Debian 10 Buster, Debian 11 Bullseye
2) one for all the Debian called "After installing the dependencies"
3) one for Debian 8 Jessie (obsolete)
4) one for Debian 7 Wheezy (even more obsolete)
After reading and following 1) or 2) or 3), you'll be invited to read and follow
the fourth section "4) After installing the dependencies"
After reading and following 1) or 3) or 4), you'll be invited to read and follow
the section "2) After installing the dependencies"
What is your Debian release?
Type on a terminal:
cat /etc/debian_version
11.x => Bullseye
10.x => Buster
9.x => Stretch
8.x => Jessie
@ -27,20 +29,20 @@ Type on a terminal:
where x is any number between 0 and 9, x is minor and to be forgotten.
First a call to Debian packagers
First, a call to Debian packagers
Why imapsync is not in Debian as a package?
Imapsync used to be in Debian from 2005-04-25 (release 1.125) to 2011-01-25 (release 1.315).
It could be in Debian nowadays, as always, my last words on this discussion were
"Do what you want, I promise I won't complain anymore about the fact imapsync is on Debian or not."
"Do what you want, I promise I won't complain anymore about the fact imapsync is in Debian or not."
https://lists.debian.org/debian-legal/2011/01/msg00058.html
The license is now "No limits to do anything with this work and this license".
Feel free.
The Imapsync license is "No limits to do anything with this work and this license".
So feel free!
See also a more detailed story at
https://lists.debian.org/debian-user/2016/11/msg00849.html
=======================================================================
1) Installing imapsync on Debian 9 Stretch or Debian 10 Buster
1) Installing imapsync on Debian 9 Stretch, 10 Buster, 11 Bullseye
=======================================================================
How to install imapsync dependencies on a Debian system?
@ -48,14 +50,19 @@ How to install imapsync dependencies on a Debian system?
The repositories are the classical ones, ie, a
/etc/apt/sources.list file with:
# Debian 9 Stretch
deb http://deb.debian.org/debian stretch main contrib
# Debian 11 Bullseye
deb http://deb.debian.org/debian bullseye main contrib
or
# Debian 10 Buster
deb http://deb.debian.org/debian buster main contrib
or
# Debian 9 Stretch
deb http://deb.debian.org/debian stretch main contrib
You need root privilege to run it, be root or use sudo.
@ -88,16 +95,60 @@ apt install -y \
libtest-deep-perl \
libtest-warn-perl \
make \
time \
cpanminus
After installing the dependencies, imapsync should be working,
go to the section "4) After installing the dependencies" below.
go to the section "2) After installing the dependencies" below.
=======================================================================
2) After installing the dependencies
=======================================================================
You don't have to be root to test and use imapsync.
Take imapsync either on github:
wget -N https://github.com/imapsync/imapsync/blob/master/imapsync
or be on the cutting edge with the upstream site:
wget -N https://imapsync.lamiral.info/imapsync
Add execution permission to the downloaded script:
chmod +x imapsync
Check the dependencies and print also the basic example:
./imapsync
Perform a live test showing imapsync job:
./imapsync --testslive
Now install imapsync on the system (need root priviledges again):
cp imapsync /usr/bin/
That's finished for the installation part.
You can now use imapsync without knowing where it is located
on the system:
imapsync
Now go read the menu http://imapsync.lamiral.info/#doc
and start with the tutorial
https://imapsync.lamiral.info/doc/TUTORIAL_Unix.html
Feedback is welcome!
=============================================
2) Installing imapsync on Debian 8 Jessie
=============================================
=======================================================================
3) Installing imapsync on Debian 8 Jessie
=======================================================================
How to install imapsync dependencies on a Debian system?
@ -165,11 +216,11 @@ the module JSON::WebToken with the command:
cpanm JSON::WebToken
After installing the dependencies, imapsync should be working,
go to the section "4) After installing the dependencies" below.
go to the section "2) After installing the dependencies" above.
=======================================================================
3) Installing imapsync on Debian 7 Wheezy
4) Installing imapsync on Debian 7 Wheezy
=======================================================================
How to install imapsync dependencies on a Debian system?
@ -234,40 +285,6 @@ the module JSON::WebToken with the command:
After installing the dependencies, imapsync should be working,
then go to the section "4) After installing the dependencies" below.
=======================================
4) After installing the dependencies
=======================================
You don't have to be root to test and use imapsync.
Take imapsync either on github or at the upstream site.
Add execution permission to the downloaded script:
chmod +x imapsync
Check the dependencies and print also the basic example:
./imapsync
Perform a live test showing imapsync job:
./imapsync --testslive
Now install imapsync on the system (need root priviledges again):
cp imapsync /usr/bin/
That's finished for the installation part.
You can now use imapsync without knowing where it is located
on the system:
imapsync
Now go to read http://imapsync.lamiral.info/#doc
and start with the tutorial.
then go to the section "2) After installing the dependencies" above.
=======================================================================
=======================================================================

44
INSTALL.d/INSTALL.Docker_build.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: INSTALL.Docker_build.txt,v 1.12 2019/11/28 14:43:48 gilles Exp gilles $
# $Id: INSTALL.Docker_build.txt,v 1.15 2021/11/29 11:56:11 gilles Exp gilles $
This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/
@ -10,22 +10,27 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.Docker_build.txt
=======================================================================
Nearly verbatim copy from https://imapsync.lamiral.info/INSTALL.d/Dockerfile
(nearly but the Dockerfile rcs tag Id: removed, that would conflict)
(verbatim but the Dockerfile rcs tag $ removed because it would conflict)
## Dockerfile for building a docker imapsync image
# Dockerfile,v 1.26 2019/11/27 15:49:13 gilles Exp gilles $
# Id: Dockerfile,v 1.38 2021/11/29 10:20:56 gilles Exp gilles $
# I use the following command to build the image:
#
# docker build -t gilleslamiral/imapsync .
#
# where this Dockerfile is in the current directory
#
# I thank you very much
# I like thanks
# I like stars
# I also like (and need) money
# I thank you very much in advance
# I also need money to keep on doing this stuff
# Number of imapsync images pulled so far (2021_11_29): 608326
# Command used:
# curl -s https://hub.docker.com/v2/repositories/gilleslamiral/imapsync/ | jq '.pull_count'
# Debian Buster is Debian 10
FROM debian:buster
@ -33,16 +38,13 @@ LABEL maintainer="Gilles LAMIRAL <gilles@lamiral.info>" \
description="Imapsync" \
documentation="https://imapsync.lamiral.info/#doc"
# Put a copy of the Dockerfile in the image itself
# It can help future maintenance, isn't it?
# I put a copy of the Dockerfile in the image itself
# It can help maintenance, isn't it?
# Also put optionally my local and usually more recent imapsync on /, for testing purpose
COPY Dockerfile /
COPY Dockerfile imapsyn[c] prerequisites_imapsyn[c] /
# Also put my local imapsync on /usr/bin/ for testing purpose
# renamed imapsync_local since would be erased by the wget
COPY imapsync /usr/bin/imapsync_local
COPY prerequisites_imapsync /usr/bin/prerequisites_imapsync_local
RUN set -xe && \
apt-get update \
@ -53,7 +55,8 @@ RUN set -xe && \
libdata-uniqid-perl \
libencode-imaputf7-perl \
libfile-copy-recursive-perl \
libfile-tail-perl \
libfile-tail-perl \
libio-compress-perl \
libio-socket-ssl-perl \
libio-socket-inet6-perl \
libio-tee-perl \
@ -77,15 +80,19 @@ RUN set -xe && \
make \
cpanminus \
lsof \
ncat \
openssl \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN set -xe \
&& cd /usr/bin/ \
&& pwd \
&& wget -N https://imapsync.lamiral.info/imapsync \
https://imapsync.lamiral.info/prerequisites_imapsync \
&& mv imapsync /usr/bin/imapsync \
&& chmod +x /usr/bin/imapsync /usr/bin/imapsync_local # just_a_comment_to_force_update 2019_11_26_12_58_34
&& wget -N --no-check-certificate https://imapsync.lamiral.info/imapsync \
https://imapsync.lamiral.info/prerequisites_imapsync \
https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/python/oauth2.py \
&& chmod +x imapsync oauth2.py \
&& /usr/bin/imapsync --testslive && /usr/bin/imapsync --tests # just_a_comment_to_force_update 2021_11_29_11_20_56
USER nobody:nogroup
@ -102,6 +109,5 @@ CMD ["/usr/bin/imapsync"]
=======================================================================
=======================================================================

9
INSTALL.d/INSTALL.OnlineUI.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: INSTALL.OnlineUI.txt,v 1.49 2021/03/19 09:39:01 gilles Exp gilles $
# $Id: INSTALL.OnlineUI.txt,v 1.50 2021/09/20 13:47:20 gilles Exp gilles $
This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/
@ -11,7 +11,7 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.OnlineUI.txt
I'm now confident with /X since the /X service is up and running quite
well since January 2017. Anyway, if you run this service on your own
well since January 2017. Anyway, if you run this service on your own,
online, you take responsibility for it.
=======================================================================
@ -135,7 +135,10 @@ Now that I have explained the general context for any system,
I'll describe concrete examples on several systems,
Debian/Ubuntu and Centos. Feedbacks show that the Centos
process is easier in case you don't know very much any of the
Linux distributions.
Linux distributions. I add that if you don't know very much any
of the Linux distributions then you shouldn't install this
imapsync service at all.
=============================================================================
A) Concrete example on a Debian server with Apache:

3
INSTALL.d/INSTALL.Ubuntu.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: INSTALL.Ubuntu.txt,v 1.17 2019/11/28 14:35:43 gilles Exp gilles $
# $Id: INSTALL.Ubuntu.txt,v 1.18 2021/07/30 15:06:43 gilles Exp gilles $
This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/
@ -49,6 +49,7 @@ libtest-nowarnings-perl \
libtest-deep-perl \
libtest-warn-perl \
make \
time \
cpanminus
In case you want to update the Perl module

18
INSTALL.d/memo_docker
View file

@ -1,5 +1,7 @@
#!/bin/sh
# $Id: memo_docker,v 1.15 2021/11/20 20:52:55 gilles Exp gilles $
echo imapsync_docker_timestamp_dockerfile
imapsync_docker_timestamp_dockerfile()
{
@ -27,6 +29,12 @@ imapsync_docker_version()
docker run gilleslamiral/imapsync imapsync --version
}
echo imapsync_local_docker_version
imapsync_local_docker_version()
{
docker run gilleslamiral/imapsync imapsync_local --version
}
echo imapsync_docker_add_tag_version
imapsync_docker_add_tag_version()
{
@ -102,13 +110,21 @@ imapsync_docker_rebuild_from_scratch()
imapsync_docker_build
}
echo is_release_number
is_release_number()
{
echo "$1" | perl -pe 'm/^\d\.\d+$/ || exit 1'
}
echo imapsync_docker_upload
imapsync_docker_upload()
{
version=`imapsync_docker_version`
# upload only after successful tests
set -x \
&& rcsdiff imapsync \
&& is_release_number "$version" \
&& rcsdiff imapsync Dockerfile memo_docker \
&& imapsync_docker_testslive \
&& imapsync_docker_tests \
&& docker run gilleslamiral/imapsync true \

70
Makefile
View file

@ -1,5 +1,5 @@
# $Id: Makefile,v 1.335 2021/07/06 02:00:46 gilles Exp gilles $
# $Id: Makefile,v 1.339 2022/01/14 14:28:38 gilles Exp gilles $
.PHONY: help usage all doc
@ -34,6 +34,7 @@ usage:
@echo "make upload_index"
@echo "make upload_FAQ # upload FAQs and documentation"
@echo "make upload_X # upload online UI"
@echo "make upload_csv # upload online CSV service"
@echo "make upload_latest # upload latest imapsync and binaries (dev)"
@echo "make upload_cgi # upload latest imapsync online, after local and remote --tests success."
@echo "make upload_cgi_memo # upload cgi_memo stat_patterns.txt to /X servers."
@ -150,7 +151,7 @@ clean_bak:
rm -f index.shtml.bak ./S/style.css.bak
clean_oauth2:
rm oauth2/typescript oauth2/D_*txt
rm -f oauth2/typescript oauth2/D_*txt
.PHONY: install dist man
@ -160,6 +161,7 @@ clean_man:
rm -f W/imapsync.1
W/imapsync.1: imapsync
mkdir -p W
pod2man imapsync > W/imapsync.1
install: testp W/imapsync.1
@ -188,7 +190,7 @@ docker:
@echo "make docker_upload_docker_hub # upload last build to https://hub.docker.com/r/gilleslamiral/imapsync"
@echo "ssh vp3 THEN cd docker/imapsync && . memo_docker"
docker_build:
docker_build: docker_copy_to_vp3
ssh vp3 'cd docker/imapsync && . memo_docker && imapsync_docker_build'
@ -199,7 +201,7 @@ docker_copy_to_vp3:
rsync -av imapsync INSTALL.d/Dockerfile INSTALL.d/memo_docker INSTALL.d/prerequisites_imapsync INSTALL.d/secret.txt vp3:docker/imapsync/
rsync -av RCS/imapsync,v INSTALL.d/RCS/Dockerfile,v INSTALL.d/RCS/memo_docker,v vp3:docker/imapsync/RCS/
docker_upload_docker_hub:
docker_upload_docker_hub: docker_build
ssh vp3 'cd docker/imapsync && . memo_docker && imapsync_docker_upload'
functree: W/imapsync_functions_tree_ppi.txt W/imapsync_functions_tree.txt
@ -517,18 +519,18 @@ W/test_tail.bat:
win64sshaccess:
ssh 'pc HP DV7'@p24 'perl -V'
win64prepa:
ssh 'pc HP DV7'@p24 'perl -V'
winprepalocal:
unix2dos W/build_exe.bat W/test_exe.bat W/install_modules.bat W/install_module_one.bat
win64prepa: winprepalocal
ssh 'pc HP DV7'@p24 'perl -V'
scp imapsync W/build_exe.bat W/install_modules.bat W/install_module_one.bat \
W/test_exe_tests.bat W/test_exe_testsdebug.bat W/test_exe.bat \
pc_HP_DV7_p24:'Desktop/imapsync_build'
ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat'
./W/check_win64err build_exe.bat
win64build:
unix2dos W/build_exe.bat W/install_modules.bat
win64build: winprepalocal
scp imapsync W/build_exe.bat W/install_modules.bat pc_HP_DV7_p24:'Desktop/imapsync_build'
ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat'
./W/check_win64err build_exe.bat
@ -537,10 +539,9 @@ imapsync.exe: imapsync_64bit.exe
cp -a imapsync_64bit.exe imapsync.exe
imapsync_64bit.exe: imapsync
imapsync_64bit.exe: imapsync winprepalocal
(date "+%s"| tr "\n" " "; echo -n "BEGIN 64bit " $(VERSION) ": "; date) >> W/.BUILD_EXE_TIME
ssh 'pc HP DV7'@p24 'perl -V'
unix2dos W/build_exe.bat W/test_exe.bat W/install_modules.bat W/install_module_one.bat
scp imapsync W/build_exe.bat W/install_modules.bat W/install_module_one.bat \
W/test_exe_tests.bat W/test_exe_testsdebug.bat W/test_exe.bat \
pc_HP_DV7_p24:'Desktop/imapsync_build'
@ -565,7 +566,7 @@ zip: dosify_bat
unix2dos ../prepa_zip/imapsync_$(VERSION)/*.txt
cd ../prepa_zip/ && rm -f ./imapsync_$(VERSION).zip && zip -r ./imapsync_$(VERSION).zip ./imapsync_$(VERSION)/
scp ../prepa_zip/imapsync_$(VERSION).zip Admin@c:'C:/msys/1.0/home/Admin/'
scp ../prepa_zip/imapsync_$(VERSION).zip pc_HP_DV7_p24:'Desktop/'
# scp ../prepa_zip/imapsync_$(VERSION).zip pc_HP_DV7_p24:'Desktop/'
cp ../prepa_zip/imapsync_$(VERSION).zip /fe/imapsync/
@ -575,19 +576,22 @@ zip: dosify_bat
mac: imapsync_bin_Darwin
maccopy:
rsync -p -e 'ssh -4 -p 995' imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync webserver \
gilleslamira@gate.polarhome.com:
rsync -av -p -e 'ssh -4 -p 995' X/ gilleslamira@gate.polarhome.com:X/
macstadiumcopy:
rsync -v imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync webserver administrator@macstadium.lamiral.info:
rsync -v examples/file.txt examples/sync_loop_darwin.sh administrator@macstadium.lamiral.info:examples/
rsync -v X/ administrator@macstadium.lamiral.info:X/
maccopy:
rsync -v -p -e 'ssh -4 -p 995' imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync webserver \
gilleslamira@gate.polarhome.com:
rsync -v -p -e 'ssh -4 -p 995' examples/file.txt examples/sync_loop_darwin.sh gilleslamira@gate.polarhome.com:examples/
rsync -v -p -e 'ssh -4 -p 995' X/ gilleslamira@gate.polarhome.com:X/
macforce: maccopy
ssh -4 -p 995 gilleslamira@gate.polarhome.com 'sh -x build_mac.sh'
imapsync_bin_Darwin: imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync
imapsync_bin_Darwin: imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync maccopy
rcsdiff imapsync
rsync -p -e 'ssh -4 -p 995' imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync gilleslamira@gate.polarhome.com:
ssh -4 -p 995 gilleslamira@gate.polarhome.com 'sh -x build_mac.sh'
rsync -P -e 'ssh -4 -p 995' gilleslamira@gate.polarhome.com:imapsync_bin_Darwin .
@ -714,12 +718,12 @@ S/imapsync_sold_by_country.txt: /g/bin/imapsync_by_country
ks:
rsync -avHz --delete --exclude '*.exe' \
. gilles@ks.lamiral.info:public_html/imapsync/
ssh root@ks.lamiral.info 'apachectl configtest && /etc/init.d/apache2 reload'
ssh root@ks.lamiral.info 'apachectl configtest && apachectl reload'
ksa:
rsync -avHz --delete -P \
. gilles@ks.lamiral.info:public_html/imapsync/
ssh root@ks.lamiral.info 'apachectl configtest && /etc/init.d/apache2 reload'
ssh root@ks.lamiral.info 'apachectl configtest && apachectl reload'
ks3:
rsync -avHz --delete -P \
@ -745,11 +749,10 @@ ks5tests: ks5tests_gilles ks5tests_root
ks5tests_gilles:
rsync -P imapsync gilles@ks.lamiral.info:public_html/imapsync/
rsync -P oauth2_access_token.txt oauth2_string_for_oauthdirect.txt gilles@ks.lamiral.info:
ssh gilles@ks.lamiral.info 'public_html/imapsync/imapsync --tests'
ks5tests_root:
rsync -P imapsync oauth2_access_token.txt oauth2_string_for_oauthdirect.txt root@ks5.lamiral.info:
rsync -P imapsync root@ks5.lamiral.info:
ssh root@ks5.lamiral.info './imapsync --tests'
@ -830,14 +833,16 @@ W/.valid.index.shtml: index.shtml S/*.shtml
upload_index: valid_index clean_permissions
rcsdiff index.shtml README_Windows.txt S/style.css S/*.shtml FAQ.d/*.txt LICENSE CREDITS TODO examples/*.bat examples/*.sh index.shtml INSTALL.d/*.txt
rcsdiff S/quiz/quiz_imapsync.html S/quiz/quiz_imapsync.js S/quiz/quiz_imapsync.css
rm -f examples/LOG_imapsync/*
rsync -avH index.shtml README_Windows.txt FAQ INSTALL NOLIMIT LICENSE CREDITS TODO S/robots.txt S/favicon.ico ../imapsync_website/
rsync -aHv --delete ./W/ks.htaccess ../imapsync_website/.htaccess
rsync -aHv --delete S/ ../imapsync_website/S/
rsync -aHv --delete ./S/ ../imapsync_website/S/
rsync -aHv --delete ./examples/ ../imapsync_website/examples/
rsync -aHv --delete ./INSTALL.d/ ../imapsync_website/INSTALL.d/
rsync -aHv --delete ./FAQ.d/ ../imapsync_website/FAQ.d/
rsync -avH --delete ./doc/ ../imapsync_website/doc/
rsync -avH --delete ./W/tools/ ../imapsync_website/W/tools/
rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/
@ -853,7 +858,7 @@ upload_latest: unitests ci_imapsync bin
upload_cgi: unitests ks5tests ks5tests_root ci_imapsync
rsync -a imapsync ./INSTALL.d/prerequisites_imapsync ../imapsync_website/
rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/
rsync -P imapsync root@ks5.lamiral.info:/home/www/apache24/cgi-bin/imapsync
rsync -P imapsync root@ks5.lamiral.info:/home/www/apache24/cgi-bin/
upload_cgi_memo:
dos2unix X/stat_patterns.txt X/server_survey_patterns.txt
@ -863,13 +868,21 @@ upload_cgi_memo:
upload_X:
./W/tools/validate_xml_html5 X/index.html X/imapsync_form.html X/imapsync_form_extra.html X/imapsync_form_extra_free.html X/imapsync_form_wrapper.html
rcsdiff X/imapsync_form.html X/imapsync_form_extra.html X/imapsync_form_wrapper.html X/imapsync_form.css X/noscript.css
rcsdiff X/imapsync_form.js X/imapsync_form_wrapper.js
rcsdiff X/imapsync_form.html X/imapsync_form_extra.html X/imapsync_form_extra_free.html X/imapsync_form_wrapper.html X/imapsync_form.css X/noscript.css
rcsdiff X/imapsync_form.js X/imapsync_form_wrapper.js
rcsdiff INSTALL.d/INSTALL.OnlineUI.txt
rsync -a ./INSTALL.d/INSTALL.OnlineUI.txt ../imapsync_website/INSTALL.d/INSTALL.OnlineUI.txt
rsync -av --delete X/ ../imapsync_website/X/
rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/
upload_csv:
./W/tools/validate_xml_html5 X/sandbox_csv.html
rcsdiff X/sandbox_csv.html X/sandbox_csv.js X/imapsync_csv_wrapper
rsync -a X/sandbox_csv.html X/sandbox_csv.js X/imapsync_csv_wrapper ../imapsync_website/X/
rsync -aHvz X/sandbox_csv.html X/sandbox_csv.js X/imapsync_csv_wrapper root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/X/
rsync X/imapsync_csv_wrapper root@ks5.lamiral.info:/home/www/apache24/cgi-bin/
upload_FAQ:
rcsdiff FAQ.d/*.txt LICENSE CREDITS TODO INSTALL.d/*.txt
rsync -avH FAQ INSTALL CREDITS TODO ../imapsync_website/
@ -891,8 +904,9 @@ upload_ks: ci tarball
rsync -aHv --delete ./INSTALL.d/ ../imapsync_website/INSTALL.d/
rsync -aHv --delete ./FAQ.d/ ../imapsync_website/FAQ.d/
rsync -avH --delete ./doc/ ../imapsync_website/doc/
rsync -avH --delete ./W/tools/ ../imapsync_website/W/tools/
rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/
ssh root@ks.lamiral.info 'apachectl configtest && /etc/init.d/apache2 reload'
ssh root@ks.lamiral.info 'apachectl configtest && apachectl reload'
upload_ks5:

38
README
View file

@ -6,7 +6,7 @@ NAME
VERSION
This documentation refers to Imapsync $Revision: 2.140 $
This documentation refers to Imapsync $Revision: 2.178 $
USAGE
@ -182,16 +182,6 @@ OPTIONS
--sslargs2 str : Pass any ssl parameter for host2 ssl or tls connection.
See --sslargs1
--timeout1 flo : Connection timeout in seconds for host1.
Default is 120 and 0 means no timeout at all.
--timeout2 flo : Connection timeout in seconds for host2.
Default is 120 and 0 means no timeout at all.
Caveat, under CGI context, you may encounter a timeout
from the webserver, killing imapsync and the imap connexions.
See the document INSTALL.OnlineUI.txt and search
for "Timeout" for how to deal with this issue.
OPTIONS/authentication
--authmech1 str : Auth mechanism to use with host1:
@ -395,6 +385,12 @@ OPTIONS
Example: 's/\000/ /g' # to replace null characters
by spaces.
--regexmess reg : and this one, etc.
--truncmess int : truncates messages when their size exceed the int
value, specified in bytes. Good to sync too big
messages or to "suppress" attachments.
Have in mind that this way, messages become
uncoherent somehow.
OPTIONS/labels
@ -595,6 +591,26 @@ OPTIONS
OPTIONS/behavior
--timeout1 flo : Connection timeout in seconds for host1.
Default is 120 and 0 means no timeout at all.
--timeout2 flo : Connection timeout in seconds for host2.
Default is 120 and 0 means no timeout at all.
Caveat, under CGI context, you may encounter a timeout
from the webserver, killing imapsync and the imap connexions.
See the document INSTALL.OnlineUI.txt and search
for "Timeout" for how to deal with this issue.
--keepalive1 : https://metacpan.org/pod/Mail::IMAPClient#Keepalive
Some firewalls and network gears like to timeout connections
prematurely if the connection sits idle.
This option enables SO_KEEPALIVE on the host1 socket.
--keepalive1 is on by default since imapsync release 2.169
Use --nokeepalive1 to disable it.
--keepalive2 : Same as --keepalive2 but for host2.
Use --nokeepalive2 to disable it.
--maxmessagespersecond flo : limits the average number of messages
transferred per second.

141
README_Windows.txt
View file

@ -1,4 +1,4 @@
# $Id: README_Windows.txt,v 1.24 2021/06/15 10:29:45 gilles Exp gilles $
# $Id: README_Windows.txt,v 1.26 2022/01/14 11:54:28 gilles Exp gilles $
#
# This is the README_Windows.txt file for imapsync
# imapsync: IMAP syncing and migration tool.
@ -21,48 +21,52 @@ A) Simplest way
A.0) Preamble for visual users looking for a visual tool.
First, the visual thing I am talking about here is not the online visual
interface I call /X that you might have seen and used at the URL
https://imapsync.lamiral.info/X/
First, the visual thing I am talking about here is not the online
visual interface I call /X that you might have seen and used at the
URL https://imapsync.lamiral.info/X/
This /X visual interface is not yet available on Windows as a service
that you can install and use in your system.
I plan to make a visual tool available on Windows but, for now,
I encountered technical issues.
that you can install and use in your system. I plan to make a visual
tool available on Windows but, for now, I encountered technical
issues.
So what visual thing am I talking about?
Imapsync itself is not a visual tool. The visual tool is Notepad or
any text editor. Many pure visual users have succeeded in using
imapsync to transfer their email accounts. As you can guess, only
the ones that tried have succeeded, so don't give up before
trying at least once. Another assumption is that visual users can read.
imapsync to transfer their email accounts. As you can guess, only the
ones that tried have succeeded, so don't give up before trying at
least once. Another assumption is that visual users can read.
Let's go for some reading!
A.1) Get imapsync.
Get imapsync at https://imapsync.lamiral.info/
You'll then have access to a zip archive file named imapsync_1.977.zip
where 1.977 is the imapsync release number.
You'll then have access to a zip archive file named imapsync_2.178.zip
where 2.178 is the imapsync release number.
A.2) Extract the zip file in a folder where you will work with imapsync.
You can work on the Desktop since the zip file extraction creates
a unique folder named imapsync_1.977/
a unique folder named imapsync_2.178/
Do not unzip the archive in what is called a "system" directory since
you may encounter permission issues.
Two-point to have in mind:
Two points to have in mind:
* You don't need to be an Administrator to unzip the zip archive.
* You don't need to be an Administrator to run imapsync.
In short, any user on your system can use imapsync.
A.3) Check the folder
In the folder extracted imapsync_1.977 you see 7 files and 2 directories,
which may be presented in a different order than the following, the order
is not important anyway. The really important files to get your mailbox
transfer job started in a few minutes are the first two files of the
In the folder extracted and called imapsync_2.178, you see 7 files
and 2 directories. Those files and directories may be presented in a
different order than the following, the order is not important
anyway. There are only two important files to get your mailbox
transfer job started in a few minutes, the first two files of the
following list:
* README_Windows.txt is the current file you are reading.
@ -80,90 +84,104 @@ A.3) Check the folder
You can copy or rename the file imapsync_example.bat as you wish,
as long as its extension remains ".bat", for example, mysync.bat
On Windows systems, a file name ending with a .bat extension means "I am a batch script".
A batch script is a file containing commands, it's a program.
Don't be afraid, a program can be very simple to modify and I hope
imapsync_example.bat is one of them.
On Windows systems, a file name ending with a .bat extension means
"I'm a batch script". A batch script is a file containing commands,
it's a program. Don't be afraid, a program can be very simple to
modify and I hope imapsync_example.bat is one of them.
The batch scripts have to stay in the same directory as
imapsync.exe because of the way they call imapsync.exe.
They use the string ".\imapsync.exe", so let them be in the same directory.
The batch scripts have to stay in the same directory as imapsync.exe
because of the way they call imapsync.exe. They use the string
".\imapsync.exe", so let them be in the same directory.
You can change the path .\ to whatever you want if you understand what
you are doing (you have to use a pathname from the script point of view).
You can change the path .\ to whatever you want if you understand
what you are doing (you have to use a pathname from the script point
of view).
For the rest of this documentation, I assume you copied
imapsync_example.bat to a file named mysync.bat
If you don't know how to copy and rename a file then use imapsync_example.bat
itself, it's ok. The original file is still in the zip file in case
you want to restart from scratch.
If you don't know how to copy and rename a file then use
imapsync_example.bat itself, it's ok. The original file is still in
the zip file in case you want to restart from scratch.
A.4) Edit the batch file
This section describes how to edit the file mysync.bat to change
the example values with yours.
This section describes how to edit the file mysync.bat and change
it by replacing example values with your values.
To edit mysync.bat, you have to right-click on it and select "modify"
in the list presented in the small window menu.
Notepad or Notepadd++ are very good editor candidates to modify it.
Notepad is already installed on any Windows system, Notepadd++
is not usually installed but if you have it then use it.
Notepad or Notepadd++ are very good editor candidates to modify the
script. Notepad is already installed on any Windows system,
Notepadd++ is not usually installed but if you have it, then use it.
Office Word or any powerful text processor is NOT good for that job.
Text processors transform files in a special format that is wrong to
make them stay a good batch file, so don't use them!
Files FAQ.txt and FAQ.d/* contain many tips and special options sometimes
needed by specific imap software servers like Exchange, Office365, or Gmail.
Those files are also available online at
The documents FAQ.txt and FAQ.d/* contain many tips. They describe
some special options that are sometimes needed by specific imap
software servers like Exchange, Office365, or Gmail.
Those documents are also available online at
https://imapsync.lamiral.info/FAQ.d/
You don't have to look into them unless you encounter problems.
A.5) Run the batch file
To run imapsync with your values just double-click on the batch file mysync.bat
To run imapsync with your values just double-click on the batch file
mysync.bat
There is no need to have administrator privileges to run imapsync.
The run happens in a DOS window; usually, this window is black.
If imapsync.exe returns immediately with the ERRORLEVEL -1, it may be
because you have a Group Policy in place to prevent the execution of
programs in the %TEMP% directory. Temporarily remove this restriction
and imapsync will work as expected. Thanks to Walter H. for this
input!
If imapsync.exe returns immediately with the ERRORLEVEL -1, it may be
because you have an Group Policy in place to prevent execution of
programs in %TEMP%. Temporarily remove this restriction and imapsync
will work as expected. Thanks to Walter H. for this input!
Technically imapsync.exe is an embedded Perl script with also Perl
and many Perl modules glued together in an archive auto-extracted.
Technically speaking, imapsync.exe is an embedded Perl script with
also the Perl interpreter and many Perl modules, all glued together
in an archive auto-extracted at run time. So it needs write-access to
the temporary directory. The temporary directory name depends on the
user but its value is in the variable %TEMP%. You can have the value
by running the command ECHO %TEMP% in a DOS window. You can have a
DOS window by launching the command cmd.exe
A.6) Look at the sync running.
You can abort the sync at any time with a quick double ctrl-c,
ie hit ctrl-c twice within one second.
You can abort the sync at any time with a quick double ctrl-c, hit
ctrl-c twice within one second.
A single ctrl-c will reconnect to both imap servers.
You can also simply abort the sync by closing the DOS window,
using the cross situated at the up-right corner.
You can also simply abort the sync by closing the DOS window, using
the cross situated at the up-right corner.
What you see in this DOS terminal is also put in a logfile located
in the subdirectory LOG_imapsync/
A.7) Control what happened.
When the sync is finished you can find the whole log file
of the output in the folder named "LOG_imapsync/".
The logfile name is based on the launching date,
hour, minute, second, milliseconds, plus the user1 and user2 parameters.
When the sync is finished you can find the whole log file of the
output in the folder named "LOG_imapsync/".
The logfile name is based on the launching date, hour, minute,
second, milliseconds, plus the user1 and user2 parameters.
For example, a file name can be
LOG_imapsync\2019_11_29_14_49_36_514_tata_titi.txt
There is one log file created for each run.
The log file name is printed at the end of the imapsync run.
There is one log file created for each run. The log file name is
printed both at the beginning and the end of the imapsync run.
IMPORTANT: When there is a problem, the problem is very often described
at the end of the log file. It means you don't have to read all this
bloody ununderstandable verbose logfile, just read the end first.
IMPORTANT: When there is a problem, the problem is very often
described at the end of the log file. It means you don't have to read
all this bloody ununderstandable verbose logfile, just read the end
first.
A.8) Loop on A.4 through A.7
@ -173,7 +191,7 @@ A.8) Loop on A.4 through A.7
* A.6) look at the run and the log file
* A.7) control what happened.
Loop on the process of editing, running, and controlling imapsync
Loop on the process of editing, running and controlling imapsync
until you solve all issues and the sync is over.
A good sign that the sync went very well is when the nearly last lines are like:
@ -195,8 +213,8 @@ This is the way for modifying imapsync.exe if needed.
B.1) Install Perl if it isn't already installed.
Strawberry Perl is a very good candidate
http://strawberryperl.com/
I use 5.30.1.1 (released 2019-11-22) but previous and later releases
should work as well (Perl 5.18 to 5.26 do).
I use 5.32.1.1 (released 2021-01-24) but previous and later releases
should work as well (Perl 5.18 to 5.30 do).
B.2) Go into the Cook/ directory
B.3) Double-click build_exe.bat
@ -205,4 +223,3 @@ It should create a binary imapsync.exe in the current Cook/ directory.
B.4) Move imapsync.exe in the upper directory and follow instructions
from A.3) to A.8)

10
S/external.shtml
View file

@ -65,9 +65,13 @@
<li> MailStore: <a href="https://www.mailstore.com/en/products/mailstore-home/">https://www.mailstore.com/en/products/mailstore-home/</a></li>
<li> mnIMAPSync: <a href="https://github.com/manusa/mnIMAPSync">https://github.com/manusa/mnIMAPSync</a></li>
<li> imap-upload: <a href="http://imap-upload.sourceforge.net/">http://imap-upload.sourceforge.net/</a>
(A tool for uploading a local mbox file to IMAP4 server)</li>
(A tool to upload a local mbox file to IMAP4 server)</li>
<li> imapbackup: <a href="https://github.com/rcarmo/imapbackup">https://github.com/rcarmo/imapbackup</a>
(A Python script for incremental backups of IMAP mailboxes)</li>
(A Python script to incremental backups of IMAP mailboxes)</li>
<li> imap-backup: <a href="https://github.com/joeyates/imap-backup">https://github.com/joeyates/imap-backup</a>
(A Ruby script to backup GMail (or other IMAP) accounts to disk)</li>
<li> BitRecover email-backup <b>99 USD, 299 USD</b> <a href="https://www.bitrecover.com/email-backup/">https://www.bitrecover.com/email-backup/</a>.</li>
@ -182,7 +186,7 @@ alt="Viewable With Any Browser" />
<!--#config timefmt="%D" -->
<!--#config timefmt="%A %B %d, %Y" -->
<b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b>
($Id: external.shtml,v 1.58 2021/06/30 20:34:23 gilles Exp gilles $)<br/>
($Id: external.shtml,v 1.59 2022/01/08 19:54:26 gilles Exp gilles $)<br/>
<a href="#TOP">Top of the page</a>
</p>

BIN
S/images/5c019d917bba312af7553b49_MacStadium-developerlogo.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 54 KiB

70
S/news.shtml
View file

@ -85,6 +85,72 @@ If you are very interrested in those projects, don't hesitate to solicit me!
</ul>
-->
<ul>
<li><b>2.178</b></li>
<li><b>General</b>: </li>
<li><b>Dependencies removed</b>: None </li>
<li><b>Dependencies added</b>: Perl module Compress::Zlib</li>
<li><b>Enhancement</b>: Added final statistic line: "CPU time and %cpu : 13.27 sec 8.3 %cpu 2.1 %allcpus</li>
<li><b>Enhancement</b>: Removed the hard limit --maxsize 35_651_584 set by --gmail2 since CAPABILITY is used and good.</li>
<li><b>Enhancement</b>: Does not checknoabletosearch if --justfolders</li>
<li><b>Enhancement</b>: When creating a folder, check if it is selectable and consider ok if it is selectable.</li>
<li><b>Enhancement</b>: Options --gmail1 --gmail2 together sets --useheader X-Gmail-Received --useheader Message-Id</li>
<li><b>Enhancement</b>: Added compression in the imap connexions.
Compression is off by default no matter the capability announced.
Use --compress1 and --compress2 to turn compression on.</li>
<li><b>Enhancement</b>: Added --keepalive1 and --keepalive2 options.
Some firewalls and network gears like to timeout connections prematurely if the connection sits idle.
This option enables SO_KEEPALIVE on the host1 socket.
--keepalive1 is on by default since imapsync release 2.169
Use --nokeepalive1 to disable it. Same thing for --keepalive2 but for --host2. See <a href="https://metacpan.org/pod/Mail::IMAPClient#Keepalive">https://metacpan.org/pod/Mail::IMAPClient#Keepalive</a></li>
<li><b>Enhancement</b>: </li>
<li><b>Enhancement</b>: </li>
<li><b>Usability</b>: Removed --fast option. It didn't do anything anyway, it was a fake option.</li>
<li><b>Usability</b>: Added advice inline advice on errors
<ul>
<li>ERR_OVERQUOTA. "The destination mailbox is 100% full, get free space on it and then resume the sync."</li>
<li>ERR_TRANSFER_EXCEEDED. "The maximum transfer size for a single sync is reached ( over 3 GB ). Relaunch the sync to sync more."</li>
<li>ERR_APPEND_SIZE. "The destination server refuses too big messages. Use --truncmess option. Read https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Too_Big.txt" </li>
<li>ERR_CONNECTION_FAILURE_HOST1. "Check that host1 on port1 is the right IMAP server to be contacted for your mailbox."</li>
<li>ERR_CONNECTION_FAILURE_HOST2. "Check that host2 on port2 is the right IMAP server to be contacted for your mailbox."</li>
<li>ERR_AUTHENTICATION_FAILURE_USER1. "Check the credentials for user1."</li>
<li>ERR_AUTHENTICATION_FAILURE_USER2. "Check the credentials for user2."</li>
</ul>
</li>
<li><b>Usability</b>: Bytes sizes human readable are now presented in KB MB GB TB PB (1000 base) instead of KiB MiB GiB TiB PiB (1024 base)</li>
<li><b>Usability</b>: Added README help for --truncmess</li>
<li><b>Usability</b>: </li>
<li><b>Usability</b>: </li>
<li><b>Usability</b>: </li>
<li><b>Usability</b>: </li>
<li><b>CGI context</b>: Experimental loaddelay forced to 0 (no delay at all). So far a huge load is rarely a problem.</li>
<li><b>CGI context</b>: Bugfix. Abort didn't set charset=UTF-8; same thing for heavy load 503 Service Unavailable</li>
<li><b>Bug fix</b>: Made reconnect mechanism works with --oauthdirect1 --oauthdirect2 --oauthaccesstoken1 --oauthaccesstoken2</li>
<li><b>Bug fix</b>: Upped the maximum number a pid can be. From 99999 to 2^22.
See <a href="https://unix.stackexchange.com/questions/16883/what-is-the-maximum-value-of-the-process-id">what-is-the-maximum-value-of-the-process-id</a> </li>
<li><b>Bug fix</b>: Load average on Mac was sometimes buggy. locale float 3,14 instead of 3.14</li>
<li><b>Bug fix</b>: errors_incr() exited with CATCH_ALL</li>
<li><b>Bug fix</b>: Memory consumption on Mac was not relevant. Use RSS instead of VSZ, on Mac only. Linux stays with VSZ.</li>
<li><b>Bug fix</b>: </li>
<li><b>Bug fix</b>: </li>
<li><b>Docker context</b>: Current working directory changed from /var/tmp to /var/tmp/uid_$EFFECTIVE_USER_ID</li>
</ul>
<ul>
<li><b>2.140</b></li>
<li><b>General</b>: New release numbering from 1.999 directly to 2.100; I want to avoid 1.1000 and sorting issues in directory listing.</li>
@ -157,8 +223,6 @@ Exit value 112 (EXIT_WITH_ERRORS_MAX) is now replaced by the most common error e
<li><b>Refactoring</b>: Continuing to deglobalize the global variables.</li>
<li><b>Refactoring</b>: 1734 unit tests.</li>
<li><b>Refactoring</b>: </li>
</ul>
@ -954,7 +1018,7 @@ by ignoring PERMANENTFLAGS (Exchange tests)</li>
<!--#config timefmt="%D" -->
<!--#config timefmt="%A %B %d, %Y" -->
<b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b>
($Id: news.shtml,v 1.78 2021/07/05 20:22:15 gilles Exp gilles $)<br />
($Id: news.shtml,v 1.83 2022/01/13 18:36:12 gilles Exp gilles $)<br />
<a href="#TOP">Top of the page</a>
</p>
</body></html>

29
S/quiz/README.txt Normal file
View file

@ -0,0 +1,29 @@
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
LICENSE
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Copyright by Code Boxx
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
MORE
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Please visit https://code-boxx.com/ for more!

45
S/quiz/comment_to_toh.txt Executable file
View file

@ -0,0 +1,45 @@
Hi Toh!
First, I thank you very much for posting and sharing this good piece
of code!
I like it because, it's simple, it's embedable, no dependency, pure
javascript.
I found your code search for a simple quiz code. I consider quizzes as
a fun learning learning experience on something.
Some things I'd like to improve on it:
ux/reload) The user has to reload the page to restart the quiz and
it's not evident. How about making the last button a possibility
to redo the quiz from start and make a better score?
ux/where I am) The user doesn't know how many questions there are in
the quiz, where he is along the quiz, how good he is. Knowing where we
are in a serie makes it easier to reach the end. This one is easy to
add, as I could do it.
css functional) The quiz.css is not so cosmetic: if you comment it in the
html, then the quiz functionally doesn't work the same; we have to
click twice on a button to go to the next question, and the second
click has to be outside the round of the radio button, on the label.
I don't know how to fix that but it's not important anyway, the css is
needed for cosmetics.
css cosmetic) this quiz code is to be embedded in a page then I suggest
that the cosmetics are inheritated.
I'm also adept of minimal css, the less to achieve a goal, the better.
dev testing) I don't know how do you test it while coding and
modifying it. A manual loop browser_reload_run_editor?
dev jslint) I paste it to jslint (https://www.jslint.com/)
and it didn't pass the lint test.
dev data load) How about putting all the Q/R in a dedicated text file
to maintain a single quiz.js and quiz.css and have multiple quizzes
using the same functional code?

46
S/quiz/quiz.css Normal file
View file

@ -0,0 +1,46 @@
/* (A) WRAPPER */
#quizWrap {
max-width: 600px;
margin: 0 auto;
}
/* (B) QUESTION */
#quizQn {
padding: 20px;
background: #4c93ba;
color: #fff;
font-size: 24px;
border-radius: 10px;
}
/* (C) ANSWERS */
#quizAns {
margin: 10px 0;
display: grid;
grid-template-columns: auto auto;
grid-gap: 10px;
}
#quizAns input[type=radio] { display: none; }
#quizAns label {
background: #fafafa;
border: 1px solid #eee;
border-radius: 10px;
padding: 10px;
font-size: 20px;
cursor: pointer;
text-align: center;
}
#quizAns label.correct {
background: #d8ffc4;
border: 1px solid #60a03f;
}
#quizAns label.wrong {
background: #ffe8e8;
border: 1px solid #c78181;
}
/* (D) BODY... DOES NOT QUITE MATTER */
html, body {
background: #74b6db;
font-family: arial, sans-serif;
}

16
S/quiz/quiz.html Normal file
View file

@ -0,0 +1,16 @@
<!DOCTYPE html>
<html>
<head>
<title>
Simple Javascript Quiz Demo
</title>
<!-- (A) LOAD QUIZ CSS + JS -->
<link href="quiz.css" rel="stylesheet">
<script src="quiz.js"></script>
</head>
<body>
<!-- (B) QUIZ CONTAINER -->
<div id="quizWrap"></div>
</body>
</html>

136
S/quiz/quiz.js Normal file
View file

@ -0,0 +1,136 @@
var quiz = {
// (A) PROPERTIES
// (A1) QUESTIONS & ANSWERS
// Q = QUESTION, O = OPTIONS, A = CORRECT ANSWER
data: [
{
q : "What is the standard distance between the target and archer in Olympics?",
o : [
"50 meters",
"70 meters",
"100 meters",
"120 meters"
],
a : 1 // arrays start with 0, so answer is 70 meters
},
{
q : "Which is the highest number on a standard roulette wheel?",
o : [
"22",
"24",
"32",
"36"
],
a : 3
},
{
q : "How much wood could a woodchuck chuck if a woodchuck would chuck wood?",
o : [
"400 pounds",
"550 pounds",
"700 pounds",
"750 pounds"
],
a : 2
},
{
q : "Which is the seventh planet from the sun?",
o : [
"Uranus",
"Earth",
"Pluto",
"Mars"
],
a : 0
},
{
q : "Which is the largest ocean on Earth?",
o : [
"Atlantic Ocean",
"Indian Ocean",
"Arctic Ocean",
"Pacific Ocean"
],
a : 3
}
],
// (A2) HTML ELEMENTS
hWrap: null, // HTML quiz container
hQn: null, // HTML question wrapper
hAns: null, // HTML answers wrapper
// (A3) GAME FLAGS
now: 0, // current question
score: 0, // current score
// (B) INIT QUIZ HTML
init: function(){
// (B1) WRAPPER
quiz.hWrap = document.getElementById("quizWrap");
// (B2) QUESTIONS SECTION
quiz.hQn = document.createElement("div");
quiz.hQn.id = "quizQn";
quiz.hWrap.appendChild(quiz.hQn);
// (B3) ANSWERS SECTION
quiz.hAns = document.createElement("div");
quiz.hAns.id = "quizAns";
quiz.hWrap.appendChild(quiz.hAns);
// (B4) GO!
quiz.draw();
},
// (C) DRAW QUESTION
draw: function(){
// (C1) QUESTION
quiz.hQn.innerHTML = quiz.data[quiz.now].q;
// (C2) OPTIONS
quiz.hAns.innerHTML = "";
for (let i in quiz.data[quiz.now].o) {
let radio = document.createElement("input");
radio.type = "radio";
radio.name = "quiz";
radio.id = "quizo" + i;
quiz.hAns.appendChild(radio);
let label = document.createElement("label");
label.innerHTML = quiz.data[quiz.now].o[i];
label.setAttribute("for", "quizo" + i);
label.dataset.idx = i;
label.addEventListener("click", quiz.select);
quiz.hAns.appendChild(label);
}
},
// (D) OPTION SELECTED
select: function(){
// (D1) DETACH ALL ONCLICK
let all = quiz.hAns.getElementsByTagName("label");
for (let label of all) {
label.removeEventListener("click", quiz.select);
}
// (D2) CHECK IF CORRECT
let correct = this.dataset.idx == quiz.data[quiz.now].a;
if (correct) {
quiz.score++;
this.classList.add("correct");
} else {
this.classList.add("wrong");
}
// (D3) NEXT QUESTION OR END GAME
quiz.now++;
setTimeout(function(){
if (quiz.now < quiz.data.length) { quiz.draw(); }
else {
quiz.hQn.innerHTML = `You have answered ${quiz.score} of ${quiz.data.length} correctly.`;
quiz.hAns.innerHTML = "";
}
}, 1000);
}
};
window.addEventListener("load", quiz.init);

83
S/quiz/quiz_imapsync.css Executable file
View file

@ -0,0 +1,83 @@
/* Taken from https://code-boxx.com/simple-javascript-quiz */
/* (A) WRAPPER */
#quizWrap {
border-style: solid;
border-collapse: collapse;
background: #e0e0e0;
}
#quizNumber,#quizScore{
text-align: center;
padding: 10px;
border-width: thin;
border-style: solid;
border-left-style: none;
border-right-style: none;
}
#quizScore{
border-bottom-style: none;
}
#quizRestart {
margin: auto;
margin-bottom: 10px;
text-align: center;
padding: 10px;
border-radius: 10px;
border-style: solid;
border-width: thin;
background: #d8ffc4;
width: 100px;
cursor: pointer;
}
#quizQn {
padding: 20px;
text-align: center;
font-weight: bold;
}
/* (C) ANSWERS */
#quizAns {
margin: 10px;
text-align: center;
border-width: thin;
display: grid;
grid-template-columns: auto auto;
grid-gap: 10px;
justify-content: space-evenly;
align-content: space-evenly;
}
#quizAns input[type=radio] { display: none; }
#quizAns label {
/* background: #cccccc;
border: 1px solid #eee;
font-size: 20px; */
cursor: pointer;
background: #ffffff;
font-weight: bold;
padding: 10px;
border-radius: 10px;
border-style: solid;
border-width: thin;
margin-top: 5px;
margin-left: 25px;
margin-right: 25px;
margin-bottom: 5px;
}
#quizAns label.correct {
background: #d8ffc4;
border: 1px solid #60a03f;
}
#quizAns label.wrong {
background: #ffe8e8;
border: 1px solid #c78181;
}

20
S/quiz/quiz_imapsync.html Executable file
View file

@ -0,0 +1,20 @@
<!DOCTYPE html>
<!-- Taken from https://code-boxx.com/simple-javascript-quiz -->
<html>
<head>
<title>
Imapsync Quiz
</title>
<!-- (A) LOAD QUIZ CSS + JS -->
<link href="quiz_imapsync.css" rel="stylesheet">
<script src="quiz_imapsync.js"></script>
</head>
<body>
<!-- (B) QUIZ CONTAINER -->
<div id="quizWrap"></div>
</body>
</html>

245
S/quiz/quiz_imapsync.js Executable file
View file

@ -0,0 +1,245 @@
// Taken from https://code-boxx.com/simple-javascript-quiz
var quiz = {
// (A) PROPERTIES
// (A1) QUESTIONS & ANSWERS
// Q = QUESTION, O = OPTIONS, A = CORRECT ANSWER
data: [
{
a : 1,
o : [
"Facebook",
"POP, IMAP, HTTP",
"SMTP, ICQ",
"Twitter",
"FTP, GOPHER",
"Instagram"
],
q : "What are the main protocols usually used to access a mailbox?"
},
{
a : 2,
o : [
"I'M A Passager",
"Instant Mailbox Access Possibility",
"Internet Message Access Protocol",
"In My Ambiguous Posture",
"In My Ambience Place",
"Izorc Muggle Ark Prout"
],
q : "What means the acronym IMAP?"
},
{
a : 2,
o : [
"No parameter is needed",
"Harry Potter's magic wang",
"1) The IMAP server name + 2) the user login + 3) the password",
"1) the user login + 2) the password"
],
q : "What are the parameters needed to access an IMAP mailbox?"
},
{
a : 4,
o : [
"0",
"2",
"3",
"4",
"6",
"8"
],
q : "How many parameters are mandatory to synchronize two IMAP mailboxes with imapsync?"
},
{
a : 0,
o : [
"No",
"Yes"
],
q : "Can Imapsync synchronize POP accounts?"
},
{
a : 1,
o : [
"No",
"Yes"
],
q : "Can Imapsync synchronize Gmail accounts?"
},
{
a : 1,
o : [
"No",
"Yes"
],
q : "Can Imapsync synchronize Office365 accounts?"
},
{
a : 0,
o : [
"No",
"Yes"
],
q : "Can Imapsync synchronize Contacts, Calendars, Chats, or Notes?"
},
{
a : 0,
o : [
"NO LIMIT",
"GPL",
"WTFPL",
"MIT",
"CC0",
"Proprietary"
],
q : "What is the Imapsync License name?"
},
{
a : 0,
o : [
"No limits to do anything with this work and this license.",
"All permissions restricted.",
"Use it for good or evil.",
"Too long to fit here."
],
q : "What is the Imapsync License main text?"
},
{
a : 0,
o : [
"Yes, sometimes with delay, he's a human",
"No, never, email is dead",
"Yes, immediately and 24h/24, he's a robot",
"What is email?"
],
q : "Does the imapsync author Gilles LAMIRAL reply to every email?"
}
],
dataDev: [
{
a : 1,
o : [
"Facebook",
"POP, IMAP, HTTP",
"SMTP, ICQ",
"Twitter",
"FTP, GOPHER",
"Instagram"
],
q : "What are the main protocols usually used to access a mailbox?"
}
],
// (B) INIT QUIZ HTML
init: function(){
// (B1) WRAPPER
quiz.hWrap = null;
quiz.hScore = null;
quiz.hRestart = null;
quiz.hQn = null;
quiz.hAns = null;
quiz.now = 0;
quiz.score = 0;
quiz.hWrap = document.getElementById("quizWrap");
quiz.hWrap.innerHTML = "";
// NUMBER SECTION
quiz.hNumber = document.createElement("div");
quiz.hNumber.id = "quizNumber";
quiz.hWrap.appendChild(quiz.hNumber);
// (B2) QUESTIONS SECTION
quiz.hQn = document.createElement("div");
quiz.hQn.id = "quizQn";
quiz.hWrap.appendChild(quiz.hQn);
// (B3) ANSWERS SECTION
quiz.hAns = document.createElement("div");
quiz.hAns.id = "quizAns";
quiz.hWrap.appendChild(quiz.hAns);
// SCORE SECTION
quiz.hScore = document.createElement("div");
quiz.hScore.id = "quizScore";
quiz.hWrap.appendChild(quiz.hScore);
// RESTART SECTION
quiz.hRestart = document.createElement("div");
quiz.hRestart.id = "quizRestart";
quiz.hWrap.appendChild(quiz.hRestart);
quiz.hRestart.innerHTML = "Restart Quiz";
document.getElementById("quizRestart").addEventListener("click", quiz.restart);
// (B4) GO!
quiz.draw();
},
// (C) DRAW QUESTION
draw: function(){
// SCORE
var QnNumber = `${quiz.now + 1}/${quiz.data.length}`;
var ScoreNb = `${quiz.score}/${quiz.now}`;
quiz.hNumber.innerHTML = `Question ${QnNumber}`;
quiz.hScore.innerHTML = `Score ${ScoreNb}`;
// (C1) QUESTION
quiz.hQn.innerHTML = quiz.data[quiz.now].q;
// (C2) OPTIONS
quiz.hAns.innerHTML = "";
for (let i in quiz.data[quiz.now].o) {
let radio = document.createElement("input");
radio.type = "radio";
radio.name = "quiz";
radio.id = "quizo" + i;
quiz.hAns.appendChild(radio);
let label = document.createElement("label");
label.innerHTML = quiz.data[quiz.now].o[i];
label.setAttribute("for", "quizo" + i);
label.dataset.idx = i;
label.addEventListener("click", quiz.select);
quiz.hAns.appendChild(label);
}
},
// (D) OPTION SELECTED
select: function(){
// (D1) DETACH ALL ONCLICK
let all = quiz.hAns.getElementsByTagName("label");
for (let label of all) {
label.removeEventListener("click", quiz.select);
}
// (D2) CHECK IF CORRECT
let correct = this.dataset.idx == quiz.data[quiz.now].a;
if (correct) {
quiz.score++;
this.classList.add("correct");
} else {
this.classList.add("wrong");
}
// (D3) NEXT QUESTION OR END GAME
quiz.now++;
setTimeout(function(){
if (quiz.now < quiz.data.length) { quiz.draw(); }
else {
// quiz.hNumber.innerHTML = "";
ScoreNb = `${quiz.score}/${quiz.now}`;
quiz.hScore.innerHTML = `Score ${ScoreNb}`;
quiz.hQn.innerHTML = "Finished!";
quiz.hAns.innerHTML = "";
}
}, 800);
},
restart:function(){
quiz.init() ;
}
};
window.addEventListener("load", quiz.init);

11
S/testimonial.shtml
View file

@ -42,6 +42,15 @@ I'm not sure I deserve such great quotes for my work on Imapsync, anyway here th
</p>
<!--#include file="S/tw-mention.html" -->
<blockquote class="twitter-tweet">
<p lang="en" dir="ltr">
imapsync is simply the best and most usable tool ever developed. Works perfect!
Thank you <a href="https://twitter.com/imapsync?ref_src=twsrc%5Etfw">@imapsync</a></p>&mdash; bkcrri (@bkcrri)
<a href="https://twitter.com/bkcrri/status/1456392869808328706?ref_src=twsrc%5Etfw">November 4, 2021</a>
</blockquote>
<hr>
<blockquote class="twitter-tweet">
<p lang="en" dir="ltr">What’s your favorite IMAP migration/sync tool? I made the best experiences with
<a href="https://twitter.com/imapsync?ref_src=twsrc%5Etfw">@imapsync</a>.
@ -157,7 +166,7 @@ alt="Viewable With Any Browser" >
<!--#config timefmt="%D" -->
<!--#config timefmt="%A %B %d, %Y" -->
<b>This document was last modified on <!--#echo var="LAST_MODIFIED" --></b>
($Id: testimonial.shtml,v 1.8 2019/11/15 11:35:16 gilles Exp gilles $)<br>
($Id: testimonial.shtml,v 1.9 2021/11/05 12:04:12 gilles Exp gilles $)<br>
<a href="#TOP">Top of the page</a>
</p>
</footer>

22
TODO
View file

@ -1,5 +1,5 @@
#!/bin/cat
# $Id: TODO,v 1.257 2021/07/05 20:19:31 gilles Exp gilles $
# $Id: TODO,v 1.262 2022/01/11 12:56:18 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
@ -9,6 +9,17 @@ TODO file for imapsync
https://imapsync.lamiral.info/TODO
SUGGESTED 2022_01_09 by Neustradamus
Add SCRAM-SHA* support
https://github.com/imapsync/imapsync/issues/313
SUGGESTED 2021_06_13 by rotemes
https://github.com/imapsync/imapsync/issues/288
Add a counter for skipped because of already copied and
because of other things.
What ENVELOPE could bring?
( 'FLAGS', 'INTERNALDATE', 'RFC822.SIZE' ) ;
( 'FLAGS', 'INTERNALDATE', 'RFC822.SIZE' , 'ENVELOPE' )
SUGGESTED 2021_05_29 on github https://github.com/imapsync/imapsync/issues/78
ripmime https://github.com/inflex/ripMIME
@ -696,6 +707,15 @@ http://asg.web.cmu.edu/cyrus/download/imapd/altnamespace.html
Now the TODO done! (or not)
===========================================================================
DONE revision 2.169 date: 2021/12/07 17:13:44;
Added --keepalive1 and --keepalive2 options. On by default.
See https://metacpan.org/pod/Mail::IMAPClient#Keepalive
DONE revision 2.159 date: 2021/11/19 14:29:31
SUGGESTED 2021_11_18 by Justas Umbrasas
Add COMPRESS to imap connexions
--compress1 --compress2
DONE 2021/04/30 revision 2.111 by Gilles
SUGGESTED 2021_04_22 by popular request
Add an option --syncduplicates that syncs duplicates.

2
VERSION
View file

@ -1 +1 @@
2.140
2.178

266
X/cgi_memo
View file

@ -1,6 +1,6 @@
#!/bin/sh
# $Id: cgi_memo,v 1.73 2021/06/30 20:10:12 gilles Exp gilles $
# $Id: cgi_memo,v 1.74 2021/11/25 11:25:13 gilles Exp gilles $
if test -n "$1"; then
echoq() { echo "$@" ; } # not quiet mode
@ -89,7 +89,7 @@ here_is_linux()
echoq list_all_logs
list_all_logs()
{
cat list_all_logs.txt
cat list_all_logs.txt list_all_logs_auto.txt | grep -v 385d7a4d8d428d7aa2b57c8982629e2bd67698ed/ | grep "$1"
}
echoq list_all_logs_generate
@ -108,17 +108,27 @@ list_log_matching()
}
statsfile()
{
base=`pattern_filename G $1 $2`
statsfile=$base.txt
echo $statsfile
}
echoq biggest_transfer
biggest_transfer()
{
bytestohuman `datamash_file_op_index G_Total_bytes_transferred.txt max 5`
statsfile=`statsfile Total_bytes_transferred $1`
bytestohuman `datamash_file_op_index "$statsfile" max 5`
}
echoq total_bytes_transferred
total_bytes_transferred()
{
datamash_file_op_index G_Total_bytes_transferred.txt sum 5
statsfile=`statsfile Total_bytes_transferred $1`
datamash_file_op_index "$statsfile" sum 5
}
# Total volume transferred
@ -126,20 +136,22 @@ echoq total_volume_transferred
total_volume_transferred()
{
#echo -n 'numfmt --to=iec-i '
bytestohuman `total_bytes_transferred`
bytestohuman `total_bytes_transferred $1`
}
echoq total_messages_transferred
total_messages_transferred()
{
datamash_file_op_index G_Messages_transferred.txt sum 4 %16.0f | tr -d ' '
statsfile=`statsfile Messages_transferred $1`
datamash_file_op_index "$statsfile" sum 4 %16.0f | tr -d ' '
}
longest_transfer()
{
LC_ALL=C printf "%.0f\n" `datamash_file_op_index G_Transfer_time.txt max 4`
statsfile=`statsfile Transfer_time $1`
LC_ALL=C printf "%.0f\n" `datamash_file_op_index "$statsfile" max 4`
}
echoq number_and_pids_of_imapsync_running
@ -178,9 +190,15 @@ oom_immune_imapsync_running() {
echoq nb_migrations_launched
nb_migrations_launched() {
/bin/ls . | egrep [a-f0-9]{40} | wc -l
list_all_logs | egrep "$1" | egrep -o [a-f0-9]{40} | sort | uniq -c | wc -l
}
nb_migrations_launched_old() {
/bin/ls . | egrep [a-f0-9]{40} | sort | uniq | wc -l
}
echoq current_stats
current_stats() {
echo -n "Nb accounts: "; nb_migrations_launched
@ -213,9 +231,10 @@ grep_in_all_logs() {
}
echoq grep_in_logs_manual
grep_in_logs_manual() {
cat << EOF
list_all_logs | tail -500 | tr '\n' '\000'| xargs -0 egrep -i LALALA | tee grep_LALALA.txt
grep_in_logs_manual()
{
cat << EOF
list_all_logs /2021_11 | tail -666 | tr '\n' '\000'| xargs -0 egrep -i LALALA | tee grep_LALALA.txt
EOF
}
@ -245,7 +264,7 @@ tests_pattern_filename()
run_test test "abcd" = "`pattern_filename ab""cd`"
run_test test "abcd" = "`pattern_filename ab""cd`"
run_test test "ab_cd" = "`pattern_filename ab" "cd`"
run_test test "ab__cd" = "`pattern_filename ab" "cd`"
run_test test "ab__cd" = "`pattern_filename ab" "cd`"
run_test test "ab__cd" = "`pattern_filename ab "" cd`"
run_test test "ab___cd" = "`pattern_filename ab " " cd`"
run_test test "ab____cd" = "`pattern_filename ab " " cd`"
@ -263,7 +282,7 @@ pattern_filename()
echoq 'grep_stats_from_list_log_matching lognamepattern'
echoq 'grep_stats_from_list_log_matching lognamepattern # time depending on lognamepattern'
grep_stats_from_list_log_matching() {
pattern="$1"
pattern_filename=`pattern_filename "$pattern"`
@ -277,32 +296,42 @@ grep_stats_from_list_log_matching() {
grep_any() {
pattern_filtered=`pattern_filename "$1" "$2"`
file="G_${pattern_filtered}.txt"
echo $file
egrep -i "$1" grep_stats_"$2".txt > $file.tmp
mv $file.tmp $file
file=`statsfile "$1" "$2"`
pattern_filtered=`pattern_filename "$2"`
echo $file
if test -f grep_stats_"$pattern_filtered".txt ; then
egrep -i "$1" grep_stats_"$pattern_filtered".txt > $file.tmp
mv $file.tmp $file
else
echo File not found: grep_stats_"$pattern_filtered".txt
fi
}
grep_load() {
echo G_Load.txt
egrep -o 'Load is ..?\... ..?\... ..?\... .*' grep_stats.txt > G_Load.txt
file=`statsfile "Load" "$1"`
pattern_filtered=`pattern_filename "$1"`
echo $file
if test -f grep_stats_"$pattern_filtered".txt ; then
egrep -o 'Load is ..?\... ..?\... ..?\... .*' grep_stats_"$pattern_filtered".txt > $file.tmp
mv $file.tmp $file
else
echo File not found: grep_stats_"$pattern_filtered".txt
fi
}
echoq 'grep_all_stat_from_patterns_list lognamepattern # long'
grep_all_stat_from_patterns_list() {
#grep_load
stat_patterns_list | while read k; do grep_any "$k" "$1"; done
}
stat_patterns_list() {
cat stat_patterns.txt | sed '/^[[:space:]]*$/d' | tr -d '^'
}
echoq 'grep_all_stat_from_patterns_list lognamepattern # long'
grep_all_stat_from_patterns_list() {
grep_load "$1"
stat_patterns_list | while read k; do grep_any "$k" "$1"; done
}
sum_first_column_G_HTTP_USER_AGENT_sorted()
{
@ -348,18 +377,20 @@ stat_load()
echoq stat_exit_value
stat_exit_value()
{
good_lines=`grep '(EX' G_Exiting_with_return_value.txt | wc -l | tr -d ' '`
grep '(EX' G_Exiting_with_return_value.txt \
statsfile=`statsfile Exiting_with_return_value "$1"`
good_lines_nb=`grep '(EX' $statsfile | wc -l | tr -d ' '`
grep '(EX' "$statsfile" \
| datamash --sort groupby 6 -W count 5 \
| awk -v good_lines=$good_lines \
'{ printf "%.2g%% %s\n", 100*$2/good_lines, $1 }' \
| awk -v good_lines_nb=$good_lines_nb \
'{ printf "%.2g%% %s\n", 100*$2/good_lines_nb, $1 }' \
| sort -n
}
echoq stat_exit_value_by_value
stat_exit_value_by_value()
{
datamash --sort groupby 5 -W count 5 < G_Exiting_with_return_value.txt
statsfile=`statsfile Exiting_with_return_value "$1"`
datamash --sort groupby 5 -W count 5 < "$statsfile"
}
@ -423,8 +454,8 @@ stat_all()
stat_any G_Messages_found_in_host2_not_in_host1.txt 9
# stat_any G_Folders_synced.txt
egrep -o '[0-9]+/[0-9]+' G_Folders_synced.txt | egrep -o '^[0-9]+' > G_Folders_synced_.txt
egrep -o '[0-9]+/[0-9]+' G_Folders_synced.txt | egrep -o '[0-9]+$' > G_Folders_total_seen.txt
egrep -o '[0-9]+/[0-9]+ synced' G_Folders_synced.txt | egrep -o '^[0-9]+' > G_Folders_synced_.txt
egrep -o '[0-9]+/[0-9]+ synced' G_Folders_synced.txt | egrep -o '[0-9]+/[0-9]+' | egrep -o '[0-9]+$' > G_Folders_total_seen.txt
stat_any G_Folders_synced_.txt 1
stat_any G_Folders_total_seen.txt 1
@ -437,7 +468,7 @@ stat_all()
stat_any G_Average_bandwidth_rate.txt 5
stat_any G_Biggest_message.txt
stat_any G_Detected_errors.txt 2
stat_any G_Exiting_with_return_value.txt 5 # GROUP
#stat_any G_Exiting_with_return_value.txt 5 # GROUP
stat_any G_Memory_consumption_at_the_end.txt 7
#stat_any G_failure_Error_login.txt
percent_stat_useragent_X ; echo
@ -448,24 +479,62 @@ stat_all()
stat_transfer_time_mean()
{
datamash_file_op_index G_Transfer_time.txt mean
statsfile=`statsfile Transfer_time "$1"`
datamash_file_op_index "$statsfile" mean
}
stat_throuput_since_day_one_in_days()
{
number_of_syncs=`number_of_syncs`
days_since_first_use=`days_since_first_use`
number_of_syncs=`number_of_syncs "$1"`
days_since_first_use=`days_since_first_use "$1"`
c "$number_of_syncs / $days_since_first_use"
}
stat_queue_mean()
stat_queue_mean_old()
{
stat_throuput_since_day_one_in_days=`stat_throuput_since_day_one_in_days`
stat_transfer_time_mean=`stat_transfer_time_mean`
stat_transfer_time_mean=`stat_transfer_time_mean "$1"`
stat_queue_mean_raw=`c "$stat_throuput_since_day_one_in_days * $stat_transfer_time_mean / 3600 / 24"`
LC_ALL=C printf "%2.2f\n" $stat_queue_mean_raw
}
stat_queue_mean()
{
first_log=`first_log "$1"`
last_log=`last_log "$1"`
number_of_syncs=`number_of_syncs "$1"`
seconds_between_files=`seconds_between_files $first_log $last_log`
stat_transfer_time_mean=`stat_transfer_time_mean "$1"`
stat_queue_mean_raw=`c "$number_of_syncs / $seconds_between_files * $stat_transfer_time_mean"`
LC_ALL=C printf "%2.2f\n" $stat_queue_mean_raw
}
first_log()
{
list_all_logs "$1"_ | grep /"$1" | head -1
}
last_log()
{
list_all_logs "$1"_ | grep /"$1" | tail -1
}
start_date()
{
first_log=`first_log "$1"`
date -r "$first_log"
}
end_date()
{
last_log=`last_log "$1"`
date -r "$last_log"
}
echoq dirs_of_syncs_finished_recently
dirs_of_syncs_finished_recently() {
find . -maxdepth 1 -mtime "${1:--1}" | grep -v "385d7a4d8d428d7aa2b57c8982629e2bd67698ed" | egrep [a-f0-9]{40} | while read d; do
@ -492,6 +561,7 @@ logfiles_finished_recently()
}
}
last_dirs_written()
{
ls -tr | tail -1800
@ -675,12 +745,38 @@ first_use() {
echo "${1:-2017} ${2:-01} ${3:-09}"
}
filedate()
{
test FreeBSD = `uname -s` && gdate -r "$1" '+%Y %m %d'
test Linux = `uname -s` && date -r "$1" '+%Y %m %d'
}
days_between_files()
{
epoch1=`epoch_of_file "$1"`
epoch2=`epoch_of_file "$2"`
echo epoch1 $epoch1 epoch2 $epoch2
expr \( $epoch2 - $epoch1 \) / 3600 / 24
}
seconds_between_files()
{
epoch1=`epoch_of_file "$1"`
epoch2=`epoch_of_file "$2"`
expr \( $epoch2 - $epoch1 \)
}
days_since_first_use() {
first_use=`first_use "$@"`
#echo $[$[$(date +%s)-$(epoch_of_y_m_d_h_m_s 2017 01 09 00 00 00)]/60/60/24]
echo $[$[$(date +%s)-$(epoch_of_y_m_d_h_m_s $first_use 00 00 00)]/60/60/24]
}
epoch_of_y_m_d_h_m_s() {
date -v -1d > /dev/null 2>&1 && date -u -v ${1:-1970}y -v ${2:-1}m -v ${3:-1}d -v ${4:-0}H -v ${5:-0}M -v ${6:-0}S +%s && return
date --date="1 day ago" > /dev/null && date -u -d "${1:-1970}-${2:-1}-${3:-1} ${4:-0}:${5:-0}:${6:-0}" +%s && return
@ -756,7 +852,7 @@ vnstat_init() {
echoq vnstat_gen
vnstat_gen() {
vnstat_init || return
for opt in s h hg hs d m y t vs 5 ; do
for opt in s h hg hs d m y t vs 5 5g ; do
test "$1" && echo vnstati -$opt -o $VNSTATI_DIR/vnstat_${opt}.png
vnstati -$opt -o $VNSTATI_DIR/vnstat_${opt}.png
done
@ -833,55 +929,72 @@ referrer_of_x() {
zegrep -h -s -o 'GET /X/? .*http[^"]+' "${@:-/var/log/apache/access.log}" | grep -o 'http.*'
}
biggest_message_seen() {
cat G_Biggest_message.txt | grep -v Memory| datamash -W max 4 | xargs bytestohuman
statsfile=`statsfile Biggest_message $1`
cat "$statsfile" | grep -v Memory | datamash -W max 4 | xargs bytestohuman
}
biggest_message_transferred() {
# With this, the "Biggest message" could not be transferred by imapsync itself.
grep 'Host2 Biggest message' < G_Biggest_message.txt | datamash -W max 4 | xargs bytestohuman
# With this, the "Biggest message" may be not be transferred by imapsync itself.
statsfile=`statsfile Biggest_message $1`
grep 'Host2 Biggest message' < "$statsfile" | datamash -W max 4 | xargs bytestohuman
}
biggest_bandwidth_rate() {
datamash_file_op_index G_Average_bandwidth_rate.txt max 5 | tr -d ' ' | tr '\n' ' '
statsfile=`statsfile Average_bandwidth_rate $1`
datamash_file_op_index "$statsfile" max 5 | tr -d ' ' | tr '\n' ' '
echo KiB/s
}
average_bandwidth_rate() {
datamash_file_op_index G_Average_bandwidth_rate.txt mean 5 | tr -d ' ' | tr '\n' ' '
statsfile=`statsfile Average_bandwidth_rate $1`
datamash_file_op_index "$statsfile" mean 5 | tr -d ' ' | tr '\n' ' '
echo KiB/s
}
max_number_of_messages_transferred() {
datamash_file_op_index G_Messages_transferred.txt max 4 "%.0f"
statsfile=`statsfile Messages_transferred $1`
datamash_file_op_index "$statsfile" max 4 "%.0f"
}
max_number_of_messages_skipped() {
datamash_file_op_index G_Messages_skipped.txt max 4 "%.0f"
statsfile=`statsfile Messages_skipped $1`
datamash_file_op_index "$statsfile" max 4 "%.0f"
}
echoq number_of_X_users
number_of_X_users() {
datamash_file_op_index G_REMOTE_ADDR.txt unique 3 | tr , '\n' | wc -l
number_of_X_users()
{
statsfile=`statsfile REMOTE_ADDR $1`
test -f $statsfile || { echo No exists $statsfile ; return ; }
datamash_file_op_index $statsfile unique 3 | tr , '\n' | wc -l
}
echoq summary_compute
summary_compute() {
summary_compute_old() {
list_all_logs_generate \
&& grep_stats_from_list_all_logs \
&& grep_all_stat_from_patterns_list \
&& summary_display
}
echoq 'summary_compute lognamepattern # time depending on lognamepattern'
summary_compute() {
grep_stats_from_list_log_matching "$1" \
&& grep_all_stat_from_patterns_list "$1" \
&& summary_display "$1"
}
number_of_syncs()
{
list_all_logs | wc -l
list_all_logs | egrep "$1"_ | wc -l
}
@ -1002,25 +1115,25 @@ server_survey_host2()
}
echoq summary_display
echoq summary_display lognamepattern
summary_display() {
vnstat_gen > /dev/null
echo "Start date of /X (aaaa mm dd): `first_use` (`days_since_first_use` days of service)"
echo -n "Number of /X users: " ; number_of_X_users
echo -n "Number of /X accounts synced: " ; nb_migrations_launched
echo -n "Number of /X syncs: " ; number_of_syncs
echo -n "Total volume /X transferred: " ; total_volume_transferred
echo -n "Total messages /X transferred: " ; total_messages_transferred
echo -n "Biggest transfer: " ; biggest_transfer
echo -n "Biggest message seen: " ; biggest_message_seen
echo -n "Biggest message transferred: " ; biggest_message_transferred
echo -n "Biggest bandwidth rate: " ; biggest_bandwidth_rate
echo -n "Average bandwidth rate: " ; average_bandwidth_rate
echo -n "Max messages transferred: " ; max_number_of_messages_transferred
echo -n "Max messages skipped: " ; max_number_of_messages_skipped
echo -n "Longest transfer: " ; seconds_to_days_hours `longest_transfer`
echo -n "Queue length mean is: " ; stat_queue_mean
echo "Data made at" `date -r grep_stats.txt`
echo "Start date : " `start_date "$1"`
echo "End date : " `end_date "$1"`
echo -n "Number of /X users: " ; number_of_X_users "$1"
echo -n "Number of /X accounts synced: " ; nb_migrations_launched "$1"
echo -n "Number of /X syncs: " ; number_of_syncs "$1"
echo -n "Total volume /X transferred: " ; total_volume_transferred "$1"
echo -n "Total messages /X transferred: " ; total_messages_transferred "$1"
echo -n "Biggest transfer: " ; biggest_transfer "$1"
echo -n "Biggest message seen: " ; biggest_message_seen "$1"
echo -n "Biggest message transferred: " ; biggest_message_transferred "$1"
echo -n "Biggest bandwidth rate: " ; biggest_bandwidth_rate "$1"
echo -n "Average bandwidth rate: " ; average_bandwidth_rate "$1"
echo -n "Max messages transferred: " ; max_number_of_messages_transferred "$1"
echo -n "Max messages skipped: " ; max_number_of_messages_skipped "$1"
echo -n "Longest transfer: " ; seconds_to_days_hours `longest_transfer "$1"`
echo -n "Queue length mean is: " ; stat_queue_mean "$1"
echo "Data made at" `date -r grep_stats_"$1".txt`
}
echoq sync_ks2_i005
@ -1173,6 +1286,9 @@ while :; do ssh root@ks5 'cd /var/tmp/imapsync_cgi/ ; . cgi_memo ; loop_number_o
cat G_Memory_consumption_at_the_end.txt | sort -g -k7 | grep 202[01] |tail -100 | cut -f1 -d: | while read f; do echo $f ; grep 'Memory consumption at the end' $f; grep 'Host. Nb messages' $f ; grep 'Biggest message' $f ; grep 'Memory/biggest message ratio' $f ; done
cat G_Host2_Nb_messages.txt | sort -g -k4 | grep 202[01] |tail -100 | cut -f1 -d: | while read f; do echo $f ; grep 'Memory consumption at the end' $f; grep 'Host. Nb messages' $f ; grep 'Biggest message' $f ; grep 'Memory/biggest message ratio' $f ; done
cat G_Host1_Nb_messages.txt | sort -g -k4 | grep 202[01] |tail -100 | cut -f1 -d: | while read f; do echo $f ; grep 'Memory consumption at the end' $f; grep 'Host. Nb messages' $f ; grep 'Biggest message' $f ; grep 'Memory/biggest message ratio' $f ; done
# Best bandwidth moments
cat /var/tmp/number_of_imapsync_running_every_60s.txt | sort -k3 -g| tail -66
EOF
}
@ -1181,13 +1297,13 @@ echoq perf_help
perf_help() {
test FreeBSD = `uname -s` && {
echo FreeBSD here
echo "nload -t 6000 em0 -u K"
echo "nload -t 6000 em0 -u K -i 100000 -o 100000"
echo "iftop -i em0 -f 'port imap or port imaps' -B # t p >"
}
test Linux = `uname -s` && {
echo Linux here
echo "nload -t 6000 eth0 -u K # Linux"
echo "nload -t 6000 eth0 -u K -i 100000 -o 100000 # Linux"
echo "iftop -i eth0 -f 'port imap or port imaps' -B # t p >"
}
}

226
X/imapsync_csv_wrapper
View file

@ -1,11 +1,14 @@
#!/bin/sh
# $Id: imapsync_csv_wrapper,v 1.5 2020/05/01 21:26:30 gilles Exp gilles $
# $Id: imapsync_csv_wrapper,v 1.13 2021/11/01 13:24:47 gilles Exp gilles $
exec 3>&1
debug=/bin/true
debug=/bin/false
debug=true
# comment the following to have debug data
debug=false
PATH=$PATH:/home/www/apache24/cgi-bin/
echo3() {
echo "$@" >&3
@ -83,8 +86,9 @@ tests_uri_unescape()
run_test test '"_a\n\n\nz_" = "_`uri_unescape a%0A%0A%0Az`_"'
run_test test '"_`uri_unescape a%0A%0A%0Az`_" = "_`uri_unescape a%0A%0A%0Az`_"'
run_test test 'A B' = 'A B'
uri_unescape 'a%0A%0A%0Az' > /tmp/$$_zzz1.txt
/bin/echo -e "a\n\n\nz" > /tmp/$$_zzz2.txt
uri_unescape 'a%0Az' > /tmp/$$_zzz1.txt
/bin/echo "a
z" > /tmp/$$_zzz2.txt
run_test diff /tmp/$$_zzz1.txt /tmp/$$_zzz2.txt
rm /tmp/$$_zzz1.txt /tmp/$$_zzz2.txt
z3=`uri_unescape a%0A%0A%0Az`
@ -208,17 +212,33 @@ tests_is_blank_or_comment()
logfailures()
{
:
run_on_each_line_failures="$run_on_each_line_failures
$@"
}
tests_logfailures()
{
:
#reset
run_on_each_line_failures=
# add Blabla
run_test logfailures Blabla
run_test test "
Blabla" = "$run_on_each_line_failures"
# add Kiki
run_test logfailures Kiki
run_test test "
Blabla
Kiki" = "$run_on_each_line_failures"
#reset again
run_on_each_line_failures=
}
run_on_each_line()
run_on_each_line()
{
echodebug '### Entering run_on_each_line ###'
test -f abort_$$.txt && { echo abort demanded so not doing normal stuff ; return ; }
@ -228,12 +248,6 @@ run_on_each_line()
csv_data_multilines="$1"
echodebug "csv_data_multilines=[$csv_data_multilines]"
test 1 -le "$#" && shift
failures_file=${1:-""}
echodebug failures_file="$failures_file"
test -x "$failures_file" && { echo3 "failure file $failures_file is an executable. Leaving run_on_each_line()." ; return ; }
test 1 -le "$#" && shift
command=${1:-echo}
echodebug command="$command" "$@"
@ -244,43 +258,49 @@ run_on_each_line()
line_number=0
csv_number=0
failures_number=0
echo "$csv_data_multilines" | while IFS=';' read h1 u1 p1 h2 u2 p2 fake; do
while IFS=';' read h1 u1 p1 h2 u2 p2 extra fake; do
test -f abort_$$.txt && { echo abort demanded so not doing normal stuff ; return ; }
line_number=`expr 1 + $line_number`
if is_blank_or_comment "$h1$u1$p1$h2$u2$p2$fake" ; then
echo3 "Ignoring line " $line_number "$h1$u1$p1$h2$u2$p2$fake"
if is_blank_or_comment "$h1$u1$p1$h2$u2$p2$extra" ; then
echo3 "Ignoring line " $line_number "$h1$u1$p1$h2$u2$p2$extra"
else
csv_number=`expr 1 + $csv_number`
echo3 "Processing line " $line_number run $csv_number "[$h1] [$u1] [xxx] [$h2] [$u2] [xxx] [$fake] $$"
$command "$@" --host1 "$h1" --user1 "$u1" --password1 "$p1" --host2 "$h2" --user2 "$u2" --password2 "$p2"
echo3 "Processing line " $line_number csv run $csv_number "[$h1] [$u1] [xxx] [$h2] [$u2] [xxx] [$extra] $$"
$command "$@" --host1 "$h1" --user1 "$u1" --password1 "$p1" --host2 "$h2" --user2 "$u2" --password2 "$p2" $extra
command_status=$?
test "0" != "$command_status" && failures_number=`expr 1 + $failures_number`
logfailures "$failures_file" "$command_status" "$line_number" "$csv_number"
echodebug command_status=$command_status failures_number=$failures_number
test "0" != "$command_status" && logfailures "line $line_number csv run $csv_number returned with status $command_status ($h1;$u1;$p1;$h2;$u2;$p2;$extra;)"
fi
done
echodebug Leaving run_on_each_line
done << EOF
$csv_data_multilines
EOF
echodebug Leaving run_on_each_line, failures_number=$failures_number
test "0" != "$failures_number" && echo "$run_on_each_line_failures"
test "0" != "$failures_number" && return 1
return 0
}
tests_run_on_each_line()
tests_run_on_each_line()
{
# 1-4
run_test test '' = "`run_on_each_line`"
run_test test '' = "`run_on_each_line ''`"
run_test test '' = "`run_on_each_line '' 'log.txt'`"
run_test test '' = "`run_on_each_line '' 'log.txt' echo imapsync`"
run_test test '' = "`run_on_each_line '' echo imapsync`"
run_test test '' = "`run_on_each_line '' echo imapsync blabla blibli`"
# 5-7 blank data
run_test test '' = "`run_on_each_line ' '`"
run_test test '' = "`run_on_each_line ' ' 'log.txt'`"
run_test test '' = "`run_on_each_line ' ' 'log.txt' echo imapsync`"
run_test test '' = "`run_on_each_line ' ' `"
run_test test '' = "`run_on_each_line ' ' echo imapsync blabla blibli`"
# 8-10 comment data
run_test test '' = "`run_on_each_line '# '`"
run_test test '' = "`run_on_each_line '# ' 'log.txt'`"
run_test test '' = "`run_on_each_line '# ' 'log.txt' echo imapsync`"
run_test test '' = "`run_on_each_line ' # '`"
run_test test '' = "`run_on_each_line '# ' echo imapsync blabla blibli`"
# 11-13 empty line
myval='
@ -289,10 +309,10 @@ tests_run_on_each_line()
myret=`run_on_each_line "$myval"`
run_test test "" = "$myret"
myret=`run_on_each_line "$myval" 'log.txt'`
myret=`run_on_each_line "$myval"`
run_test test "" = "$myret"
myret=`run_on_each_line "$myval" 'log.txt' echo imapsync`
myret=`run_on_each_line "$myval" echo imapsync blabla blibli`
run_test test "" = "$myret"
@ -308,39 +328,33 @@ tests_run_on_each_line()
myret=`run_on_each_line "$myval"`
run_test test "" = "$myret"
myret=`run_on_each_line "$myval" 'log.txt'`
myret=`run_on_each_line "$myval" `
run_test test "" = "$myret"
myret=`run_on_each_line "$myval" 'log.txt' echo imapsync`
myret=`run_on_each_line "$myval" echo imapsync blabla blibli`
run_test test "" = "$myret"
# 17-21
# 17-18
# csv data no blank no comment
lines='a1;b1;c1;d1;e1;f1
a2;b2;c2;d2;e2;f2
a3;b3;c3;d3;e3;f3'
# No failure file (so /dev/null), no command (so echo)
# no command (so echo)
lines_out='--host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1
--host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2
--host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3'
run_test test "$lines_out" = "`run_on_each_line "$lines"`"
# failure file is a command => BAD => do nothing
cp /bin/echo "/tmp/myecho$$"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$`"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$ echo`"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$ echo imapsync blabla bla`"
rm "/tmp/myecho$$"
# No failure file (so /dev/null), command = echo imapsync
# command = echo imapsync
lines_out='imapsync --host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1
imapsync --host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2
imapsync --host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3'
run_test test "$lines_out" = "`run_on_each_line "$lines" '' echo imapsync`"
run_test test "$lines_out" = "`run_on_each_line "$lines" echo imapsync`"
# 22-27
# 19-20
# csv data + blanks + comments
lines='a1;b1;c1;d1;e1;f1
# blabla 2
@ -349,34 +363,97 @@ a2;b2;c2;d2;e2;f2
a3;b3;c3;d3;e3;f3
# end'
# No failure file (so /dev/null), no command (so echo)
# no command (so echo)
lines_out='--host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1
--host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2
--host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3'
run_test test "$lines_out" = "`run_on_each_line "$lines"`"
# failure file is a command => BAD => do nothing
cp /bin/echo "/tmp/myecho$$"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$`"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$ echo`"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$ echo imapsync blabla bla`"
rm "/tmp/myecho$$"
# No failure file (so /dev/null), command = echo imapsync
# command = echo imapsync
lines_out='imapsync --host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1
imapsync --host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2
imapsync --host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3'
run_test test "$lines_out" = "`run_on_each_line "$lines" '' echo imapsync`"
run_test test "$lines_out" = "`run_on_each_line "$lines" echo imapsync`"
# 21-22
# csv data 3/3;/6/6;/7/7;/8/8; parameters + blanks + comments
# runs 1 2 3 4 5 6 7 8
lines='a1;b1;c1
# blabla 2
a2;b2;c2;
# bloblo 3
a3;b3;c3;d3;e3;f3
a4;b4;c4;d4;e4;f4;
a5;b5;c5;d5;e5;f5;--extra blurp
a6;b6;c6;d6;e6;f6;--extra blurp;
a7;b7;c7;d7;e7;f7;--extra blurp; ignored
a8;b8;c8;d8;e8;f8;--extra blurp; ignored;
# end'
# no command (so echo)
lines_out='--host1 a1 --user1 b1 --password1 c1 --host2 --user2 --password2
--host1 a2 --user1 b2 --password1 c2 --host2 --user2 --password2
--host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3
--host1 a4 --user1 b4 --password1 c4 --host2 d4 --user2 e4 --password2 f4
--host1 a5 --user1 b5 --password1 c5 --host2 d5 --user2 e5 --password2 f5 --extra blurp
--host1 a6 --user1 b6 --password1 c6 --host2 d6 --user2 e6 --password2 f6 --extra blurp
--host1 a7 --user1 b7 --password1 c7 --host2 d7 --user2 e7 --password2 f7 --extra blurp
--host1 a8 --user1 b8 --password1 c8 --host2 d8 --user2 e8 --password2 f8 --extra blurp'
run_test test "$lines_out" = "`run_on_each_line "$lines"`"
# command = echo imapsync
lines_out='imapsync --host1 a1 --user1 b1 --password1 c1 --host2 --user2 --password2
imapsync --host1 a2 --user1 b2 --password1 c2 --host2 --user2 --password2
imapsync --host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3
imapsync --host1 a4 --user1 b4 --password1 c4 --host2 d4 --user2 e4 --password2 f4
imapsync --host1 a5 --user1 b5 --password1 c5 --host2 d5 --user2 e5 --password2 f5 --extra blurp
imapsync --host1 a6 --user1 b6 --password1 c6 --host2 d6 --user2 e6 --password2 f6 --extra blurp
imapsync --host1 a7 --user1 b7 --password1 c7 --host2 d7 --user2 e7 --password2 f7 --extra blurp
imapsync --host1 a8 --user1 b8 --password1 c8 --host2 d8 --user2 e8 --password2 f8 --extra blurp'
run_test test "$lines_out" = "`run_on_each_line "$lines" echo imapsync`"
# 23-2x
# "" input
lines='a1;b1;c1;d1;e1;f1;--caca "KK"'
# no command (so echo)
lines_out='--host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1 --caca "KK"'
run_test test "$lines_out" = "`run_on_each_line "$lines"`"
# command = echo imapsync
lines_out='imapsync --host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1 --caca "KK"'
run_test test "$lines_out" = "`run_on_each_line "$lines" echo imapsync`"
}
remove_double_quotes()
{
echo "$1" | tr -d '"'
}
tests_remove_double_quotes()
{
echo 'Entering tests_remove_double_quotes()'
run_test test -z `remove_double_quotes`
run_test test 'abc' = `remove_double_quotes 'abc'`
run_test test -z `remove_double_quotes '"'`
run_test test -z `remove_double_quotes '""'`
run_test test 'abc' = `remove_double_quotes '"ab"c'`
echo 'Leaving tests_remove_double_quotes()'
}
hashsync()
{
#set -x
mystring=${1:-''}
mykey=${2:-''}
mystring=`remove_double_quotes ${1:-''}`
mykey=${2:-''}
# impressive! is not it? quoting shit!
perl -MDigest::HMAC_SHA1 -e 'print Digest::HMAC_SHA1::hmac_sha1_hex( "'"$mystring"'", "'"$mykey"'" )'
#set +x
@ -385,11 +462,16 @@ hashsync()
tests_hashsync()
{
echo 'Entering tests_hashsync()'
run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync`
run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync ''`
run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync '' ''`
run_test test 'e86a28a3611c1e7bbaf8057cd00ae122781a11fe' = `hashsync 'zzz' ''`
run_test test '6a7b451ac99eab1531ad8e6cd544b32420c552ac' = `hashsync 'zzz' 'A'`
run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync '""'`
run_test test 'e86a28a3611c1e7bbaf8057cd00ae122781a11fe' = `hashsync '"zzz"'`
echo 'Leaving tests_hashsync()'
}
tests_caca()
@ -480,7 +562,9 @@ tests()
tests_run_on_each_line \
tests_hashsync \
tests_is_blank_or_comment \
tests_logfailures
tests_logfailures \
tests_remove_double_quotes \
}
@ -595,7 +679,7 @@ justlogin()
date
test -f abort_$$.txt && { echo Abort demanded so not doing normal stuff ; return ; }
echo Now run imapsync --justlogin to check all credentials are ok
if run_on_each_line "$csv_data_unescaped" "" \
if run_on_each_line "$csv_data_unescaped" \
imapsync --no-modulesversion --justlogin --tmpdir .
then
echo success of the just login check round
@ -612,8 +696,8 @@ justfoldersizes()
######################## just folders sizes counting round #####################'
date
test -f abort_$$.txt && { echo Abort demanded so not doing normal stuff ; return ; }
echo Now run imapsync --justlogin to check all credentials are ok
run_on_each_line "$csv_data_unescaped" "" \
echo Now run imapsync --justfoldersizes to show the volume to be synced
run_on_each_line "$csv_data_unescaped" \
imapsync --no-modulesversion --justfoldersizes --tmpdir . \
|| return 1
}
@ -625,8 +709,8 @@ sync_all()
######################## now sync them all round ###############################'
date
test -f abort_$$.txt && { echo Abort demanded so not doing normal stuff ; return ; }
echo Now run imapsync --justlogin to check all credentials are ok
run_on_each_line "$csv_data_unescaped" "" \
echo Now run imapsync on the data given
run_on_each_line "$csv_data_unescaped" \
imapsync --no-modulesversion --tmpdir . \
|| return 1
}
@ -647,11 +731,13 @@ main()
# All stderr to stdin, yeah I am like that
exec 2>&1
# Some tests
# Some tests, uncomment which one you work with and also the return one
#run_tests tests_is_blank_or_comment
#run_tests tests_run_on_each_line
#run_tests tests_remove_blanks_and_comments
#tests_hashsync
#run_tests tests_logfailures
#tests_remove_double_quotes
#return
tests_all_verbose_if_failure || return 1
@ -662,7 +748,9 @@ main()
getcsvdata
hashsync=`hashsync $csv_data_pure`
hashsync=`hashsync "$csv_data_pure"`
test -z $hashsync && return 1
change_working_directory_if_under_cgi /var/tmp/imapsync_cgi/$hashsync/
@ -678,9 +766,8 @@ main()
unset SERVER_SOFTWARE
# Now the real stuff
justlogin || return 1
justfoldersizes
#justlogin || return 1
#justfoldersizes
sync_all || return 1
@ -694,6 +781,3 @@ main()
tests_initialisation
main
return 0

6
X/imapsync_form.css
View file

@ -1,6 +1,6 @@
/* $Id: imapsync_form.css,v 1.9 2020/06/15 11:55:50 gilles Exp gilles $ */
/* $Id: imapsync_form.css,v 1.10 2021/09/21 09:25:53 gilles Exp gilles $ */
body {
color: black;
@ -29,6 +29,10 @@ body {
height: 50px ;
}
#imapsync_log_bottom {
scroll-margin-top: 300px ;
}
.progress {
/* text-align: center ; */
/* font-size: 1.4em ; */

62
X/imapsync_form.js
View file

@ -1,5 +1,5 @@
// $Id: imapsync_form.js,v 1.20 2021/05/29 10:25:53 gilles Exp gilles $
// $Id: imapsync_form.js,v 1.26 2021/12/18 19:01:52 gilles Exp gilles $
/*jslint browser: true*/ /*global $*/
@ -55,7 +55,30 @@ $(document).ready(
var note = function note( message )
{
$("#tests").append( message ) ;
}
} ;
var tests_last_x_lines = function tests_last_x_lines()
{
is( "", last_x_lines(), "last_x_lines: no args => empty string" ) ;
is( "", last_x_lines(""), "last_x_lines: empty string => empty string" ) ;
is( "abc", last_x_lines("abc"), "last_x_lines: abc => abc" ) ;
is( "abc\ndef", last_x_lines("abc\ndef"), "last_x_lines: abc\ndef => abc\ndef" ) ;
is( "def", last_x_lines("abc\ndef", -1), "last_x_lines: abc\ndef -1 => def\n" ) ;
is( "", last_x_lines("abc\ndef", 0), "last_x_lines: abc\ndef 0 => empty string" ) ;
is( "abc\ndef", last_x_lines("abc\ndef", -10), "last_x_lines: last 10 of 2 lines => 2 lines" ) ;
is( "4\n5\n", last_x_lines("1\n2\n3\n4\n5\n", -3), "last_x_lines: last 3 lines of 5 lines" ) ;
is( "3\n4\n5", last_x_lines("1\n2\n3\n4\n5", -3), "last_x_lines: last 3 lines of 5 lines" ) ;
} ;
var last_x_lines = function last_x_lines( string, num )
{
if ( undefined === string || 0 === num )
{
return "" ;
}
return string.split(/\r?\n/).slice(num).join("\n") ;
} ;
var last_eta = function last_eta( string )
{
@ -82,6 +105,8 @@ $(document).ready(
return "ETA: unknown" ;
}
}
var tests_last_eta = function tests_last_eta()
{
@ -257,7 +282,7 @@ $(document).ready(
}
else
{
slice_length = -240 ;
slice_length = -2400 ;
}
slice_log = xhr.responseText.slice( slice_length ) ;
eta_str = last_eta( slice_log ) ;
@ -297,24 +322,24 @@ $(document).ready(
if ( xhr.readyState === 4 )
{
// end of sync
$("#progress-txt").text(
$("#progress-txt").text(
"Ended. It remains "
+ eta_obj.msgs_left + " messages to be synced" ) ;
$( "#output" ).text( xhr.responseText ) ;
}
else
{
eta_str = eta_obj.str + " (refresh every " + refresh_interval_s + " s)" ;
eta_str = eta_str.replace(/(\r\n|\n|\r)/gm, "") ; // trim newline
//$("#tests").append( "refreshLog eta_str: " + eta_str + "\n" ) ;
$("#progress-txt").text( eta_str ) ;
eta_str = eta_obj.str + " (refresh every " + refresh_interval_s + " s)" ;
eta_str = eta_str.replace(/(\r\n|\n|\r)/gm, "") ; // trim newlines
//$("#tests").append( "refreshLog eta_str: " + eta_str + "\n" ) ;
$( "#progress-txt" ).text( eta_str ) ;
var last_lines = last_x_lines( xhr.responseText.slice(-2000), -10)
$( "#output" ).text( last_lines ) ;
}
$( "#output" ).text( xhr.responseText ) ;
}
var handleRun = function handleRun(xhr, timerRefreshLog)
{
@ -326,7 +351,6 @@ $(document).ready(
// var headers = xhr.getAllResponseHeaders();
// $("#console").append(headers);
// $("#console").append("See the completed log\n");
$("#link_to_bottom").show() ;
clearInterval( timerRefreshLog ) ;
refreshLog( xhr ) ; // a last time
// back to enable state for next run
@ -650,7 +674,6 @@ $(document).ready(
// in case of a manual refresh, start with
$("#bt-sync").prop("disabled", false);
$("#bt-abort").prop("disabled", false);
$("#link_to_bottom").hide();
$("#progress-bar-left").css( "width", 100 + "%" ).attr( "aria-valuenow", 100 ) ;
$("#showpassword1").click(
@ -678,7 +701,6 @@ $(document).ready(
$( "#imapsync_current" ).load( "imapsync_current.txt" ) ;
$("#bt-sync").prop("disabled", true) ;
$("#bt-abort").prop("disabled", false) ;
$("#link_to_bottom").hide() ;
$("#progress-txt").text( "ETA: coming soon" ) ;
store_form() ;
imapsync() ;
@ -729,6 +751,9 @@ $(document).ready(
if ( "imapsync.lamiral.info" === location.hostname )
{
$( "#status_24h" ).attr('src', 'https://lstu.fr/imapsync_online_status_24h_1200x70') ;
$( "#status_7d" ).attr('src', 'https://lstu.fr/imapsync_online_status_7d') ;
$( "#status_2m" ).attr('src', 'https://lstu.fr/imapsync_online_status_2months_1200x70') ;
$( "#local_bandwidth" ).collapse( "show" ) ;
$( "#local_status_dbmon" ).collapse( "show" ) ;
$( "#local_status_hetrix" ).collapse( "show" ) ;
@ -737,6 +762,7 @@ $(document).ready(
}
else if ( "lamiral.info" === location.hostname )
{
$( "#local_bandwidth" ).collapse( "show" ) ;
$( "#local_status_dbmon" ).collapse( "show" ) ;
$( "#local_status_hetrix" ).collapse( "show" ) ;
@ -778,6 +804,7 @@ $(document).ready(
tests_store_retrieve( ) ;
tests_last_eta( ) ;
tests_decompose_eta_line( ) ;
tests_last_x_lines( ) ;
// tests_cryptojs( ) ;
// The following test can be used to check that if a test fails
@ -785,15 +812,16 @@ $(document).ready(
//is( 0, 1, "this test always fails" ) ;
tests_bilan( nb_attended_test ) ;
// If you want to always see the tests, uncomment the following
// line
// $("#tests").collapse("show") ;
//$("#tests").collapse("show") ;
}
}
init( ) ;
tests( 29 ) ;
tests( 38 ) ;
retrieve_form( ) ;
}

35
X/imapsync_form_extra.html
View file

@ -1,6 +1,6 @@
<!DOCTYPE html>
<!-- $Id: imapsync_form_extra.html,v 1.18 2021/05/30 05:45:15 gilles Exp gilles $ -->
<!-- $Id: imapsync_form_extra.html,v 1.22 2021/12/18 19:03:46 gilles Exp gilles $ -->
<html lang="en" id="top">
@ -273,7 +273,12 @@
<a id="buttons"></a>
<hr>
<!-- Classical button to go to the log only -->
<div class="text-center center-block">
If you <b>close</b> this <b>window</b> (or tab) <b>during</b> the synchronization,
it will <b>abort</b> the synchronization, it's like <b>hitting</b> the red button <b>"Abort!"</b> below.
</div>
<!-- Classical button to go to the log only, when javascript is turned off -->
<noscript>
<div class="row">
<div class="col-sm-12 padd0" >
@ -336,7 +341,6 @@
</pre>
<a id="link_to_bottom" href="#bottom">Bottom of imapsync log</a>
</div>
<div class="col-sm-6 well">
@ -351,7 +355,10 @@
</div>
<h2 class="text-center scripton">Log of imapsync run</h2>
<h2 id="imapsync_log_beginning" class="text-center scripton">Log of imapsync run</h2>
<div class="text-center scripton">
<a href="#imapsync_log_bottom">Link to the <b>bottom</b> of the imapsync log file</a>
</div>
<pre id="output" class="scripton">
@ -374,6 +381,10 @@
</pre>
<div id="imapsync_log_bottom" class="text-center scripton">
<a href="#imapsync_log_beginning">Link to the <b>beginning</b> of the imapsync log file</a>
</div>
<div id="local_bandwidth" class="collapse">
<hr>
<p class="text-center">
@ -390,17 +401,17 @@
<hr>
<p class="text-center">
Imapsync <b>Online Status</b> over the <b>last 24h</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_24h_1200x70">
<img id="status_24h" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_24h_1200x70.png">
</p>
<p class="text-center">
Imapsync <b>Online Status</b> over the <b>last week</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_7d">
<img id="status_7d" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_7d_1200x70.png">
</p>
<p class="text-center">
Imapsync <b>Online Status</b> over the <b>last two months</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_2months_1200x70"><br>
<img id="status_2m" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_2months_1200x70.png"><br>
</p>
@ -411,19 +422,19 @@ Uncomment the following when the data are available
<p class="text-center">
Imapsync <b>Online Status</b> over the <b>last year</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_1year_1200x70"><br>
<img id="status_1y" class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_1year_1200x70"><br>
</p>
<p class="text-center">
Imapsync <b>Online Status</b> over the <b>last 10 years</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_10years_1200x70-2"><br>
<img id="status_10y" class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_10years_1200x70-2"><br>
</p>
-->
</div>
<div id="local_status_hetrix" class="collapse in">
<div id="local_status_hetrix" class="collapse">
<hr>
<p class="text-center">
@ -452,7 +463,7 @@ monitor page powered by the <a href="https://hetrixtools.com/uptime-monitor/4143
<!-- <a href="#buttons" class="btn btn-info scripton" role="button">Consoles</a> -->
<a href="#bottom" title="Bottom of the page" class="btn btn-info active" role="button">Bottom</a>
<br>
<small> ($Id: imapsync_form_extra.html,v 1.18 2021/05/30 05:45:15 gilles Exp gilles $) </small><br>
<small> ($Id: imapsync_form_extra.html,v 1.22 2021/12/18 19:03:46 gilles Exp gilles $) </small><br>
Terms and conditions for anything: <a href="https://imapsync.lamiral.info/LICENSE">No limits to do anything with this work and this license!</a><br>
</div>
</div>
@ -482,7 +493,5 @@ src="imapsync_form.js"
>
</script>
<img src="https://static.scarf.sh/a.png?x-pxid=28b508c3-c51f-4109-9b88-29f3f6e4cf2f" alt="Scarf Tracker"/>
</body>
</html>

36
X/imapsync_form_extra_free.html
View file

@ -1,6 +1,6 @@
<!DOCTYPE html>
<!-- $Id: imapsync_form_extra_free.html,v 1.18 2021/05/30 05:45:00 gilles Exp gilles $ -->
<!-- $Id: imapsync_form_extra_free.html,v 1.23 2021/12/18 19:03:46 gilles Exp gilles $ -->
<html lang="en" id="top">
@ -332,7 +332,12 @@ It's <b>ok</b> if you <b>don't</b> have a VAT number.<br />
<a id="buttons"></a>
<hr>
<!-- Classical button to go to the log only -->
<div class="text-center center-block">
If you <b>close</b> this <b>window</b> (or tab) <b>during</b> the synchronization,
it will <b>abort</b> the synchronization, it's like <b>hitting</b> the red button <b>"Abort!"</b> below.
</div>
<!-- Classical button to go to the log only, when javascript is turned off -->
<noscript>
<div class="row">
<div class="col-sm-12 padd0" >
@ -395,7 +400,7 @@ It's <b>ok</b> if you <b>don't</b> have a VAT number.<br />
</pre>
<a id="link_to_bottom" href="#bottom">Bottom of imapsync log</a>
</div>
<div class="col-sm-6 well">
@ -410,8 +415,10 @@ It's <b>ok</b> if you <b>don't</b> have a VAT number.<br />
</div>
<h2 class="text-center scripton">Log of imapsync run</h2>
<h2 id="imapsync_log_beginning" class="text-center scripton">Log of imapsync run</h2>
<div class="text-center scripton">
<a href="#imapsync_log_bottom">Link to the <b>bottom</b> of the imapsync log file</a>
</div>
<pre id="output" class="scripton">
@ -433,6 +440,10 @@ It's <b>ok</b> if you <b>don't</b> have a VAT number.<br />
</pre>
<div id="imapsync_log_bottom" class="text-center scripton">
<a href="#imapsync_log_beginning">Link to the <b>beginning</b> of the imapsync log file</a>
</div>
<div id="local_bandwidth" class="collapse">
<hr>
<p class="text-center">
@ -449,17 +460,17 @@ It's <b>ok</b> if you <b>don't</b> have a VAT number.<br />
<hr>
<p class="text-center">
Imapsync <b>Online Status</b> over the <b>last 24h</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_24h_1200x70">
<img id="status_24h" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_24h_1200x70.png">
</p>
<p class="text-center">
Imapsync <b>Online Status</b> over the <b>last week</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_7d">
<img id="status_7d" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_7d_1200x70.png">
</p>
<p class="text-center">
Imapsync <b>Online Status</b> over the <b>last two months</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_2months_1200x70"><br>
<img id="status_2m" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_2months_1200x70.png"><br>
</p>
@ -470,19 +481,19 @@ Uncomment the following when the data are available
<p class="text-center">
Imapsync <b>Online Status</b> over the <b>last year</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_1year_1200x70"><br>
<img id="status_1y" class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_1year_1200x70"><br>
</p>
<p class="text-center">
Imapsync <b>Online Status</b> over the <b>last 10 years</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_10years_1200x70-2"><br>
<img id="status_10y" class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_10years_1200x70-2"><br>
</p>
-->
</div>
<div id="local_status_hetrix" class="collapse in">
<div id="local_status_hetrix" class="collapse">
<hr>
<p class="text-center">
@ -511,7 +522,7 @@ monitor page powered by the <a href="https://hetrixtools.com/uptime-monitor/4143
<!-- <a href="#buttons" class="btn btn-info scripton" role="button">Consoles</a> -->
<a href="#bottom" title="Bottom of the page" class="btn btn-info active" role="button">Bottom</a>
<br>
<small> ($Id: imapsync_form_extra_free.html,v 1.18 2021/05/30 05:45:00 gilles Exp gilles $) </small><br>
<small> ($Id: imapsync_form_extra_free.html,v 1.23 2021/12/18 19:03:46 gilles Exp gilles $) </small><br>
Terms and conditions for anything: <a href="https://imapsync.lamiral.info/LICENSE">No limits to do anything with this work and this license!</a><br>
</div>
</div>
@ -541,7 +552,6 @@ src="imapsync_form.js"
>
</script>
<img src="https://static.scarf.sh/a.png?x-pxid=28b508c3-c51f-4109-9b88-29f3f6e4cf2f" alt="Scarf Tracker"/>
</body>
</html>

BIN
X/imapsync_online_status_24h_1200x70.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 17 KiB

BIN
X/imapsync_online_status_2months_1200x70.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 19 KiB

BIN
X/imapsync_online_status_7d_1200x70.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 19 KiB

158
X/sandbox_csv.html
View file

@ -1,57 +1,77 @@
<!DOCTYPE html>
<html>
<html lang="en">
<head>
<title>Imapsync CSV online</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- -->
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
<link rel="stylesheet" href="../S/style.css" type="text/css" />
<link rel="license" href="https://imapsync.lamiral.info/NOLIMIT">
<!-- -->
</head>
<body>
<h1>Imapsync online</h1>
<h1>Imapsync CSV online</h1>
<p>
This file is a sandbox to play with, to learn, understand what happens, what can be done etc.<br/>
<a href="#BOTTOM">Bottom of this page</a>
This Imapsync CSV online service is a prototype, a working prototype.<br/>
</p>
Some explanations about this service:
<ul>
<li>The input textarea is multi-lines, each line describe a synchronization from the source account to the destination account.</li>
<li>Blank lines are ignored. Blanks lines have only spaces or tabs or nothing but a CR</li>
<li>Lines beginning with # are also ignored, use that for commenting.</li>
<li>A basic line will sync all messages in all folders from the source account to the destination account.</li>
<li>Each line has seven columns, six for the credentials, and one extra to add one or several parameters to the imapsync command.</li>
<li>Each line has seven semi-colons characters ";", each one delimiting a column. The last column can be empty.</li>
<li>Each line is processed serially, one synchronization at a time.</li>
<li>At the end of the log, the lines that failed are listed.</li>
<li>The input textarea is prefilled with values, as an example, replace them with yours or remove them.</li>
<li></li>
<li></li>
<li></li>
</ul>
<form id="form" action="/cgi-bin/imapsync_csv_wrapper" method="post" autocomplete="on">
<textarea name="csv_data" rows="10" cols="120">
# This example is a real one, ie, truly working in the real world.
test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;secret2;
<textarea id="csv_data" name="csv_data" rows="10" cols="120">
# This example is a real one, ie, a truly working in the real world.
test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;secret2;;
# The reverse
test2.lamiral.info;test2;secret2;test1.lamiral.info;test1;secret1;
# The first again but this time with an authentication failure
test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;wrong_secret;;
# The first again but with authentication failure
test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;wrong;
# The first again but this time with extra parameters
test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;secret2;--justfoldersizes --dry --minage 55 ;
</textarea>
<br/>
</form>
<button type="button"
onclick="imapsync( handleRun )"
>Run imapsync on all csv data
<p>
<button id="bt-sync" type="button">
Run imapsync on all csv data!
</button>
<button type="button"
onclick="abort( )"
>Abort imapsync
<button id="bt-abort" type="button">
Abort the run!
</button>
</p>
<p>
<a href="#BOTTOM">Link to the bottom of this page (end of the log)</a>
</p>
<h2>Console of imapsync runs</h2>
@ -63,13 +83,11 @@ test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;wrong;
<pre id="abort">
</pre>
<h2>Log of imapsync runs</h2>
<h2>Log of all imapsync runs</h2>
<pre id="output" >
</pre>
<h2>Links</h2>
<div id="BOTTOM">
@ -78,89 +96,11 @@ test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;wrong;
</p>
</div>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
<script>
var readyStateStr = {
0: "Request not initialized",
1: "Server connection established",
2: "Response headers received",
3: "Processing request",
4: "Finished and response is ready"
} ;
function imapsync( cFunction ) {
var xhr ;
xhr = new XMLHttpRequest( ) ;
var timerRefreshLog = setInterval( function() { refreshLog( xhr ) }, 3000 ) ;
xhr.onreadystatechange = function( ) {
cFunction( this, timerRefreshLog ) ;
} ;
var form_querystring = $("#form").serialize() ;
$("#form_querystring").text( form_querystring ) ;
xhr.open( "POST", "/cgi-bin/imapsync_csv_wrapper", true ) ;
xhr.setRequestHeader( "Content-type",
"application/x-www-form-urlencoded" ) ;
xhr.send( form_querystring ) ;
$("#output").text("Here comes the log!\n\n") ;
xhr.send( ) ;
}
function handleRun( xhr, timerRefreshLog ) {
$( "#console" ).text( "Status: " + xhr.status + " " + xhr.statusText + ".\n"
+ "State: " + readyStateStr[ xhr.readyState ] + "\n" ) ;
if ( xhr.status == 200 && xhr.readyState == 4 ) {
var headers = xhr.getAllResponseHeaders( ) ;
clearInterval( timerRefreshLog ) ;
refreshLog( xhr ) ; // a last time
}
}
function refreshLog( xhr ) {
$( "#output" ).text( xhr.responseText ) ;
}
function abort()
{
var querystring = $("#form").serialize() + "&abort=on";
var xhr;
xhr = new XMLHttpRequest();
xhr.onreadystatechange = function ()
{
handleAbort(xhr);
};
xhr.open("POST", "/cgi-bin/imapsync_csv_wrapper", true);
xhr.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
xhr.send(querystring);
}
function handleAbort( xhr ) {
$( "#abort" ).text( "Status: " + xhr.status + " " + xhr.statusText + ".\n"
+ "State: " + readyStateStr[ xhr.readyState ] + "\n" ) ;
if ( xhr.status == 200 && xhr.readyState == 4 ) {
var headers = xhr.getAllResponseHeaders( ) ;
// $( "#abort" ).append( "\n" + headers + "\n" ) ;
$( "#abort" ).append( xhr.responseText ) ;
}
}
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js">
</script>
<script src="sandbox_csv.js">
</script>
</body>
</html>

112
X/sandbox_csv.js Executable file
View file

@ -0,0 +1,112 @@
$(document).ready(
function ()
{
var readyStateStr = {
0: "Request not initialized",
1: "Server connection established",
2: "Response headers received",
3: "Processing request",
4: "Finished and response is ready"
} ;
function imapsync( cFunction ) {
var xhr ;
xhr = new XMLHttpRequest( ) ;
var timerRefreshLog = setInterval( function() { refreshLog( xhr ) }, 6000 ) ;
xhr.onreadystatechange = function( ) {
cFunction( this, timerRefreshLog ) ;
} ;
var form_querystring = $("#form").serialize() ;
$("#form_querystring").text( form_querystring ) ;
xhr.open( "POST", "/cgi-bin/imapsync_csv_wrapper", true ) ;
xhr.setRequestHeader( "Content-type",
"application/x-www-form-urlencoded" ) ;
xhr.send( form_querystring ) ;
$("#output").text("Here comes the log!\n\n") ;
}
function handleRun( xhr, timerRefreshLog ) {
$( "#console" ).text( "Status: " + xhr.status + " " + xhr.statusText + ".\n"
+ "State: " + readyStateStr[ xhr.readyState ] + "\n" ) ;
if ( xhr.status == 200 && xhr.readyState == 4 ) {
var headers = xhr.getAllResponseHeaders( ) ;
clearInterval( timerRefreshLog ) ;
refreshLog( xhr ) ; // a last time
$("#bt-sync").prop("disabled", false) ;
$("#csv_data").prop('readonly', false);
}
}
function refreshLog( xhr ) {
$( "#output" ).text( xhr.responseText ) ;
}
function abort()
{
var querystring = $("#form").serialize() + "&abort=on";
var xhr;
xhr = new XMLHttpRequest();
xhr.onreadystatechange = function ()
{
handleAbort(xhr);
};
xhr.open("POST", "/cgi-bin/imapsync_csv_wrapper", true);
xhr.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
xhr.send(querystring);
}
function handleAbort( xhr ) {
$( "#abort" ).text( "Status: " + xhr.status + " " + xhr.statusText + ".\n"
+ "State: " + readyStateStr[ xhr.readyState ] + "\n" ) ;
if ( xhr.status == 200 && xhr.readyState == 4 ) {
var headers = xhr.getAllResponseHeaders( ) ;
// $( "#abort" ).append( "\n" + headers + "\n" ) ;
$( "#abort" ).append( xhr.responseText ) ;
$("#csv_data").prop('readonly', false);
$( "#bt-sync" ).prop("disabled", false);
$( "#bt-abort" ).prop("disabled", false);
}
}
$("#bt-sync").click(
function ()
{
$("#bt-sync").prop("disabled", true) ;
$("#csv_data").prop('readonly', true);
$("#bt-abort").prop("disabled", false) ;
$("#abort").text("") ;
imapsync( handleRun ) ;
}
);
$("#bt-abort").click(
function ()
{
$("#bt-sync").prop("disabled", true);
$("#csv_data").prop('readonly', true);
$("#bt-abort").prop("disabled", true);
abort();
}
);
// in case of a manual refresh, start with
$("#csv_data").prop('readonly', false);
$("#bt-sync").prop("disabled", false);
$("#bt-abort").prop("disabled", false);
}
) ;

1
X/stat_patterns.txt
View file

@ -36,6 +36,7 @@
^Message rate
^Average bandwidth rate
^Biggest message transferred
^Biggest message:
^Detected.*errors
^Ended by a signal
^Exiting with return value

1
X/vnstati.html
View file

@ -17,6 +17,7 @@ small { font-size: 8px; color: #cbcbcb; }
<body bgcolor="#ffffff">
<img src="vnstat_s.png" border="0" alt="summary"><br>
<img src="vnstat_5g.png" border="0" alt="5 minutes graphic"><br>
<img src="vnstat_5.png" border="0" alt="5 minutes"><br>
<img src="vnstat_h.png" border="0" alt="hourly"><br>
<img src="vnstat_d.png" border="0" alt="daily"><br>

123
doc/TUTORIAL_Unix.html
View file

@ -7,7 +7,7 @@
<CENTER>
<H1>Imapsync tutorial</H1>
<FONT SIZE="4"><I>Gilles LAMIRAL gilles@lamiral.info</I></FONT><BR>
<FONT SIZE="4">$Id: TUTORIAL_Unix.t2t,v 1.26 2019/02/10 12:33:17 gilles Exp gilles $ </FONT>
<FONT SIZE="4">$Id: TUTORIAL_Unix.t2t,v 1.27 2022/01/14 12:42:51 gilles Exp gilles $ </FONT>
</CENTER>
<P></P>
@ -18,14 +18,14 @@
<LI><A HREF="#toc1">1. Good practices overview</A>
<LI><A HREF="#toc2">2. Basic steps</A>
<UL>
<LI><A HREF="#toc3">2.1. Verifying imapsync works well on its own</A>
<LI><A HREF="#toc3">2.1. Verifying imapsync works well</A>
<LI><A HREF="#toc4">2.2. Working with your data</A>
<LI><A HREF="#toc5">2.3. Prepare your credentials</A>
<LI><A HREF="#toc6">2.4. Take a real user account as source</A>
<LI><A HREF="#toc7">2.5. Take a test user account as destination</A>
<LI><A HREF="#toc6">2.4. Take a real user account as a source</A>
<LI><A HREF="#toc7">2.5. Take a test user account as a destination</A>
<LI><A HREF="#toc8">2.6. Edit your own script mysync</A>
</UL>
<LI><A HREF="#toc9">3. Background knowledge about emailboxes</A>
<LI><A HREF="#toc9">3. Background knowledge about mailboxes</A>
<LI><A HREF="#toc10">4. Imapsync presentation</A>
<LI><A HREF="#toc11">5. Conventions</A>
<LI><A HREF="#toc12">6. Why start with a test account on destination host2?</A>
@ -53,24 +53,24 @@ without much pain. So try imapsync with a real account at the source, aka host1,
and a test account at the destination, aka host2.
<P></P>
<LI>Next, once familiar and satisfied by the result on the host2 test account,
change to a real user account on host2 or just stop consider it a test one.
change to a real user account on host2 or just stop considering it a test one.
</UL>
<A NAME="toc2"></A>
<H1>2. Basic steps</H1>
<A NAME="toc3"></A>
<H2>2.1. Verifying imapsync works well on its own</H2>
<H2>2.1. Verifying imapsync works well</H2>
<P>
Open a terminal and go to the imapsync directory.
The imapsync directory is the directory created by extraction
of the tarball (.tgz), its name is <CODE>imapsync-1.xxx</CODE> where <CODE>1.xxx</CODE>
is imapsync release number (1.882 at the time of this writing).
The imapsync directory is the directory created by the extraction
of the tarball (.tgz), its name is <CODE>imapsync-2.xxx</CODE> where <CODE>2.xxx</CODE>
is imapsync release number (2.178 at the time of this proofread).
</P>
<PRE>
cd imapsync-1.xxx/
cd imapsync-2.xxx/
</PRE>
<P>
@ -86,14 +86,14 @@ It should output the help message.
</P>
<P>
If the previous command fails then there is an installation issue.
Go back to <A HREF="https://imapsync.lamiral.info/#install">https://imapsync.lamiral.info/#install</A> then
and read and apply the installation file corresponding to your
Go back to the <A HREF="https://imapsync.lamiral.info/#install">https://imapsync.lamiral.info/#install</A> section, then
read and apply the installation file corresponding to your
system and drop me an email about your issue.
</P>
<P>
Next, verify imapsync runs live tests. This check needs an Internet
Next, verify imapsync runs well with live tests. This check needs Internet
access. It does a simple sync between two real dedicated
imap maiboxes located at the host test.lamiral.info
imap mailboxes located at the host test.lamiral.info
</P>
<PRE>
@ -109,7 +109,7 @@ Now verify that the script examples/imapsync_example.sh runs fine:
</PRE>
<P>
This script does the same thing than "<CODE>imapsync --testslive</CODE>" but it
This script does the same thing as "<CODE>imapsync --testslive</CODE>" but it
uses explicitly the 6 parameters so it will be a good start
for your future own script.
</P>
@ -148,28 +148,28 @@ An IMAP account is accessed with 3 parameters,
</P>
<UL>
<LI>the imap server <B>host</B>. It's a server name or an ip address
<LI>the imap server <B>host</B>. It's a server name or an IP address
<LI>the <B>user</B> name
<LI>the <B>password</B>
</UL>
<P>
Since imapsync job is to sync two imap accounts we need 3 + 3 = 6 parameters:
Since imapsync job is to sync two imap accounts, we need 3 + 3 = 6 parameters:
</P>
<UL>
<LI>Three parameters in order to read data from the source account: <B>host1</B>, <B>user1</B> and <B>password1</B>
<LI>Three parameters in order to copy this data to the destination account: <B>host2</B>, <B>user2</B> and <B>password2</B>
<LI>Three parameters to read data from the source account: <B>host1</B>, <B>user1</B> and <B>password1</B>
<LI>Three parameters to copy this data to the destination account: <B>host2</B>, <B>user2</B> and <B>password2</B>
</UL>
<A NAME="toc6"></A>
<H2>2.4. Take a real user account as source</H2>
<H2>2.4. Take a real user account as a source</H2>
<P>
Even during the learning time with imapsync, you can take a
real user account as a source. There is also no problem if this account is
currently used by a user. By default, this account will only be read, no change will
be made by imapsync on it.
be made by imapsync on it. It's safe to use a normal and live account as a source, even to learn imapsync.
</P>
<P>
Assuming that the imap source server name host1 is <B>origin.example.com</B>,
@ -184,13 +184,13 @@ we now have the first three parameters.
</UL>
<A NAME="toc7"></A>
<H2>2.5. Take a test user account as destination</H2>
<H2>2.5. Take a test user account as a destination</H2>
<P>
Unlike the source side, the destination side will be modified by
imapsync. Therefore, for learning, checking and adjusting,
it is not a very good idea to use a real user imap account
the first times you play with imapsync.
imapsync. Therefore, for learning, checking, and adjusting,
it is not a good idea to use a real user imap account
the first time you play with imapsync.
</P>
<P>
If you really can't afford a test account on host2, it's ok,
@ -213,7 +213,7 @@ we now have the next three parameters.
<H2>2.6. Edit your own script mysync</H2>
<P>
Now edit the script <CODE>mysync</CODE> and replace the test values by yours.
Now edit the script <CODE>mysync</CODE> and replace the test values with yours.
</P>
<P>
You're ready for a dry test on your accounts.
@ -227,22 +227,21 @@ You're ready for a dry test on your accounts.
Since the <CODE>mysync</CODE> script is a copy of <CODE>examples/imapsync_example.sh</CODE>,
your first run with your data should include three other options
<CODE>--automap</CODE> <CODE>--justfolders</CODE> <CODE>--dry</CODE>.
With <CODE>--dry</CODE> option, nothing will really be done on host2
With <CODE>--dry</CODE> option, nothing will be done for real on host2
but yet it will test whether the credentials are ok on both sides.
You'll encounter two successful logins, or one failure on host1,
or a successful login on host1 but a failure on host2.
If the login are ok, you will also observe if the folders mapping is ok.
You'll encounter two successful logins, or one login failure on host1 or host2, or two failures on host1 and host2.
If the logins are ok, you will be able to observe that the folders mapping is ok.
</P>
<P>
If a login fails then double-check all three values that identify
If login fails then double-check all three values that identify
the account, which are the host, the login name, and the password.
</P>
<P>
If the folders mapping proposed is not ok then you can fix it with
option <CODE>--f1f2</CODE>. The following example maps the source folder
the option <CODE>--f1f2</CODE>. The following example maps the source folder
"Sent Messages" to the destination folder "Sent". The double-quotes
are not part of the folders names but they should be used when special
characters like blanks are in the folders names:
are not part of the names of the folders but they should be used when special
characters like blanks are in the names of the folders:
</P>
<PRE>
@ -250,7 +249,7 @@ characters like blanks are in the folders names:
</PRE>
<P>
As explained in the inline help or in the README:
As explained in the inline help or the README:
</P>
<PRE>
@ -267,7 +266,7 @@ folders. Remove <CODE>--dry</CODE> from the <CODE>mysync</CODE> script and rerun
</PRE>
<A NAME="toc9"></A>
<H1>3. Background knowledge about emailboxes</H1>
<H1>3. Background knowledge about mailboxes</H1>
<P>
Three Internet protocols are used to access almost all email accounts:
@ -289,7 +288,7 @@ they are unavailable from any other system located elsewhere.
<P>
The second protocol to deal with email messages is IMAP, Internet Message Access Protocol.
IMAP gives access to a hierarchy of mailboxes also called folders. Other IMAP features are
concurrent accesses, tagging with flags, search by many criterium like date, subject, size etc.
concurrent accesses, tagging with flags, search by many criteriums like date, subject, size, etc.
</P>
<P>
The IMAP protocol presents most of the features POP lacks.
@ -304,11 +303,11 @@ HTTP is the protocol to browse the web.
Web browsers like Google Chrome, Mozilla Firefox, Internet Explorer and Safari
are HTTP client software tools. You already know that so what's the point with HTTP mailboxes?
HTTP mailboxes are called webmails.
Webmails often offer the same features than imap servers
Webmails often offer the same features as imap servers
because webmails underlying storage systems are often imap servers.
</P>
<P>
Webmails systems like Gmail, Yahoo, Exchange, Zimbra or Office365 are also accessible via imap.
Webmails systems like Gmail, Yahoo, Exchange, Zimbra, or Office365 are also accessible via imap.
</P>
<P>
The conclusion of this protocol review is that mailboxes can be
@ -325,12 +324,12 @@ you can use the tool <CODE>pop2imap</CODE> to sync them.
<H1>4. Imapsync presentation</H1>
<P>
Software imapsync is a command line tool to
copy, migrate, backup or synchronize IMAP mailboxes.
Software imapsync is a command-line tool to
copy, migrate, backup, or synchronize IMAP mailboxes.
</P>
<P>
Command line means imapsync is not graphical, it is textual.
Usually with command line tools you have to type characters
Usually, with command-line tools, you have to type characters
on your keyboard. But your fingers won't suffer much pain
typing on the keyboard because script examples are given,
nearly ready to run. Most of the time you only have to change
@ -343,25 +342,25 @@ select/copy/paste complete examples,
and run the little script with a double-click.
</P>
<P>
Imapsync runs on Linux, Windows and OS X (Macintosh world).
Imapsync runs on Linux, Windows, and OS X (Macintosh/Darwin world).
Imapsync is written in the Perl language and, thanks to the
Perl developers, Perl runs mostly everywhere. So does imapsync.
</P>
<P>
While operating systems have a lot in common, they sometimes differ,
especially within their syntax. I won't blame anyone, historically Windows
came after Unix. The marvelous designers in this old times
decided it would be very cool to not share exactly the same syntax
came after Unix. The marvelous designers in these old times
decided it would be very cool to not share the same syntax
for doing the same things.
</P>
<P>
To designate an end of line, Unix uses the character \n
To designate an end of the line, Unix uses the character \n
Windows uses two characters \r\n and Mac use \r.
Thanks guys, great great thinking!
Thanks to you guys, great thinking!
Fifty years later, we still suffer from this...daily.
</P>
<P>
To avoid you some headaches with systems no one masters
To avoid some headaches with systems that no one masters
I will give examples in both worlds, Unix and Windows.
OS X users are in the Unix world nowadays so they must follow
the Unix examples.
@ -371,7 +370,7 @@ the Unix examples.
<H1>5. Conventions</H1>
<P>
In order to simplify display or print,
To simplify display or print,
each imapsync command line is usually written in several lines
but it could be written in one single line.
</P>
@ -379,11 +378,11 @@ but it could be written in one single line.
If you prefer to use the whole command written in one single line then
just remove the last visible character of each line ( \ or ^ ) and
also the carriage return character.
The last visible character means "command continues on next line";
it is the backslash \ character on Unix and the caret ^ character on Windows.
The last visible character \ or ^ means "command continues on next line";
it's the backslash \ character on Unix and the caret ^ character on Windows.
</P>
<P>
For example, on Unix, a command like the following
For example, on Unix, a command like the following:
</P>
<PRE>
@ -395,7 +394,7 @@ For example, on Unix, a command like the following
</PRE>
<P>
is equivalent to
is equivalent to:
</P>
<PRE>
@ -403,7 +402,7 @@ is equivalent to
</PRE>
<P>
and on Windows
and on Windows, a command like the following:
</P>
<PRE>
@ -415,7 +414,7 @@ and on Windows
</PRE>
<P>
is equivalent to
is equivalent to:
</P>
<PRE>
@ -435,16 +434,16 @@ messages that couldn't be transferred stay on host1.
</P>
<P>
It's not the same for destination accounts as imapsync writes on host2 accounts.
Imapsync creates folders on them, add messages, set flags on messages.
It isn't a safe behavior on a real account. So don't use a real user account
Imapsync creates folders on them, adds messages, and sets flags on messages.
It isn't safe on a real account. So don't use a real user account
to test imapsync. Learn to use it and see what it does on a test account at host2.
</P>
<P>
What can badly happen? The most common bad behavior is
the folders mapping won't be what you expect because it is strictly
reproduced from host1 to host2. The second bad behavior is
duplicates on second run and after, it's rare but it can happen
when a imap server software changes headers "<CODE>Message-Id</CODE>" or "<CODE>Received</CODE>".
that folders mapping won't be what you expect because it is strictly
reproduced from host1 to host2. The second bad behavior is getting
duplicates on the second run and after; it's rare but it can happen
when an imap server software changes the headers "<CODE>Message-Id</CODE>" or "<CODE>Received</CODE>".
Solutions to avoid duplicates are often easy (There's a FAQ called <CODE>FAQ.Duplicates.txt</CODE> about that).
It's also possible to remove the duplicates on host2 but it's better to avoid them on user accounts at first,
users won't like that you mess up their mailboxes.

115
doc/TUTORIAL_Unix.t2t
View file

@ -1,6 +1,6 @@
Imapsync tutorial
Gilles LAMIRAL gilles@lamiral.info
$Id: TUTORIAL_Unix.t2t,v 1.26 2019/02/10 12:33:17 gilles Exp gilles $
$Id: TUTORIAL_Unix.t2t,v 1.27 2022/01/14 12:42:51 gilles Exp gilles $
+ Good practices overview +
@ -16,20 +16,20 @@ without much pain. So try imapsync with a real account at the source, aka host1,
and a test account at the destination, aka host2.
- Next, once familiar and satisfied by the result on the host2 test account,
change to a real user account on host2 or just stop consider it a test one.
change to a real user account on host2 or just stop considering it a test one.
+ Basic steps +
++ Verifying imapsync works well on its own ++
++ Verifying imapsync works well ++
Open a terminal and go to the imapsync directory.
The imapsync directory is the directory created by extraction
of the tarball (.tgz), its name is ``imapsync-1.xxx`` where ``1.xxx``
is imapsync release number (1.882 at the time of this writing).
The imapsync directory is the directory created by the extraction
of the tarball (.tgz), its name is ``imapsync-2.xxx`` where ``2.xxx``
is imapsync release number (2.178 at the time of this proofread).
```
cd imapsync-1.xxx/
cd imapsync-2.xxx/
```
Verify imapsync runs on your system
@ -39,13 +39,13 @@ Verify imapsync runs on your system
It should output the help message.
If the previous command fails then there is an installation issue.
Go back to https://imapsync.lamiral.info/#install then
and read and apply the installation file corresponding to your
Go back to the https://imapsync.lamiral.info/#install section, then
read and apply the installation file corresponding to your
system and drop me an email about your issue.
Next, verify imapsync runs live tests. This check needs an Internet
Next, verify imapsync runs well with live tests. This check needs Internet
access. It does a simple sync between two real dedicated
imap maiboxes located at the host test.lamiral.info
imap mailboxes located at the host test.lamiral.info
```
./imapsync --testslive
```
@ -54,7 +54,7 @@ Now verify that the script examples/imapsync_example.sh runs fine:
```
sh examples/imapsync_example.sh
```
This script does the same thing than "``imapsync --testslive``" but it
This script does the same thing as "``imapsync --testslive``" but it
uses explicitly the 6 parameters so it will be a good start
for your future own script.
@ -76,22 +76,22 @@ So far so good, now we're going to work with your data.
++ Prepare your credentials ++
An IMAP account is accessed with 3 parameters,
- the imap server **host**. It's a server name or an ip address
- the imap server **host**. It's a server name or an IP address
- the **user** name
- the **password**
Since imapsync job is to sync two imap accounts we need 3 + 3 = 6 parameters:
- Three parameters in order to read data from the source account: **host1**, **user1** and **password1**
- Three parameters in order to copy this data to the destination account: **host2**, **user2** and **password2**
Since imapsync job is to sync two imap accounts, we need 3 + 3 = 6 parameters:
- Three parameters to read data from the source account: **host1**, **user1** and **password1**
- Three parameters to copy this data to the destination account: **host2**, **user2** and **password2**
++ Take a real user account as source ++
++ Take a real user account as a source ++
Even during the learning time with imapsync, you can take a
real user account as a source. There is also no problem if this account is
currently used by a user. By default, this account will only be read, no change will
be made by imapsync on it.
be made by imapsync on it. It's safe to use a normal and live account as a source, even to learn imapsync.
Assuming that the imap source server name host1 is **origin.example.com**,
the user1 account name is **myuser1** and its password is **mysecret1**,
@ -102,12 +102,12 @@ we now have the first three parameters.
- --password1 **mysecret1**
++ Take a test user account as destination ++
++ Take a test user account as a destination ++
Unlike the source side, the destination side will be modified by
imapsync. Therefore, for learning, checking and adjusting,
it is not a very good idea to use a real user imap account
the first times you play with imapsync.
imapsync. Therefore, for learning, checking, and adjusting,
it is not a good idea to use a real user imap account
the first time you play with imapsync.
If you really can't afford a test account on host2, it's ok,
imapsync is not that bad but you may have some work to do to fix some unwanted behavior.
@ -124,7 +124,7 @@ we now have the next three parameters.
++ Edit your own script mysync ++
Now edit the script ``mysync`` and replace the test values by yours.
Now edit the script ``mysync`` and replace the test values with yours.
You're ready for a dry test on your accounts.
@ -135,27 +135,26 @@ sh mysync
Since the ``mysync`` script is a copy of ``examples/imapsync_example.sh``,
your first run with your data should include three other options
``--automap`` ``--justfolders`` ``--dry``.
With ``--dry`` option, nothing will really be done on host2
With ``--dry`` option, nothing will be done for real on host2
but yet it will test whether the credentials are ok on both sides.
You'll encounter two successful logins, or one failure on host1,
or a successful login on host1 but a failure on host2.
If the login are ok, you will also observe if the folders mapping is ok.
You'll encounter two successful logins, or one login failure on host1 or host2, or two failures on host1 and host2.
If the logins are ok, you will be able to observe that the folders mapping is ok.
If a login fails then double-check all three values that identify
If login fails then double-check all three values that identify
the account, which are the host, the login name, and the password.
If the folders mapping proposed is not ok then you can fix it with
option ``--f1f2``. The following example maps the source folder
the option ``--f1f2``. The following example maps the source folder
"Sent Messages" to the destination folder "Sent". The double-quotes
are not part of the folders names but they should be used when special
characters like blanks are in the folders names:
are not part of the names of the folders but they should be used when special
characters like blanks are in the names of the folders:
```
./imapsync ... --f1f2 "Sent Messages=Sent"
```
As explained in the inline help or in the README:
As explained in the inline help or the README:
```
--f1f2 str1=str2 : Force folder str1 to be synced to str2.
```
@ -169,7 +168,7 @@ sh mysync
+ Background knowledge about emailboxes +
+ Background knowledge about mailboxes +
Three Internet protocols are used to access almost all email accounts:
POP3, IMAP, HTTP.
@ -187,7 +186,7 @@ they are unavailable from any other system located elsewhere.
The second protocol to deal with email messages is IMAP, Internet Message Access Protocol.
IMAP gives access to a hierarchy of mailboxes also called folders. Other IMAP features are
concurrent accesses, tagging with flags, search by many criterium like date, subject, size etc.
concurrent accesses, tagging with flags, search by many criteriums like date, subject, size, etc.
The IMAP protocol presents most of the features POP lacks.
Messages stay on the imap server so any client on the network can access them
@ -199,10 +198,10 @@ HTTP is the protocol to browse the web.
Web browsers like Google Chrome, Mozilla Firefox, Internet Explorer and Safari
are HTTP client software tools. You already know that so what's the point with HTTP mailboxes?
HTTP mailboxes are called webmails.
Webmails often offer the same features than imap servers
Webmails often offer the same features as imap servers
because webmails underlying storage systems are often imap servers.
Webmails systems like Gmail, Yahoo, Exchange, Zimbra or Office365 are also accessible via imap.
Webmails systems like Gmail, Yahoo, Exchange, Zimbra, or Office365 are also accessible via imap.
The conclusion of this protocol review is that mailboxes can be
accessed using the IMAP protocol, most of the time.
@ -215,11 +214,11 @@ you can use the tool ``pop2imap`` to sync them.
+ Imapsync presentation +
Software imapsync is a command line tool to
copy, migrate, backup or synchronize IMAP mailboxes.
Software imapsync is a command-line tool to
copy, migrate, backup, or synchronize IMAP mailboxes.
Command line means imapsync is not graphical, it is textual.
Usually with command line tools you have to type characters
Usually, with command-line tools, you have to type characters
on your keyboard. But your fingers won't suffer much pain
typing on the keyboard because script examples are given,
nearly ready to run. Most of the time you only have to change
@ -230,22 +229,22 @@ You can still use the mouse to launch an editor,
select/copy/paste complete examples,
and run the little script with a double-click.
Imapsync runs on Linux, Windows and OS X (Macintosh world).
Imapsync runs on Linux, Windows, and OS X (Macintosh/Darwin world).
Imapsync is written in the Perl language and, thanks to the
Perl developers, Perl runs mostly everywhere. So does imapsync.
While operating systems have a lot in common, they sometimes differ,
especially within their syntax. I won't blame anyone, historically Windows
came after Unix. The marvelous designers in this old times
decided it would be very cool to not share exactly the same syntax
came after Unix. The marvelous designers in these old times
decided it would be very cool to not share the same syntax
for doing the same things.
To designate an end of line, Unix uses the character \n
To designate an end of the line, Unix uses the character \n
Windows uses two characters \r\n and Mac use \r.
Thanks guys, great great thinking!
Thanks to you guys, great thinking!
Fifty years later, we still suffer from this...daily.
To avoid you some headaches with systems no one masters
To avoid some headaches with systems that no one masters
I will give examples in both worlds, Unix and Windows.
OS X users are in the Unix world nowadays so they must follow
the Unix examples.
@ -253,17 +252,17 @@ the Unix examples.
+ Conventions +
In order to simplify display or print,
To simplify display or print,
each imapsync command line is usually written in several lines
but it could be written in one single line.
If you prefer to use the whole command written in one single line then
just remove the last visible character of each line ( \ or ^ ) and
also the carriage return character.
The last visible character means "command continues on next line";
it is the backslash \ character on Unix and the caret ^ character on Windows.
The last visible character \ or ^ means "command continues on next line";
it's the backslash \ character on Unix and the caret ^ character on Windows.
For example, on Unix, a command like the following
For example, on Unix, a command like the following:
```
imapsync \
@ -272,12 +271,12 @@ For example, on Unix, a command like the following
--password1 secret1 \
...
```
is equivalent to
is equivalent to:
```
imapsync --host1 test.lamiral.info --user1 test1 --password1 secret1 ...
```
and on Windows
and on Windows, a command like the following:
```
imapsync.exe ^
--host1 test.lamiral.info ^
@ -285,7 +284,7 @@ and on Windows
--password1 secret1 ^
...
```
is equivalent to
is equivalent to:
```
imapsync --host1 test.lamiral.info --user1 test1 --password1 secret1 ...
```
@ -301,15 +300,15 @@ messages that couldn't be transferred stay on host1.
It's not the same for destination accounts as imapsync writes on host2 accounts.
Imapsync creates folders on them, add messages, set flags on messages.
It isn't a safe behavior on a real account. So don't use a real user account
Imapsync creates folders on them, adds messages, and sets flags on messages.
It isn't safe on a real account. So don't use a real user account
to test imapsync. Learn to use it and see what it does on a test account at host2.
What can badly happen? The most common bad behavior is
the folders mapping won't be what you expect because it is strictly
reproduced from host1 to host2. The second bad behavior is
duplicates on second run and after, it's rare but it can happen
when a imap server software changes headers "``Message-Id``" or "``Received``".
that folders mapping won't be what you expect because it is strictly
reproduced from host1 to host2. The second bad behavior is getting
duplicates on the second run and after; it's rare but it can happen
when an imap server software changes the headers "``Message-Id``" or "``Received``".
Solutions to avoid duplicates are often easy (There's a FAQ called ``FAQ.Duplicates.txt`` about that).
It's also possible to remove the duplicates on host2 but it's better to avoid them on user accounts at first,
users won't like that you mess up their mailboxes.

27
examples/file.txt
View file

@ -1,35 +1,36 @@
# Example file.txt for imapsync massive migration.
#
# $Id: file.txt,v 1.15 2020/03/10 19:49:52 gilles Exp gilles $
# $Id: file.txt,v 1.17 2022/01/09 09:49:25 gilles Exp gilles $
#
# Each line contains at least 6 columns, columns are parameters for
# Each line contains at least 6 columns
# The separator is the character semi-colon ";"
# The columns are the values for the parameters
# --host1 --user1 --password1 --host2 --user2 --password2
#
# Extra columns can be used to pass extra parameters but the script reading
# this file have to read them into some variables.
# A extra column can be used to pass some extra parameters but the script reading
# this file have to be able to get them.
#
# Last, don't forget the last semicolon.
# Don't forget the last semicolon.
#
# Windows: see the script examples/sync_loop_windows.bat
# Unix: see the script examples/sync_loop_unix.sh
# Power users: see the script examples/sync_parallel_unix.sh
# Lines starting with a # are usually comments and ignored
# Blank lines are ignored as well
# Now the data example
host001_1;user001_1;password001_1;host001_2;user001_2;password001_2;
host002_1;user002_1;password002_1;host002_2;user002_2;password002_2;
host003_1;user003_1;password003_1;host003_2;user003_2;password003_2;
host001_1;user001_1;password001_1;host001_2;user001_2;password001_2;;
host002_1;user002_1;password002_1;host002_2;user002_2;password002_2;--subfolder2 foo;
host003_1;user003_1;password003_1;host003_2;user003_2;password003_2;--justfolders --automap --subfolder2 foo;
# Another comment blabla
host004_1;user004_1;password004_1;host004_2;user004_2;password004_2;
host004_1;user004_1;password004_1;host004_2;user004_2;password004_2;;
# This last example is a real one, ie, truly working in the real world.
test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;secret2;
test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;secret2;;
# The reverse
test2.lamiral.info;test2;secret2;test1.lamiral.info;test1;secret1;
# The reverse with some extra parameters
test2.lamiral.info;test2;secret2;test1.lamiral.info;test1;secret1;--dry --subfolder2 foo;

65
examples/sync_loop_darwin.sh
View file

@ -1,41 +1,64 @@
#!/bin/sh
#
# $Id: sync_loop_darwin.sh,v 1.2 2018/02/12 21:54:59 gilles Exp gilles $
# $Id: sync_loop_darwin.sh,v 1.3 2022/01/10 08:03:46 gilles Exp gilles $
# Example for imapsync massive migration on Unix systems.
# See also http://imapsync.lamiral.info/FAQ.d/FAQ.Massive.txt
#
# Data is supposed to be in file.txt in the following format:
# host001_1;user001_1;password001_1;host001_2;user001_2;password001_2;
# host001_1;user001_1;password001_1;host001_2;user001_2;password001_2;;
# ...
# Separator is character semi-colon ";" it can be changed by any character changing IFS=';'
# in the while loop below.
# # Each line contains 6 columns, columns are parameter values for
# The separator is the character semi-colon ";"
# this separator character can be changed to any character
# by changing IFS=';' in the while loop below.
#
# Each line contains 7 columns. These columns are the 6 parameter values
# for the imapsync command options
# --host1 --user1 --password1 --host2 --user2 --password2
# and a trailing empty fake column to avaid CR LF part going
# in the 6th parameter password2. Don't forget the last semicolon.
# plus an extra column for extra parameters and a trailing fake column
# to avoid CR LF part going in the 7th parameter extra.
# So don't forget the last semicolon, especially on MacOS systems.
#
# You can add extra options after the variable "$@"
# Use character backslash \ at the end of each suplementary line, except for the last one.
# You can also add extra options in this script after the variable "$@"
# Those options will be applied in every imapsync run, for every line.
# The imapsync command below is written in two lines to avoid a long line.
# The character backslash \ at the end of the first line is means
# "the command continues on the next line".
#
# Use character backslash \ at the end of each supplementary line,
# except for the last line.
#
# You can also pass extra options via the parameters of this script since
# they will be in "$@"
# they will be in "$@". Shell knowledge is your friend.
# The credentials filename "file.txt" used for the loop can be renamed
# by changing "file.txt" below.
# The file file_failures.txt will contain the lines from file.txt that ended
# up in error, for whatever reason. It's there to notice and replay easily
# the failed imapsync runs. Is is emptied at the beginning of the loop run.
# I let you junggle with it.
echo Looping on account credentials found in file.txt
echo Looping on accounts credentials found in file.txt
echo
{ while IFS=';' read h1 u1 p1 h2 u2 p2 fake
line_counter=0
# Empty the error listing
> file_failures.txt
{ while IFS=';' read h1 u1 p1 h2 u2 p2 extra fake
do
{ echo "$h1" | egrep '^#|^ *$' ; } > /dev/null && continue # this skip commented lines in file.txt
echo "==== Starting imapsync from host1 $h1 user1 $u1 to host2 $h2 user2 $u2 ===="
../imapsync_bin_Darwin --host1 "$h1" --user1 "$u1" --password1 "$p1" \
--host2 "$h2" --user2 "$u2" --password2 "$p2" \
"$@"
echo "==== Ended imapsync from host1 $h1 user1 $u1 to host2 $h2 user2 $u2 ===="
line_counter=`expr 1 + $line_counter`
{ echo "$h1" | tr -d '\r' | egrep '^#|^ *$' ; } > /dev/null && continue # this skip commented lines in file.txt
echo "==== Starting imapsync with --host1 $h1 --user1 $u1 --host2 $h2 --user2 $u2 $extra $@ ===="
if ../imapsync_bin_Darwin --host1 "$h1" --user1 "$u1" --password1 "$p1" \
--host2 "$h2" --user2 "$u2" --password2 "$p2" $extra "$@"
then
echo "success sync for line $line_counter "
else
echo "$h1;$u1;$p1;$h2;$u2;$p2;$extra;" | tee -a file_failures.txt
fi
echo "==== Ended imapsync with --host1 $h1 --user1 $u1 --host2 $h2 --user2 $u2 $extra $@ ===="
echo
done
done
} < file.txt

54
examples/sync_loop_unix.sh
View file

@ -1,48 +1,64 @@
#!/bin/sh
#
# $Id: sync_loop_unix.sh,v 1.10 2020/12/11 18:09:11 gilles Exp gilles $
# $Id: sync_loop_unix.sh,v 1.13 2022/01/09 09:53:47 gilles Exp gilles $
# Example for imapsync massive migration on Unix systems.
# See also http://imapsync.lamiral.info/FAQ.d/FAQ.Massive.txt
#
# Data is supposed to be in file.txt in the following format:
# host001_1;user001_1;password001_1;host001_2;user001_2;password001_2;
# host001_1;user001_1;password001_1;host001_2;user001_2;password001_2;;
# ...
# Separator is character semi-colon ";" it can be changed by any character changing IFS=';'
# in the while loop below.
# # Each line contains 6 columns, columns are parameter values for
# The separator is the character semi-colon ";"
# this separator character can be changed to any character
# by changing IFS=';' in the while loop below.
#
# Each line contains 7 columns. These columns are the 6 parameter values
# for the imapsync command options
# --host1 --user1 --password1 --host2 --user2 --password2
# and a trailing empty fake column to avoid CR LF part going
# in the 6th parameter password2. Don't forget the last semicolon.
# plus an extra column for extra parameters and a trailing fake column
# to avoid CR LF part going in the 7th parameter extra.
# So don't forget the last semicolon, especially on MacOS systems.
#
# You can add extra options after the variable "$@"
# Use character backslash \ at the end of each supplementary line, except for the last one.
# You can also add extra options in this script after the variable "$@"
# Those options will be applied in every imapsync run, for every line.
# The imapsync command below is written in two lines to avoid a long line.
# The character backslash \ at the end of the first line is means
# "the command continues on the next line".
#
# Use character backslash \ at the end of each supplementary line,
# except for the last line.
#
# You can also pass extra options via the parameters of this script since
# they will be in "$@"
# they will be in "$@". Shell knowledge is your friend.
# The credentials filename "file.txt" used for the loop can be renamed
# by changing "file.txt" below.
# The file file_failures.txt will contain the lines from file.txt that ended
# up in error, for whatever reason. It's there to notice and replay easily
# the failed imapsync runs. Is is emptied at the beginning of the loop run.
# I let you junggle with it.
echo Looping on account credentials found in file.txt
echo Looping on accounts credentials found in file.txt
echo
line_counter=0
# Empty the error listing
> file_failures.txt
{ while IFS=';' read h1 u1 p1 h2 u2 p2 fake
{ while IFS=';' read h1 u1 p1 h2 u2 p2 extra fake
do
line_counter=`expr 1 + $line_counter`
{ echo "$h1" | tr -d '\r' | egrep '^#|^ *$' ; } > /dev/null && continue # this skip commented lines in file.txt
echo "==== Starting imapsync from host1 $h1 user1 $u1 to host2 $h2 user2 $u2 ===="
echo "==== Starting imapsync with --host1 $h1 --user1 $u1 --host2 $h2 --user2 $u2 $extra $@ ===="
if imapsync --host1 "$h1" --user1 "$u1" --password1 "$p1" \
--host2 "$h2" --user2 "$u2" --password2 "$p2" \
"$@"
--host2 "$h2" --user2 "$u2" --password2 "$p2" $extra "$@"
then
echo "success sync for line $line_counter "
else
echo "$h1;$u1;$p1;$h2;$u2;$p2;" | tee -a file_failures.txt
echo "$h1;$u1;$p1;$h2;$u2;$p2;$extra;" | tee -a file_failures.txt
fi
echo "==== Ended imapsync from host1 $h1 user1 $u1 to host2 $h2 user2 $u2 ===="
echo "==== Ended imapsync with --host1 $h1 --user1 $u1 --host2 $h2 --user2 $u2 $extra $@ ===="
echo
done
done
} < file.txt

6
examples/sync_loop_windows.bat
View file

@ -1,5 +1,5 @@
@REM
@REM $Id: sync_loop_windows.bat,v 1.18 2018/05/24 11:45:42 gilles Exp gilles $
@REM $Id: sync_loop_windows.bat,v 1.19 2022/01/09 09:52:44 gilles Exp gilles $
@REM
@REM imapsync massive sync example batch for Windows users
@REM lines beginning with @REM are just comments
@ -85,11 +85,11 @@ SET csvfile=file.txt
@REM Blank lines are usually ignored. Dumping the tokens in [] in case debugging is needed
@ECHO GOT those values from %csvfile% presented inside brackets: [%%G] [%%H] [%%I] [%%J] [%%K] [%%L] [%%M]
@REM You can add extra arguments to imapsync after the variable named %arguments%
@ECHO ==== Starting imapsync from --host1 %%G --user1 %%H to --host2 %%J --user2 %%K ====
@ECHO ==== Starting imapsync with --host1 %%G --user1 %%H --host2 %%J --user2 %%K %%M %arguments% ====
@imapsync ^
--host1 %%G --user1 %%H --password1 %%I ^
--host2 %%J --user2 %%K --password2 %%L %%M %arguments%
@ECHO ==== Ended imapsync from --host1 %%G --user1 %%H to --host2 %%J --user2 %%K ====
@ECHO ==== Ended imapsync with --host1 %%G --user1 %%H --host2 %%J --user2 %%K %%M %arguments% ====
@ECHO.
)

17
examples/sync_parallel_unix.sh
View file

@ -1,6 +1,6 @@
#!/bin/sh
#
# $Id: sync_parallel_unix.sh,v 1.9 2021/02/19 13:41:51 gilles Exp gilles $
# $Id: sync_parallel_unix.sh,v 1.11 2022/01/13 12:56:28 gilles Exp gilles $
# If you're on Windows there is a possibility to install and use parallel
# but I have never tested it. I found:
@ -44,10 +44,10 @@
# The parallel command is then followed by its parameters.
# parallel parameters explained:
#
# --max-procs 7 means parallel will parallelize up to 7 jobs at a time,
# --max-procs 3 means parallel will parallelize up to 3 jobs at a time,
# adjust this value by monitoring your system capacity.
#
# --delay 1.1 means parallel will pause 1.1 seconds after starting each job.
# --delay 1.4 means parallel will pause 1.4 seconds (1400 ms) after starting each job.
#
# --colsep ';' means the separator between values is the character semi-colon ;
#
@ -96,6 +96,8 @@
# paralelized runs
# The current script does not take into account what is in the 7th column
check_parallel_is_here() {
parallel --version > /dev/null || { echo "parallel command is not installed. Install it first."; return 1; }
}
@ -114,9 +116,14 @@ DRYRUN=echo
# since the previous echo value will be discarded
DRYRUN=
parallel --max-procs 7 --delay 1.1 --colsep ';' --arg-file file.txt --line-buffer --tagstring "from {2} to {5} : " \
parallel --max-procs 3 --delay 1.4 --colsep ';' --arg-file file.txt --line-buffer --tagstring "from {2} to {5} : " \
'echo {1} | egrep "^#|^ *$" > /dev/null ||' \
$DRYRUN imapsync --host1 {1} --user1 {2} --password1 {3} \
--host2 {4} --user2 {5} --password2 {6} "$@" --simulong 5
--host2 {4} --user2 {5} --password2 {6} "$@" --simulong 5
# A question to ask to the parallel mailing-list, Ole Tange
# does not work like I want, it passes all the 7th column as only one argument to imapsync:
# '{=7 split / /, $arg[7] =}'
# I want a list of arguments

1466
imapsync
View file

File diff suppressed because it is too large Load diff

38
index.shtml
View file

@ -23,6 +23,8 @@
<!--
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" />
-->
<link href="S/quiz/quiz_imapsync.css" rel="stylesheet" />
<script src="S/quiz/quiz_imapsync.js" type="text/javascript"></script>
</head>
@ -68,10 +70,12 @@
<img width="486" height="309" alt="imapsync logo" src="S/images/logo_imapsync.png" />
</a>
<h1>Afraid of losing all your precious emails?<br/>
<h1>The Mailbox Changer
</h1>
<h2>Afraid of losing all your precious emails?<br/>
It's time to copy all of them elsewhere!<br/>
Safely.
</h1>
</h2>
<!-- I use SSI here only to pass a strict xhtml validation that facebook/twitter/coinbase can't pass -->
<!-- remove space before #include to activate these ssi -->
@ -83,9 +87,11 @@ Safely.
Site last updated on <b><!--#echo var="LAST_MODIFIED" --></b><br/>
<a href="FAQ.d/FAQ.Old_Style_Web_Design.txt">Why this website looks so old fashion?</a>
</div>
<h2>What is imapsync? <a href="#TOP" id="WHATIS"><small>(back to menu)</small></a>
</h2>
@ -176,6 +182,9 @@ See detailed explanation and motivation here when
</p>
-->
<h2>Imapsync Quiz
</h2>
<div id="quizWrap"></div>
<h2>Contact the author, Gilles LAMIRAL <a href="#TOP" id="AUTHOR"><small>(back to menu)</small></a>
@ -272,7 +281,8 @@ problems, you're sure to get my attention and my time.
<ul>
<li>Imapsync <b>full professional support</b> (that costs also &euro;120 EUR by itself, see why below).</li>
<li>Access without any limit to the imapsync online service.
<li>Access without any limit to the imapsync <b>online service</b>, like <b><a href="https://imapsync.lamiral.info/X/">/X</a></b>
but <b>without</b> the 3 GB limit.
</li>
<li>Standalone <b>imapsync.exe</b> for win32, easy installation done by a zip extraction anywhere.
@ -282,8 +292,8 @@ problems, you're sure to get my attention and my time.
<li>Imapsync Perl <b>source code</b> for any operating system, Unix, Windows, OS X.</li>
<li>The <b>online visual interface</b>, like <b><a href="https://imapsync.lamiral.info/X/">/X</a></b>,
to install the service on a <b>Linux server</b> (not working on Windows yet).</li>
<li>The <b>visual interface</b>, similar to <b><a href="https://imapsync.lamiral.info/X/">/X</a></b>,
to install the service on your own <b>Linux server</b> (not working on Windows yet).</li>
<li><b>Lifetime</b> of imapsync <b>updates</b> without extra payment.</li>
@ -298,7 +308,7 @@ problems, you're sure to get my attention and my time.
<p>
For <b>&euro;120 EUR</b> independently, you get imapsync <b>full professional support</b>,
provided by the imapsync designer and developer, Gilles LAMIRAL, your servitor, who has
been supporting imap migrations with imapsync for more than 19 years (I started in 2001).
been supporting imap migrations with imapsync for more than 20 years (I started in 2001).
See a detailed <a href="#SUPPORT"><b>support description</b></a> below.
</p>
@ -399,7 +409,7 @@ how to deal with special imap software servers (Gmail, Office365, Exchange etc.)
choosing and explaining options, explaining error messages,
solving authentication issues,
dealing with speed, quotas, special context,
running in parallel, etc., so in summary: <b>anything!</b>
running in parallel, etc., so in summary: <b>anything!</b> except doing the job for you.
</p>
<p>
@ -426,8 +436,9 @@ that is 120 EUR.
</h3>
<p>
The funny thing about crypto payments is that I receive very few of them, one or two per year, and in small amount, one or two dollars equivalent...
Maybe one day those little crypto coins will be like gold? (I would like that!)
The funny thing about crypto payments is that I receive very few of them, one or two per year,
and in small amount, one or two dollars equivalent...
Maybe one day those little crypto coins will be like gold? (I will like that!)
</p>
<!--
@ -799,9 +810,11 @@ has a <b>network access to both IMAP servers</b>, the source and the destination
<p>What to do next?<br/>
Depending on the system where you will run imapsync:</p>
<ul>
<li>Windows users, read directly <a href="README_Windows.txt">README_Windows.txt</a>.</li>
<li>OS X users, read <a href="INSTALL.d/INSTALL.Darwin.txt">INSTALL.Darwin.txt</a>.</li>
<li>OS X users, read <a href="INSTALL.d/INSTALL.Darwin.txt">INSTALL.Darwin.txt</a>. <a href="https://www.macstadium.com/"><img src="S/images/5c019d917bba312af7553b49_MacStadium-developerlogo.png" alt="MacStadium logo" width="150" height="61" /></a></li>
<li>Centos users, read <a href="INSTALL.d/INSTALL.Centos.txt">INSTALL.Centos.txt</a>.</li>
<li>CPanel users, read <a href="INSTALL.d/INSTALL.CPanel.txt">INSTALL.CPanel.txt</a>.</li>
<li>Debian users, read <a href="INSTALL.d/INSTALL.Debian.txt">INSTALL.Debian.txt</a>.</li>
@ -906,7 +919,10 @@ you some struggling hours later</b>, believe me. So read the three columns main
<li> <a href="FAQ.d/FAQ.Folders_Sizes.txt">Sizes of folders</a>. </li>
<li> <a href="FAQ.d/FAQ.Admin_Authentication.txt">Admin authentication</a>. </li>
<li> <a href="FAQ.d/FAQ.Archiving.txt">Archiving tips</a>. </li>
<li> <a href="FAQ.d/FAQ.General.txt">General FAQ</a>. </li>
<li> <a href="FAQ.d/FAQ.Fun_Things.txt">Fun things to do with Imapsync</a>. </li>
<li> <a href="FAQ.d/FAQ.POP3.txt">POP3 issues</a>. </li>
<li> <a href="FAQ.d/FAQ.Reporting_Bugs.txt">Reporting bugs</a>. </li>
<li> <a href="FAQ.d/FAQ.Big_Mailbox.txt">Big mailboxes</a>. </li>
@ -971,7 +987,7 @@ like written in the <a href="LICENSE">https://imapsync.lamiral.info/LICENSE</a>
<!--#config timefmt="%D" -->
<!--#config timefmt="%A %B %d, %Y" -->
<b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b>
($Id: index.shtml,v 1.499 2021/06/08 09:57:27 gilles Exp gilles $)<br />
($Id: index.shtml,v 1.508 2021/12/20 17:49:51 gilles Exp gilles $)<br />
<a href="#TOP">Top of the page</a>
</p>

13
oauth2/generate_gmail_token
View file

@ -25,7 +25,7 @@ get_refresh_token()
access_token_file="`access_token_file $1`"
# 1 Get an access_token and a refresh_token
./oauth2_google.py --user="$1" \
${my_dirname}/oauth2_google.py --user="$1" \
--client_id="108687549524-86sjq07f3ch8otl9fnr56mjnniltdrvn.apps.googleusercontent.com" \
--client_secret=zAJO4PLxzeJ4yOaiJRk6f69k \
--generate_oauth2_token --quiet
@ -44,12 +44,12 @@ regenerate_access_token()
access_token_file="`access_token_file $1`"
# 2 Regenerate an access_token if time passes too long (3600s)
refresh_token="`cat $refresh_token_file|tr -d '\r'`"
echo ./oauth2_google.py --quiet --user="$1" \
echo ${my_dirname}/oauth2_google.py --quiet --user="$1" \
--client_id="108687549524-86sjq07f3ch8otl9fnr56mjnniltdrvn.apps.googleusercontent.com" \
--client_secret=zAJO4PLxzeJ4yOaiJRk6f69k \
--refresh_token="$refresh_token"
./oauth2_google.py --quiet --user="$1" \
${my_dirname}/oauth2_google.py --quiet --user="$1" \
--client_id="108687549524-86sjq07f3ch8otl9fnr56mjnniltdrvn.apps.googleusercontent.com" \
--client_secret=zAJO4PLxzeJ4yOaiJRk6f69k \
--refresh_token="$refresh_token" | tee "$access_token_file"
@ -86,7 +86,7 @@ generate_oauthdirect()
access_token="`cat $access_token_file`"
oauthdirect_file="`oauthdirect_file $1`"
# 3 Generate oauth2_string for imap authentication
./oauth2_google.py --generate_oauth2_string --user="$1" \
${my_dirname}/oauth2_google.py --generate_oauth2_string --user="$1" \
--access_token="$access_token" | tail -1 | tee "$oauthdirect_file"
echo oauth2_string saved in "$oauthdirect_file"
echo Give this to imapsync --oauthdirect1 or --oauthdirect2
@ -101,7 +101,7 @@ authenticate_imap()
access_token_file="`access_token_file $1`"
access_token="`cat $access_token_file`"
echo $access_token
./oauth2_imap_from_token "$1" "$access_token"
${my_dirname}/oauth2_imap_from_token "$1" "$access_token"
}
@ -112,7 +112,7 @@ authenticate_imap_google_from_access_token()
access_token_file="`access_token_file $1`"
access_token="`cat $access_token_file`"
echo "$access_token"
./oauth2_google.py --user="$1" \
${my_dirname}/oauth2_google.py --user="$1" \
--access_token="$access_token" --test_imap_authentication
}
@ -185,5 +185,6 @@ generate_token()
}
my_dirname=`dirname $0`
generate_token "$@"

192
tests.sh
View file

@ -1,6 +1,6 @@
#!/bin/sh
# $Id: tests.sh,v 1.365 2021/07/05 20:19:12 gilles Exp gilles $
# $Id: tests.sh,v 1.367 2022/01/13 12:59:05 gilles Exp gilles $
# To run these tests, you need a running imap server somewhere
# with several accounts. And be on Linux or Unix.
@ -218,6 +218,7 @@ set_return_code_variables()
EXIT_ERR_SELECT=117
EXIT_TRANSFER_EXCEEDED=118
EXIT_ERR_APPEND_VIRUS=119
EXIT_ERR_FLAGS=120
EXIT_TESTS_FAILED=254 # Like Test::More API
}
@ -272,6 +273,19 @@ option_extra_arguments() {
test "$?" = "$EX_USAGE"
}
option_extra() {
(
mkdir -p W/tmp/tests/options_extra/ || return 1
cd W/tmp/tests/options_extra/ || return 1
echo '--debugimap' > options_extra.txt
test -f ../../../../imapsync
../../../../imapsync --testslive
test "$?" = "$EX_OK"
)
pwd
}
passwords_masked() {
$CMD_PERL ./imapsync --host1 boumboum --password1 secret --justbanner | grep MASKED
}
@ -415,6 +429,19 @@ ll_INBOX() {
--folder INBOX
}
ll_daily_digest() {
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST1 --user2 tata \
--passfile2 ../../var/pass/secret.tata \
--folder INBOX --dry --nodry1 --maxage 5 \
--truncmess 1000 --debugcontent --f1f2 INBOX=INBOX.Fake \
| egrep 'From:|To:|Subject:|Date:|=====|msg '
}
ll_acl() {
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
@ -641,7 +668,7 @@ ll_abort_noprocess()
# In mandatory_tests
ll_abort_not_a_pid_number()
{
echo 999999 > /tmp/imapsync_fake.pid
echo 12345678 > /tmp/imapsync_fake.pid
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
@ -649,7 +676,7 @@ ll_abort_not_a_pid_number()
--passfile2 ../../var/pass/secret.titi \
--abort --pidfile /tmp/imapsync_fake.pid \
--logfile ll_abort_not_a_pid_number.log
grep 'pid 999999 in /tmp/imapsync_fake.pid is not a pid number' LOG_imapsync/ll_abort_not_a_pid_number.log
grep 'pid 12345678 in /tmp/imapsync_fake.pid is not a pid number' LOG_imapsync/ll_abort_not_a_pid_number.log
}
@ -884,7 +911,7 @@ ll_final_diff() {
--folder INBOX --f1f2 INBOX=INBOX.final_diff --maxage 30
}
ll_with_errors() {
ll_with_flags_errors() {
can_send && sendtestmessage
can_send && sendtestmessage
$CMD_PERL ./imapsync \
@ -892,9 +919,9 @@ ll_with_errors() {
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--folder INBOX --maxage 30 \
--regexflag 's/.*/PasGlop \\PasGlopRe/'
test "$EXIT_WITH_ERRORS" = "$?"
--folder INBOX --maxage 300 \
--regexflag 's/.*/PasGlop \\PasGlopRe/' --errorsmax 5
test "$EXIT_ERR_FLAGS" = "$?"
}
@ -905,10 +932,11 @@ ll_errorsmax() {
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--nofoldersizes --folder INBOX.errors --regexflag 's/.*/PasGlop \\PasGlopRe/' --errorsmax 5 --delete2
--nofoldersizes --folder INBOX.errors --regexflag 's/.*/PasGlop \\PasGlopRe/' --errorsmax 5 \
| grep 'Maximum number of errors 5 reached'
#--pipemess 'grep lalalala' --nopipemesscheck --dry --debugcontent --debugflags
#test "$EXIT_WITH_ERRORS_MAX" = "$?" # no longer used since errors classification
test "$EXIT_WITH_ERRORS" = "$?"
#test "$EXIT_ERR_FLAGS" = "$?"
}
ll_debug()
@ -1182,7 +1210,7 @@ ll_timeout1_timeout2() {
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--folder INBOX --timeout1 0.99 --timeout2 0.95 --justlogin
--folder INBOX --timeout1 1.99 --timeout2 1.95 --justlogin
}
ll_timeout_timeout1() {
@ -1777,19 +1805,68 @@ gmail_l_automap() {
ll_justfolders() {
$CMD_PERL ./imapsync \
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--justfolders
echo "sudo rm -rf /home/vmail/titi/.new_folder/"
echo "sudo rm -rf /home/vmail/titi/.new_folder/"
}
ll_create_folder_New1()
{
./W/learn/create_folder localhost tata `cat ../../var/pass/secret.tata` INBOX.New1 INBOX.New1.New1 INBOX.New1.New1.New1
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--justfolders --include New1 --folderfirst INBOX.New1.New1
}
ll_delete_folder_New1()
{
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--include New1 --folderfirst INBOX.New1.New1 --delete1emptyfolders --delete1
}
ll_create_folder_encoding_accent()
{
./W/learn/create_folder localhost tata `cat ../../var/pass/secret.tata` INBOX.New1 'INBOX.New1.E&AwE-le&AwE-ments envoye&AwE-s' 'INBOX.New1.&AMk-l&AOk-ments envoy&AOk-s'
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--justfolders --include New1
}
ll_create_folder_encoding_accent_365()
{
#./W/learn/create_folder localhost tata `cat ../../var/pass/secret.tata` 'INBOX.E&AwE-le&AwE-ments envoye&AwE-s' 'INBOX.&AMk-l&AOk-ments envoy&AOk-s'
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 imap-mail.outlook.com --user2 gilles.lamiral@outlook.com \
--passfile2 ../../var/pass/secret.outlook.com \
--justfolders --include 'ments envoy' --automap --exclude New1
}
ll_justfolders_delete1emptyfolders() {
./W/learn/create_folder localhost tata `cat /g/var/pass/secret.tata` INBOX.Empty INBOX.Empty.Empty INBOX.Empty.Empty.Empty
$CMD_PERL ./imapsync \
./W/learn/create_folder localhost tata `cat ../../var/pass/secret.tata` INBOX.Empty INBOX.Empty.Empty INBOX.Empty.Empty.Empty
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
@ -1798,10 +1875,9 @@ ll_justfolders_delete1emptyfolders() {
}
ll_delete1_delete1emptyfolders() {
./W/learn/create_folder localhost tata `cat /g/var/pass/secret.tata` INBOX.Empty INBOX.Empty.Empty INBOX.Empty.Empty.Empty
$CMD_PERL ./imapsync \
./W/learn/create_folder localhost tata `cat ../../var/pass/secret.tata` INBOX.Empty INBOX.Empty.Empty INBOX.Empty.Empty.Empty
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
@ -1812,7 +1888,7 @@ ll_delete1_delete1emptyfolders() {
ll_justfolders_skipemptyfolders() {
$CMD_PERL ./imapsync \
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
@ -2646,6 +2722,27 @@ ll_search_UNSEEN_SENTSINCE()
}
ll_search_FROM_TO_CC()
{
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--folder INBOX --dry --search 'OR FROM gilles@localhost (OR TO gilles@localhost (CC gilles@localhost))'
}
ll_search_FROM()
{
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--folder INBOX --dry --search 'FROM gilles@localhost'
}
ll_maxage_nonew()
{
@ -2771,6 +2868,23 @@ ll_exitwhenover()
}
ll_exitwhenover_noerrorsdump()
{
sendtestmessage
sendtestmessage
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--maxage 1 --folder INBOX --nofoldersizes \
--exitwhenover 100 --noerrorsdump
test "$EXIT_TRANSFER_EXCEEDED" = "$?"
}
ll_folder_INBOX()
@ -3046,6 +3160,20 @@ ll_regextrans2()
--folder 'INBOX.yop.yap' --debug
}
ll_add_suffix()
{
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--justfolders \
--regextrans2 's,$,\@easterngraphics.com,' \
--folderrec 'INBOX.yop' --dry --justfolders
}
ll_regextrans2_ucfirst_downcase_last_folder()
{
# lowercase the last basename part
@ -3724,6 +3852,7 @@ ll_regexmess_truncate_long_message_substr()
}
ll_regexmess_truncate_long_message_truncmess()
{
#
@ -6085,6 +6214,15 @@ ll_justlogin_notls() {
--justlogin --notls1 --notls2
}
ll_justlogin_nocompress() {
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--justlogin --nocompress2
}
@ -6465,6 +6603,11 @@ office365_justlogin_ssl1_ssl2()
--justlogin
}
outlook_login()
{
office365_justlogin_ssl1_ssl2
}
office365_justlogin_tls()
{
$CMD_PERL ./imapsync \
@ -6743,7 +6886,7 @@ firstclass_fullfill() {
}
Alessandro_error_11()
Alessandro_error_11()
{
# $CMD_PERL ./imapsync \
# --host1 $HOST1 --user1 tata \
@ -6757,10 +6900,11 @@ Alessandro_error_11()
--passfile1 ../../var/pass/secret.tata \
--host2 imap-mail.outlook.com --ssl2 --user2 gilles.lamiral@outlook.com \
--passfile2 ../../var/pass/secret.outlook.com \
--folder INBOX.error_11 --debugcontent --nodry --nodry1 --pipemess 'cat /g/Alessandro_error_11.txt'
--folder INBOX.error_11 --debugcontent --nodry --nodry1 --pipemess 'cat /g/Alessandro_error_11.txt' \
--regexmess "s{\QSubject: =?TELETEX?Q?Fw=3APresentation_Storia_dell=5C=27Informatica?=\E}{Subject: Presentation Storia dell'Informatica}"
# Subject: =?TELETEX?Q?Fw=3APresentation_Storia_dell=5C=27Informatica?=
# Subject: Presentation Storia dell'Informatica
#--pipemess W/tools/fix_email_for_exchange.py
#--pipemess 'reformime -r7'
@ -7237,6 +7381,7 @@ option_releasecheck
option_noreleasecheck
option_bad_delete2
option_extra_arguments
option_extra
passfile1_noexist
passfile2_noexist
passwords_masked
@ -7395,9 +7540,10 @@ ll_noheader_force
ll_noheader
ll_domino1_domino2
ll_domino2
ll_with_errors
ll_with_flags_errors
ll_errorsmax
ll_exitwhenover
ll_exitwhenover_noerrorsdump
fuzz_basic
fuzz_network
testslive