zydronium/imapsync
1
0
Fork 0
mirror of https://github.com/imapsync/imapsync.git synced 2025-06-06 20:55:29 +02:00
Code Issues Projects Releases Packages Wiki Activity

2.178

Browse source
This commit is contained in:
Nick Bebout 2022-02-15 12:29:45 -06:00
parent f98c9946e5
commit 2f815205a9
72 changed files with 4567 additions and 1469 deletions
Download patch file Download diff file Expand all files Collapse all files

178
ChangeLog
View file

@ -1,17 +1,187 @@
RCS file: RCS/imapsync,v RCS file: RCS/imapsync,v
Working file: imapsync Working file: imapsync
head: 2.140 head: 2.178
branch: branch:
locks: strict locks: strict
gilles: 2.140 gilles: 2.178
access list: access list:
symbolic names: symbolic names:
keyword substitution: kv keyword substitution: kv
total revisions: 1040; selected revisions: 1040 total revisions: 1078; selected revisions: 1078
description: description:
---------------------------- ----------------------------
revision 2.140 locked by: gilles; revision 2.178 locked by: gilles;
date: 2022/01/12 21:28:37; author: gilles; state: Exp; lines: +75 -13
Added tests_compress_ssl()
----------------------------
revision 2.177
date: 2021/12/23 20:38:44; author: gilles; state: Exp; lines: +8 -8
Bugfix. 1849 -> 1848 unit tests
----------------------------
revision 2.176
date: 2021/12/23 20:19:04; author: gilles; state: Exp; lines: +35 -23
Bugfix. memory consumption on Mac was not relevant. Use RSS instead of VSZ, on Mac only. Linux stays with VSZ.
----------------------------
revision 2.175
date: 2021/12/21 17:46:14; author: gilles; state: Exp; lines: +23 -21
Bugix. Darwin, fixed loadavg tests, no more W/t/loadavg.out needed.
----------------------------
revision 2.174
date: 2021/12/14 15:51:58; author: gilles; state: Exp; lines: +62 -17
Added advice for:
ERR_APPEND_SIZE
ERR_CONNECTION_FAILURE_HOST1
ERR_CONNECTION_FAILURE_HOST2
ERR_AUTHENTICATION_FAILURE_USER1
ERR_AUTHENTICATION_FAILURE_USER2
----------------------------
revision 2.173
date: 2021/12/09 18:30:34; author: gilles; state: Exp; lines: +61 -40
Bugfix. Made reconnect mechanism works with --oauthdirect1 --oauthdirect2 --oauthaccesstoken1 --oauthaccesstoken2
----------------------------
revision 2.172
date: 2021/12/07 18:19:42; author: gilles; state: Exp; lines: +27 -17
Added some tests to tests_get_options_from_string() and tests_get_options_extra()
I can not have the unknown options back with Getopt::Long::GetOptionsFromString(), just the unknown values, ie, caca but not --caca
----------------------------
revision 2.171
date: 2021/12/07 17:50:12; author: gilles; state: Exp; lines: +14 -8
Added --ssl1 --ssl2 --tls1 --tls2 --compress1 --compress2 to get_options_from_string()
----------------------------
revision 2.170
date: 2021/12/07 17:19:44; author: gilles; state: Exp; lines: +12 -8
Added --keepalive1 --keepalive2 --reconnectretry1 --reconnectretry2 in get_options_from_string()
----------------------------
revision 2.169
date: 2021/12/07 17:13:44; author: gilles; state: Exp; lines: +101 -36
Added --keepalive1 and --keepalive2 options. On by default.
See https://metacpan.org/pod/Mail::IMAPClient#Keepalive
Use --nokeepalive1 and --nokeepalive2 to disable Keepalive.
----------------------------
revision 2.168
date: 2021/12/05 23:27:27; author: gilles; state: Exp; lines: +29 -11
Added --debugflags --errorsmax --folder --timeout in options from file
----------------------------
revision 2.167
date: 2021/12/02 16:59:51; author: gilles; state: Exp; lines: +9 -9
Moved the call to extra options after the chdir of the CGI context.
----------------------------
revision 2.166
date: 2021/12/02 15:23:37; author: gilles; state: Exp; lines: +107 -9
Now read extra options on the first line of the file ./options_extra.txt if it is present and readable.
----------------------------
revision 2.165
date: 2021/11/30 20:27:38; author: gilles; state: Exp; lines: +30 -11
cpu_time rounded with 2 decimals.
Added 4 tests when no compression is on.
----------------------------
revision 2.164
date: 2021/11/24 21:26:46; author: gilles; state: Exp; lines: +12 -11
Compression no more on by default. Just to see.
----------------------------
revision 2.163
date: 2021/11/20 20:11:46; author: gilles; state: Exp; lines: +8 -7
Bugfix. imapsync --version was buggy in docker context.
----------------------------
revision 2.162
date: 2021/11/20 10:16:33; author: gilles; state: Exp; lines: +14 -12
Docker. Current working directory changed from /var/tmp to /var/tmp/uid_$EFFECTIVE_USER_ID
----------------------------
revision 2.161
date: 2021/11/19 18:17:48; author: gilles; state: Exp; lines: +8 -8
Bugfix. inline doc mentioned always --nocompress1
----------------------------
revision 2.160
date: 2021/11/19 14:40:58; author: gilles; state: Exp; lines: +10 -7
Added inline documentation about --nocompress1 and --nocompress2
----------------------------
revision 2.159
date: 2021/11/19 14:29:31; author: gilles; state: Exp; lines: +113 -12
Dependency. Added use Compress::Zlib
Enhancement. Added compression in imap. On by default no matter the capability announced. Use --nocompress1 and --nocompress2 to turn it off.
----------------------------
revision 2.158
date: 2021/11/13 20:01:35; author: gilles; state: Exp; lines: +13 -7
Added README help for --truncmess
----------------------------
revision 2.157
date: 2021/10/23 21:51:43; author: gilles; state: Exp; lines: +61 -16
Refactor. Deglobalized $debugcontent. Added debugcontent() function.
1800 unit tests.
----------------------------
revision 2.156
date: 2021/10/12 13:36:36; author: gilles; state: Exp; lines: +72 -71
Added comment for ERR_FLAGS "Many STORE errors with FLAGS. Retry with the option --noresyncflags"
Refactored errors_incr() and errors_listing()
Bugfix. errors_incr() existed with CATCH_ALL
----------------------------
revision 2.155
date: 2021/10/01 07:46:55; author: gilles; state: Exp; lines: +9 -8
Like --useheader X-Gmail-Received --useheader Message-Id when --gmail1 --gmail2
----------------------------
revision 2.154
date: 2021/09/29 20:38:04; author: gilles; state: Exp; lines: +8 -8
Bugfix. Added 'E&AwE-le&AwE-ments envoye&AwE-s' in possible_special
----------------------------
revision 2.153
date: 2021/09/29 13:56:15; author: gilles; state: Exp; lines: +34 -12
Does not checknoabletosearch if --justfolders
When creating folder check if selectable and consider ok if selectable
----------------------------
revision 2.152
date: 2021/09/29 12:03:12; author: gilles; state: Exp; lines: +362 -58
Added inline advice on error ERR_OVERQUOTA
Added inline advice on error ERR_TRANSFER_EXCEEDED
Renamed function bytes_display_string() bytes_display_string_bin()
bytes_display_string_bin converts bytes to human KiB MiB GiB TiB PiB
Added function bytes_display_string_dec() to convert bytes to human decimal KB MB GB TB PB (1000 base)
----------------------------
revision 2.151
date: 2021/09/08 10:34:25; author: gilles; state: Exp; lines: +23 -18
Bugfix. load average on macosx was sometimes buggy. locale float 3,14 instead of 3.14
----------------------------
revision 2.150
date: 2021/08/08 12:10:51; author: gilles; state: Exp; lines: +9 -10
Refactor. Deglobalized warn_release variable
----------------------------
revision 2.149
date: 2021/07/31 11:24:24; author: gilles; state: Exp; lines: +7 -26
Removed --fast option.
----------------------------
revision 2.148
date: 2021/07/22 14:21:09; author: gilles; state: Exp; lines: +10 -10
Bugfix. abort didn't do charset=UTF-8; same for heavy load 503 Service Unavailable
----------------------------
revision 2.147
date: 2021/07/22 13:13:50; author: gilles; state: Exp; lines: +12 -12
Bugfix. The maximum number a pid can be is 2^22, not 2^32.
----------------------------
revision 2.146
date: 2021/07/22 12:48:21; author: gilles; state: Exp; lines: +8 -8
1742 unit tests.
----------------------------
revision 2.145
date: 2021/07/22 12:45:31; author: gilles; state: Exp; lines: +16 -13
Bugfix. Up the maximum number a pid can be. From 99999 to 2^32. See https://unix.stackexchange.com/questions/16883/what-is-the-maximum-value-of-the-process-id
----------------------------
revision 2.144
date: 2021/07/17 18:07:59; author: gilles; state: Exp; lines: +13 -10
Just an experimental loaddelay forced to 0.
----------------------------
revision 2.143
date: 2021/07/12 19:45:44; author: gilles; state: Exp; lines: +111 -15
Bugfix. Under docker context --version was buggy, too verbose.
----------------------------
revision 2.142
date: 2021/07/12 14:04:01; author: gilles; state: Exp; lines: +17 -19
Removed that hard limit --maxsize 35_651_584 set by --gmail2 since CAPABILITY is used and good.
----------------------------
revision 2.141
date: 2021/07/06 02:59:19; author: gilles; state: Exp; lines: +8 -8
cpu_time param
----------------------------
revision 2.140
date: 2021/07/05 20:14:18; author: gilles; state: Exp; lines: +14 -10 date: 2021/07/05 20:14:18; author: gilles; state: Exp; lines: +14 -10
Fixed perlcrit 4 severity 4 violations. Fixed perlcrit 4 severity 4 violations.
---------------------------- ----------------------------

52
FAQ.d/FAQ.Archiving.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Archiving.txt,v 1.25 2021/05/04 16:56:59 gilles Exp gilles $ $Id: FAQ.Archiving.txt,v 1.26 2022/01/11 12:55:53 gilles Exp gilles $
This documentation is also available online at This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -14,12 +14,12 @@ Questions answered in this FAQ are:
Q. How can I archive different accounts on the same destination account, Q. How can I archive different accounts on the same destination account,
each account on a separate folder? each account on a separate folder?
Q. How can I restore a complete account archived in a subfolder, like Q. How can I restore a complete account backuped in a subfolder, like
with the option --subfolder2? a backup done with the option --subfolder2?
Q. How to move emails from one IMAP folder to another either on the Q. How to move emails from one IMAP folder to another, either on the
same IMAP server or a different one? same IMAP server or on a different one? Example:
For example, all messages older than 1 year, move from INBOX to Archive. How to move all messages older than 1 year, from INBOX to Archive?
Q. Can imapsync be used to maintain and restore a local offline copy Q. Can imapsync be used to maintain and restore a local offline copy
of a mailbox, eg for backup purposes, using Mbox or Maildir of a mailbox, eg for backup purposes, using Mbox or Maildir
@ -47,8 +47,12 @@ R. Use --subfolder2
Examples: Examples:
To backup the account Foo in a subfolder called FooArchiv:
imapsync ... --user1 Foo --subfolder2 FooArchiv imapsync ... --user1 Foo --subfolder2 FooArchiv
To backup the account Bar in a subfolder called BarArchiv:
imapsync ... --user1 Bar --subfolder2 BarArchiv imapsync ... --user1 Bar --subfolder2 BarArchiv
In case you need a very strict sync, add options In case you need a very strict sync, add options
@ -59,13 +63,19 @@ ie, the subfolder "FooArchiv".
Examples: Examples:
To mirror the account Foo in a subfolder called FooArchiv:
imapsync ... --user1 Foo --subfolder2 FooArchiv --delete2 --delete2foldersonly "FooArchiv" imapsync ... --user1 Foo --subfolder2 FooArchiv --delete2 --delete2foldersonly "FooArchiv"
To mirror the account Bar in a subfolder called BarArchiv:
imapsync ... --user1 Bar --subfolder2 BarArchiv --delete2 --delete2foldersonly "BarArchiv" imapsync ... --user1 Bar --subfolder2 BarArchiv --delete2 --delete2foldersonly "BarArchiv"
See how to restore those with the next Q/R.
======================================================================= =======================================================================
Q. How can I restore a complete account archived in a subfolder, like Q. How can I restore a complete account backuped in a subfolder, like
with the option --subfolder2? a backup done with the option --subfolder2?
R. Use --subfolder1 R. Use --subfolder1
@ -81,26 +91,29 @@ R. Use --subfolder1
Examples: Examples:
To restore the whole folder FooArchiv hierarchy into the Foo account:
imapsync ... --user2 Foo --subfolder1 FooArchiv imapsync ... --user2 Foo --subfolder1 FooArchiv
imapsync ... --user2 Bar --subfolder1 BarArchiv To restore the whole folder BarArchiv hierarchy into the Bar account:
imapsync ... --user2 Bar --subfolder1 BarArchiv
Read the previous FAQ item "How can I archive different accounts on the same Read the previous FAQ item "How can I archive different accounts on the same
destination account, each account on a separate folder?" destination account, each account on a separate folder?"
======================================================================= =======================================================================
Q. How to move emails from one IMAP folder to another either on the Q. How to move emails from one IMAP folder to another, either on the
same IMAP server or a different one? same IMAP server or on a different one? Example:
For example, all messages older than 1 year, move from INBOX to Archive. How to move all messages older than 1 year, from INBOX to Archive?
R1. Solution: R1. Solution:
imapsync ... --minage 365 --folder Inbox --f1f2 INBOX=Archive --delete1 --noexpungeaftereach imapsync ... --minage 365 --folder Inbox --f1f2 INBOX=Archive --delete1 --noexpungeaftereach
Add option --noexpungeaftereach if speed is a concern. Use the option --noexpungeaftereach if speed is a concern.
Add option --delete1 if you want to move messages, instead of just copy/sync them. Use the option --delete1 if you want to move messages, instead of just copy/sync them.
R2. See also file FAQ.Folders_Mapping.txt R2. See also file FAQ.Folders_Mapping.txt
@ -145,20 +158,23 @@ I have never used any of this tools so asking me for support with them
won't go far away. won't go far away.
======================================================================= =======================================================================
Q. How to archive a whole year in a subfolder? Q. How to backup a whole year in a subfolder?
R. Example: R. To backup the whole year 2018 in a subfolder called ARCHIVE_2018:
imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \ imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \
--folder2 ARCHIVE_2018 --subfolder2 ARCHIVE_2018
======================================================================= =======================================================================
Q. How to archive a whole year in subfolders but without subsubfolders? Q. How to archive a whole year in subfolders but without subsubfolders?
R. Example: R. Instead of the solution with --subfolder2, use:
imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \ imapsync ... --search "SENTSINCE 1-Jan-2018 SENTBEFORE 31-Dec-2018" \
--sep2 _ --regextrans2 's{(.*)}{Archive_$1_2018}' --sep2 _ --regextrans2 's{(.*)}{Archive_$1_2018}'
For each folder "FFFF" in the source account, it will create a destination
folder called "Archive_FFFF_2018", at the top level hierarchy.
======================================================================= =======================================================================
======================================================================= =======================================================================

11
FAQ.d/FAQ.Authentication_failure.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Authentication_failure.txt,v 1.13 2021/03/02 19:24:22 gilles Exp gilles $ $Id: FAQ.Authentication_failure.txt,v 1.14 2021/11/05 11:35:07 gilles Exp gilles $
This documentation is also available online at This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -26,14 +26,16 @@ Host1 failure: Error login on [imap.example.com] with user [foo] auth [LOGIN]: 2
R. One over four imapsync syncs ends up quickly with the error message R. One over four imapsync syncs ends up quickly with the error message
"Authentication failed" or "NO LOGIN failed" or similar. "Authentication failed" or "NO LOGIN failed" or a similar message.
Authentication failure is the primary failure with imapsync Authentication failure is the primary failure with imapsync
and since nothing tangible can be done without authentication, and since nothing tangible can be done without authentication,
this stage must succeed to go further. this stage must succeed to go further.
Here are some advices to get you pass this difficult stage of authentication: Here are some advices to get you pass this difficult stage of authentication:
* Triple check each credential parameter, there are three parameters at each side: * Triple check each credential parameter. It's not always a password problem,
it can be the wrong host or the wrong user.
There are three parameters at each side:
* triple check --host1 * triple check --host1
* triple check --user1 * triple check --user1
* triple check --password1 * triple check --password1
@ -53,8 +55,7 @@ Here are some advices to get you pass this difficult stage of authentication:
instead of the string MASKED. It helps for debugging quoting issues. instead of the string MASKED. It helps for debugging quoting issues.
Option --showpasswords shows passwords again when the IMAP dialog Option --showpasswords shows passwords again when the IMAP dialog
is dumped by --debugimap option. Search for a line like is dumped by --debugimap option. Search for a line like
"Sending: 2 LOGIN test1 secret1" (secret1 is the password here) "Sending: 2 LOGIN test1 secret1" (secret1 is the password here).
* It is sometimes very hard to quote correctly unusual characters, * It is sometimes very hard to quote correctly unusual characters,
especially on Windows (and Mac sometimes). See especially on Windows (and Mac sometimes). See

21
FAQ.d/FAQ.Contacts_Calendars.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Contacts_Calendars.txt,v 1.14 2020/11/02 16:04:09 gilles Exp gilles $ $Id: FAQ.Contacts_Calendars.txt,v 1.15 2021/09/06 14:32:19 gilles Exp gilles $
This documentation is also available online at This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -11,21 +11,20 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Contacts_Calendars.txt
Questions answered in this FAQ are: Questions answered in this FAQ are:
Q. Can I copy or sync Calendars or Contacts with imapsync? Q. Can I copy or sync Contacts, Calendars, Tasks or Chat messages with imapsync?
Q. Can I copy or sync Chat messages with imapsync?
Q. How can I avoid copying Calendar or Contacts folders?
Q. How can I copy or synchronize Calendars or Contacts? Q. How can I copy or synchronize Calendars or Contacts?
Q. How can I avoid copying Calendar or Contacts folders?
Now the questions again with their answers. Now the questions again with their answers.
======================================================================= =======================================================================
Q. Can I copy or sync Calendars or Contacts with imapsync? Q. Can I copy or sync Contacts, Calendars, Tasks or Chat messages with imapsync?
R. No, Imapsync can't migrate Contacts and Calendars. R. No, Imapsync can't migrate Contacts, Calendars, Tasks nor Chat messages.
It's because most IMAP servers don't get contacts and calendars It's because most IMAP servers don't get contacts and calendars
events via IMAP. In other words, messages synced by imapsync from events via IMAP. In other words, messages synced by imapsync from
Calendars or Contacts folders are not used by email servers to set Contacts/Calendars/Tasks/Chat folders are not used by email servers to set
or get the contacts or calendars. or get the contacts, calendars, tasks or chat messages.
No way via IMAP, no way via imapsync. No way via IMAP, no way via imapsync.
So it's a good idea to avoid syncing contacts and calendars when So it's a good idea to avoid syncing contacts and calendars when
@ -35,12 +34,6 @@ R. No, Imapsync can't migrate Contacts and Calendars.
But see also the question below But see also the question below
"How can I copy or synchronize Calendars or Contacts?". "How can I copy or synchronize Calendars or Contacts?".
=======================================================================
Q. Can I copy or sync Chat messages with imapsync?
R. No, Imapsync can't migrate Chat messages (those in GSuite for example).
The explanation is the same as with Calendars or Contacts.
======================================================================= =======================================================================
Q. How can I copy or synchronize Calendars or Contacts? Q. How can I copy or synchronize Calendars or Contacts?

73
FAQ.d/FAQ.Docker.txt
View file

@ -1,13 +1,13 @@
#!/bin/cat #!/bin/cat
# $Id: FAQ.Docker.txt,v 1.8 2021/05/04 16:57:09 gilles Exp gilles $ # $Id: FAQ.Docker.txt,v 1.10 2021/07/12 21:22:14 gilles Exp gilles $
This document is also available online at This document is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Docker.txt https://imapsync.lamiral.info/FAQ.d/FAQ.Docker.txt
=========================================================================== ======================================================================
Installing and using imapsync docker image Installing and using imapsync docker image
=========================================================================== ======================================================================
Questions answered in this FAQ are: Questions answered in this FAQ are:
@ -23,27 +23,29 @@ Q. Where is the imapsync Docker hub location?
Q. How can I mask the passwords on the command line without mounting Q. How can I mask the passwords on the command line without mounting
inside the container? inside the container?
===========================================================================
Q. Why imapsync isn't logging to a file in Docker context. Q. Why imapsync isn't logging to a file in Docker context.
How can I allow it? How can I allow it?
Q. Any tips for the Docker Mailcow distribution? Q. Any tips for the Docker Mailcow distribution?
Q. How can I build my own image?
Q. What do you do to update the docker hub image?
Now the questions again with their answers. Now the questions again with their answers.
=========================================================================== ======================================================================
Q. How can I install and use the imapsync Docker image on my amd64 system? Q. How can I install and use the imapsync Docker image on my amd64 system?
R. Install Docker on your system. Once Docker is installed on your system, R. Install Docker on your system. Once Docker is installed on your system,
all you have to do in order to run imapsync is the command line: all you have to do in order install and run imapsync is the command line:
docker run gilleslamiral/imapsync imapsync <usual imapsync arguments> docker run gilleslamiral/imapsync imapsync <usual imapsync arguments>
=========================================================================== ======================================================================
Q. How can I install and use the imapsync Docker image on my i386 system? Q. How can I install and use the imapsync Docker image on my i386 system?
R. You can't for now. It will throw this error when running it: R1. You can't do it directly for now. It will throw this error when running it:
docker run gilleslamiral/imapsync imapsync docker run gilleslamiral/imapsync imapsync
standard_init_linux.go:211: exec user process caused "exec format error" standard_init_linux.go:211: exec user process caused "exec format error"
@ -51,21 +53,24 @@ R. You can't for now. It will throw this error when running it:
I plan to make the image multi-plateform. Drop me a note if you are I plan to make the image multi-plateform. Drop me a note if you are
eager to have it. eager to have it.
=========================================================================== R2. Build the image on a i386 system with the exact same Dockerfile provided
and run it on your i386 system.
======================================================================
Q. How can I install or update only the imapsync Docker image on my system? Q. How can I install or update only the imapsync Docker image on my system?
R. To install or update the imapsync image, run: R. To install or update the imapsync image, run:
docker pull gilleslamiral/imapsync docker pull gilleslamiral/imapsync
=========================================================================== ======================================================================
Q. Where is the imapsync Docker hub location? Q. Where is the imapsync Docker hub location?
R. Here: R. Here:
https://hub.docker.com/r/gilleslamiral/imapsync/ https://hub.docker.com/r/gilleslamiral/imapsync/
=========================================================================== ======================================================================
Q. How can I mask the passwords on the command line without mounting Q. How can I mask the passwords on the command line without mounting
anything inside the container? anything inside the container?
@ -84,7 +89,7 @@ See:
https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file
https://stackoverflow.com/a/30494145/491175 https://stackoverflow.com/a/30494145/491175
=========================================================================== ======================================================================
Q. Why imapsync isn't logging to a file in Docker context. Q. Why imapsync isn't logging to a file in Docker context.
How can I allow it? How can I allow it?
@ -93,14 +98,14 @@ R. In Docker context, writing the log is disabled by default because you
inside the docker and no longer available after the sync, unless inside the docker and no longer available after the sync, unless
a special mount is done before. a special mount is done before.
Starting with imapsync release 2.113 the logging can be set on Starting with imapsync release 2.113 the logging can be turned on
by using the option --log. You can add access to the logfile with a mount by using the option --log. You can add access to the logfile with a mount
or similar. See https://docs.docker.com/storage/volumes/" or similar. See https://docs.docker.com/storage/volumes/"
See also See also
https://github.com/imapsync/imapsync/issues/283 https://github.com/imapsync/imapsync/issues/283
=========================================================================== ======================================================================
Q. Any tips for the Docker Mailcow distribution? Q. Any tips for the Docker Mailcow distribution?
R. With the Mailcow distribution, imapsync is running inside a Docker R. With the Mailcow distribution, imapsync is running inside a Docker
@ -116,6 +121,40 @@ See the original Mathilde's comment:
https://github.com/imapsync/imapsync/issues/201#issuecomment-559500077 https://github.com/imapsync/imapsync/issues/201#issuecomment-559500077
======================================================================
Q. How can I build my own image?
=========================================================================== R. Use the following command to build the image:
===========================================================================
docker build -t gilleslamiral/imapsync .
with the Dockerfile in the current directory.
The latest Dockerfile can be found at
https://imapsync.lamiral.info/INSTALL.d/Dockerfile
or
https://hub.docker.com/r/gilleslamiral/imapsync
or
https://imapsync.lamiral.info/INSTALL.d/INSTALL.Docker_build.txt
======================================================================
Q. What do you do to update the docker hub image?
R. I do in my imapsync directory:
make docker_build
make docker_upload_docker_hub
Then I go to
https://hub.docker.com/r/gilleslamiral/imapsync
I edit and update the docker "Readme" with the last Dockerfile I used.
I also update INSTALL.Docker_build.txt with the last Dockerfile
I also update the last Dockerfile on the imapsync site with:
make upload_index
======================================================================
======================================================================

9
FAQ.d/FAQ.Dovecot.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Dovecot.txt,v 1.12 2021/04/30 13:02:06 gilles Exp gilles $ $Id: FAQ.Dovecot.txt,v 1.13 2021/12/03 12:51:23 gilles Exp gilles $
This documentation is also available online at This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -13,6 +13,8 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Dovecot.txt
Questions answered in this FAQ are: Questions answered in this FAQ are:
Q. Is there imapsync specific options to deal with Dovecot?
Q. Subfolders are not created by imapsync and the error is Q. Subfolders are not created by imapsync and the error is
"Mailbox doesn't allow inferior mailboxes" "Mailbox doesn't allow inferior mailboxes"
@ -28,6 +30,11 @@ Q. Imapsync encounters timeouts and Dovecot log says
Now the questions again with their answers. Now the questions again with their answers.
=======================================================================
Q. Is there imapsync specific options to deal with Dovecot?
R. Dovecot is ok with imapsync, nothing special is needed, just the
basic option, the credentials, are needed.
======================================================================= =======================================================================
Q. How to migrate to Dovecot with an admin/MasterUser account? Q. How to migrate to Dovecot with an admin/MasterUser account?

16
FAQ.d/FAQ.Duplicates.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Duplicates.txt,v 1.24 2021/07/04 22:59:56 gilles Exp gilles $ $Id: FAQ.Duplicates.txt,v 1.25 2021/10/06 20:21:13 gilles Exp gilles $
This documentation is also available online at This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -12,6 +12,10 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
Questions answered in this FAQ are: Questions answered in this FAQ are:
Q. Without imapsync, I made several copies that partially failed and it
ended with many duplicates/triplicates messages or more. Can I clean
up the account with imapsync and how?
Q. How does imapsync identify messages and duplicates? Q. How does imapsync identify messages and duplicates?
Q. How can I know if imapsync will generate duplicates on a second run? Q. How can I know if imapsync will generate duplicates on a second run?
@ -30,6 +34,14 @@ Q. How can I remove duplicates in a lone account?
Now the questions again with their answers. Now the questions again with their answers.
=======================================================================
Q. Without imapsync, I made several copies that partially failed and it
ended with many duplicates/triplicates messages or more. Can I clean
up the account with imapsync and how?
R. Yes.
See the Q/R "How can I remove duplicates in a lone account?" below.
======================================================================= =======================================================================
Q. How does imapsync identify messages and duplicates? Q. How does imapsync identify messages and duplicates?
@ -261,5 +273,7 @@ R. In order to remove duplicates in a lone account, just run imapsync
option --delete2duplicates, ie, with option --delete2duplicates, ie, with
host1 == host2, user1 == user2, password1 == password2 host1 == host2, user1 == user2, password1 == password2
imapsync ... --delete2duplicates
======================================================================= =======================================================================
======================================================================= =======================================================================

46
FAQ.d/FAQ.Emptying.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Emptying.txt,v 1.11 2020/10/13 16:24:43 gilles Exp gilles $ $Id: FAQ.Emptying.txt,v 1.12 2021/09/09 11:53:53 gilles Exp gilles $
This documentation is also available online at This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -13,10 +13,18 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Emptying.txt
Questions answered in this FAQ are: Questions answered in this FAQ are:
Q. How to delete all emails of all folders of an account with imapsync? Q. How to delete all emails of all folders of an account with imapsync?
Q. How to delete all folders of an account with imapsync? Q. How to delete all folders of an account with imapsync?
Q. How to delete some folders of an account with imapsync? Q. How to delete some folders of an account with imapsync?
Q. How to revert a sync that went to the wrong account? Q. How to revert a sync that went to the wrong account?
Q. How to delete emails older than 90 days with imapsync?
Q. How to delete emails matching a certain criterium with imapsync?
Now the questions again with their answers. Now the questions again with their answers.
======================================================================= =======================================================================
@ -27,11 +35,11 @@ R. Use the same account on both sides with option --delete1
Option --noexpungeaftereach is just to speed up the deletions. Option --noexpungeaftereach is just to speed up the deletions.
Example (real example): Example:
imapsync \ imapsync \
--host1 test.lamiral.info --user1 empty --password1 secret \ --host1 test.lamiral.info --user1 foo --password1 secret \
--host2 test.lamiral.info --user2 empty --password2 secret \ --host2 test.lamiral.info --user2 foo --password2 secret \
--delete1 --noexpungeaftereach --delete1 --noexpungeaftereach
======================================================================= =======================================================================
@ -90,5 +98,35 @@ R. If all the wrong emails have some common characteristic that is not
If the wrongly emails messages synced have no special common If the wrongly emails messages synced have no special common
characteristic then take an handkerchief and cry. characteristic then take an handkerchief and cry.
=======================================================================
Q. How to delete emails older than 90 days with imapsync?
R. Use the same account on both sides with option --delete1
--noexpungeaftereach --minage 90
Example:
imapsync \
--host1 test.lamiral.info --user1 foo --password1 secret \
--host2 test.lamiral.info --user2 foo --password2 secret \
--delete1 --noexpungeaftereach --minage 90
=======================================================================
Q. How to delete emails matching a certain criterium with imapsync?
R. Use the --search option to select the messages you want to delete
and then use the same account on both sides with the suplementary
options --delete1 --noexpungeaftereach
To select the messages read the document FAQ.Messages_Selection.txt
at https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
Example:
imapsync \
--host1 test.lamiral.info --user1 foo --password1 secret \
--host2 test.lamiral.info --user2 foo --password2 secret \
--delete1 --noexpungeaftereach --search "SENTBEFORE 31-Dec-2010"
======================================================================= =======================================================================
======================================================================= =======================================================================

42
FAQ.d/FAQ.Exchange.txt
View file

@ -1,5 +1,5 @@
$Id: FAQ.Exchange.txt,v 1.75 2021/05/25 10:23:33 gilles Exp gilles $ $Id: FAQ.Exchange.txt,v 1.79 2021/12/17 11:59:48 gilles Exp gilles $
This documentation is also available online at This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -31,6 +31,9 @@ Q. Exchange fails with "User is authenticated but not connected".
Q. Exchange fails with "BAD Command received in Invalid state". Q. Exchange fails with "BAD Command received in Invalid state".
Q. Exchange fails with "NO Server Unavailable. 15" and imapsync says
"could not append ... NO Server Unavailable. 15"
Q. From XXX to Exchange 2013, read receipts are all resent again Q. From XXX to Exchange 2013, read receipts are all resent again
after a sync. Even for old messages. How can I fix that? after a sync. Even for old messages. How can I fix that?
@ -201,6 +204,14 @@ R2. Miguel Alameda reported understanding and solving this issue
like this, the context was admin/authuser: like this, the context was admin/authuser:
"The admin user had not permission in the target mailbox." "The admin user had not permission in the target mailbox."
R3. Jason from magiclamp.net adds this tip (2021_11_23).
"In GoDaddy Exchange admin center, under recipients, you can
double click on a mailbox, and in the "mailbox delegation" scrolling
down to the bottom it shows a input field named "Full Access".
Add the admin account there."
====================================================================== ======================================================================
Q. I see "NO Maximum size of appendable message has been exceeded" Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that? What can I do with that?
@ -298,6 +309,35 @@ R2. This message might also happen when authenticating with ssl and tls.
It won't work, you can't do a direct double encryption, few It won't work, you can't do a direct double encryption, few
servers allow that. servers allow that.
======================================================================
Q. Exchange fails with "NO Server Unavailable. 15" and imapsync says
"could not append ... NO Server Unavailable. 15"
R1. The problem could be a mailbox quota limitation.
In the Exchange log file, if you see many
"Microsoft.Exchange.Data.Storage.QuotaExceededException",
it looks like "Quota Exceeded" could be the issue.
The solution is to increase the quota for this mailbox.
Quoting
https://docs.microsoft.com/en-us/exchange/recipients/user-mailboxes/storage-quotas?view=exchserver-2019
In the Exchange admin center (EAC), navigate to Recipients > Mailboxes.
In the list of user mailboxes, click the mailbox that you want to change the storage quotas for,
and then click Edit icon, then change "Prohibit send and receive at (GB)", then save your changes.
Or use the Exchange Management Shell, aka PowerShell:
Set-Mailbox -Identity "Joe Healy" -IssueWarningQuota 499gb -ProhibitSendQuota 500gb -ProhibitSendReceiveQuota 500gb -UseDatabaseQuotaDefaults $false
R2. A user fixed it by enabling IMAP logging on Exchange which is
quite crazy as a solution. But why not, here it is:
https://www.mantisbt.org/forums/viewtopic.php?p=68835&sid=8b1ecd2d976e85218fbf33fcd38c209d#p68835
by TerranceM
Set-ImapSettings -Server <server-name> -ProtocolLogEnabled $true
====================================================================== ======================================================================
Q. From XXX to Exchange 2013, read receipts are all resent again Q. From XXX to Exchange 2013, read receipts are all resent again
after a sync. Even for old messages. How can I fix that? after a sync. Even for old messages. How can I fix that?

80
FAQ.d/FAQ.Fun_Things.txt Normal file
View file

@ -0,0 +1,80 @@
#!/bin/cat
$Id: FAQ.Fun_Things.txt,v 1.2 2021/10/15 13:37:05 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Fun_Things.txt
=======================================================================
Imapsync fun things to do with
=======================================================================
Questions answered in this FAQ are:
Q. How can I have a daily digest of my mailbox?
Now the questions again with their answers.
=======================================================================
Q. How can I have a daily digest of my mailbox?
R. Use the same account as source and destination (not mandatory)
imapsync \
--host1 imap.example.com --user1 foo --password1 secret \
--host2 imap.example.com --user2 foo --password2 secret \
--folder INBOX --maxage 1 --dry --nodry1 --debugcontent \
--truncmess 1000 --f1f2 INBOX=Unused
Some explanations on the options suggested:
--folder INBOX to only focus on the folder INBOX
--maxage 1 to select messages less old than 1 day
--dry --nodry1 to avoid really syncing the messages
--debugcontent to display the content
--truncmess 1000 to display only the first 1000 bytes
--f1f2 INBOX=Unused to force imapsync to do something.
Since it is the same mailbox as source and destination,
imapsync usually doesn't sync the messages as they are
already at the destination. The folder Unused has to
exist, otherwise imapsync won't display the messages.
If you pipe this command with a filter like egrep, it becomes like
magic:
imapsync \
--host1 imap.example.com --user1 foo --password1 secret \
--host2 imap.example.com --user2 foo --password2 secret \
--folder INBOX --maxage 1 --dry --nodry1 --debugcontent \
--truncmess 1000 --f1f2 INBOX=Unused \
| egrep 'From:|To:|Subject:|Date:|=====|msg '
Output example:
$RCSfile: FAQ.Fun_Things.txt,v $ $Revision: 1.2 $ $Date: 2021/10/15 13:37:05 $
msg INBOX/22359 copying to INBOX.Fake (not really since --dry mode) ETA: vendredi 15 octobre 2021-10-15 15:31:38 +0200 CEST 57 s 38/38 msgs left
================================================================================
Subject: test: lei7eeCh4Fulahei
To: <tata@localhost.info>
Date: Mon, 11 Oct 2021 11:56:05 +0200 (CEST)
From: gilles@localhost.info (Gilles LAMIRAL)
================================================================================
msg INBOX/22360 copying to INBOX.Fake (not really since --dry mode) ETA: vendredi 15 octobre 2021-10-15 15:31:38 +0200 CEST 57 s 37/38 msgs left
================================================================================
Subject: test: yahShee3Ohvu9mai
To: <tata@localhost.info>
Date: Mon, 11 Oct 2021 11:56:05 +0200 (CEST)
From: gilles@localhost.info (Gilles LAMIRAL)
================================================================================
Thanks to John Fedewa for this trick!
=======================================================================
=======================================================================

5
FAQ.d/FAQ.Gmail.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Gmail.txt,v 1.80 2021/07/06 02:30:11 gilles Exp gilles $ $Id: FAQ.Gmail.txt,v 1.81 2021/10/06 20:21:26 gilles Exp gilles $
This document is also available online at This document is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -236,7 +236,7 @@ imapsync \
--password2 gmailsecret2 \ --password2 gmailsecret2 \
--gmail1 --gmail2 --gmail1 --gmail2
Or, replacing what does --gmail1 --gmail2 (values are from imapsync 1.970): Or, replacing what does --gmail1 --gmail2 (values are from imapsync 2.155):
imapsync \ imapsync \
--host1 imap.gmail.com \ --host1 imap.gmail.com \
@ -251,6 +251,7 @@ imapsync \
--maxbytesafter 1_000_000_000 \ --maxbytesafter 1_000_000_000 \
--automap \ --automap \
--maxsleep 2 \ --maxsleep 2 \
--useheader X-Gmail-Received --useheader Message-Id \
--synclabels \ --synclabels \
--resynclabels \ --resynclabels \
--exclude "\[Gmail\]$" \ --exclude "\[Gmail\]$" \

206
FAQ.d/FAQ.ISP.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.ISP.txt,v 1.9 2018/05/24 11:34:30 gilles Exp gilles $ $Id: FAQ.ISP.txt,v 1.13 2021/12/06 20:02:39 gilles Exp gilles $
This document is also available online at This document is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -9,36 +9,24 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.ISP.txt
Imapsync tips for ISP. Specific issues and solutions. Imapsync tips for ISP. Specific issues and solutions.
======================================================================= =======================================================================
* IMAPSync - usage scenario with ISP - by Flávio Zarur Lucarelli. * IMAPSync - usage scenario with ISP - by Flavio Zarur Lucarelli (https://www.lucanet.com.br)
I thought Id write a quick step by step on my attempts to learn the I thought I had to write a quick step by step on my attempts to learn
imapsync features that matter the most, so it works as we expected in the imapsync features that matter the most, so it works as we expected
the cenario in which we use it, which is to migrate customers from in the scenario in which we use it, which is to migrate customers from
their old ISP to our ISP/email hosting. Thanks to the master Gilles their old ISP to our ISP/email hosting. Thanks to the master Gilles
Lamiral for all his help and hard work. Lamiral for all his help and hard work.
First of all, remember to use --dry to test things first always and First of all, remember to use --dry to test things first always and
check the log file to see what would actually happen. check the log file to see what would actually happen.
Initially, I used a method where I'd do an exact sync of source to This type of copy keeps destination exactly like source (except, it doesn't
destination, deleting messages which were in destination, but not copy duplicates), so use it ONLY before changing your MX to the new server,
source. I call this "Method 2", below. In this cenario, customer as it will delete whatever is in destination which isn't in source.
shouldn't be using the destination account yet. Then, after I switch MX,
I'd do a final sync based on date. The big advantage of this is, you
get an exact sync.
Easier, however, is method 1, where I sync based on message ID, this
way, I can use the same syntax before and after MX change, with no
worries. Only disadvantage, you might not get an exact sync, there
might be some difference in terms of total emails in folders, due to
duplicates, emails that had same message ID in source server.
* Method 1 - sync based on message ID, can use same syntax before and
after MX change
imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \ imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd \ --host2 imap.myisp.com --user2 user@domain.com --password2 pwd \
--addheader --subscribeall --delete2 --delete2folders --addheader
Note: add header adds message ID when it doesn't exist. Note: add header adds message ID when it doesn't exist.
@ -46,57 +34,141 @@ This syntax can also be used to sync different source accounts to one
same destination account, simply execute it as many times as desired, same destination account, simply execute it as many times as desired,
switching source user (user1). switching source user (user1).
* Method 2 - exact sync source do destination, then sync based on date If you have a list of accounts to sync, use this:
after MX change https://imapsync.lamiral.info/examples/sync_loop_unix.sh
My first goal is to have an exact sync of an account from Adjust the parameters according to the original syntax which I
current/source host to the new/destination host and be able to sync explained above.
several times. The --useuid parameter is very important for that
purpose. This is what I use:
imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \ The Sync loop Unix script now outputs an error log, which can be
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \ useful.
--useuid --delete2 --delete2folders
This makes it so imap.myisp.com (destination) is an exact copy of the We also have shell scripts to check for errors, such as:
account at imap.gmail.com (source). This is not a problem, since the
user is not using the new host yet. ]You can check Imapsync log files
and surely you will see the final difference should be 0. Check also
for any possible errors in the log (search for "error").
The second goal is to lower the TTL (ex: 5 min) for the host - Accounts over quota
associated with the MX record, in the domain's DNS server. Let's say
customer has a host mail which his MX points to, with a high TTL grep -R "EXIT_OVERQUOTA" *.txt
(usually 1 hour). Lower it to 5 min so that, when you change the MX,
it propagates faster. - Login error
grep -R "Error login" *.txt
- Check just critical errors
egrep 'EXIT_AUTHENTICATION_FAILURE|EXIT_CONNECTION_FAILURE|EXIT_TLS_FAILURE|EXIT_OVERQUOTA' *.txt
- Check all sorts of errors (can be too much info)
grep 'Exiting with return value' *.txt | grep -v 'Exiting with return value 0'
Also, I like to check log file sizes and compare account sizes on
source and destination.
--
Back to it.
After you do a full sync, check that everything is consistant and
let's proceed to the final stage.
Lower the TTL (ex: 5 min) for the host associated with the MX record,
in the domain's DNS server. Let's say customer has a host mail which
his MX points to, with a high TTL (usually 1 hour). Lower it to 5 min
so that, when you change the MX, it propagates faster.
When comes time to switch over to the new host, do a final sync with When comes time to switch over to the new host, do a final sync with
above syntax, before changing the MX. Then, change the MX and tell above syntax, before changing the MX. Then, change the MX and tell
your users to start using exclusively the new host. your users to start using exclusively the new host.
A few hours after the MX change, we will run Imapsync again. We have A few hours after the MX change, we will run Imapsync again. We have
to start preserving emails users move or flag in the new host, which to start preserving emails users move or flags in the new host, which
they started using, so we can't do an exact sync anymore. they started using, so we can't do an exact sync anymore.
The best solution for me was to Sync any new emails (maxage:1) from One simple solution is the same syntax as before, but without
source (that could arrive in source even after MX change, due to --delete2 and --delete2folders.
cache) and delete such emails from source server. This way, customer's
mailbox is still intact on the source server, except new emails, which
get synced to new server and deleted from source.
imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \ So:
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \
--folder INBOX --useuid --noexpungeaftereach --skipemptyfolders --maxage 1 --delete1
I personally prefer to keep a copy of users box intact in source, but imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \
if that's not an issue for you, you can remove --folder INBOX and even --host2 imap.myisp.com --user2 user@domain.com --password2 pwd \
--maxage, but then, all emails in source will be deleted. You can use --subscribeall --addheader
--maxage 1 with --delete1, however, for all folder (without specifying
--folder INBOX), so only any new email that arrives at source is
copied to destination and deleted from source.
My next goal was to automate the process, so I followed this advice: The only issue with that syntax is, it would simply copy what is in
http://imapsync.lamiral.info/examples/sync_loop_unix.sh source that is not in destination. For example, user might have moved
emails to some sub-folder in host2. However, if you do this once, one
hour after MX change, for example, it's comprehensible that
destination is not exactly as the source anymore.
What we've been doing, especially in the case of huge migrations,
since our destination server auto-archives emails in sub-folders and
changes the hierarchy, is syncing just the last day or few days of
email and just of INBOX.
So we do the sync initially explained in this doc, before the MX
change. Then we auto-archive accounts and after auto-archiving, we
change MX and do a final sync using maxage.
Something like this:
imapsync --host1 imap.myisp.com --user1 user@domain.com --password1 pwd \
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd \
--subscribeall --addheader --maxage 1
If your last full sync was 2 days ago, then use --maxage 2
Goal is to get all the new emails (2 last days) from source.
Another alternative for syncing after the MX change, which Lamiral
suggests, involves deleting emails from source from the last day.
I prefer the method explained above. Anyway, here is this other cenario...
Something like this:
imapsync --host1 imap.gmail.com --user1 user@domain.com --password1 pwd --ssl1 \
--host2 imap.myisp.com --user2 user@domain.com --password2 pwd --ssl2 \
--folder INBOX --noexpungeaftereach --skipemptyfolders --maxage 1 --delete1
This way, any new email that arrived at source is copied to
destination and deleted from source. Without the --delete1 option,
messages moved on host2 by the user are coming back from host1, and
flags changed on host2 are also reset to what they are on host1.
However, as I wrote before, if you run it just once without --delete1,
perhaps not that bad.
*** Other tips for ISPs
1) Sync entire account into 1 folder of another account
imapsync --host1 xxx --user1 user1@domain.com --password1 secret1 --ssl1 \
--host2 yyy --user2 user2@domain.com --password2 secret2 --ssl2 \
--addheader --subfolder2 "otheraccountfolder" --delete2 --delete2foldersonly /otheraccountfolder/
Above, user must not be using such destination folder in destination
account until you finish syncing.
I've used it without --delete2 and --delete2foldersonly, as it caused
problems for us, of that folder being unsubscribed in destination...
Don't think it should be a problem as this copy is done usually
without using really using the folder in host2 until you instruct him
to.
Another nice scenario, sync just one folder hierarchy of an account to
another folder of another account.
--subfolder1 test --subfolder2 test2 --delete2 --delete2foldersonly /test2/ --addheader
Also, very useful is to be move emails older than one year, from an
account to another one, to offload the source account.
imapsync ... --subfolder2 "account_user1" --addheader --minage 365 --delete1
So everything older than 365 days is deleted from source. You can even
specify same account for source and recipient, if you want to "delete
older than x days" for the account itself, without moving old emails
to another account.
2) Translate folder names
I also ended up requiring a regex to translate folder names. On the I also ended up requiring a regex to translate folder names. On the
old server (Gmail), Sent items were in a folder called [Gmail]/E-mails old server (Gmail), Sent items were in a folder called [Gmail]/E-mails
@ -110,17 +182,21 @@ So this was added:
--regextrans2 "s,Lixeira,Trash," \ --regextrans2 "s,Lixeira,Trash," \
--regextrans2 "s,Rascunhos,Drafts," --regextrans2 "s,Rascunhos,Drafts,"
*** Other cenarios Option --automap is also good to do most of the translate job.
- Sync entire account into 1 folder of another account 3) Folders to exclude, to speed up syncs, etc.
imapsync --host1 xxx --user1 user1@domain.com --password1 secret1 --ssl1 \ --exclude "^Backups" --exclude "(?i)spam" --exclude "(?i)junk" --exclude "(?i)trash" --exclude "(?i)lixeira" --exclude "(?i)Deleted Items"
--host2 yyy --user2 user2@domain.com --password2 secret2 --ssl2 \
--useuid --subfolder2 otheraccountfolder --delete2 --delete2foldersonly /otheraccountfolder/
Above is based on message UID, advantage of no dupes, however, user 4) Delete older than 1 year from an account.
must not be using such destination folder in destination account until
you finish syncing. Use the account itself as source and destination and use:
--minage 365 --delete1
5) Check size of folders in an account. Use Imapsync with same account as source and destination, with this parameter:
--justfoldersizes
======================================================================= =======================================================================
======================================================================= =======================================================================

41
FAQ.d/FAQ.Logfile.txt Normal file
View file

@ -0,0 +1,41 @@
$Id: FAQ.Logfile.txt,v 1.2 2021/07/23 12:27:31 gilles Exp gilles $
This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Logfile.txt
======================================================================
Imapsync log file
======================================================================
Questions answered in this FAQ are:
Q. I have these error "could not be fetched", like thos one:
Err 1/1: - msg INBOX/17219 {0} S[36592] F[] I[16-Jul-2021 12:04:15 +0000] could not be fetched
What can I do about it?
Now the questions again with their answers.
======================================================================
Q. I have these error "could not be fetched", like thos one:
Err 1/1: - msg INBOX/17219 {0} S[36592] F[] I[16-Jul-2021 12:04:15 +0000] could not be fetched
What can I do about it?
R. Explanation of a log line:
- msg INBOX/17219 {0} S[36592] F[] I[16-Jul-2021 12:04:15 +0000]
INBOX = folder
17219 = UID
{0} = got 0 bytes
S[36592] = should have got 36592 bytes (RFC size announced)
F[] = Flags, none here
I[16-Jul-2021 12:04:15 +0000] = Internal date
Usually this error doesn't appear on next runs, at least not for the
same messages. So rerun imapsync until all messages are synced.
======================================================================
======================================================================

8
FAQ.d/FAQ.Massive.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Massive.txt,v 1.31 2020/10/15 01:53:52 gilles Exp gilles $ $Id: FAQ.Massive.txt,v 1.32 2021/11/01 11:30:48 gilles Exp gilles $
This document is also available online at This document is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -56,9 +56,9 @@ R2. To estimate the presync period, consider the mean imapsync transfer
rate to be around 340 Kbytes/s, ie, 2.8 Mbps, no matter the local rate to be around 340 Kbytes/s, ie, 2.8 Mbps, no matter the local
link bandwidth. link bandwidth.
It's a mean, measured upon various different syncs, coming from It's a mean, measured upon various different syncs, coming from
the online service /X where the network card flow rate is 100 Mbps the online service /X where the network card flow rate is 200 Mbps
and the provider bandwidth is also 100 Mbps. Max seen is a transfer (Rx 100 Mps, Tx 100 Mps) and the provider bandwidth is also 200 Mbps.
at 21 Mbps. The maximum global rate seen is 22 MiB/s (176 Mbps).
At 340 Kbytes/s, At 340 Kbytes/s,
1 TB to transfer and one sync at a time will end in 35 days (1024^3/340/3600/24). 1 TB to transfer and one sync at a time will end in 35 days (1024^3/340/3600/24).

142
FAQ.d/FAQ.Messages_Too_Big.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Messages_Too_Big.txt,v 1.5 2021/06/22 15:31:39 gilles Exp gilles $ $Id: FAQ.Messages_Too_Big.txt,v 1.8 2021/12/16 11:44:26 gilles Exp gilles $
This document is also available online at This document is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -11,76 +11,128 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Too_Big.txt
Questions answered in this FAQ are: Questions answered in this FAQ are:
Q. What does imapsync with big messages? Q. Imapsync reports errors like these ones:
"could not append...Maximum size of appendable message has been exceeded"
"could not append...maximum message size exceeded"
"The most frequent error is ERR_APPEND_SIZE"
What does it mean?
Q. Does imapsync limits big messages?
Q. What are the well known servers messages size limits?
Q. What can I do to transfer messages bigger than what allows the Q. What can I do to transfer messages bigger than what allows the
imap destination server? imap destination server?
Q. The solution --truncmess xxxx doesn't work with Gmail.
What can I do?
Now the questions again with their answers. Now the questions again with their answers.
======================================================================= =======================================================================
Q. What does imapsync with big messages? Q. Imapsync reports errors like these ones:
"could not append...Maximum size of appendable message has been exceeded"
"could not append...maximum message size exceeded"
"The most frequent error is ERR_APPEND_SIZE"
What does it mean?
R. By default, imapsync checks if the host2 imap server announces a R. It means the message couldn't be transferred to the destination
message size limit with "APPENDLIMIT=xxxx" in its CAPABILITY response. account because it is too big, imapsync tries to APPEND it to the
Then imapsync takes this value to set automatically the --maxsize server (that's IMAP jargon) but the server refuses it by saying
option, unless --maxsize is already set on the command line and has a either "NO. Maximum size of appendable message has been exceeded"
smaller value than the APPENDLIMIT value. or "NO. maximum message size exceeded".
If the host2 imap server doesn't announce any message size limit then
imapsync syncs all messages no matter their sizes. =======================================================================
Q. Does imapsync limits big messages?
R. No. Imapsync synchronizes messages no matter their sizes.
When a message is copied to the destination server, the destination When a message is copied to the destination server, the destination
server accepts it or not but it does it as a whole, it takes the full server accepts it or not. It does it as a whole, it takes the full
message or nothing. Nothing triggers an error in imapsync. message or nothing. The destination server can refuse a too big
message, it then triggers an APPEND error message in imapsync.
* Gmail announces APPENDLIMIT=35651584 (June 2021). So far, December 2021, the biggest message transferred by imapsync on
* Office365 announces nothing. the online /X service is 1.51 GiB. Who can top that?
* Dovecot announces nothing.
* Usually imap servers announce nothing.
=======================================================================
Q. What are the well known servers messages size limits?
* Gmail message size limit is 35 MB.
Gmail announces APPENDLIMIT=35651584 (June 2021).
* Office365 message size limit is 45 MB by default
and it can be upped by configuration. See
https://imapsync.lamiral.info/FAQ.d/FAQ.Office365.txt
Q. I see "NO Maximum size of appendable message has been exceeded"
* Exchange message size limit is 10 MB by default
and it can be upped by configuration. See
https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt
Q. I see "NO Maximum size of appendable message has been exceeded"
* Dovecot message size limit is variable. It is defined
by the sysadmin. For example, quota_max_mail_size = 100M
https://doc.dovecot.org/configuration_manual/quota/
* Usually, any IMAP server has a message size limit, defined
by the sysadmin. You discover it sooner or later
======================================================================= =======================================================================
Q. What can I do to transfer messages bigger than what allows the Q. What can I do to transfer messages bigger than what allows the
imap destination server? imap destination server?
R1. A solution is to truncate the message to the maximum size allowed R1. A solution is ask imapsync to truncate the message to the maximum
before copying it to host2. That's a little dirty but it's ok if you size allowed before copying it to host2 by using the option
prefer having a truncated message than no message at all. The --truncmess.
attachments after the truncation won't be available but everything
before should be ok. It may also depend on the software tool reading
the email message.
For example, Gmail currently limits messages to 35651584 bytes, That's a little dirty but it's ok if you prefer having a truncated
aka 34 MiB (June 2021), message than no message at all. The attachments after the truncation
won't be available but everything before should be ok. It may also
depend on the software tool reading the email message.
Starting with imapsync release 1.938 option --truncmess xxxx truncates
messages bigger than the given size xxxx, size given in bytes.
imapsync --truncmess 35651584 --appendlimit 1000000000000 To truncate messages to 10 MB, use:
The --appendlimit option is only needed imap servers that offer the imapsync ... --truncmess 10_000_000
APPENDLIMIT capability, for example, Gmail. Using it doesn't hurt
anyway.
Prior to imapsync release 1.938, instead of --truncmess:
On Linux, use:
imapsync ... --pipemess 'perl -0ne "print substr \$_,0,35651584" '
On windows, use (I'm not sure of, not tested yet):
imapsync ... --pipemess "perl -0ne 'print substr $_,0,35651584' "
R2. A not-so-dirty solution would be to transform the big messages, R2. A not-so-dirty solution would be to transform the big messages,
replace the attachments with links to the attachments stored elsewhere. replace the attachments with links to the attachments stored
Well, it's not done and it's too much complicated compared to a simpler elsewhere. Well, it's not done and it's too much complicated compared
solution which is to allow bigger messages on the destination account. to a simpler solution which is to allow bigger messages on the
destination account.
As time goes on, disk space is cheaper and cheaper. Don't make the As time goes on, disk space is cheaper and cheaper. Don't make the
new user account, the destination account, worst than his previous, new user account, the destination account, worst than his previous
buy disks. Monitor and rule your resources, not the users. account: buy disks.
Monitor and rule your hardware resources, serve your users,
not the contrary.
=======================================================================
Q. The solution --truncmess xxxx doesn't work with Gmail.
What can I do?
R. Gmail currently limits messages size up to 35651584 bytes, aka 34
MiB (June 2021) but imapsync knows it so it avoid to transfer these
big messages. Solution, use:
imapsync ... --truncmess 35_651_584 --appendlimit 1_000_000_000_000
Explanation. By default, imapsync checks if the host2 imap server
announces a message size limit with "APPENDLIMIT=xxxx" in its
CAPABILITY response, like Gmail does. Imapsync takes this APPENDLIMIT
value to set automatically the --maxsize option, unless --maxsize is
already set on the command line and has a smaller value than the
APPENDLIMIT value.
1_000_000_000_000 is one TB, I guess you don't have any message of
that size yet.
======================================================================= =======================================================================
======================================================================= =======================================================================

124
FAQ.d/FAQ.Migration_Plan.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Migration_Plan.txt,v 1.7 2021/03/02 22:31:36 gilles Exp gilles $ $Id: FAQ.Migration_Plan.txt,v 1.9 2021/12/19 10:00:32 gilles Exp gilles $
This document is also available online at This document is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -8,23 +8,23 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Migration_Plan.txt
===================================================================== =====================================================================
Imapsync. Suggestions for a good, low impact on users, Imapsync. Suggestions for a good, low impact on users,
well executed email migration plan. well-executed email migration plan.
===================================================================== =====================================================================
There is two main different scenarios depending on the response to the There are two main different scenarios. Choosing which one fits your
following question: context depends on the response to the following question:
Will the imap software tools used by the users use the exact same Will the imap software tools used by the users use the same
credentials triplet for both imap servers, the old server host1 and credentials triplet for both imap servers, the old server host1 and
the new server host2? the new server host2?
The credentials triplet is hostname/username/password. The credentials triplet is hostname/username/password.
If the answer is yes, ie, clients email tools use the exact same If the answer is yes, ie, clients' email tools use the same triplet
triplet credentials, then it is possible to perform a migration credentials, then it is possible to perform a migration without
without changing anything on the users side. This may be a very time changing anything on the users' side. This may be a very time-saving
saving option. But it's a rare condition so I'll describe this option. But it's a rare condition so I'll describe this scenario later
scenario later in this document. in this document.
===================================================================== =====================================================================
@ -40,104 +40,124 @@ Classical scenario, credentials triplets are different on both sides
users are already playing with the new mailboxes on host2, don't users are already playing with the new mailboxes on host2, don't
follow this scenario. follow this scenario.
* Presync all the mailboxes from the old server host1 to the new * Pre-synchronize all the mailboxes from the old server host1 to the
server host2. If the imap server names are going to change their IP new server host2. If an imap server name is going to change its IP
resolution then don't use those names, use names that will always address, then don't use this name, use a name that will always
match the same imap servers, or use their IP addresses. match the same imap servers, or use their IP addresses.
Presyncs can usefully be done with --delete2 in order to get an Pre-synchronizations can usefully be done with --delete2 to get an
exact sync. But never use the option --delete2 once users have exact synchronization. But never use the option --delete2 once the
started to play with their new account on host2, their play will be users have started to play with their new account on host2, their
lost on the next presync, or when the MX is changed, since INBOX play will be lost on the next synchronization. Don't use --delete2
will start to receive new messages that are not on host1. either when the MX is changed since INBOX will start to receive new
messages that are not on host1 and then removing them is not a good
idea.
* Decide a migration day/hour. * Decide a migration day/hour.
* Repeat the presyncs (with the --delete2 options) daily until the * Repeat the pre-synchronizations (with the --delete2 options) daily
migration hour. This repeated process will show how long should until the migration hour. This repeated process will show how long
take the last sync. should take the last synchronization.
* At the migration hour, cut access to the users to the old server * At the migration hour, cut access to the users to the old server
host1, if you can. Or tell them to not use it anymore. host1, if you can. Or tell them to not use it anymore.
* Do a last presync exactly like previous ones. * Do the last pre-synchronization exactly like the previous ones.
* Change the MX, the new messages should start to arrive in the new * Change the MX, the new messages should start to arrive in the new
imap server host2. imap server host2.
* Wait the TTL value, aka 5 minutes. Now, new messages should * Wait for the TTL value, aka 5 minutes. Now, new messages should
not arrive to the old server host1. not arrive at the old server host1.
* Tell the users that the old imap server host1 is down and no * Tell the users that the old imap server host1 is down and no
longer available. longer available.
* Do a postsync. A postsync is a sync with the following options * Do a post-synchronization. A post-synchronization is a run with the
--maxage 1 --delete1 --folder INBOX following options: --folder INBOX --delete1 --maxage 1
This postsync will move the last new messages arrived on host1 to This post-synchronization will copy the messages arrived in the
host2 during the TTL interval and delete them on host1. Do not use last day (--maxage 1) in the folder INBOX (--folder INBOX) on the
the option --delete2 in a postsync. source account, to the destination account. It will also delete
them on host1 (--delete1). It's --delete1, it's not --delete2.
Remember, do not use the option --delete2 in a post
synchronization, as users won't appreciate seeing their newly
arrived messages disappear because of you.
* Give access to new accounts to the users with their new credential * Give access to new accounts to the users with their new credential
triplet hostname/username/password. If the way to contact users is triplet hostname/username/password. If the way to contact users is
email then you should give this long before shutting down the old by email then you should give them the new credentials long before
server. shutting down the old server.
* Migration done. * Migration done.
* In case there are still messages arriving at the old imap server * In case there are still messages arriving at the old imap server
host1, you can perform more postsyncs, ie, syncs every day host1, you can perform more post-synchronizations, ie, runs every
with the options: day with the options: --maxage 1 --delete1 --folder INBOX
--maxage 1 --delete1 --folder INBOX
* Increase the TTL of the MX back to its previous value, usually
24 hours, 86400 seconds. You don't want all your email system
to break down completely when your DNS are not available
temporarily, keeping dns values in cache for a 24h is a savvy
practice.
===================================================================== =====================================================================
Lucky scenario, credentials triplets are the same on both sides Lucky scenario, credentials triplets are the same on both sides
===================================================================== =====================================================================
* Decrease the TTL of the MX, as well as the imap hostname resolution, * Decrease the TTL of the MX, as well as the imap hostname resolution,
to 5 minutes (or even less). See FAQ.TTL.txt to understand why. to 5 minutes (or even less). The document FAQ.TTL.txt explains why.
* Create the new mailboxes on the destination server host2. * Create the new mailboxes on the destination server host2.
* Presync all the mailboxes from the old host1 to the new server host2, * Pre-synchronize all the mailboxes from the old host1 to the new
using different names that the ones used by the imap software server host2, using different names than the ones used by the imap
clients (use their IP for example). software clients (use their IP for example). Presyncs have to be
Presyncs have to be done with --delete2 but never use --delete2 done with --delete2 but never use --delete2 once users have started
once users have started playing with their new account on host2. playing with their new account on host2.
* Decide a migration day/hour. * Decide a migration day/hour.
* repeat the presyncs (with the --delete2 options) daily until the * Repeat the pre-synchronizations (the runs with the --delete2
migration hour. This repeated process will show how long should options) daily until the migration hour. This repeated process will
take the last sync. show how long should take the last sync.
* At the migration hour, cut access to the users to the old server. * At the migration hour, cut access to the users to the old server.
You can do this by changing the imap host1 hostname to a non-imap You can do this by changing the imap host1 hostname to a non-imap
server for example, or by changing their password on host1. server for example, or by changing their password on host1.
* Do a last sync exactly like the presyncs, not using the imap hostname. * Do the last run exactly like the pre-synchronizations.
* Change also the MX resolution, the new messages should start * Change also the MX resolution, the new messages should start
to arrive in the new imap server very soon. to arrive in the new imap server very soon.
* Wait the TTL value, aka 5 minutes. Now, new messages should * Wait for the TTL value, aka 5 minutes. Now, new messages should
not arrive to the old server host1. not arrive at the old server host1.
* Do a postsync. A postsync is a sync with the following options * Do a post-synchronization. A post-synchronization is a run with the
--maxage 1 --delete1 --folder INBOX following options: --folder INBOX --delete1 --maxage 1
This postsync will move the last new messages arrived on host1 to This post-synchronization will copy the messages arrived in the
host2 during the TTL interval and delete them on host1. Do not use last day (--maxage 1) in the folder INBOX (--folder INBOX) on the
the option --delete2 in a postsync. source account, to the destination account. It will also delete
them on host1 (--delete1). It's --delete1, it's not --delete2.
Remember, do not use the option --delete2 in a post
synchronization, as users won't appreciate seeing their newly
arrived messages disappear because of you.
* Shutdown the old imap server. * Shut down the old imap server.
* Change the user imap hostname resolution from the old IP of host1 * Change the user imap hostname resolution from the old IP of host1
to the IP of the new imap server host2. to the IP of the new imap server host2.
* Migration done. * Migration done.
* Increase the TTL of the MX back to its previous value, usually
24 hours, 86400 seconds. You don't want all your email system
to break down completely when your DNS are not available
temporarily, keeping dns values in cache for a 24h is a savvy
practice.
======================================================================= =======================================================================
======================================================================= =======================================================================

228
FAQ.d/FAQ.Office365.txt
View file

@ -1,5 +1,5 @@
$Id: FAQ.Office365.txt,v 1.23 2021/07/06 02:30:01 gilles Exp gilles $ $Id: FAQ.Office365.txt,v 1.33 2022/01/05 11:17:21 gilles Exp gilles $
This documentation is also available online at This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -14,14 +14,24 @@ Questions answered in this FAQ are:
Q. Can I use imapsync to transfer from or to Office365 accounts? Q. Can I use imapsync to transfer from or to Office365 accounts?
Q. Does IMAPSync support OAUTH2 authentication in Office 365? Q. Can imapsync work for users that are administrators for
an Office 365 domain? (quick answer: no)
Q. Does imapsync support OAUTH2 authentication for Office365 accounts?
Q. How to sync from Office365 to XXX? Q. How to sync from Office365 to XXX?
Q. How to sync from XXX to Office365 Q. How to sync from XXX to Office365
Q. For Office365 I have double and triple-checked the username and Q. For Office365 I have double and triple-checked the username and
password spelling but I still get a "LOGIN failed". Any clue? password spelling but I still get a "LOGIN failed". Any clue?
Q. How can I access an Office365 shared mailbox?
Q. How can I use a shared account as a backup account for several
mailboxes and so avoid spending too many dollars in backup accounts?
Q. Office365 fails with "User is authenticated but not connected".
Q. I see "NO Maximum size of appendable message has been exceeded" Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that? What can I do with that?
@ -33,6 +43,8 @@ Q. Office365 throttles the sync and says:
"Request is throttled. Suggested Backoff Time: 299961 milliseconds". "Request is throttled. Suggested Backoff Time: 299961 milliseconds".
What can I do with that? What can I do with that?
Q. What are the receive and sending limits of Office365?
Q. The sync fails with many "Trying command when NOT connected!". Q. The sync fails with many "Trying command when NOT connected!".
What can I do? What can I do?
@ -68,20 +80,101 @@ Now the questions again with their answers.
====================================================================== ======================================================================
Q. Can I use imapsync to transfer from or to Office365 accounts? Q. Can I use imapsync to transfer from or to Office365 accounts?
R. Yes. But IMAP access to Office365 account is not always allowed by default R0. Yes. But IMAP access to Office365 account is not always allowed
so it has to be allowed in the server configuration part. by default so it has to be allowed in the server configuration part.
https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365 https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
R1. Imapsync doesn't support OAUTH2 authentication for Office365
accounts yet. That's not totally true, if you have an access token
to access your mailbox then you can authenticate with imapsync this
way:
imapsync ... --oauthaccesstoken1 tokenfile
where "tokenfile" is a file containing the access token.
R2. Enable double-step authentication and configure it but after use
an "app password" with imapsync. Details:
a) Go to
https://account.microsoft.com/security
b) Click on "Advanced Security Options"
Turn "Two-step verification" on. Follow the steps and finish".
c) Then now "App passwords" is available.
Click on "Create a new app password".
Use this password to authenticate with imapsync.
d) Delete this app password when the job with imapsync is finished.
R3. Also, check a license is assigned to that account in Office365.
R4. Microsoft introduced something called "security defaults" which is enabled
by default for new tenants. One of the rules blocks IMAP access as of imapsync.
The funny thing is that you can't disable a single rule of this
security package without buying additional licenses.
Switching the whole thing off allows the IMAP login.
Also, disable double-step authentication on the Azure/Active Directory portal.
See here:
https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults
Thanks to Stephan Buhre for this R4 answer.
R5. Are there special characters in the password?
https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
R6. Triple check the hostname then. Try all of these:
* outlook.office365.com
* imap-mail.outlook.com
* imap.outlook.com
======================================================================
Q. Can imapsync work for users that are administrators for
an Office 365 domain? (quick answer: no)
R. I doesn't seem possible to use imap for administrators, so imapsync
won't be able to work for an administrator's mailbox.
See:
https://github.com/imapsync/imapsync/issues/310#issuecomment-1002396218
https://exhaust.lewiscollard.com/post/146866104/office365-to-migadu-migration/
Solution: as explained in the article above, use Davmail as a proxy to
access this mailbox. http://davmail.sourceforge.net/
Thanks to Lewis Collard for this report and solution.
====================================================================== ======================================================================
Q. Does imapsync support OAUTH2 authentication for Office365 accounts? Q. Does imapsync support OAUTH2 authentication for Office365 accounts?
R. No. But I plan to add it during the year 2021. R. Yes but partially. Imapsync won't help you getting an access token
but if you have one then you can use it with imapsync this way:
Office365 as source:
imapsync ... --oauthaccesstoken1 tokenfile1
Office365 as destination:
imapsync ... --oauthaccesstoken2 tokenfile2
====================================================================== ======================================================================
Q. How to sync from Office365 to XXX? Q. How to sync from Office365 to XXX?
R0. IMAP is not enabled by default on Office365, see how to enable it:
https://docs.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/enable-or-disable-pop3-or-imap4-access
https://docs.microsoft.com/en-us/exchange/troubleshoot/configure-mailboxes/pop3-imap-owa-activesync-office-365
If IMAP cannot be enabled or if it doesn't work well, you can try DavMail
(http://davmail.sourceforge.net/) which acts as an IMAP gateway
through Outlook Web Access.
R. On Windows, use: R. On Windows, use:
imapsync.exe ... --office1 imapsync.exe ... --office1
@ -142,41 +235,64 @@ It often does some miracles on messages.
====================================================================== ======================================================================
Q. For Office365 I have double and triple-checked the username and Q. For Office365 I have double and triple-checked the username and
password spelling but I still get a "LOGIN failed". Any clue? password spelling but I still get a "LOGIN failed". Any clue?
R0. Imapsync doesn't support OAUTH2 authentication for Office365 accounts yet. R. Go to "Q. Can I use imapsync to transfer from or to Office365 accounts?"
R1. Disable double-step authentication, also known as 2-factor, ======================================================================
2-step authentication on the Azure/Active Directory portal. Q. How can I access an Office365 shared mailbox?
R2. Also, check a license is assigned to that account in Office365. R. First, create a shared mailbox, for example shared@example.com.
Then give full permissions to a licensed account user@example.com.
Now with imapsync, use the licensed user login with the syntax:
R3. Microsoft introduced something called "security defaults" which is enabled user@example.com\shared@example.com
by default for new tenants. One of the rules blocks IMAP access as of imapsync.
The funny thing is that you can't disable single rules of this security
package without buying additional licenses.
Switching the whole thing off allows the login.0
Also, disable double-step authentication on the Azure/Active Directory portal.
See here:
https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults
Thanks to Stephan Buhre for this R3 answer.
R4. Are there special characters in the password? and the password for user@example.com
https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt
https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt
R5. Triple check the hostname then. Try all of these: Caveat: Character \ is a special character, so
* outlook.office365.com use double-quotes around the user login, like:
* imap-mail.outlook.com
* imap.outlook.com imapsync ... --user1 "user@example.com\shared@example.com"
Sources:
https://adam-hand.com/2017/07/25/connect-a-shared-mailbox-from-o365-to-outlook-via-imap/
https://www.arclab.com/en/kb/email/imap-settings-shared-mailbox-office-exchange-online.html
https://social.technet.microsoft.com/Forums/en-US/336e02ee-6767-4810-90a0-1352bd7cc9e9/office-365-how-to-access-a-shared-mailbox-using-imap-client?forum=onlineservicesexchange
======================================================================
Q. How can I use a shared account as a backup account for several
mailboxes and so avoid spending too many dollars in backup accounts?
R. Use the option --subfolder2 backup_foo
imapsync ... --user2 sharedloginsyntax --subfolder2 backup_foo
To restore, do the reverse with:
imapsync ... --user1 sharedloginsyntax --subfolder1 backup_foo
======================================================================
Q. Office365 fails with "User is authenticated but not connected".
R1. "The message User is authenticated but not connected is due to a
bug in the Office365 server's IMAP implementation. If the client
presents a valid user name but an invalid password, the server
accepts the login, but subsequent commands fail with the
aforementioned error message." Source:
https://unix.stackexchange.com/questions/164823/user-is-authenticated-but-not-connected-after-changing-my-exchange-password
Thanks to James Abbottsmith for this link and explanation at
https://github.com/imapsync/imapsync/issues/32#issuecomment-153561647
R2. Miguel Alameda reported understanding and solving this issue
like this, the context was admin/authuser:
"The admin user had not permission in the target mailbox."
====================================================================== ======================================================================
Q. I see "NO Maximum size of appendable message has been exceeded" Q. I see "NO Maximum size of appendable message has been exceeded"
What can I do with that? What can I do with that?
R. Office365 supports send/receive max message sizes of up to 150MB R. Office365 supports send/receive max message sizes of up to 150MB
but you need to make changes in your tenant(s) to support it. but you need to make changes in your tenant(s) to support it.
@ -204,14 +320,15 @@ Q. The imap connection to Office365 is not working very efficiently,
R. Yes. Try DavMail R. Yes. Try DavMail
http://davmail.sourceforge.net/ http://davmail.sourceforge.net/
I don't use it myself but a user, Yannick Palanque, reported great results using it. I don't use it myself but a user, Yannick Palanque, reported
great results using it.
====================================================================== ======================================================================
Q. The sync fails with many "Trying command when NOT connected!". Q. The sync fails with many "Trying command when NOT connected!".
What can I do? What can I do?
R. The --debugimap option can show you more details, especially some R. The --debugimap option can show you more details, especially
message like: messages like this one:
"BAD Request is throttled. Suggested Backoff Time: 178755 milliseconds" "BAD Request is throttled. Suggested Backoff Time: 178755 milliseconds"
In that case, see the next faq item and its fixes. In that case, see the next faq item and its fixes.
@ -249,6 +366,29 @@ R2. Play with options --maxbytespersecond or --maxmessagespersecond
I don't know the upper value that avoids the default throttling from 0365 I don't know the upper value that avoids the default throttling from 0365
and I guess it changes over time. and I guess it changes over time.
======================================================================
Q. What are the receive and sending limits of Office365?
R1. 3600 messages per hour (22 July 2021). Maybe 1200 as imapsync can
be viewed as a single sender. Tell me what you experienced,
I'll update this point.
https://docs.microsoft.com/en-us/office365/servicedescriptions/exchange-online-service-description/exchange-online-limits#receiving-and-sending-limits
In that case, imapsync can adapt to this with (1 message/second = 3600 messages per hour)
imapsync ... --maxmessagespersecond 1
or
imapsync ... --maxmessagespersecond 0.33
R2. I also found "Microsoft theoretically allows for about 300MB of
throughput per user per hour." at
https://www.systools.in/blog/microsoft-office-365-throttling-policy/
In that case, imapsync can adapt to this with (83333 bytes/second = 300 MBytes/hour)
imapsync ... --maxbytespersecond 83333
====================================================================== ======================================================================
Q. Office365 refuses to create the folder named "Files" with the error Q. Office365 refuses to create the folder named "Files" with the error
"NO Folder name is reserved". What happens? "NO Folder name is reserved". What happens?
@ -278,11 +418,12 @@ R0. To fix this, add --exclude Files
"Files_renamed_by_imapsync", but no complaining. "Files_renamed_by_imapsync", but no complaining.
R1. This folder "Files" seems to be a standard folder in Exchange Online, R1. This folder "Files" seems to be a standard folder in Exchange Online,
but it is not. This folder contains all attachments in every email but it is not. This folder contains all attachments in every email
that is in the mailbox but without any headers. that is in the mailbox but without any headers.
This causes some confusion for users as these appear in their search
results as duplicate lines but without the sender details or even This causes some confusion for users as these appear in their search
the message body. results as duplicate lines but without the sender details or even
the message body.
This folder seems to be usually hidden so IMAP clients can’t see it, This folder seems to be usually hidden so IMAP clients can’t see it,
but for some reason sometimes it becomes visible. but for some reason sometimes it becomes visible.
@ -351,7 +492,7 @@ Q. From XXX Office365 I get this error message sometimes:
"BAD Command Argument Error 11". What does it mean? "BAD Command Argument Error 11". What does it mean?
R. This error message comes from the Office365 IMAP server when it R. This error message comes from the Office365 IMAP server when it
encounters any problem. Most of the time it is one of the following: encounters any problem. Most of the time it is one of the following:
* Some messages are bigger than the size limit. * Some messages are bigger than the size limit.
45 MB by default on Office365. 45 MB by default on Office365.
@ -437,7 +578,8 @@ but previously in Office365, you must do something like the following, using Pow
Note from Betsy Lawlor: You can use global modern authentication with two factor Note from Betsy Lawlor: You can use global modern authentication with two factor
on Exchange Online (M365) but you must have "AllowBasicAuthImap" on the admin account you are using to migrate the mail. on Exchange Online (M365) but you must have "AllowBasicAuthImap" on
the admin account you are using to migrate the mail.
Remark: PLAIN authentication is the only way to go with --authuser1 for now. Remark: PLAIN authentication is the only way to go with --authuser1 for now.
So don't use --authmech1 SOMETHING with --authuser1 admin_user, So don't use --authmech1 SOMETHING with --authuser1 admin_user,
@ -453,7 +595,7 @@ so if two factors authentication is enabled you will no be able to sync
the mail (it will not connect to host2). the mail (it will not connect to host2).
OWA (I think this is the two-factor authentication) needs to be disabled OWA (I think this is the two-factor authentication) needs to be disabled
globally and enabled individually on the users you want to. globally and enabled individually on the users you want to.
One account can have the rights to access other mailboxes no matter if admin or One account can have the right to access other mailboxes no matter if admin or
any other account but needs to have OWA disabled to be able to connect any other account but needs to have OWA disabled to be able to connect
to the office365 server. to the office365 server.
@ -473,8 +615,8 @@ Date: Mon, 1 Jun 2015 17:53:54 +0000
Q. Couldn't create folder [trash] "Mailbox already exists". Q. Couldn't create folder [trash] "Mailbox already exists".
R. Some servers take care of character cases in folder names, R. Some servers take care of character cases in folder names,
some servers do not, like Exchange. Since not respecting the case can merge some servers do not, like Exchange. Since not respecting the case
two different folders into one then imapsync respects the case. can merge two different folders into one then imapsync respects the case.
For example, if a host1 server has a folder name called "trash" For example, if a host1 server has a folder name called "trash"
and the host2 server already has a folder "Trash" or "TRASH" and the host2 server already has a folder "Trash" or "TRASH"

102
FAQ.d/FAQ.OnlineUI.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.OnlineUI.txt,v 1.29 2021/05/05 21:38:32 gilles Exp gilles $ $Id: FAQ.OnlineUI.txt,v 1.31 2021/12/24 21:02:52 gilles Exp gilles $
This document is also available online at This document is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -38,47 +38,52 @@ Let's do it anyway, you'll be the judge.
R1. Some figures R1. Some figures
Date of this report: 22 April 2021. Date of this report: 24 December 2021. Happy Christmas!
The online imapsync service /X started 9 January 2017 The online imapsync service /X started 9 January 2017,
(1564 days of service). 5 years ago (1810 days of service).
On average, /X has 51 users per day, each user lunches on average On average, /X has 51 users per day, each user lunches on average
6 account migrations, from just one launch to many (hundreds). 6 account migrations, from just 1 launch to many (hundreds).
The total volume /X transferred so far is around 210 TiB coming The total volume /X transferred so far is around 240 TiB coming
from nearly 450 000 imap account migrations and 700 million email messages. from nearly 520 000 imap account migrations and 810 million email messages.
R2. Pros & Cons R2. Pros & Cons
The online imapsync service /X runs on HTTPS only, with a The online imapsync service /X runs on HTTPS only, with a letsencrypt
letsencrypt certificate, a certificate overall rated "A+" at certificate, an up to date certificate overall rated "A+" at
https://www.ssllabs.com/ssltest/analyze.html?d=imapsync.lamiral.info https://www.ssllabs.com/ssltest/analyze.html?d=imapsync.lamiral.info
Because of the HTTPS usage, what the users enter in their browser, Because of the HTTPS usage, what the users enter in their browser, the
the imap logins and passwords, can't be eavesdropped on the network. imap logins and passwords, can't be eavesdropped on the network.
Imapsync itself takes care of encryption for the imap sessions, Imapsync itself takes care of encryption for the imap sessions, if
if possible. First, imapsync tries to use SSL on port 993, possible. First, imapsync tries to use SSL on port 993, then TLS on
then TLS on port 143 if the servers announce TLS, then no encryption at all. port 143 if the servers announce TLS, then no encryption at all.
Concerning encryption, what is done with the source imap server host1 Concerning encryption, what is done with the source imap server host1
is independent of what is done with the destination imap server host2. is independent of what is done with the destination imap server host2.
On the date of 22 April 2021, there is no security problem On the date of 24 December 2021, there is no security problem detected
detected or reported to me (Gilles LAMIRAL), so far. or reported to me (Gilles LAMIRAL), so far.
Feel free to attack the service and feel free to report any
hole encountered. Have in mind I can watch what you do
from the server-side and take measures if the service suffers from
your acts. Drop me a note before will be fair play.
As the owner of the service, it could have been 450 000 pairs of Concerning the Log4j threats, the imapsync server doesn't use Log4j at
credentials collected and nearly 210 terabytes of email messages. all, so it should be safe about this issue.
I haven't kept them but I can't prove I haven't. It's just trust,
like nearly every online service in the universe.
The imap server certificates are not checked for authenticity Feel free to attack the service and feel free to report any hole
(by default) because too many imap servers are crappy configured encountered. Have in mind that I can watch what you do from the
regarding certified certificates. server-side, and then take measures if the service suffers from your
acts. Drop me a note before will be fair play and I will let you act
as harsh as you can.
As the owner of the service, it could have been 520 000 pairs of
credentials collected and nearly 240 terabytes of email messages.
That's massive. I haven't kept them but I can't prove I haven't. It's
just trust, like nearly every online service in the universe.
Concerning imapsync transfers, the imap server certificates are not
checked for authenticity by default because too many imap servers are
crappy configured regarding their certified certificates.
This default behavior is chosen like this because users of /X This default behavior is chosen like this because users of /X
want their emails transferred, instead of being not transferred want their emails transferred, instead of being not transferred
@ -88,35 +93,38 @@ I admit that this part, checking imap ssl/tls certificates,
could be improved from my side by including well known could be improved from my side by including well known
certificates directly in imapsync. certificates directly in imapsync.
If the imap servers don't honor ssl nor tls, then logins, passwords If an imap server doesn't honor ssl nor tls, then logins, passwords
and everything will go clear text during the imap transfers. and everything will go clear text during the imap transfers. That's
That's not good at all but what "comforts" me is that if an not good at all! But what "comforts" me is that if an imap server does
imap server does only clear text transfers, then it's also true only clear text transfers, then it's also true for all the imap
for all the imap sessions the account owner encounters, sessions the account owner encounters, imapsync is just one of them;
imapsync is just one of them; the imap servers are then always the imap servers are then always unsecured for any imap software
unsecured for any imap software client. client.
Last point, who could be sure that no cracker cracked the online hosts
and that he isn't currently sniffing the credentials?
Last point, who could be sure that no cracker cracked the online No one! I'm not sure myself, even if I do take care of that
hosts and that he isn't currently sniffing the credentials?
No one, I'm not sure myself, even if I do take care of that
possibility. So changing the imap accounts passwords after possibility. So changing the imap accounts passwords after
the sync is a safe and recommended practice! the sync is a safe and recommended practice! You can even
change the password just after imapsync has started its job,
just after a successful login. This way you know the previously
given password is no longer working just after imapsync has
started its job.
===================================================================== =====================================================================
Q. Does the online service store any sensitive information Q. Does the online service store any sensitive information
like my passwords? like my passwords?
No. No.
The online passwords are kept by your browser. The goal is
that the next time you run a sync from your browser, all the The online passwords are kept by your browser. The goal is that the
parameters are already there without having to re-enter them. next time you run a sync from your browser, all the parameters are
The save is done when you click on the "Sync or resync" green already there without having to re-enter them. The save is done when
button. So, if you don't want your browser to keep your password, you click on the "Sync or resync" green button. So, if you don't want
enter a wrong one or empty the password field and click your browser to keep your password, enter a wrong one or empty the
on the "Sync or resync" green button, the wrong password password field and click on the "Sync or resync" green button, the
will be saved in your browser and no sync will be done, wrong password will be saved in your browser and no sync will be done,
since a good login is mandatory to do anything on your mailboxes. since a good login is mandatory to do anything on your mailboxes.
===================================================================== =====================================================================

17
FAQ.d/FAQ.Release_Checklist.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.Release_Checklist.txt,v 1.20 2021/07/06 02:29:51 gilles Exp gilles $ $Id: FAQ.Release_Checklist.txt,v 1.22 2022/01/14 12:46:38 gilles Exp gilles $
This documentation is also available online at This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -12,6 +12,16 @@ https://imapsync.lamiral.info/FAQ.d/FAQ.Release_Checklist.txt
Checklist before release a new release: Checklist before release a new release:
- Make "imapsync --tests" pass on Linux
- Make "imapsync --tests" pass on Mac
- Make "imapsync --tests" pass on Windows 32 bits
- Make "imapsync --tests" pass on Windows 64 bits
- Make "imapsync --testslive" pass on Linux
- Make "imapsync --testslive" pass on Mac
- Make "imapsync --testslive" pass on Windows 32 bits
- Make "imapsync --testslive" pass on Windows 64 bits
- Add a new section in S/news.shtml reading "rlog imapsync" - Add a new section in S/news.shtml reading "rlog imapsync"
- In case of a new dependency, add it to all the INSTALL/* files. - In case of a new dependency, add it to all the INSTALL/* files.
- Generate the README - Generate the README
@ -20,6 +30,11 @@ Checklist before release a new release:
- Read the OPTIONS section of README and read it very slowly - Read the OPTIONS section of README and read it very slowly
- Read slowly README_Windows.txt - Read slowly README_Windows.txt
- Read slowly the TUTORIAL_Unix file in html - Read slowly the TUTORIAL_Unix file in html
- Make "rcsdiff imapsync" give nothing relevant and exit 0
- make ci pass.
- Review the newsletter by running: - Review the newsletter by running:
m4 -P W/ml_announce.in.txt m4 -P W/ml_announce.in.txt

28
FAQ.d/FAQ.SSL_errors.txt
View file

@ -1,14 +1,14 @@
#!/bin/cat #!/bin/cat
$Id: FAQ.SSL_errors.txt,v 1.15 2021/06/22 15:35:10 gilles Exp gilles $ $Id: FAQ.SSL_errors.txt,v 1.17 2021/08/01 12:49:00 gilles Exp gilles $
This document is also available online at This document is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt
======================================================================= ======================================================================
Imapsync SSL errors Imapsync SSL errors
======================================================================= ======================================================================
Questions answered in this FAQ are: Questions answered in this FAQ are:
@ -25,11 +25,14 @@ Q. SSL connect attempt failed SSL
SSL connect attempt failed SSL SSL connect attempt failed SSL
SSL routines:ssl_choose_client_version:unsupported protocol SSL routines:ssl_choose_client_version:unsupported protocol
Q. fatal SSL error: SSL connect attempt failed with unknown error
SSL wants a read first
Q. How to see the certificate and identify problems in it? Q. How to see the certificate and identify problems in it?
Now the questions again with their answers. Now the questions again with their answers.
======================================================================= ======================================================================
Q. What are the errors Q. What are the errors
DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error DEBUG: .../IO/Socket/SSL.pm:1165: local error: SSL write error
or or
@ -45,11 +48,11 @@ it's often Exchange or Office365. I don't know what exactly happens.
Those errors happen more often on Windows than on Linux. Those errors happen more often on Windows than on Linux.
======================================================================= ======================================================================
Q. What can I do to avoid those "SSL read/write errors"? Q. What can I do to avoid those "SSL read/write errors"?
R0. Windows users: upgrade to imapsync.exe release 1.836 (or next ones) R0. Windows users: upgrade to imapsync.exe release 1.836 (or next ones)
Those errors don't appear with recent releases, post 1.836 Those errors appear less often with imapsync releases post 1.836
R1. Remove all ssl/tls encryption R1. Remove all ssl/tls encryption
@ -70,7 +73,7 @@ R5. Set up a ssltunnel proxy to the host.
Read the file FAQ.Security.txt for an example to set up Read the file FAQ.Security.txt for an example to set up
a ssltunnel proxy. a ssltunnel proxy.
======================================================================= ======================================================================
Q. SSL connect attempt failed SSL Q. SSL connect attempt failed SSL
routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
or or
@ -145,8 +148,13 @@ when they receive a TLS version 1.1 request.
In this case setting the version In this case setting the version
to 'SSLv23:!SSLv2:!SSLv3:!TLSv1_1:!TLSv1_2' might help. to 'SSLv23:!SSLv2:!SSLv3:!TLSv1_1:!TLSv1_2' might help.
======================================================================
Q. fatal SSL error: SSL connect attempt failed with unknown error
SSL wants a read first
======================================================================= R. If you're using --ssl1 or --ssl2, try instead --tls1 or --tls2
======================================================================
Q. How to see the certificate and identify problems in it? Q. How to see the certificate and identify problems in it?
R. Use the command openssl like this: R. Use the command openssl like this:
@ -221,5 +229,5 @@ ggEBANuPNbYLPMZ4vPa9NBoHAUdIXqpi0eqdXMXd2sT+qRmqxS5ihr999BHOROcr
... ...
======================================================================= ======================================================================
======================================================================= ======================================================================

205
FAQ.d/FAQ.Security.txt
View file

@ -1,45 +1,122 @@
#!/bin/cat #!/bin/cat
# $Id: FAQ.Security.txt,v 1.23 2021/06/22 15:32:10 gilles Exp gilles $ # $Id: FAQ.Security.txt,v 1.26 2021/07/11 12:11:38 gilles Exp gilles $
This document is also available online at This document is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Security.txt https://imapsync.lamiral.info/FAQ.d/FAQ.Security.txt
======================================================================= =======================================================================
Imapsync tips about security. Issues and solutions. Imapsync tips about security. Issues and solutions.
======================================================================= =======================================================================
SSL: going to encryption before the imap session start. Questions answered in this FAQ are:
It is on port 993.
TLS: going to encryption after the imap session start but before Q. Is running this program a secure method of transferring emails?
the credential are sent. It is on port 143. Are there any security concerns?
Q. Does imapsync refer to SSL security protocols with --ssl1 and --ssl2
and does it refer to TLS security protocols with the --tls1 and --tls2
options?
Short answer: No.
Q. I noticed that the online UI has no option for TLS/SSL.
Is this secure?
Is this more secure than using the .bat file on my computer?
Q. Are transferred emails/attachments stored on any other
server/location aside from my originating/destination server(s)?
Q. Other than changing passwords on the originating/destination email
accounts once the relevant emails have been moved,
are there any other security tips I should know?
Q. I need to transfer mail from an imap server to another imap server.
Which ports need to be open on the firewall to make this possible?
Q. Does imapsync support the IMAP command STARTTLS?
Q. Does imapsync support IMAP over SSL/TLS (IMAPS)?
Q. How can I test an ssl/tls imap connection without imapsync?
Q. How can I manually test login using --ssl?
Q. Imapsync used to use SSL_VERIFY_PEER now it uses SSL_VERIFY_NONE.
How can I change this back to the more secure SSL_VERIFY_PEER?
Q: How can I have an imaps server?
Now the questions again with their answers.
======================================================================= =======================================================================
Q. Is running this program a secure method of transferring emails? Q. Is running this program a secure method of transferring emails?
Are there any security concerns? Are there any security concerns?
R. Well, it depends. Use encryption, secure the access to the host running R. Well, it depends. Use encryption and secure access to the host running
imapsync and everything shall be safe. imapsync then everything shall be safe.
======================================================================= =======================================================================
Q. I noticed that the online UI has no option for TLS/SSL Q. Does imapsync refer to SSL security protocols with --ssl1 and --ssl2
Is this secure? Is this more secure that using the .bat file and does it refer to TLS security protocols with the --tls1 and --tls2
on my computer? options?
Short answer: No.
R1. The online UI does TLS/SSL imap connections if the imap R. No.
servers support TLS/SSL. If you are concerned by security then
using the .bat file or .sh on your computer should be more secure Imapsync behaviour:
since you can examine and secure it by yourself, no matter high is
your paranoid spirit. The online UI security is mine, I am --ssl: Goes to encryption before the imap session start.
concerned by security, not to the upmost high level possible The connection is on port 993.
but I won't give you direct access to the host to discover my level. Can use the TLS or SSL security protocols.
With a good guy spirit, feel free to try to break the online UI Fails if encryption cannot be established.
security and report me any security issue you encounter, I'll do my
best to fix them as soon as possible. Drop me a note before --tls: Goes to encryption after the imap session start but before the
starting because I may detect a sort of abuse and ban credential are sent.
definitively your IPs. The connection is on port 143.
Can use the TLS or SSL securityprotocols.
Fails if encryption cannot be established.
Explanation:
To force transferring emails over an encrypted connection, you can use
the imapsync parameters --ssl1 and --ssl2 or the parameters --tls1 and
--tls2, they all force the connection to be encrypted by the security
protocols series SSL/TLS.
In a security context, SSL refers to all deprecated Secure Sockets
Layer protocols. TLS refers to the SSL successors, Transport Layer
Security protocols. But TLS 1.0 and TLS 1.1 are also deprecated.
Current endorsed versions of TLS are only TLS 1.2 and TLS 1.3 (July 2021).
The whole story is detailed here:
https://en.wikipedia.org/wiki/Transport_Layer_Security
Options --ssl1 and --ssl2 are a little more paranoid than --tls1 and
--tls2 because they verify that the hostname of the certificate is the
same as the one used by imapsync. Other than that, the security
behavior is the same.
=======================================================================
Q. I noticed that the online UI has no option for TLS/SSL.
Is this secure?
Is this more secure than using the .bat file on my computer?
R1. The online UI does TLS/SSL imap connections if the imap servers
support TLS/SSL.
If you are concerned about security then using the .bat file or .sh on
your computer should be more secure since you can examine and secure
it by yourself, no matter high is your paranoid spirit compared to
mine.
The online UI security is mine, I am concerned by security, not to the
utmost high level possible but I won't give you direct access to the
host to discover my level. With a good guy spirit, feel free to try
to break the online UI security and report me any security issue you
encounter, I'll do my best to fix them as soon as possible. Drop me a
note before starting because I may detect a sort of abuse and ban your
IPs.
======================================================================= =======================================================================
Q. Are transferred emails/attachments stored on any other Q. Are transferred emails/attachments stored on any other
@ -52,11 +129,11 @@ Q. Other than changing passwords on the originating/destination email
accounts once the relevant emails have been moved, accounts once the relevant emails have been moved,
are there any other security tips I should know? are there any other security tips I should know?
R. Secure the host where imapsync is running since credentials are R. Yes. Secure the host where imapsync is running since credentials
on it. are on it.
======================================================================= =======================================================================
Q. I need to transfer mail from an imap server to an other imap server. Q. I need to transfer mail from an imap server to another imap server.
Which ports need to be open on the firewall to make this possible? Which ports need to be open on the firewall to make this possible?
R. It depends. Open either: R. It depends. Open either:
@ -64,29 +141,31 @@ R. It depends. Open either:
* port 993 in ssl mode (--ssl1 or --ssl2) * port 993 in ssl mode (--ssl1 or --ssl2)
======================================================================= =======================================================================
Q. Does imapsync support IMAP TLS? Q. Does imapsync support the IMAP command STARTTLS?
R1. Yes. R1. Yes.
Use --tls1 and/or --tls2 options:
--tls1 tells imapsync to use tls on host1. Use --tls1 and --tls2 options:
--tls2 tells imapsync to use tls on host2.
R2. Since imapsync release 1.755 TLS mode is activated automatically --tls1 tells imapsync to use STARTTLS on host1.
if the server announce it supports it, STARTTLS inside the response --tls2 tells imapsync to use STARTTLS on host2.
to CAPABILITY, and if neither --notls nor --ssl is explicitely
mentioned on the command line options. R2. Since imapsync release 1.755 STARTTLS mode is activated
automatically when the server announces that it supports it by
listing STARTTLS inside the response to the CAPABILITY command.
If either --notls or --ssl are explicitly mentioned on the
command-line options then STARTTLS won't be done.
======================================================================= =======================================================================
Q. Does imapsync support IMAP over SSL (IMAPS)? Q. Does imapsync support IMAP over SSL/TLS (IMAPS)?
R. Yes natively since release 1.161. R. Yes natively since release 1.161.
Still, there are 2 ways, at least, to use ssl: Still, there are 2 ways, at least, to use ssl:
a) Use native --ssl1 and/or --ssl2 options a) Use native --ssl1 and/or --ssl2 options
--ssl1 tells imapsync to use ssl on host1. --ssl1 tells imapsync to use ssl/tls on host1.
--ssl2 tells imapsync to use ssl on host2. --ssl2 tells imapsync to use ssl/tls on host2.
b) Use stunnel b) Use stunnel
@ -108,18 +187,18 @@ b) Use stunnel
If the local port 143 is already taken then use a free one, If the local port 143 is already taken then use a free one,
like 10143 for example. like 10143 for example.
c) Other example for accessing gmail with no local root access c) Another example for accessing Gmail with no local root access
to open port 143 to open port 143:
stunnel3 -P '' -c -d 9993 -r imap.gmail.com:993 -f stunnel3 -P '' -c -d 9993 -r imap.gmail.com:993 -f
Then, to access gmail as host2 use: Then, to access Gmail as host2 use:
imapsync ... --host2 localhost --port2 9993 --nossl2 imapsync ... --host2 localhost --port2 9993 --nossl2
======================================================================= =======================================================================
Q.How can I test an ssl imap connection without imapsync? Q.How can I test an ssl/tls imap connection without imapsync?
R1. Use either ncat or telnet-ssl or openssl commands like in the R1. Use either ncat or telnet-ssl or openssl commands like in the
following examples with imap.gmail.com server: following examples with imap.gmail.com server:
@ -133,37 +212,39 @@ R1. Use either ncat or telnet-ssl or openssl commands like in the
The previous commands are interactive, hit ctrl-c The previous commands are interactive, hit ctrl-c
to finish them. If you want to finish automatically, then use: to finish them. If you want to finish automatically, then use:
{ sleep 2; echo "a logout"; sleep 1; } | ncat --ssl -C imap.gmail.com 993 { sleep 2; echo "A LOGOUT"; sleep 1; } | ncat --ssl -C imap.gmail.com 993
======================================================================= =======================================================================
Q. How can I manually test a login via ssl? Q. How can I manually test login using --ssl?
R. Use either ncat or telnet-ssl or openssl commands like in the R. Use either ncat or telnet-ssl or openssl commands like in the
following examples with imap.gmail.com server: following examples with imap.gmail.com server:
ncat --ssl -C imap.gmail.com 993 ncat --ssl -C imap.gmail.com 993
telnet-ssl -z ssl imap.gmail.com 993 telnet-ssl -z ssl imap.gmail.com 993
openssl s_client -crlf -connect imap.gmail.com:993 openssl s_client -crlf -connect imap.gmail.com:993
Tipical dialog for an imap LOGIN command: Typical dialog for an imap LOGIN command:
* OK Gimap ready for requests from 78.196.254.58 q1mb175739668wix * OK Gimap ready for requests from 78.196.254.58 q1mb175739668wix
a LOGIN "gilles.lamiral@gmail.com" "secret" A1 LOGIN "gilles.lamiral@gmail.com" "secret"
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE ... ESEARCH * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE ... ESEARCH
a OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success) A1 OK gilles.lamiral@gmail.com Gilles Lamiral authenticated (Success)
b LOGOUT A2 LOGOUT
* BYE LOGOUT Requested * BYE LOGOUT Requested
b OK 73 good day (Success) A2 OK 73 good day (Success)
The client part you have to type is The client part you have to type is
a LOGIN ... A1 LOGIN ...
b LOGOUT A2 LOGOUT
while replacing ... by your credentials values, while replacing ... by your credentials values,
other lines are the server responses. other lines are the server responses.
======================================================================= =======================================================================
Q.How can I test an tls imap connection without imapsync? Q. How can I test a STARTTLS imap connection without imapsync?
R1. Use openssl command like the following example with R1. Use openssl command like the following example with
an outlook.office365.com server: an outlook.office365.com server:
@ -177,13 +258,12 @@ to finish them. If you want to finish automatically, then use:
Replace outlook.office365.com with your imap server name. Replace outlook.office365.com with your imap server name.
====================================================================== ======================================================================
Q. Imapsync used to use SSL_VERIFY_PEER now it uses SSL_VERIFY_NONE. Q. Imapsync used to use SSL_VERIFY_PEER now it uses SSL_VERIFY_NONE.
How can I change this back to the more secure SSL_VERIFY_PEER? How can I change this back to the more secure SSL_VERIFY_PEER?
R2. After imapsync 1.673, R. After imapsync 1.673,
to set SSL_verify_mode to SSL_VERIFY_PEER on host1 to set SSL_verify_mode to SSL_VERIFY_PEER on host1
and SSL_verify_mode to SSL_VERIFY_NONE on host2 and SSL_verify_mode to SSL_VERIFY_NONE on host2
@ -191,18 +271,20 @@ R2. After imapsync 1.673,
--sslargs1 SSL_verify_mode=1 \ --sslargs1 SSL_verify_mode=1 \
--sslargs2 SSL_verify_mode=0 --sslargs2 SSL_verify_mode=0
See perldoc IO::Socket::SSL for all possibilities, also at See "perldoc IO::Socket::SSL" for all possibilities, also at
http://search.cpan.org/perldoc?IO%3A%3ASocket%3A%3ASSL http://search.cpan.org/perldoc?IO%3A%3ASocket%3A%3ASSL
It might be possible you need an extra: It might be possible you need an extra option
--sslargs1 SSL_ca_file=/etc/ssl/certs/ca-certificates.crt
--sslargs1 SSL_ca_file=/etc/ssl/certs/ca-certificates.crt
to help the ssl software verifying the server certificate. to help the ssl software verifying the server certificate.
The file ca-certificates.crt may be elsewhere on your system, even The file ca-certificates.crt may be elsewhere on your system, even
named differently. named differently.
The imap server certificates are not checked for authenticity The imap server certificates are not checked for authenticity
(by default) because too many imap servers are crappy configured by imapsync by default because too many imap servers are crappy
regarding certified certificates. configured regarding certified certificates.
This default behavior is chosen like this because users This default behavior is chosen like this because users
want their emails transferred, instead of being not transferred want their emails transferred, instead of being not transferred
@ -211,12 +293,13 @@ because of an incompetent imap server sysadmin.
I admit that this part, checking imap ssl/tls certificates, I admit that this part, checking imap ssl/tls certificates,
could be improved from my side by including well known could be improved from my side by including well known
certificates directly in imapsync. certificates directly in imapsync.
Drop me a note to encourage me. Drop me a note to encourage me, I'm lazy.
======================================================================= =======================================================================
Q: How to have an imaps server? Q: How can I have an imaps server?
R. Three solutions.
R.
a) Install one a) Install one
b) or use stunnel : b) or use stunnel :

31
FAQ.d/FAQ.Yahoo.txt
View file

@ -1,5 +1,5 @@
$Id: FAQ.Yahoo.txt,v 1.12 2021/05/04 11:04:51 gilles Exp gilles $ $Id: FAQ.Yahoo.txt,v 1.13 2021/12/03 12:38:58 gilles Exp gilles $
This document is also available online at This document is also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
@ -18,24 +18,6 @@ Q. Synchronizing from XXX to Yahoo
Now the questions again with their answers. Now the questions again with their answers.
Some explanation about the options given later:
https://imapsync.lamiral.info/README
...
--noabletosearch : Makes --minage and --maxage options use the internal
dates given by a FETCH imap command instead of the
"Date:" header. Internal date is the arrival date
in the mailbox.
--noabletosearch equals --noabletosearch1 --noabletosearch2
--regexmess reg : Apply the whole regex to each message before transfer.
Example: 's/\000/ /g' # to replace null by space.
--regexmess reg : and this one, etc.
I'm not sure anymore about the need of --noabletosearch with Yahoo, maybe
they fixed the issue about the standard imap searching command SEARCH
they didn't honor before.
======================================================================= =======================================================================
Q. Synchronizing from Yahoo to XXX Q. Synchronizing from Yahoo to XXX
@ -49,8 +31,7 @@ Example:
--password1 billysecreta \ --password1 billysecreta \
--host2 XXX \ --host2 XXX \
--user2 Billy@other.com \ --user2 Billy@other.com \
--password2 billysecretb \ --password2 billysecretb
--noabletosearch
You also need to go to Yahoo, security and enable You also need to go to Yahoo, security and enable
@ -90,7 +71,7 @@ R. Example:
--host2 imap.mail.yahoo.com \ --host2 imap.mail.yahoo.com \
--user2 Billy@yahoo.com \ --user2 Billy@yahoo.com \
--password2 billysecretb \ --password2 billysecretb \
--noabletosearch --regexmess "s{(?<![\n])\z}{\r\n}gxms" --regexmess "s{(?<![\n])\z}{\r\n}gxms"
You wonder why there is this ugly regex, what it does and where does it You wonder why there is this ugly regex, what it does and where does it
come from? come from?
@ -101,5 +82,11 @@ when copying some messages to Yahoo.
It adds a final \r\n on a message if a final \r\n is missing. It adds a final \r\n on a message if a final \r\n is missing.
See https://github.com/imapsync/imapsync/issues/209 See https://github.com/imapsync/imapsync/issues/209
https://imapsync.lamiral.info/README
...
--regexmess reg : Apply the whole regex to each message before transfer.
Example: 's/\000/ /g' # to replace null by space.
--regexmess reg : and this one, etc.
======================================================================= =======================================================================
======================================================================= =======================================================================

12
FAQ.d/htaccess.txt
View file

@ -1,5 +1,5 @@
# $Id: htaccess.txt,v 1.31 2021/05/29 08:09:11 gilles Exp gilles $ # $Id: htaccess.txt,v 1.32 2021/07/23 12:38:19 gilles Exp gilles $
AddDescription "<b>Back</b> to Imapsync main page." .. AddDescription "<b>Back</b> to Imapsync main page." ..
@ -40,13 +40,15 @@ AddDescription "<b>Flags tips and issues</b>." FAQ.Flag
AddDescription "<b>General Data Protection Regulation</b> (GDPR)." FAQ.GDPR.txt AddDescription "<b>General Data Protection Regulation</b> (GDPR)." FAQ.GDPR.txt
AddDescription "<b>General</b> and <b>pot-pourri</b> issues" FAQ.General.txt AddDescription "<b>General</b> and <b>pot-pourri</b> issues" FAQ.General.txt
AddDescription "<b>Gmail</b> accounts." FAQ.Gmail.txt AddDescription "<b>Gmail</b> accounts." FAQ.Gmail.txt
AddDescription "<b>GoDaddy</b> accounts." FAQ.GoDaddy.txt
AddDescription "<b>IceWarp</b> accounts." FAQ.IceWarp.txt AddDescription "<b>IceWarp</b> accounts." FAQ.IceWarp.txt
AddDescription "<b>ISP tips</b>." FAQ.ISP.txt AddDescription "<b>ISP tips</b>." FAQ.ISP.txt
AddDescription "<b>Kerio</b> accounts." FAQ.Kerio.txt AddDescription "<b>Kerio</b> accounts." FAQ.Kerio.txt
AddDescription "<b>MailEnable</b> accounts." FAQ.MailEnable.txt AddDescription "<b>MailEnable</b> accounts." FAQ.MailEnable.txt
AddDescription "<b>Massive/bulk migrations</b>." FAQ.Massive.txt AddDescription "<b>Massive and bulk</b> migrations." FAQ.Massive.txt
AddDescription "<b>Memory issues</b>." FAQ.Memory.txt AddDescription "<b>Log file, content, errors and tips</b>." FAQ.Logfile.txt
AddDescription "<b>Migration Plan</b>." FAQ.Migration_Plan.txt AddDescription "<b>Memory</b> issues." FAQ.Memory.txt
AddDescription "Migration <b>Plan</b>." FAQ.Migration_Plan.txt
AddDescription "<b>Office365</b> accounts." FAQ.Office365.txt AddDescription "<b>Office365</b> accounts." FAQ.Office365.txt
AddDescription "<b>The Online UI service Q&R</b>." FAQ.OnlineUI.txt AddDescription "<b>The Online UI service Q&R</b>." FAQ.OnlineUI.txt
AddDescription "<b>Password & special characters on Windows</b>." FAQ.Passwords_on_Windows.txt AddDescription "<b>Password & special characters on Windows</b>." FAQ.Passwords_on_Windows.txt
@ -56,7 +58,7 @@ AddDescription "<b>POP3 issues</b>." FAQ.POP3
AddDescription "<b>Principles & design decisions</b>." FAQ.Principles.txt AddDescription "<b>Principles & design decisions</b>." FAQ.Principles.txt
AddDescription "<b>Selecting messages</b>." FAQ.Messages_Selection.txt AddDescription "<b>Selecting messages</b>." FAQ.Messages_Selection.txt
AddDescription "<b>How to transfer too big messages</b>." FAQ.Messages_Too_Big.txt AddDescription "<b>How to transfer too big messages</b>." FAQ.Messages_Too_Big.txt
AddDescription "<b>Why the imapsync web site looks so old?</b>" FAQ.Old_Style_Web_Design.txt AddDescription "Why the imapsync web site <b>looks so old?</b>" FAQ.Old_Style_Web_Design.txt
AddDescription "<b>UCS</b> accounts." FAQ.UCS.txt AddDescription "<b>UCS</b> accounts." FAQ.UCS.txt
AddDescription "<b>Checklist before release a new release</b>." FAQ.Release_Checklist.txt AddDescription "<b>Checklist before release a new release</b>." FAQ.Release_Checklist.txt

33
INSTALL.d/Dockerfile Executable file → Normal file
View file

@ -1,6 +1,6 @@
## Dockerfile for building a docker imapsync image ## Dockerfile for building a docker imapsync image
# $Id: Dockerfile,v 1.30 2020/03/26 07:15:04 gilles Exp gilles $ # $Id: Dockerfile,v 1.38 2021/11/29 10:20:56 gilles Exp gilles $
# I use the following command to build the image: # I use the following command to build the image:
# #
# docker build -t gilleslamiral/imapsync . # docker build -t gilleslamiral/imapsync .
@ -10,9 +10,9 @@
# I thank you very much # I thank you very much
# I like thanks # I like thanks
# I like stars # I like stars
# I also like (and need) money # I also need money to keep on doing this stuff
# Number of imapsync images pulled so far (2021_05_29): 368881 # Number of imapsync images pulled so far (2021_11_29): 608326
# Command used: # Command used:
# curl -s https://hub.docker.com/v2/repositories/gilleslamiral/imapsync/ | jq '.pull_count' # curl -s https://hub.docker.com/v2/repositories/gilleslamiral/imapsync/ | jq '.pull_count'
@ -24,16 +24,13 @@ LABEL maintainer="Gilles LAMIRAL <gilles@lamiral.info>" \
description="Imapsync" \ description="Imapsync" \
documentation="https://imapsync.lamiral.info/#doc" documentation="https://imapsync.lamiral.info/#doc"
# Put a copy of the Dockerfile in the image itself # I put a copy of the Dockerfile in the image itself
# It can help future maintenance, isn't it? # It can help maintenance, isn't it?
# Also put optionally my local and usually more recent imapsync on /, for testing purpose
COPY Dockerfile / COPY Dockerfile imapsyn[c] prerequisites_imapsyn[c] /
# Also put my local imapsync on /usr/bin/ for testing purpose
# renamed imapsync_local since would be erased by the wget
COPY imapsync /usr/bin/imapsync_local
COPY prerequisites_imapsync /usr/bin/prerequisites_imapsync_local
RUN set -xe && \ RUN set -xe && \
apt-get update \ apt-get update \
@ -44,7 +41,8 @@ RUN set -xe && \
libdata-uniqid-perl \ libdata-uniqid-perl \
libencode-imaputf7-perl \ libencode-imaputf7-perl \
libfile-copy-recursive-perl \ libfile-copy-recursive-perl \
libfile-tail-perl \ libfile-tail-perl \
libio-compress-perl \
libio-socket-ssl-perl \ libio-socket-ssl-perl \
libio-socket-inet6-perl \ libio-socket-inet6-perl \
libio-tee-perl \ libio-tee-perl \
@ -70,16 +68,17 @@ RUN set -xe && \
lsof \ lsof \
ncat \ ncat \
openssl \ openssl \
ca-certificates \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
RUN set -xe \ RUN set -xe \
&& cd /usr/bin/ \
&& pwd \ && pwd \
&& wget -N https://imapsync.lamiral.info/imapsync \ && wget -N --no-check-certificate https://imapsync.lamiral.info/imapsync \
https://imapsync.lamiral.info/prerequisites_imapsync \ https://imapsync.lamiral.info/prerequisites_imapsync \
https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/python/oauth2.py https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/python/oauth2.py \
&& mv imapsync oauth2.py /usr/bin/ \ && chmod +x imapsync oauth2.py \
&& chmod +x /usr/bin/imapsync /usr/bin/imapsync_local /usr/bin/oauth2.py # just_a_comment_to_force_update 2019_11_26_12_58_34 && /usr/bin/imapsync --testslive && /usr/bin/imapsync --tests # just_a_comment_to_force_update 2021_11_29_11_20_56
USER nobody:nogroup USER nobody:nogroup

51
INSTALL.d/INSTALL.Centos.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
# $Id: INSTALL.Centos.txt,v 1.25 2021/02/02 17:08:46 gilles Exp gilles $ # $Id: INSTALL.Centos.txt,v 1.29 2021/12/17 14:52:20 gilles Exp gilles $
This documentation is also located online at This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/ https://imapsync.lamiral.info/INSTALL.d/
@ -9,37 +9,43 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.Centos.txt
Installing imapsync on CentOS Installing imapsync on CentOS
======================================================================= =======================================================================
There is one section for Centos 8, There is one section for AlmaLinux and Centos 8,
one for Centos 7, one for Centos 7,
and one for Centos 6. and one for Centos 6.
======================================================================= =======================================================================
Centos 8 AlmaLinux and Centos 8
======================================================================= =======================================================================
How to install imapsync on AlmaLinux and Centos 8:
How to install the latest imapsync on Centos 8 dnf install --enablerepo=powertools imapsync
imapsync --testslive
dnf install wget perl-App-cpanminus gcc
# then =======================================================================
cpanm lazy Test::MockObject AlmaLinux and Centos 8 and latest imapsync
wget https://imapsync.lamiral.info/imapsync =======================================================================
perl -Mlazy imapsync
At the time of this writing (December 2021), the powertools repository
contains imapsync release 1.977, which is not the latest available
imapsync release. (1.977 date is 2019/12/23 while 2.174 is 2021/12/14)
In order to install the latest imapsync I suggest the following process:
Install the powertools imapsync 1.977 release via dnf like mentioned above.
It will install the needed packages, Perl itself and Perl modules
for release 1.977.
The latest imapsync 2.174 needs no more Perl modules.
wget -N https://imapsync.lamiral.info/imapsync
chmod +x imapsync chmod +x imapsync
./imapsync --testslive ./imapsync --testslive
./imapsync --tests ./imapsync --tests
You may get an even more recent imapsync release this way.
If you encounter the following error during the run of "perl -Mlazy imapsync"
...
Can't locate object method "new" via package "Archive::Tar" at ...
FAIL install IO-String-1.08
fix it with:
wget http://mirror.centos.org/centos/8/PowerTools/x86_64/os/Packages/perl-IO-String-1.08-31.el8.noarch.rpm
yum install perl-IO-String-1.08-31.el8.noarch.rpm -y
See also https://github.com/imapsync/imapsync/issues/211
======================================================================= =======================================================================
Centos 7 Centos 7
@ -72,7 +78,7 @@ Unit tests:
At the time of this writing (March 2020), the epel7 repository still At the time of this writing (March 2020), the epel7 repository still
contains imapsync release 1.727, which is not the latest available contains imapsync release 1.727, which is not the latest available
imapsync release. (1.727 date is 2016/08/19 while 1.979 is 2020/01/14) imapsync release. (1.727 date is 2016/08/19 while 2.174 is 2021/12/14)
In order to install the latest imapsync I suggest the following process: In order to install the latest imapsync I suggest the following process:
@ -102,7 +108,8 @@ Then:
perl-Test-NoWarnings \ perl-Test-NoWarnings \
perl-Test-Simple \ perl-Test-Simple \
perl-Test-Warn \ perl-Test-Warn \
perl-Sub-Uplevel perl-Sub-Uplevel \
ca-certificates
Also, run the following commands: Also, run the following commands:

51
INSTALL.d/INSTALL.Darwin.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
# $Id: INSTALL.Darwin.txt,v 1.34 2021/06/15 10:02:37 gilles Exp gilles $ # $Id: INSTALL.Darwin.txt,v 1.36 2021/12/24 21:08:13 gilles Exp gilles $
This documentation is also located online at This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/ https://imapsync.lamiral.info/INSTALL.d/
@ -23,18 +23,18 @@ A) Installing imapsync binary on Darwin / Mac OS X
There is a standalone imapsync binary for Mac OS X There is a standalone imapsync binary for Mac OS X
called "imapsync_bin_Darwin" (without the quotes), called "imapsync_bin_Darwin" (without the quotes),
available in the compressed tarball called imapsync-1.977.tgz available in the compressed tarball called imapsync-2.140.tgz
where 1.977 is the imapsync version number. where 2.140 is the imapsync version number.
In case you haven't buy it yet or forgot where the download link is, In case you haven't buy it yet or forgot where the download link is,
check https://imapsync.lamiral.info/dist/ check https://imapsync.lamiral.info/dist/
Download the tarball imapsync-1.977.tgz, not the binary imapsync_bin_Darwin Download the tarball imapsync-2.140.tgz, not the binary imapsync_bin_Darwin
because the binary is also in the tarball and there are other useful files because the binary is also in the tarball and there are other useful files
in the tarball. The binary present in /dist/ is just there for a fast upgrade in the tarball. The binary present in /dist/ is just there for a fast upgrade
when you need one. when you need one.
I suppose this tarball imapsync-1.977.tgz is downloaded under your $HOME directory, I suppose this tarball imapsync-2.140.tgz is downloaded under your $HOME directory,
let say /Users/gilles/, but you can put it anywhere. In real, your $HOME directory let say /Users/gilles/, but you can put it anywhere. In real, your $HOME directory
is not /Users/gilles/, it maybe /Users/john/ or /Users/zoey/ where John or Zoey is not /Users/gilles/, it maybe /Users/john/ or /Users/zoey/ where John or Zoey
is you login name. is you login name.
@ -45,19 +45,19 @@ Untar the tarball:
cd cd
pwd pwd
tar xzvf imapsync-1.977.tgz tar xzvf imapsync-2.140.tgz
In case the previous command fails, it means the tarball file called imapsync-1.977.tgz In case the previous command fails, it means the tarball file called imapsync-2.140.tgz
is not in your $HOME directory, you may have downloaded it elsewhere on the file system. is not in your $HOME directory, you may have downloaded it elsewhere on the file system.
A way to find it is the command: A way to find it is the command:
find / | grep imapsync find / | grep imapsync
Now that the tarball is extracted, it created a directory called imapsync-1.977/ Now that the tarball is extracted, it created a directory called imapsync-2.140/
Go into the directory imapsync-1.977 with the command: Go into the directory imapsync-2.140 with the command:
cd imapsync-1.977 cd imapsync-2.140
First let's have a simple run to see if imapsync_bin_Darwin works. First let's have a simple run to see if imapsync_bin_Darwin works.
@ -178,18 +178,19 @@ Install the "make" command in case it is not already installed.
First let us install cpanminus locally in ~/perl5 First let us install cpanminus locally in ~/perl5
wget --no-check-certificate -O- http://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib curl -L https://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib
Then take this install into account in the current environment Then take this install into account in the current environment
eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib` perl -I ~/perl5/lib/perl5 -Mlocal::lib # just to see the variables
perl -I ~/perl5/lib/perl5 -Mlocal::lib eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib` # doing the variables assigments
If you want to have always this setting in your environment then run the commands If you want to have always this setting in your environment then run the commands
echo 'eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`' >> ~/.profile echo 'eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`' >> ~/.profile
echo 'export MANPATH=$HOME/perl5/man:$MANPATH' >> ~/.profile echo 'export MANPATH=$HOME/perl5/man:$MANPATH' >> ~/.profile
cat ~/.profile cat ~/.profile
. ~/.profile
Now let's update the standard CPAN Perl module Now let's update the standard CPAN Perl module
@ -202,18 +203,26 @@ helps to verify what is needed to install on your system
sh prerequisites_imapsync sh prerequisites_imapsync
cpanm Authen::NTLM cpanm Authen::NTLM
cpanm File::Copy::Recursive IO::Tee cpanm IO::Tee
cpanm Mail::IMAPClient cpanm Mail::IMAPClient # File::Copy::Recursive?
cpanm Readonly
cpanm Unicode::String cpanm Unicode::String
cpanm Sys::MemInfo
cpanm File::Tail
cpanm Test::MockObject
cpanm Readonly
cpanm Data::Uniqid cpanm Data::Uniqid
cpanm JSON::WebToken cpanm JSON::WebToken
We're ready to install and test the latest imapsync We're ready to install and test the latest imapsync
wget -c http://imapsync.lamiral.info/imapsync curl -L http://imapsync.lamiral.info/imapsync > imapsync
chmod +x imapsync
./imapsync ./imapsync
./imapsync --modules ./imapsync --testslive
./imapsync --tests
You can rerun "sh prerequisites_imapsync" You can rerun "sh prerequisites_imapsync"
to verify everything is ok: to verify everything is ok:
@ -223,10 +232,6 @@ to verify everything is ok:
When everything is ok the script execution ends with this sentence When everything is ok the script execution ends with this sentence
"All needed modules are already installed" "All needed modules are already installed"
Now imapsync should work on your system.
./imapsync --testslive
======================================================================= =======================================================================
E) Building imapsync binary on Darwin / Mac OS X E) Building imapsync binary on Darwin / Mac OS X
======================================================================= =======================================================================
@ -234,7 +239,7 @@ E) Building imapsync binary on Darwin / Mac OS X
cpanm Module::ScanDeps cpanm Module::ScanDeps
cpanm PAR::Packer cpanm PAR::Packer
pp -o imapsync.bin imapsync pp -x -u -o imapsync.bin imapsync
./imapsync.bin ./imapsync.bin
./imapsync.bin --testslive ./imapsync.bin --testslive

127
INSTALL.d/INSTALL.Debian.txt
View file

@ -1,24 +1,26 @@
#!/bin/cat #!/bin/cat
# $Id: INSTALL.Debian.txt,v 1.32 2020/12/11 09:27:29 gilles Exp gilles $ # $Id: INSTALL.Debian.txt,v 1.35 2021/12/17 20:30:33 gilles Exp gilles $
This documentation is also located online at This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/ https://imapsync.lamiral.info/INSTALL.d/
https://imapsync.lamiral.info/INSTALL.d/INSTALL.Debian.txt https://imapsync.lamiral.info/INSTALL.d/INSTALL.Debian.txt
There is three install sections in this document, There is four install sections in this document,
1) one for Debian 9 Stretch and Debian 10 Buster 1) one for Debian 9 Stretch, Debian 10 Buster, Debian 11 Bullseye
2) one for Debian 8 Jessie 2) one for all the Debian called "After installing the dependencies"
3) one for Debian 7 Wheezy 3) one for Debian 8 Jessie (obsolete)
4) one for Debian 7 Wheezy (even more obsolete)
After reading and following 1) or 2) or 3), you'll be invited to read and follow After reading and following 1) or 3) or 4), you'll be invited to read and follow
the fourth section "4) After installing the dependencies" the section "2) After installing the dependencies"
What is your Debian release? What is your Debian release?
Type on a terminal: Type on a terminal:
cat /etc/debian_version cat /etc/debian_version
11.x => Bullseye
10.x => Buster 10.x => Buster
9.x => Stretch 9.x => Stretch
8.x => Jessie 8.x => Jessie
@ -27,20 +29,20 @@ Type on a terminal:
where x is any number between 0 and 9, x is minor and to be forgotten. where x is any number between 0 and 9, x is minor and to be forgotten.
First a call to Debian packagers First, a call to Debian packagers
Why imapsync is not in Debian as a package? Why imapsync is not in Debian as a package?
Imapsync used to be in Debian from 2005-04-25 (release 1.125) to 2011-01-25 (release 1.315). Imapsync used to be in Debian from 2005-04-25 (release 1.125) to 2011-01-25 (release 1.315).
It could be in Debian nowadays, as always, my last words on this discussion were It could be in Debian nowadays, as always, my last words on this discussion were
"Do what you want, I promise I won't complain anymore about the fact imapsync is on Debian or not." "Do what you want, I promise I won't complain anymore about the fact imapsync is in Debian or not."
https://lists.debian.org/debian-legal/2011/01/msg00058.html https://lists.debian.org/debian-legal/2011/01/msg00058.html
The license is now "No limits to do anything with this work and this license". The Imapsync license is "No limits to do anything with this work and this license".
Feel free. So feel free!
See also a more detailed story at See also a more detailed story at
https://lists.debian.org/debian-user/2016/11/msg00849.html https://lists.debian.org/debian-user/2016/11/msg00849.html
======================================================================= =======================================================================
1) Installing imapsync on Debian 9 Stretch or Debian 10 Buster 1) Installing imapsync on Debian 9 Stretch, 10 Buster, 11 Bullseye
======================================================================= =======================================================================
How to install imapsync dependencies on a Debian system? How to install imapsync dependencies on a Debian system?
@ -48,14 +50,19 @@ How to install imapsync dependencies on a Debian system?
The repositories are the classical ones, ie, a The repositories are the classical ones, ie, a
/etc/apt/sources.list file with: /etc/apt/sources.list file with:
# Debian 9 Stretch # Debian 11 Bullseye
deb http://deb.debian.org/debian stretch main contrib deb http://deb.debian.org/debian bullseye main contrib
or or
# Debian 10 Buster # Debian 10 Buster
deb http://deb.debian.org/debian buster main contrib deb http://deb.debian.org/debian buster main contrib
or
# Debian 9 Stretch
deb http://deb.debian.org/debian stretch main contrib
You need root privilege to run it, be root or use sudo. You need root privilege to run it, be root or use sudo.
@ -88,16 +95,60 @@ apt install -y \
libtest-deep-perl \ libtest-deep-perl \
libtest-warn-perl \ libtest-warn-perl \
make \ make \
time \
cpanminus cpanminus
After installing the dependencies, imapsync should be working, After installing the dependencies, imapsync should be working,
go to the section "4) After installing the dependencies" below. go to the section "2) After installing the dependencies" below.
=======================================================================
2) After installing the dependencies
=======================================================================
You don't have to be root to test and use imapsync.
Take imapsync either on github:
wget -N https://github.com/imapsync/imapsync/blob/master/imapsync
or be on the cutting edge with the upstream site:
wget -N https://imapsync.lamiral.info/imapsync
Add execution permission to the downloaded script:
chmod +x imapsync
Check the dependencies and print also the basic example:
./imapsync
Perform a live test showing imapsync job:
./imapsync --testslive
Now install imapsync on the system (need root priviledges again):
cp imapsync /usr/bin/
That's finished for the installation part.
You can now use imapsync without knowing where it is located
on the system:
imapsync
Now go read the menu http://imapsync.lamiral.info/#doc
and start with the tutorial
https://imapsync.lamiral.info/doc/TUTORIAL_Unix.html
Feedback is welcome!
=============================================
2) Installing imapsync on Debian 8 Jessie =======================================================================
============================================= 3) Installing imapsync on Debian 8 Jessie
=======================================================================
How to install imapsync dependencies on a Debian system? How to install imapsync dependencies on a Debian system?
@ -165,11 +216,11 @@ the module JSON::WebToken with the command:
cpanm JSON::WebToken cpanm JSON::WebToken
After installing the dependencies, imapsync should be working, After installing the dependencies, imapsync should be working,
go to the section "4) After installing the dependencies" below. go to the section "2) After installing the dependencies" above.
======================================================================= =======================================================================
3) Installing imapsync on Debian 7 Wheezy 4) Installing imapsync on Debian 7 Wheezy
======================================================================= =======================================================================
How to install imapsync dependencies on a Debian system? How to install imapsync dependencies on a Debian system?
@ -234,40 +285,6 @@ the module JSON::WebToken with the command:
After installing the dependencies, imapsync should be working, After installing the dependencies, imapsync should be working,
then go to the section "4) After installing the dependencies" below. then go to the section "2) After installing the dependencies" above.
=======================================
4) After installing the dependencies
=======================================
You don't have to be root to test and use imapsync.
Take imapsync either on github or at the upstream site.
Add execution permission to the downloaded script:
chmod +x imapsync
Check the dependencies and print also the basic example:
./imapsync
Perform a live test showing imapsync job:
./imapsync --testslive
Now install imapsync on the system (need root priviledges again):
cp imapsync /usr/bin/
That's finished for the installation part.
You can now use imapsync without knowing where it is located
on the system:
imapsync
Now go to read http://imapsync.lamiral.info/#doc
and start with the tutorial.
======================================================================= =======================================================================
=======================================================================

44
INSTALL.d/INSTALL.Docker_build.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
# $Id: INSTALL.Docker_build.txt,v 1.12 2019/11/28 14:43:48 gilles Exp gilles $ # $Id: INSTALL.Docker_build.txt,v 1.15 2021/11/29 11:56:11 gilles Exp gilles $
This documentation is also located online at This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/ https://imapsync.lamiral.info/INSTALL.d/
@ -10,22 +10,27 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.Docker_build.txt
======================================================================= =======================================================================
Nearly verbatim copy from https://imapsync.lamiral.info/INSTALL.d/Dockerfile Nearly verbatim copy from https://imapsync.lamiral.info/INSTALL.d/Dockerfile
(nearly but the Dockerfile rcs tag Id: removed, that would conflict) (verbatim but the Dockerfile rcs tag $ removed because it would conflict)
## Dockerfile for building a docker imapsync image ## Dockerfile for building a docker imapsync image
# Dockerfile,v 1.26 2019/11/27 15:49:13 gilles Exp gilles $ # Id: Dockerfile,v 1.38 2021/11/29 10:20:56 gilles Exp gilles $
# I use the following command to build the image: # I use the following command to build the image:
# #
# docker build -t gilleslamiral/imapsync . # docker build -t gilleslamiral/imapsync .
# #
# where this Dockerfile is in the current directory # where this Dockerfile is in the current directory
# #
# I thank you very much
# I like thanks # I like thanks
# I like stars # I like stars
# I also like (and need) money # I also need money to keep on doing this stuff
# I thank you very much in advance
# Number of imapsync images pulled so far (2021_11_29): 608326
# Command used:
# curl -s https://hub.docker.com/v2/repositories/gilleslamiral/imapsync/ | jq '.pull_count'
# Debian Buster is Debian 10
FROM debian:buster FROM debian:buster
@ -33,16 +38,13 @@ LABEL maintainer="Gilles LAMIRAL <gilles@lamiral.info>" \
description="Imapsync" \ description="Imapsync" \
documentation="https://imapsync.lamiral.info/#doc" documentation="https://imapsync.lamiral.info/#doc"
# Put a copy of the Dockerfile in the image itself # I put a copy of the Dockerfile in the image itself
# It can help future maintenance, isn't it? # It can help maintenance, isn't it?
# Also put optionally my local and usually more recent imapsync on /, for testing purpose
COPY Dockerfile / COPY Dockerfile imapsyn[c] prerequisites_imapsyn[c] /
# Also put my local imapsync on /usr/bin/ for testing purpose
# renamed imapsync_local since would be erased by the wget
COPY imapsync /usr/bin/imapsync_local
COPY prerequisites_imapsync /usr/bin/prerequisites_imapsync_local
RUN set -xe && \ RUN set -xe && \
apt-get update \ apt-get update \
@ -53,7 +55,8 @@ RUN set -xe && \
libdata-uniqid-perl \ libdata-uniqid-perl \
libencode-imaputf7-perl \ libencode-imaputf7-perl \
libfile-copy-recursive-perl \ libfile-copy-recursive-perl \
libfile-tail-perl \ libfile-tail-perl \
libio-compress-perl \
libio-socket-ssl-perl \ libio-socket-ssl-perl \
libio-socket-inet6-perl \ libio-socket-inet6-perl \
libio-tee-perl \ libio-tee-perl \
@ -77,15 +80,19 @@ RUN set -xe && \
make \ make \
cpanminus \ cpanminus \
lsof \ lsof \
ncat \
openssl \
ca-certificates \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
RUN set -xe \ RUN set -xe \
&& cd /usr/bin/ \
&& pwd \ && pwd \
&& wget -N https://imapsync.lamiral.info/imapsync \ && wget -N --no-check-certificate https://imapsync.lamiral.info/imapsync \
https://imapsync.lamiral.info/prerequisites_imapsync \ https://imapsync.lamiral.info/prerequisites_imapsync \
&& mv imapsync /usr/bin/imapsync \ https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/python/oauth2.py \
&& chmod +x /usr/bin/imapsync /usr/bin/imapsync_local # just_a_comment_to_force_update 2019_11_26_12_58_34 && chmod +x imapsync oauth2.py \
&& /usr/bin/imapsync --testslive && /usr/bin/imapsync --tests # just_a_comment_to_force_update 2021_11_29_11_20_56
USER nobody:nogroup USER nobody:nogroup
@ -102,6 +109,5 @@ CMD ["/usr/bin/imapsync"]
======================================================================= =======================================================================
======================================================================= =======================================================================

9
INSTALL.d/INSTALL.OnlineUI.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
# $Id: INSTALL.OnlineUI.txt,v 1.49 2021/03/19 09:39:01 gilles Exp gilles $ # $Id: INSTALL.OnlineUI.txt,v 1.50 2021/09/20 13:47:20 gilles Exp gilles $
This documentation is also located online at This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/ https://imapsync.lamiral.info/INSTALL.d/
@ -11,7 +11,7 @@ https://imapsync.lamiral.info/INSTALL.d/INSTALL.OnlineUI.txt
I'm now confident with /X since the /X service is up and running quite I'm now confident with /X since the /X service is up and running quite
well since January 2017. Anyway, if you run this service on your own well since January 2017. Anyway, if you run this service on your own,
online, you take responsibility for it. online, you take responsibility for it.
======================================================================= =======================================================================
@ -135,7 +135,10 @@ Now that I have explained the general context for any system,
I'll describe concrete examples on several systems, I'll describe concrete examples on several systems,
Debian/Ubuntu and Centos. Feedbacks show that the Centos Debian/Ubuntu and Centos. Feedbacks show that the Centos
process is easier in case you don't know very much any of the process is easier in case you don't know very much any of the
Linux distributions. Linux distributions. I add that if you don't know very much any
of the Linux distributions then you shouldn't install this
imapsync service at all.
============================================================================= =============================================================================
A) Concrete example on a Debian server with Apache: A) Concrete example on a Debian server with Apache:

3
INSTALL.d/INSTALL.Ubuntu.txt
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
# $Id: INSTALL.Ubuntu.txt,v 1.17 2019/11/28 14:35:43 gilles Exp gilles $ # $Id: INSTALL.Ubuntu.txt,v 1.18 2021/07/30 15:06:43 gilles Exp gilles $
This documentation is also located online at This documentation is also located online at
https://imapsync.lamiral.info/INSTALL.d/ https://imapsync.lamiral.info/INSTALL.d/
@ -49,6 +49,7 @@ libtest-nowarnings-perl \
libtest-deep-perl \ libtest-deep-perl \
libtest-warn-perl \ libtest-warn-perl \
make \ make \
time \
cpanminus cpanminus
In case you want to update the Perl module In case you want to update the Perl module

18
INSTALL.d/memo_docker
View file

@ -1,5 +1,7 @@
#!/bin/sh #!/bin/sh
# $Id: memo_docker,v 1.15 2021/11/20 20:52:55 gilles Exp gilles $
echo imapsync_docker_timestamp_dockerfile echo imapsync_docker_timestamp_dockerfile
imapsync_docker_timestamp_dockerfile() imapsync_docker_timestamp_dockerfile()
{ {
@ -27,6 +29,12 @@ imapsync_docker_version()
docker run gilleslamiral/imapsync imapsync --version docker run gilleslamiral/imapsync imapsync --version
} }
echo imapsync_local_docker_version
imapsync_local_docker_version()
{
docker run gilleslamiral/imapsync imapsync_local --version
}
echo imapsync_docker_add_tag_version echo imapsync_docker_add_tag_version
imapsync_docker_add_tag_version() imapsync_docker_add_tag_version()
{ {
@ -102,13 +110,21 @@ imapsync_docker_rebuild_from_scratch()
imapsync_docker_build imapsync_docker_build
} }
echo is_release_number
is_release_number()
{
echo "$1" | perl -pe 'm/^\d\.\d+$/ || exit 1'
}
echo imapsync_docker_upload echo imapsync_docker_upload
imapsync_docker_upload() imapsync_docker_upload()
{ {
version=`imapsync_docker_version` version=`imapsync_docker_version`
# upload only after successful tests # upload only after successful tests
set -x \ set -x \
&& rcsdiff imapsync \ && is_release_number "$version" \
&& rcsdiff imapsync Dockerfile memo_docker \
&& imapsync_docker_testslive \ && imapsync_docker_testslive \
&& imapsync_docker_tests \ && imapsync_docker_tests \
&& docker run gilleslamiral/imapsync true \ && docker run gilleslamiral/imapsync true \

68
Makefile
View file

@ -1,5 +1,5 @@
# $Id: Makefile,v 1.335 2021/07/06 02:00:46 gilles Exp gilles $ # $Id: Makefile,v 1.339 2022/01/14 14:28:38 gilles Exp gilles $
.PHONY: help usage all doc .PHONY: help usage all doc
@ -34,6 +34,7 @@ usage:
@echo "make upload_index" @echo "make upload_index"
@echo "make upload_FAQ # upload FAQs and documentation" @echo "make upload_FAQ # upload FAQs and documentation"
@echo "make upload_X # upload online UI" @echo "make upload_X # upload online UI"
@echo "make upload_csv # upload online CSV service"
@echo "make upload_latest # upload latest imapsync and binaries (dev)" @echo "make upload_latest # upload latest imapsync and binaries (dev)"
@echo "make upload_cgi # upload latest imapsync online, after local and remote --tests success." @echo "make upload_cgi # upload latest imapsync online, after local and remote --tests success."
@echo "make upload_cgi_memo # upload cgi_memo stat_patterns.txt to /X servers." @echo "make upload_cgi_memo # upload cgi_memo stat_patterns.txt to /X servers."
@ -150,7 +151,7 @@ clean_bak:
rm -f index.shtml.bak ./S/style.css.bak rm -f index.shtml.bak ./S/style.css.bak
clean_oauth2: clean_oauth2:
rm oauth2/typescript oauth2/D_*txt rm -f oauth2/typescript oauth2/D_*txt
.PHONY: install dist man .PHONY: install dist man
@ -160,6 +161,7 @@ clean_man:
rm -f W/imapsync.1 rm -f W/imapsync.1
W/imapsync.1: imapsync W/imapsync.1: imapsync
mkdir -p W
pod2man imapsync > W/imapsync.1 pod2man imapsync > W/imapsync.1
install: testp W/imapsync.1 install: testp W/imapsync.1
@ -188,7 +190,7 @@ docker:
@echo "make docker_upload_docker_hub # upload last build to https://hub.docker.com/r/gilleslamiral/imapsync" @echo "make docker_upload_docker_hub # upload last build to https://hub.docker.com/r/gilleslamiral/imapsync"
@echo "ssh vp3 THEN cd docker/imapsync && . memo_docker" @echo "ssh vp3 THEN cd docker/imapsync && . memo_docker"
docker_build: docker_build: docker_copy_to_vp3
ssh vp3 'cd docker/imapsync && . memo_docker && imapsync_docker_build' ssh vp3 'cd docker/imapsync && . memo_docker && imapsync_docker_build'
@ -199,7 +201,7 @@ docker_copy_to_vp3:
rsync -av imapsync INSTALL.d/Dockerfile INSTALL.d/memo_docker INSTALL.d/prerequisites_imapsync INSTALL.d/secret.txt vp3:docker/imapsync/ rsync -av imapsync INSTALL.d/Dockerfile INSTALL.d/memo_docker INSTALL.d/prerequisites_imapsync INSTALL.d/secret.txt vp3:docker/imapsync/
rsync -av RCS/imapsync,v INSTALL.d/RCS/Dockerfile,v INSTALL.d/RCS/memo_docker,v vp3:docker/imapsync/RCS/ rsync -av RCS/imapsync,v INSTALL.d/RCS/Dockerfile,v INSTALL.d/RCS/memo_docker,v vp3:docker/imapsync/RCS/
docker_upload_docker_hub: docker_upload_docker_hub: docker_build
ssh vp3 'cd docker/imapsync && . memo_docker && imapsync_docker_upload' ssh vp3 'cd docker/imapsync && . memo_docker && imapsync_docker_upload'
functree: W/imapsync_functions_tree_ppi.txt W/imapsync_functions_tree.txt functree: W/imapsync_functions_tree_ppi.txt W/imapsync_functions_tree.txt
@ -517,18 +519,18 @@ W/test_tail.bat:
win64sshaccess: win64sshaccess:
ssh 'pc HP DV7'@p24 'perl -V' ssh 'pc HP DV7'@p24 'perl -V'
winprepalocal:
win64prepa:
ssh 'pc HP DV7'@p24 'perl -V'
unix2dos W/build_exe.bat W/test_exe.bat W/install_modules.bat W/install_module_one.bat unix2dos W/build_exe.bat W/test_exe.bat W/install_modules.bat W/install_module_one.bat
win64prepa: winprepalocal
ssh 'pc HP DV7'@p24 'perl -V'
scp imapsync W/build_exe.bat W/install_modules.bat W/install_module_one.bat \ scp imapsync W/build_exe.bat W/install_modules.bat W/install_module_one.bat \
W/test_exe_tests.bat W/test_exe_testsdebug.bat W/test_exe.bat \ W/test_exe_tests.bat W/test_exe_testsdebug.bat W/test_exe.bat \
pc_HP_DV7_p24:'Desktop/imapsync_build' pc_HP_DV7_p24:'Desktop/imapsync_build'
ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat' ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat'
./W/check_win64err build_exe.bat ./W/check_win64err build_exe.bat
win64build: win64build: winprepalocal
unix2dos W/build_exe.bat W/install_modules.bat
scp imapsync W/build_exe.bat W/install_modules.bat pc_HP_DV7_p24:'Desktop/imapsync_build' scp imapsync W/build_exe.bat W/install_modules.bat pc_HP_DV7_p24:'Desktop/imapsync_build'
ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat' ssh 'pc HP DV7'@p24 'Desktop/imapsync_build/build_exe.bat'
./W/check_win64err build_exe.bat ./W/check_win64err build_exe.bat
@ -537,10 +539,9 @@ imapsync.exe: imapsync_64bit.exe
cp -a imapsync_64bit.exe imapsync.exe cp -a imapsync_64bit.exe imapsync.exe
imapsync_64bit.exe: imapsync imapsync_64bit.exe: imapsync winprepalocal
(date "+%s"| tr "\n" " "; echo -n "BEGIN 64bit " $(VERSION) ": "; date) >> W/.BUILD_EXE_TIME (date "+%s"| tr "\n" " "; echo -n "BEGIN 64bit " $(VERSION) ": "; date) >> W/.BUILD_EXE_TIME
ssh 'pc HP DV7'@p24 'perl -V' ssh 'pc HP DV7'@p24 'perl -V'
unix2dos W/build_exe.bat W/test_exe.bat W/install_modules.bat W/install_module_one.bat
scp imapsync W/build_exe.bat W/install_modules.bat W/install_module_one.bat \ scp imapsync W/build_exe.bat W/install_modules.bat W/install_module_one.bat \
W/test_exe_tests.bat W/test_exe_testsdebug.bat W/test_exe.bat \ W/test_exe_tests.bat W/test_exe_testsdebug.bat W/test_exe.bat \
pc_HP_DV7_p24:'Desktop/imapsync_build' pc_HP_DV7_p24:'Desktop/imapsync_build'
@ -565,7 +566,7 @@ zip: dosify_bat
unix2dos ../prepa_zip/imapsync_$(VERSION)/*.txt unix2dos ../prepa_zip/imapsync_$(VERSION)/*.txt
cd ../prepa_zip/ && rm -f ./imapsync_$(VERSION).zip && zip -r ./imapsync_$(VERSION).zip ./imapsync_$(VERSION)/ cd ../prepa_zip/ && rm -f ./imapsync_$(VERSION).zip && zip -r ./imapsync_$(VERSION).zip ./imapsync_$(VERSION)/
scp ../prepa_zip/imapsync_$(VERSION).zip Admin@c:'C:/msys/1.0/home/Admin/' scp ../prepa_zip/imapsync_$(VERSION).zip Admin@c:'C:/msys/1.0/home/Admin/'
scp ../prepa_zip/imapsync_$(VERSION).zip pc_HP_DV7_p24:'Desktop/' # scp ../prepa_zip/imapsync_$(VERSION).zip pc_HP_DV7_p24:'Desktop/'
cp ../prepa_zip/imapsync_$(VERSION).zip /fe/imapsync/ cp ../prepa_zip/imapsync_$(VERSION).zip /fe/imapsync/
@ -575,19 +576,22 @@ zip: dosify_bat
mac: imapsync_bin_Darwin mac: imapsync_bin_Darwin
maccopy: macstadiumcopy:
rsync -p -e 'ssh -4 -p 995' imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync webserver \ rsync -v imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync webserver administrator@macstadium.lamiral.info:
gilleslamira@gate.polarhome.com: rsync -v examples/file.txt examples/sync_loop_darwin.sh administrator@macstadium.lamiral.info:examples/
rsync -av -p -e 'ssh -4 -p 995' X/ gilleslamira@gate.polarhome.com:X/ rsync -v X/ administrator@macstadium.lamiral.info:X/
maccopy:
rsync -v -p -e 'ssh -4 -p 995' imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync webserver \
gilleslamira@gate.polarhome.com:
rsync -v -p -e 'ssh -4 -p 995' examples/file.txt examples/sync_loop_darwin.sh gilleslamira@gate.polarhome.com:examples/
rsync -v -p -e 'ssh -4 -p 995' X/ gilleslamira@gate.polarhome.com:X/
macforce: maccopy macforce: maccopy
ssh -4 -p 995 gilleslamira@gate.polarhome.com 'sh -x build_mac.sh' ssh -4 -p 995 gilleslamira@gate.polarhome.com 'sh -x build_mac.sh'
imapsync_bin_Darwin: imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync maccopy
imapsync_bin_Darwin: imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync
rcsdiff imapsync rcsdiff imapsync
rsync -p -e 'ssh -4 -p 995' imapsync W/build_mac.sh INSTALL.d/prerequisites_imapsync gilleslamira@gate.polarhome.com:
ssh -4 -p 995 gilleslamira@gate.polarhome.com 'sh -x build_mac.sh' ssh -4 -p 995 gilleslamira@gate.polarhome.com 'sh -x build_mac.sh'
rsync -P -e 'ssh -4 -p 995' gilleslamira@gate.polarhome.com:imapsync_bin_Darwin . rsync -P -e 'ssh -4 -p 995' gilleslamira@gate.polarhome.com:imapsync_bin_Darwin .
@ -714,12 +718,12 @@ S/imapsync_sold_by_country.txt: /g/bin/imapsync_by_country
ks: ks:
rsync -avHz --delete --exclude '*.exe' \ rsync -avHz --delete --exclude '*.exe' \
. gilles@ks.lamiral.info:public_html/imapsync/ . gilles@ks.lamiral.info:public_html/imapsync/
ssh root@ks.lamiral.info 'apachectl configtest && /etc/init.d/apache2 reload' ssh root@ks.lamiral.info 'apachectl configtest && apachectl reload'
ksa: ksa:
rsync -avHz --delete -P \ rsync -avHz --delete -P \
. gilles@ks.lamiral.info:public_html/imapsync/ . gilles@ks.lamiral.info:public_html/imapsync/
ssh root@ks.lamiral.info 'apachectl configtest && /etc/init.d/apache2 reload' ssh root@ks.lamiral.info 'apachectl configtest && apachectl reload'
ks3: ks3:
rsync -avHz --delete -P \ rsync -avHz --delete -P \
@ -745,11 +749,10 @@ ks5tests: ks5tests_gilles ks5tests_root
ks5tests_gilles: ks5tests_gilles:
rsync -P imapsync gilles@ks.lamiral.info:public_html/imapsync/ rsync -P imapsync gilles@ks.lamiral.info:public_html/imapsync/
rsync -P oauth2_access_token.txt oauth2_string_for_oauthdirect.txt gilles@ks.lamiral.info:
ssh gilles@ks.lamiral.info 'public_html/imapsync/imapsync --tests' ssh gilles@ks.lamiral.info 'public_html/imapsync/imapsync --tests'
ks5tests_root: ks5tests_root:
rsync -P imapsync oauth2_access_token.txt oauth2_string_for_oauthdirect.txt root@ks5.lamiral.info: rsync -P imapsync root@ks5.lamiral.info:
ssh root@ks5.lamiral.info './imapsync --tests' ssh root@ks5.lamiral.info './imapsync --tests'
@ -830,14 +833,16 @@ W/.valid.index.shtml: index.shtml S/*.shtml
upload_index: valid_index clean_permissions upload_index: valid_index clean_permissions
rcsdiff index.shtml README_Windows.txt S/style.css S/*.shtml FAQ.d/*.txt LICENSE CREDITS TODO examples/*.bat examples/*.sh index.shtml INSTALL.d/*.txt rcsdiff index.shtml README_Windows.txt S/style.css S/*.shtml FAQ.d/*.txt LICENSE CREDITS TODO examples/*.bat examples/*.sh index.shtml INSTALL.d/*.txt
rcsdiff S/quiz/quiz_imapsync.html S/quiz/quiz_imapsync.js S/quiz/quiz_imapsync.css
rm -f examples/LOG_imapsync/* rm -f examples/LOG_imapsync/*
rsync -avH index.shtml README_Windows.txt FAQ INSTALL NOLIMIT LICENSE CREDITS TODO S/robots.txt S/favicon.ico ../imapsync_website/ rsync -avH index.shtml README_Windows.txt FAQ INSTALL NOLIMIT LICENSE CREDITS TODO S/robots.txt S/favicon.ico ../imapsync_website/
rsync -aHv --delete ./W/ks.htaccess ../imapsync_website/.htaccess rsync -aHv --delete ./W/ks.htaccess ../imapsync_website/.htaccess
rsync -aHv --delete S/ ../imapsync_website/S/ rsync -aHv --delete ./S/ ../imapsync_website/S/
rsync -aHv --delete ./examples/ ../imapsync_website/examples/ rsync -aHv --delete ./examples/ ../imapsync_website/examples/
rsync -aHv --delete ./INSTALL.d/ ../imapsync_website/INSTALL.d/ rsync -aHv --delete ./INSTALL.d/ ../imapsync_website/INSTALL.d/
rsync -aHv --delete ./FAQ.d/ ../imapsync_website/FAQ.d/ rsync -aHv --delete ./FAQ.d/ ../imapsync_website/FAQ.d/
rsync -avH --delete ./doc/ ../imapsync_website/doc/ rsync -avH --delete ./doc/ ../imapsync_website/doc/
rsync -avH --delete ./W/tools/ ../imapsync_website/W/tools/
rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/ rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/
@ -853,7 +858,7 @@ upload_latest: unitests ci_imapsync bin
upload_cgi: unitests ks5tests ks5tests_root ci_imapsync upload_cgi: unitests ks5tests ks5tests_root ci_imapsync
rsync -a imapsync ./INSTALL.d/prerequisites_imapsync ../imapsync_website/ rsync -a imapsync ./INSTALL.d/prerequisites_imapsync ../imapsync_website/
rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/ rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/
rsync -P imapsync root@ks5.lamiral.info:/home/www/apache24/cgi-bin/imapsync rsync -P imapsync root@ks5.lamiral.info:/home/www/apache24/cgi-bin/
upload_cgi_memo: upload_cgi_memo:
dos2unix X/stat_patterns.txt X/server_survey_patterns.txt dos2unix X/stat_patterns.txt X/server_survey_patterns.txt
@ -863,13 +868,21 @@ upload_cgi_memo:
upload_X: upload_X:
./W/tools/validate_xml_html5 X/index.html X/imapsync_form.html X/imapsync_form_extra.html X/imapsync_form_extra_free.html X/imapsync_form_wrapper.html ./W/tools/validate_xml_html5 X/index.html X/imapsync_form.html X/imapsync_form_extra.html X/imapsync_form_extra_free.html X/imapsync_form_wrapper.html
rcsdiff X/imapsync_form.html X/imapsync_form_extra.html X/imapsync_form_wrapper.html X/imapsync_form.css X/noscript.css rcsdiff X/imapsync_form.html X/imapsync_form_extra.html X/imapsync_form_extra_free.html X/imapsync_form_wrapper.html X/imapsync_form.css X/noscript.css
rcsdiff X/imapsync_form.js X/imapsync_form_wrapper.js rcsdiff X/imapsync_form.js X/imapsync_form_wrapper.js
rcsdiff INSTALL.d/INSTALL.OnlineUI.txt rcsdiff INSTALL.d/INSTALL.OnlineUI.txt
rsync -a ./INSTALL.d/INSTALL.OnlineUI.txt ../imapsync_website/INSTALL.d/INSTALL.OnlineUI.txt rsync -a ./INSTALL.d/INSTALL.OnlineUI.txt ../imapsync_website/INSTALL.d/INSTALL.OnlineUI.txt
rsync -av --delete X/ ../imapsync_website/X/ rsync -av --delete X/ ../imapsync_website/X/
rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/ rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/
upload_csv:
./W/tools/validate_xml_html5 X/sandbox_csv.html
rcsdiff X/sandbox_csv.html X/sandbox_csv.js X/imapsync_csv_wrapper
rsync -a X/sandbox_csv.html X/sandbox_csv.js X/imapsync_csv_wrapper ../imapsync_website/X/
rsync -aHvz X/sandbox_csv.html X/sandbox_csv.js X/imapsync_csv_wrapper root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/X/
rsync X/imapsync_csv_wrapper root@ks5.lamiral.info:/home/www/apache24/cgi-bin/
upload_FAQ: upload_FAQ:
rcsdiff FAQ.d/*.txt LICENSE CREDITS TODO INSTALL.d/*.txt rcsdiff FAQ.d/*.txt LICENSE CREDITS TODO INSTALL.d/*.txt
rsync -avH FAQ INSTALL CREDITS TODO ../imapsync_website/ rsync -avH FAQ INSTALL CREDITS TODO ../imapsync_website/
@ -891,8 +904,9 @@ upload_ks: ci tarball
rsync -aHv --delete ./INSTALL.d/ ../imapsync_website/INSTALL.d/ rsync -aHv --delete ./INSTALL.d/ ../imapsync_website/INSTALL.d/
rsync -aHv --delete ./FAQ.d/ ../imapsync_website/FAQ.d/ rsync -aHv --delete ./FAQ.d/ ../imapsync_website/FAQ.d/
rsync -avH --delete ./doc/ ../imapsync_website/doc/ rsync -avH --delete ./doc/ ../imapsync_website/doc/
rsync -avH --delete ./W/tools/ ../imapsync_website/W/tools/
rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/ rsync -aHvz --delete ../imapsync_website/ root@ks5.lamiral.info:/usr/local/www/apache24/data/imapsync/
ssh root@ks.lamiral.info 'apachectl configtest && /etc/init.d/apache2 reload' ssh root@ks.lamiral.info 'apachectl configtest && apachectl reload'
upload_ks5: upload_ks5:

38
README
View file

@ -6,7 +6,7 @@ NAME
VERSION VERSION
This documentation refers to Imapsync $Revision: 2.140 $ This documentation refers to Imapsync $Revision: 2.178 $
USAGE USAGE
@ -182,16 +182,6 @@ OPTIONS
--sslargs2 str : Pass any ssl parameter for host2 ssl or tls connection. --sslargs2 str : Pass any ssl parameter for host2 ssl or tls connection.
See --sslargs1 See --sslargs1
--timeout1 flo : Connection timeout in seconds for host1.
Default is 120 and 0 means no timeout at all.
--timeout2 flo : Connection timeout in seconds for host2.
Default is 120 and 0 means no timeout at all.
Caveat, under CGI context, you may encounter a timeout
from the webserver, killing imapsync and the imap connexions.
See the document INSTALL.OnlineUI.txt and search
for "Timeout" for how to deal with this issue.
OPTIONS/authentication OPTIONS/authentication
--authmech1 str : Auth mechanism to use with host1: --authmech1 str : Auth mechanism to use with host1:
@ -396,6 +386,12 @@ OPTIONS
by spaces. by spaces.
--regexmess reg : and this one, etc. --regexmess reg : and this one, etc.
--truncmess int : truncates messages when their size exceed the int
value, specified in bytes. Good to sync too big
messages or to "suppress" attachments.
Have in mind that this way, messages become
uncoherent somehow.
OPTIONS/labels OPTIONS/labels
Gmail present labels as folders in imap. Imapsync can accelerate the Gmail present labels as folders in imap. Imapsync can accelerate the
@ -595,6 +591,26 @@ OPTIONS
OPTIONS/behavior OPTIONS/behavior
--timeout1 flo : Connection timeout in seconds for host1.
Default is 120 and 0 means no timeout at all.
--timeout2 flo : Connection timeout in seconds for host2.
Default is 120 and 0 means no timeout at all.
Caveat, under CGI context, you may encounter a timeout
from the webserver, killing imapsync and the imap connexions.
See the document INSTALL.OnlineUI.txt and search
for "Timeout" for how to deal with this issue.
--keepalive1 : https://metacpan.org/pod/Mail::IMAPClient#Keepalive
Some firewalls and network gears like to timeout connections
prematurely if the connection sits idle.
This option enables SO_KEEPALIVE on the host1 socket.
--keepalive1 is on by default since imapsync release 2.169
Use --nokeepalive1 to disable it.
--keepalive2 : Same as --keepalive2 but for host2.
Use --nokeepalive2 to disable it.
--maxmessagespersecond flo : limits the average number of messages --maxmessagespersecond flo : limits the average number of messages
transferred per second. transferred per second.

141
README_Windows.txt
View file

@ -1,4 +1,4 @@
# $Id: README_Windows.txt,v 1.24 2021/06/15 10:29:45 gilles Exp gilles $ # $Id: README_Windows.txt,v 1.26 2022/01/14 11:54:28 gilles Exp gilles $
# #
# This is the README_Windows.txt file for imapsync # This is the README_Windows.txt file for imapsync
# imapsync: IMAP syncing and migration tool. # imapsync: IMAP syncing and migration tool.
@ -21,48 +21,52 @@ A) Simplest way
A.0) Preamble for visual users looking for a visual tool. A.0) Preamble for visual users looking for a visual tool.
First, the visual thing I am talking about here is not the online visual First, the visual thing I am talking about here is not the online
interface I call /X that you might have seen and used at the URL visual interface I call /X that you might have seen and used at the
https://imapsync.lamiral.info/X/ URL https://imapsync.lamiral.info/X/
This /X visual interface is not yet available on Windows as a service This /X visual interface is not yet available on Windows as a service
that you can install and use in your system. that you can install and use in your system. I plan to make a visual
I plan to make a visual tool available on Windows but, for now, tool available on Windows but, for now, I encountered technical
I encountered technical issues. issues.
So what visual thing am I talking about? So what visual thing am I talking about?
Imapsync itself is not a visual tool. The visual tool is Notepad or Imapsync itself is not a visual tool. The visual tool is Notepad or
any text editor. Many pure visual users have succeeded in using any text editor. Many pure visual users have succeeded in using
imapsync to transfer their email accounts. As you can guess, only imapsync to transfer their email accounts. As you can guess, only the
the ones that tried have succeeded, so don't give up before ones that tried have succeeded, so don't give up before trying at
trying at least once. Another assumption is that visual users can read. least once. Another assumption is that visual users can read.
Let's go for some reading! Let's go for some reading!
A.1) Get imapsync. A.1) Get imapsync.
Get imapsync at https://imapsync.lamiral.info/ Get imapsync at https://imapsync.lamiral.info/
You'll then have access to a zip archive file named imapsync_1.977.zip You'll then have access to a zip archive file named imapsync_2.178.zip
where 1.977 is the imapsync release number. where 2.178 is the imapsync release number.
A.2) Extract the zip file in a folder where you will work with imapsync. A.2) Extract the zip file in a folder where you will work with imapsync.
You can work on the Desktop since the zip file extraction creates You can work on the Desktop since the zip file extraction creates
a unique folder named imapsync_1.977/ a unique folder named imapsync_2.178/
Do not unzip the archive in what is called a "system" directory since Do not unzip the archive in what is called a "system" directory since
you may encounter permission issues. you may encounter permission issues.
Two-point to have in mind: Two points to have in mind:
* You don't need to be an Administrator to unzip the zip archive. * You don't need to be an Administrator to unzip the zip archive.
* You don't need to be an Administrator to run imapsync. * You don't need to be an Administrator to run imapsync.
In short, any user on your system can use imapsync.
A.3) Check the folder A.3) Check the folder
In the folder extracted imapsync_1.977 you see 7 files and 2 directories, In the folder extracted and called imapsync_2.178, you see 7 files
which may be presented in a different order than the following, the order and 2 directories. Those files and directories may be presented in a
is not important anyway. The really important files to get your mailbox different order than the following, the order is not important
transfer job started in a few minutes are the first two files of the anyway. There are only two important files to get your mailbox
transfer job started in a few minutes, the first two files of the
following list: following list:
* README_Windows.txt is the current file you are reading. * README_Windows.txt is the current file you are reading.
@ -80,90 +84,104 @@ A.3) Check the folder
You can copy or rename the file imapsync_example.bat as you wish, You can copy or rename the file imapsync_example.bat as you wish,
as long as its extension remains ".bat", for example, mysync.bat as long as its extension remains ".bat", for example, mysync.bat
On Windows systems, a file name ending with a .bat extension means "I am a batch script". On Windows systems, a file name ending with a .bat extension means
A batch script is a file containing commands, it's a program. "I'm a batch script". A batch script is a file containing commands,
Don't be afraid, a program can be very simple to modify and I hope it's a program. Don't be afraid, a program can be very simple to
imapsync_example.bat is one of them. modify and I hope imapsync_example.bat is one of them.
The batch scripts have to stay in the same directory as The batch scripts have to stay in the same directory as imapsync.exe
imapsync.exe because of the way they call imapsync.exe. because of the way they call imapsync.exe. They use the string
They use the string ".\imapsync.exe", so let them be in the same directory. ".\imapsync.exe", so let them be in the same directory.
You can change the path .\ to whatever you want if you understand what You can change the path .\ to whatever you want if you understand
you are doing (you have to use a pathname from the script point of view). what you are doing (you have to use a pathname from the script point
of view).
For the rest of this documentation, I assume you copied For the rest of this documentation, I assume you copied
imapsync_example.bat to a file named mysync.bat imapsync_example.bat to a file named mysync.bat
If you don't know how to copy and rename a file then use imapsync_example.bat If you don't know how to copy and rename a file then use
itself, it's ok. The original file is still in the zip file in case imapsync_example.bat itself, it's ok. The original file is still in
you want to restart from scratch. the zip file in case you want to restart from scratch.
A.4) Edit the batch file A.4) Edit the batch file
This section describes how to edit the file mysync.bat to change This section describes how to edit the file mysync.bat and change
the example values with yours. it by replacing example values with your values.
To edit mysync.bat, you have to right-click on it and select "modify" To edit mysync.bat, you have to right-click on it and select "modify"
in the list presented in the small window menu. in the list presented in the small window menu.
Notepad or Notepadd++ are very good editor candidates to modify it. Notepad or Notepadd++ are very good editor candidates to modify the
Notepad is already installed on any Windows system, Notepadd++ script. Notepad is already installed on any Windows system,
is not usually installed but if you have it then use it. Notepadd++ is not usually installed but if you have it, then use it.
Office Word or any powerful text processor is NOT good for that job. Office Word or any powerful text processor is NOT good for that job.
Text processors transform files in a special format that is wrong to Text processors transform files in a special format that is wrong to
make them stay a good batch file, so don't use them! make them stay a good batch file, so don't use them!
Files FAQ.txt and FAQ.d/* contain many tips and special options sometimes The documents FAQ.txt and FAQ.d/* contain many tips. They describe
needed by specific imap software servers like Exchange, Office365, or Gmail. some special options that are sometimes needed by specific imap
Those files are also available online at software servers like Exchange, Office365, or Gmail.
Those documents are also available online at
https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/
You don't have to look into them unless you encounter problems. You don't have to look into them unless you encounter problems.
A.5) Run the batch file A.5) Run the batch file
To run imapsync with your values just double-click on the batch file mysync.bat To run imapsync with your values just double-click on the batch file
mysync.bat
There is no need to have administrator privileges to run imapsync. There is no need to have administrator privileges to run imapsync.
The run happens in a DOS window; usually, this window is black. The run happens in a DOS window; usually, this window is black.
If imapsync.exe returns immediately with the ERRORLEVEL -1, it may be
because you have a Group Policy in place to prevent the execution of
programs in the %TEMP% directory. Temporarily remove this restriction
and imapsync will work as expected. Thanks to Walter H. for this
input!
If imapsync.exe returns immediately with the ERRORLEVEL -1, it may be Technically speaking, imapsync.exe is an embedded Perl script with
because you have an Group Policy in place to prevent execution of also the Perl interpreter and many Perl modules, all glued together
programs in %TEMP%. Temporarily remove this restriction and imapsync in an archive auto-extracted at run time. So it needs write-access to
will work as expected. Thanks to Walter H. for this input! the temporary directory. The temporary directory name depends on the
Technically imapsync.exe is an embedded Perl script with also Perl user but its value is in the variable %TEMP%. You can have the value
and many Perl modules glued together in an archive auto-extracted. by running the command ECHO %TEMP% in a DOS window. You can have a
DOS window by launching the command cmd.exe
A.6) Look at the sync running. A.6) Look at the sync running.
You can abort the sync at any time with a quick double ctrl-c, You can abort the sync at any time with a quick double ctrl-c, hit
ie hit ctrl-c twice within one second. ctrl-c twice within one second.
A single ctrl-c will reconnect to both imap servers. A single ctrl-c will reconnect to both imap servers.
You can also simply abort the sync by closing the DOS window,
using the cross situated at the up-right corner. You can also simply abort the sync by closing the DOS window, using
the cross situated at the up-right corner.
What you see in this DOS terminal is also put in a logfile located What you see in this DOS terminal is also put in a logfile located
in the subdirectory LOG_imapsync/ in the subdirectory LOG_imapsync/
A.7) Control what happened. A.7) Control what happened.
When the sync is finished you can find the whole log file When the sync is finished you can find the whole log file of the
of the output in the folder named "LOG_imapsync/". output in the folder named "LOG_imapsync/".
The logfile name is based on the launching date,
hour, minute, second, milliseconds, plus the user1 and user2 parameters. The logfile name is based on the launching date, hour, minute,
second, milliseconds, plus the user1 and user2 parameters.
For example, a file name can be For example, a file name can be
LOG_imapsync\2019_11_29_14_49_36_514_tata_titi.txt LOG_imapsync\2019_11_29_14_49_36_514_tata_titi.txt
There is one log file created for each run. There is one log file created for each run. The log file name is
The log file name is printed at the end of the imapsync run. printed both at the beginning and the end of the imapsync run.
IMPORTANT: When there is a problem, the problem is very often described IMPORTANT: When there is a problem, the problem is very often
at the end of the log file. It means you don't have to read all this described at the end of the log file. It means you don't have to read
bloody ununderstandable verbose logfile, just read the end first. all this bloody ununderstandable verbose logfile, just read the end
first.
A.8) Loop on A.4 through A.7 A.8) Loop on A.4 through A.7
@ -173,7 +191,7 @@ A.8) Loop on A.4 through A.7
* A.6) look at the run and the log file * A.6) look at the run and the log file
* A.7) control what happened. * A.7) control what happened.
Loop on the process of editing, running, and controlling imapsync Loop on the process of editing, running and controlling imapsync
until you solve all issues and the sync is over. until you solve all issues and the sync is over.
A good sign that the sync went very well is when the nearly last lines are like: A good sign that the sync went very well is when the nearly last lines are like:
@ -195,8 +213,8 @@ This is the way for modifying imapsync.exe if needed.
B.1) Install Perl if it isn't already installed. B.1) Install Perl if it isn't already installed.
Strawberry Perl is a very good candidate Strawberry Perl is a very good candidate
http://strawberryperl.com/ http://strawberryperl.com/
I use 5.30.1.1 (released 2019-11-22) but previous and later releases I use 5.32.1.1 (released 2021-01-24) but previous and later releases
should work as well (Perl 5.18 to 5.26 do). should work as well (Perl 5.18 to 5.30 do).
B.2) Go into the Cook/ directory B.2) Go into the Cook/ directory
B.3) Double-click build_exe.bat B.3) Double-click build_exe.bat
@ -205,4 +223,3 @@ It should create a binary imapsync.exe in the current Cook/ directory.
B.4) Move imapsync.exe in the upper directory and follow instructions B.4) Move imapsync.exe in the upper directory and follow instructions
from A.3) to A.8) from A.3) to A.8)

10
S/external.shtml
View file

@ -65,9 +65,13 @@
<li> MailStore: <a href="https://www.mailstore.com/en/products/mailstore-home/">https://www.mailstore.com/en/products/mailstore-home/</a></li> <li> MailStore: <a href="https://www.mailstore.com/en/products/mailstore-home/">https://www.mailstore.com/en/products/mailstore-home/</a></li>
<li> mnIMAPSync: <a href="https://github.com/manusa/mnIMAPSync">https://github.com/manusa/mnIMAPSync</a></li> <li> mnIMAPSync: <a href="https://github.com/manusa/mnIMAPSync">https://github.com/manusa/mnIMAPSync</a></li>
<li> imap-upload: <a href="http://imap-upload.sourceforge.net/">http://imap-upload.sourceforge.net/</a> <li> imap-upload: <a href="http://imap-upload.sourceforge.net/">http://imap-upload.sourceforge.net/</a>
(A tool for uploading a local mbox file to IMAP4 server)</li> (A tool to upload a local mbox file to IMAP4 server)</li>
<li> imapbackup: <a href="https://github.com/rcarmo/imapbackup">https://github.com/rcarmo/imapbackup</a> <li> imapbackup: <a href="https://github.com/rcarmo/imapbackup">https://github.com/rcarmo/imapbackup</a>
(A Python script for incremental backups of IMAP mailboxes)</li> (A Python script to incremental backups of IMAP mailboxes)</li>
<li> imap-backup: <a href="https://github.com/joeyates/imap-backup">https://github.com/joeyates/imap-backup</a>
(A Ruby script to backup GMail (or other IMAP) accounts to disk)</li>
<li> BitRecover email-backup <b>99 USD, 299 USD</b> <a href="https://www.bitrecover.com/email-backup/">https://www.bitrecover.com/email-backup/</a>.</li> <li> BitRecover email-backup <b>99 USD, 299 USD</b> <a href="https://www.bitrecover.com/email-backup/">https://www.bitrecover.com/email-backup/</a>.</li>
@ -182,7 +186,7 @@ alt="Viewable With Any Browser" />
<!--#config timefmt="%D" --> <!--#config timefmt="%D" -->
<!--#config timefmt="%A %B %d, %Y" --> <!--#config timefmt="%A %B %d, %Y" -->
<b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b> <b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b>
($Id: external.shtml,v 1.58 2021/06/30 20:34:23 gilles Exp gilles $)<br/> ($Id: external.shtml,v 1.59 2022/01/08 19:54:26 gilles Exp gilles $)<br/>
<a href="#TOP">Top of the page</a> <a href="#TOP">Top of the page</a>
</p> </p>

BIN
S/images/5c019d917bba312af7553b49_MacStadium-developerlogo.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 54 KiB

70
S/news.shtml
View file

@ -85,6 +85,72 @@ If you are very interrested in those projects, don't hesitate to solicit me!
</ul> </ul>
--> -->
<ul>
<li><b>2.178</b></li>
<li><b>General</b>: </li>
<li><b>Dependencies removed</b>: None </li>
<li><b>Dependencies added</b>: Perl module Compress::Zlib</li>
<li><b>Enhancement</b>: Added final statistic line: "CPU time and %cpu : 13.27 sec 8.3 %cpu 2.1 %allcpus</li>
<li><b>Enhancement</b>: Removed the hard limit --maxsize 35_651_584 set by --gmail2 since CAPABILITY is used and good.</li>
<li><b>Enhancement</b>: Does not checknoabletosearch if --justfolders</li>
<li><b>Enhancement</b>: When creating a folder, check if it is selectable and consider ok if it is selectable.</li>
<li><b>Enhancement</b>: Options --gmail1 --gmail2 together sets --useheader X-Gmail-Received --useheader Message-Id</li>
<li><b>Enhancement</b>: Added compression in the imap connexions.
Compression is off by default no matter the capability announced.
Use --compress1 and --compress2 to turn compression on.</li>
<li><b>Enhancement</b>: Added --keepalive1 and --keepalive2 options.
Some firewalls and network gears like to timeout connections prematurely if the connection sits idle.
This option enables SO_KEEPALIVE on the host1 socket.
--keepalive1 is on by default since imapsync release 2.169
Use --nokeepalive1 to disable it. Same thing for --keepalive2 but for --host2. See <a href="https://metacpan.org/pod/Mail::IMAPClient#Keepalive">https://metacpan.org/pod/Mail::IMAPClient#Keepalive</a></li>
<li><b>Enhancement</b>: </li>
<li><b>Enhancement</b>: </li>
<li><b>Usability</b>: Removed --fast option. It didn't do anything anyway, it was a fake option.</li>
<li><b>Usability</b>: Added advice inline advice on errors
<ul>
<li>ERR_OVERQUOTA. "The destination mailbox is 100% full, get free space on it and then resume the sync."</li>
<li>ERR_TRANSFER_EXCEEDED. "The maximum transfer size for a single sync is reached ( over 3 GB ). Relaunch the sync to sync more."</li>
<li>ERR_APPEND_SIZE. "The destination server refuses too big messages. Use --truncmess option. Read https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Too_Big.txt" </li>
<li>ERR_CONNECTION_FAILURE_HOST1. "Check that host1 on port1 is the right IMAP server to be contacted for your mailbox."</li>
<li>ERR_CONNECTION_FAILURE_HOST2. "Check that host2 on port2 is the right IMAP server to be contacted for your mailbox."</li>
<li>ERR_AUTHENTICATION_FAILURE_USER1. "Check the credentials for user1."</li>
<li>ERR_AUTHENTICATION_FAILURE_USER2. "Check the credentials for user2."</li>
</ul>
</li>
<li><b>Usability</b>: Bytes sizes human readable are now presented in KB MB GB TB PB (1000 base) instead of KiB MiB GiB TiB PiB (1024 base)</li>
<li><b>Usability</b>: Added README help for --truncmess</li>
<li><b>Usability</b>: </li>
<li><b>Usability</b>: </li>
<li><b>Usability</b>: </li>
<li><b>Usability</b>: </li>
<li><b>CGI context</b>: Experimental loaddelay forced to 0 (no delay at all). So far a huge load is rarely a problem.</li>
<li><b>CGI context</b>: Bugfix. Abort didn't set charset=UTF-8; same thing for heavy load 503 Service Unavailable</li>
<li><b>Bug fix</b>: Made reconnect mechanism works with --oauthdirect1 --oauthdirect2 --oauthaccesstoken1 --oauthaccesstoken2</li>
<li><b>Bug fix</b>: Upped the maximum number a pid can be. From 99999 to 2^22.
See <a href="https://unix.stackexchange.com/questions/16883/what-is-the-maximum-value-of-the-process-id">what-is-the-maximum-value-of-the-process-id</a> </li>
<li><b>Bug fix</b>: Load average on Mac was sometimes buggy. locale float 3,14 instead of 3.14</li>
<li><b>Bug fix</b>: errors_incr() exited with CATCH_ALL</li>
<li><b>Bug fix</b>: Memory consumption on Mac was not relevant. Use RSS instead of VSZ, on Mac only. Linux stays with VSZ.</li>
<li><b>Bug fix</b>: </li>
<li><b>Bug fix</b>: </li>
<li><b>Docker context</b>: Current working directory changed from /var/tmp to /var/tmp/uid_$EFFECTIVE_USER_ID</li>
</ul>
<ul> <ul>
<li><b>2.140</b></li> <li><b>2.140</b></li>
<li><b>General</b>: New release numbering from 1.999 directly to 2.100; I want to avoid 1.1000 and sorting issues in directory listing.</li> <li><b>General</b>: New release numbering from 1.999 directly to 2.100; I want to avoid 1.1000 and sorting issues in directory listing.</li>
@ -157,8 +223,6 @@ Exit value 112 (EXIT_WITH_ERRORS_MAX) is now replaced by the most common error e
<li><b>Refactoring</b>: Continuing to deglobalize the global variables.</li> <li><b>Refactoring</b>: Continuing to deglobalize the global variables.</li>
<li><b>Refactoring</b>: 1734 unit tests.</li> <li><b>Refactoring</b>: 1734 unit tests.</li>
<li><b>Refactoring</b>: </li>
</ul> </ul>
@ -954,7 +1018,7 @@ by ignoring PERMANENTFLAGS (Exchange tests)</li>
<!--#config timefmt="%D" --> <!--#config timefmt="%D" -->
<!--#config timefmt="%A %B %d, %Y" --> <!--#config timefmt="%A %B %d, %Y" -->
<b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b> <b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b>
($Id: news.shtml,v 1.78 2021/07/05 20:22:15 gilles Exp gilles $)<br /> ($Id: news.shtml,v 1.83 2022/01/13 18:36:12 gilles Exp gilles $)<br />
<a href="#TOP">Top of the page</a> <a href="#TOP">Top of the page</a>
</p> </p>
</body></html> </body></html>

29
S/quiz/README.txt Normal file
View file

@ -0,0 +1,29 @@
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
LICENSE
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Copyright by Code Boxx
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
MORE
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Please visit https://code-boxx.com/ for more!

45
S/quiz/comment_to_toh.txt Executable file
View file

@ -0,0 +1,45 @@
Hi Toh!
First, I thank you very much for posting and sharing this good piece
of code!
I like it because, it's simple, it's embedable, no dependency, pure
javascript.
I found your code search for a simple quiz code. I consider quizzes as
a fun learning learning experience on something.
Some things I'd like to improve on it:
ux/reload) The user has to reload the page to restart the quiz and
it's not evident. How about making the last button a possibility
to redo the quiz from start and make a better score?
ux/where I am) The user doesn't know how many questions there are in
the quiz, where he is along the quiz, how good he is. Knowing where we
are in a serie makes it easier to reach the end. This one is easy to
add, as I could do it.
css functional) The quiz.css is not so cosmetic: if you comment it in the
html, then the quiz functionally doesn't work the same; we have to
click twice on a button to go to the next question, and the second
click has to be outside the round of the radio button, on the label.
I don't know how to fix that but it's not important anyway, the css is
needed for cosmetics.
css cosmetic) this quiz code is to be embedded in a page then I suggest
that the cosmetics are inheritated.
I'm also adept of minimal css, the less to achieve a goal, the better.
dev testing) I don't know how do you test it while coding and
modifying it. A manual loop browser_reload_run_editor?
dev jslint) I paste it to jslint (https://www.jslint.com/)
and it didn't pass the lint test.
dev data load) How about putting all the Q/R in a dedicated text file
to maintain a single quiz.js and quiz.css and have multiple quizzes
using the same functional code?

46
S/quiz/quiz.css Normal file
View file

@ -0,0 +1,46 @@
/* (A) WRAPPER */
#quizWrap {
max-width: 600px;
margin: 0 auto;
}
/* (B) QUESTION */
#quizQn {
padding: 20px;
background: #4c93ba;
color: #fff;
font-size: 24px;
border-radius: 10px;
}
/* (C) ANSWERS */
#quizAns {
margin: 10px 0;
display: grid;
grid-template-columns: auto auto;
grid-gap: 10px;
}
#quizAns input[type=radio] { display: none; }
#quizAns label {
background: #fafafa;
border: 1px solid #eee;
border-radius: 10px;
padding: 10px;
font-size: 20px;
cursor: pointer;
text-align: center;
}
#quizAns label.correct {
background: #d8ffc4;
border: 1px solid #60a03f;
}
#quizAns label.wrong {
background: #ffe8e8;
border: 1px solid #c78181;
}
/* (D) BODY... DOES NOT QUITE MATTER */
html, body {
background: #74b6db;
font-family: arial, sans-serif;
}

16
S/quiz/quiz.html Normal file
View file

@ -0,0 +1,16 @@
<!DOCTYPE html>
<html>
<head>
<title>
Simple Javascript Quiz Demo
</title>
<!-- (A) LOAD QUIZ CSS + JS -->
<link href="quiz.css" rel="stylesheet">
<script src="quiz.js"></script>
</head>
<body>
<!-- (B) QUIZ CONTAINER -->
<div id="quizWrap"></div>
</body>
</html>

136
S/quiz/quiz.js Normal file
View file

@ -0,0 +1,136 @@
var quiz = {
// (A) PROPERTIES
// (A1) QUESTIONS & ANSWERS
// Q = QUESTION, O = OPTIONS, A = CORRECT ANSWER
data: [
{
q : "What is the standard distance between the target and archer in Olympics?",
o : [
"50 meters",
"70 meters",
"100 meters",
"120 meters"
],
a : 1 // arrays start with 0, so answer is 70 meters
},
{
q : "Which is the highest number on a standard roulette wheel?",
o : [
"22",
"24",
"32",
"36"
],
a : 3
},
{
q : "How much wood could a woodchuck chuck if a woodchuck would chuck wood?",
o : [
"400 pounds",
"550 pounds",
"700 pounds",
"750 pounds"
],
a : 2
},
{
q : "Which is the seventh planet from the sun?",
o : [
"Uranus",
"Earth",
"Pluto",
"Mars"
],
a : 0
},
{
q : "Which is the largest ocean on Earth?",
o : [
"Atlantic Ocean",
"Indian Ocean",
"Arctic Ocean",
"Pacific Ocean"
],
a : 3
}
],
// (A2) HTML ELEMENTS
hWrap: null, // HTML quiz container
hQn: null, // HTML question wrapper
hAns: null, // HTML answers wrapper
// (A3) GAME FLAGS
now: 0, // current question
score: 0, // current score
// (B) INIT QUIZ HTML
init: function(){
// (B1) WRAPPER
quiz.hWrap = document.getElementById("quizWrap");
// (B2) QUESTIONS SECTION
quiz.hQn = document.createElement("div");
quiz.hQn.id = "quizQn";
quiz.hWrap.appendChild(quiz.hQn);
// (B3) ANSWERS SECTION
quiz.hAns = document.createElement("div");
quiz.hAns.id = "quizAns";
quiz.hWrap.appendChild(quiz.hAns);
// (B4) GO!
quiz.draw();
},
// (C) DRAW QUESTION
draw: function(){
// (C1) QUESTION
quiz.hQn.innerHTML = quiz.data[quiz.now].q;
// (C2) OPTIONS
quiz.hAns.innerHTML = "";
for (let i in quiz.data[quiz.now].o) {
let radio = document.createElement("input");
radio.type = "radio";
radio.name = "quiz";
radio.id = "quizo" + i;
quiz.hAns.appendChild(radio);
let label = document.createElement("label");
label.innerHTML = quiz.data[quiz.now].o[i];
label.setAttribute("for", "quizo" + i);
label.dataset.idx = i;
label.addEventListener("click", quiz.select);
quiz.hAns.appendChild(label);
}
},
// (D) OPTION SELECTED
select: function(){
// (D1) DETACH ALL ONCLICK
let all = quiz.hAns.getElementsByTagName("label");
for (let label of all) {
label.removeEventListener("click", quiz.select);
}
// (D2) CHECK IF CORRECT
let correct = this.dataset.idx == quiz.data[quiz.now].a;
if (correct) {
quiz.score++;
this.classList.add("correct");
} else {
this.classList.add("wrong");
}
// (D3) NEXT QUESTION OR END GAME
quiz.now++;
setTimeout(function(){
if (quiz.now < quiz.data.length) { quiz.draw(); }
else {
quiz.hQn.innerHTML = `You have answered ${quiz.score} of ${quiz.data.length} correctly.`;
quiz.hAns.innerHTML = "";
}
}, 1000);
}
};
window.addEventListener("load", quiz.init);

83
S/quiz/quiz_imapsync.css Executable file
View file

@ -0,0 +1,83 @@
/* Taken from https://code-boxx.com/simple-javascript-quiz */
/* (A) WRAPPER */
#quizWrap {
border-style: solid;
border-collapse: collapse;
background: #e0e0e0;
}
#quizNumber,#quizScore{
text-align: center;
padding: 10px;
border-width: thin;
border-style: solid;
border-left-style: none;
border-right-style: none;
}
#quizScore{
border-bottom-style: none;
}
#quizRestart {
margin: auto;
margin-bottom: 10px;
text-align: center;
padding: 10px;
border-radius: 10px;
border-style: solid;
border-width: thin;
background: #d8ffc4;
width: 100px;
cursor: pointer;
}
#quizQn {
padding: 20px;
text-align: center;
font-weight: bold;
}
/* (C) ANSWERS */
#quizAns {
margin: 10px;
text-align: center;
border-width: thin;
display: grid;
grid-template-columns: auto auto;
grid-gap: 10px;
justify-content: space-evenly;
align-content: space-evenly;
}
#quizAns input[type=radio] { display: none; }
#quizAns label {
/* background: #cccccc;
border: 1px solid #eee;
font-size: 20px; */
cursor: pointer;
background: #ffffff;
font-weight: bold;
padding: 10px;
border-radius: 10px;
border-style: solid;
border-width: thin;
margin-top: 5px;
margin-left: 25px;
margin-right: 25px;
margin-bottom: 5px;
}
#quizAns label.correct {
background: #d8ffc4;
border: 1px solid #60a03f;
}
#quizAns label.wrong {
background: #ffe8e8;
border: 1px solid #c78181;
}

20
S/quiz/quiz_imapsync.html Executable file
View file

@ -0,0 +1,20 @@
<!DOCTYPE html>
<!-- Taken from https://code-boxx.com/simple-javascript-quiz -->
<html>
<head>
<title>
Imapsync Quiz
</title>
<!-- (A) LOAD QUIZ CSS + JS -->
<link href="quiz_imapsync.css" rel="stylesheet">
<script src="quiz_imapsync.js"></script>
</head>
<body>
<!-- (B) QUIZ CONTAINER -->
<div id="quizWrap"></div>
</body>
</html>

245
S/quiz/quiz_imapsync.js Executable file
View file

@ -0,0 +1,245 @@
// Taken from https://code-boxx.com/simple-javascript-quiz
var quiz = {
// (A) PROPERTIES
// (A1) QUESTIONS & ANSWERS
// Q = QUESTION, O = OPTIONS, A = CORRECT ANSWER
data: [
{
a : 1,
o : [
"Facebook",
"POP, IMAP, HTTP",
"SMTP, ICQ",
"Twitter",
"FTP, GOPHER",
"Instagram"
],
q : "What are the main protocols usually used to access a mailbox?"
},
{
a : 2,
o : [
"I'M A Passager",
"Instant Mailbox Access Possibility",
"Internet Message Access Protocol",
"In My Ambiguous Posture",
"In My Ambience Place",
"Izorc Muggle Ark Prout"
],
q : "What means the acronym IMAP?"
},
{
a : 2,
o : [
"No parameter is needed",
"Harry Potter's magic wang",
"1) The IMAP server name + 2) the user login + 3) the password",
"1) the user login + 2) the password"
],
q : "What are the parameters needed to access an IMAP mailbox?"
},
{
a : 4,
o : [
"0",
"2",
"3",
"4",
"6",
"8"
],
q : "How many parameters are mandatory to synchronize two IMAP mailboxes with imapsync?"
},
{
a : 0,
o : [
"No",
"Yes"
],
q : "Can Imapsync synchronize POP accounts?"
},
{
a : 1,
o : [
"No",
"Yes"
],
q : "Can Imapsync synchronize Gmail accounts?"
},
{
a : 1,
o : [
"No",
"Yes"
],
q : "Can Imapsync synchronize Office365 accounts?"
},
{
a : 0,
o : [
"No",
"Yes"
],
q : "Can Imapsync synchronize Contacts, Calendars, Chats, or Notes?"
},
{
a : 0,
o : [
"NO LIMIT",
"GPL",
"WTFPL",
"MIT",
"CC0",
"Proprietary"
],
q : "What is the Imapsync License name?"
},
{
a : 0,
o : [
"No limits to do anything with this work and this license.",
"All permissions restricted.",
"Use it for good or evil.",
"Too long to fit here."
],
q : "What is the Imapsync License main text?"
},
{
a : 0,
o : [
"Yes, sometimes with delay, he's a human",
"No, never, email is dead",
"Yes, immediately and 24h/24, he's a robot",
"What is email?"
],
q : "Does the imapsync author Gilles LAMIRAL reply to every email?"
}
],
dataDev: [
{
a : 1,
o : [
"Facebook",
"POP, IMAP, HTTP",
"SMTP, ICQ",
"Twitter",
"FTP, GOPHER",
"Instagram"
],
q : "What are the main protocols usually used to access a mailbox?"
}
],
// (B) INIT QUIZ HTML
init: function(){
// (B1) WRAPPER
quiz.hWrap = null;
quiz.hScore = null;
quiz.hRestart = null;
quiz.hQn = null;
quiz.hAns = null;
quiz.now = 0;
quiz.score = 0;
quiz.hWrap = document.getElementById("quizWrap");
quiz.hWrap.innerHTML = "";
// NUMBER SECTION
quiz.hNumber = document.createElement("div");
quiz.hNumber.id = "quizNumber";
quiz.hWrap.appendChild(quiz.hNumber);
// (B2) QUESTIONS SECTION
quiz.hQn = document.createElement("div");
quiz.hQn.id = "quizQn";
quiz.hWrap.appendChild(quiz.hQn);
// (B3) ANSWERS SECTION
quiz.hAns = document.createElement("div");
quiz.hAns.id = "quizAns";
quiz.hWrap.appendChild(quiz.hAns);
// SCORE SECTION
quiz.hScore = document.createElement("div");
quiz.hScore.id = "quizScore";
quiz.hWrap.appendChild(quiz.hScore);
// RESTART SECTION
quiz.hRestart = document.createElement("div");
quiz.hRestart.id = "quizRestart";
quiz.hWrap.appendChild(quiz.hRestart);
quiz.hRestart.innerHTML = "Restart Quiz";
document.getElementById("quizRestart").addEventListener("click", quiz.restart);
// (B4) GO!
quiz.draw();
},
// (C) DRAW QUESTION
draw: function(){
// SCORE
var QnNumber = `${quiz.now + 1}/${quiz.data.length}`;
var ScoreNb = `${quiz.score}/${quiz.now}`;
quiz.hNumber.innerHTML = `Question ${QnNumber}`;
quiz.hScore.innerHTML = `Score ${ScoreNb}`;
// (C1) QUESTION
quiz.hQn.innerHTML = quiz.data[quiz.now].q;
// (C2) OPTIONS
quiz.hAns.innerHTML = "";
for (let i in quiz.data[quiz.now].o) {
let radio = document.createElement("input");
radio.type = "radio";
radio.name = "quiz";
radio.id = "quizo" + i;
quiz.hAns.appendChild(radio);
let label = document.createElement("label");
label.innerHTML = quiz.data[quiz.now].o[i];
label.setAttribute("for", "quizo" + i);
label.dataset.idx = i;
label.addEventListener("click", quiz.select);
quiz.hAns.appendChild(label);
}
},
// (D) OPTION SELECTED
select: function(){
// (D1) DETACH ALL ONCLICK
let all = quiz.hAns.getElementsByTagName("label");
for (let label of all) {
label.removeEventListener("click", quiz.select);
}
// (D2) CHECK IF CORRECT
let correct = this.dataset.idx == quiz.data[quiz.now].a;
if (correct) {
quiz.score++;
this.classList.add("correct");
} else {
this.classList.add("wrong");
}
// (D3) NEXT QUESTION OR END GAME
quiz.now++;
setTimeout(function(){
if (quiz.now < quiz.data.length) { quiz.draw(); }
else {
// quiz.hNumber.innerHTML = "";
ScoreNb = `${quiz.score}/${quiz.now}`;
quiz.hScore.innerHTML = `Score ${ScoreNb}`;
quiz.hQn.innerHTML = "Finished!";
quiz.hAns.innerHTML = "";
}
}, 800);
},
restart:function(){
quiz.init() ;
}
};
window.addEventListener("load", quiz.init);

11
S/testimonial.shtml
View file

@ -42,6 +42,15 @@ I'm not sure I deserve such great quotes for my work on Imapsync, anyway here th
</p> </p>
<!--#include file="S/tw-mention.html" --> <!--#include file="S/tw-mention.html" -->
<blockquote class="twitter-tweet">
<p lang="en" dir="ltr">
imapsync is simply the best and most usable tool ever developed. Works perfect!
Thank you <a href="https://twitter.com/imapsync?ref_src=twsrc%5Etfw">@imapsync</a></p>&mdash; bkcrri (@bkcrri)
<a href="https://twitter.com/bkcrri/status/1456392869808328706?ref_src=twsrc%5Etfw">November 4, 2021</a>
</blockquote>
<hr>
<blockquote class="twitter-tweet"> <blockquote class="twitter-tweet">
<p lang="en" dir="ltr">What’s your favorite IMAP migration/sync tool? I made the best experiences with <p lang="en" dir="ltr">What’s your favorite IMAP migration/sync tool? I made the best experiences with
<a href="https://twitter.com/imapsync?ref_src=twsrc%5Etfw">@imapsync</a>. <a href="https://twitter.com/imapsync?ref_src=twsrc%5Etfw">@imapsync</a>.
@ -157,7 +166,7 @@ alt="Viewable With Any Browser" >
<!--#config timefmt="%D" --> <!--#config timefmt="%D" -->
<!--#config timefmt="%A %B %d, %Y" --> <!--#config timefmt="%A %B %d, %Y" -->
<b>This document was last modified on <!--#echo var="LAST_MODIFIED" --></b> <b>This document was last modified on <!--#echo var="LAST_MODIFIED" --></b>
($Id: testimonial.shtml,v 1.8 2019/11/15 11:35:16 gilles Exp gilles $)<br> ($Id: testimonial.shtml,v 1.9 2021/11/05 12:04:12 gilles Exp gilles $)<br>
<a href="#TOP">Top of the page</a> <a href="#TOP">Top of the page</a>
</p> </p>
</footer> </footer>

22
TODO
View file

@ -1,5 +1,5 @@
#!/bin/cat #!/bin/cat
# $Id: TODO,v 1.257 2021/07/05 20:19:31 gilles Exp gilles $ # $Id: TODO,v 1.262 2022/01/11 12:56:18 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc This documentation is also at http://imapsync.lamiral.info/#doc
@ -9,6 +9,17 @@ TODO file for imapsync
https://imapsync.lamiral.info/TODO https://imapsync.lamiral.info/TODO
SUGGESTED 2022_01_09 by Neustradamus
Add SCRAM-SHA* support
https://github.com/imapsync/imapsync/issues/313
SUGGESTED 2021_06_13 by rotemes
https://github.com/imapsync/imapsync/issues/288
Add a counter for skipped because of already copied and
because of other things.
What ENVELOPE could bring?
( 'FLAGS', 'INTERNALDATE', 'RFC822.SIZE' ) ;
( 'FLAGS', 'INTERNALDATE', 'RFC822.SIZE' , 'ENVELOPE' )
SUGGESTED 2021_05_29 on github https://github.com/imapsync/imapsync/issues/78 SUGGESTED 2021_05_29 on github https://github.com/imapsync/imapsync/issues/78
ripmime https://github.com/inflex/ripMIME ripmime https://github.com/inflex/ripMIME
@ -696,6 +707,15 @@ http://asg.web.cmu.edu/cyrus/download/imapd/altnamespace.html
Now the TODO done! (or not) Now the TODO done! (or not)
=========================================================================== ===========================================================================
DONE revision 2.169 date: 2021/12/07 17:13:44;
Added --keepalive1 and --keepalive2 options. On by default.
See https://metacpan.org/pod/Mail::IMAPClient#Keepalive
DONE revision 2.159 date: 2021/11/19 14:29:31
SUGGESTED 2021_11_18 by Justas Umbrasas
Add COMPRESS to imap connexions
--compress1 --compress2
DONE 2021/04/30 revision 2.111 by Gilles DONE 2021/04/30 revision 2.111 by Gilles
SUGGESTED 2021_04_22 by popular request SUGGESTED 2021_04_22 by popular request
Add an option --syncduplicates that syncs duplicates. Add an option --syncduplicates that syncs duplicates.

2
VERSION
View file

@ -1 +1 @@
2.140 2.178

266
X/cgi_memo
View file

@ -1,6 +1,6 @@
#!/bin/sh #!/bin/sh
# $Id: cgi_memo,v 1.73 2021/06/30 20:10:12 gilles Exp gilles $ # $Id: cgi_memo,v 1.74 2021/11/25 11:25:13 gilles Exp gilles $
if test -n "$1"; then if test -n "$1"; then
echoq() { echo "$@" ; } # not quiet mode echoq() { echo "$@" ; } # not quiet mode
@ -89,7 +89,7 @@ here_is_linux()
echoq list_all_logs echoq list_all_logs
list_all_logs() list_all_logs()
{ {
cat list_all_logs.txt cat list_all_logs.txt list_all_logs_auto.txt | grep -v 385d7a4d8d428d7aa2b57c8982629e2bd67698ed/ | grep "$1"
} }
echoq list_all_logs_generate echoq list_all_logs_generate
@ -108,17 +108,27 @@ list_log_matching()
} }
statsfile()
{
base=`pattern_filename G $1 $2`
statsfile=$base.txt
echo $statsfile
}
echoq biggest_transfer echoq biggest_transfer
biggest_transfer() biggest_transfer()
{ {
bytestohuman `datamash_file_op_index G_Total_bytes_transferred.txt max 5` statsfile=`statsfile Total_bytes_transferred $1`
bytestohuman `datamash_file_op_index "$statsfile" max 5`
} }
echoq total_bytes_transferred echoq total_bytes_transferred
total_bytes_transferred() total_bytes_transferred()
{ {
datamash_file_op_index G_Total_bytes_transferred.txt sum 5 statsfile=`statsfile Total_bytes_transferred $1`
datamash_file_op_index "$statsfile" sum 5
} }
# Total volume transferred # Total volume transferred
@ -126,20 +136,22 @@ echoq total_volume_transferred
total_volume_transferred() total_volume_transferred()
{ {
#echo -n 'numfmt --to=iec-i ' #echo -n 'numfmt --to=iec-i '
bytestohuman `total_bytes_transferred` bytestohuman `total_bytes_transferred $1`
} }
echoq total_messages_transferred echoq total_messages_transferred
total_messages_transferred() total_messages_transferred()
{ {
datamash_file_op_index G_Messages_transferred.txt sum 4 %16.0f | tr -d ' ' statsfile=`statsfile Messages_transferred $1`
datamash_file_op_index "$statsfile" sum 4 %16.0f | tr -d ' '
} }
longest_transfer() longest_transfer()
{ {
LC_ALL=C printf "%.0f\n" `datamash_file_op_index G_Transfer_time.txt max 4` statsfile=`statsfile Transfer_time $1`
LC_ALL=C printf "%.0f\n" `datamash_file_op_index "$statsfile" max 4`
} }
echoq number_and_pids_of_imapsync_running echoq number_and_pids_of_imapsync_running
@ -178,9 +190,15 @@ oom_immune_imapsync_running() {
echoq nb_migrations_launched echoq nb_migrations_launched
nb_migrations_launched() { nb_migrations_launched() {
/bin/ls . | egrep [a-f0-9]{40} | wc -l list_all_logs | egrep "$1" | egrep -o [a-f0-9]{40} | sort | uniq -c | wc -l
} }
nb_migrations_launched_old() {
/bin/ls . | egrep [a-f0-9]{40} | sort | uniq | wc -l
}
echoq current_stats echoq current_stats
current_stats() { current_stats() {
echo -n "Nb accounts: "; nb_migrations_launched echo -n "Nb accounts: "; nb_migrations_launched
@ -213,9 +231,10 @@ grep_in_all_logs() {
} }
echoq grep_in_logs_manual echoq grep_in_logs_manual
grep_in_logs_manual() { grep_in_logs_manual()
cat << EOF {
list_all_logs | tail -500 | tr '\n' '\000'| xargs -0 egrep -i LALALA | tee grep_LALALA.txt cat << EOF
list_all_logs /2021_11 | tail -666 | tr '\n' '\000'| xargs -0 egrep -i LALALA | tee grep_LALALA.txt
EOF EOF
} }
@ -245,7 +264,7 @@ tests_pattern_filename()
run_test test "abcd" = "`pattern_filename ab""cd`" run_test test "abcd" = "`pattern_filename ab""cd`"
run_test test "abcd" = "`pattern_filename ab""cd`" run_test test "abcd" = "`pattern_filename ab""cd`"
run_test test "ab_cd" = "`pattern_filename ab" "cd`" run_test test "ab_cd" = "`pattern_filename ab" "cd`"
run_test test "ab__cd" = "`pattern_filename ab" "cd`" run_test test "ab__cd" = "`pattern_filename ab" "cd`"
run_test test "ab__cd" = "`pattern_filename ab "" cd`" run_test test "ab__cd" = "`pattern_filename ab "" cd`"
run_test test "ab___cd" = "`pattern_filename ab " " cd`" run_test test "ab___cd" = "`pattern_filename ab " " cd`"
run_test test "ab____cd" = "`pattern_filename ab " " cd`" run_test test "ab____cd" = "`pattern_filename ab " " cd`"
@ -263,7 +282,7 @@ pattern_filename()
echoq 'grep_stats_from_list_log_matching lognamepattern' echoq 'grep_stats_from_list_log_matching lognamepattern # time depending on lognamepattern'
grep_stats_from_list_log_matching() { grep_stats_from_list_log_matching() {
pattern="$1" pattern="$1"
pattern_filename=`pattern_filename "$pattern"` pattern_filename=`pattern_filename "$pattern"`
@ -277,32 +296,42 @@ grep_stats_from_list_log_matching() {
grep_any() { grep_any() {
pattern_filtered=`pattern_filename "$1" "$2"` file=`statsfile "$1" "$2"`
file="G_${pattern_filtered}.txt" pattern_filtered=`pattern_filename "$2"`
echo $file echo $file
egrep -i "$1" grep_stats_"$2".txt > $file.tmp if test -f grep_stats_"$pattern_filtered".txt ; then
mv $file.tmp $file egrep -i "$1" grep_stats_"$pattern_filtered".txt > $file.tmp
mv $file.tmp $file
else
echo File not found: grep_stats_"$pattern_filtered".txt
fi
} }
grep_load() { grep_load() {
echo G_Load.txt file=`statsfile "Load" "$1"`
egrep -o 'Load is ..?\... ..?\... ..?\... .*' grep_stats.txt > G_Load.txt pattern_filtered=`pattern_filename "$1"`
echo $file
if test -f grep_stats_"$pattern_filtered".txt ; then
egrep -o 'Load is ..?\... ..?\... ..?\... .*' grep_stats_"$pattern_filtered".txt > $file.tmp
mv $file.tmp $file
else
echo File not found: grep_stats_"$pattern_filtered".txt
fi
} }
echoq 'grep_all_stat_from_patterns_list lognamepattern # long'
grep_all_stat_from_patterns_list() {
#grep_load
stat_patterns_list | while read k; do grep_any "$k" "$1"; done
}
stat_patterns_list() { stat_patterns_list() {
cat stat_patterns.txt | sed '/^[[:space:]]*$/d' | tr -d '^' cat stat_patterns.txt | sed '/^[[:space:]]*$/d' | tr -d '^'
} }
echoq 'grep_all_stat_from_patterns_list lognamepattern # long'
grep_all_stat_from_patterns_list() {
grep_load "$1"
stat_patterns_list | while read k; do grep_any "$k" "$1"; done
}
sum_first_column_G_HTTP_USER_AGENT_sorted() sum_first_column_G_HTTP_USER_AGENT_sorted()
{ {
@ -348,18 +377,20 @@ stat_load()
echoq stat_exit_value echoq stat_exit_value
stat_exit_value() stat_exit_value()
{ {
good_lines=`grep '(EX' G_Exiting_with_return_value.txt | wc -l | tr -d ' '` statsfile=`statsfile Exiting_with_return_value "$1"`
grep '(EX' G_Exiting_with_return_value.txt \ good_lines_nb=`grep '(EX' $statsfile | wc -l | tr -d ' '`
grep '(EX' "$statsfile" \
| datamash --sort groupby 6 -W count 5 \ | datamash --sort groupby 6 -W count 5 \
| awk -v good_lines=$good_lines \ | awk -v good_lines_nb=$good_lines_nb \
'{ printf "%.2g%% %s\n", 100*$2/good_lines, $1 }' \ '{ printf "%.2g%% %s\n", 100*$2/good_lines_nb, $1 }' \
| sort -n | sort -n
} }
echoq stat_exit_value_by_value echoq stat_exit_value_by_value
stat_exit_value_by_value() stat_exit_value_by_value()
{ {
datamash --sort groupby 5 -W count 5 < G_Exiting_with_return_value.txt statsfile=`statsfile Exiting_with_return_value "$1"`
datamash --sort groupby 5 -W count 5 < "$statsfile"
} }
@ -423,8 +454,8 @@ stat_all()
stat_any G_Messages_found_in_host2_not_in_host1.txt 9 stat_any G_Messages_found_in_host2_not_in_host1.txt 9
# stat_any G_Folders_synced.txt # stat_any G_Folders_synced.txt
egrep -o '[0-9]+/[0-9]+' G_Folders_synced.txt | egrep -o '^[0-9]+' > G_Folders_synced_.txt egrep -o '[0-9]+/[0-9]+ synced' G_Folders_synced.txt | egrep -o '^[0-9]+' > G_Folders_synced_.txt
egrep -o '[0-9]+/[0-9]+' G_Folders_synced.txt | egrep -o '[0-9]+$' > G_Folders_total_seen.txt egrep -o '[0-9]+/[0-9]+ synced' G_Folders_synced.txt | egrep -o '[0-9]+/[0-9]+' | egrep -o '[0-9]+$' > G_Folders_total_seen.txt
stat_any G_Folders_synced_.txt 1 stat_any G_Folders_synced_.txt 1
stat_any G_Folders_total_seen.txt 1 stat_any G_Folders_total_seen.txt 1
@ -437,7 +468,7 @@ stat_all()
stat_any G_Average_bandwidth_rate.txt 5 stat_any G_Average_bandwidth_rate.txt 5
stat_any G_Biggest_message.txt stat_any G_Biggest_message.txt
stat_any G_Detected_errors.txt 2 stat_any G_Detected_errors.txt 2
stat_any G_Exiting_with_return_value.txt 5 # GROUP #stat_any G_Exiting_with_return_value.txt 5 # GROUP
stat_any G_Memory_consumption_at_the_end.txt 7 stat_any G_Memory_consumption_at_the_end.txt 7
#stat_any G_failure_Error_login.txt #stat_any G_failure_Error_login.txt
percent_stat_useragent_X ; echo percent_stat_useragent_X ; echo
@ -448,24 +479,62 @@ stat_all()
stat_transfer_time_mean() stat_transfer_time_mean()
{ {
datamash_file_op_index G_Transfer_time.txt mean statsfile=`statsfile Transfer_time "$1"`
datamash_file_op_index "$statsfile" mean
} }
stat_throuput_since_day_one_in_days() stat_throuput_since_day_one_in_days()
{ {
number_of_syncs=`number_of_syncs` number_of_syncs=`number_of_syncs "$1"`
days_since_first_use=`days_since_first_use` days_since_first_use=`days_since_first_use "$1"`
c "$number_of_syncs / $days_since_first_use" c "$number_of_syncs / $days_since_first_use"
} }
stat_queue_mean() stat_queue_mean_old()
{ {
stat_throuput_since_day_one_in_days=`stat_throuput_since_day_one_in_days` stat_throuput_since_day_one_in_days=`stat_throuput_since_day_one_in_days`
stat_transfer_time_mean=`stat_transfer_time_mean` stat_transfer_time_mean=`stat_transfer_time_mean "$1"`
stat_queue_mean_raw=`c "$stat_throuput_since_day_one_in_days * $stat_transfer_time_mean / 3600 / 24"` stat_queue_mean_raw=`c "$stat_throuput_since_day_one_in_days * $stat_transfer_time_mean / 3600 / 24"`
LC_ALL=C printf "%2.2f\n" $stat_queue_mean_raw LC_ALL=C printf "%2.2f\n" $stat_queue_mean_raw
} }
stat_queue_mean()
{
first_log=`first_log "$1"`
last_log=`last_log "$1"`
number_of_syncs=`number_of_syncs "$1"`
seconds_between_files=`seconds_between_files $first_log $last_log`
stat_transfer_time_mean=`stat_transfer_time_mean "$1"`
stat_queue_mean_raw=`c "$number_of_syncs / $seconds_between_files * $stat_transfer_time_mean"`
LC_ALL=C printf "%2.2f\n" $stat_queue_mean_raw
}
first_log()
{
list_all_logs "$1"_ | grep /"$1" | head -1
}
last_log()
{
list_all_logs "$1"_ | grep /"$1" | tail -1
}
start_date()
{
first_log=`first_log "$1"`
date -r "$first_log"
}
end_date()
{
last_log=`last_log "$1"`
date -r "$last_log"
}
echoq dirs_of_syncs_finished_recently echoq dirs_of_syncs_finished_recently
dirs_of_syncs_finished_recently() { dirs_of_syncs_finished_recently() {
find . -maxdepth 1 -mtime "${1:--1}" | grep -v "385d7a4d8d428d7aa2b57c8982629e2bd67698ed" | egrep [a-f0-9]{40} | while read d; do find . -maxdepth 1 -mtime "${1:--1}" | grep -v "385d7a4d8d428d7aa2b57c8982629e2bd67698ed" | egrep [a-f0-9]{40} | while read d; do
@ -492,6 +561,7 @@ logfiles_finished_recently()
} }
} }
last_dirs_written() last_dirs_written()
{ {
ls -tr | tail -1800 ls -tr | tail -1800
@ -675,12 +745,38 @@ first_use() {
echo "${1:-2017} ${2:-01} ${3:-09}" echo "${1:-2017} ${2:-01} ${3:-09}"
} }
filedate()
{
test FreeBSD = `uname -s` && gdate -r "$1" '+%Y %m %d'
test Linux = `uname -s` && date -r "$1" '+%Y %m %d'
}
days_between_files()
{
epoch1=`epoch_of_file "$1"`
epoch2=`epoch_of_file "$2"`
echo epoch1 $epoch1 epoch2 $epoch2
expr \( $epoch2 - $epoch1 \) / 3600 / 24
}
seconds_between_files()
{
epoch1=`epoch_of_file "$1"`
epoch2=`epoch_of_file "$2"`
expr \( $epoch2 - $epoch1 \)
}
days_since_first_use() { days_since_first_use() {
first_use=`first_use "$@"` first_use=`first_use "$@"`
#echo $[$[$(date +%s)-$(epoch_of_y_m_d_h_m_s 2017 01 09 00 00 00)]/60/60/24] #echo $[$[$(date +%s)-$(epoch_of_y_m_d_h_m_s 2017 01 09 00 00 00)]/60/60/24]
echo $[$[$(date +%s)-$(epoch_of_y_m_d_h_m_s $first_use 00 00 00)]/60/60/24] echo $[$[$(date +%s)-$(epoch_of_y_m_d_h_m_s $first_use 00 00 00)]/60/60/24]
} }
epoch_of_y_m_d_h_m_s() { epoch_of_y_m_d_h_m_s() {
date -v -1d > /dev/null 2>&1 && date -u -v ${1:-1970}y -v ${2:-1}m -v ${3:-1}d -v ${4:-0}H -v ${5:-0}M -v ${6:-0}S +%s && return date -v -1d > /dev/null 2>&1 && date -u -v ${1:-1970}y -v ${2:-1}m -v ${3:-1}d -v ${4:-0}H -v ${5:-0}M -v ${6:-0}S +%s && return
date --date="1 day ago" > /dev/null && date -u -d "${1:-1970}-${2:-1}-${3:-1} ${4:-0}:${5:-0}:${6:-0}" +%s && return date --date="1 day ago" > /dev/null && date -u -d "${1:-1970}-${2:-1}-${3:-1} ${4:-0}:${5:-0}:${6:-0}" +%s && return
@ -756,7 +852,7 @@ vnstat_init() {
echoq vnstat_gen echoq vnstat_gen
vnstat_gen() { vnstat_gen() {
vnstat_init || return vnstat_init || return
for opt in s h hg hs d m y t vs 5 ; do for opt in s h hg hs d m y t vs 5 5g ; do
test "$1" && echo vnstati -$opt -o $VNSTATI_DIR/vnstat_${opt}.png test "$1" && echo vnstati -$opt -o $VNSTATI_DIR/vnstat_${opt}.png
vnstati -$opt -o $VNSTATI_DIR/vnstat_${opt}.png vnstati -$opt -o $VNSTATI_DIR/vnstat_${opt}.png
done done
@ -833,55 +929,72 @@ referrer_of_x() {
zegrep -h -s -o 'GET /X/? .*http[^"]+' "${@:-/var/log/apache/access.log}" | grep -o 'http.*' zegrep -h -s -o 'GET /X/? .*http[^"]+' "${@:-/var/log/apache/access.log}" | grep -o 'http.*'
} }
biggest_message_seen() {
cat G_Biggest_message.txt | grep -v Memory| datamash -W max 4 | xargs bytestohuman biggest_message_seen() {
statsfile=`statsfile Biggest_message $1`
cat "$statsfile" | grep -v Memory | datamash -W max 4 | xargs bytestohuman
} }
biggest_message_transferred() { biggest_message_transferred() {
# With this, the "Biggest message" could not be transferred by imapsync itself. # With this, the "Biggest message" may be not be transferred by imapsync itself.
grep 'Host2 Biggest message' < G_Biggest_message.txt | datamash -W max 4 | xargs bytestohuman statsfile=`statsfile Biggest_message $1`
grep 'Host2 Biggest message' < "$statsfile" | datamash -W max 4 | xargs bytestohuman
} }
biggest_bandwidth_rate() { biggest_bandwidth_rate() {
datamash_file_op_index G_Average_bandwidth_rate.txt max 5 | tr -d ' ' | tr '\n' ' ' statsfile=`statsfile Average_bandwidth_rate $1`
datamash_file_op_index "$statsfile" max 5 | tr -d ' ' | tr '\n' ' '
echo KiB/s echo KiB/s
} }
average_bandwidth_rate() { average_bandwidth_rate() {
datamash_file_op_index G_Average_bandwidth_rate.txt mean 5 | tr -d ' ' | tr '\n' ' ' statsfile=`statsfile Average_bandwidth_rate $1`
datamash_file_op_index "$statsfile" mean 5 | tr -d ' ' | tr '\n' ' '
echo KiB/s echo KiB/s
} }
max_number_of_messages_transferred() { max_number_of_messages_transferred() {
datamash_file_op_index G_Messages_transferred.txt max 4 "%.0f" statsfile=`statsfile Messages_transferred $1`
datamash_file_op_index "$statsfile" max 4 "%.0f"
} }
max_number_of_messages_skipped() { max_number_of_messages_skipped() {
datamash_file_op_index G_Messages_skipped.txt max 4 "%.0f" statsfile=`statsfile Messages_skipped $1`
datamash_file_op_index "$statsfile" max 4 "%.0f"
} }
echoq number_of_X_users echoq number_of_X_users
number_of_X_users() { number_of_X_users()
datamash_file_op_index G_REMOTE_ADDR.txt unique 3 | tr , '\n' | wc -l {
statsfile=`statsfile REMOTE_ADDR $1`
test -f $statsfile || { echo No exists $statsfile ; return ; }
datamash_file_op_index $statsfile unique 3 | tr , '\n' | wc -l
} }
echoq summary_compute
summary_compute() {
summary_compute_old() {
list_all_logs_generate \ list_all_logs_generate \
&& grep_stats_from_list_all_logs \ && grep_stats_from_list_all_logs \
&& grep_all_stat_from_patterns_list \ && grep_all_stat_from_patterns_list \
&& summary_display && summary_display
} }
echoq 'summary_compute lognamepattern # time depending on lognamepattern'
summary_compute() {
grep_stats_from_list_log_matching "$1" \
&& grep_all_stat_from_patterns_list "$1" \
&& summary_display "$1"
}
number_of_syncs() number_of_syncs()
{ {
list_all_logs | wc -l list_all_logs | egrep "$1"_ | wc -l
} }
@ -1002,25 +1115,25 @@ server_survey_host2()
} }
echoq summary_display echoq summary_display lognamepattern
summary_display() { summary_display() {
vnstat_gen > /dev/null echo "Start date : " `start_date "$1"`
echo "Start date of /X (aaaa mm dd): `first_use` (`days_since_first_use` days of service)" echo "End date : " `end_date "$1"`
echo -n "Number of /X users: " ; number_of_X_users echo -n "Number of /X users: " ; number_of_X_users "$1"
echo -n "Number of /X accounts synced: " ; nb_migrations_launched echo -n "Number of /X accounts synced: " ; nb_migrations_launched "$1"
echo -n "Number of /X syncs: " ; number_of_syncs echo -n "Number of /X syncs: " ; number_of_syncs "$1"
echo -n "Total volume /X transferred: " ; total_volume_transferred echo -n "Total volume /X transferred: " ; total_volume_transferred "$1"
echo -n "Total messages /X transferred: " ; total_messages_transferred echo -n "Total messages /X transferred: " ; total_messages_transferred "$1"
echo -n "Biggest transfer: " ; biggest_transfer echo -n "Biggest transfer: " ; biggest_transfer "$1"
echo -n "Biggest message seen: " ; biggest_message_seen echo -n "Biggest message seen: " ; biggest_message_seen "$1"
echo -n "Biggest message transferred: " ; biggest_message_transferred echo -n "Biggest message transferred: " ; biggest_message_transferred "$1"
echo -n "Biggest bandwidth rate: " ; biggest_bandwidth_rate echo -n "Biggest bandwidth rate: " ; biggest_bandwidth_rate "$1"
echo -n "Average bandwidth rate: " ; average_bandwidth_rate echo -n "Average bandwidth rate: " ; average_bandwidth_rate "$1"
echo -n "Max messages transferred: " ; max_number_of_messages_transferred echo -n "Max messages transferred: " ; max_number_of_messages_transferred "$1"
echo -n "Max messages skipped: " ; max_number_of_messages_skipped echo -n "Max messages skipped: " ; max_number_of_messages_skipped "$1"
echo -n "Longest transfer: " ; seconds_to_days_hours `longest_transfer` echo -n "Longest transfer: " ; seconds_to_days_hours `longest_transfer "$1"`
echo -n "Queue length mean is: " ; stat_queue_mean echo -n "Queue length mean is: " ; stat_queue_mean "$1"
echo "Data made at" `date -r grep_stats.txt` echo "Data made at" `date -r grep_stats_"$1".txt`
} }
echoq sync_ks2_i005 echoq sync_ks2_i005
@ -1173,6 +1286,9 @@ while :; do ssh root@ks5 'cd /var/tmp/imapsync_cgi/ ; . cgi_memo ; loop_number_o
cat G_Memory_consumption_at_the_end.txt | sort -g -k7 | grep 202[01] |tail -100 | cut -f1 -d: | while read f; do echo $f ; grep 'Memory consumption at the end' $f; grep 'Host. Nb messages' $f ; grep 'Biggest message' $f ; grep 'Memory/biggest message ratio' $f ; done cat G_Memory_consumption_at_the_end.txt | sort -g -k7 | grep 202[01] |tail -100 | cut -f1 -d: | while read f; do echo $f ; grep 'Memory consumption at the end' $f; grep 'Host. Nb messages' $f ; grep 'Biggest message' $f ; grep 'Memory/biggest message ratio' $f ; done
cat G_Host2_Nb_messages.txt | sort -g -k4 | grep 202[01] |tail -100 | cut -f1 -d: | while read f; do echo $f ; grep 'Memory consumption at the end' $f; grep 'Host. Nb messages' $f ; grep 'Biggest message' $f ; grep 'Memory/biggest message ratio' $f ; done cat G_Host2_Nb_messages.txt | sort -g -k4 | grep 202[01] |tail -100 | cut -f1 -d: | while read f; do echo $f ; grep 'Memory consumption at the end' $f; grep 'Host. Nb messages' $f ; grep 'Biggest message' $f ; grep 'Memory/biggest message ratio' $f ; done
cat G_Host1_Nb_messages.txt | sort -g -k4 | grep 202[01] |tail -100 | cut -f1 -d: | while read f; do echo $f ; grep 'Memory consumption at the end' $f; grep 'Host. Nb messages' $f ; grep 'Biggest message' $f ; grep 'Memory/biggest message ratio' $f ; done cat G_Host1_Nb_messages.txt | sort -g -k4 | grep 202[01] |tail -100 | cut -f1 -d: | while read f; do echo $f ; grep 'Memory consumption at the end' $f; grep 'Host. Nb messages' $f ; grep 'Biggest message' $f ; grep 'Memory/biggest message ratio' $f ; done
# Best bandwidth moments
cat /var/tmp/number_of_imapsync_running_every_60s.txt | sort -k3 -g| tail -66
EOF EOF
} }
@ -1181,13 +1297,13 @@ echoq perf_help
perf_help() { perf_help() {
test FreeBSD = `uname -s` && { test FreeBSD = `uname -s` && {
echo FreeBSD here echo FreeBSD here
echo "nload -t 6000 em0 -u K" echo "nload -t 6000 em0 -u K -i 100000 -o 100000"
echo "iftop -i em0 -f 'port imap or port imaps' -B # t p >" echo "iftop -i em0 -f 'port imap or port imaps' -B # t p >"
} }
test Linux = `uname -s` && { test Linux = `uname -s` && {
echo Linux here echo Linux here
echo "nload -t 6000 eth0 -u K # Linux" echo "nload -t 6000 eth0 -u K -i 100000 -o 100000 # Linux"
echo "iftop -i eth0 -f 'port imap or port imaps' -B # t p >" echo "iftop -i eth0 -f 'port imap or port imaps' -B # t p >"
} }
} }

218
X/imapsync_csv_wrapper
View file

@ -1,11 +1,14 @@
#!/bin/sh #!/bin/sh
# $Id: imapsync_csv_wrapper,v 1.5 2020/05/01 21:26:30 gilles Exp gilles $ # $Id: imapsync_csv_wrapper,v 1.13 2021/11/01 13:24:47 gilles Exp gilles $
exec 3>&1 exec 3>&1
debug=/bin/true debug=true
debug=/bin/false # comment the following to have debug data
debug=false
PATH=$PATH:/home/www/apache24/cgi-bin/
echo3() { echo3() {
echo "$@" >&3 echo "$@" >&3
@ -83,8 +86,9 @@ tests_uri_unescape()
run_test test '"_a\n\n\nz_" = "_`uri_unescape a%0A%0A%0Az`_"' run_test test '"_a\n\n\nz_" = "_`uri_unescape a%0A%0A%0Az`_"'
run_test test '"_`uri_unescape a%0A%0A%0Az`_" = "_`uri_unescape a%0A%0A%0Az`_"' run_test test '"_`uri_unescape a%0A%0A%0Az`_" = "_`uri_unescape a%0A%0A%0Az`_"'
run_test test 'A B' = 'A B' run_test test 'A B' = 'A B'
uri_unescape 'a%0A%0A%0Az' > /tmp/$$_zzz1.txt uri_unescape 'a%0Az' > /tmp/$$_zzz1.txt
/bin/echo -e "a\n\n\nz" > /tmp/$$_zzz2.txt /bin/echo "a
z" > /tmp/$$_zzz2.txt
run_test diff /tmp/$$_zzz1.txt /tmp/$$_zzz2.txt run_test diff /tmp/$$_zzz1.txt /tmp/$$_zzz2.txt
rm /tmp/$$_zzz1.txt /tmp/$$_zzz2.txt rm /tmp/$$_zzz1.txt /tmp/$$_zzz2.txt
z3=`uri_unescape a%0A%0A%0Az` z3=`uri_unescape a%0A%0A%0Az`
@ -208,12 +212,28 @@ tests_is_blank_or_comment()
logfailures() logfailures()
{ {
: run_on_each_line_failures="$run_on_each_line_failures
$@"
} }
tests_logfailures() tests_logfailures()
{ {
: #reset
run_on_each_line_failures=
# add Blabla
run_test logfailures Blabla
run_test test "
Blabla" = "$run_on_each_line_failures"
# add Kiki
run_test logfailures Kiki
run_test test "
Blabla
Kiki" = "$run_on_each_line_failures"
#reset again
run_on_each_line_failures=
} }
@ -228,12 +248,6 @@ run_on_each_line()
csv_data_multilines="$1" csv_data_multilines="$1"
echodebug "csv_data_multilines=[$csv_data_multilines]" echodebug "csv_data_multilines=[$csv_data_multilines]"
test 1 -le "$#" && shift
failures_file=${1:-""}
echodebug failures_file="$failures_file"
test -x "$failures_file" && { echo3 "failure file $failures_file is an executable. Leaving run_on_each_line()." ; return ; }
test 1 -le "$#" && shift test 1 -le "$#" && shift
command=${1:-echo} command=${1:-echo}
echodebug command="$command" "$@" echodebug command="$command" "$@"
@ -244,21 +258,27 @@ run_on_each_line()
line_number=0 line_number=0
csv_number=0 csv_number=0
failures_number=0 failures_number=0
echo "$csv_data_multilines" | while IFS=';' read h1 u1 p1 h2 u2 p2 fake; do while IFS=';' read h1 u1 p1 h2 u2 p2 extra fake; do
test -f abort_$$.txt && { echo abort demanded so not doing normal stuff ; return ; } test -f abort_$$.txt && { echo abort demanded so not doing normal stuff ; return ; }
line_number=`expr 1 + $line_number` line_number=`expr 1 + $line_number`
if is_blank_or_comment "$h1$u1$p1$h2$u2$p2$fake" ; then if is_blank_or_comment "$h1$u1$p1$h2$u2$p2$extra" ; then
echo3 "Ignoring line " $line_number "$h1$u1$p1$h2$u2$p2$fake" echo3 "Ignoring line " $line_number "$h1$u1$p1$h2$u2$p2$extra"
else else
csv_number=`expr 1 + $csv_number` csv_number=`expr 1 + $csv_number`
echo3 "Processing line " $line_number run $csv_number "[$h1] [$u1] [xxx] [$h2] [$u2] [xxx] [$fake] $$" echo3 "Processing line " $line_number csv run $csv_number "[$h1] [$u1] [xxx] [$h2] [$u2] [xxx] [$extra] $$"
$command "$@" --host1 "$h1" --user1 "$u1" --password1 "$p1" --host2 "$h2" --user2 "$u2" --password2 "$p2" $command "$@" --host1 "$h1" --user1 "$u1" --password1 "$p1" --host2 "$h2" --user2 "$u2" --password2 "$p2" $extra
command_status=$? command_status=$?
test "0" != "$command_status" && failures_number=`expr 1 + $failures_number` test "0" != "$command_status" && failures_number=`expr 1 + $failures_number`
logfailures "$failures_file" "$command_status" "$line_number" "$csv_number" echodebug command_status=$command_status failures_number=$failures_number
test "0" != "$command_status" && logfailures "line $line_number csv run $csv_number returned with status $command_status ($h1;$u1;$p1;$h2;$u2;$p2;$extra;)"
fi fi
done done << EOF
echodebug Leaving run_on_each_line $csv_data_multilines
EOF
echodebug Leaving run_on_each_line, failures_number=$failures_number
test "0" != "$failures_number" && echo "$run_on_each_line_failures"
test "0" != "$failures_number" && return 1 test "0" != "$failures_number" && return 1
return 0 return 0
} }
@ -268,19 +288,19 @@ tests_run_on_each_line()
# 1-4 # 1-4
run_test test '' = "`run_on_each_line`" run_test test '' = "`run_on_each_line`"
run_test test '' = "`run_on_each_line ''`" run_test test '' = "`run_on_each_line ''`"
run_test test '' = "`run_on_each_line '' 'log.txt'`" run_test test '' = "`run_on_each_line '' echo imapsync`"
run_test test '' = "`run_on_each_line '' 'log.txt' echo imapsync`" run_test test '' = "`run_on_each_line '' echo imapsync blabla blibli`"
# 5-7 blank data # 5-7 blank data
run_test test '' = "`run_on_each_line ' '`" run_test test '' = "`run_on_each_line ' '`"
run_test test '' = "`run_on_each_line ' ' 'log.txt'`" run_test test '' = "`run_on_each_line ' ' `"
run_test test '' = "`run_on_each_line ' ' 'log.txt' echo imapsync`" run_test test '' = "`run_on_each_line ' ' echo imapsync blabla blibli`"
# 8-10 comment data # 8-10 comment data
run_test test '' = "`run_on_each_line '# '`" run_test test '' = "`run_on_each_line '# '`"
run_test test '' = "`run_on_each_line '# ' 'log.txt'`" run_test test '' = "`run_on_each_line ' # '`"
run_test test '' = "`run_on_each_line '# ' 'log.txt' echo imapsync`" run_test test '' = "`run_on_each_line '# ' echo imapsync blabla blibli`"
# 11-13 empty line # 11-13 empty line
myval=' myval='
@ -289,10 +309,10 @@ tests_run_on_each_line()
myret=`run_on_each_line "$myval"` myret=`run_on_each_line "$myval"`
run_test test "" = "$myret" run_test test "" = "$myret"
myret=`run_on_each_line "$myval" 'log.txt'` myret=`run_on_each_line "$myval"`
run_test test "" = "$myret" run_test test "" = "$myret"
myret=`run_on_each_line "$myval" 'log.txt' echo imapsync` myret=`run_on_each_line "$myval" echo imapsync blabla blibli`
run_test test "" = "$myret" run_test test "" = "$myret"
@ -308,39 +328,33 @@ tests_run_on_each_line()
myret=`run_on_each_line "$myval"` myret=`run_on_each_line "$myval"`
run_test test "" = "$myret" run_test test "" = "$myret"
myret=`run_on_each_line "$myval" 'log.txt'` myret=`run_on_each_line "$myval" `
run_test test "" = "$myret" run_test test "" = "$myret"
myret=`run_on_each_line "$myval" 'log.txt' echo imapsync` myret=`run_on_each_line "$myval" echo imapsync blabla blibli`
run_test test "" = "$myret" run_test test "" = "$myret"
# 17-21 # 17-18
# csv data no blank no comment # csv data no blank no comment
lines='a1;b1;c1;d1;e1;f1 lines='a1;b1;c1;d1;e1;f1
a2;b2;c2;d2;e2;f2 a2;b2;c2;d2;e2;f2
a3;b3;c3;d3;e3;f3' a3;b3;c3;d3;e3;f3'
# No failure file (so /dev/null), no command (so echo) # no command (so echo)
lines_out='--host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1 lines_out='--host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1
--host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2 --host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2
--host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3' --host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3'
run_test test "$lines_out" = "`run_on_each_line "$lines"`" run_test test "$lines_out" = "`run_on_each_line "$lines"`"
# failure file is a command => BAD => do nothing
cp /bin/echo "/tmp/myecho$$"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$`"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$ echo`"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$ echo imapsync blabla bla`"
rm "/tmp/myecho$$"
# No failure file (so /dev/null), command = echo imapsync # command = echo imapsync
lines_out='imapsync --host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1 lines_out='imapsync --host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1
imapsync --host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2 imapsync --host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2
imapsync --host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3' imapsync --host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3'
run_test test "$lines_out" = "`run_on_each_line "$lines" '' echo imapsync`" run_test test "$lines_out" = "`run_on_each_line "$lines" echo imapsync`"
# 22-27 # 19-20
# csv data + blanks + comments # csv data + blanks + comments
lines='a1;b1;c1;d1;e1;f1 lines='a1;b1;c1;d1;e1;f1
# blabla 2 # blabla 2
@ -349,33 +363,96 @@ a2;b2;c2;d2;e2;f2
a3;b3;c3;d3;e3;f3 a3;b3;c3;d3;e3;f3
# end' # end'
# No failure file (so /dev/null), no command (so echo) # no command (so echo)
lines_out='--host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1 lines_out='--host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1
--host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2 --host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2
--host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3' --host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3'
run_test test "$lines_out" = "`run_on_each_line "$lines"`" run_test test "$lines_out" = "`run_on_each_line "$lines"`"
# failure file is a command => BAD => do nothing # command = echo imapsync
cp /bin/echo "/tmp/myecho$$"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$`"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$ echo`"
run_test test "" = "`run_on_each_line "$lines" /tmp/myecho$$ echo imapsync blabla bla`"
rm "/tmp/myecho$$"
# No failure file (so /dev/null), command = echo imapsync
lines_out='imapsync --host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1 lines_out='imapsync --host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1
imapsync --host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2 imapsync --host1 a2 --user1 b2 --password1 c2 --host2 d2 --user2 e2 --password2 f2
imapsync --host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3' imapsync --host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3'
run_test test "$lines_out" = "`run_on_each_line "$lines" '' echo imapsync`" run_test test "$lines_out" = "`run_on_each_line "$lines" echo imapsync`"
# 21-22
# csv data 3/3;/6/6;/7/7;/8/8; parameters + blanks + comments
# runs 1 2 3 4 5 6 7 8
lines='a1;b1;c1
# blabla 2
a2;b2;c2;
# bloblo 3
a3;b3;c3;d3;e3;f3
a4;b4;c4;d4;e4;f4;
a5;b5;c5;d5;e5;f5;--extra blurp
a6;b6;c6;d6;e6;f6;--extra blurp;
a7;b7;c7;d7;e7;f7;--extra blurp; ignored
a8;b8;c8;d8;e8;f8;--extra blurp; ignored;
# end'
# no command (so echo)
lines_out='--host1 a1 --user1 b1 --password1 c1 --host2 --user2 --password2
--host1 a2 --user1 b2 --password1 c2 --host2 --user2 --password2
--host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3
--host1 a4 --user1 b4 --password1 c4 --host2 d4 --user2 e4 --password2 f4
--host1 a5 --user1 b5 --password1 c5 --host2 d5 --user2 e5 --password2 f5 --extra blurp
--host1 a6 --user1 b6 --password1 c6 --host2 d6 --user2 e6 --password2 f6 --extra blurp
--host1 a7 --user1 b7 --password1 c7 --host2 d7 --user2 e7 --password2 f7 --extra blurp
--host1 a8 --user1 b8 --password1 c8 --host2 d8 --user2 e8 --password2 f8 --extra blurp'
run_test test "$lines_out" = "`run_on_each_line "$lines"`"
# command = echo imapsync
lines_out='imapsync --host1 a1 --user1 b1 --password1 c1 --host2 --user2 --password2
imapsync --host1 a2 --user1 b2 --password1 c2 --host2 --user2 --password2
imapsync --host1 a3 --user1 b3 --password1 c3 --host2 d3 --user2 e3 --password2 f3
imapsync --host1 a4 --user1 b4 --password1 c4 --host2 d4 --user2 e4 --password2 f4
imapsync --host1 a5 --user1 b5 --password1 c5 --host2 d5 --user2 e5 --password2 f5 --extra blurp
imapsync --host1 a6 --user1 b6 --password1 c6 --host2 d6 --user2 e6 --password2 f6 --extra blurp
imapsync --host1 a7 --user1 b7 --password1 c7 --host2 d7 --user2 e7 --password2 f7 --extra blurp
imapsync --host1 a8 --user1 b8 --password1 c8 --host2 d8 --user2 e8 --password2 f8 --extra blurp'
run_test test "$lines_out" = "`run_on_each_line "$lines" echo imapsync`"
# 23-2x
# "" input
lines='a1;b1;c1;d1;e1;f1;--caca "KK"'
# no command (so echo)
lines_out='--host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1 --caca "KK"'
run_test test "$lines_out" = "`run_on_each_line "$lines"`"
# command = echo imapsync
lines_out='imapsync --host1 a1 --user1 b1 --password1 c1 --host2 d1 --user2 e1 --password2 f1 --caca "KK"'
run_test test "$lines_out" = "`run_on_each_line "$lines" echo imapsync`"
} }
remove_double_quotes()
{
echo "$1" | tr -d '"'
}
tests_remove_double_quotes()
{
echo 'Entering tests_remove_double_quotes()'
run_test test -z `remove_double_quotes`
run_test test 'abc' = `remove_double_quotes 'abc'`
run_test test -z `remove_double_quotes '"'`
run_test test -z `remove_double_quotes '""'`
run_test test 'abc' = `remove_double_quotes '"ab"c'`
echo 'Leaving tests_remove_double_quotes()'
}
hashsync() hashsync()
{ {
#set -x #set -x
mystring=${1:-''} mystring=`remove_double_quotes ${1:-''}`
mykey=${2:-''} mykey=${2:-''}
# impressive! is not it? quoting shit! # impressive! is not it? quoting shit!
perl -MDigest::HMAC_SHA1 -e 'print Digest::HMAC_SHA1::hmac_sha1_hex( "'"$mystring"'", "'"$mykey"'" )' perl -MDigest::HMAC_SHA1 -e 'print Digest::HMAC_SHA1::hmac_sha1_hex( "'"$mystring"'", "'"$mykey"'" )'
@ -385,11 +462,16 @@ hashsync()
tests_hashsync() tests_hashsync()
{ {
echo 'Entering tests_hashsync()'
run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync` run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync`
run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync ''` run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync ''`
run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync '' ''` run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync '' ''`
run_test test 'e86a28a3611c1e7bbaf8057cd00ae122781a11fe' = `hashsync 'zzz' ''` run_test test 'e86a28a3611c1e7bbaf8057cd00ae122781a11fe' = `hashsync 'zzz' ''`
run_test test '6a7b451ac99eab1531ad8e6cd544b32420c552ac' = `hashsync 'zzz' 'A'` run_test test '6a7b451ac99eab1531ad8e6cd544b32420c552ac' = `hashsync 'zzz' 'A'`
run_test test 'fbdb1d1b18aa6c08324b7d64b71fb76370690e1d' = `hashsync '""'`
run_test test 'e86a28a3611c1e7bbaf8057cd00ae122781a11fe' = `hashsync '"zzz"'`
echo 'Leaving tests_hashsync()'
} }
tests_caca() tests_caca()
@ -480,7 +562,9 @@ tests()
tests_run_on_each_line \ tests_run_on_each_line \
tests_hashsync \ tests_hashsync \
tests_is_blank_or_comment \ tests_is_blank_or_comment \
tests_logfailures tests_logfailures \
tests_remove_double_quotes \
} }
@ -595,7 +679,7 @@ justlogin()
date date
test -f abort_$$.txt && { echo Abort demanded so not doing normal stuff ; return ; } test -f abort_$$.txt && { echo Abort demanded so not doing normal stuff ; return ; }
echo Now run imapsync --justlogin to check all credentials are ok echo Now run imapsync --justlogin to check all credentials are ok
if run_on_each_line "$csv_data_unescaped" "" \ if run_on_each_line "$csv_data_unescaped" \
imapsync --no-modulesversion --justlogin --tmpdir . imapsync --no-modulesversion --justlogin --tmpdir .
then then
echo success of the just login check round echo success of the just login check round
@ -612,8 +696,8 @@ justfoldersizes()
######################## just folders sizes counting round #####################' ######################## just folders sizes counting round #####################'
date date
test -f abort_$$.txt && { echo Abort demanded so not doing normal stuff ; return ; } test -f abort_$$.txt && { echo Abort demanded so not doing normal stuff ; return ; }
echo Now run imapsync --justlogin to check all credentials are ok echo Now run imapsync --justfoldersizes to show the volume to be synced
run_on_each_line "$csv_data_unescaped" "" \ run_on_each_line "$csv_data_unescaped" \
imapsync --no-modulesversion --justfoldersizes --tmpdir . \ imapsync --no-modulesversion --justfoldersizes --tmpdir . \
|| return 1 || return 1
} }
@ -625,8 +709,8 @@ sync_all()
######################## now sync them all round ###############################' ######################## now sync them all round ###############################'
date date
test -f abort_$$.txt && { echo Abort demanded so not doing normal stuff ; return ; } test -f abort_$$.txt && { echo Abort demanded so not doing normal stuff ; return ; }
echo Now run imapsync --justlogin to check all credentials are ok echo Now run imapsync on the data given
run_on_each_line "$csv_data_unescaped" "" \ run_on_each_line "$csv_data_unescaped" \
imapsync --no-modulesversion --tmpdir . \ imapsync --no-modulesversion --tmpdir . \
|| return 1 || return 1
} }
@ -647,11 +731,13 @@ main()
# All stderr to stdin, yeah I am like that # All stderr to stdin, yeah I am like that
exec 2>&1 exec 2>&1
# Some tests # Some tests, uncomment which one you work with and also the return one
#run_tests tests_is_blank_or_comment #run_tests tests_is_blank_or_comment
#run_tests tests_run_on_each_line #run_tests tests_run_on_each_line
#run_tests tests_remove_blanks_and_comments #run_tests tests_remove_blanks_and_comments
#tests_hashsync #tests_hashsync
#run_tests tests_logfailures
#tests_remove_double_quotes
#return #return
tests_all_verbose_if_failure || return 1 tests_all_verbose_if_failure || return 1
@ -662,7 +748,9 @@ main()
getcsvdata getcsvdata
hashsync=`hashsync $csv_data_pure` hashsync=`hashsync "$csv_data_pure"`
test -z $hashsync && return 1
change_working_directory_if_under_cgi /var/tmp/imapsync_cgi/$hashsync/ change_working_directory_if_under_cgi /var/tmp/imapsync_cgi/$hashsync/
@ -678,9 +766,8 @@ main()
unset SERVER_SOFTWARE unset SERVER_SOFTWARE
# Now the real stuff # Now the real stuff
justlogin || return 1 #justlogin || return 1
#justfoldersizes
justfoldersizes
sync_all || return 1 sync_all || return 1
@ -694,6 +781,3 @@ main()
tests_initialisation tests_initialisation
main main
return 0 return 0

6
X/imapsync_form.css
View file

@ -1,6 +1,6 @@
/* $Id: imapsync_form.css,v 1.9 2020/06/15 11:55:50 gilles Exp gilles $ */ /* $Id: imapsync_form.css,v 1.10 2021/09/21 09:25:53 gilles Exp gilles $ */
body { body {
color: black; color: black;
@ -29,6 +29,10 @@ body {
height: 50px ; height: 50px ;
} }
#imapsync_log_bottom {
scroll-margin-top: 300px ;
}
.progress { .progress {
/* text-align: center ; */ /* text-align: center ; */
/* font-size: 1.4em ; */ /* font-size: 1.4em ; */

62
X/imapsync_form.js
View file

@ -1,5 +1,5 @@
// $Id: imapsync_form.js,v 1.20 2021/05/29 10:25:53 gilles Exp gilles $ // $Id: imapsync_form.js,v 1.26 2021/12/18 19:01:52 gilles Exp gilles $
/*jslint browser: true*/ /*global $*/ /*jslint browser: true*/ /*global $*/
@ -55,7 +55,30 @@ $(document).ready(
var note = function note( message ) var note = function note( message )
{ {
$("#tests").append( message ) ; $("#tests").append( message ) ;
} } ;
var tests_last_x_lines = function tests_last_x_lines()
{
is( "", last_x_lines(), "last_x_lines: no args => empty string" ) ;
is( "", last_x_lines(""), "last_x_lines: empty string => empty string" ) ;
is( "abc", last_x_lines("abc"), "last_x_lines: abc => abc" ) ;
is( "abc\ndef", last_x_lines("abc\ndef"), "last_x_lines: abc\ndef => abc\ndef" ) ;
is( "def", last_x_lines("abc\ndef", -1), "last_x_lines: abc\ndef -1 => def\n" ) ;
is( "", last_x_lines("abc\ndef", 0), "last_x_lines: abc\ndef 0 => empty string" ) ;
is( "abc\ndef", last_x_lines("abc\ndef", -10), "last_x_lines: last 10 of 2 lines => 2 lines" ) ;
is( "4\n5\n", last_x_lines("1\n2\n3\n4\n5\n", -3), "last_x_lines: last 3 lines of 5 lines" ) ;
is( "3\n4\n5", last_x_lines("1\n2\n3\n4\n5", -3), "last_x_lines: last 3 lines of 5 lines" ) ;
} ;
var last_x_lines = function last_x_lines( string, num )
{
if ( undefined === string || 0 === num )
{
return "" ;
}
return string.split(/\r?\n/).slice(num).join("\n") ;
} ;
var last_eta = function last_eta( string ) var last_eta = function last_eta( string )
{ {
@ -83,6 +106,8 @@ $(document).ready(
} }
} }
var tests_last_eta = function tests_last_eta() var tests_last_eta = function tests_last_eta()
{ {
is( "", last_eta( ), "last_eta: no args => empty string" ) ; is( "", last_eta( ), "last_eta: no args => empty string" ) ;
@ -257,7 +282,7 @@ $(document).ready(
} }
else else
{ {
slice_length = -240 ; slice_length = -2400 ;
} }
slice_log = xhr.responseText.slice( slice_length ) ; slice_log = xhr.responseText.slice( slice_length ) ;
eta_str = last_eta( slice_log ) ; eta_str = last_eta( slice_log ) ;
@ -297,24 +322,24 @@ $(document).ready(
if ( xhr.readyState === 4 ) if ( xhr.readyState === 4 )
{ {
// end of sync // end of sync
$("#progress-txt").text( $("#progress-txt").text(
"Ended. It remains " "Ended. It remains "
+ eta_obj.msgs_left + " messages to be synced" ) ; + eta_obj.msgs_left + " messages to be synced" ) ;
$( "#output" ).text( xhr.responseText ) ;
} }
else else
{ {
eta_str = eta_obj.str + " (refresh every " + refresh_interval_s + " s)" ; eta_str = eta_obj.str + " (refresh every " + refresh_interval_s + " s)" ;
eta_str = eta_str.replace(/(\r\n|\n|\r)/gm, "") ; // trim newline eta_str = eta_str.replace(/(\r\n|\n|\r)/gm, "") ; // trim newlines
//$("#tests").append( "refreshLog eta_str: " + eta_str + "\n" ) ; //$("#tests").append( "refreshLog eta_str: " + eta_str + "\n" ) ;
$("#progress-txt").text( eta_str ) ; $( "#progress-txt" ).text( eta_str ) ;
var last_lines = last_x_lines( xhr.responseText.slice(-2000), -10)
$( "#output" ).text( last_lines ) ;
} }
$( "#output" ).text( xhr.responseText ) ;
} }
var handleRun = function handleRun(xhr, timerRefreshLog) var handleRun = function handleRun(xhr, timerRefreshLog)
{ {
@ -326,7 +351,6 @@ $(document).ready(
// var headers = xhr.getAllResponseHeaders(); // var headers = xhr.getAllResponseHeaders();
// $("#console").append(headers); // $("#console").append(headers);
// $("#console").append("See the completed log\n"); // $("#console").append("See the completed log\n");
$("#link_to_bottom").show() ;
clearInterval( timerRefreshLog ) ; clearInterval( timerRefreshLog ) ;
refreshLog( xhr ) ; // a last time refreshLog( xhr ) ; // a last time
// back to enable state for next run // back to enable state for next run
@ -650,7 +674,6 @@ $(document).ready(
// in case of a manual refresh, start with // in case of a manual refresh, start with
$("#bt-sync").prop("disabled", false); $("#bt-sync").prop("disabled", false);
$("#bt-abort").prop("disabled", false); $("#bt-abort").prop("disabled", false);
$("#link_to_bottom").hide();
$("#progress-bar-left").css( "width", 100 + "%" ).attr( "aria-valuenow", 100 ) ; $("#progress-bar-left").css( "width", 100 + "%" ).attr( "aria-valuenow", 100 ) ;
$("#showpassword1").click( $("#showpassword1").click(
@ -678,7 +701,6 @@ $(document).ready(
$( "#imapsync_current" ).load( "imapsync_current.txt" ) ; $( "#imapsync_current" ).load( "imapsync_current.txt" ) ;
$("#bt-sync").prop("disabled", true) ; $("#bt-sync").prop("disabled", true) ;
$("#bt-abort").prop("disabled", false) ; $("#bt-abort").prop("disabled", false) ;
$("#link_to_bottom").hide() ;
$("#progress-txt").text( "ETA: coming soon" ) ; $("#progress-txt").text( "ETA: coming soon" ) ;
store_form() ; store_form() ;
imapsync() ; imapsync() ;
@ -729,6 +751,9 @@ $(document).ready(
if ( "imapsync.lamiral.info" === location.hostname ) if ( "imapsync.lamiral.info" === location.hostname )
{ {
$( "#status_24h" ).attr('src', 'https://lstu.fr/imapsync_online_status_24h_1200x70') ;
$( "#status_7d" ).attr('src', 'https://lstu.fr/imapsync_online_status_7d') ;
$( "#status_2m" ).attr('src', 'https://lstu.fr/imapsync_online_status_2months_1200x70') ;
$( "#local_bandwidth" ).collapse( "show" ) ; $( "#local_bandwidth" ).collapse( "show" ) ;
$( "#local_status_dbmon" ).collapse( "show" ) ; $( "#local_status_dbmon" ).collapse( "show" ) ;
$( "#local_status_hetrix" ).collapse( "show" ) ; $( "#local_status_hetrix" ).collapse( "show" ) ;
@ -737,6 +762,7 @@ $(document).ready(
} }
else if ( "lamiral.info" === location.hostname ) else if ( "lamiral.info" === location.hostname )
{ {
$( "#local_bandwidth" ).collapse( "show" ) ; $( "#local_bandwidth" ).collapse( "show" ) ;
$( "#local_status_dbmon" ).collapse( "show" ) ; $( "#local_status_dbmon" ).collapse( "show" ) ;
$( "#local_status_hetrix" ).collapse( "show" ) ; $( "#local_status_hetrix" ).collapse( "show" ) ;
@ -778,6 +804,7 @@ $(document).ready(
tests_store_retrieve( ) ; tests_store_retrieve( ) ;
tests_last_eta( ) ; tests_last_eta( ) ;
tests_decompose_eta_line( ) ; tests_decompose_eta_line( ) ;
tests_last_x_lines( ) ;
// tests_cryptojs( ) ; // tests_cryptojs( ) ;
// The following test can be used to check that if a test fails // The following test can be used to check that if a test fails
@ -785,15 +812,16 @@ $(document).ready(
//is( 0, 1, "this test always fails" ) ; //is( 0, 1, "this test always fails" ) ;
tests_bilan( nb_attended_test ) ; tests_bilan( nb_attended_test ) ;
// If you want to always see the tests, uncomment the following // If you want to always see the tests, uncomment the following
// line // line
// $("#tests").collapse("show") ; //$("#tests").collapse("show") ;
} }
} }
init( ) ; init( ) ;
tests( 29 ) ; tests( 38 ) ;
retrieve_form( ) ; retrieve_form( ) ;
} }

35
X/imapsync_form_extra.html
View file

@ -1,6 +1,6 @@
<!DOCTYPE html> <!DOCTYPE html>
<!-- $Id: imapsync_form_extra.html,v 1.18 2021/05/30 05:45:15 gilles Exp gilles $ --> <!-- $Id: imapsync_form_extra.html,v 1.22 2021/12/18 19:03:46 gilles Exp gilles $ -->
<html lang="en" id="top"> <html lang="en" id="top">
@ -273,7 +273,12 @@
<a id="buttons"></a> <a id="buttons"></a>
<hr> <hr>
<!-- Classical button to go to the log only --> <div class="text-center center-block">
If you <b>close</b> this <b>window</b> (or tab) <b>during</b> the synchronization,
it will <b>abort</b> the synchronization, it's like <b>hitting</b> the red button <b>"Abort!"</b> below.
</div>
<!-- Classical button to go to the log only, when javascript is turned off -->
<noscript> <noscript>
<div class="row"> <div class="row">
<div class="col-sm-12 padd0" > <div class="col-sm-12 padd0" >
@ -336,7 +341,6 @@
</pre> </pre>
<a id="link_to_bottom" href="#bottom">Bottom of imapsync log</a>
</div> </div>
<div class="col-sm-6 well"> <div class="col-sm-6 well">
@ -351,7 +355,10 @@
</div> </div>
<h2 class="text-center scripton">Log of imapsync run</h2> <h2 id="imapsync_log_beginning" class="text-center scripton">Log of imapsync run</h2>
<div class="text-center scripton">
<a href="#imapsync_log_bottom">Link to the <b>bottom</b> of the imapsync log file</a>
</div>
<pre id="output" class="scripton"> <pre id="output" class="scripton">
@ -374,6 +381,10 @@
</pre> </pre>
<div id="imapsync_log_bottom" class="text-center scripton">
<a href="#imapsync_log_beginning">Link to the <b>beginning</b> of the imapsync log file</a>
</div>
<div id="local_bandwidth" class="collapse"> <div id="local_bandwidth" class="collapse">
<hr> <hr>
<p class="text-center"> <p class="text-center">
@ -390,17 +401,17 @@
<hr> <hr>
<p class="text-center"> <p class="text-center">
Imapsync <b>Online Status</b> over the <b>last 24h</b><br> Imapsync <b>Online Status</b> over the <b>last 24h</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_24h_1200x70"> <img id="status_24h" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_24h_1200x70.png">
</p> </p>
<p class="text-center"> <p class="text-center">
Imapsync <b>Online Status</b> over the <b>last week</b><br> Imapsync <b>Online Status</b> over the <b>last week</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_7d"> <img id="status_7d" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_7d_1200x70.png">
</p> </p>
<p class="text-center"> <p class="text-center">
Imapsync <b>Online Status</b> over the <b>last two months</b><br> Imapsync <b>Online Status</b> over the <b>last two months</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_2months_1200x70"><br> <img id="status_2m" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_2months_1200x70.png"><br>
</p> </p>
@ -411,19 +422,19 @@ Uncomment the following when the data are available
<p class="text-center"> <p class="text-center">
Imapsync <b>Online Status</b> over the <b>last year</b><br> Imapsync <b>Online Status</b> over the <b>last year</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_1year_1200x70"><br> <img id="status_1y" class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_1year_1200x70"><br>
</p> </p>
<p class="text-center"> <p class="text-center">
Imapsync <b>Online Status</b> over the <b>last 10 years</b><br> Imapsync <b>Online Status</b> over the <b>last 10 years</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_10years_1200x70-2"><br> <img id="status_10y" class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_10years_1200x70-2"><br>
</p> </p>
--> -->
</div> </div>
<div id="local_status_hetrix" class="collapse in"> <div id="local_status_hetrix" class="collapse">
<hr> <hr>
<p class="text-center"> <p class="text-center">
@ -452,7 +463,7 @@ monitor page powered by the <a href="https://hetrixtools.com/uptime-monitor/4143
<!-- <a href="#buttons" class="btn btn-info scripton" role="button">Consoles</a> --> <!-- <a href="#buttons" class="btn btn-info scripton" role="button">Consoles</a> -->
<a href="#bottom" title="Bottom of the page" class="btn btn-info active" role="button">Bottom</a> <a href="#bottom" title="Bottom of the page" class="btn btn-info active" role="button">Bottom</a>
<br> <br>
<small> ($Id: imapsync_form_extra.html,v 1.18 2021/05/30 05:45:15 gilles Exp gilles $) </small><br> <small> ($Id: imapsync_form_extra.html,v 1.22 2021/12/18 19:03:46 gilles Exp gilles $) </small><br>
Terms and conditions for anything: <a href="https://imapsync.lamiral.info/LICENSE">No limits to do anything with this work and this license!</a><br> Terms and conditions for anything: <a href="https://imapsync.lamiral.info/LICENSE">No limits to do anything with this work and this license!</a><br>
</div> </div>
</div> </div>
@ -482,7 +493,5 @@ src="imapsync_form.js"
> >
</script> </script>
<img src="https://static.scarf.sh/a.png?x-pxid=28b508c3-c51f-4109-9b88-29f3f6e4cf2f" alt="Scarf Tracker"/>
</body> </body>
</html> </html>

36
X/imapsync_form_extra_free.html
View file

@ -1,6 +1,6 @@
<!DOCTYPE html> <!DOCTYPE html>
<!-- $Id: imapsync_form_extra_free.html,v 1.18 2021/05/30 05:45:00 gilles Exp gilles $ --> <!-- $Id: imapsync_form_extra_free.html,v 1.23 2021/12/18 19:03:46 gilles Exp gilles $ -->
<html lang="en" id="top"> <html lang="en" id="top">
@ -332,7 +332,12 @@ It's <b>ok</b> if you <b>don't</b> have a VAT number.<br />
<a id="buttons"></a> <a id="buttons"></a>
<hr> <hr>
<!-- Classical button to go to the log only --> <div class="text-center center-block">
If you <b>close</b> this <b>window</b> (or tab) <b>during</b> the synchronization,
it will <b>abort</b> the synchronization, it's like <b>hitting</b> the red button <b>"Abort!"</b> below.
</div>
<!-- Classical button to go to the log only, when javascript is turned off -->
<noscript> <noscript>
<div class="row"> <div class="row">
<div class="col-sm-12 padd0" > <div class="col-sm-12 padd0" >
@ -395,7 +400,7 @@ It's <b>ok</b> if you <b>don't</b> have a VAT number.<br />
</pre> </pre>
<a id="link_to_bottom" href="#bottom">Bottom of imapsync log</a>
</div> </div>
<div class="col-sm-6 well"> <div class="col-sm-6 well">
@ -410,8 +415,10 @@ It's <b>ok</b> if you <b>don't</b> have a VAT number.<br />
</div> </div>
<h2 class="text-center scripton">Log of imapsync run</h2> <h2 id="imapsync_log_beginning" class="text-center scripton">Log of imapsync run</h2>
<div class="text-center scripton">
<a href="#imapsync_log_bottom">Link to the <b>bottom</b> of the imapsync log file</a>
</div>
<pre id="output" class="scripton"> <pre id="output" class="scripton">
@ -433,6 +440,10 @@ It's <b>ok</b> if you <b>don't</b> have a VAT number.<br />
</pre> </pre>
<div id="imapsync_log_bottom" class="text-center scripton">
<a href="#imapsync_log_beginning">Link to the <b>beginning</b> of the imapsync log file</a>
</div>
<div id="local_bandwidth" class="collapse"> <div id="local_bandwidth" class="collapse">
<hr> <hr>
<p class="text-center"> <p class="text-center">
@ -449,17 +460,17 @@ It's <b>ok</b> if you <b>don't</b> have a VAT number.<br />
<hr> <hr>
<p class="text-center"> <p class="text-center">
Imapsync <b>Online Status</b> over the <b>last 24h</b><br> Imapsync <b>Online Status</b> over the <b>last 24h</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_24h_1200x70"> <img id="status_24h" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_24h_1200x70.png">
</p> </p>
<p class="text-center"> <p class="text-center">
Imapsync <b>Online Status</b> over the <b>last week</b><br> Imapsync <b>Online Status</b> over the <b>last week</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_7d"> <img id="status_7d" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_7d_1200x70.png">
</p> </p>
<p class="text-center"> <p class="text-center">
Imapsync <b>Online Status</b> over the <b>last two months</b><br> Imapsync <b>Online Status</b> over the <b>last two months</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_2months_1200x70"><br> <img id="status_2m" class="img-responsive center-block" alt="Imapsync Online Status" src="imapsync_online_status_2months_1200x70.png"><br>
</p> </p>
@ -470,19 +481,19 @@ Uncomment the following when the data are available
<p class="text-center"> <p class="text-center">
Imapsync <b>Online Status</b> over the <b>last year</b><br> Imapsync <b>Online Status</b> over the <b>last year</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_1year_1200x70"><br> <img id="status_1y" class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_1year_1200x70"><br>
</p> </p>
<p class="text-center"> <p class="text-center">
Imapsync <b>Online Status</b> over the <b>last 10 years</b><br> Imapsync <b>Online Status</b> over the <b>last 10 years</b><br>
<img class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_10years_1200x70-2"><br> <img id="status_10y" class="img-responsive center-block" alt="Imapsync Online Status" src="https://lstu.fr/imapsync_online_status_10years_1200x70-2"><br>
</p> </p>
--> -->
</div> </div>
<div id="local_status_hetrix" class="collapse in"> <div id="local_status_hetrix" class="collapse">
<hr> <hr>
<p class="text-center"> <p class="text-center">
@ -511,7 +522,7 @@ monitor page powered by the <a href="https://hetrixtools.com/uptime-monitor/4143
<!-- <a href="#buttons" class="btn btn-info scripton" role="button">Consoles</a> --> <!-- <a href="#buttons" class="btn btn-info scripton" role="button">Consoles</a> -->
<a href="#bottom" title="Bottom of the page" class="btn btn-info active" role="button">Bottom</a> <a href="#bottom" title="Bottom of the page" class="btn btn-info active" role="button">Bottom</a>
<br> <br>
<small> ($Id: imapsync_form_extra_free.html,v 1.18 2021/05/30 05:45:00 gilles Exp gilles $) </small><br> <small> ($Id: imapsync_form_extra_free.html,v 1.23 2021/12/18 19:03:46 gilles Exp gilles $) </small><br>
Terms and conditions for anything: <a href="https://imapsync.lamiral.info/LICENSE">No limits to do anything with this work and this license!</a><br> Terms and conditions for anything: <a href="https://imapsync.lamiral.info/LICENSE">No limits to do anything with this work and this license!</a><br>
</div> </div>
</div> </div>
@ -541,7 +552,6 @@ src="imapsync_form.js"
> >
</script> </script>
<img src="https://static.scarf.sh/a.png?x-pxid=28b508c3-c51f-4109-9b88-29f3f6e4cf2f" alt="Scarf Tracker"/>
</body> </body>
</html> </html>

BIN
X/imapsync_online_status_24h_1200x70.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 17 KiB

BIN
X/imapsync_online_status_2months_1200x70.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 19 KiB

BIN
X/imapsync_online_status_7d_1200x70.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 19 KiB

158
X/sandbox_csv.html
View file

@ -1,57 +1,77 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html lang="en">
<head> <head>
<title>Imapsync CSV online</title> <title>Imapsync CSV online</title>
<meta charset="utf-8"> <meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<!-- --> <!-- -->
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css"> <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
<link rel="stylesheet" href="../S/style.css" type="text/css" />
<link rel="license" href="https://imapsync.lamiral.info/NOLIMIT">
<!-- --> <!-- -->
</head> </head>
<body> <body>
<h1>Imapsync online</h1> <h1>Imapsync CSV online</h1>
<p> <p>
This file is a sandbox to play with, to learn, understand what happens, what can be done etc.<br/> This Imapsync CSV online service is a prototype, a working prototype.<br/>
<a href="#BOTTOM">Bottom of this page</a>
</p> </p>
Some explanations about this service:
<ul>
<li>The input textarea is multi-lines, each line describe a synchronization from the source account to the destination account.</li>
<li>Blank lines are ignored. Blanks lines have only spaces or tabs or nothing but a CR</li>
<li>Lines beginning with # are also ignored, use that for commenting.</li>
<li>A basic line will sync all messages in all folders from the source account to the destination account.</li>
<li>Each line has seven columns, six for the credentials, and one extra to add one or several parameters to the imapsync command.</li>
<li>Each line has seven semi-colons characters ";", each one delimiting a column. The last column can be empty.</li>
<li>Each line is processed serially, one synchronization at a time.</li>
<li>At the end of the log, the lines that failed are listed.</li>
<li>The input textarea is prefilled with values, as an example, replace them with yours or remove them.</li>
<li></li>
<li></li>
<li></li>
</ul>
<form id="form" action="/cgi-bin/imapsync_csv_wrapper" method="post" autocomplete="on"> <form id="form" action="/cgi-bin/imapsync_csv_wrapper" method="post" autocomplete="on">
<textarea name="csv_data" rows="10" cols="120"> <textarea id="csv_data" name="csv_data" rows="10" cols="120">
# This example is a real one, ie, truly working in the real world. # This example is a real one, ie, a truly working in the real world.
test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;secret2; test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;secret2;;
# The reverse # The first again but this time with an authentication failure
test2.lamiral.info;test2;secret2;test1.lamiral.info;test1;secret1; test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;wrong_secret;;
# The first again but with authentication failure # The first again but this time with extra parameters
test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;wrong; test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;secret2;--justfoldersizes --dry --minage 55 ;
</textarea> </textarea>
<br/> <br/>
</form> </form>
<button type="button" <p>
onclick="imapsync( handleRun )" <button id="bt-sync" type="button">
>Run imapsync on all csv data Run imapsync on all csv data!
</button> </button>
<button id="bt-abort" type="button">
Abort the run!
<button type="button"
onclick="abort( )"
>Abort imapsync
</button> </button>
</p>
<p>
<a href="#BOTTOM">Link to the bottom of this page (end of the log)</a>
</p>
<h2>Console of imapsync runs</h2> <h2>Console of imapsync runs</h2>
@ -63,13 +83,11 @@ test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;wrong;
<pre id="abort"> <pre id="abort">
</pre> </pre>
<h2>Log of imapsync runs</h2> <h2>Log of all imapsync runs</h2>
<pre id="output" > <pre id="output" >
</pre> </pre>
<h2>Links</h2> <h2>Links</h2>
<div id="BOTTOM"> <div id="BOTTOM">
@ -78,89 +96,11 @@ test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;wrong;
</p> </p>
</div> </div>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script> <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js">
<script>
var readyStateStr = {
0: "Request not initialized",
1: "Server connection established",
2: "Response headers received",
3: "Processing request",
4: "Finished and response is ready"
} ;
function imapsync( cFunction ) {
var xhr ;
xhr = new XMLHttpRequest( ) ;
var timerRefreshLog = setInterval( function() { refreshLog( xhr ) }, 3000 ) ;
xhr.onreadystatechange = function( ) {
cFunction( this, timerRefreshLog ) ;
} ;
var form_querystring = $("#form").serialize() ;
$("#form_querystring").text( form_querystring ) ;
xhr.open( "POST", "/cgi-bin/imapsync_csv_wrapper", true ) ;
xhr.setRequestHeader( "Content-type",
"application/x-www-form-urlencoded" ) ;
xhr.send( form_querystring ) ;
$("#output").text("Here comes the log!\n\n") ;
xhr.send( ) ;
}
function handleRun( xhr, timerRefreshLog ) {
$( "#console" ).text( "Status: " + xhr.status + " " + xhr.statusText + ".\n"
+ "State: " + readyStateStr[ xhr.readyState ] + "\n" ) ;
if ( xhr.status == 200 && xhr.readyState == 4 ) {
var headers = xhr.getAllResponseHeaders( ) ;
clearInterval( timerRefreshLog ) ;
refreshLog( xhr ) ; // a last time
}
}
function refreshLog( xhr ) {
$( "#output" ).text( xhr.responseText ) ;
}
function abort()
{
var querystring = $("#form").serialize() + "&abort=on";
var xhr;
xhr = new XMLHttpRequest();
xhr.onreadystatechange = function ()
{
handleAbort(xhr);
};
xhr.open("POST", "/cgi-bin/imapsync_csv_wrapper", true);
xhr.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
xhr.send(querystring);
}
function handleAbort( xhr ) {
$( "#abort" ).text( "Status: " + xhr.status + " " + xhr.statusText + ".\n"
+ "State: " + readyStateStr[ xhr.readyState ] + "\n" ) ;
if ( xhr.status == 200 && xhr.readyState == 4 ) {
var headers = xhr.getAllResponseHeaders( ) ;
// $( "#abort" ).append( "\n" + headers + "\n" ) ;
$( "#abort" ).append( xhr.responseText ) ;
}
}
</script> </script>
<script src="sandbox_csv.js">
</script>
</body> </body>
</html> </html>

112
X/sandbox_csv.js Executable file
View file

@ -0,0 +1,112 @@
$(document).ready(
function ()
{
var readyStateStr = {
0: "Request not initialized",
1: "Server connection established",
2: "Response headers received",
3: "Processing request",
4: "Finished and response is ready"
} ;
function imapsync( cFunction ) {
var xhr ;
xhr = new XMLHttpRequest( ) ;
var timerRefreshLog = setInterval( function() { refreshLog( xhr ) }, 6000 ) ;
xhr.onreadystatechange = function( ) {
cFunction( this, timerRefreshLog ) ;
} ;
var form_querystring = $("#form").serialize() ;
$("#form_querystring").text( form_querystring ) ;
xhr.open( "POST", "/cgi-bin/imapsync_csv_wrapper", true ) ;
xhr.setRequestHeader( "Content-type",
"application/x-www-form-urlencoded" ) ;
xhr.send( form_querystring ) ;
$("#output").text("Here comes the log!\n\n") ;
}
function handleRun( xhr, timerRefreshLog ) {
$( "#console" ).text( "Status: " + xhr.status + " " + xhr.statusText + ".\n"
+ "State: " + readyStateStr[ xhr.readyState ] + "\n" ) ;
if ( xhr.status == 200 && xhr.readyState == 4 ) {
var headers = xhr.getAllResponseHeaders( ) ;
clearInterval( timerRefreshLog ) ;
refreshLog( xhr ) ; // a last time
$("#bt-sync").prop("disabled", false) ;
$("#csv_data").prop('readonly', false);
}
}
function refreshLog( xhr ) {
$( "#output" ).text( xhr.responseText ) ;
}
function abort()
{
var querystring = $("#form").serialize() + "&abort=on";
var xhr;
xhr = new XMLHttpRequest();
xhr.onreadystatechange = function ()
{
handleAbort(xhr);
};
xhr.open("POST", "/cgi-bin/imapsync_csv_wrapper", true);
xhr.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
xhr.send(querystring);
}
function handleAbort( xhr ) {
$( "#abort" ).text( "Status: " + xhr.status + " " + xhr.statusText + ".\n"
+ "State: " + readyStateStr[ xhr.readyState ] + "\n" ) ;
if ( xhr.status == 200 && xhr.readyState == 4 ) {
var headers = xhr.getAllResponseHeaders( ) ;
// $( "#abort" ).append( "\n" + headers + "\n" ) ;
$( "#abort" ).append( xhr.responseText ) ;
$("#csv_data").prop('readonly', false);
$( "#bt-sync" ).prop("disabled", false);
$( "#bt-abort" ).prop("disabled", false);
}
}
$("#bt-sync").click(
function ()
{
$("#bt-sync").prop("disabled", true) ;
$("#csv_data").prop('readonly', true);
$("#bt-abort").prop("disabled", false) ;
$("#abort").text("") ;
imapsync( handleRun ) ;
}
);
$("#bt-abort").click(
function ()
{
$("#bt-sync").prop("disabled", true);
$("#csv_data").prop('readonly', true);
$("#bt-abort").prop("disabled", true);
abort();
}
);
// in case of a manual refresh, start with
$("#csv_data").prop('readonly', false);
$("#bt-sync").prop("disabled", false);
$("#bt-abort").prop("disabled", false);
}
) ;

1
X/stat_patterns.txt
View file

@ -36,6 +36,7 @@
^Message rate ^Message rate
^Average bandwidth rate ^Average bandwidth rate
^Biggest message transferred ^Biggest message transferred
^Biggest message:
^Detected.*errors ^Detected.*errors
^Ended by a signal ^Ended by a signal
^Exiting with return value ^Exiting with return value

1
X/vnstati.html
View file

@ -17,6 +17,7 @@ small { font-size: 8px; color: #cbcbcb; }
<body bgcolor="#ffffff"> <body bgcolor="#ffffff">
<img src="vnstat_s.png" border="0" alt="summary"><br> <img src="vnstat_s.png" border="0" alt="summary"><br>
<img src="vnstat_5g.png" border="0" alt="5 minutes graphic"><br>
<img src="vnstat_5.png" border="0" alt="5 minutes"><br> <img src="vnstat_5.png" border="0" alt="5 minutes"><br>
<img src="vnstat_h.png" border="0" alt="hourly"><br> <img src="vnstat_h.png" border="0" alt="hourly"><br>
<img src="vnstat_d.png" border="0" alt="daily"><br> <img src="vnstat_d.png" border="0" alt="daily"><br>

123
doc/TUTORIAL_Unix.html
View file

@ -7,7 +7,7 @@
<CENTER> <CENTER>
<H1>Imapsync tutorial</H1> <H1>Imapsync tutorial</H1>
<FONT SIZE="4"><I>Gilles LAMIRAL gilles@lamiral.info</I></FONT><BR> <FONT SIZE="4"><I>Gilles LAMIRAL gilles@lamiral.info</I></FONT><BR>
<FONT SIZE="4">$Id: TUTORIAL_Unix.t2t,v 1.26 2019/02/10 12:33:17 gilles Exp gilles $ </FONT> <FONT SIZE="4">$Id: TUTORIAL_Unix.t2t,v 1.27 2022/01/14 12:42:51 gilles Exp gilles $ </FONT>
</CENTER> </CENTER>
<P></P> <P></P>
@ -18,14 +18,14 @@
<LI><A HREF="#toc1">1. Good practices overview</A> <LI><A HREF="#toc1">1. Good practices overview</A>
<LI><A HREF="#toc2">2. Basic steps</A> <LI><A HREF="#toc2">2. Basic steps</A>
<UL> <UL>
<LI><A HREF="#toc3">2.1. Verifying imapsync works well on its own</A> <LI><A HREF="#toc3">2.1. Verifying imapsync works well</A>
<LI><A HREF="#toc4">2.2. Working with your data</A> <LI><A HREF="#toc4">2.2. Working with your data</A>
<LI><A HREF="#toc5">2.3. Prepare your credentials</A> <LI><A HREF="#toc5">2.3. Prepare your credentials</A>
<LI><A HREF="#toc6">2.4. Take a real user account as source</A> <LI><A HREF="#toc6">2.4. Take a real user account as a source</A>
<LI><A HREF="#toc7">2.5. Take a test user account as destination</A> <LI><A HREF="#toc7">2.5. Take a test user account as a destination</A>
<LI><A HREF="#toc8">2.6. Edit your own script mysync</A> <LI><A HREF="#toc8">2.6. Edit your own script mysync</A>
</UL> </UL>
<LI><A HREF="#toc9">3. Background knowledge about emailboxes</A> <LI><A HREF="#toc9">3. Background knowledge about mailboxes</A>
<LI><A HREF="#toc10">4. Imapsync presentation</A> <LI><A HREF="#toc10">4. Imapsync presentation</A>
<LI><A HREF="#toc11">5. Conventions</A> <LI><A HREF="#toc11">5. Conventions</A>
<LI><A HREF="#toc12">6. Why start with a test account on destination host2?</A> <LI><A HREF="#toc12">6. Why start with a test account on destination host2?</A>
@ -53,24 +53,24 @@ without much pain. So try imapsync with a real account at the source, aka host1,
and a test account at the destination, aka host2. and a test account at the destination, aka host2.
<P></P> <P></P>
<LI>Next, once familiar and satisfied by the result on the host2 test account, <LI>Next, once familiar and satisfied by the result on the host2 test account,
change to a real user account on host2 or just stop consider it a test one. change to a real user account on host2 or just stop considering it a test one.
</UL> </UL>
<A NAME="toc2"></A> <A NAME="toc2"></A>
<H1>2. Basic steps</H1> <H1>2. Basic steps</H1>
<A NAME="toc3"></A> <A NAME="toc3"></A>
<H2>2.1. Verifying imapsync works well on its own</H2> <H2>2.1. Verifying imapsync works well</H2>
<P> <P>
Open a terminal and go to the imapsync directory. Open a terminal and go to the imapsync directory.
The imapsync directory is the directory created by extraction The imapsync directory is the directory created by the extraction
of the tarball (.tgz), its name is <CODE>imapsync-1.xxx</CODE> where <CODE>1.xxx</CODE> of the tarball (.tgz), its name is <CODE>imapsync-2.xxx</CODE> where <CODE>2.xxx</CODE>
is imapsync release number (1.882 at the time of this writing). is imapsync release number (2.178 at the time of this proofread).
</P> </P>
<PRE> <PRE>
cd imapsync-1.xxx/ cd imapsync-2.xxx/
</PRE> </PRE>
<P> <P>
@ -86,14 +86,14 @@ It should output the help message.
</P> </P>
<P> <P>
If the previous command fails then there is an installation issue. If the previous command fails then there is an installation issue.
Go back to <A HREF="https://imapsync.lamiral.info/#install">https://imapsync.lamiral.info/#install</A> then Go back to the <A HREF="https://imapsync.lamiral.info/#install">https://imapsync.lamiral.info/#install</A> section, then
and read and apply the installation file corresponding to your read and apply the installation file corresponding to your
system and drop me an email about your issue. system and drop me an email about your issue.
</P> </P>
<P> <P>
Next, verify imapsync runs live tests. This check needs an Internet Next, verify imapsync runs well with live tests. This check needs Internet
access. It does a simple sync between two real dedicated access. It does a simple sync between two real dedicated
imap maiboxes located at the host test.lamiral.info imap mailboxes located at the host test.lamiral.info
</P> </P>
<PRE> <PRE>
@ -109,7 +109,7 @@ Now verify that the script examples/imapsync_example.sh runs fine:
</PRE> </PRE>
<P> <P>
This script does the same thing than "<CODE>imapsync --testslive</CODE>" but it This script does the same thing as "<CODE>imapsync --testslive</CODE>" but it
uses explicitly the 6 parameters so it will be a good start uses explicitly the 6 parameters so it will be a good start
for your future own script. for your future own script.
</P> </P>
@ -148,28 +148,28 @@ An IMAP account is accessed with 3 parameters,
</P> </P>
<UL> <UL>
<LI>the imap server <B>host</B>. It's a server name or an ip address <LI>the imap server <B>host</B>. It's a server name or an IP address
<LI>the <B>user</B> name <LI>the <B>user</B> name
<LI>the <B>password</B> <LI>the <B>password</B>
</UL> </UL>
<P> <P>
Since imapsync job is to sync two imap accounts we need 3 + 3 = 6 parameters: Since imapsync job is to sync two imap accounts, we need 3 + 3 = 6 parameters:
</P> </P>
<UL> <UL>
<LI>Three parameters in order to read data from the source account: <B>host1</B>, <B>user1</B> and <B>password1</B> <LI>Three parameters to read data from the source account: <B>host1</B>, <B>user1</B> and <B>password1</B>
<LI>Three parameters in order to copy this data to the destination account: <B>host2</B>, <B>user2</B> and <B>password2</B> <LI>Three parameters to copy this data to the destination account: <B>host2</B>, <B>user2</B> and <B>password2</B>
</UL> </UL>
<A NAME="toc6"></A> <A NAME="toc6"></A>
<H2>2.4. Take a real user account as source</H2> <H2>2.4. Take a real user account as a source</H2>
<P> <P>
Even during the learning time with imapsync, you can take a Even during the learning time with imapsync, you can take a
real user account as a source. There is also no problem if this account is real user account as a source. There is also no problem if this account is
currently used by a user. By default, this account will only be read, no change will currently used by a user. By default, this account will only be read, no change will
be made by imapsync on it. be made by imapsync on it. It's safe to use a normal and live account as a source, even to learn imapsync.
</P> </P>
<P> <P>
Assuming that the imap source server name host1 is <B>origin.example.com</B>, Assuming that the imap source server name host1 is <B>origin.example.com</B>,
@ -184,13 +184,13 @@ we now have the first three parameters.
</UL> </UL>
<A NAME="toc7"></A> <A NAME="toc7"></A>
<H2>2.5. Take a test user account as destination</H2> <H2>2.5. Take a test user account as a destination</H2>
<P> <P>
Unlike the source side, the destination side will be modified by Unlike the source side, the destination side will be modified by
imapsync. Therefore, for learning, checking and adjusting, imapsync. Therefore, for learning, checking, and adjusting,
it is not a very good idea to use a real user imap account it is not a good idea to use a real user imap account
the first times you play with imapsync. the first time you play with imapsync.
</P> </P>
<P> <P>
If you really can't afford a test account on host2, it's ok, If you really can't afford a test account on host2, it's ok,
@ -213,7 +213,7 @@ we now have the next three parameters.
<H2>2.6. Edit your own script mysync</H2> <H2>2.6. Edit your own script mysync</H2>
<P> <P>
Now edit the script <CODE>mysync</CODE> and replace the test values by yours. Now edit the script <CODE>mysync</CODE> and replace the test values with yours.
</P> </P>
<P> <P>
You're ready for a dry test on your accounts. You're ready for a dry test on your accounts.
@ -227,22 +227,21 @@ You're ready for a dry test on your accounts.
Since the <CODE>mysync</CODE> script is a copy of <CODE>examples/imapsync_example.sh</CODE>, Since the <CODE>mysync</CODE> script is a copy of <CODE>examples/imapsync_example.sh</CODE>,
your first run with your data should include three other options your first run with your data should include three other options
<CODE>--automap</CODE> <CODE>--justfolders</CODE> <CODE>--dry</CODE>. <CODE>--automap</CODE> <CODE>--justfolders</CODE> <CODE>--dry</CODE>.
With <CODE>--dry</CODE> option, nothing will really be done on host2 With <CODE>--dry</CODE> option, nothing will be done for real on host2
but yet it will test whether the credentials are ok on both sides. but yet it will test whether the credentials are ok on both sides.
You'll encounter two successful logins, or one failure on host1, You'll encounter two successful logins, or one login failure on host1 or host2, or two failures on host1 and host2.
or a successful login on host1 but a failure on host2. If the logins are ok, you will be able to observe that the folders mapping is ok.
If the login are ok, you will also observe if the folders mapping is ok.
</P> </P>
<P> <P>
If a login fails then double-check all three values that identify If login fails then double-check all three values that identify
the account, which are the host, the login name, and the password. the account, which are the host, the login name, and the password.
</P> </P>
<P> <P>
If the folders mapping proposed is not ok then you can fix it with If the folders mapping proposed is not ok then you can fix it with
option <CODE>--f1f2</CODE>. The following example maps the source folder the option <CODE>--f1f2</CODE>. The following example maps the source folder
"Sent Messages" to the destination folder "Sent". The double-quotes "Sent Messages" to the destination folder "Sent". The double-quotes
are not part of the folders names but they should be used when special are not part of the names of the folders but they should be used when special
characters like blanks are in the folders names: characters like blanks are in the names of the folders:
</P> </P>
<PRE> <PRE>
@ -250,7 +249,7 @@ characters like blanks are in the folders names:
</PRE> </PRE>
<P> <P>
As explained in the inline help or in the README: As explained in the inline help or the README:
</P> </P>
<PRE> <PRE>
@ -267,7 +266,7 @@ folders. Remove <CODE>--dry</CODE> from the <CODE>mysync</CODE> script and rerun
</PRE> </PRE>
<A NAME="toc9"></A> <A NAME="toc9"></A>
<H1>3. Background knowledge about emailboxes</H1> <H1>3. Background knowledge about mailboxes</H1>
<P> <P>
Three Internet protocols are used to access almost all email accounts: Three Internet protocols are used to access almost all email accounts:
@ -289,7 +288,7 @@ they are unavailable from any other system located elsewhere.
<P> <P>
The second protocol to deal with email messages is IMAP, Internet Message Access Protocol. The second protocol to deal with email messages is IMAP, Internet Message Access Protocol.
IMAP gives access to a hierarchy of mailboxes also called folders. Other IMAP features are IMAP gives access to a hierarchy of mailboxes also called folders. Other IMAP features are
concurrent accesses, tagging with flags, search by many criterium like date, subject, size etc. concurrent accesses, tagging with flags, search by many criteriums like date, subject, size, etc.
</P> </P>
<P> <P>
The IMAP protocol presents most of the features POP lacks. The IMAP protocol presents most of the features POP lacks.
@ -304,11 +303,11 @@ HTTP is the protocol to browse the web.
Web browsers like Google Chrome, Mozilla Firefox, Internet Explorer and Safari Web browsers like Google Chrome, Mozilla Firefox, Internet Explorer and Safari
are HTTP client software tools. You already know that so what's the point with HTTP mailboxes? are HTTP client software tools. You already know that so what's the point with HTTP mailboxes?
HTTP mailboxes are called webmails. HTTP mailboxes are called webmails.
Webmails often offer the same features than imap servers Webmails often offer the same features as imap servers
because webmails underlying storage systems are often imap servers. because webmails underlying storage systems are often imap servers.
</P> </P>
<P> <P>
Webmails systems like Gmail, Yahoo, Exchange, Zimbra or Office365 are also accessible via imap. Webmails systems like Gmail, Yahoo, Exchange, Zimbra, or Office365 are also accessible via imap.
</P> </P>
<P> <P>
The conclusion of this protocol review is that mailboxes can be The conclusion of this protocol review is that mailboxes can be
@ -325,12 +324,12 @@ you can use the tool <CODE>pop2imap</CODE> to sync them.
<H1>4. Imapsync presentation</H1> <H1>4. Imapsync presentation</H1>
<P> <P>
Software imapsync is a command line tool to Software imapsync is a command-line tool to
copy, migrate, backup or synchronize IMAP mailboxes. copy, migrate, backup, or synchronize IMAP mailboxes.
</P> </P>
<P> <P>
Command line means imapsync is not graphical, it is textual. Command line means imapsync is not graphical, it is textual.
Usually with command line tools you have to type characters Usually, with command-line tools, you have to type characters
on your keyboard. But your fingers won't suffer much pain on your keyboard. But your fingers won't suffer much pain
typing on the keyboard because script examples are given, typing on the keyboard because script examples are given,
nearly ready to run. Most of the time you only have to change nearly ready to run. Most of the time you only have to change
@ -343,25 +342,25 @@ select/copy/paste complete examples,
and run the little script with a double-click. and run the little script with a double-click.
</P> </P>
<P> <P>
Imapsync runs on Linux, Windows and OS X (Macintosh world). Imapsync runs on Linux, Windows, and OS X (Macintosh/Darwin world).
Imapsync is written in the Perl language and, thanks to the Imapsync is written in the Perl language and, thanks to the
Perl developers, Perl runs mostly everywhere. So does imapsync. Perl developers, Perl runs mostly everywhere. So does imapsync.
</P> </P>
<P> <P>
While operating systems have a lot in common, they sometimes differ, While operating systems have a lot in common, they sometimes differ,
especially within their syntax. I won't blame anyone, historically Windows especially within their syntax. I won't blame anyone, historically Windows
came after Unix. The marvelous designers in this old times came after Unix. The marvelous designers in these old times
decided it would be very cool to not share exactly the same syntax decided it would be very cool to not share the same syntax
for doing the same things. for doing the same things.
</P> </P>
<P> <P>
To designate an end of line, Unix uses the character \n To designate an end of the line, Unix uses the character \n
Windows uses two characters \r\n and Mac use \r. Windows uses two characters \r\n and Mac use \r.
Thanks guys, great great thinking! Thanks to you guys, great thinking!
Fifty years later, we still suffer from this...daily. Fifty years later, we still suffer from this...daily.
</P> </P>
<P> <P>
To avoid you some headaches with systems no one masters To avoid some headaches with systems that no one masters
I will give examples in both worlds, Unix and Windows. I will give examples in both worlds, Unix and Windows.
OS X users are in the Unix world nowadays so they must follow OS X users are in the Unix world nowadays so they must follow
the Unix examples. the Unix examples.
@ -371,7 +370,7 @@ the Unix examples.
<H1>5. Conventions</H1> <H1>5. Conventions</H1>
<P> <P>
In order to simplify display or print, To simplify display or print,
each imapsync command line is usually written in several lines each imapsync command line is usually written in several lines
but it could be written in one single line. but it could be written in one single line.
</P> </P>
@ -379,11 +378,11 @@ but it could be written in one single line.
If you prefer to use the whole command written in one single line then If you prefer to use the whole command written in one single line then
just remove the last visible character of each line ( \ or ^ ) and just remove the last visible character of each line ( \ or ^ ) and
also the carriage return character. also the carriage return character.
The last visible character means "command continues on next line"; The last visible character \ or ^ means "command continues on next line";
it is the backslash \ character on Unix and the caret ^ character on Windows. it's the backslash \ character on Unix and the caret ^ character on Windows.
</P> </P>
<P> <P>
For example, on Unix, a command like the following For example, on Unix, a command like the following:
</P> </P>
<PRE> <PRE>
@ -395,7 +394,7 @@ For example, on Unix, a command like the following
</PRE> </PRE>
<P> <P>
is equivalent to is equivalent to:
</P> </P>
<PRE> <PRE>
@ -403,7 +402,7 @@ is equivalent to
</PRE> </PRE>
<P> <P>
and on Windows and on Windows, a command like the following:
</P> </P>
<PRE> <PRE>
@ -415,7 +414,7 @@ and on Windows
</PRE> </PRE>
<P> <P>
is equivalent to is equivalent to:
</P> </P>
<PRE> <PRE>
@ -435,16 +434,16 @@ messages that couldn't be transferred stay on host1.
</P> </P>
<P> <P>
It's not the same for destination accounts as imapsync writes on host2 accounts. It's not the same for destination accounts as imapsync writes on host2 accounts.
Imapsync creates folders on them, add messages, set flags on messages. Imapsync creates folders on them, adds messages, and sets flags on messages.
It isn't a safe behavior on a real account. So don't use a real user account It isn't safe on a real account. So don't use a real user account
to test imapsync. Learn to use it and see what it does on a test account at host2. to test imapsync. Learn to use it and see what it does on a test account at host2.
</P> </P>
<P> <P>
What can badly happen? The most common bad behavior is What can badly happen? The most common bad behavior is
the folders mapping won't be what you expect because it is strictly that folders mapping won't be what you expect because it is strictly
reproduced from host1 to host2. The second bad behavior is reproduced from host1 to host2. The second bad behavior is getting
duplicates on second run and after, it's rare but it can happen duplicates on the second run and after; it's rare but it can happen
when a imap server software changes headers "<CODE>Message-Id</CODE>" or "<CODE>Received</CODE>". when an imap server software changes the headers "<CODE>Message-Id</CODE>" or "<CODE>Received</CODE>".
Solutions to avoid duplicates are often easy (There's a FAQ called <CODE>FAQ.Duplicates.txt</CODE> about that). Solutions to avoid duplicates are often easy (There's a FAQ called <CODE>FAQ.Duplicates.txt</CODE> about that).
It's also possible to remove the duplicates on host2 but it's better to avoid them on user accounts at first, It's also possible to remove the duplicates on host2 but it's better to avoid them on user accounts at first,
users won't like that you mess up their mailboxes. users won't like that you mess up their mailboxes.

115
doc/TUTORIAL_Unix.t2t
View file

@ -1,6 +1,6 @@
Imapsync tutorial Imapsync tutorial
Gilles LAMIRAL gilles@lamiral.info Gilles LAMIRAL gilles@lamiral.info
$Id: TUTORIAL_Unix.t2t,v 1.26 2019/02/10 12:33:17 gilles Exp gilles $ $Id: TUTORIAL_Unix.t2t,v 1.27 2022/01/14 12:42:51 gilles Exp gilles $
+ Good practices overview + + Good practices overview +
@ -16,20 +16,20 @@ without much pain. So try imapsync with a real account at the source, aka host1,
and a test account at the destination, aka host2. and a test account at the destination, aka host2.
- Next, once familiar and satisfied by the result on the host2 test account, - Next, once familiar and satisfied by the result on the host2 test account,
change to a real user account on host2 or just stop consider it a test one. change to a real user account on host2 or just stop considering it a test one.
+ Basic steps + + Basic steps +
++ Verifying imapsync works well on its own ++ ++ Verifying imapsync works well ++
Open a terminal and go to the imapsync directory. Open a terminal and go to the imapsync directory.
The imapsync directory is the directory created by extraction The imapsync directory is the directory created by the extraction
of the tarball (.tgz), its name is ``imapsync-1.xxx`` where ``1.xxx`` of the tarball (.tgz), its name is ``imapsync-2.xxx`` where ``2.xxx``
is imapsync release number (1.882 at the time of this writing). is imapsync release number (2.178 at the time of this proofread).
``` ```
cd imapsync-1.xxx/ cd imapsync-2.xxx/
``` ```
Verify imapsync runs on your system Verify imapsync runs on your system
@ -39,13 +39,13 @@ Verify imapsync runs on your system
It should output the help message. It should output the help message.
If the previous command fails then there is an installation issue. If the previous command fails then there is an installation issue.
Go back to https://imapsync.lamiral.info/#install then Go back to the https://imapsync.lamiral.info/#install section, then
and read and apply the installation file corresponding to your read and apply the installation file corresponding to your
system and drop me an email about your issue. system and drop me an email about your issue.
Next, verify imapsync runs live tests. This check needs an Internet Next, verify imapsync runs well with live tests. This check needs Internet
access. It does a simple sync between two real dedicated access. It does a simple sync between two real dedicated
imap maiboxes located at the host test.lamiral.info imap mailboxes located at the host test.lamiral.info
``` ```
./imapsync --testslive ./imapsync --testslive
``` ```
@ -54,7 +54,7 @@ Now verify that the script examples/imapsync_example.sh runs fine:
``` ```
sh examples/imapsync_example.sh sh examples/imapsync_example.sh
``` ```
This script does the same thing than "``imapsync --testslive``" but it This script does the same thing as "``imapsync --testslive``" but it
uses explicitly the 6 parameters so it will be a good start uses explicitly the 6 parameters so it will be a good start
for your future own script. for your future own script.
@ -76,22 +76,22 @@ So far so good, now we're going to work with your data.
++ Prepare your credentials ++ ++ Prepare your credentials ++
An IMAP account is accessed with 3 parameters, An IMAP account is accessed with 3 parameters,
- the imap server **host**. It's a server name or an ip address - the imap server **host**. It's a server name or an IP address
- the **user** name - the **user** name
- the **password** - the **password**
Since imapsync job is to sync two imap accounts we need 3 + 3 = 6 parameters: Since imapsync job is to sync two imap accounts, we need 3 + 3 = 6 parameters:
- Three parameters in order to read data from the source account: **host1**, **user1** and **password1** - Three parameters to read data from the source account: **host1**, **user1** and **password1**
- Three parameters in order to copy this data to the destination account: **host2**, **user2** and **password2** - Three parameters to copy this data to the destination account: **host2**, **user2** and **password2**
++ Take a real user account as source ++ ++ Take a real user account as a source ++
Even during the learning time with imapsync, you can take a Even during the learning time with imapsync, you can take a
real user account as a source. There is also no problem if this account is real user account as a source. There is also no problem if this account is
currently used by a user. By default, this account will only be read, no change will currently used by a user. By default, this account will only be read, no change will
be made by imapsync on it. be made by imapsync on it. It's safe to use a normal and live account as a source, even to learn imapsync.
Assuming that the imap source server name host1 is **origin.example.com**, Assuming that the imap source server name host1 is **origin.example.com**,
the user1 account name is **myuser1** and its password is **mysecret1**, the user1 account name is **myuser1** and its password is **mysecret1**,
@ -102,12 +102,12 @@ we now have the first three parameters.
- --password1 **mysecret1** - --password1 **mysecret1**
++ Take a test user account as destination ++ ++ Take a test user account as a destination ++
Unlike the source side, the destination side will be modified by Unlike the source side, the destination side will be modified by
imapsync. Therefore, for learning, checking and adjusting, imapsync. Therefore, for learning, checking, and adjusting,
it is not a very good idea to use a real user imap account it is not a good idea to use a real user imap account
the first times you play with imapsync. the first time you play with imapsync.
If you really can't afford a test account on host2, it's ok, If you really can't afford a test account on host2, it's ok,
imapsync is not that bad but you may have some work to do to fix some unwanted behavior. imapsync is not that bad but you may have some work to do to fix some unwanted behavior.
@ -124,7 +124,7 @@ we now have the next three parameters.
++ Edit your own script mysync ++ ++ Edit your own script mysync ++
Now edit the script ``mysync`` and replace the test values by yours. Now edit the script ``mysync`` and replace the test values with yours.
You're ready for a dry test on your accounts. You're ready for a dry test on your accounts.
@ -135,27 +135,26 @@ sh mysync
Since the ``mysync`` script is a copy of ``examples/imapsync_example.sh``, Since the ``mysync`` script is a copy of ``examples/imapsync_example.sh``,
your first run with your data should include three other options your first run with your data should include three other options
``--automap`` ``--justfolders`` ``--dry``. ``--automap`` ``--justfolders`` ``--dry``.
With ``--dry`` option, nothing will really be done on host2 With ``--dry`` option, nothing will be done for real on host2
but yet it will test whether the credentials are ok on both sides. but yet it will test whether the credentials are ok on both sides.
You'll encounter two successful logins, or one failure on host1, You'll encounter two successful logins, or one login failure on host1 or host2, or two failures on host1 and host2.
or a successful login on host1 but a failure on host2. If the logins are ok, you will be able to observe that the folders mapping is ok.
If the login are ok, you will also observe if the folders mapping is ok.
If a login fails then double-check all three values that identify If login fails then double-check all three values that identify
the account, which are the host, the login name, and the password. the account, which are the host, the login name, and the password.
If the folders mapping proposed is not ok then you can fix it with If the folders mapping proposed is not ok then you can fix it with
option ``--f1f2``. The following example maps the source folder the option ``--f1f2``. The following example maps the source folder
"Sent Messages" to the destination folder "Sent". The double-quotes "Sent Messages" to the destination folder "Sent". The double-quotes
are not part of the folders names but they should be used when special are not part of the names of the folders but they should be used when special
characters like blanks are in the folders names: characters like blanks are in the names of the folders:
``` ```
./imapsync ... --f1f2 "Sent Messages=Sent" ./imapsync ... --f1f2 "Sent Messages=Sent"
``` ```
As explained in the inline help or in the README: As explained in the inline help or the README:
``` ```
--f1f2 str1=str2 : Force folder str1 to be synced to str2. --f1f2 str1=str2 : Force folder str1 to be synced to str2.
``` ```
@ -169,7 +168,7 @@ sh mysync
+ Background knowledge about emailboxes + + Background knowledge about mailboxes +
Three Internet protocols are used to access almost all email accounts: Three Internet protocols are used to access almost all email accounts:
POP3, IMAP, HTTP. POP3, IMAP, HTTP.
@ -187,7 +186,7 @@ they are unavailable from any other system located elsewhere.
The second protocol to deal with email messages is IMAP, Internet Message Access Protocol. The second protocol to deal with email messages is IMAP, Internet Message Access Protocol.
IMAP gives access to a hierarchy of mailboxes also called folders. Other IMAP features are IMAP gives access to a hierarchy of mailboxes also called folders. Other IMAP features are
concurrent accesses, tagging with flags, search by many criterium like date, subject, size etc. concurrent accesses, tagging with flags, search by many criteriums like date, subject, size, etc.
The IMAP protocol presents most of the features POP lacks. The IMAP protocol presents most of the features POP lacks.
Messages stay on the imap server so any client on the network can access them Messages stay on the imap server so any client on the network can access them
@ -199,10 +198,10 @@ HTTP is the protocol to browse the web.
Web browsers like Google Chrome, Mozilla Firefox, Internet Explorer and Safari Web browsers like Google Chrome, Mozilla Firefox, Internet Explorer and Safari
are HTTP client software tools. You already know that so what's the point with HTTP mailboxes? are HTTP client software tools. You already know that so what's the point with HTTP mailboxes?
HTTP mailboxes are called webmails. HTTP mailboxes are called webmails.
Webmails often offer the same features than imap servers Webmails often offer the same features as imap servers
because webmails underlying storage systems are often imap servers. because webmails underlying storage systems are often imap servers.
Webmails systems like Gmail, Yahoo, Exchange, Zimbra or Office365 are also accessible via imap. Webmails systems like Gmail, Yahoo, Exchange, Zimbra, or Office365 are also accessible via imap.
The conclusion of this protocol review is that mailboxes can be The conclusion of this protocol review is that mailboxes can be
accessed using the IMAP protocol, most of the time. accessed using the IMAP protocol, most of the time.
@ -215,11 +214,11 @@ you can use the tool ``pop2imap`` to sync them.
+ Imapsync presentation + + Imapsync presentation +
Software imapsync is a command line tool to Software imapsync is a command-line tool to
copy, migrate, backup or synchronize IMAP mailboxes. copy, migrate, backup, or synchronize IMAP mailboxes.
Command line means imapsync is not graphical, it is textual. Command line means imapsync is not graphical, it is textual.
Usually with command line tools you have to type characters Usually, with command-line tools, you have to type characters
on your keyboard. But your fingers won't suffer much pain on your keyboard. But your fingers won't suffer much pain
typing on the keyboard because script examples are given, typing on the keyboard because script examples are given,
nearly ready to run. Most of the time you only have to change nearly ready to run. Most of the time you only have to change
@ -230,22 +229,22 @@ You can still use the mouse to launch an editor,
select/copy/paste complete examples, select/copy/paste complete examples,
and run the little script with a double-click. and run the little script with a double-click.
Imapsync runs on Linux, Windows and OS X (Macintosh world). Imapsync runs on Linux, Windows, and OS X (Macintosh/Darwin world).
Imapsync is written in the Perl language and, thanks to the Imapsync is written in the Perl language and, thanks to the
Perl developers, Perl runs mostly everywhere. So does imapsync. Perl developers, Perl runs mostly everywhere. So does imapsync.
While operating systems have a lot in common, they sometimes differ, While operating systems have a lot in common, they sometimes differ,
especially within their syntax. I won't blame anyone, historically Windows especially within their syntax. I won't blame anyone, historically Windows
came after Unix. The marvelous designers in this old times came after Unix. The marvelous designers in these old times
decided it would be very cool to not share exactly the same syntax decided it would be very cool to not share the same syntax
for doing the same things. for doing the same things.
To designate an end of line, Unix uses the character \n To designate an end of the line, Unix uses the character \n
Windows uses two characters \r\n and Mac use \r. Windows uses two characters \r\n and Mac use \r.
Thanks guys, great great thinking! Thanks to you guys, great thinking!
Fifty years later, we still suffer from this...daily. Fifty years later, we still suffer from this...daily.
To avoid you some headaches with systems no one masters To avoid some headaches with systems that no one masters
I will give examples in both worlds, Unix and Windows. I will give examples in both worlds, Unix and Windows.
OS X users are in the Unix world nowadays so they must follow OS X users are in the Unix world nowadays so they must follow
the Unix examples. the Unix examples.
@ -253,17 +252,17 @@ the Unix examples.
+ Conventions + + Conventions +
In order to simplify display or print, To simplify display or print,
each imapsync command line is usually written in several lines each imapsync command line is usually written in several lines
but it could be written in one single line. but it could be written in one single line.
If you prefer to use the whole command written in one single line then If you prefer to use the whole command written in one single line then
just remove the last visible character of each line ( \ or ^ ) and just remove the last visible character of each line ( \ or ^ ) and
also the carriage return character. also the carriage return character.
The last visible character means "command continues on next line"; The last visible character \ or ^ means "command continues on next line";
it is the backslash \ character on Unix and the caret ^ character on Windows. it's the backslash \ character on Unix and the caret ^ character on Windows.
For example, on Unix, a command like the following For example, on Unix, a command like the following:
``` ```
imapsync \ imapsync \
@ -272,12 +271,12 @@ For example, on Unix, a command like the following
--password1 secret1 \ --password1 secret1 \
... ...
``` ```
is equivalent to is equivalent to:
``` ```
imapsync --host1 test.lamiral.info --user1 test1 --password1 secret1 ... imapsync --host1 test.lamiral.info --user1 test1 --password1 secret1 ...
``` ```
and on Windows and on Windows, a command like the following:
``` ```
imapsync.exe ^ imapsync.exe ^
--host1 test.lamiral.info ^ --host1 test.lamiral.info ^
@ -285,7 +284,7 @@ and on Windows
--password1 secret1 ^ --password1 secret1 ^
... ...
``` ```
is equivalent to is equivalent to:
``` ```
imapsync --host1 test.lamiral.info --user1 test1 --password1 secret1 ... imapsync --host1 test.lamiral.info --user1 test1 --password1 secret1 ...
``` ```
@ -301,15 +300,15 @@ messages that couldn't be transferred stay on host1.
It's not the same for destination accounts as imapsync writes on host2 accounts. It's not the same for destination accounts as imapsync writes on host2 accounts.
Imapsync creates folders on them, add messages, set flags on messages. Imapsync creates folders on them, adds messages, and sets flags on messages.
It isn't a safe behavior on a real account. So don't use a real user account It isn't safe on a real account. So don't use a real user account
to test imapsync. Learn to use it and see what it does on a test account at host2. to test imapsync. Learn to use it and see what it does on a test account at host2.
What can badly happen? The most common bad behavior is What can badly happen? The most common bad behavior is
the folders mapping won't be what you expect because it is strictly that folders mapping won't be what you expect because it is strictly
reproduced from host1 to host2. The second bad behavior is reproduced from host1 to host2. The second bad behavior is getting
duplicates on second run and after, it's rare but it can happen duplicates on the second run and after; it's rare but it can happen
when a imap server software changes headers "``Message-Id``" or "``Received``". when an imap server software changes the headers "``Message-Id``" or "``Received``".
Solutions to avoid duplicates are often easy (There's a FAQ called ``FAQ.Duplicates.txt`` about that). Solutions to avoid duplicates are often easy (There's a FAQ called ``FAQ.Duplicates.txt`` about that).
It's also possible to remove the duplicates on host2 but it's better to avoid them on user accounts at first, It's also possible to remove the duplicates on host2 but it's better to avoid them on user accounts at first,
users won't like that you mess up their mailboxes. users won't like that you mess up their mailboxes.

27
examples/file.txt
View file

@ -1,35 +1,36 @@
# Example file.txt for imapsync massive migration. # Example file.txt for imapsync massive migration.
# #
# $Id: file.txt,v 1.15 2020/03/10 19:49:52 gilles Exp gilles $ # $Id: file.txt,v 1.17 2022/01/09 09:49:25 gilles Exp gilles $
# #
# Each line contains at least 6 columns, columns are parameters for # Each line contains at least 6 columns
# The separator is the character semi-colon ";"
# The columns are the values for the parameters
# --host1 --user1 --password1 --host2 --user2 --password2 # --host1 --user1 --password1 --host2 --user2 --password2
# #
# Extra columns can be used to pass extra parameters but the script reading # A extra column can be used to pass some extra parameters but the script reading
# this file have to read them into some variables. # this file have to be able to get them.
# #
# Last, don't forget the last semicolon. # Don't forget the last semicolon.
# #
# Windows: see the script examples/sync_loop_windows.bat # Windows: see the script examples/sync_loop_windows.bat
# Unix: see the script examples/sync_loop_unix.sh # Unix: see the script examples/sync_loop_unix.sh
# Power users: see the script examples/sync_parallel_unix.sh # Power users: see the script examples/sync_parallel_unix.sh
# Lines starting with a # are usually comments and ignored # Lines starting with a # are usually comments and ignored
# Blank lines are ignored as well # Blank lines are ignored as well
# Now the data example # Now the data example
host001_1;user001_1;password001_1;host001_2;user001_2;password001_2; host001_1;user001_1;password001_1;host001_2;user001_2;password001_2;;
host002_1;user002_1;password002_1;host002_2;user002_2;password002_2; host002_1;user002_1;password002_1;host002_2;user002_2;password002_2;--subfolder2 foo;
host003_1;user003_1;password003_1;host003_2;user003_2;password003_2; host003_1;user003_1;password003_1;host003_2;user003_2;password003_2;--justfolders --automap --subfolder2 foo;
# Another comment blabla # Another comment blabla
host004_1;user004_1;password004_1;host004_2;user004_2;password004_2; host004_1;user004_1;password004_1;host004_2;user004_2;password004_2;;
# This last example is a real one, ie, truly working in the real world. # This last example is a real one, ie, truly working in the real world.
test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;secret2; test1.lamiral.info;test1;secret1;test2.lamiral.info;test2;secret2;;
# The reverse # The reverse with some extra parameters
test2.lamiral.info;test2;secret2;test1.lamiral.info;test1;secret1; test2.lamiral.info;test2;secret2;test1.lamiral.info;test1;secret1;--dry --subfolder2 foo;

65
examples/sync_loop_darwin.sh
View file

@ -1,41 +1,64 @@
#!/bin/sh #!/bin/sh
# #
# $Id: sync_loop_darwin.sh,v 1.2 2018/02/12 21:54:59 gilles Exp gilles $ # $Id: sync_loop_darwin.sh,v 1.3 2022/01/10 08:03:46 gilles Exp gilles $
# Example for imapsync massive migration on Unix systems. # Example for imapsync massive migration on Unix systems.
# See also http://imapsync.lamiral.info/FAQ.d/FAQ.Massive.txt # See also http://imapsync.lamiral.info/FAQ.d/FAQ.Massive.txt
# #
# Data is supposed to be in file.txt in the following format: # Data is supposed to be in file.txt in the following format:
# host001_1;user001_1;password001_1;host001_2;user001_2;password001_2; # host001_1;user001_1;password001_1;host001_2;user001_2;password001_2;;
# ... # ...
# Separator is character semi-colon ";" it can be changed by any character changing IFS=';' # The separator is the character semi-colon ";"
# in the while loop below. # this separator character can be changed to any character
# # Each line contains 6 columns, columns are parameter values for # by changing IFS=';' in the while loop below.
# --host1 --user1 --password1 --host2 --user2 --password2
# and a trailing empty fake column to avaid CR LF part going
# in the 6th parameter password2. Don't forget the last semicolon.
# #
# You can add extra options after the variable "$@" # Each line contains 7 columns. These columns are the 6 parameter values
# Use character backslash \ at the end of each suplementary line, except for the last one. # for the imapsync command options
# --host1 --user1 --password1 --host2 --user2 --password2
# plus an extra column for extra parameters and a trailing fake column
# to avoid CR LF part going in the 7th parameter extra.
# So don't forget the last semicolon, especially on MacOS systems.
#
# You can also add extra options in this script after the variable "$@"
# Those options will be applied in every imapsync run, for every line.
# The imapsync command below is written in two lines to avoid a long line.
# The character backslash \ at the end of the first line is means
# "the command continues on the next line".
#
# Use character backslash \ at the end of each supplementary line,
# except for the last line.
#
# You can also pass extra options via the parameters of this script since # You can also pass extra options via the parameters of this script since
# they will be in "$@" # they will be in "$@". Shell knowledge is your friend.
# The credentials filename "file.txt" used for the loop can be renamed # The credentials filename "file.txt" used for the loop can be renamed
# by changing "file.txt" below. # by changing "file.txt" below.
# The file file_failures.txt will contain the lines from file.txt that ended
# up in error, for whatever reason. It's there to notice and replay easily
# the failed imapsync runs. Is is emptied at the beginning of the loop run.
# I let you junggle with it.
echo Looping on account credentials found in file.txt echo Looping on accounts credentials found in file.txt
echo echo
line_counter=0
{ while IFS=';' read h1 u1 p1 h2 u2 p2 fake # Empty the error listing
> file_failures.txt
{ while IFS=';' read h1 u1 p1 h2 u2 p2 extra fake
do do
{ echo "$h1" | egrep '^#|^ *$' ; } > /dev/null && continue # this skip commented lines in file.txt line_counter=`expr 1 + $line_counter`
echo "==== Starting imapsync from host1 $h1 user1 $u1 to host2 $h2 user2 $u2 ====" { echo "$h1" | tr -d '\r' | egrep '^#|^ *$' ; } > /dev/null && continue # this skip commented lines in file.txt
../imapsync_bin_Darwin --host1 "$h1" --user1 "$u1" --password1 "$p1" \ echo "==== Starting imapsync with --host1 $h1 --user1 $u1 --host2 $h2 --user2 $u2 $extra $@ ===="
--host2 "$h2" --user2 "$u2" --password2 "$p2" \ if ../imapsync_bin_Darwin --host1 "$h1" --user1 "$u1" --password1 "$p1" \
"$@" --host2 "$h2" --user2 "$u2" --password2 "$p2" $extra "$@"
echo "==== Ended imapsync from host1 $h1 user1 $u1 to host2 $h2 user2 $u2 ====" then
echo "success sync for line $line_counter "
else
echo "$h1;$u1;$p1;$h2;$u2;$p2;$extra;" | tee -a file_failures.txt
fi
echo "==== Ended imapsync with --host1 $h1 --user1 $u1 --host2 $h2 --user2 $u2 $extra $@ ===="
echo echo
done done
} < file.txt } < file.txt

54
examples/sync_loop_unix.sh
View file

@ -1,48 +1,64 @@
#!/bin/sh #!/bin/sh
# #
# $Id: sync_loop_unix.sh,v 1.10 2020/12/11 18:09:11 gilles Exp gilles $ # $Id: sync_loop_unix.sh,v 1.13 2022/01/09 09:53:47 gilles Exp gilles $
# Example for imapsync massive migration on Unix systems. # Example for imapsync massive migration on Unix systems.
# See also http://imapsync.lamiral.info/FAQ.d/FAQ.Massive.txt # See also http://imapsync.lamiral.info/FAQ.d/FAQ.Massive.txt
# #
# Data is supposed to be in file.txt in the following format: # Data is supposed to be in file.txt in the following format:
# host001_1;user001_1;password001_1;host001_2;user001_2;password001_2; # host001_1;user001_1;password001_1;host001_2;user001_2;password001_2;;
# ... # ...
# Separator is character semi-colon ";" it can be changed by any character changing IFS=';' # The separator is the character semi-colon ";"
# in the while loop below. # this separator character can be changed to any character
# # Each line contains 6 columns, columns are parameter values for # by changing IFS=';' in the while loop below.
# --host1 --user1 --password1 --host2 --user2 --password2
# and a trailing empty fake column to avoid CR LF part going
# in the 6th parameter password2. Don't forget the last semicolon.
# #
# You can add extra options after the variable "$@" # Each line contains 7 columns. These columns are the 6 parameter values
# Use character backslash \ at the end of each supplementary line, except for the last one. # for the imapsync command options
# --host1 --user1 --password1 --host2 --user2 --password2
# plus an extra column for extra parameters and a trailing fake column
# to avoid CR LF part going in the 7th parameter extra.
# So don't forget the last semicolon, especially on MacOS systems.
#
# You can also add extra options in this script after the variable "$@"
# Those options will be applied in every imapsync run, for every line.
# The imapsync command below is written in two lines to avoid a long line.
# The character backslash \ at the end of the first line is means
# "the command continues on the next line".
#
# Use character backslash \ at the end of each supplementary line,
# except for the last line.
#
# You can also pass extra options via the parameters of this script since # You can also pass extra options via the parameters of this script since
# they will be in "$@" # they will be in "$@". Shell knowledge is your friend.
# The credentials filename "file.txt" used for the loop can be renamed # The credentials filename "file.txt" used for the loop can be renamed
# by changing "file.txt" below. # by changing "file.txt" below.
# The file file_failures.txt will contain the lines from file.txt that ended
# up in error, for whatever reason. It's there to notice and replay easily
# the failed imapsync runs. Is is emptied at the beginning of the loop run.
# I let you junggle with it.
echo Looping on account credentials found in file.txt echo Looping on accounts credentials found in file.txt
echo echo
line_counter=0 line_counter=0
# Empty the error listing
> file_failures.txt > file_failures.txt
{ while IFS=';' read h1 u1 p1 h2 u2 p2 fake { while IFS=';' read h1 u1 p1 h2 u2 p2 extra fake
do do
line_counter=`expr 1 + $line_counter` line_counter=`expr 1 + $line_counter`
{ echo "$h1" | tr -d '\r' | egrep '^#|^ *$' ; } > /dev/null && continue # this skip commented lines in file.txt { echo "$h1" | tr -d '\r' | egrep '^#|^ *$' ; } > /dev/null && continue # this skip commented lines in file.txt
echo "==== Starting imapsync from host1 $h1 user1 $u1 to host2 $h2 user2 $u2 ====" echo "==== Starting imapsync with --host1 $h1 --user1 $u1 --host2 $h2 --user2 $u2 $extra $@ ===="
if imapsync --host1 "$h1" --user1 "$u1" --password1 "$p1" \ if imapsync --host1 "$h1" --user1 "$u1" --password1 "$p1" \
--host2 "$h2" --user2 "$u2" --password2 "$p2" \ --host2 "$h2" --user2 "$u2" --password2 "$p2" $extra "$@"
"$@"
then then
echo "success sync for line $line_counter " echo "success sync for line $line_counter "
else else
echo "$h1;$u1;$p1;$h2;$u2;$p2;" | tee -a file_failures.txt echo "$h1;$u1;$p1;$h2;$u2;$p2;$extra;" | tee -a file_failures.txt
fi fi
echo "==== Ended imapsync from host1 $h1 user1 $u1 to host2 $h2 user2 $u2 ====" echo "==== Ended imapsync with --host1 $h1 --user1 $u1 --host2 $h2 --user2 $u2 $extra $@ ===="
echo echo
done done
} < file.txt } < file.txt

6
examples/sync_loop_windows.bat
View file

@ -1,5 +1,5 @@
@REM @REM
@REM $Id: sync_loop_windows.bat,v 1.18 2018/05/24 11:45:42 gilles Exp gilles $ @REM $Id: sync_loop_windows.bat,v 1.19 2022/01/09 09:52:44 gilles Exp gilles $
@REM @REM
@REM imapsync massive sync example batch for Windows users @REM imapsync massive sync example batch for Windows users
@REM lines beginning with @REM are just comments @REM lines beginning with @REM are just comments
@ -85,11 +85,11 @@ SET csvfile=file.txt
@REM Blank lines are usually ignored. Dumping the tokens in [] in case debugging is needed @REM Blank lines are usually ignored. Dumping the tokens in [] in case debugging is needed
@ECHO GOT those values from %csvfile% presented inside brackets: [%%G] [%%H] [%%I] [%%J] [%%K] [%%L] [%%M] @ECHO GOT those values from %csvfile% presented inside brackets: [%%G] [%%H] [%%I] [%%J] [%%K] [%%L] [%%M]
@REM You can add extra arguments to imapsync after the variable named %arguments% @REM You can add extra arguments to imapsync after the variable named %arguments%
@ECHO ==== Starting imapsync from --host1 %%G --user1 %%H to --host2 %%J --user2 %%K ==== @ECHO ==== Starting imapsync with --host1 %%G --user1 %%H --host2 %%J --user2 %%K %%M %arguments% ====
@imapsync ^ @imapsync ^
--host1 %%G --user1 %%H --password1 %%I ^ --host1 %%G --user1 %%H --password1 %%I ^
--host2 %%J --user2 %%K --password2 %%L %%M %arguments% --host2 %%J --user2 %%K --password2 %%L %%M %arguments%
@ECHO ==== Ended imapsync from --host1 %%G --user1 %%H to --host2 %%J --user2 %%K ==== @ECHO ==== Ended imapsync with --host1 %%G --user1 %%H --host2 %%J --user2 %%K %%M %arguments% ====
@ECHO. @ECHO.
) )

17
examples/sync_parallel_unix.sh
View file

@ -1,6 +1,6 @@
#!/bin/sh #!/bin/sh
# #
# $Id: sync_parallel_unix.sh,v 1.9 2021/02/19 13:41:51 gilles Exp gilles $ # $Id: sync_parallel_unix.sh,v 1.11 2022/01/13 12:56:28 gilles Exp gilles $
# If you're on Windows there is a possibility to install and use parallel # If you're on Windows there is a possibility to install and use parallel
# but I have never tested it. I found: # but I have never tested it. I found:
@ -44,10 +44,10 @@
# The parallel command is then followed by its parameters. # The parallel command is then followed by its parameters.
# parallel parameters explained: # parallel parameters explained:
# #
# --max-procs 7 means parallel will parallelize up to 7 jobs at a time, # --max-procs 3 means parallel will parallelize up to 3 jobs at a time,
# adjust this value by monitoring your system capacity. # adjust this value by monitoring your system capacity.
# #
# --delay 1.1 means parallel will pause 1.1 seconds after starting each job. # --delay 1.4 means parallel will pause 1.4 seconds (1400 ms) after starting each job.
# #
# --colsep ';' means the separator between values is the character semi-colon ; # --colsep ';' means the separator between values is the character semi-colon ;
# #
@ -96,6 +96,8 @@
# paralelized runs # paralelized runs
# The current script does not take into account what is in the 7th column
check_parallel_is_here() { check_parallel_is_here() {
parallel --version > /dev/null || { echo "parallel command is not installed. Install it first."; return 1; } parallel --version > /dev/null || { echo "parallel command is not installed. Install it first."; return 1; }
} }
@ -114,9 +116,14 @@ DRYRUN=echo
# since the previous echo value will be discarded # since the previous echo value will be discarded
DRYRUN= DRYRUN=
parallel --max-procs 7 --delay 1.1 --colsep ';' --arg-file file.txt --line-buffer --tagstring "from {2} to {5} : " \ parallel --max-procs 3 --delay 1.4 --colsep ';' --arg-file file.txt --line-buffer --tagstring "from {2} to {5} : " \
'echo {1} | egrep "^#|^ *$" > /dev/null ||' \ 'echo {1} | egrep "^#|^ *$" > /dev/null ||' \
$DRYRUN imapsync --host1 {1} --user1 {2} --password1 {3} \ $DRYRUN imapsync --host1 {1} --user1 {2} --password1 {3} \
--host2 {4} --user2 {5} --password2 {6} "$@" --simulong 5 --host2 {4} --user2 {5} --password2 {6} "$@" --simulong 5
# A question to ask to the parallel mailing-list, Ole Tange
# does not work like I want, it passes all the 7th column as only one argument to imapsync:
# '{=7 split / /, $arg[7] =}'
# I want a list of arguments

1440
imapsync
View file

File diff suppressed because it is too large Load diff

38
index.shtml
View file

@ -23,6 +23,8 @@
<!-- <!--
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" /> <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" />
--> -->
<link href="S/quiz/quiz_imapsync.css" rel="stylesheet" />
<script src="S/quiz/quiz_imapsync.js" type="text/javascript"></script>
</head> </head>
@ -68,10 +70,12 @@
<img width="486" height="309" alt="imapsync logo" src="S/images/logo_imapsync.png" /> <img width="486" height="309" alt="imapsync logo" src="S/images/logo_imapsync.png" />
</a> </a>
<h1>Afraid of losing all your precious emails?<br/> <h1>The Mailbox Changer
</h1>
<h2>Afraid of losing all your precious emails?<br/>
It's time to copy all of them elsewhere!<br/> It's time to copy all of them elsewhere!<br/>
Safely. Safely.
</h1> </h2>
<!-- I use SSI here only to pass a strict xhtml validation that facebook/twitter/coinbase can't pass --> <!-- I use SSI here only to pass a strict xhtml validation that facebook/twitter/coinbase can't pass -->
<!-- remove space before #include to activate these ssi --> <!-- remove space before #include to activate these ssi -->
@ -83,9 +87,11 @@ Safely.
Site last updated on <b><!--#echo var="LAST_MODIFIED" --></b><br/> Site last updated on <b><!--#echo var="LAST_MODIFIED" --></b><br/>
<a href="FAQ.d/FAQ.Old_Style_Web_Design.txt">Why this website looks so old fashion?</a> <a href="FAQ.d/FAQ.Old_Style_Web_Design.txt">Why this website looks so old fashion?</a>
</div> </div>
<h2>What is imapsync? <a href="#TOP" id="WHATIS"><small>(back to menu)</small></a> <h2>What is imapsync? <a href="#TOP" id="WHATIS"><small>(back to menu)</small></a>
</h2> </h2>
@ -176,6 +182,9 @@ See detailed explanation and motivation here when
</p> </p>
--> -->
<h2>Imapsync Quiz
</h2>
<div id="quizWrap"></div>
<h2>Contact the author, Gilles LAMIRAL <a href="#TOP" id="AUTHOR"><small>(back to menu)</small></a> <h2>Contact the author, Gilles LAMIRAL <a href="#TOP" id="AUTHOR"><small>(back to menu)</small></a>
@ -272,7 +281,8 @@ problems, you're sure to get my attention and my time.
<ul> <ul>
<li>Imapsync <b>full professional support</b> (that costs also &euro;120 EUR by itself, see why below).</li> <li>Imapsync <b>full professional support</b> (that costs also &euro;120 EUR by itself, see why below).</li>
<li>Access without any limit to the imapsync online service. <li>Access without any limit to the imapsync <b>online service</b>, like <b><a href="https://imapsync.lamiral.info/X/">/X</a></b>
but <b>without</b> the 3 GB limit.
</li> </li>
<li>Standalone <b>imapsync.exe</b> for win32, easy installation done by a zip extraction anywhere. <li>Standalone <b>imapsync.exe</b> for win32, easy installation done by a zip extraction anywhere.
@ -282,8 +292,8 @@ problems, you're sure to get my attention and my time.
<li>Imapsync Perl <b>source code</b> for any operating system, Unix, Windows, OS X.</li> <li>Imapsync Perl <b>source code</b> for any operating system, Unix, Windows, OS X.</li>
<li>The <b>online visual interface</b>, like <b><a href="https://imapsync.lamiral.info/X/">/X</a></b>, <li>The <b>visual interface</b>, similar to <b><a href="https://imapsync.lamiral.info/X/">/X</a></b>,
to install the service on a <b>Linux server</b> (not working on Windows yet).</li> to install the service on your own <b>Linux server</b> (not working on Windows yet).</li>
<li><b>Lifetime</b> of imapsync <b>updates</b> without extra payment.</li> <li><b>Lifetime</b> of imapsync <b>updates</b> without extra payment.</li>
@ -298,7 +308,7 @@ problems, you're sure to get my attention and my time.
<p> <p>
For <b>&euro;120 EUR</b> independently, you get imapsync <b>full professional support</b>, For <b>&euro;120 EUR</b> independently, you get imapsync <b>full professional support</b>,
provided by the imapsync designer and developer, Gilles LAMIRAL, your servitor, who has provided by the imapsync designer and developer, Gilles LAMIRAL, your servitor, who has
been supporting imap migrations with imapsync for more than 19 years (I started in 2001). been supporting imap migrations with imapsync for more than 20 years (I started in 2001).
See a detailed <a href="#SUPPORT"><b>support description</b></a> below. See a detailed <a href="#SUPPORT"><b>support description</b></a> below.
</p> </p>
@ -399,7 +409,7 @@ how to deal with special imap software servers (Gmail, Office365, Exchange etc.)
choosing and explaining options, explaining error messages, choosing and explaining options, explaining error messages,
solving authentication issues, solving authentication issues,
dealing with speed, quotas, special context, dealing with speed, quotas, special context,
running in parallel, etc., so in summary: <b>anything!</b> running in parallel, etc., so in summary: <b>anything!</b> except doing the job for you.
</p> </p>
<p> <p>
@ -426,8 +436,9 @@ that is 120 EUR.
</h3> </h3>
<p> <p>
The funny thing about crypto payments is that I receive very few of them, one or two per year, and in small amount, one or two dollars equivalent... The funny thing about crypto payments is that I receive very few of them, one or two per year,
Maybe one day those little crypto coins will be like gold? (I would like that!) and in small amount, one or two dollars equivalent...
Maybe one day those little crypto coins will be like gold? (I will like that!)
</p> </p>
<!-- <!--
@ -799,9 +810,11 @@ has a <b>network access to both IMAP servers</b>, the source and the destination
<p>What to do next?<br/> <p>What to do next?<br/>
Depending on the system where you will run imapsync:</p> Depending on the system where you will run imapsync:</p>
<ul> <ul>
<li>Windows users, read directly <a href="README_Windows.txt">README_Windows.txt</a>.</li> <li>Windows users, read directly <a href="README_Windows.txt">README_Windows.txt</a>.</li>
<li>OS X users, read <a href="INSTALL.d/INSTALL.Darwin.txt">INSTALL.Darwin.txt</a>.</li> <li>OS X users, read <a href="INSTALL.d/INSTALL.Darwin.txt">INSTALL.Darwin.txt</a>. <a href="https://www.macstadium.com/"><img src="S/images/5c019d917bba312af7553b49_MacStadium-developerlogo.png" alt="MacStadium logo" width="150" height="61" /></a></li>
<li>Centos users, read <a href="INSTALL.d/INSTALL.Centos.txt">INSTALL.Centos.txt</a>.</li> <li>Centos users, read <a href="INSTALL.d/INSTALL.Centos.txt">INSTALL.Centos.txt</a>.</li>
<li>CPanel users, read <a href="INSTALL.d/INSTALL.CPanel.txt">INSTALL.CPanel.txt</a>.</li> <li>CPanel users, read <a href="INSTALL.d/INSTALL.CPanel.txt">INSTALL.CPanel.txt</a>.</li>
<li>Debian users, read <a href="INSTALL.d/INSTALL.Debian.txt">INSTALL.Debian.txt</a>.</li> <li>Debian users, read <a href="INSTALL.d/INSTALL.Debian.txt">INSTALL.Debian.txt</a>.</li>
@ -906,7 +919,10 @@ you some struggling hours later</b>, believe me. So read the three columns main
<li> <a href="FAQ.d/FAQ.Folders_Sizes.txt">Sizes of folders</a>. </li> <li> <a href="FAQ.d/FAQ.Folders_Sizes.txt">Sizes of folders</a>. </li>
<li> <a href="FAQ.d/FAQ.Admin_Authentication.txt">Admin authentication</a>. </li> <li> <a href="FAQ.d/FAQ.Admin_Authentication.txt">Admin authentication</a>. </li>
<li> <a href="FAQ.d/FAQ.Archiving.txt">Archiving tips</a>. </li> <li> <a href="FAQ.d/FAQ.Archiving.txt">Archiving tips</a>. </li>
<li> <a href="FAQ.d/FAQ.General.txt">General FAQ</a>. </li> <li> <a href="FAQ.d/FAQ.General.txt">General FAQ</a>. </li>
<li> <a href="FAQ.d/FAQ.Fun_Things.txt">Fun things to do with Imapsync</a>. </li>
<li> <a href="FAQ.d/FAQ.POP3.txt">POP3 issues</a>. </li> <li> <a href="FAQ.d/FAQ.POP3.txt">POP3 issues</a>. </li>
<li> <a href="FAQ.d/FAQ.Reporting_Bugs.txt">Reporting bugs</a>. </li> <li> <a href="FAQ.d/FAQ.Reporting_Bugs.txt">Reporting bugs</a>. </li>
<li> <a href="FAQ.d/FAQ.Big_Mailbox.txt">Big mailboxes</a>. </li> <li> <a href="FAQ.d/FAQ.Big_Mailbox.txt">Big mailboxes</a>. </li>
@ -971,7 +987,7 @@ like written in the <a href="LICENSE">https://imapsync.lamiral.info/LICENSE</a>
<!--#config timefmt="%D" --> <!--#config timefmt="%D" -->
<!--#config timefmt="%A %B %d, %Y" --> <!--#config timefmt="%A %B %d, %Y" -->
<b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b> <b>This document last modified on <!--#echo var="LAST_MODIFIED" --></b>
($Id: index.shtml,v 1.499 2021/06/08 09:57:27 gilles Exp gilles $)<br /> ($Id: index.shtml,v 1.508 2021/12/20 17:49:51 gilles Exp gilles $)<br />
<a href="#TOP">Top of the page</a> <a href="#TOP">Top of the page</a>
</p> </p>

13
oauth2/generate_gmail_token
View file

@ -25,7 +25,7 @@ get_refresh_token()
access_token_file="`access_token_file $1`" access_token_file="`access_token_file $1`"
# 1 Get an access_token and a refresh_token # 1 Get an access_token and a refresh_token
./oauth2_google.py --user="$1" \ ${my_dirname}/oauth2_google.py --user="$1" \
--client_id="108687549524-86sjq07f3ch8otl9fnr56mjnniltdrvn.apps.googleusercontent.com" \ --client_id="108687549524-86sjq07f3ch8otl9fnr56mjnniltdrvn.apps.googleusercontent.com" \
--client_secret=zAJO4PLxzeJ4yOaiJRk6f69k \ --client_secret=zAJO4PLxzeJ4yOaiJRk6f69k \
--generate_oauth2_token --quiet --generate_oauth2_token --quiet
@ -44,12 +44,12 @@ regenerate_access_token()
access_token_file="`access_token_file $1`" access_token_file="`access_token_file $1`"
# 2 Regenerate an access_token if time passes too long (3600s) # 2 Regenerate an access_token if time passes too long (3600s)
refresh_token="`cat $refresh_token_file|tr -d '\r'`" refresh_token="`cat $refresh_token_file|tr -d '\r'`"
echo ./oauth2_google.py --quiet --user="$1" \ echo ${my_dirname}/oauth2_google.py --quiet --user="$1" \
--client_id="108687549524-86sjq07f3ch8otl9fnr56mjnniltdrvn.apps.googleusercontent.com" \ --client_id="108687549524-86sjq07f3ch8otl9fnr56mjnniltdrvn.apps.googleusercontent.com" \
--client_secret=zAJO4PLxzeJ4yOaiJRk6f69k \ --client_secret=zAJO4PLxzeJ4yOaiJRk6f69k \
--refresh_token="$refresh_token" --refresh_token="$refresh_token"
./oauth2_google.py --quiet --user="$1" \ ${my_dirname}/oauth2_google.py --quiet --user="$1" \
--client_id="108687549524-86sjq07f3ch8otl9fnr56mjnniltdrvn.apps.googleusercontent.com" \ --client_id="108687549524-86sjq07f3ch8otl9fnr56mjnniltdrvn.apps.googleusercontent.com" \
--client_secret=zAJO4PLxzeJ4yOaiJRk6f69k \ --client_secret=zAJO4PLxzeJ4yOaiJRk6f69k \
--refresh_token="$refresh_token" | tee "$access_token_file" --refresh_token="$refresh_token" | tee "$access_token_file"
@ -86,7 +86,7 @@ generate_oauthdirect()
access_token="`cat $access_token_file`" access_token="`cat $access_token_file`"
oauthdirect_file="`oauthdirect_file $1`" oauthdirect_file="`oauthdirect_file $1`"
# 3 Generate oauth2_string for imap authentication # 3 Generate oauth2_string for imap authentication
./oauth2_google.py --generate_oauth2_string --user="$1" \ ${my_dirname}/oauth2_google.py --generate_oauth2_string --user="$1" \
--access_token="$access_token" | tail -1 | tee "$oauthdirect_file" --access_token="$access_token" | tail -1 | tee "$oauthdirect_file"
echo oauth2_string saved in "$oauthdirect_file" echo oauth2_string saved in "$oauthdirect_file"
echo Give this to imapsync --oauthdirect1 or --oauthdirect2 echo Give this to imapsync --oauthdirect1 or --oauthdirect2
@ -101,7 +101,7 @@ authenticate_imap()
access_token_file="`access_token_file $1`" access_token_file="`access_token_file $1`"
access_token="`cat $access_token_file`" access_token="`cat $access_token_file`"
echo $access_token echo $access_token
./oauth2_imap_from_token "$1" "$access_token" ${my_dirname}/oauth2_imap_from_token "$1" "$access_token"
} }
@ -112,7 +112,7 @@ authenticate_imap_google_from_access_token()
access_token_file="`access_token_file $1`" access_token_file="`access_token_file $1`"
access_token="`cat $access_token_file`" access_token="`cat $access_token_file`"
echo "$access_token" echo "$access_token"
./oauth2_google.py --user="$1" \ ${my_dirname}/oauth2_google.py --user="$1" \
--access_token="$access_token" --test_imap_authentication --access_token="$access_token" --test_imap_authentication
} }
@ -185,5 +185,6 @@ generate_token()
} }
my_dirname=`dirname $0`
generate_token "$@" generate_token "$@"

190
tests.sh
View file

@ -1,6 +1,6 @@
#!/bin/sh #!/bin/sh
# $Id: tests.sh,v 1.365 2021/07/05 20:19:12 gilles Exp gilles $ # $Id: tests.sh,v 1.367 2022/01/13 12:59:05 gilles Exp gilles $
# To run these tests, you need a running imap server somewhere # To run these tests, you need a running imap server somewhere
# with several accounts. And be on Linux or Unix. # with several accounts. And be on Linux or Unix.
@ -218,6 +218,7 @@ set_return_code_variables()
EXIT_ERR_SELECT=117 EXIT_ERR_SELECT=117
EXIT_TRANSFER_EXCEEDED=118 EXIT_TRANSFER_EXCEEDED=118
EXIT_ERR_APPEND_VIRUS=119 EXIT_ERR_APPEND_VIRUS=119
EXIT_ERR_FLAGS=120
EXIT_TESTS_FAILED=254 # Like Test::More API EXIT_TESTS_FAILED=254 # Like Test::More API
} }
@ -272,6 +273,19 @@ option_extra_arguments() {
test "$?" = "$EX_USAGE" test "$?" = "$EX_USAGE"
} }
option_extra() {
(
mkdir -p W/tmp/tests/options_extra/ || return 1
cd W/tmp/tests/options_extra/ || return 1
echo '--debugimap' > options_extra.txt
test -f ../../../../imapsync
../../../../imapsync --testslive
test "$?" = "$EX_OK"
)
pwd
}
passwords_masked() { passwords_masked() {
$CMD_PERL ./imapsync --host1 boumboum --password1 secret --justbanner | grep MASKED $CMD_PERL ./imapsync --host1 boumboum --password1 secret --justbanner | grep MASKED
} }
@ -415,6 +429,19 @@ ll_INBOX() {
--folder INBOX --folder INBOX
} }
ll_daily_digest() {
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST1 --user2 tata \
--passfile2 ../../var/pass/secret.tata \
--folder INBOX --dry --nodry1 --maxage 5 \
--truncmess 1000 --debugcontent --f1f2 INBOX=INBOX.Fake \
| egrep 'From:|To:|Subject:|Date:|=====|msg '
}
ll_acl() { ll_acl() {
$CMD_PERL ./imapsync \ $CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \ --host1 $HOST1 --user1 tata \
@ -641,7 +668,7 @@ ll_abort_noprocess()
# In mandatory_tests # In mandatory_tests
ll_abort_not_a_pid_number() ll_abort_not_a_pid_number()
{ {
echo 999999 > /tmp/imapsync_fake.pid echo 12345678 > /tmp/imapsync_fake.pid
$CMD_PERL ./imapsync \ $CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \ --host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \ --passfile1 ../../var/pass/secret.tata \
@ -649,7 +676,7 @@ ll_abort_not_a_pid_number()
--passfile2 ../../var/pass/secret.titi \ --passfile2 ../../var/pass/secret.titi \
--abort --pidfile /tmp/imapsync_fake.pid \ --abort --pidfile /tmp/imapsync_fake.pid \
--logfile ll_abort_not_a_pid_number.log --logfile ll_abort_not_a_pid_number.log
grep 'pid 999999 in /tmp/imapsync_fake.pid is not a pid number' LOG_imapsync/ll_abort_not_a_pid_number.log grep 'pid 12345678 in /tmp/imapsync_fake.pid is not a pid number' LOG_imapsync/ll_abort_not_a_pid_number.log
} }
@ -884,7 +911,7 @@ ll_final_diff() {
--folder INBOX --f1f2 INBOX=INBOX.final_diff --maxage 30 --folder INBOX --f1f2 INBOX=INBOX.final_diff --maxage 30
} }
ll_with_errors() { ll_with_flags_errors() {
can_send && sendtestmessage can_send && sendtestmessage
can_send && sendtestmessage can_send && sendtestmessage
$CMD_PERL ./imapsync \ $CMD_PERL ./imapsync \
@ -892,9 +919,9 @@ ll_with_errors() {
--passfile1 ../../var/pass/secret.tata \ --passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \ --host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \ --passfile2 ../../var/pass/secret.titi \
--folder INBOX --maxage 30 \ --folder INBOX --maxage 300 \
--regexflag 's/.*/PasGlop \\PasGlopRe/' --regexflag 's/.*/PasGlop \\PasGlopRe/' --errorsmax 5
test "$EXIT_WITH_ERRORS" = "$?" test "$EXIT_ERR_FLAGS" = "$?"
} }
@ -905,10 +932,11 @@ ll_errorsmax() {
--passfile1 ../../var/pass/secret.tata \ --passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \ --host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \ --passfile2 ../../var/pass/secret.titi \
--nofoldersizes --folder INBOX.errors --regexflag 's/.*/PasGlop \\PasGlopRe/' --errorsmax 5 --delete2 --nofoldersizes --folder INBOX.errors --regexflag 's/.*/PasGlop \\PasGlopRe/' --errorsmax 5 \
| grep 'Maximum number of errors 5 reached'
#--pipemess 'grep lalalala' --nopipemesscheck --dry --debugcontent --debugflags #--pipemess 'grep lalalala' --nopipemesscheck --dry --debugcontent --debugflags
#test "$EXIT_WITH_ERRORS_MAX" = "$?" # no longer used since errors classification #test "$EXIT_WITH_ERRORS_MAX" = "$?" # no longer used since errors classification
test "$EXIT_WITH_ERRORS" = "$?" #test "$EXIT_ERR_FLAGS" = "$?"
} }
ll_debug() ll_debug()
@ -1182,7 +1210,7 @@ ll_timeout1_timeout2() {
--passfile1 ../../var/pass/secret.tata \ --passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \ --host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \ --passfile2 ../../var/pass/secret.titi \
--folder INBOX --timeout1 0.99 --timeout2 0.95 --justlogin --folder INBOX --timeout1 1.99 --timeout2 1.95 --justlogin
} }
ll_timeout_timeout1() { ll_timeout_timeout1() {
@ -1777,19 +1805,68 @@ gmail_l_automap() {
ll_justfolders() { ll_justfolders() {
$CMD_PERL ./imapsync \ $CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \ --host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \ --passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \ --host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \ --passfile2 ../../var/pass/secret.titi \
--justfolders --justfolders
echo "sudo rm -rf /home/vmail/titi/.new_folder/" echo "sudo rm -rf /home/vmail/titi/.new_folder/"
} }
ll_create_folder_New1()
{
./W/learn/create_folder localhost tata `cat ../../var/pass/secret.tata` INBOX.New1 INBOX.New1.New1 INBOX.New1.New1.New1
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--justfolders --include New1 --folderfirst INBOX.New1.New1
}
ll_delete_folder_New1()
{
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--include New1 --folderfirst INBOX.New1.New1 --delete1emptyfolders --delete1
}
ll_create_folder_encoding_accent()
{
./W/learn/create_folder localhost tata `cat ../../var/pass/secret.tata` INBOX.New1 'INBOX.New1.E&AwE-le&AwE-ments envoye&AwE-s' 'INBOX.New1.&AMk-l&AOk-ments envoy&AOk-s'
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--justfolders --include New1
}
ll_create_folder_encoding_accent_365()
{
#./W/learn/create_folder localhost tata `cat ../../var/pass/secret.tata` 'INBOX.E&AwE-le&AwE-ments envoye&AwE-s' 'INBOX.&AMk-l&AOk-ments envoy&AOk-s'
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 imap-mail.outlook.com --user2 gilles.lamiral@outlook.com \
--passfile2 ../../var/pass/secret.outlook.com \
--justfolders --include 'ments envoy' --automap --exclude New1
}
ll_justfolders_delete1emptyfolders() { ll_justfolders_delete1emptyfolders() {
./W/learn/create_folder localhost tata `cat /g/var/pass/secret.tata` INBOX.Empty INBOX.Empty.Empty INBOX.Empty.Empty.Empty ./W/learn/create_folder localhost tata `cat ../../var/pass/secret.tata` INBOX.Empty INBOX.Empty.Empty INBOX.Empty.Empty.Empty
$CMD_PERL ./imapsync \ $CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \ --host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \ --passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \ --host2 $HOST2 --user2 titi \
@ -1798,10 +1875,9 @@ ll_justfolders_delete1emptyfolders() {
} }
ll_delete1_delete1emptyfolders() { ll_delete1_delete1emptyfolders() {
./W/learn/create_folder localhost tata `cat /g/var/pass/secret.tata` INBOX.Empty INBOX.Empty.Empty INBOX.Empty.Empty.Empty ./W/learn/create_folder localhost tata `cat ../../var/pass/secret.tata` INBOX.Empty INBOX.Empty.Empty INBOX.Empty.Empty.Empty
$CMD_PERL ./imapsync \ $CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \ --host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \ --passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \ --host2 $HOST2 --user2 titi \
@ -1812,7 +1888,7 @@ ll_delete1_delete1emptyfolders() {
ll_justfolders_skipemptyfolders() { ll_justfolders_skipemptyfolders() {
$CMD_PERL ./imapsync \ $CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \ --host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \ --passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \ --host2 $HOST2 --user2 titi \
@ -2646,6 +2722,27 @@ ll_search_UNSEEN_SENTSINCE()
} }
ll_search_FROM_TO_CC()
{
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--folder INBOX --dry --search 'OR FROM gilles@localhost (OR TO gilles@localhost (CC gilles@localhost))'
}
ll_search_FROM()
{
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--folder INBOX --dry --search 'FROM gilles@localhost'
}
ll_maxage_nonew() ll_maxage_nonew()
{ {
@ -2771,6 +2868,23 @@ ll_exitwhenover()
} }
ll_exitwhenover_noerrorsdump()
{
sendtestmessage
sendtestmessage
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--maxage 1 --folder INBOX --nofoldersizes \
--exitwhenover 100 --noerrorsdump
test "$EXIT_TRANSFER_EXCEEDED" = "$?"
}
ll_folder_INBOX() ll_folder_INBOX()
@ -3046,6 +3160,20 @@ ll_regextrans2()
--folder 'INBOX.yop.yap' --debug --folder 'INBOX.yop.yap' --debug
} }
ll_add_suffix()
{
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--justfolders \
--regextrans2 's,$,\@easterngraphics.com,' \
--folderrec 'INBOX.yop' --dry --justfolders
}
ll_regextrans2_ucfirst_downcase_last_folder() ll_regextrans2_ucfirst_downcase_last_folder()
{ {
# lowercase the last basename part # lowercase the last basename part
@ -3724,6 +3852,7 @@ ll_regexmess_truncate_long_message_substr()
} }
ll_regexmess_truncate_long_message_truncmess() ll_regexmess_truncate_long_message_truncmess()
{ {
# #
@ -6085,6 +6214,15 @@ ll_justlogin_notls() {
--justlogin --notls1 --notls2 --justlogin --notls1 --notls2
} }
ll_justlogin_nocompress() {
$CMD_PERL ./imapsync \
--host1 $HOST1 --user1 tata \
--passfile1 ../../var/pass/secret.tata \
--host2 $HOST2 --user2 titi \
--passfile2 ../../var/pass/secret.titi \
--justlogin --nocompress2
}
@ -6465,6 +6603,11 @@ office365_justlogin_ssl1_ssl2()
--justlogin --justlogin
} }
outlook_login()
{
office365_justlogin_ssl1_ssl2
}
office365_justlogin_tls() office365_justlogin_tls()
{ {
$CMD_PERL ./imapsync \ $CMD_PERL ./imapsync \
@ -6757,10 +6900,11 @@ Alessandro_error_11()
--passfile1 ../../var/pass/secret.tata \ --passfile1 ../../var/pass/secret.tata \
--host2 imap-mail.outlook.com --ssl2 --user2 gilles.lamiral@outlook.com \ --host2 imap-mail.outlook.com --ssl2 --user2 gilles.lamiral@outlook.com \
--passfile2 ../../var/pass/secret.outlook.com \ --passfile2 ../../var/pass/secret.outlook.com \
--folder INBOX.error_11 --debugcontent --nodry --nodry1 --pipemess 'cat /g/Alessandro_error_11.txt' --folder INBOX.error_11 --debugcontent --nodry --nodry1 --pipemess 'cat /g/Alessandro_error_11.txt' \
--regexmess "s{\QSubject: =?TELETEX?Q?Fw=3APresentation_Storia_dell=5C=27Informatica?=\E}{Subject: Presentation Storia dell'Informatica}"
# Subject: =?TELETEX?Q?Fw=3APresentation_Storia_dell=5C=27Informatica?=
# Subject: Presentation Storia dell'Informatica
#--pipemess W/tools/fix_email_for_exchange.py #--pipemess W/tools/fix_email_for_exchange.py
#--pipemess 'reformime -r7' #--pipemess 'reformime -r7'
@ -7237,6 +7381,7 @@ option_releasecheck
option_noreleasecheck option_noreleasecheck
option_bad_delete2 option_bad_delete2
option_extra_arguments option_extra_arguments
option_extra
passfile1_noexist passfile1_noexist
passfile2_noexist passfile2_noexist
passwords_masked passwords_masked
@ -7395,9 +7540,10 @@ ll_noheader_force
ll_noheader ll_noheader
ll_domino1_domino2 ll_domino1_domino2
ll_domino2 ll_domino2
ll_with_errors ll_with_flags_errors
ll_errorsmax ll_errorsmax
ll_exitwhenover ll_exitwhenover
ll_exitwhenover_noerrorsdump
fuzz_basic fuzz_basic
fuzz_network fuzz_network
testslive testslive