zydronium/imapsync
1
0
Fork 0
mirror of https://github.com/imapsync/imapsync.git synced 2025-06-03 03:07:14 +02:00
Code Issues Projects Releases Packages Wiki Activity

1.213

Browse source
This commit is contained in:
Nick Bebout 2011-03-12 02:44:32 +00:00
parent 602c0768ee
commit 820693d0b3
10 changed files with 373 additions and 58 deletions
Download patch file Download diff file Expand all files Collapse all files

12
CREDITS
View file

@ -7,6 +7,18 @@ http://www.amazon.com/gp/registry/wishlist/1C9UNDIH3P7R7/
I thank very much all of these people.
Simon Matter
Had problem with append_string()
from Cyrus-IMAPd to Lotus Notes 7.0.2
Simon's friend (his name ?)
Wrote a patch to add flags and date to append_file()
Foo Kok Wai
MDaemon 9.5.4 success
Mercury 4.1 success
UW v12.264 success
Enzo Michelangeli
Made me try a second LOGIN login on first login failure.

28
ChangeLog
View file

@ -1,15 +1,39 @@
RCS file: RCS/imapsync,v
Working file: imapsync
head: 1.209
head: 1.213
branch:
locks: strict
access list:
symbolic names:
keyword substitution: kv
total revisions: 209; selected revisions: 209
total revisions: 213; selected revisions: 213
description:
----------------------------
revision 1.213
date: 2007/02/16 04:07:19; author: gilles; state: Exp; lines: +168 -11
- Back to append_file() modified -> append_file2()
----------------------------
revision 1.212
date: 2007/02/13 05:18:15; author: gilles; state: Exp; lines: +12 -12
32 imap server softwares supported.
Fixed a long standing bug (header keywords weren't
uppercased like said in the comment above)
----------------------------
revision 1.211
date: 2007/02/08 03:25:36; author: gilles; state: Exp; lines: +23 -20
MDaemon 9.5.4 success
Mercury 4.1 success
UW v12.264 success
Docum: link to wishlist
----------------------------
revision 1.210
date: 2007/02/08 02:58:52; author: gilles; state: Exp; lines: +19 -10
- Added whole command line output (I'm fed up to ask for)
- Added link to the imapsync thanks wishlist. I like to help
people for free, but I like books too :-)
- Added count of skipped message due to --dry mode
----------------------------
revision 1.209
date: 2007/02/02 02:06:50; author: gilles; state: Exp; lines: +9 -6
- Started to list the distributions containing an imapsync port.

55
FAQ
View file

@ -369,4 +369,59 @@ Q. From Cyrus to Notes
Juhu! --useheader 'Message-ID' --skipsize does wonders! :)
======================================================================
Q. From cyrus to dbmail
R. (Given by Michael Monnerie, left as is)
dbmail creates a header like
X-DBMail-PhysMessage-ID:94348
on messages. There was another problem with cyrus, other headers get
modified also, so I recommend this for a full sync:
imapsync --host1 cyrushost --user1 u1 --password1 p1 \
--host2 dbmailhost --user2 u2 --password2 p2 \
--useheader 'Message-ID' \
--skipsize \
--syncinternaldates
I needed it because there are obviously some
encoding differences between the two servers, and modified headers lead
to all messages being doubled. Examples:
dbmail:
Content-Type:text/plain; format=flowed; charset=iso-8859-1;
reply-type=original
cyrus:
Content-Type:text/plain; format=flowed; charset="iso-8859-1";
reply-type=original
And also the "Received" headers got blanks added in DBmail, etc.
******************
For a full server copy from cyrus to dbmail I used:
imapsync --host1 cyrus --user1 x --authuser1 x --password1 x --ssl1 \
--host2 dbmail --user2 x --authuser2 x --password2 x \
--skipheader '(^X-|^Received|^MIME-|^Content-Type|^Disposition-|^From|^Cc|^Reply-|^Subject|^To|^DomainKey).*' \
--skipsize \
--sep1 '/' --exclude 'user/demo/Trash' \
--regextrans2 's/^user.//' --syncinternaldates
The 'exclude user/demo/Trash' was used because there was one message
there with 8 bit headers which dbmail doesn't accept, so I had to skip
the whole folder. It would be nice to have an option to just ignore and
log unsyncable messages, but do the rest, instead of stopping.
******************
There are two other major problems:
1) dbmail doesn't accept utf8 header, while cyrus does. imapsync stops
in that case, making sync impossible
To convert the wholes messages from 8bit to 7bit, use option :
--regexmess 's/[\x80-\xff]/X/g'
======================================================================

36
README
View file

@ -1,9 +1,9 @@
NAME
imapsync - IMAP synchronisation, sync, copy or migration tool.
Synchronise mailboxes between two imap servers. Good at IMAP migration.
More than 25 different IMAP server softwares supported with success.
More than 32 different IMAP server softwares supported with success.
$Revision: 1.209 $
$Revision: 1.213 $
INSTALL
imapsync works fine under any Unix OS with perl.
@ -151,13 +151,15 @@ EXIT STATUS
AUTHOR
Gilles LAMIRAL <lamiral@linux-france.org>
Feedback good or bad is always welcome. The first you send me an email
you will receive a confirmation request before I really read your
message.
Feedback good or bad is always welcome.
The newsgroup comp.mail.imap is a good place to talk about imapsync. I
read it when imapsync is concerned.
Gilles LAMIRAL earn his living writing, installing, configuring and
teaching free open and gratis softwares. Do not hesitate to pay him for
that services.
LICENSE
imapsync is free, gratis and open source software cover by the GNU
General Public License. See the GPL file included in the distribution or
@ -181,9 +183,11 @@ BUGS
And also, if it can help :
- operating systems on both sides.
- operating systems on both sides and the third side in case
you run imapsync on a foreign host from the both.
- imapsync with all the options you use, the full command line
you use (except the passwords of course)
you use (except the passwords of course). This can be found
at the beginning of the output.
- output given with --debug --debugimap near the failure point.
IMAP SERVERS
@ -195,7 +199,7 @@ IMAP SERVERS
- dkimap4 2.39
- Imail 7.04 (maybe).
Success stories reported with the following 31 imap servers (softwares
Success stories reported with the following 32 imap servers (softwares
names are in alphabetic order) :
- BincImap 1.2.3 (GPL) (http://www.bincimap.org/)
@ -220,7 +224,8 @@ IMAP SERVERS
- Groupwise IMAP (Novell) 6.x and 7.0. Buggy so see the FAQ.
- iPlanet Messaging server 4.15, 5.1, 5.2
- IMail 7.15 (Ipswitch/Win2003), 8.12
- MDaemon 7.0.1, 8.1
- MDaemon 7.0.1, 8.1, 9.5.4 (Windows server 2003 R2 platform)
- Mercury 4.1 (Windows server 2000 platform)
- Microsoft Exchange Server 5.5
- Netscape Mail Server 3.6 (Wintel !)
- Netscape Messaging Server 4.15 Patch 7
@ -235,7 +240,7 @@ IMAP SERVERS
- Sun Java System Messaging Server 6.2-2.05
- Surgemail 3.6f5-5
- UW-imap servers (imap-2000b) rijkkramer IMAP4rev1 2000.287
(RedHat uses UW like 2003.338rh) (OSI Approved)
(RedHat uses UW like 2003.338rh), v12.264 Solaris 5.7 (OSI Approved)
(http://www.washington.edu/imap/)
- UW - QMail v2.1
- Imap part of TCP/IP suite of VMS 7.3.2
@ -254,7 +259,9 @@ IMAP SERVERS
imapsync --host1 imap.troc.org --host2 imap.trac.org --justconnect
And please rate imapsync at http://freshmeat.net/projects/imapsync/
Please rate imapsync at http://freshmeat.net/projects/imapsync/ or
better give the author a book, he likes books:
http://www.amazon.com/gp/registry/wishlist/1C9UNDIH3P7R7/
HUGE MIGRATION
Have a special attention on options --subscribed --subscribe --delete
@ -302,10 +309,5 @@ SIMILAR SOFTWARES
Feedback (good or bad) will be always welcome.
AUTHOR
Gilles LAMIRAL earn his living writing, installing, configuring and
teaching free open and gratis softwares. Don't hesitate to pay him for
that services.
$Id: imapsync,v 1.209 2007/02/02 02:06:50 gilles Exp $
$Id: imapsync,v 1.213 2007/02/16 04:07:19 gilles Exp $

7
TODO
View file

@ -1,6 +1,13 @@
TODO file for imapsync
----------------------
Add an option to make imapsync automatically
reconnect when the connection drops?
Add an entry to
http://lsm.execpc.com/lsm/
See template lsm.imapsync
Update the distribution list including imapsync in the
INSTALL chapter.

2
VERSION
View file

@ -1 +1 @@
1.209
1.213

6
freshmeat_submition.inp
View file

@ -5,14 +5,14 @@
#RELEASE_FOCUS="Code cleanup"
#RELEASE_FOCUS="Minor feature enhancements"
#RELEASE_FOCUS="Major feature enhancements"
RELEASE_FOCUS="Minor bugfixes"
#RELEASE_FOCUS="Major bugfixes"
#RELEASE_FOCUS="Minor bugfixes"
RELEASE_FOCUS="Major bugfixes"
#RELEASE_FOCUS="Minor security fixes"
#RELEASE_FOCUS="Major security fixes"
#TEXT_BODY="Syntax cleanup"
#TEXT_BODY="Updated documentation"
#TEXT_BODY="Bug fix: be case insensitive with header keywords."
TEXT_BODY="Bug fix: corrected memory problem with big messages."
TEXT_BODY="Bug fix: rewrote the way to store messages to avoid freeze problems with some imap servers"

239
imapsync
View file

@ -4,10 +4,10 @@
imapsync - IMAP synchronisation, sync, copy or migration
tool. Synchronise mailboxes between two imap servers. Good
at IMAP migration. More than 25 different IMAP server softwares
at IMAP migration. More than 32 different IMAP server softwares
supported with success.
$Revision: 1.209 $
$Revision: 1.213 $
=head1 INSTALL
@ -175,13 +175,16 @@ in a Bourne shell:
Gilles LAMIRAL <lamiral@linux-france.org>
Feedback good or bad is always welcome. The first you send
me an email you will receive a confirmation request before I
really read your message.
Feedback good or bad is always welcome.
The newsgroup comp.mail.imap is a good place to talk about
imapsync. I read it when imapsync is concerned.
Gilles LAMIRAL earn his living writing, installing,
configuring and teaching free open and gratis
softwares. Do not hesitate to pay him for that services.
=head1 LICENSE
imapsync is free, gratis and open source software cover by
@ -191,8 +194,6 @@ http://www.gnu.org/licenses/licenses.html
=head1 BUGS
No known serious bug. Report any bug to the author.
Before reporting bugs, read the FAQ, this README and the
TODO files.
@ -211,9 +212,11 @@ In your report, please include:
And also, if it can help :
- operating systems on both sides.
- operating systems on both sides and the third side in case
you run imapsync on a foreign host from the both.
- imapsync with all the options you use, the full command line
you use (except the passwords of course)
you use (except the passwords of course). This can be found
at the beginning of the output.
- output given with --debug --debugimap near the failure point.
=head1 IMAP SERVERS
@ -226,7 +229,7 @@ Failure stories reported with the following 4 imap servers :
- dkimap4 2.39
- Imail 7.04 (maybe).
Success stories reported with the following 31 imap servers
Success stories reported with the following 32 imap servers
(softwares names are in alphabetic order) :
- BincImap 1.2.3 (GPL) (http://www.bincimap.org/)
@ -251,7 +254,8 @@ Success stories reported with the following 31 imap servers
- Groupwise IMAP (Novell) 6.x and 7.0. Buggy so see the FAQ.
- iPlanet Messaging server 4.15, 5.1, 5.2
- IMail 7.15 (Ipswitch/Win2003), 8.12
- MDaemon 7.0.1, 8.1
- MDaemon 7.0.1, 8.1, 9.5.4 (Windows server 2003 R2 platform)
- Mercury 4.1 (Windows server 2000 platform)
- Microsoft Exchange Server 5.5
- Netscape Mail Server 3.6 (Wintel !)
- Netscape Messaging Server 4.15 Patch 7
@ -266,7 +270,7 @@ Success stories reported with the following 31 imap servers
- Sun Java System Messaging Server 6.2-2.05
- Surgemail 3.6f5-5
- UW-imap servers (imap-2000b) rijkkramer IMAP4rev1 2000.287
(RedHat uses UW like 2003.338rh) (OSI Approved)
(RedHat uses UW like 2003.338rh), v12.264 Solaris 5.7 (OSI Approved)
(http://www.washington.edu/imap/)
- UW - QMail v2.1
- Imap part of TCP/IP suite of VMS 7.3.2
@ -287,7 +291,10 @@ Example :
imapsync --host1 imap.troc.org --host2 imap.trac.org --justconnect
And please rate imapsync at http://freshmeat.net/projects/imapsync/
Please rate imapsync at http://freshmeat.net/projects/imapsync/
or better give the author a book, he likes books:
http://www.amazon.com/gp/registry/wishlist/1C9UNDIH3P7R7/
=head1 HUGE MIGRATION
@ -353,13 +360,9 @@ Entries for imapsync:
Feedback (good or bad) will be always welcome.
=head1 AUTHOR
$Id: imapsync,v 1.213 2007/02/16 04:07:19 gilles Exp $
Gilles LAMIRAL earn his living writing, installing,
configuring and teaching free open and gratis
softwares. Don't hesitate to pay him for that services.
$Id: imapsync,v 1.209 2007/02/02 02:06:50 gilles Exp $
=cut
@ -403,7 +406,7 @@ my(
$mess_size_total_trans,
$mess_size_total_skipped,
$mess_size_total_error,
$mess_trans, $mess_skipped,
$mess_trans, $mess_skipped, $mess_skipped_dry,
$timeout, # whr (ESS/PRW)
$timestart, $timeend, $timediff,
$timesize, $timebefore,
@ -416,7 +419,7 @@ my(
use vars qw ($opt_G); # missing code for this will be option.
$rcs = ' $Id: imapsync,v 1.209 2007/02/02 02:06:50 gilles Exp $ ';
$rcs = ' $Id: imapsync,v 1.213 2007/02/16 04:07:19 gilles Exp $ ';
$rcs =~ m/,v (\d+\.\d+)/;
$VERSION = ($1) ? $1 : "UNKNOWN";
@ -429,7 +432,7 @@ check_lib_version() or
$mess_size_total_trans = 0;
$mess_size_total_skipped = 0;
$mess_size_total_error = 0;
$mess_trans = $mess_skipped = 0;
$mess_trans = $mess_skipped = $mess_skipped_dry = 0;
sub check_lib_version {
@ -453,11 +456,13 @@ $error=0;
my $banner = join("",
'$RCSfile: imapsync,v $ ',
'$Revision: 1.209 $ ',
'$Date: 2007/02/02 02:06:50 $ ',
'$Revision: 1.213 $ ',
'$Date: 2007/02/16 04:07:19 $ ',
"\n",
"Mail::IMAPClient version used here is ",
$VERSION_IMAPClient,"\n"
$VERSION_IMAPClient,"\n",
"Command line used :\n",
"$0 @ARGV\n",
);
unless(defined(&_SYSEXITS_H)) {
@ -1120,12 +1125,13 @@ FOLDER: foreach my $f_fold (@f_folders) {
unlink($message_file);
$from->message_to_file($message_file, $f_msg);
my $string = file_to_string($message_file);
#unlink($message_file);
if (@regexmess) {
foreach my $regexmess (@regexmess) {
$debug and print "eval \$string =~ $regexmess\n";
eval("\$string =~ $regexmess");
}
#string_to_file($string, $message_file);
string_to_file($string, $message_file);
}
$debug and print "F message content begin next line\n",
$string,
@ -1146,9 +1152,10 @@ FOLDER: foreach my $f_fold (@f_folders) {
print "flags from : [$flags_f][$d]\n";
last FOLDER if $to->IsUnconnected();
unless ($dry) {
unless($new_id = $to->append_string($t_fold,$string, $flags_f, $d)){
#unless($new_id = $to->append_file($t_fold, $message_file, $flags_f, $d)){
warn "Couldn't append msg #$f_msg (Subject:[".$from->subject($f_msg)."]) to folder $t_fold: ",
#unless($new_id = $to->append_string($t_fold,$string, $flags_f, $d)){
unless($new_id = $to->append_file2($t_fold, $message_file, "", $flags_f, $d)){
warn "Couldn't append msg #$f_msg (Subject:[".
$from->subject($f_msg)."]) to folder $t_fold: ",
$to->LastError, "\n";
$error++;
$mess_size_total_error += $f_size;
@ -1167,6 +1174,8 @@ FOLDER: foreach my $f_fold (@f_folders) {
$from->expunge() if ($expunge and not $dry);
}
}
}else{
$mess_skipped_dry += 1;
}
unlink($message_file);
next MESS;
@ -1296,17 +1305,21 @@ sub select_msgs {
return(@msgs);
}
sub stats {
print "++++ Statistics ++++\n";
print "Time : $timediff sec\n";
print "Messages transferred : $mess_trans\n";
print "Messages transferred : $mess_trans ";
print "(could be $mess_skipped_dry without dry mode)" if ($dry);
print "\n";
print "Messages skipped : $mess_skipped\n";
print "Total bytes transferred: $mess_size_total_trans\n";
print "Total bytes skipped : $mess_size_total_skipped\n";
print "Total bytes error : $mess_size_total_error\n";
print "Detected $error errors\n";
print "Please, rate imapsync at http://freshmeat.net/projects/imapsync/\n";
print "?Happy with this free, open source and gratis GPL software?\n",
"Feel free to thank the author by giving him a book:\n",
"http://www.amazon.com/gp/registry/wishlist/1C9UNDIH3P7R7/\n";
}
@ -1409,15 +1422,15 @@ sub parse_header_msg1 {
# remove the first blanks (dbmail bug ?)
# and uppercase header keywords
# (dbmail and dovecot)
$val =~ s/^\s*(.+?):(.+)$/\U$1\E:$2/;
$val =~ s/^\s*(.+)$/$1/;
my $H = uc($h);
# show stuff in debug mode
$debug and print "${s}H $h:", $val, "\n";
if ($skipheader and $h =~ m/$skipheader/) {
$debug and print "${s}H $H:", $val, "\n";
if ($skipheader and $H =~ m/$skipheader/i) {
$debug and print "Skipping header $h\n";
next;
}
$headstr .= "$h:". $val;
$headstr .= "$H:". $val;
}
}
#return unless ($headstr);
@ -1633,6 +1646,162 @@ sub Split {
return $self->{SPLIT};
}
# From IMAPClient.pm
sub append_file2 {
my $self = shift;
my $folder = $self->Massage(shift);
my $file = shift;
my $control = shift || undef;
my $count = $self->Count($self->Count+1);
my $flags = shift || undef;
my $date = shift || undef;
if (defined($flags)) {
$flags =~ s/^\s+//g;
$flags =~ s/\s+$//g;
}
if (defined($date)) {
$date =~ s/^\s+//g;
$date =~ s/\s+$//g;
}
$flags = "($flags)" if $flags and $flags !~ /^\(.*\)$/ ;
$date = qq/"$date"/ if $date and $date !~ /^"/ ;
unless ( -f $file ) {
$self->LastError("File $file not found.\n");
return undef;
}
my $fh = IO::File->new($file) ;
unless ($fh) {
$self->LastError("Unable to open $file: $!\n");
$@ = "Unable to open $file: $!" ;
carp "unable to open $file: $!" if $^W;
return undef;
}
my $bare_nl_count = scalar grep { /^\x0a$|[^\x0d]\x0a$/} <$fh>;
seek($fh,0,0);
my $clear = $self->Clear;
$self->Clear($clear)
if $self->Count >= $clear and $clear > 0;
my $length = ( -s $file ) + $bare_nl_count;
my $string = "$count APPEND $folder " .
( $flags ? "$flags " : "" ) .
( $date ? "$date " : "" ) .
"{" . $length . "}\x0d\x0a" ;
$self->_record($count,[ $self->_next_index($count), "INPUT", "$string" ] );
my $feedback = $self->_send_line("$string");
unless ($feedback) {
$self->LastError("Error sending '$string' to IMAP: $!\n");
close $fh;
return undef;
}
my ($code, $output) = ("","");
until ( $code ) {
$output = $self->_read_line or close $fh, return undef;
foreach my $o (@$output) {
$self->_record($count,$o); # $o is already an array ref
($code) = $o->[DATA] =~ /(^\+|^\d+\sNO|^\d+\sBAD)/i;
if ($o->[DATA] =~ /^\*\s+BYE/) {
carp $o->[DATA] if $^W;
$self->State(Unconnected);
close $fh;
return undef ;
} elsif ( $o->[DATA]=~ /^\d+\s+(NO|BAD)/i ) {
carp $o->[DATA] if $^W;
close $fh;
return undef;
}
}
}
{ # Narrow scope
# Slurp up headers: later we'll make this more efficient I guess
local $/ = "\x0d\x0a\x0d\x0a";
my $text = <$fh>;
$text =~ s/\x0d?\x0a/\x0d\x0a/g;
$self->_record($count,[ $self->_next_index($count), "INPUT", "{From file $file}" ] ) ;
$feedback = $self->_send_line($text);
unless ($feedback) {
$self->LastError("Error sending append msg text to IMAP: $!\n");
close $fh;
return undef;
}
_debug $self, "control points to $$control\n" if ref($control) and $self->Debug;
$/ = ref($control) ? "\x0a" : $control ? $control : "\x0a";
while (defined($text = <$fh>)) {
$text =~ s/\x0d?\x0a/\x0d\x0a/g;
$self->_record( $count,
[ $self->_next_index($count), "INPUT", "{from $file}\x0d\x0a" ]
);
$feedback = $self->_send_line($text,1);
unless ($feedback) {
$self->LastError("Error sending append msg text to IMAP: $!\n");
close $fh;
return undef;
}
}
$feedback = $self->_send_line("\x0d\x0a");
unless ($feedback) {
$self->LastError("Error sending append msg text to IMAP: $!\n");
close $fh;
return undef;
}
}
# Now for the crucial test: Did the append work or not?
($code, $output) = ("","");
my $uid = undef;
until ( $code ) {
$output = $self->_read_line or return undef;
foreach my $o (@$output) {
$self->_record($count,$o); # $o is already an array ref
$self->_debug("append_file: Deciding if " . $o->[DATA] . " has the code.\n")
if $self->Debug;
($code) = $o->[DATA] =~ /^\d+\s(NO|BAD|OK)/i;
# try to grab new msg's uid from o/p
$o->[DATA] =~ m#UID\s+\d+\s+(\d+)\]# and $uid = $1;
if ($o->[DATA] =~ /^\*\s+BYE/) {
carp $o->[DATA] if $^W;
$self->State(Unconnected);
close $fh;
return undef ;
} elsif ( $o->[DATA]=~ /^\d+\s+(NO|BAD)/i ) {
carp $o->[DATA] if $^W;
close $fh;
return undef;
}
}
}
close $fh;
if ($code !~ /^OK/i) {
return undef;
}
return defined($uid) ? $uid : $self;
}
# From IMAPClient.pm
sub fetch_hash2 {

30
learn/delete Executable file
View file

@ -0,0 +1,30 @@
#!/usr/bin/perl -w
use Mail::IMAPClient;
$ARGV[3] or die "usage: $0 host user password folder uid1 uid2 ...\n";
$host = $ARGV[0];
$user = $ARGV[1];
$password = $ARGV[2];
$folder = $ARGV[3];
my $imap = Mail::IMAPClient->new();
$imap->Debug(1);
$imap->Server($host);
$imap->connect() or die;
$imap->User($user);
$imap->Password($password);
$imap->login() or die;
$imap->Uid(1);
$imap->Peek(1);
$imap->select($folder) or die;
foreach $uid (@ARGV[4..$#ARGV]) {
print "deleting $uid\n";
$imap->delete_message($uid);
$imap->expunge();
}
$imap->close();

16
lsm.imapsync Normal file
View file

@ -0,0 +1,16 @@
Begin4
Title: imapsync
Version: 1.209
Entered-date: 2007-01-09
Description: IMAP synchronisation, sync, copy or migration tool.
Synchronise mailboxes between two imap servers. Good at IMAP migration.
More than 32 different IMAP server softwares supported with success.
Keywords: IMAP synchronisation mail
Author: lamiral@linux-france.org (Gilles LAMIRAL)
Maintained-by: lamiral@linux-france.org (Gilles LAMIRAL)
Primary-site: http://www.linux-france.org/prj/imapsync/dist/
Alternate-site:
Original-site: http://www.linux-france.org/prj/imapsync/dist/
Platforms: UNIX Windows
Copying-policy: GPL
End