1.200

CREDITS

@@ -7,6 +7,15 @@ http://www.amazon.com/gp/registry/wishlist/1C9UNDIH3P7R7/
 
 I thank very much all of these people.
 
+Florin Andrei 
+Suggested to remove the bad behavior with INBOX
+and --prefix2 (prefix2 was ignored)
+Had a good discussion about memory in cyrus ml.
+
+Tashfeen Bhimdi
+IO::Socket::SSL have to be optional
+PLAIN without Capability AUTH=PLAIN is needed.
+
 Jalal
 Found --include and --include unuseful
 Wrote --include or  --include patch
@@ -17,7 +26,7 @@ Gave OpenMail to Cyrus success
 
 Nicola De Marco
 Had authenticate problems with PLAIN
-and gave me a donation with the imapsync wishlist.
+and gave me a beautiful donation with the imapsync wishlist.
 
 Arunav Mandal
 Success ?
@@ -32,6 +41,7 @@ Success from Microsoft Exchange 5.5 to Zimbra 4.0.4
 
 Damjan Perenic
 Success from cyrus to notes.
+Pb with big messages.
 
 Tom Allison
 Had problem with dovecot and dbmail
@@ -45,6 +55,8 @@ RISKO Gergely
 Maintain imapsync debian package.
 
 Marten Lehmann
+Made me talk about imapsync cpu and memory hungriness.
+Made me remove the exit(1) in case "No CAPABILITY for AUTHENTICATE"
 Found bug : imapsync does not delete messages immediately
 after a successful transfer but the next run.
 

ChangeLog

@@ -1,15 +1,32 @@
 
 RCS file: RCS/imapsync,v
 Working file: imapsync
-head: 1.197
+head: 1.200
 branch:
 locks: strict
 access list:
 symbolic names:
 keyword substitution: kv
-total revisions: 197;	selected revisions: 197
+total revisions: 200;	selected revisions: 200
 description:
 ----------------------------
+revision 1.200
+date: 2007/01/02 08:27:04;  author: gilles;  state: Exp;  lines: +48 -15
+use message_to_file()instead of message_string()
+use append_file() instead of append_string()
+fixed INBOX with --prefix2 problem.
+Authmechanism() can't receive 'LOGIN' as argument (bug ?).
+----------------------------
+revision 1.199
+date: 2006/12/30 15:53:51;  author: gilles;  state: Exp;  lines: +7 -6
+Sun Java System Messaging Server success
+----------------------------
+revision 1.198
+date: 2006/12/30 04:23:36;  author: gilles;  state: Exp;  lines: +19 -15
+Added a comment about CPU and memory hungriness.
+Removed exit(1) on No auth CAPABILITY.
+Authenticate is always done now.
+----------------------------
 revision 1.197
 date: 2006/12/10 02:49:39;  author: gilles;  state: Exp;  lines: +7 -10
 Updated online documantation about new

README

@@ -3,7 +3,7 @@ NAME
     Synchronise mailboxes between two imap servers. Good at IMAP migration.
     More than 25 different IMAP server softwares supported with success.
 
-    $Revision: 1.197 $
+    $Revision: 1.200 $
 
 INSTALL
      imapsync works fine under any Unix OS.
@@ -76,7 +76,9 @@ DESCRIPTION
     sides. Same headers, same message size and the transfer is done only
     once. All flags are preserved, unread will stay unread, read will stay
     read, deleted will stay deleted. You can stop the transfer at any time
-    and restart it later, imapsync is adapted to a bad connection.
+    and restart it later, imapsync is adapted to a bad connection. imapsync
+    is CPU hungry so nice and renice commands can be a good help. imapsync
+    can be memory hungry too, especially with large messages.
 
     You can decide to delete the messages from the source mailbox after a
     successful transfer (it is a good feature when migrating). In that case,
@@ -200,7 +202,8 @@ IMAP SERVERS
      - Critical Path (7.0.020)
      - Cyrus IMAP 1.5, 1.6, 2.1, 2.1.15, 2.1.16, 2.1.18 
        2.2.1, 2.2.2-BETA, 2.2.10, 2.2.12, 2.3-alpha (OSI Approved),
-       v2.2.12-Invoca-RPM-2.2.12-3.RHEL4.1
+       v2.2.12-Invoca-RPM-2.2.12-3.RHEL4.1,
+       v2.3.1-Invoca-RPM-2.3.1-2.7.fc5,
        (http://asg.web.cmu.edu/cyrus/)
      - David Tobit V8 (proprietary Message system).
      - DBMail 1.2.1, 2.0.4, 2.0.9, 2.2rc1 (GPL) (http://www.dbmail.org/).
@@ -216,13 +219,14 @@ IMAP SERVERS
      - Microsoft Exchange Server 5.5
      - Netscape Mail Server 3.6 (Wintel !)
      - Netscape Messaging Server 4.15 Patch 7
-     - OpenMail IMAP server B.07.00.k0
+     - OpenMail IMAP server B.07.00.k0 (Samsung Contact ?)
      - OpenWave
      - Qualcomm Worldmail (NT)
      - Rockliffe Mailsite 5.3.11
      - Samsung Contact IMAP server 8.5.0
      - Scalix v10.1, 10.0.1.3
      - SunONE Messaging server 5.2, 6.0 (SUN JES - Java Enterprise System)
+     - Sun Java System Messaging Server 6.2-2.05
      - Surgemail 3.6f5-5
      - UW-imap servers (imap-2000b) rijkkramer IMAP4rev1 2000.287
        (RedHat uses UW like 2003.338rh) (OSI Approved) 
@@ -297,5 +301,5 @@ AUTHOR
     teaching free open and gratis softwares. Don't hesitate to pay him for
     that services.
 
-    $Id: imapsync,v 1.197 2006/12/10 02:49:39 gilles Exp $
+    $Id: imapsync,v 1.200 2007/01/02 08:27:04 gilles Exp $
 

VERSION

@@ -1 +1 @@
-1.197
+1.200

imapsync

@@ -7,7 +7,7 @@ tool. Synchronise mailboxes between two imap servers. Good
 at IMAP migration. More than 25 different IMAP server softwares
 supported with success.
 
-$Revision: 1.197 $
+$Revision: 1.200 $
 
 =head1 INSTALL
 
@@ -89,7 +89,9 @@ and the transfer is done only once. All flags are
 preserved, unread will stay unread, read will stay read,
 deleted will stay deleted. You can stop the transfer at any
 time and restart it later, imapsync is adapted to a bad
-connection.
+connection. imapsync is CPU hungry so nice and renice 
+commands can be a good help. imapsync can be memory hungry too,
+especially with large messages.
 
 You can decide to delete the messages from the source mailbox
 after a successful transfer (it is a good feature when migrating).
@@ -186,6 +188,8 @@ http://www.gnu.org/licenses/licenses.html
 
 =head1 BUGS
 
+
+
 No known serious bug.  Report any bug to the author.
 Before reporting bugs, read the FAQ, this README and the
 TODO files.
@@ -229,7 +233,8 @@ Success stories reported with the following 29 imap servers
  - Critical Path (7.0.020)
  - Cyrus IMAP 1.5, 1.6, 2.1, 2.1.15, 2.1.16, 2.1.18 
    2.2.1, 2.2.2-BETA, 2.2.10, 2.2.12, 2.3-alpha (OSI Approved),
-   v2.2.12-Invoca-RPM-2.2.12-3.RHEL4.1
+   v2.2.12-Invoca-RPM-2.2.12-3.RHEL4.1,
+   v2.3.1-Invoca-RPM-2.3.1-2.7.fc5,
    (http://asg.web.cmu.edu/cyrus/)
  - David Tobit V8 (proprietary Message system).
  - DBMail 1.2.1, 2.0.4, 2.0.9, 2.2rc1 (GPL) (http://www.dbmail.org/).
@@ -245,13 +250,14 @@ Success stories reported with the following 29 imap servers
  - Microsoft Exchange Server 5.5
  - Netscape Mail Server 3.6 (Wintel !)
  - Netscape Messaging Server 4.15 Patch 7
- - OpenMail IMAP server B.07.00.k0
+ - OpenMail IMAP server B.07.00.k0 (Samsung Contact ?)
  - OpenWave
  - Qualcomm Worldmail (NT)
  - Rockliffe Mailsite 5.3.11
  - Samsung Contact IMAP server 8.5.0
  - Scalix v10.1, 10.0.1.3
  - SunONE Messaging server 5.2, 6.0 (SUN JES - Java Enterprise System)
+ - Sun Java System Messaging Server 6.2-2.05
  - Surgemail 3.6f5-5
  - UW-imap servers (imap-2000b) rijkkramer IMAP4rev1 2000.287
    (RedHat uses UW like 2003.338rh) (OSI Approved) 
@@ -347,7 +353,7 @@ Gilles LAMIRAL earn his living writing, installing,
 configuring and teaching free open and gratis
 softwares. Don't hesitate to pay him for that services.
 
-$Id: imapsync,v 1.197 2006/12/10 02:49:39 gilles Exp $
+$Id: imapsync,v 1.200 2007/01/02 08:27:04 gilles Exp $
 
 =cut
 
@@ -362,6 +368,7 @@ use Term::ReadKey;
 use MIME::Base64;
 use English;
 use POSIX qw(uname);
+use Fcntl;
 
 eval { require 'usr/include/sysexits.ph' };
 
@@ -403,7 +410,7 @@ my(
 use vars qw ($opt_G); # missing code for this will be option.
 
 
-$rcs = ' $Id: imapsync,v 1.197 2006/12/10 02:49:39 gilles Exp $ ';
+$rcs = ' $Id: imapsync,v 1.200 2007/01/02 08:27:04 gilles Exp $ ';
 $rcs =~ m/,v (\d+\.\d+)/;
 $VERSION = ($1) ? $1 : "UNKNOWN";
 
@@ -440,8 +447,8 @@ $error=0;
 
 my $banner = join("", 
 		  '$RCSfile: imapsync,v $ ',
-		  '$Revision: 1.197 $ ',
-		  '$Date: 2006/12/10 02:49:39 $ ',
+		  '$Revision: 1.200 $ ',
+		  '$Date: 2007/01/02 08:27:04 $ ',
 		  "\n",
 		  "Mail::IMAPClient version used here is ",
 		  $VERSION_IMAPClient,"\n"
@@ -615,23 +622,23 @@ sub login_imap {
 	}
 	print "Banner : ", server_banner($imap);
 	
-	if ($authmech eq "LOGIN") {
-		# Default mode for Mail::IMAPClient, so don't do anything.
-	} elsif ($imap->has_capability("AUTH=$authmech")
-		or $imap->has_capability($authmech)
-		) {
-		$imap->Authmechanism($authmech);
-		$imap->Authcallback(\&plainauth) if $authmech eq "PLAIN";
+	if ($imap->has_capability("AUTH=$authmech")
+	    or $imap->has_capability($authmech)
+	   ) {
+		printf("Host %s says it has CAPABILITY for AUTHENTICATE %s\n",
+		       $imap->Server, $authmech);
 	} else {
-		printf("%s: No CAPABILITY for AUTHENTICATE %s\n",
+		printf("Host %s says it has NO CAPABILITY for AUTHENTICATE %s\n",
 		       $imap->Server, $authmech);
 		if ($authmech eq 'PLAIN') {
 			print "Frequently PLAIN is only supported with SSL, ",
 			  "try --ssl1 or --ssl2 option\n";
 		}
-		exit(1);
 	}
 	
+	$imap->Authmechanism($authmech) unless ($authmech eq 'LOGIN');
+	$imap->Authcallback(\&plainauth) if $authmech eq "PLAIN";
+
 	$imap->User($user);
 	$imap->Authuser($authuser);
 	$imap->Password($password);
@@ -883,7 +890,8 @@ sub to_folder_name {
 	$t_fold = separator_invert($x_fold,$f_sep, $t_sep);
 	$debug and print "inverted   separators : [$t_fold]\n";
 	# Adding the prefix supplied by namespace or the --prefix2 option
-	$t_fold = $t_prefix . $t_fold unless($t_fold eq 'INBOX');
+	$t_fold = $t_prefix . $t_fold 
+	  unless($t_prefix eq "INBOX." and $t_fold eq "INBOX");
 	$debug and print "added   target prefix : [$t_fold]\n";
 
 	# Transforming the folder name by the --regextrans2 option(s)
@@ -1070,13 +1078,21 @@ FOLDER: foreach my $f_fold (@f_folders) {
 			# copy
 			print "+ Copying msg #$f_msg:$f_size to folder $t_fold\n";
 			last FOLDER if $from->IsUnconnected();
-			my $string = $from->message_string($f_msg);
-			foreach my $regexmess (@regexmess) {
-				$debug and print "eval \$string =~ $regexmess\n";
-				eval("\$string =~ $regexmess");
+			#my $string = $from->message_string($f_msg);			
+			my $message_file = "tmp_imapsync_$$";
+			unlink($message_file);
+			$from->message_to_file($message_file, $f_msg);
+			
+			if (@regexmess) {
+				my $string = file_to_string($message_file);
+				foreach my $regexmess (@regexmess) {
+					$debug and print "eval \$string =~ $regexmess\n";
+					eval("\$string =~ $regexmess");
+				}
+				string_to_file($string, $message_file);
 			}
 			$debug and print "F message content begin next line\n",
-			  $string,
+			  file_to_string($message_file),
 			    "F message content ended on previous line\n";
 			my $d = "";
 			if ($syncinternaldates) {
@@ -1094,7 +1110,8 @@ FOLDER: foreach my $f_fold (@f_folders) {
 			print "flags from : [$flags_f][$d]\n";
 			last FOLDER if   $to->IsUnconnected();
 			unless ($dry) {
-				unless($new_id = $to->append_string($t_fold,$string, $flags_f, $d)){
+				#unless($new_id = $to->append_string($t_fold,$string, $flags_f, $d)){
+				unless($new_id = $to->append_file($t_fold, $message_file, $flags_f, $d)){
 					warn "Couldn't append msg #$f_msg (Subject:[".$from->subject($f_msg)."]) to folder $t_fold: ",
 					  $to->LastError, "\n";
 					$error++;
@@ -1115,6 +1132,7 @@ FOLDER: foreach my $f_fold (@f_folders) {
 					}
 				}
 			}
+			unlink($message_file);
 			next MESS;
 		}else{
 			$debug and print "Message id [$m_id] found in t:$t_fold\n";
@@ -1404,6 +1422,26 @@ sub  firstline {
         return $line;
 }
 
+
+sub file_to_string {
+	my($file) = @_;
+	my @string;
+	open FILE, $file or die("$! $file");
+	@string = <FILE>;
+	close FILE;
+	return join("", @string);
+}
+
+
+sub string_to_file {
+	my($string, $file) = @_;
+	sysopen(FILE, $file,O_WRONLY|O_TRUNC|O_CREAT, 0600) or die("$! $file");
+	print FILE $string;
+	close FILE;
+}
+
+
+
 sub usage {
         print <<EOF;
 

learn/file_string

@@ -0,0 +1,27 @@
+#!/usr/bin/perl -w
+
+
+
+sub file_to_string {
+	my($file) = @_;
+	my @string;
+	open FILE, $file or die("$! $file");
+	@string = <FILE>;
+	close FILE;
+	return join("", @string);
+}
+
+use Fcntl;
+sub string_to_file {
+	my($string, $file) = @_;
+	sysopen(FILE, $file,O_WRONLY|O_TRUNC|O_CREAT, 0600) or die("$! $file");
+	print FILE $string;
+	close FILE;
+}
+
+
+
+
+string_to_file("blabla $$ \n", "/tmp/imapsync_t01");
+print file_to_string("/tmp/imapsync_t01");
+#unlink("/tmp/imapsync_t01");

tests.sh

@@ -1,6 +1,6 @@
 #!/bin/sh
 
-# $Id: tests.sh,v 1.57 2006/12/10 02:48:14 gilles Exp gilles $	
+# $Id: tests.sh,v 1.58 2007/01/02 08:23:04 gilles Exp $	
 
 #### Shell pragmas
 
@@ -640,13 +640,13 @@ ll_bigmail() {
 		--host2 localhost --user2 titi@est.belle \
 		--passfile2 /var/tmp/secret.titi \
 		--folder INBOX.bigmail
+		echo 'rm  /home/vmail/titi/.bigmail/cur/*'
 	else
 		:
 	fi
 }
 
 
-
 ##########################
 # specific tests
 ##########################
@@ -852,7 +852,8 @@ test $# -eq 0 && run_tests \
 	ll_authmech_CRAMMD5 \
 	ll_authuser \
 	ll_delete2 \
-	ll_folderrec
+	ll_folderrec \
+	ll_bigmail