IMAP Accounts Migration
When you move your email account around, either between different hosting providers or to/from self-hosted system, you usually want to take your mailbox with you. Depending on the size of the mailbox it may be as easy as drag-n-drop your emails between accounts for relatively small email history. However, if you have GBs of emails in multiple folders with large attachments drag-n-drop method would not work. You something else, a rubost, method that can work for hours or days, can handle disconnections to reliable transfer all your emails.
There are external services who offer transferring your mailboxes but from what I have seen, they are quite costly for what they offer, plus yuo basically give them full access to all your mailbox content.
There is another way, a command line utility called imapsync which can do the work for free, very reliably and efficiently.
Personal notes: My personal notes on how to setup stuff to make it easier to repeat next time.
imapsync is a command line utility that can transfer all your emails, folders and attachments from one IMAP account to another. It handles disconnections. It can be run multiple times, avoiding disconnections and ensuring all your emails are eventually transfered.
Here is how to do it.
Step 1: Installation
Download the most recent release:
wget https://github.com/imapsync/imapsync/archive/refs/tags/imapsync-2.200.tar.gz
Example:
~$ wget https://github.com/imapsync/imapsync/archive/refs/tags/imapsync-2.200.tar.gz
--2022-07-04 19:17:22-- https://github.com/imapsync/imapsync/archive/refs/tags/imapsync-2.200.tar.gz
Resolving github.com (github.com)... 192.30.255.113
Connecting to github.com (github.com)|192.30.255.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/imapsync/imapsync/tar.gz/refs/tags/imapsync-2.200 [following]
--2022-07-04 19:17:23-- https://codeload.github.com/imapsync/imapsync/tar.gz/refs/tags/imapsync-2.200
Resolving codeload.github.com (codeload.github.com)... 192.30.255.121
Connecting to codeload.github.com (codeload.github.com)|192.30.255.121|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘imapsync-2.200.tar.gz’
imapsync-2.200.tar.gz [ <=> ] 1.86M 3.02MB/s in 0.6s
2022-07-04 19:17:24 (3.02 MB/s) - ‘imapsync-2.200.tar.gz’ saved [1948095]
Unpack it:
tar -xzf imapsync-2.200.tar.gz
Step 2: Setup
Then, inside the folder, there is all you need to run the tool:
~$ cd imapsync-imapsync-2.200/
~/imapsync-imapsync-2.200$ ll
total 1524
drwxrwxr-x 10 kobit kobit 4096 Jun 1 15:48 ./
drwxr-xr-x 12 kobit kobit 4096 Jul 4 19:17 ../
-rw-rw-r-- 1 kobit kobit 224664 Jun 1 15:48 ChangeLog
-rw-rw-r-- 1 kobit kobit 29457 Jun 1 15:48 CREDITS
drwxrwxr-x 3 kobit kobit 4096 Jun 1 15:48 doc/
drwxrwxr-x 2 kobit kobit 4096 Jun 1 15:48 examples/
lrwxrwxrwx 1 kobit kobit 21 Jun 1 15:48 FAQ -> FAQ.d/FAQ.General.txt
drwxrwxr-x 2 kobit kobit 4096 Jun 1 15:48 FAQ.d/
drwxrwxr-x 3 kobit kobit 4096 Jun 1 15:48 .github/
-rw-rw-r-- 1 kobit kobit 177 Jun 1 15:48 .gitignore
-rwxrwxr-x 1 kobit kobit 802386 Jun 1 15:48 imapsync*
-rw-rw-r-- 1 kobit kobit 38751 Jun 1 15:48 index.shtml
lrwxrwxrwx 1 kobit kobit 25 Jun 1 15:48 INSTALL -> INSTALL.d/INSTALL.ANY.txt
drwxrwxr-x 2 kobit kobit 4096 Jun 1 15:48 INSTALL.d/
-rw-rw-r-- 1 kobit kobit 320 Jun 1 15:48 LICENSE
-rw-rw-r-- 1 kobit kobit 37011 Jun 1 15:48 Makefile
-rw-rw-r-- 1 kobit kobit 320 Jun 1 15:48 NOLIMIT
drwxrwxr-x 2 kobit kobit 4096 Jun 1 15:48 oauth2/
-rw-rw-r-- 1 kobit kobit 44411 Jun 1 15:48 README
lrwxrwxrwx 1 kobit kobit 6 Jun 1 15:48 README.txt -> README
-rw-rw-r-- 1 kobit kobit 8530 Jun 1 15:48 README_Windows.txt
drwxrwxr-x 5 kobit kobit 4096 Jun 1 15:48 S/
-rw-rw-r-- 1 kobit kobit 239783 Jun 1 15:48 tests.sh
-rw-rw-r-- 1 kobit kobit 49440 Jun 1 15:48 TODO
-rw-rw-r-- 1 kobit kobit 6 Jun 1 15:48 VERSION
-rwxrwxr-x 1 kobit kobit 4725 Jun 1 15:48 webserver*
drwxrwxr-x 3 kobit kobit 4096 Jun 1 15:48 X/
The main part is, of course the imapsync file, which is a perl script. There is tons of options and ways to run the sync. Some dedicated to specific services like gmail. Through my trials and errors I found a combination of options which work quite well.
Step 3. Executing the transfer
Executing the tool is quite simple. You just need to give it a source account details and destination account details and that’s it. Like this:
./imapsync --host1 ${SRC_HOST} --user1 ${SRC_USER} --password1 "${SRC_PASS}" \
--host2 ${DEST_HOST} --user2 ${DEST_USER} --password2 "${DEST_PASS}"
However, there are some additional options to tune the transfer for speed, folders mapping, authentication and so on.
Here are a few examples.
GMail
GMail is quite simple because, there are options dedicated for GMail which actually set bunch of other parameters tuned for GMail service.
./imapsync --host1 ${SRC_HOST} --user1 ${SRC_USER} --password1 "${SRC_PASS}" \
--host2 ${DEST_HOST} --user2 ${DEST_USER} --password2 "${DEST_PASS}" \
--gmail1 "$@"
The --gmail1 option is crucial here.
ICloud, Fastmail, …
Many providers have certain limits on the traffic we can generate, transfer speed and volume. It is worth spending some time on reading about the limits to avoid ban or other consequences of abuse.
Here is a command you can run to tune the traffic you generate:
./imapsync --host1 ${SRC_HOST} --user1 ${SRC_USER} --password1 "${SRC_PASS}" \
--host2 ${DEST_HOST} --user2 ${DEST_USER} --password2 "${DEST_PASS}" \
--automap --maxbytespersecond 50_000 --maxbytesafter 500_000_000 "$@"
The --maxbytesafter option usually needs special value depending on the provider. --automap ensures correct folders mapping. If it does not work, you may need ro provide your own mapping for certain folders.
Full example
If you have very big mailbox, I suggest to write super simple script, which can be run multiple times in case the connection is broken or other problems.
The script can look like this:
#!/bin/bash
DEST_HOST=imap.fastmail.com
DEST_USER=myfastmailuser@mydomain.com
DEST_PASS="myfastmailassword"
SRC_HOST=imap.mail.me.com
SRC_USER=myiclouduser@icloud.com
SRC_PASS="myicloudpasswrod"
./imapsync --host1 ${SRC_HOST} --user1 {$SRC_USER} --password1 "${SRC_PASS}" \
--host2 ${DEST_HOST} --user2 ${DEST_USER} --password2 "${DEST_PASS}" \
--automap --maxbytespersecond 50_000 --maxbytesafter 500_000_000 "$@"
And just run the script:
./my_imapsync.sh