How to move files for Moon+ Reader while preserving metadata (bookmarks, highlights) or recover lost metadata

Overview of Problem

This is about using the Android electronic book reader app "Moon+ Reader" to move your files around successfully without losing the metadata (bookmarks, notes, highlights, progress, etc.). This technique can also be used to recover those if you lost them.

Problem

My books were stored in "/sdcard/Books/Changeling: The Dreaming" (for this example). I needed to move these PDFs and CBZs to a different location. Using a File Manager I moved the files to "/storage/extSdCard/Changeling The Dreaming". (I had to remove the ':' in the name because of the Fat32 filesystem format on the SDCard. Curse you, Microsoft.)

Then I re-imported the books into Moon+ Reader. The app removed the previous item entries and imported the new ones, but it failed to recognize they were the same PDF and CBZ files. It did not transfer the metadata. When I say metadata I refer to the bookmarks, the highlights, the notes, the reading progress, or even the names I manually assigned to the books, etc. (Some of the PDFs needed better names.)

This bug breaks lot of features every time there is moving or renaming files/folders. I love being able to highlight my books, but Moon Reader needs to be able to handle if I move my files around.

Solution 1 -- If you have not yet lost your metadata

0) In case it needs to be said, always make a backup of your book files and your information. Moon+ Reader lets you store backups of your metadata easily through its menus. Do this before proceeding.

1) In Moon+ Reader, go to My Files. (Not "My Shelf". You need to go to My Files specifically.)

2) Using Moon+ Reader, Select and then Move any existing items.

Note that Moon+ Reader may not be able to actually move the files. The reason for this is in Android version 4.4.2, Google revoked the privilege for any apps to modify files on external SD cards. (Some limited Google specific apps can use your SD card though.)

My files were on my external SD, so Moon+ Reader could not actually move them. If your files are only on the internal SD card, it seems to be able to move them just fine.

(WARNING: Because of the warped hybrid of external SD and internal SD storage, Moon+ Reader actually deleted any duplicate files at the target move location, so watch out! Your experience may be different, but always have a backup.)

3) Remove the selected item entries from within Moon+ Reader itself.

4) Now in your file manager, move the actual file object itself to new target location.
5) Back in Moon+ Reader, now do a normal Import Books.

Your metadata should have been transferred when you re-import the new files.

Solution 2 -- If you have lost your metadata 

This is when you have moved your files, and re-imported them, and your metadata didn't show up. Here is how to fix the problem.

0) In case it needs to be said, always make a backup of your book files and your information. Moon+ Reader lets you store backups of your metadata easily through its menus. Do this before proceeding.

1) In Moon+ Reader, remove any of the new items you just imported.

In my case, I had moved files /sdcard/Books/Changeling: The Dreaming (for this example) to a different location at "/storage/extSdCard/Changeling The Dreaming". I went back through Moon+ Reader and removed the files now at "/storage/extSdCard/Changeling The Dreaming".

2) Now Moon+ Reader should have none of the botched entries stored in it.

3) In your file manager, move the files back to where they were before Moon+ Reader lost the metadata. (If your files are stored on an external SD card, remember that Android 4.4.2 doesn't allow most apps to modify external SD contents, so may need to remove the SD card and do the file management on your desktop.)

4) In Moon+ Reader, re-import the old files from their old location.

5) Now Moon+ Reader should restore all your old metadata (bookmarks, notes, highlights, etc.)

Now you are ready to migrate your files in the special way to preserve your metadata.

6) In Moon+ Reader, go to My Files. (Not "My Shelf". You need to go to My Files specifically.)

7) Using Moon+ Reader, Select and then Move any existing items. Note Moon+ Reader will not actually move the files. (WARNING: My setup with Moon+ Reader actually deleted any duplicate files at the target move location, so watch out! Your experience may be different, but always have a backup. )

8) Remove the selected item entries from within Moon+ Reader itself.

9) Now in your file manager, move the actual file object itself to new target location.
10) Back in Moon+ Reader, now do a normal Import Books.

Troubleshooting

If you get "import: 0", then you did not remove the entries before trying to import. Be sure to check if you double imported them, because they might have nasty duplicates at the new import location where you were trying that do not have the transferred meta data.

Hack Editing the Backup Files Directly

Be careful when editing the Backup files yourself. You'll probably need some really basic knowledge of how SQL commands work, but not much.

About Moon+ Reader's Backup Files
Moon+ Reader's backup files have the file extension .mrpro (on the Pro version anyway). This is actually a Zip file. If you extract it, inside is a folder called "com.flyersoft.moonreaderp". Inside that is a bunch of files. In my case, there is typically about 60 files, most of them numbered files from 1.tag to 57.tag files. The important one you want is one of the high numbers. Look at the file size also to help you. One of them is actually the SQLite3 database. Mine was about 1.1 MB, while the rest were tiny, which tipped me off it was the real contents.

SQL Editing
I used an SQL editing program to view the SQLite3 database file. If I remember, I used "Sqliteman" which was simple enough for my needs. Using some simple find and replace string commands in SQL, I edited the existing entries with my meta data (highlights, notes, bookmarks, etc) changing them from the old filepaths to point to the working ones.

Finishing -- Zipping and Re-Importing and Restoring
After editing the appropriate .tag file (which is the SQLite3 database file actually), I zipped the entire folder back up, and gave it a name with a newer date, then imported that through Moon+ Reader, restoring from there. This fixed my paths.

Though now I just prefer to root my devices which allow me to give admin privileges to certain apps to move files around.
 

Addendum: Proposed Solution

I sent an email to the developer of Moon+ Reader about a proposed solution to fix this. I suggested he use file checksums (such as sha128 or md5sum) on each item that is imported. This way he could match files that move or are duplicates.

Moon+ Reader uses SQLite version 3 underneath and all the highlighting, bookmarks, notes, etc. are stored in these. You can even gain direct access to these by having the program use the "Backup" and "Restore" features. So if you want you can edit the SQLite database manually. I have actually hacked my Moon+ Reader SQLite files quite a bit at this point to clean up some bad stored book-entries.

While the developer did kindly respond, he never said that he would do this or not. I hope he does. But he did give me a hint on how to what had happened.