Fight for the Internet 1!

Wednesday, September 24, 2014

LibreOffice Color Scheme

Quick blurb about changing color schemes in LibreOffice. I typically use Google Docs for most things but for everything else, it's either Vim or LibreOffice. I prefer dark color schemes, as pure white ones can make my eyes hurt. But I've never been successful with changing or adjusting LibreOffice colors for long or very thoroughly.

While googling for some help, I found this solution:

Tools > Options > LibreOffice > Accessibility > Automatically detect high-contrast mode of the operating system 
In LibreOffice help it says: “High contrast is an operating system setting that changes the system color scheme to improve readability.” I just had to share this since it solved my problem.

 Be sure to also check "Use automatic font color for screen display."

Saturday, July 12, 2014

Amarok FLAC Problems -- Gstreamers vs. VLC Backend

Problem

So I really like the music player Amarok. It's overall excellent, even if I don't use most of its exotic features.

The majority of my music library is in FLAC format now (and growing, since I'm converting). For at least 2 years the Gstreamers backend has performed abysmally with stop-and-go-glitches in FLAC playback. I always figured they would fix it, since this is a major format and it's a serious obvious glitch. But I'm done waiting. In Amarok, this meant if two FLAC files were in my playlist, there was a 50-75% chance it would have glitched playback.

As far as I can tell, this is NOT an Amarok bug at all. The fault lays 100% in Gstreamers. In (many?) desktop environments, the gstreamers audio backend provides support. We need to replaces this, immediately. So here's the fix.

Solution


VLC comes to rescue with an actually functioning sound backend.

1. Install the VLC backend.

In Ubuntu or Debian:
apt-get install phonon-backend-vlc
In Fedora:
yum install phonon-backend-vlc
2. Change backend selection in Phonon: Go to System SettingsMultimedia → Backend
Choose VLC and click Prefer. Your screen should look like:
https://wirejungle.files.wordpress.com/2010/08/phonon-backend.png

Note: KDE used to support the Xine backend but this has since been deprecated.
3. Restart KDE, or possibly restart the entire system. I'm not sure if the PulseAudio system requires a full restart.

This will also solve problems with Volume Override that can occur under GStreamers, even if you have disabled it.

Caveat

Replay Gain doesn't work under VLC backend sadly.

Monday, June 9, 2014

How to move files for Moon+ Reader while preserving metadata (bookmarks, highlights) or recover lost metadata

Overview of Problem
This is about using the Android electronic book reader app "Moon+ Reader" to move your files around successfully without losing the metadata (bookmarks, notes, highlights, progress, etc.). This technique can also be used to recover those if you lost them.

Problem
My books were stored in "/sdcard/Books/Changeling: The Dreaming" (for this example). I needed to move these PDFs and CBZs to a different location. Using a File Manager I moved the files to "/storage/extSdCard/Changeling The Dreaming". (I had to remove the ':' in the name because of the Fat32 filesystem format on the SDCard. Curse you, Microsoft.)

Then I re-imported the books into Moon+ Reader. The app removed the previous item entries and imported the new ones, but it failed to recognize they were the same PDF and CBZ files. It did not transfer the metadata. When I say metadata I refer to the bookmarks, the highlights, the notes, the reading progress, or even the names I manually assigned to the books, etc. (Some of the PDFs needed better names.)
This bug breaks lot of features every time there is moving or renaming files/folders. I love being able to highlight my books, but Moon Reader needs to be able to handle if I move my files around.

Solution 1 -- If you have not yet lost your metadata

0) In case it needs to be said, always make a backup of your book files and your information. Moon+ Reader lets you store backups of your metadata easily through its menus. Do this before proceeding.
1) In Moon+ Reader, go to My Files. (Not "My Shelf". You need to go to My Files specifically.)
2) Using Moon+ Reader, Select and then Move any existing items.

Note that Moon+ Reader may not be able to actually move the files. The reason for this is in Android version 4.4.2, Google revoked the privilege for any apps to modify files on external SD cards. (Some limited Google specific apps can use your SD card though.)

My files were on my external SD, so Moon+ Reader could not actually move them. If your files are only on the internal SD card, it seems to be able to move them just fine.

(WARNING: Because of the warped hybrid of external SD and internal SD storage, Moon+ Reader actually deleted any duplicate files at the target move location, so watch out! Your experience may be different, but always have a backup.)
3) Remove the selected item entries from within Moon+ Reader itself.
4) Now in your file manager, move the actual file object itself to new target location.
5) Back in Moon+ Reader, now do a normal Import Books.

Your metadata should have been transferred when you re-import the new files.


Solution 2 -- If you have lost your metadata 
This is when you have moved your files, and re-imported them, and your metadata didn't show up. Here is how to fix the problem.

0) In case it needs to be said, always make a backup of your book files and your information. Moon+ Reader lets you store backups of your metadata easily through its menus. Do this before proceeding.
1) In Moon+ Reader, remove any of the new items you just imported.

In my case, I had moved files /sdcard/Books/Changeling: The Dreaming (for this example) to a different location at "/storage/extSdCard/Changeling The Dreaming". I went back through Moon+ Reader and removed the files now at "/storage/extSdCard/Changeling The Dreaming".

2) Now Moon+ Reader should have none of the botched entries stored in it.
3) In your file manager, move the files back to where they were before Moon+ Reader lost the metadata. (If your files are stored on an external SD card, remember that Android 4.4.2 doesn't allow most apps to modify external SD contents, so may need to remove the SD card and do the file management on your desktop.)
4) In Moon+ Reader, re-import the old files from their old location.
5) Now Moon+ Reader should restore all your old metadata (bookmarks, notes, highlights, etc.)

Now you are ready to migrate your files in the special way to preserve your metadata.

6) In Moon+ Reader, go to My Files. (Not "My Shelf". You need to go to My Files specifically.)
7) Using Moon+ Reader, Select and then Move any existing items. Note Moon+ Reader will not actually move the files. (WARNING: My setup with Moon+ Reader actually deleted any duplicate files at the target move location, so watch out! Your experience may be different, but always have a backup. )
8) Remove the selected item entries from within Moon+ Reader itself.
9) Now in your file manager, move the actual file object itself to new target location.
10) Back in Moon+ Reader, now do a normal Import Books.

Troubleshooting
If you get "import: 0", then you did not remove the entries before trying to import. Be sure to check if you double imported them, because they might have nasty duplicates at the new import location where you were trying that do not have the transferred meta data.

Hack Editing the Backup Files Directly
Be careful when editing the Backup files yourself. You'll probably need some really basic knowledge of how SQL commands work, but not much.

About Moon+ Reader's Backup Files
Moon+ Reader's backup files have the file extension .mrpro (on the Pro version anyway). This is actually a Zip file. If you extract it, inside is a folder called "com.flyersoft.moonreaderp". Inside that is a bunch of files. In my case, there is typically about 60 files, most of them numbered files from 1.tag to 57.tag files. The important one you want is one of the high numbers. Look at the file size also to help you. One of them is actually the SQLite3 database. Mine was about 1.1 MB, while the rest were tiny, which tipped me off it was the real contents.

SQL Editing
I used an SQL editing program to view the SQLite3 database file. If I remember, I used "Sqliteman" which was simple enough for my needs. Using some simple find and replace string commands in SQL, I edited the existing entries with my meta data (highlights, notes, bookmarks, etc) changing them from the old filepaths to point to the working ones.

Finishing -- Zipping and Re-Importing and Restoring
After editing the appropriate .tag file (which is the SQLite3 database file actually), I zipped the entire folder back up, and gave it a name with a newer date, then imported that through Moon+ Reader, restoring from there. This fixed my paths.

Though now I just prefer to root my devices which allow me to give admin privileges to certain apps to move files around.
 

Addendum: Proposed Solution
I sent an email to the developer of Moon+ Reader about a proposed solution to fix this. I suggested he use file checksums (such as sha128 or md5sum) on each item that is imported. This way he could match files that move or are duplicates.
Moon+ Reader uses SQLite version 3 underneath and all the highlighting, bookmarks, notes, etc. are stored in these. You can even gain direct access to these by having the program use the "Backup" and "Restore" features. So if you want you can edit the SQLite database manually. I have actually hacked my Moon+ Reader SQLite files quite a bit at this point to clean up some bad stored book-entries.
While the developer did kindly respond, he never said that he would do this or not. I hope he does. But he did give me a hint on how to what had happened.

Friday, April 4, 2014

Harddrive Problems Solved

Overview of Problem
Had a bit of a saga trying to trying to get a harddrive to work properly on my computer. Here are the steps I took to fix the issues. Hopefully they may help someone else someday.

Hardware
New Harddrive: Seagate 4TB, 5900 RPM, SATA3 6.0Gps
Motherboard: Brand: ASRock, Model: Z77 Extreme4.

Problem
I tried copying several terabytes of data to the new drive. It doesn't seem to have a problem with lots of small files, but it eventually has an error when copying files that are several gigabyes in size. However the size threshold which triggered errors were inconsistent. Sometimes it would copy 4.5GB files, and other times it would error if I tried copying 1.8GB files. (I later figured out why.)

Upon error, it would remount the drive into read-only mode and no attempts to remount it writeable seemed to work. I always had to reboot. Sometimes konqueror would report "Errno: 30" upon a failed copy.

Occasionally I think it even managed to hard-lock my system. I was impatient so i didn't give it long to wait, (maybe a minute or something), before hard-resetting the power on my computer. (What can I say? I trust EXT3/4's journaling system to recover my work and all my other programs recover as well.)

Diagnosis
I ran some tests using the SMART diagnosis, but it said the drive was healthy. I haven't run a full test yet, but I will just in case. The quick tests reported no problems.

I checked the cabling and I found the drive was on the same power-cable as 3TB Seagate. I don't think they would have been drawing too much electricity over the same line to cause power fluctuations (since they are the only two Serial ATA/SATA drives in the whole system and nothing else was running.) Still I gave the new 4TB Seagate it's own dedicated power line.

Next I did an important step, which I didn't realize until later. I deleted the partition table of the problem drive (it only had one massive 4TB partition), and created a new one. I also reformated the drive. (For most of this article I used EXT4, but I did experiments with XFS. XFS never triggered the error but I didn't wait hours to force a trigger so my results can't be judged either way with that format.)

I have read several accounts stating that once you have triggered the errors, you must correct them or they will continue to happen. These symptoms are consistent with what I encountered, particularly trigger errors with the different large file sizes, none of which were consistent.

I simply wiped/formated the drive, but others claim they have been able to recover using fsck.ext4 to correct the problems. I cannot comment on this.

More Diagnosis
I finally clued into checking the dmesg utility, and found some very useful information.

[ 891.079292] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
[11668.781082] ata4.00: exception Emask 0x10 SAct 0x7fffffff SErr 0x400100 action 0x6 frozen
[11668.781086] ata4.00: irq_stat 0x08000000, interface fatal error
[11668.781087] ata4: SError: { UnrecovData Handshk }
[11668.781089] ata4.00: failed command: WRITE FPDMA QUEUED
[11668.781091] ata4.00: cmd 61/00:00:00:18:b9/04:00:9b:00:00/40 tag 0 ncq 524288 out
res 40/00:d0:00:80:b9/00:00:9b:00:00/40 Emask 0x10 (ATA bus error)
[11668.781093] ata4.00: status: { DRDY }

......
(Snip)
.....
[11846.581765] ata4: hard resetting link
[11846.886462] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[11846.887197] ata4.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
[11846.887199] ata4.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
[11846.887200] ata4.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
[11846.888785] ata4.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
[11846.888788] ata4.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
[11846.888789] ata4.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
[11846.889558] ata4.00: configured for UDMA/133
[11846.889594] ata4: EH complete
[11892.733319] ata4.00: exception Emask 0x10 SAct 0x7fffffff SErr 0x400100 action 0x6 frozen
[11892.733322] ata4.00: irq_stat 0x08000000, interface fatal error
[11892.733324] ata4: SError: { UnrecovData Handshk }
[11892.733325] ata4.00: failed command: WRITE FPDMA QUEUED
[11892.733328] ata4.00: cmd 61/00:00:00:3c:76/04:00:9e:00:00/40 tag 0 ncq 524288 out



Some users online claimed the SATA cable itself could be going bad. This is a genuine possibility, so I switched to a different one.

Also, there are long threads reporting problems like this for SATA microcontrollers, particularly the Marvell 9123 controller. This is a software issue I believe, and not a hardware failure. But I'm no kernel dev. Others have reported issues for JMicron controllers also. I checked my motherboard and found a post by a different user using a very similar model to my own with the same problem. My motherboard has the following (checked by running the 'lspci' command.)

The chipset for the controller is "Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)".

Finally I checked the cabling again in my computer. Mistakenly I had hooked up the harddrive to the SATA-3.0 ports. (There are 8-ports on my system, and it was a simple mistake, but ugh...) In fact, looking at the dmesg output you can even see the drive is reporting running at SATA2-3.0Gps speeds, when it is capable of running at SATA3 speed.

I switched its connection to a SATA3 port.


Solution

I applied several solutions to this problem so far. I'm not sure which one solved it.

1) Gave the device a dedicated power line. It's a big harddrive and sharing power with another big harddrive could have caused minor fluctuations.
2) Recreated the partition table of the problem drive and formated the drive to  EXT4. I made sure to do this after every error / failure.
3) Connected the drive to a different SATA micro-controller on my motherboard.