Harddrive Problems Solved

Overview of Problem
Had a bit of a saga trying to trying to get a harddrive to work properly on my computer. Here are the steps I took to fix the issues. Hopefully they may help someone else someday.

Hardware
New Harddrive: Seagate 4TB, 5900 RPM, SATA3 6.0Gps
Motherboard: Brand: ASRock, Model: Z77 Extreme4.

Problem
I tried copying several terabytes of data to the new drive. It doesn't seem to have a problem with lots of small files, but it eventually has an error when copying files that are several gigabyes in size. However the size threshold which triggered errors were inconsistent. Sometimes it would copy 4.5GB files, and other times it would error if I tried copying 1.8GB files. (I later figured out why.)

Upon error, it would remount the drive into read-only mode and no attempts to remount it writeable seemed to work. I always had to reboot. Sometimes konqueror would report "Errno: 30" upon a failed copy.

Occasionally I think it even managed to hard-lock my system. I was impatient so i didn't give it long to wait, (maybe a minute or something), before hard-resetting the power on my computer. (What can I say? I trust EXT3/4's journaling system to recover my work and all my other programs recover as well.)

Diagnosis
I ran some tests using the SMART diagnosis, but it said the drive was healthy. I haven't run a full test yet, but I will just in case. The quick tests reported no problems.

I checked the cabling and I found the drive was on the same power-cable as 3TB Seagate. I don't think they would have been drawing too much electricity over the same line to cause power fluctuations (since they are the only two Serial ATA/SATA drives in the whole system and nothing else was running.) Still I gave the new 4TB Seagate it's own dedicated power line.

Next I did an important step, which I didn't realize until later. I deleted the partition table of the problem drive (it only had one massive 4TB partition), and created a new one. I also reformated the drive. (For most of this article I used EXT4, but I did experiments with XFS. XFS never triggered the error but I didn't wait hours to force a trigger so my results can't be judged either way with that format.)

I have read several accounts stating that once you have triggered the errors, you must correct them or they will continue to happen. These symptoms are consistent with what I encountered, particularly trigger errors with the different large file sizes, none of which were consistent.

I simply wiped/formated the drive, but others claim they have been able to recover using fsck.ext4 to correct the problems. I cannot comment on this.

More Diagnosis
I finally clued into checking the dmesg utility, and found some very useful information.

[ 891.079292] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
[11668.781082] ata4.00: exception Emask 0x10 SAct 0x7fffffff SErr 0x400100 action 0x6 frozen
[11668.781086] ata4.00: irq_stat 0x08000000, interface fatal error
[11668.781087] ata4: SError: { UnrecovData Handshk }
[11668.781089] ata4.00: failed command: WRITE FPDMA QUEUED
[11668.781091] ata4.00: cmd 61/00:00:00:18:b9/04:00:9b:00:00/40 tag 0 ncq 524288 out
res 40/00:d0:00:80:b9/00:00:9b:00:00/40 Emask 0x10 (ATA bus error)
[11668.781093] ata4.00: status: { DRDY }
......
(Snip)
.....
[11846.581765] ata4: hard resetting link
[11846.886462] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[11846.887197] ata4.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
[11846.887199] ata4.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
[11846.887200] ata4.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
[11846.888785] ata4.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
[11846.888788] ata4.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
[11846.888789] ata4.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
[11846.889558] ata4.00: configured for UDMA/133
[11846.889594] ata4: EH complete
[11892.733319] ata4.00: exception Emask 0x10 SAct 0x7fffffff SErr 0x400100 action 0x6 frozen
[11892.733322] ata4.00: irq_stat 0x08000000, interface fatal error
[11892.733324] ata4: SError: { UnrecovData Handshk }
[11892.733325] ata4.00: failed command: WRITE FPDMA QUEUED
[11892.733328] ata4.00: cmd 61/00:00:00:3c:76/04:00:9e:00:00/40 tag 0 ncq 524288 out


Some users online claimed the SATA cable itself could be going bad. This is a genuine possibility, so I switched to a different one.

Also, there are long threads reporting problems like this for SATA microcontrollers, particularly the Marvell 9123 controller. This is a software issue I believe, and not a hardware failure. But I'm no kernel dev. Others have reported issues for JMicron controllers also. I checked my motherboard and found a post by a different user using a very similar model to my own with the same problem. My motherboard has the following (checked by running the 'lspci' command.)

The chipset for the controller is "Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)".

Finally I checked the cabling again in my computer. Mistakenly I had hooked up the harddrive to the SATA-3.0 ports. (There are 8-ports on my system, and it was a simple mistake, but ugh...) In fact, looking at the dmesg output you can even see the drive is reporting running at SATA2-3.0Gps speeds, when it is capable of running at SATA3 speed.

I switched its connection to a SATA3 port.


Solution
I applied several solutions to this problem so far. I'm not sure which one solved it.

1) Gave the device a dedicated power line. It's a big harddrive and sharing power with another big harddrive could have caused minor fluctuations.
2) Recreated the partition table of the problem drive and formated the drive to  EXT4. I made sure to do this after every error / failure.
3) Connected the drive to a different SATA micro-controller on my motherboard.

Pulse Audio Dynamic Volumes (Take-2)

So today I tried using PulseAudio Dynamic volumes again, and the quick conclusion is this: They are bad, so very very bad.

Previously I had been cruelly subjected to Dynamic Volumes on with my Desktop through it's several speakers and sub-woofers. Now I tried it on my laptop using headphones / earbuds. The experience was actually worse.

On the surface Dynamic volumes seems nice, because in theory it should raise the total volume of the system as necessary. There is however one drawback. It raises the MASTER VOLUME for the entire system, which means all other applications and any subsequent applications will inherit that volume level.

So say you raise the volume on a piece of music you are playing (in Amarok for example), which is nice. They you load a video in your VLC. The video's audio is louder to begin with AND the VLC volume is set to the previous amount, which was 140%. Suddenly your ears are blasted with maximum audio volume. Painful and wrong.

Seriously. I'd fucking sue a company for earing damage if this wasn't free software with absolutely no warranty. (Okay, I wouldn't sue an open-source free company, even then, because I love open-source, but you get the point.)

HP Photosmart c4280 scanner in Fedora 19 Linux

I own an HP Photosmart c4280 scanner printer combo. I cannot vouch for the printing capabilities, but I can declare that I have got it working with scanning. Here is how to quickly get it setup in Fedora 19 Linux:

For the printing capabilities, run:

sudo yum install hplip hplip-gui libsane-hpaio hpijs  hplip-libs

Then run:

sudo hp-setup

If you just want to use the scanner, you still have to do the Printing Setup steps, but after that, it is ready to go. This setup uses the "Sane" scanning/printing system backend, so any graphic interface that uses that backend will now work.

NOTE: You may need to shutdown/startup the device again, and also restart Sane (xsane).

For a simple straightforward graphic app to help you scan, I suggest installing 'xsane', though there are also Gimp Sane plugins and one for KDE as well I believe.

How to Convert APE (cd images) in Linux

Sometimes when I download music, the format is in APE which isn't convenient in Linux.

To convert a single APE file to a single FLAC file:

ffmpeg -i inputaudio.ape outputaudio.flac

To rip the multiple tracks from an entire CD image (with a .cue file) from a single APE file:

1st, convert the entire large ape to flac.

ffmpeg -i inputaudio.ape outputaudio.flac

2nd, edit the .cue file: Look for the line which lists the file name of the .ape file.

FILE "The Shrimp Shack Shooters - The Album.ape" WAVE

Change this to the converted output flac file:

FILE "The Shrimp Shack Shooters - The Album.flac" WAVE

3rd, then open the .cue file in K3b

4th, switch K3b to Conversion mode, by going to the menu: Project -> Convert Tracks.

Now you can output the tracks from the flac file individually.

Yet More Comments on Fedora's Usability

Quick post: SELinux is not for the masses. I honestly don't think it should be enabled for a desktop version of Linux. It's such a colossal pain with so little gain, and making the changes permanent is an exercise in genuine pain. AppArmor never troubled me and provided 99.99% of the gains.

The other big problem is the Firewall. Making the changes permanent is also not trivial and absolutely not obvious. Yes it can be done, but heavens it's not apparent. It was a serious source of frustration until I had to sit down, while seething with anger, to find the not-obvious permanent area. That GUI needs major reworks, because this will frustrate others. Actually, the fact that the firewall got in the way is something that shouldn't happen in the first place.

I have to genuinely give Ubuntu credit. They make using Linux much easier in many many ways... except for the whole Mir / Unity thing.

More Experiences Moving to Fedora 18

More Experiences Moving to Fedora 18

Here are more experiences more experiences moving from Ubuntu to Fedora 18.

Pros

Smoother integration of certain services, like the Updater. It doesn’t require a password much of the time.

Firewall-Config

After a month or two in Fedora, I discovered a really nice graphical program for configuring my firewall! At first glance and after a simple few tasks, I think it works very nicely with SELinux as well. This is a very pleasant surprise.

Neutral

SELinux: Though I was aware of what SELinux, I had never specifically used it. I must give credit to the nice desktop user interface. For example, I needed to enable additional permissions for Firefox to use some Flash at one point. The GUI for SELinux was really easy to understand; just copy and paste this command into a console to enable that permission.

But on the slight downside, I was not able to configure my SSH exactly as I wanted until I figured out how to configure the firewall. This actually makes sense but wasn't easy for me to find the problem spot. I spent a long time attempting to configure SELinux and months later stumbled upon , and I'm not sure if it is a SELinux problem or not.

Cons

General Users

SELinux can be a pain to the average user unless there's something there to help them. I'm pretty a pretty advanced user in Linux and using SELinux is not simple or convenient. Fedora has a nice little helper System-Tray application but that's still not very useful.

For example, I routinely browse a website that uses flash. To allow certain features, such as saving a screen capture of the flash portion of the website, I have to run a SELinux command.

This wouldn't be a problem, except it is not simple or trivial to make this setting permanent. It's not readily apparent how to do this, so I essentially have to run this command every single day, every time I boot my computer up. Doing such a needlessly repetitive task to enable a certain feature is very Microsoft-Windows-esc, and should not be a problem on the Linux Desktop. There should be an easier way to do this.

Sadly I have to now go read some long and incredibly detailed manuals on SELinux, when I just want to browse a fucking website. Ugh.

This is a real Con, and I never had to do with this before.

Development

Third-Party Fedora Support
Interestingly, I didn’t realize that the world had really jumped onboard with the Ubuntu craze. So many places where I have downloaded software for use in development now only list Ubuntu, instead of Fedora/Debian/Ubuntu or some more generic Linux form.

I really hope this is somewhat problematic for me, other than a minor headache with papercuts in using their Ubuntu-focused software.

Fedora Free Open-Source Software Ideology
While I really really like the Free and Open-Source Software ideology, this has actually been a little troubling in Fedora. For example, it is currently quite difficult to add an proper Oracle Java repository to my system.

While I really like OpenJDK when it works, current development projects of mine require the official Oracle JDK. I also want a repository on my system to handle this, especially for updates. I shouldn’t have to watch the news like a hawk for Java updates.

Yum
There are some features I really like about Yum, such as transaction history and undo-features, and a very nice console interface. But they are also not perfect. Aside from being slower than Apt, it is missing two very nice features. “apt-get autoremove” and the ability to perform a “dry-run” sort of simulation, though the later is really not that big of a deal for me.

I’ve yet to find an easy and convenient way to automatically remove unused dependencies (aka “orphaned” packages).

Also the dist-upgrade command doesn't seem quite as straightforward as the apt version.

Google Music

Not sure that Fedora should be blamed for this, but the Google Music manager is NOT working for me. I've tried the 32 and 64 bit versions, including completely deleting the configuration files under my home directory. Nothing works.

I don't know what to do but this is very annoying.

Downloading GIT sources from GNOME.org Difficult as Expected

Keeping in line perfectly with my experience every time trying to interact with Gnome or anything vaguely related to that community, I am having incredible difficult downloading from their GIT repositories.

I want to get the GIMP soc-2013-psd branch. But it is absolutely fucking unclear how to do this.

God damn it Gnome. Get it to together. I swear you do so much harm by thinking everyone in the world processes information and thought the obscure and unusual way you do. (Gnome3 ring a bell for anyone?)

God almighty the gnome.org git repository has a worse layout than Tumblr. This is abjectly awful.