MidnightReign.org

Sloppiness really irritates me. Case in point – I install some updates for the server, and schedule a midnight (my local time) reboot so that some of the updates can take effect. It turns out that one of the updates decided to disable the mysql daemon from starting up. Nice going Mr. Package Maintainer, thank you.

Unfortunately, not only do my web sites rely on it, but my mail system does too.

It could have been worse though.

(Thanks Drusky)

Server hard disk crashed, so the site was offline for awhile.

Had lots of fun recovering the disk. Did a bit-for-bit image of the old disk to the new one, and the problems followed along. sigh

So we repartitioned from scratch without using LVM (Which I suspect was causing the problem the whole time),  and manually copied everything over.  Thankfully that worked without a hitch.

I then decided, in my genius, to attempt an OS upgrade.  That went more-or-less smoothly; the few issues I had were resolved rather quickly.  Since the new OS upgraded the PHP and MySQL to version 5, I had a few issues with my sites.  This one had a problem with one of the text formatting plugins (I’m tired, so the name escapes me right now), so I had to disable it.  If any of the entries look funky, that’s why.  Hopefully there’s a bug fix for the plugin soon, cuz I kind of like it.

The other (more serious) issue I had was transferring my vbulletin forum from its backup host to this one.  I didn’t think there would be much of an issue, since the backup host is running PHP & MySQL 5.  Turns out that Murphy has a sense of humor.

There’s an obscure bug with one of the queries that vbulletin runs when it opens a forum for viewing that causes MySQL 5.0.0.21 (or whatever) to crash.  Fun fun.  Thankfully the guys at vbulletin.com have a temporary code fix to sort out the issue until a new version of MySQL comes out.  It took me a frikkin’ age to get the patch applied (had to do it manually unfortunately), cuz I was tired and I just could not grasp what I was doing.  The code block that I had to change was about 20 lines or so, but only about 3 lines actually changed.  I must’ve tried the patch about 5 times before I realised a missing brace was stuffing me around the whole time.  Arrrgh!  It was really annoying, because you go through a stage (after about the 3rd attempt) that you start thinking the patch just doesn’t work in your environment.   Thankfully that was not the case.

In other news, updating on the Sleep Apnoea bit; I got a call from the neurophysiologist, and she told me that according to the sleep lab tests, I test positive for Obstructive Sleep Apnoea.  Joy.

I get to go do another sleep lab on the 7th where they’re going to test CPAP on me.

Freshly installed server. No relation whatsoever to the old machine.

I lie – the only file I copied over was the apache virtual hosts config, and the websites themselves – and the databases.

Suddenly the file system changes to read only. – I pick this up because I start getting mysql errors when trying to access my other site – and I recognise this problem, because it happened once before on the old box, shortly before we took it down.

I managed to reboot the box, and all is well. I decide to check if there are any updates… so I run yum check-update… and yum fails with an import error. Module ‘yum’ cannot be found. ????

I find some obscure references in Google, but no definite fix, but I get some pointers, so I start verifying rpms. While I’m about it, I log into my local test box, and run the same verifies so I know what to expect. — The odd thing is that the package versions for yum and its dependants are exactly the same on both boxes. One works, the other doesn’t.

Yum verifies.

Python returns an md5 checksum error on one of the files that my local box doesn’t.

I verify the python package a couple of times, and it fails each time. Yum still returns the same error.

2 minutes later, I run the verify again (Ok ok, I have OCD), but this time it passes!????

Try yum, and it works.

Now I’m scratching my head.

Wow, it’s been an interesting week or so.

You may not realise it, but the site is now running on new hardware. The old box kept randomly crapping out, especially with all the new traffic from my other site.

I spent a week (or slightly less than), setting up the new box on Fedora Core 4, and coming up with a compatible (but better) email solution. I wasn’t worried about the sites at all.. I mean, damn, they’re easy enough to do.

My brother had the machine sitting on his home network behind a NAT (Network Address Translation), with only SSH access. He couldn’t get port 80 to forward for some reason, so I had to make do. Btw – elinks is a VERY cool text-based (console) browser – it rawks, because it understood my mouse clicks in my PuTTY session! Cool beans! It really sucked using phpmyadmin via text based browser tho, so I resorted to command line mysql client more often than not (I’m a lazy bum, I like a pretty face on things, but I’m definitely not scared of commandline).

I got most things working, but it was very difficult to test things like multiple domains when you’re sitting on a NATted host, so I had to hope for the best when the box was put onto the live network.

Fortunately for me, most of the things I did worked ‘out of the box’ so to speak, however there were a few issues remaining with the email system which I pain-stakingly sorted out.

While doing this I was constantly getting disconnected, and as you can imagine, my frustrations grew monumental. Eventually I completely lost connection to the box.

After quite a bit of downtime, it turns out that a newly installed switch was causing all my problems. Now that it has been replaced, the box is performing beautifully, and no more disconnects! Awesome!

I friggin’ hate upgrading the blog. Damn be-hatch takes awhile. Considering I have to ‘diff -br’ the live site with the new files to see what I need to replace. It’s not a straight forward process, unfortunately, since I have all sorts of anti-spam crap hooked into some of the files — Which seems to be pretty effective, by the way.

I’ve installed the new Spam Karma 2 beta for spam handling – It seems to be quite different to the old version. — Hopefully it works as well.

But… as usual, there are problems. Seems the server is missing curl, which is needed by SK2 for some mumbo jumbo… and it’s been a bitch installing it. It never rains. It pours.

Typically, there is no php-curl package in th main OS install repository. Weird how there’s source for it in the php-devel package, now isn’t it. At least some bright spark thought to add it to the contrib repo. Had the same problem with php-xml come to think of it. Another reason why I’ve tossed this friggin’ distro.

I’m a complete dolt. Yes, I had upgraded to the bleeding edge version, but it wasn’t right on the edge. It was more like 6 feet back from the edge, you know, where you can see the ocean far below, but you’re too chicken to look straight down. It helps if I read the list properly. Instead of taking the file from the bottom of the listing where the newest stuff was, I took it from the top. D’oh. Oh well, it was late. That’s what my excuse is anyway.

At any rate, the backend software is now running bleeding edge, razor sharp, most-recent-nightly-build of the software.

You may be wondering (but most likely you aren’t), “why?”

Because I was bored, allright? I felt like doing it. So I did. So there.

Bleh.

The site decided to go all funky and not display the style sheets. Ahh well. It sorted itself out somehow, and I also took the opportunity to upgrade the backend, not that I really notice the difference.

shrug