My name is Earl Chris…

Just as I was showing off the stability of my server to Aram (his host’s server keeps falling over), what should happen? You guessed it, my server crashes.

It looks as though the kernel panicked for some reason, although I can’t really tell why since nothing was written to the logs. It then proceeded to reboot, but for some reason the bootloader, grub, decided to break just at that point. Not being able to get the machine to boot from anything but the hard drives (both with the broken grub), I had to call in some helping hands from 49pence. At roughly 00:30 on a Saturday morning.

After waiting for about an hour for a call from them, I went to bed, and could hardly sleep. I take this all too seriously… Anyway, I gave ’em another call this morning and quickly jumped in the shower. This time the call came in while I was still in the shower. I jumped out and got the ball rolling again. I got them to turn on serial console redirection (so I could work on the box remotely), and pop in a Debian install CD. Using that, and some cunning, I managed to roll back to a working version of grub and reboot. Problem solved, about 11.5 hours after the initial crash.

Lessons learned:

  • Do not upgrade anything as critical as a bootloader if it works and has no security holes. Just don’t do it. No matter how much better the new one is.
  • Keep an install CD (or other rescue CD) in the drive to boot off in emergencies. I could have had the machine back up again within 30 minutes.
  • Keep serial console redirection on all the time.
  • Don’t brag.

So, problem solved, and I won’t be needing helping hands again unless hardware breaks. Oh, and I got the helping hands free since they missed my first call somehow.

Prices go up… As do other things.

The price of natural gas has risen by 300% in the last few years. That translates directly to a rise in price of electricity. What does that mean? That means that RedBus, the company that provides space to 49pence (my colo provider), has had to increase its prices. By 122%. From £350/month to £780/month. What that means is the price of running this server has gone up from £29/month to £39/month (34%). Phew, I was expecting much, much more of a rise.

49pence, however, haven’t just raised the price of colocation. No. They decided to sweeten the deal quite a bit. How did they do that? Rise the bandwidth limit from 100GB/month to 2000GB/month. Yes, that’s two terabytes bandwidth allowance per month. Isn’t that just insane? That’s a 1,900% increase.

So, while I was previously slightly worried that I would be placed on a slightly more expensive pay bracket for going quite close to 100GB, I can now rest easy knowing I can blow my current limit through the roof and not get charged extra.

Anyone need a mirror?

Still, I need to figure out where to pluck the extra £120/year from.