Store Your Recordings on a Separate Drive (or drives)

The installation of mythtv recommends storing your recordings on a separate partition from the rest of the operating system. This is done to insulate the recordings files from the key operating system files, and, in general, it is a Best Practice. However, for optimal performance, this should be taken a step further: Set up a separate drive or drives to hold your recordings. This will separate all of the disk interaction between the OS and its related files from mythtv and its recordings. The result will be a smoother end user experience.

An internal SATA drive is probably the best and least expensive solution. An external SATA drive is good, too, but more expensive. A PATA drive would work well, but it is best to keep that drive on a different controller than the drive that your OS is on.

Reduce The Size Of Your Guide

By default, MythTV will collect 14 days of guide data to get information about future shows. This is actually great for searching for shows, and performing other user-related options, but it can wreak havoc on the system performance. A dead give-away is in th mythbackend.log file:

2007-12-23 09:31:54.099 Scheduled 871 items in 12.4 = 0.20 match + 12.17 place

If the scheduled items creeps up too high (typically greater than 700), then your system is working pretty heavily to schedule a lot of shows based upon the 14 days of guide data. This can cause the database to become rather large and slow down all database queries, which, in turn, slows down the end user experience.

A happy medium is reducing the number of days of guide data from 14 to 9.

Optimize the IVTV Paramaters

The default kernel parameters for the IVTV based cards do not take advantage of the buffers on the cards. This means that you will see a lot of logged error messages referring to “application not reading fast enough”. This could result in choppy recorded video which is a Bad Thing(tm).

options ivtv enc_yuv_buffers=8 enc_mpg_buffers=32 enc_vbi_buffers=8 enc_pcm_buffers=8

Know Your Hardware

One of the largest problems is that the underlying nature of MythTV is to take full use of the hardware it has at hand. One cannot help but try to squeeze everything (and more) out of their hardware. However, keep in mind that if you have a single processor frontend/backend system, you don’t want to be recording too many shows at any one time or running too many concurrent commercial flagging processes because the single processor just cannot handle it.

The most optimal set up is to have a fully networked frontend/backend system:

• Have a frontend (client) system for each television in the house. It is probably best to have these as identical systems with minimal hardware requirements since they will only be acting as display systems.
  • NFS mounting of videos, mp3′s, photos, etc.
• Have a backend (server) system stowed away in a closet. This will be the system that you will end up beefing up. A dual-processor system is recommended since it will be performing all of the work (recording, streaming to the frontend processes, commercial flagging, etc).
  • SATA storage for recordings
  • Separate storage for video, mp3s, photos, etc.
• 100MB networking from the backend system to each of the frontend systems.

By having the same frontend systems, if you ever have a failure of a frontend, you can always easily shuffle them around while you repair or replace the failed components.

The same goes for remote controls. Although the temptation is to just reuse the multifunction remote you already have for your TV, cable box, Tivo, etc, it just isn’t worth the hassle. Simplicity in the setup will definitely improve everyone’s WAF (Wife Approval Factor). If each frontend has the same remote (although configured specifically for the equipment attached to that frontend), then, as a user (not the admin), you just have to learn the buttons on the one remote, and it applies to all of the frontends.

Grab yourself an inexpensive multi-function remote like the OneForAll (http://www.amazon.com/gp/product/B0000ALPBA) for each of your frontend systems. I was very hesitant at buying a remote I couldn’t test out since many multi-function remotes disable certain buttons based upon the mode you are in. However, this remote specifically has a PVR mode and you can move button functions from one mode to another.

Know What You Are Displaying To

The K.I.S.S. (Keep It Simple, Stupid!) Principle really helps when maintaining your MythTV systems. By having your frontend system as similar as possible, the level of administration needed on those system drops. In other words, if you tinker with one, you know you just have to make that same change on all of the others. Unfortunately, it is harder to control what types of televisions you are displaying to.

Remember: essentially, your expensive TV screen is just a simple computer monitor at this point. You are not using the internal tuner and you are not using all of the various input (DVD player, Antenna, Cable, etc) connections. You are just using the one input on the TV that is coming out from the computer. It is nice and simple.

IF it is possible, try to stick to the same kind of input, even though you will end up with different TV’s throughout your home. For example, if your TVs all have component input as the best quality input, then make sure that all of your frontend systems can output component video.

Try to avoid using any of the ‘PC’ inputs on TV’s. Component, HDMI, Composite, S-Video, and coaxial are all bound by TV broadcast standards. Once you start fiddling with the PC inputs, you have a lot of extra work to perform in order to get it to properly display the broadcast video on the screen. Stick with the standards, and you will be satisfied.

In my case, all of ours have HDMI connections on them. Some video HDMI as evil, but that’s another topic all together. I got away with purchasing some inexpensive DVI to HDMI cables (10ft, ~$10ea. on-line).

By keeping things similar or the same, the X configuration for each of the frontend systems will be the same.

Optimize the database

The MySQL database behind MythTV takes an enormous beating as shows are recorded, scanned for commercials, watched, bookmarked, and deleted, just to name a few causes. It does not take a very long time for the database to become more ‘fragmented’ and provide slower responses due to the increasing level of ‘fragmentation’. This has a direct impact on the user experience in a very negative way.

The solution to this is to periodically optimize the database. The problem with optimizing the database is that it is intensive and will only make things worse before it makes it better. So, it is much better to run this off-hours.

The easiest solution is to run a wrapper around mythfilldatabase which optimizes the database. But, because mythfilldatabase is run according to the grabber’s suggested schedule, this might end up running in the middle of the day.

What I found as the happy medium between the two was to craft the wrapper such that the database gets optimized only one day a week:

#!/bin/sh
# mythoptdb – Call this in mythfilldatabase’s place in
# mythtv-setup

if [ `date +%a` = "Sun" ]; then
optimize_db.sh
fi

/usr/bin/mythfilldatabase –remove-new-channels –quiet –max-days 9 $*

The “optimize_db.pl” script that comes as part of the contrib from MythTV has one shortcoming: it doesn’t always work properly. It worked for me for a long time, then started hanging on the optimization/repair when my database grew rather large. I ended up replacing it with optimize_db.sh which does the job more efficiently. Still, you must keep in mind that the database will be unavailable during the optimization/repair of all tables. The goal is to keep this (very) short.

Understand the order of your inputs

Setting up MythTV can be very confusing: there are so many options in the mythtv-setup screens that is can be easy to either miss an important option or be unable to find that elusive option later on. Add in to the equation the fact that there are easily just as many (if not more) options in the frontend setup menus, it becomes almost overwhelming.

One item that took a couple of emails to the mailing list to figure out is the ordering of the tuners. When recording, the backend server process is set up to select the first available tuner, and when you go to watch live tv (which I almost never do now-a-days), it also is set up to select the first available tuner. Additionally, if you are watching live TV and mythtv needs to record something, you get the expected warning that it needs the tuner for that recording.

All of that sounds normal, and it is. However, you can change the settings to have mythtv’s backend select the tuners in reverse order so as to avoid conflict

High Definition recordings vs Standard Definition recordings

Standard Definition (SD) recordings, when displayed via MythTV on that nice screen just don’t look as good as that old 4:3 tube television. The problem there is that SD is broadcast as interlaced video, and the computer (the mythtv box) only displays in progressive mode (see this great description of the differences). That, and the quality of SD is so low, many of the artifacts that you didn’t notice previously now show up amplified because of the larger screen size, the better screen quality, and the MPEG2 encoding. However, SD has one advantage: it is small. Typical hour-long shows run about 300-500MB.

High Definition (HD) recordings are an entirely different beast, and they come in many different flavors. Aside from an awesome increase in picture quality, they all have one important factor in common: they are huge. A typical 1 hour recording can run between 3 and 6 gigabytes.

PLAN YOUR STORAGE ACCORDINGLY.

My experience is limited to only Over-The-Air (OTA) HD transmissions from the local affiliate stations, but I definitely recommend the HDHomeRun for an OTA setup. MythTV supports it right out of the box, it is easy to configure, and you can get HD recordings set up within minutes.

Obtaining HD recordings off of your cable/satellite box is an entirely different beast, and, unfortunately, I have not gone down that route just yet.

BE PREPARED TO ADMINISTER THIS SETUP

MythTV is still in a beta phase, and, although there are a lot of people who are using it, it still is not entirely ready for prime-time; the system isn’t made to run entirely on its own and it does need some tlc. I cannot stress this enough: don’t go cheap on your backend system’s motherboard. Make sure that it is a solid board with well supported chips. Some inexpensive, and tempting, motherboards use chipsets that work well as windows systems, but perform with numerous problems under Linux (some Jetway, PCChips, MSI, and some via-based motherboards) which will only give you major headaches.

This is not to say that you couldn’t have a successful system that you don’t need touch for weeks on end, but it is likely that things may creep up that you, and no one else, expected. Besides, you will want to tinker with it over time to fully optimize the system.

A great example is the impact of cpu frequency scaling on a MythTV backend: don’t do it. If you have multiple cards and a couple of drives (OS and video), cpuscaling can wreak havoc on the system: ivtv drivers have been known to stop responding (requiring the backend to be shut down and the drivers unloaded and reloaded) and active/busy SATA interfaces have been known to falsely believe that there are ext3 journal errors which results in the drive being put in to read-only mode.

With all of the above said, you can create an excellent home-wide DVR system using COTS (Common Off The Shelf equipment) that is DRM-free, completely customizable to your liking, and extremely versatile.

This has been put together for my own archive since there is no guarantee that existing How-To’s are going to be available when needed (there was a key point in time when the Wilsonet.com site was down, and I really needed access to the steps).

Background

After fighting with MythTV (posted here), I was determined to just get it right. It became near obsession, and I started driving my wife a little crazy since I spent a couple of hours each night tackling different aspects of MythTV.

However, I must say that it has been worth it. And my wife sees that too, as she sees it a being something that could replace our Directv TiVo.

Also, when I built my system, I attacked it as if it were a “Proof Of Concept”. In other words, I did not get too crazy about the processor type, the filesystem layout, the computer case, or really any of that stuff. I just wanted to build a system that worked. Once I got to the point where everything worked as it should, I figured I would then tackle the other items (like a smaller, nicer, and quieter case). I believe that taking this approach made it easier in that there was a lot less pressure: I didn’t really invest much money in to the project in the beginning, and anything that I did buy I could always re-sell on eBay.
Prereqisits

At the very least, you will need the following:

• A power supply
• A hard drive
• A motherboard
• A processor (CPU)
• A video capture card
• Memory
• Internet Access
• A copy of Fedora Core
• A Computer case capable of holding all of the above equipment.

Components

This is more of an aside, but I thought it was worth mentioning some more details about the individual components.

The computer case I chose was a standard tower case. Yes, it was ugly. The power supply in it was one that I got for free, after rebates, a while back (I bought it as an emergency spare) and it was LOUD. The hard drive was another spare item I had around (a 160GB drive). I had gotten the video card a long while back for very little money, after rebates. I had the motherboard and processor and memory (all pretty cheap after rebates.. I hate rebates!) available as well.

I had most of the stuff because it was being used as a temporary machine for my in-laws during a period of time that they stayed with us.

Anyhow, the point is that I already had most of the stuff. I didn’t want to invest a heck of a lot of money in to building a Myth PVR if the end result was going to be something that I couldn’t sell to my wife. When it all comes down to it, if she didn’t like it, then it was highly unlikely that we would use it. She, in essence, was my customer.

Assembling the System

I pieced together the 400W power supply, the tower computer case, the MachSpeed Viper K8M8MS MicroATX motherboard, the AMD64 Sempron 3000 (Socket 754) processor with stock heatsink and fan, the Ultra DDR400 512MB memory, and the 160GB Maxtor ATA100 Hard drive. I added in an XFX Geforce 6200 fanless video card with S-Video out since I would be testing the install on a standard 27-inch CRT TV.
With it all together, I made sure things powered up and were all properly recognized by the system. Since I know that this will eventually be a system without a mouse or keyboard, I made sure that the BIOS was set to ignore any keyboard related errors.

Installing the OS

Installation of the OS was pretty simple. Although I ended up doing it about 6 times due to testing things with MythTV and breaking things, in truth you should really only have to do it once.

I didn’t bother partitioning the drive. I didn’t see the sense in it. MythTV has it’s own space management logic which removes the worry of it filling the filesystem to full capacity. Besides, I figured that, down the road, I would have all of the recordings on a central server rather than local to any one machine.

I did make sure that I selected:

• Servers -> Windows File Server
• Servers -> Network Servers (vnc was selected by default)
• Servers -> Web Server
• Servers -> MySQL Database
• Development -> Development Libraries (recommended!)
• Development -> Development Tools (recommended)
• Applications -> Sound and Video
• Applications -> Graphical Internet

The Wilsonet.com how-to lists KDE as a requirement, and I installed it, but I ran everything under Gnome without any problems (KDE is a bit of a resource hog, IMHO.. not that Gnome is much better).

Keep in mind that this was a “proof of concept” machine, so I wasn’t too concerned about optimizations.

Right after the installation of the OS, when “firstboot” came up, I configured up a “mythtv” user, pointed it to my NTP server, and configured authentication to also go through my LDAP server.

Immediately after updating, I logged in as “root” and ran “yum -y update” to get all of the latest updates to the distribution. This took nearly as long as it did to install the base os!

Install Myth

The best thing to do is to go to Wilsonet.com and follow the instructions there. Aside from downloading all of the updates from the ATrpms site, there are a number of other little gotchas that are covered on that page.

Not sure of what I mean? Well, the idea is relatively simple:

1. Set up one big, fat, noisy machine as your BackEnd. Allow for plenty of expandibility because, in the future, who know what you might want to have MythTV do. Also, because it is a BackEnd (BE), you don’t care too much about how it looks or how noisy the drives and fans are because this system will be stuffed in a corner or in a closet for no one to see or hear. (Be sure it is a place with adequate cooling, though!)
2. Set up any number of sleek, pretty, and quiet machines throughout your house (with network access, of course) as you FrontEnd (FE) systems connected to some displays.

Now, nearly everyone who is playing around with their own Tivo-like homebrew setup is probably very familiar with the first machine (the BE). But, the other machine(s) (the FE) are a whole different ball game. Simple things can get expensive here, and the more silent you wish to have your FE, the more expensive it will get. People can go nuts over this (take a look at SilentPCReview.com), and for good reason: you don’t want a noisy machine ruining your viewing experience.

The real question that you have to ask yourself is: How quiet is quiet enough? And remember, Quiet is not equal to Silent. Silent is equal to very expensive. Quiet is equal to not as expensive. I can only go by my tastes, but I tried out a couple of different cases before I got to one that I felt was acceptable.

First, I bought the Antec Overture II case. It’s a nice looking case with a sharp Piano Black finish and is very quiet. However, right after I got it, I realized that it is just plain HUGE. It’s like taking a full-sized ATX case and laying it on its side. The airflow in the unit is awesome and it has bays for everything. Unfortunately, that is not what I wanted. I wanted a small system that would be, well, for all intents and purposes, nearly bare-bones. I mean, everything was going to be on the server, so I didn’t need something capable of holding terabytes of data at the frontend.

After selling that on eBay, I bought a smaller case off of TigerDirect.com. I picked up
an Ultra MicroFly case with a supposedly quiet 400Watt power supply. Now, when it comes to bargain stuff, I usually take the words “quiet” and “near silent” to be worthless marketing buzzword bingo. And, yes, this goes for the MicroFly. “Quiet” is a relative term, and, in MicroFly, “Quiet” can be defined as: Being less noisy than a tower system.

Being dissatisfied with this purchase, I sold it on ebay and tried out a Shuttle SK21G from Newegg.com. No, I wasn’t all that happy that the case was silver. I’m a “black-entertainment-center-equipment” type of person, but, I was somewhat tied to the AMD64 754 socket since I had two processors of that design laying around. I also wasn’t all that thrilled with getting a custom case with a custom motherboard, but I figured that if I didn’t like it, I could re-sell it on ebay, too.

However, after the quick assembly, the SilenX + ICE setup that the Shuttle has, the system is very quiet and cool.  The assembly process really isn’t much since the motherboard is already in place, but you still have to be careful and take your time.  I realized that the cabling was already run for 1 SATA device (a hard drive) and 1 IDE device (an optical drive).  Since I only had IDE drives, I was forced to use the IDE cable and not install an optical drive.  I figure that I won’t be using the system as a DVD player just yet anyhow.

With this case, there is no processor fan.   This case comes with what they call SilentX and ICE, which is really a water cooled system for the processor.  With that in mind, you have to figure that will impact the price of the system, but, when you compare it with comparable systems, you’re really not paying all that much more.  Because of the water cooled system, there is a large radiator at the back of the case (inside) which has a smart (variable speed) fan strategically placed to cool it off.

My main fear with a Shuttle system was that they would run much hotter than a normal system.  I quickly discovered that this was not the case since the processor ran around 37°C and the case fan speed hung around 950RPM.

The system’s physical size and overall looks made it ideal for my purposes.  It fits right in to most entertainment centers and can be easily hidden away.  It doesn’t pull much power, especially since I am using AMD’s Cool-n-Quiet technology with Fedora Core 64, which happily slows the clock speed down and speeds it back up on demand.  This translates in to the system running cooler, so the case fan runs slower, and, hence, quieter.  That coupled with the liquid cooling really makes this system comfortably quiet.

I’m satisfied with this case as a good choice for me.  It wasn’t until after I set the case up that I discovered that Shuttle has an ebay store where they sell refurbished units.  The cost of a refurbished unit was a lot lower than NewEgg‘s price on a new unit, and, since I don’t mind having a factory refurbished system, I bought my other two cases from them.

Keep in mind that the noise level of any given case is also directly dependent upon the other items in the case: hard drive, optical drive (not so much since it isn’t constantly running), and video card.  Because of the application I am using these systems for, I had to use nVidia cards, and I noticed the sound difference between my PNY 6200 with the bun-fan on it versus the fanless XFX 6200 I have.  I also noticed an immediate difference between an older hard drive that I had versus a new Hitachi DeskStar drive I just picked up (the Hitachi is dead quiet).

From the site http://www.pdp8.net/directv/directv.shtml, I pulled down the following SetTopBox script script, and it worked beautifully.

However, one thing didn’t work with the standard “./directv.pl setup_channel” command: the on screen display would stay up (yuck). The directv.pl script is supposed to do all of the necessary things when you use the setup_channel option, but it didn’t seem to for me. Since MythTV executes the command with the last argument as the channel number, I threw together a simple wrapper script (Change Channel) for MythTV to use.

Finally, since the directv.pl script works so well, I wrote a quick init script to turn on and off the set top box when the system is turned on or off. It works rather well, and, really, we don’t need the set top box on when the Myth system is off, do we?

I’ve been watching the whole MythTV arena for a while now.. watching it in the aspect of planning my attack (on getting a system up and running). I decided at the beginning of the week to give it all a try.

I have an AMD64 system (memory, motherboard, case, processor, hard drive, DVDROM drive) sitting idle, so I took it, loaded up FC5_x64 on it, and began following the Fedora Myth(TV)ology How-To page. Everything went rather smoothly, actually, and all was good in the world. Of course, the key component I was missing was a TV-tuner/capture card.

During the MythTV setup, the system would seemingly hang on me. Not all of the time, but most of the time. I would lose all cursor and mouse capabilities, and it just seemed completely locked. A hard reboot of the system brought things back.

On buy.com, I picked up the Hauppauge PVR-150 since that is one of the most popular cards to use under Linux. I bought it on a Monday at around 4pm, and got the thing on Tuesday around 6pm! I popped the card in, and started testing it. It didn’t take me long before the card was working, and, again, all was good.

I started tackling the MythTV setup again, and the lock-ups were more frequent. On a whim, I tried logging in to the system with my laptop, and, to my surprise, I was able to log in. It looked as if X was hung as it was taking up 99% of the cpu. Killing it off brought things back.

There was a brief moment of about 20 minutes where I had everything but the remote working. I uttered a big “woo-hoo!” But, as I continued testing, I started getting more and more lock-ups. I ran another yum update, and, to my surprise, Myth 0.20 had been released. Everything updated happily.

From there on out, it was all down hill. The problems with X locking up persisted, and, after some research, it looks like it is a problem with the linux version of the nvidia display driver and X. Ugh.

So, I decided to reload the machine from scratch. That’s been a nightmare unto itself since the FC5_x64 updates are taking FOREVER to download (the downside of YUM is that it doesn’t pick a mirror local to you.. I was getting mirrors in New Zeland, Japan, Germany.). I abandoned that, and decided to populate my own rsync’ed copy of the updates tree before going back to the install of this machine.

Additionally, I decided to try out the FC5 i386 tree vs the FC5 x64 tree since some of what I have read suggests that there might be problems related to the x64 bit versions of things. Eh.. we’ll see.