Project Remyth

[directhex]

It's been over two years since I began work on Project Bloaty, a Media Center PC running MythTV on Debian Linux. It's been educational. And now, it's being sold.

Bloaty served its purpose relatively well, but needs change, and I no longer find myself wanting a big computer in the living room, no matter how "silent" it may be. Part of the motivation here is that my old file server died, and I was unable to justify spending lots of money on a replacement - two computers on 24/7? No thanks.

So what's the plan? Well, the plan is to make use of MythTV's client-server design, and build a new Überserver - files, MythTV, and anything else I can throw at it. I can then drop cheapish "frontend" machines around the house to do the actual watching - or use the MythTV frontend software on existing Linux systems like my desktop and laptop - or use the web-based interface and VLC under Windows like the missus does - or tinker with the uPnP support in Myth which is meant to make it interoperate with a number of Windows-based Media Center apps like BeyondTV or PowerCinema.

First, some hardware specs:

• Coolermaster Stacker 810
• Tagan TG500-U25
• Asus P5WDG2-WS
• Intel Pentium-D 830
• 2x Crucial 512MiB PC4300 DDR2 ECC
• 8x Samsung Spinpoint 250GB P120 SATA2 SP2504C
• 2x Hauppauge WinTV Nova-T 90003
• TechnoTrend TT-budget T-1500 w/ CI Daughterboard
• LSI SATA MegaRAID 300-8X PCI-X
• Optiarc AD-5170 18x DVD±RW
• XFX Geforce 7300LE PCIe 64MiB

Note the list contains 3 TV cards (one of which can record Top-Up TV), an industrial grade hardware RAID controller, and ECC RAM. Some serious shizzle going on here. The reason for some slightly ill-chosen components (e.g the Pentium D) is the length of time over which some components were bought - the P5WDG2-WS doesn't take Core chips, and can't be updated to support them - and considering what the board costs, I'm not replacing it.

Onto the fairly beastly system is going Ubuntu Linux 6.06, plus some of my own "backports" (recent software made available for an older distribution). I prefer to do things this way as the hardware is well supported, and the core apps less prone to "interesting" issues than the less-well supported later releases - but I can still install bleeding edge MythTV without worry.

[nichomach]

Sounds like a great system - and while the P-D is obviously not BRILLIANT, it's certainly an adequate performer. I built a couple of machines around P-D 930s fairly recently, because we wanted to use MSI Crystal all-in-one barebones chassis. The thing is, they're also cheap at the mo' so you can get quite a lot of bang for not much buck. Also, the P4/P-D was always a reasonable media encoder.

[directhex]

The hardware has been "up and running" for 2 days now. The disks are arranged into a RAID5 array, totaling 1630GiB. Of this, a 500MiB EXT3 partition is dedicated to "/boot" (which contains kernels and the GRUB boot loader), 1.2GiB to swap space (unlikely to be needed unless the machine wants to suspend), and the remaining 1.6TiB is a JFS partition for the root "/" drive, which will contain the core OS and apps, plus all data.

The reason for /boot in this scenario (rather than one big partition) is that GRUB dislikes booting from "interesting" file systems, but EXT3 is poor for large files, and ReiserFS is dead - by having a small /boot, the problem is solved.

So, how'd the hardware build go? PERFECTLY! I made a single schoolboy error and put the RAM into single-channel mode as a result, but overall the kit booted fine first try. The stacker is a GIT to work with (those drive bezels do NOT like to just pop into place), but all the kit just about fits. I also had to move the power buttons etc from the top of the case to the bottom, to avoid stretched wires - since the buttons come in a 5.25" bay in the Stacker, the move was simple enough.

Pics this evening, of course.

[nichomach]

Congrats, glad to hear it's gone so well. I have a stacker as my main workstation here, and the space is very useful.

[Steve B]

stacker is my workhorse too
cant wait to see the pictures of this badboy
did you use 2 4-in-3 modules?

[directhex]

So, now the software. I've installed the desktop (not server) version of Ubuntu for assorted reasons, which gives a fairly nice clean slate to work with.

Step 1: Time & Sharing. Under System/Administration/Time and Date, ticking the "Keep Synchronised" button prompts the system to install time sync support (an "NTP" client). This is a good thing when one is trying to schedule TV shows. Under System/Administration/Shared Folders, when you first go in it will prompt you to install support for SMB (Windows file sharing) and/or NFS (UNIX file sharing). Both is good.



Step 2: Get the monitor back upstairs where it belongs. Using your package manager of choice, install the "ssh" package (e.g. "sudo aptitude install ssh"). This allows remote logins to the machine (the server version includes this by default, the desktop doesn't). This allows you to go and sit somewhere comfy instead of on the floor.

Step 3: Fix your package lists. Connect remotely using an SSH client, and sudo nano /etc/apt/sources.list. It should read as follows:

deb http://gb.archive.ubuntu.com/ubuntu/ dapper-updates main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ dapper-updates main restricted universe multiverse

deb http://archive.ubuntu.com/ubuntu/ dapper universe multiverse main restricted
deb-src http://gb.archive.ubuntu.com/ubuntu/ dapper universe multiverse main restricted

deb http://gb.archive.ubuntu.com/ubuntu/ dapper-backports main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ dapper-backports main restricted universe multiverse

deb http://security.ubuntu.com/ubuntu dapper-security main restricted universe multiverse
deb-src http://security.ubuntu.com/ubuntu dapper-security main restricted universe multiverse

deb http://directhex.mfgames.com/ ./
deb-src http://directhex.mfgames.com/ ./

Update your packages list with your favourite package manager (e.g "sudo aptitude update"), then run an upgrade ("sudo aptitude dist-upgrade"). At this point it'll probably ask to reboot, so do so ("sudo reboot")

[directhex]

did you use 2 4-in-3 modules?

yes

[directhex]

Step 4: Contribute to global piracy. Using your package manager, "sudo aptitude install mysql-server-5.0 php5-mysql; sudo aptitude install torrentflux". This will install and configure the wonderful TorrentFlux web-based torrent client, which allows you to easily download torrents on the server - and even includes integrated searching with most of the major sources. Once it's installed, visit "http://machinehostname/torrentflux" with a web browser (Linux desktops will need to install the "winbind" package and add "wins" to the "hosts" line of "/etc/nsswitch.conf" before you can contact machines purely by hostname). This will launch the TorrentFlux login page. The first time you log in, it'll store those details as the admin user - you can add more users later by re-using the same username & password. Tinker with the settings as you see fit.



Your first useful service, configured and working!

[Steve B]

thats the best step yet

[directhex]

Step 5: Set a static IP. DHCP is fantastic. I love DHCP. But there are two issues - 1) The DHCP servers built into consumer routers are forgetful (Ever had to change all your port forwarding settings whenever the router gets turned off? Me too) 2) In order to replace the router's DHCP server, first we need a working static IP. Log into the server, and open the /etc/network/interfaces file for editing. It should look something like this:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
     address 192.168.1.2
     netmask 255.255.255.0
     network 192.168.1.0
     broadcast 192.168.1.255
     gateway 192.168.1.1
     dns-nameservers 192.168.1.1 212.135.1.36 195.40.1.36

This assumes 1) Your router uses 192.168.1.x addresses 2) There's no chance of conflicts, e.g. your router starts counting at 64 instead of 2 3) Your DNS servers happen to be UKOnline's DNS servers. All of these are true for me. Cross fingers and reboot - You should, after a wait, have the machine come back up with the new IP set. If not, time to take that monitor back downstairs to work out what went wrong. Remember you can now edit your router's port forwarding settings and TorrentFlux so they're on the same wavelength.

[directhex]

step 6 involves something i've never done before. it's new and exciting, it is!

[directhex]

Step 6: Remote Desktop, but with Linux! This is a remarkably poorly documented facility, so pay close attention. Everybody knows it's easy to log into a Linux machine with "ssh". Fewer people know that it's possible to run GUI applications from a remote computer, as long as you have an X11 server running (such as Cygwin-X):



Even fewer people know that it's possible to get the same behavior from Linux as Windows - i.e. to have a complete, functional, desktop on the remote system (without involving SSH or Cygwin). Here's how!

First, you need to enable a feature of the login screen called XDMCP. XDMCP allows an arbitrary X server A to view a login screen provided by an X11 login server potentially running on a different computer - the problem is it's not very secure, but not to worry. We need to enable it. Ubuntu uses GDM as its login daemon, so open the file /etc/gdm/gdm.conf-custom as root ("sudo nano /etc/gdm/gdm.conf-custom"). It should have a bunch of section headings - ensure that the "daemon" and "xdmcp" headings look as follows:

[daemon]
RemoteGreeter=/usr/lib/gdm/gdmgreeter

[xdmcp]
Enable=true
HonorIndirect=false

This will enable XDMCP. You need to restart GDM for it to take effect ("sudo /etc/init.d/gdm restart")

So now, we can make an X server on a machine access the remote login screen - great, but not ideal, as it still demands use of an X server. However, what we do now is install and configure a VNC server - users connect to the VNC server, which spawns a new virtual X server on the server machine, connects to the login screen on the server machine, and sends back everything from then on to the client machine - effectively giving you a remote desktop.

So. How do we do it? We need to combine two types of server - an "inetd super-server" (which starts programs when connections are made on a given port number, in this case the OpenBSD Internet Superserver, and RealVNC 4.x (which handles creating a virtual X server, and sending it to a client). These are in the packages "openbsd-inetd" and "vnc4server", so install them. ("sudo aptitude install openbsd-inetd vnc4server").

Three config files need changing. Firstly, we edit /etc/services. This file details port numbers, and nicknames for the services that use them (for example, port 22 has the nickname "ssh"). We need to add an entry here, for inetd to handle the port->program mapping. VNC port numbers count from 5900 (a VNC connection to machine:0 uses port 5900, machine:55 uses 5955) - and one port is needed for each resolution you want to be able to serve over VNC. I'm only interested in 1024x768x24, so only one entry for me:

myvnc	5900/tcp

Next, we need to configure inetd what program to run when port 5900 comes a'knockin'. This is specified in /etc/inetd.conf:

myvnc	stream	tcp	nowait	nobody	/usr/bin/Xvnc	Xvnc -inetd -query localhost -once -depth 24 -geometry 1024x768 -AlwaysShared -SecurityTypes=None

Don't be alarmed by the Security bit - it means VNC won't ask you for a connection password (you still need a valid username/password to login). Finally, we edit the RealVNC config file, /etc/vnc.conf. We need to tell it where the config file is for the current working X server (i.e. XOrg), so it can extract vital info like font paths. Add the line:

$XFConfigPath = "/etc/X11/xorg.conf";

Finally, restart inetd ("sudo /etc/init.d/openbsd-inetd restart").

Tada:



Multi-user, and works with assorted Windows VNC clients too (tested with TightVNC)

[aidanjt]

That's good going Hexy, the hardware looks sweet, even the Pentium D will easily handle all you need to throw at it.

How stable is JFS?.. I've had some nightmareish experiences with XFS, and ReiserFS likes to piss it's pants with files bigger than a gigabyte. And was the dhcpd.conf info I gave you useful?

[directhex]

That's good going Hexy, the hardware looks sweet, even the Pentium D will easily handle all you need to throw at it.

How stable is JFS?.. I've had some nightmareish experiences with XFS, and ReiserFS likes to piss it's pants with files bigger than a gigabyte. And was the dhcpd.conf info I gave you useful?

Been tinkering with Samba this evening. And resisting playing Sam & Max 4 or the new Linux Defcon beta. For I am strong. dhcpd will come in time.

JFS and XFS both belong to a big-iron class of filesystem which are very robust against kernel issues - but by design not robust against power failure. The long term plan involves an UPS, so this doesn't fuss me too much - and both XFS and JFS are infinitely superior to the common choices for large files

[kidzer]

Ooh, i'm bookmarking this guide for the summer, building a new machine and a SFF box that I was going to use as a MythTV box, I shall definetley do that now!

Top stuff!

Having a read through project bloaty atm aswell

[directhex]

If anyone's wondering why there's been a long silence on here, I was having a major problem I wanted to solve before posting.

The problem was MASSIVE processor load during large file transfers, rendering the box unusable.

Whilst I haven't fixed the problem of high load, I have fixed it enough to keep the system mostly responsive, which is enough for me.

The key, for the record, was installing a server-optimized kernel (linux-amd64-server) instead of a desktop kernel (linux-amd64-generic) - this is configured to use an IO scheduler which deals better with concurrent serveresque jobs.

I'll now be picking up this log again, as well as working on something exciting alongside