You get to rule out some of the most expensive components with your build: SSD and graphics card!
Your motherboard doesn't have to cost an arm and a leg, because you don't need a bunch of PCI Express slots or anything.. you don't need silicone sealed, full-size etc., just a standard mobo that suits your needs.
If you're going for raw computing, look at a middle-of-the-line i7 (for economical purposes) such as this:
Intel I7 3770k
As far as a hard drive, you could get away with an SSD Hybrid Drive such as this:
SSD Hybrid Drive - 1TB
Then, run as much RAM as your budget will allow (I'd do 16gb):
Corsair 16gb DDR3 Ram
As far as processor socket, be sure that you give yourself something that is at least somewhat flexible. The above socket is decent, but you can probably find a better socket (Such as LGA2011 or so) that will have more flexibility.
You could STILL cut cost by going with a less expensive RAM kit, and with those components, you're under $500. A decent GPU, case, and cooling system (just need good fans) will be less than $300. Boom. $800 and you have a decent build (sans OS).
To be honest, I ruled out using a PC as a media center anymore. We just use a NAS drive connected to my router, download media to it, and stream it using a gaming console (Xbox 360/PS3). Works stupid good, and a 3TB NAS drive was like $200. Much cheaper than a PC.