Voltages:

Car off: 12.2 V
Cranking: 5 V (wouldn't start, had to bump it)
Just started up no accessories on: 13.6 V
Took it for a drive and came back, at idle with stereo blasting, air on, and lights on: 10 V
Same but no accessories on: 12.6 V

Those 2 bits, don't look good. How old is the battery?

Not sure, came with the previous owner. It's the battery, innit? Not terrible, I'll just go get a lightweight lawnmower battery like jared has, :p

Most auto parts stores will check your alternator and battery for free. It could be both are bad. Also, either one can take out the other. I would charge the battery up over night so that it has a good charge and take it to some place that can check it for you. The battery needs to have a full charge to test it correctly.

Hmm, I need a battery charger, :p

Anyway, we'll see.

It is either not any good or not charging - if you rev the engine with all the accessories on and the voltage drops to 10 volts then your alternator is shot or not putting out enough. Higher voltage should be on the battery terminals when the engine is running. Pull the caps off and check for bubbling and don't get the sulfuric acid spray from the bubbles popping in your eyes or on your cloths. If you find one or two cells showing dry plates then you should be getting a new battery. The dry cells were the probably the good ones and got over charged and the ones that are full may be shorted. Take the old one with you when you go for the new one to save a core charge and another trip to the store. Running the car on the battery with the engine off trying to save gas killed the last bunch of charge discharge cycle of it.

Twas the alternator dying, battery is fine, life is reasonable.