There are a few caveats, but not many:
It appears that you have to purchase a computer from Apple to be able to have them recycle your old one. It doesn't say anything about MP3 players besides iPods are accepted, but both cell phones and iPods can be sent to Apple at any time. After you enter info on-line, you're given the choice of a pre-paid label with your own shipping materials (your only choice for a CPU recycle), or a pre-paid mailer sent to you.
-Thanks to TeamSugar for tipping us off!