MP3 players that sought to kill the iPod were in turn killed by the iPod...resulting in thousands of new-in-box casualties that you can benefit from for pennies on the dollar. Slashdot links to an article on MP3Newswire that highlights some of the good models you can benefit from by scoring a deal. (They also tell you what to avoid, because some were short-lived for a reason: they sucked.)
The Virgin Player pictured here uses the same 5GB memory as the iPod Mini, and got great reviews from CNET and PCMag; it was seen available on eBay for a mere $39.
Image from MP3Newswire.