It could be leaking from your home right now. Escaping through cracks around your windows you can't even see. Your temperature-controlled air is like tiny dollars disappearing from your home and your wallet. Ok so that's a bit dramatic, but really your home could be losing some serious efficiency through walls and windows. The best way to check? A thermographic inspection.
Thermographic inspections are performed by reading the surface temperature of objects, in this case the walls of your house, with an infrared scanner. There are different kinds from spot radiometers (which measures one spot at a time) to thermal line scanners (radiant temperature viewed on a line over a thermographic image) and up to thermal imaging cameras (the most accurate, a live video thermogram).
Looking through the viewing screen on these cameras you can see areas of warmth measured in color, from white for hot to black for cold. What you're looking is what the U.S. Department of Energy (DOE) calls thermal defects and air leakages so places where you need more insulation or to patch holes. These will often stand out against the color you're trying to keep in (so if it's winter you probably want to look for darker spots where the cool air is coming in, for example).
You can buy thermographic scanners on the web or at some home improvement stores, but given the cost for a good one starts at around $1200, you may want to rent. Fluke and Flir are big players. If you're not comfortable doing it or would like more professional results there are thermographic auditing companies that will scan your home and provide you with a report. These vary by area so, as always, check with some of their (hopefully happy) customers. The DOE often suggests folding this in with the normal home inspection when purchasing a house and stipulating in the contract that a thermographic scan is required, even if your home is new.
To get accurate results from the scan you'll need to prepare you house a bit. Move furniture away from the walls and remove drapes from the windows. To get the most out of your scan you'll need a large temperature difference between inside and outside. If you can, wait till winter if you live in the north and summer in the south. Happy hunting!
Have you had or plan to have your house thermographically audited?