AT really needs to do follow up questions. It's impossible to tell how herbs are dying without basic information. They shouldn't be dying after a few days. The containers are a little small, but I don't think that would be the cause of their quick demise. My quick advice is to reduce the amount of plants at the window. Four at a window that small probably means there's not enough sunlight to go around.

Questions:

1. How much sun do you get? Some herbs need full sun and some need only partial sun.
2. How much are you watering them? Some herbs need more water than others.
3. What kind of drainage is in those containers and did you change the soil they came in?
4. Have you tried nutrients for the soil?

I struggled with the same problem with my indoor garden. A good friend is a landscape architect and says that the seeds used for potted herbs in grocery stores are not very robust, and you're much better off growing your own herbs from a seed packet. Gorcery stores usually use clippings from the same seed, versus a scattering of lots of seeds.

Plants need light, water and fresh air. If you think your plant isn't getting enough light, you can buy a "plant" light bulb (try OttLite bulbs) for the nearest light fixture.

Most indoor plants die from being overwatered. If leaves turn yellow you are overwatering. Even when the top layer of the soil looks dry, tuck your finger down 1" into the soil and check to see if it's wet underneath. If it's still damp, wait and water the next day. For these little pots, I would think watering twice a week would be about right. Don't soak or flood the plant, just give it a drink. Also, if you forget to water for a few days, do not overcompensate by flooding the plant as that only makes it worse. Water normally and check it the next day or two. Clip off any dead or dying parts.

How cold does it get at night? If it gets below freezing outside, the plants could be getting too cold at night sitting in an exposed window. Also, I agree with Pi, they probably could use more light, preferably a southern facing window that gets sun all day long.

Also you have to clip them regularly, to encourage new growth. Sometimes I'll get a plant that I only need for one or two recipes, and over time the old branches get really dry and yellow. Just remember to chop those off! And don't over-water :)

[Ideally] lots of sunlight and good drainage is key. You could also get a grow light that clamps onto the cabinet.
Overwatering is usually the cause of death for a lot of indoor plants, so make sure they don't remain too wet. It's easier for them to recover from being too dry than too wet (root rot).

How great that you're container growing! Are you using a sprayer from the sink to water them? Sometimes it looks like the plant is getting lots of water but it hardly gets any. Check the soil with your finger to see if it's actually getting wet deep enough to feed the herbs. Know the species you're working with too, and see what each needs.

Light, light, and more light. There are very few herbs that do not need direct sunlight. Even if this is a south facing window, there may be trees or an overhang outside that cut down on the amount of light that is coming through the glass. The glass itself, especially UV glass, also cuts down on the amount of light the plants receive. I am able to grow rosemary inside over the winter in an unobstructed east facing window, but it really limps along until I can get it outside again. Artificial light helps if it is close enough to the leaves (just a few inches away.)
The soil should be loose and fast draining. Most herbs prefer to be on the dry side, so water infrequently, but when you water you need to be sure the water gets to the entire root system. So I disagree with the advice not to flood the plant. Take the pot to the sink and water it enough so water runs all the way out the bottom, and let it drain before putting it back on its saucer. Do not allow any water to sit in the saucer below the plant. Then let the soil dry out to an inch below the surface of the soil before watering again. This could be in a few days; it could be over a week. It depends on the temperature and humidity of the room, the type of soil, and the size of the pot in relation to the root ball.
If the window they are in is very cold and the soil is damp, they might rot, but on the other hand they should not be too close to a heat source.
I hope all this helps!

I used to kill rosemary like it was my job. Turns out it, like a lot of herbs, needs far less water and far better drainage than you might think, and if it gets the right setup it's really hardy. Get some terra cotta pots (they are super cheap) that are NOT glazed but will let the soil breathe, put a thick layer of styrofoam pellets at the bottom of the pot to allow water to drain, and don't let it just sit in a saucer--you've got to elevate it so the water drains away. If you get two saucers for each pot, one the right size (i.e. a little bigger than the bottom of the pot) and one little one that you flip upside down inside the bigger saucer and put the pot on top of, that works well. Water maybe once a week. This worked for me for dryer-climate herbs like rosemary and thyme. Just keep at it and figure out what combination of stuff each pot needs and you should be able to keep them going for at least a couple years. Also know that some herbs are perennial and should last a long time (like rosemary and thyme,) and others, like basil, are annual and have to be restarted every year.

related question: i have a mint plant in my kitchen which acts really weirdly. none of the leaves will grow more than like half an inch long - and then they dry up and die. i've tried watering it more, watering it less; clipping it back pretty far (it grew back and did the same thing); fertilizing it; not fertilizing it. it's in the sun; it's in a pretty big pot. what the heck?

Deens is right - the 'growing' herbs from grocery stores have very little chance of surviving - they've been kept in all sorts of conditions, deliberately over watered to make them bushy and potted in cheap coir compost with no nutrient content.
You really need to get some potted herbs from a proper garden nursery. If you mention they are for indoor growing they should help you choose varieties which are more tolerant, although I think you're going to struggle for light at the particular window shown. Use a compost designed for indoor growing, make sure your containers have drainage and look out for over watering and you should be fine!

hanc - mint is one of those herbs that needs a ton of space. I had potted mint that came back year 2 (I left it outside over the winter as I did with the majority of my container garden), but year 2 it came back growing from around the edges of the pot. Year 3 it did as you're describing, and I did some research - after a couple of years in one pot, it stops performing well. If you change out the soil but put it back in the same pot it may do better, or change the pot and the soil.

I've only ever attempted to grow basil, but the one basil plant I had years ago came from a grocery store and is still thriving.

Totally agree with the flood with water, but in the sink in a well-draining container so all the excess drains off right away. And with using unglazed terra cotta pots. For all my houseplants, I use a bunch of shards from terra cotta pots I've broken over the years in a layer in the bottom of the pot to make sure it drains well. Others use stones. Repot when needed so roots have enough room, unless it is a plant that thrives on being pot-bound.

There are not too many pots on those windows. And since the photo appears to be taken at night, I don't know how anyone can tell how good the light is. Obviously, use the best light you have, even if that is not in the kitchen.

It is good to hear about the quality of seeds to use, as I have thought about doing this, and would like to do so in the large, sunny kitchen window of the place I just moved into.

Ditto the comments on grocery store potted herbs - 9 times out of 10, they won't grow for you, in fact I've noticed that the cut herbs in bags last longer than the potted ones - get your potted herbs from a plant store, preferably a small locally owned one. You can replant them into terra cotta, but you don't have to, likewise you don't have to change the soil when you get them. After a year, if they're still growing, you can repot. Use a "soilless" potting mix, or cactus soil to which you add an equal quantity of perlite. Major requirement for growing is light; a south window is almost a requirement. If plants aren't growing well, or "stretching," add some electric light, but it has to be close to the plants - putting it in the kitchen fixture won't help. Also watering, always feel the moisture content of the soil at least 1/2 way down into the pot, don't add more water until it feels just barely moist, then when you water pour in enough that it's running out the drainage holes - you need to make sure all the roots are moistened. You can leave up to 1/2" water in the drainage saucers, as the plant will use it. You won't overwater if you always test the soil moisture before you water again.

Too much water, not enough drainage have been my problems in the past. I didn't have a garden, I had Death Row. I have now been rehabilitated and repotted with much better drainage. Plants are still alive after 6 months.

You would likely do better with a single box than several pots. I have found that containers that dry out quickly are plant killers. another trick is to water continuously through the bottom by having a reservoir of water and feed a wick up into the root ball. use the correct kind of planting medium for your purpose - folks at a reputable plant place will point you in the right direction, it costs more but is well worth it. good luck

Herbs need LOTS OF LIGHT, especially what looks like basil and parsley that you have there. Only a south facing, sun all day window is going to be enough natural light for them to actually grow especially if you are using up their leaves.. I would suggest getting a small plant bulb and putting it up above the containers.

Agree with natural terra cotta pottery, not glazed, using shards of unglazed or rocks or stones in the bottom (please no toxic styrofoam, you want to eat these herbs!), and LOTS of sun, meaning that most herbs need 6-8 hours of DIRECT SUN every day, and any glass does dilute the light, so you may need to supplement with growth bulbs. Most herbs like "lean" soil (not real rich and damp), and like to be on the dryish side. You may need different types of potting mix for different herbs. Some do better in pots than others. Learn about the requirements of the individual herbs you are growing (parsley can take a little less light and a little more dampness than some other common ones). Tiny leaves usually means not enough light, or possibly not enough nutrition (use food grade compost). Fresh soil every 6-12 months can also be helpful. Once you figure each herb out and provide what it needs, they are usually easy care and fairly long-lived. There's also lots of fun herb lore to learn - they say sage reflects the happiness and good fortune of the woman of the house!

What Emmi said: I agree with everything people said about watering, except I wanted to make sure you understand what Emmi was getting at when she talked about checking the dampness.

Sometimes, if a pot gets dry enough, the soil will actually seem to repel water. It will flow out the bottom and you will think you watered the plant, but if you check below the surface, it might all still be dry, because the water will just flow right down the sides of the container and out the hole. (The dried-out hunk of soil shrinks a bit, allowing this to happen.) If you don't investigate deep down, you won't even know this is happening.

*When* this happens, you need to really soak the whole thing, and get all the dirt really really wet. (In the sink, preferably.) It's enough of a pain to re-wet really dry soil that you maybe might as well just take the plant out and repot it at that time. Like people said, the best place to get advice is a local indie plant store. The secondbest place, imho, is at gardenweb forums, I'm sure there's a forum on container gardening, there's one on houseplants, I'm sure there's one on herbs and maybe even on indoor herb growing, for all I know. These are the most plant obsessed people I've ever found, and it is the niche-iest of niche-making sites. Somebody on there has already had this problem and you can find a ton of info.

On the positive side, once you re-wet the soil, it should behave normally after that, and then, you should follow what everyone here said about not overwatering. It's okay if the soil gets dry -- it only repels the dirt if it gets super, extremely dry.

Just wondering, can herbs be grown in a terrarium? I am also a plant killer and had been thinking that might work.