In one sense, I don't believe in trunking stories. In principle, every story has a home somewhere. That's a philosophical statement, not a serious practical opinion, but I thought I'd start with that. Feel free to disagree. At the same time, some stories are trunkable for one of several reasons.
First, a story might get trunked if it is too narrowly-oriented. If I wrote a story for Strange New Worlds (ignoring the qualification issues), and it didn't place in the contest, that's it. Trunk. There are no other legal, paying markets for Star Trek fiction. That's a bit of an extreme case, but it is possible to exhaust all reasonable markets for a certain kind of story. There aren't all that many places that publish short high fantasy, what Gardner Dozois has in the past referred to as "elfie-welfie stuff," for example. This isn't necessarily a story quality issue so much as a theme/(sub)genre issue.
Second, a story might get trunked for topical reasons. Anyone holding a tsunami story in inventory after the Indian Ocean tsunami at the end of last year almost certainly considered trunking the story. Likewise stories about the impending millenium which didn't sell by 1999 or so.
Note that in both of the above cases, the passage of time, change of editors or opening of new markets can cause a story to be marketable again.
Third, a story might get trunked for reasons of quality. I do this sometimes. I'll look at an old story, think, well, this isn't up to the standards I have for myself now, even though I was quite proud of it in 2001 (or whenever). This is dangerous in one sense, because I am a lousy judge of my own work. I have stories I deeply love which have never stirred the slightest feather of interest in an editor. (Any of you ever read my "'Love,' I Told the Tide"?) I have stories which have sold, gotten good reviews, even direct compliments via fanmail or Con chatter, that I wondered about. On the other hand, I can certainly look at the line level prose, what I was doing with all the crafty bits, and see how solid a piece is. From time to time, I'll take a piece that falls in this category and do a blank sheet rewrite, that is to say, write it again from memory without having the old mss in front of me. Afterward I'll compare the two versions to see if there was anything cool from the original that I neglected to steal.
Fourth, a story might get trunked because I simply run out of markets for it. Not in the narrow sense cited above, but at some point when a story's been out 30 or 40 times I've run out of places to send it. I know some folks disagree -- including literary markets, a story could literally go a hundred places over the course of years -- but I have a very informal rule that after 20 rejections I take a careful look at it. I've never yet trunked a story based solely on the 20 rejection (or 30 or 40) rule, but that step has led me to do a quality trunk.
Any trunk story has potential to be reworked or rewritten or redrafted of course. Even in the trunk, they're not dead. I've made several sales in the past year or so from stories I trunked in the mid-1990s, then eventually came back to.
But for me, there is no simple formula.
So, questions for you guys..
1) Do you trunk stories?
2) If so, what's your trunking algorithm, if any?
3) Do you have reasons other than the ones I've cited?