Hey Wisco Kid... I know the feeling well, I have passed on several small to mid-size bucks over the years only to have someone else take the first shot they get at them. It's a downer to be sure, but it is still the better thing to do. If you shoot him, you KNOW he won't get any bigger. If you let him go, he might.
One time an approximate 100" 8 point, a prime candidate for next year if you ask me, was right underneath me during gun season for several minutes and when he finally went over the hill, BAM! He didn't last 45 seconds past when he left me. BTW, the guy that shot him was tresspassing no less! I nearly jumped out of my stand at the shot AND I was let down that he got whacked right away AND I gave the tresspasser some thoughts to think too.
But all of that said, I can definitely testify that the practice of passing on the little ones will pay off, if not on every buck, but in general. It can take a few years too, but you can usually get others to go along with you too once they can see some fruit. In fact, if I were you I would talk to the guy that shot him about QDM while he is still feeling high about his buck. He might raise his standards in future, you never know.