Question: I am handling a paper as an associate editor that proposed an algorithm that I find to be weak. In fact, I was able to show that a very simple, brute-force approach actually has a better running time than their algorithm. Therefore, I will recommend rejecting this paper. Do I have an obligation to share my proof that the brute-force running time is better? I want the higher-level editors to have confidence in the rejection, but it also occurs to me that I might be able to improve my own result and publish it independently. Is this a violation of ethics?

UPDATE: Well this certainly took off! I would like to add the following:

* The overwhelming consensus is that it would be unethical for me to "scoop" the other authors, so I will not do that. The advice from all is greatly appreciated.
* The journal is the top in its field, so we have to be extremely selective. The problem proposed is fairly interesting, but overall the paper does not meet our threshold.
* When I say that brute force is better than their method, I mean it in a provable, big O sense.
* The authors' proposed scheme is not only inefficient, it is written in a very confusing way. In fact, I asked them to compare their approach to brute-force as a way of helping them clarify their argument, and they did a bad job of it, which is what led me to look into it in the first place.
* The fact that brute force performs better than their scheme is not totally trivial because it relies on a combinatorial argument that is not amazing, but not completely obvious either.
* I will share my proof with the Editor in chief, but I have decided not to give it to the authors; I will consider publishing independently in the future if their work ever appears elsewhere.
* Their paper is not on arXiv or any other website.

Answer: 
This is one of the most feared and contentious acts in the publication of science. We all have heard stories of the paper that was rejected, only to serve as impetus for a subsequent publication by one of the rejecting reviewers. The worst case, which you would not be guilty of, is rejecting the paper for the sole purpose of benefiting from the idea. That's not the case here. If the optimality of the algorithm is the *point* of the paper, and you can clearly demonstrate their logic is flawed, you have every right to reject the paper on those grounds. But you *should* lay this out for them in the review, and give them a chance to rebut. If your logic is iron-clad, you'll win the argument.

Whether it is ethically sound to then move forward with your own publication on the topic is unclear. Did the authors present this work at a conference? If so, and had you seen that talk, you would be justified in moving forward. If not, the manuscript is considered confidential, and your idea was spurred only through a confidential review process. It would be wise to discuss this with the general editor of the publication for guidance. Some day, they may review your own paper.
I think you have an obligation to academia to disprove his algorhithm with a brute force algorithm. Give your brute force proof to everybody and put it in the public domain. After you disprove it, if you have an algorithm that's better than his and better than brute force, you could offer to help him write a better algorithm. You and him can both be coauthors of the new paper with your new algorithm. Wait until everybody accepts that his original algorithm has been rejected before offering to coauthor with him the new algorithm. Tell him that you have a new, better algorithm, but don't tell him what it is until he agrees to coauthor with you in a written, signed document. That way, you get credit for helping, but are not taking away his idea. However, remember that some ideas are proposed in odd ways because of a need to avoid similar patents. An inefficient algorithm that is patentable is better than an efficient algorithm that looks like someone else's invention.