The word unwittingly seems close, though you may have to qualify it.
I found an example
I think demonstrates this use (see page 140 in the search results).
english.stackexchange.com/users/1787/Andy
How about a new word: corraccidentally
english.stackexchange.com/users/4292/beetstra
I think this question can't have a clear answer because this phrase is quite open.
Think about all the situations where this could be applied.
So going to the root of the problem, I think what this phrase is expressing is that the person isn't aware of all the circumstances (or has the wrong idea) when doing something.
So I agree with @Andy that it might be unwittingly.
Or in my opinion inadvertently would be a closer fit to.
That said, it might also be expressed unintentional, since he/she is doing something wrong, thinking its the right thing.
But all those meaning don't cover the harshest way to use this phrase that would make the "accused" a naif, confused or misguided
(I like this last actually) since you could be meaning: "although you are doing the right thing, there is not much merit to it, since it's almost by chance, you don't have a philosophy behind your actions (or a wrong one)"
english.stackexchange.com/users/1693/Trufa
Serendipitously is the word I would use, although it means something more like "doing the right thing for no reason at all".
english.stackexchange.com/users/4040/Malvolio
The word "Lucky" is frequently used in that situation.
For example, in a case where someone confronted with two doors, decorated with writing in a foreign language, selects the one that is on the right as the "entrance" on the basis that everyone is right handed.
He has made a correct selection, based on incorrect information (that is not why the door is the entrance).
One might say to him: "
Wow, so you just happened to pick the correct choice despite having faulty reasoning?
What a lucky break!"
Although now that I'm thinking about it, the word "Fortuitous" is closer to what I meant.
english.stackexchange.com/users/4300/GWLlosa
Ulteriorly.
The individual doing the "right thing for the wrong reason" did it ulteriorly.
english.stackexchange.com/users/4347/Dustin Ledden
I think this might be a candidate: unassumingly
english.stackexchange.com/users/4291/Bnjmn
An egoist or megalomaniac...is probably someone who would do nice things for someone to build their own rep etc.
thus doing something "right" for the "wrong" reason.
english.stackexchange.com/users/None/
I'd say the person has reached the outcome "fortuitously" or "accidentally" (both mean "by accident or chance")
english.stackexchange.com/users/24728/Alex
Is there a word that means  doing the right thing for the wrong reason  or  getting the right conclusion from the wrong set of presumptions ?
Is there a word that means "doing the right thing for the wrong reason"?
english.stackexchange.com
<single-word-requests>
english.stackexchange.com/questions/10466
english.stackexchange.com/users/4304/platframe
This phrase is quite open and as a result, this question may not have a clear answer. However, some suggestions to describe this concept include the words "unwittingly", "ulteriorly", "lucky" or "unassumingly". There are a number of suggestions which include mainly adverbsbut also adjectives. such as "unwittingly" , "inadvertently" or "unintentional" also "serendiptiously", "lucky", "ulteriorly", "unassumingly", and "accidentally" . A newly coined word was also suggested "coraccidentally"
This phrase is quite open and as a result, this question may not have a clear answer. However, some suggestions to describe this concept include the words "unwittingly", "ulteriorly", "lucky" or "unassumingly". If the meaning is that it was done for ethically wrong reasons, you could use 'ulteriorly'. 'Unwittingly', 'serendipitously' , 'fortuitously' or 'accidentally' could be used when the outcome was unexpectedly good regardless of the wrong method.
This phrase is quite open, so no clear answers, however there are some words which could be used. These words include; unwittingly, unintentional, serendipitously, lucky, ulteriorly or unassumingly. The question might be too open for a precise answer to be possible. However, there appear to be many possible terms to describe such a phenomenon. Some examples include "corraccidentally", "unintentional", or "lucky".
Unwittingly may be the answer. new word:  corraccidentally Serendiptiously Lucky Lucky Fortutituos S0: Ulteriorly. Unassumingly Egoist or megalomaniac Accidently 
If the meaning is that it was done for ethically wrong reasons, you could use 'ulteriorly'. Unwittingly', 'serendipitously' , 'fortuitously' or 'accidentally' could be used when the outcome was unexpectedly good regardless of the wrong method.
The question might be too open to provide a precise answer. There appear to be many possible terms to describe such a phenomenon, but some examples include "corraccidentally", "unintentional", or "lucky".