Question: Can I use this epistemic expression: “He ***won’t have gone*** there yesterday”? It may not be ungrammatical, I think. But I have two questions. First, is this ‘*won’t have p.p.*’ used casually or freely to say that something might have happened in the past? Second, If it were present perfect, adverbs which denote the past would not be used. Then Is it, *yesterday*, okay in the example above?

Answer: 
English does use what's sometimes called the "suppositional future," which is what your example is called. There is nothing wrong with the sentence, but it tends to be used in a rather restricted context.

The simple suppositional future is equivalent to the present tense plus words like "must". We use it very rarely these days. "You boys will be wanting some supper!" Which means "You boys must want some supper."

The suppositional future perfect (which you have) is a bit more common. It equates to the present perfect tense, but with uncertainty. So "He will have reached the boat by now" means "he must have reached the boat by now" or "I'm sure he's reached the boat by now." In question form, it's much the same as adding "I wonder". "Will he have reached the boat by now?" means "I wonder if he's reached the boat by now?"

Using this form adds a good bit of urgency to the statement. It sounds like you're very anxious to know what's happening. It's something to use sparingly, if at all, but you do want to understand it if you see it in print.

French, Spanish, and Italian all use the suppositional future much, much more than English does. Students of English who are native speakers of those languages should be careful not to assume that English is the same.
I've been in the US for 7 years (in the metropolitan area around Washington DC) and haven't heard the FUTURE perfect tense (will have + p.p) used once to indicate something in the PAST.

In this case I would simply ask "Didn't he go there yesterday?"
In an informal context, you may ask "Hasn't he gone there, like yesterday?" This is a short form for "Hasn't he gone there? I thought he went there yesterday."