Question: Can I use this epistemic expression: “He ***won’t have gone*** there yesterday”? It may not be ungrammatical, I think. But I have two questions. First, is this ‘*won’t have p.p.*’ used casually or freely to say that something might have happened in the past? Second, If it were present perfect, adverbs which denote the past would not be used. Then Is it, *yesterday*, okay in the example above?

Answer: 
This has nothing whatsoever to do with the future.

*He won't have gone there yesterday* is a modalized version of *He went there yesterday*, with the past time meaning carried by the perfect auxiliary *have*. The modality expressed by *won't* is a slight epistemic weakening. It would typically be taken as indicating that the speaker does not know this for a fact, and is merely stating confident speculation.

This expression could be used when contrasting with another time that they did go or will go.
I've been in the US for 7 years (in the metropolitan area around Washington DC) and haven't heard the FUTURE perfect tense (will have + p.p) used once to indicate something in the PAST.

In this case I would simply ask "Didn't he go there yesterday?"
In an informal context, you may ask "Hasn't he gone there, like yesterday?" This is a short form for "Hasn't he gone there? I thought he went there yesterday."