Explaining importance of interaction that is statistically insigificant Consider the following dataset composed of two different groups blue and red which are observed over time. The super-imposed lines are regression lines fit to each group separately, i.e. for each group we fit a model $y=a + b \times Time$. In my mind, it is very clear that there is an interaction between Time and the grouping of the point, but when I fit a single model to the entire data: $y=a + b \times Time + c \times Group + d \times Time\times Group$, the third interaction term always shows up as not significant. This is based on the p-value that I get in R (for reference, it's around 0.55).
My question is, how should one explain this phenomenon? The red group is clearly under represented at both extremes and if one were to take a hold-out approach to model selection, it is clear that the model with interaction should always out-perform a model without the interaction. Is there a rigorous way of arguing why the p-value from the hypothesis test computed by R is not reliable in this scenario though ?

update
Based on the suggestion by @Frank Harrell, I generated the second plot.  

The p-value tells you that you cannot conclude that the blue and red lines have different slopes. Looking at the graph, I think that makes sense - the lines aren't exactly parallel, but they look pretty close to parallel.

Is there a rigorous way of arguing why the p-value from the hypothesis
test computed by R is not reliable in this scenario?

You could look at the standard error on the estimated red slope and blue slope. I suspect you'll see that the uncertainty in estimating each slope is big compared to the difference between them. But this is not a reason to disbelieve the p-value; it's a way to satisfy yourself that the p-value is correct.
By the way, I have sometimes heard the incorrect idea that the interaction tells you whether time depends on group. By any chance is that what you mean when you say it is clear to you that the interaction should be significant?
