What to do when results vary wildly based on coding the predictors as counts or frequencies? I don't want to explain my real data, but I will make up an example that is equivalent.
Let's say people are judging the quality of paintings and can say "like" or "dislike".
Paintings can have five colours in them: red, yellow, blue, green and/or orange. The paintings can be any size.
Count Approach
Let's say I categorize each square cm based on its colour (and that in this example each square cm can only be one colour). Because paintings can be of different sizes, the totals will differ across paintings. I create a model that is the number of square cms that are each colour.
So my model is: Liked = TotalRed + TotalYellow + TotalBlue + TotalGreen + TotalOrange
Analyzed like this, my results are something like this:




Term
B




Intercept
.8*


TotalRed
-.2


TotalYellow
.04


TotalBlue
-.5


TotalGreen
-.5*


TotalOrange
-.3*




*= p < .05
Proportion Approach
Let's say for each painting I calculate the proportion of the painting that is each colour. This time I leave out blue because it is the least frequent and I want to avoid perfect redundancy among my predictors.
So my model is: Liked = PropRed + PropYellow + PropGreen + PropOrange
Analyzed like this, my results are something like this:




Term
B




Intercept
-.4


PropRed
1


PropYellow
1.5*


PropGreen
-.01


PropOrange
-.15




*= p < .05
Some predictors change signs, different predictors are significant, and my intercept changes.
How do I know which one is "more right"?
Edit:
To give some more detail, this is a logistic regression, with multiple predictors. I have about 450 observations.

While answering the question all kinds of critical questions about your method pop up (For instance, how did you compute the p-values, is it based on anova or on t-tests? How much data do you have? Why choose a linear regression? Are these reasonable/good models to start with? What is the point of the experiment? Etc)...
...but, answering the main question, generally one could compare the results based on some comparison of performance. Either based on some theoretical ideas about the distribution of the error (like computing AIC), or based on evaluation of performance with a holdout/test data set (like cross-validation/machine-learning).
Personally, I wouldn't attack this problem with a straightforward linear regression, and instead I would like to observe and explore the results in order to gain a better idea of a potential model and during such exploring work I would try to involve theories and background of previous knowledge from the field.
