Question: I thought it would be so because the mass inside the earth would attract us however they would be at a larger distance than masses on the surface of the earth.

For example, consider having a tall building near the object. Would the effective force vector of the object be pointing into the center of the earth or deviated?

Answer: 
The force vector would only point towards the centre of the Earth if it were spherically symmetric and non-rotating (except perhaps at the equator or poles). There are significant deviations from this and local gravity can deviate by tenths of a degree from a line towards the centre of the Earth.

e.g. see <https://www.wolframalpha.com/widgets/view.jsp?id=e856809e0d522d3153e2e7e8ec263bf2>

I believe that the rotational effects (which are easily predicted) are usually somewhat bigger than the effects caused by local geography.

These deviations were used historically to estimate the [mean density of the Earth](https://en.wikipedia.org/wiki/Schiehallion_experiment) by Neville Maskelyne among others, who used the deviation of a plumbline caused by the mountain Schiehallion.
The force vector would be deviated. That the gravitational force is pointed to the center of the earth is a simplification that assumes the earth is a point mass.
In fact all masses pull on each other, so the mass of your tall building will pull on the object, too, deviating it.
It might even be that the mass of the earth isn't equally distributed - quiet sure - and this causes some (very small) deviation.