Question: I thought it would be so because the mass inside the earth would attract us however they would be at a larger distance than masses on the surface of the earth.

For example, consider having a tall building near the object. Would the effective force vector of the object be pointing into the center of the earth or deviated?

Answer: 
A [large skyscraper's mass](https://www.quora.com/How-much-does-a-skyscraper-weigh):

$$M=2.5\times 10^{8}\,\mathrm{kg}$$ 

For the benefit of the doubt, let's assume a centre-of-mass at the ground, and that you are standing at the building's wall, possibly 50 metres away.

$$r=50\,\mathrm{m}$$

The skyscrapers gravity on a $m=75\,\mathrm{kg}$ person is:

$$F\_{g,\text{skyscraper}}=G\frac{mM}{r^2}=0.0005\,\mathrm N$$

Compare this with the $F\_{g,\text{earth}}=mg=735\,\mathrm N$ gravity from Earth. The skyscraper is adding a perpendicular component, so the total force of gravity on the person is a vector, which now is slightly turned:

$$F\_g=\begin{pmatrix}735\\0.0005\end{pmatrix}\,\mathrm N$$

The angle in degrees between this vector $\begin{pmatrix}735\\0.0005\end{pmatrix}\,\mathrm N$ and the original vector $\begin{pmatrix}735\\0\end{pmatrix}\,\mathrm N$ is:

$$\theta=3.9\times 10^{-5} ~^\circ$$

This is a very, very small deflection. Just the fact that gravity from Earth does not pull directly to the centre due to uneven density, centrifugal effects, altitude etc., might far outweigh this tiny deflection.
The force vector would be deviated. That the gravitational force is pointed to the center of the earth is a simplification that assumes the earth is a point mass.
In fact all masses pull on each other, so the mass of your tall building will pull on the object, too, deviating it.
It might even be that the mass of the earth isn't equally distributed - quiet sure - and this causes some (very small) deviation.