One number is removed from the set of integers from $1$ to $n.$ The average of the remaining numbers is $163/4.$ Which integer was removed? 
One number is removed from the set of integers from $1$ to $n.$ The average of the remaining numbers is $\dfrac{163}4$.  Which integer was removed?

Source. British Mathematical Olympiad 2010/11, Round 1, Problem 1
I was hoping if someone could spot the flaw in my working for this question.
Attempt.
I began by letting the integer that was removed be $x$.
Then: $$\frac{1 + 2 + \cdots + (x-1) + (x+1) +\cdots + n} {n-1} = \frac{163}{4}$$
There is two arithmetic sums in the denominator, the first from 1 to $x$ and the second from $x+1$ to $n$.
These are equal to $\frac{x(x-1)}{2}$ and $\frac{(n-x)(n+x+1)}{2}$, and subbing in to first equation this gives:
$$\frac{x(x-1) + (n-x)(n+x+1)}{2(n-1)} = \frac{163}{4}$$
which reduces to:
$$\frac{n^2 + n - 2x}{2(n-1)} = \frac {163}{4}$$
And then:
$$2(n^2 + n -2x) = 163(n-1)$$
At first I thought you could consider factors, as 163 was prime then:
$n-1 = 2$ giving $n = 3$ and $n^2 + n - 2x = 163$, which using $n=3$ gives $x= -75.5$ which isn't our positive integer.
I then tried considering a quadratic in $n$ and using the discriminant but again that just looked to give a negative value of $x.$ I would be grateful for any help

The average of $1,2,3,\ldots,n$ is the number halfway between the endpoints, $(n+1)/2,$ so the sum is $n(n+1)/2.$ Omitting $x$ from among $1,2,3,\ldots,n,$ we get the sum $n(n+1)/2-x.$
Thus the average of $1,2,3,\ldots,n$ must be a weighted average of $\big( n(n+1)/2-x\big)/(n-1)$ and $x,$ with respective weights $(n-1)/n$ and $1/n.$
$$
\frac{n-1} n \left( \frac{n(n+1)/2} {n-1} - \frac x {n-1} \right) + \frac 1 n\cdot x = \frac{n+1} 2
$$
Therefore we have:
\begin{align}
& \frac{n-1} n \cdot \frac{163} 4 + \frac x n = \frac{n+1} 2 \\[8pt]
& \frac{n(n+1)}{2(n-1)} - \frac x {n-1} = \frac{163} 4
\end{align}
So we get a system of two equations that is quadratic in $n$ and linear in $x.$ I'd try solving for one of those two in terms of the other and then substituting and solving the remaining equation.
