Question: What is the Classifying Space of the Discrete Heisenberg Group? Which paper/book contains a detailed proof?

Thank you for your time.

Answer: 
For the primes in the denominator, there is an amusing heuristic based on the fact that $n \equiv n^{-1}\pmod p$ holds for all $n\geq 1$ (coprime to $p$) only for $p=2$ and $p=3$. So for these primes the series is like the series $1+1/2+1/3+\cdots$ for $\zeta(1)$, which has a true pole...
First, it seems that this equation was actually first ``proved'' by Euler. In the preface to the book "Elementary theory of $L$-functions and Eisenstein series" by Haruzo Hida, the author gives a beautiful exposition of Euler's manipulations leading to this formula. 

The connection to $p$-adic zeta functions seems to be via the Kummer congruences: looking at Euler's formulae, Kummer was apparently lead to the congruences that bear his name, and an appropriate interpretation of the latter by Kubota and Leopoldt half century ago gave rise to the first construction of the $p$-adic analogue of the Riemann zeta function -- the so-called Kubota-Leopoldt $p$-adic zeta function. 

Regarding your questions: for the first, I can only say that the zeta values in the Kummer congruences need to be amended to give rise to a continuous function of a $p$-adic variable $s$ (essentially, one needs to remove the corresponding values of the Euler factor $1-p^{-s}$); for the second, I do not see any a priori ``obvious'' reason why $2$ and $3$ should be the only primes in the denominator in Euler's formula.

The first four pages of Pierre Colmez's notes <http://www.math.jussieu.fr/~colmez/Kubota-Leopodt.pdf> are an excellent reference for the mathematical facts referred to in the last two paragraphs.