Question: Is there a self-adjoint extension of an operator that corresponds to a particle in a box $[a,b] \times [c,d] \subset \mathbb{R}^2$ with a delta potential, i.e., $-\Delta + \lambda \delta\_y $ on $L^2([a,b] \times [c,d])$ ?

All the references I've been able to find mention a particle on the line or on $\mathbb{R}^2$.

Answer: 
Maybe the following references will be helpful:

<http://arxiv.org/abs/quant-ph/0103153> (Self-adjoint extensions of operators and the teaching of quantum mechanics, by G. Bonneau, J. Faraut and G. Valent).

<https://www.amherst.edu/media/view/10264/original/gopalakrishnan06.pdf> (Self-Adjointness and the Renormalization of Singular Potentials, Bachelor
thesis by S. Gopalakrishnan).
The paper Ph. Blanchard, R. Figari, A. Mantile "Point Interaction Hamiltonians in Bounded Domains" <http://arxiv.org/abs/0704.3249> may contain some answers. In any case, if you have an operator $H$ in $L^2(\Omega)$ and want to study its perturbations by zero-range potentials, you need to know the integral kernel of the resolvent $R(z)=(H-z)^{-1}$, see the discussion in Section 1.4.3 of J. Brüning, V. Geyler, K. Pankrashkin: Spectra of self-adjoint extensions and applications to solvable Schrödinger operators <http://arxiv.org/abs/math-ph/0611088> or the older paper V.A. Geyler, V. A. Margulis, I. I. Chuchaev: Potentials of zero radius and Carleman operators, Siberian Math. J. 36 (1995) 714–726.