Question: I am looking for a way to test whether a boundary threshold exists in a physiological response – a sample of the data is plotted below. My hypothesis is that the X-variable imposes a physiological constraint on Y-values, thus producing a boundary 'ceiling' for maximum Y-values that decreases at higher X-values (indicated by the red line on figure). I assume any Y-values below the boundary are limited by some other factor not included in this model.

Essentially, my goal is to determine if the boundary exists and if so to derive a confidence interval for the boundary line model – similar to a linear regression model, but describing the upper bound of the Y-values, rather than the center of mass.

I'm sure something like this exists, but I haven't come across it before. Also, I would appreciate any suggestions on a better title or tags for this post – I assume there are more accurate terms for what I'm describing that would help folks find this post.

![threshold](https://i.stack.imgur.com/1BQBk.jpg)

Answer: 
Such a pattern would often occur when no "boundary" actually exists.

Here I generate X and Y as *independent* right-skew random variates, yet such a pattern occurs:

![enter image description here](https://i.stack.imgur.com/VKEcy.png)

The impression of any sense of a boundary in my plot is *completely bogus*, yet it looks very similar to yours. (There's an actual *vertical* boundary in this bivariate distribution at $x=80$, but I could generate very similar looking plots without any boundaries at all.)

Here's the code I used to generate the plot (in R):

```
x = rbeta(1000,1,10)*80
y = rbeta(1000,1,3)/1.5+.3
plot(x,y,ylim=c(0,1))

```

Trying it a few more times it looks like about a third of the time it gives a plot that seems to have such a slanting boundary.

No doubt a little fiddling with distributions could improve the proportion of times it occurs and at the same time make it look even more like your picture (this shifted/scaled beta(1,10)$\times$beta(1,3) was the very first counterexample I tried).

Given my picture doesn't actually have any boundary there, one should be careful of over-interpreting such a pattern. You'd need a characterization of what makes it a boundary that *wouldn't* generate lots of false positives on examples like the one I give.
My intention on how this problem might be solved is:

1. Calculate **linear model** to receive the regression line $r$.
2. Calculate the **normal vector** $v$ to the resulting regression line.
3. Shift $r$ by $v$ till all data points are under $r$.

To optimize $r$, you might rotate it by some angle $\alpha$ and stop for the best $\alpha$ you found, maybe using the [Residual Sum of Squares](http://en.wikipedia.org/wiki/Residual_sum_of_squares) as reference term.

Like I tried to show in this figure:

![enter image description here](https://i.stack.imgur.com/Dw615.png) 

Another approach could be to use [Support Vector machines](http://de.wikipedia.org/wiki/Support_Vector_Machine). I don't know if this is possible with your data, but maybe you can produce some dummy points located above your data and split them from your original points using a SVM. This is just some idea I came up with. Though, I would prefer the first method.