Although your question does not say it, I'm assuming that you do not want windows to overlap.
One approach to this problem is to use a constraint solver such as Choco. One simply writes down the constraints encoding your problem, tunes the solver to act in a smart way, and then let it run. This means that all the thinking you need to do will be spent on finding a good way of encoding the problem, not on devising an algorithm and doing the programming and tuning. Here is a partial answer to get you started.
Assume that the screen size is by $x_{max}\times y_{max}$.
For each window, $W_i$, you'll have a set of variables $x_i, y_i, h_i, w_i$ and constraints
* $x_i,y_i,h_i,w_i\ge 0$
* $x_i + w_i \le x_{max}$
* $y_i + h_i \le y_{max}$
* Perhaps also some constraints on the minimal size of windows, e.g., $h_i\ge 100$ and so forth.
* Aspect constraints: If aspect ratio is 3:4, the constraint could be something like $4h_i - \epsilon \le 3 w_i \le 4 h_i + \epsilon$, where $\epsilon$ is some small non-zero error term to allow for non-perfect window sizes, as otherwise you'd over constraint the problem.
Now you need to take care of window overlap. 946
For each pair of windows, $W_i, W_j$, where $i\neq j$, you'll generate constraints like the following, which capture that no corner of $W_j$ appears within $W_i$. For $(x,y)\in{(x_j,y_j), (x_j+w_j,y_j), (x_j,y_j+h_j), (x_j+w_j,y_j+h_j)}$, generate constraint:
* $\neg(x_i \le x\le x_i + w_j \wedge y_i\le y \le y_i+h_j)$.
The constraints specified thus far only describe non-overlapping windows that do not spill over the sides of the screen, that satisfy some minimal size constraints, and that preserve their aspect ratio.
In order to get a good fit, you need to specify a metric that captures what it means to be a good layout. One possibility is to assume that you want to keep windows roughly equal in size and/or that you want to minimize "white space". I do not think that this can be specified using Choco, but it may be possible with another constraint solve (someone else might be able to help here).
Choco does allow one to maximize wrt to an objective function specified as a single variable. Based on this idea, you could maximize the following:
* $\sum_i (h_i + w_i)$
by writing a constraint $\mathit{cost}=\sum_i (h_i + w_i)$ and telling <PERSON> to maximize $\mathit{cost}$. 433