How to numerically optimize affine transformations?
I need to optimize affine transformations for of a set of triangles using energy function based on the connectivity.
The energy of an edge $e_j$ between triangles $T_a, T_b$ is given by $$ E_j = \left[(A_a v_{a,1} + t_a) - (A_b v_{1,b} + t_b) \right]+ \left[(A_a v_{a,2} + t_a) - (A_b v_{2,b} + t_b) \right] $$
where $(A_a, t_a)$ and $(A_b, t_b)$ are affine transformations on the triangles adjacent to the edge and $v_1$ and $v_2$ are the vertices of the edge. The solution $v_i \equiv 0$ is prevented by an additional deformation energy on the triangles, like the conformal energy.
After the optimization, we want to have
$$v_{a,1}=v_{b,1}, \quad v_{a,2}=v_{b,2} \quad \Rightarrow E_j = 0$$
My problem is, that this is a non-linear problem, which is hard for solvers. I am using Google Ceres with line-search, but it often fails to find a valid line-search stepsize when optimizing
$$ \begin{pmatrix} a_1 & a_2 \ a_3 & a_4 \end{pmatrix}, \begin{pmatrix}t_1 \ t_2\end{pmatrix} $$
as a vector $(a_1, a_2, a_3, a_4, \dots, t_1, t_2, \dots)$.
For some transformations, there are more favorable parameterizations, e.g.
Rotation can be parameterizized as
$$ \begin{pmatrix} \cos \alpha & -\sin\alpha \ \sin \alpha & \cos \alpha \end{pmatrix}, \begin{pmatrix}t_1 \ t_2\end{pmatrix} $$
optimizing the vector $(\alpha, \dots, t_1, t_2, \dots)$.
Shearing can be optimized using
$$ \begin{pmatrix} 1 & s_1 \ s_2 & 1 \end{pmatrix}, \begin{pmatrix}t_1 \ t_2\end{pmatrix} $$
optimizing the vector $(s_1, s_2, \dots, t_1, t_2, \dots)$.
But for general deformations, I do not see which parameterization of the transformation matrix would fit and optimizing the coefficients directly probably lets the solver fail (or not converge) because it yields no good gradient.
So what is the best way to optimize affine transformations to find triangle deformations?
Here a graphic what should be optimized:
The final optimization objective is:
Given a set of Triangles
$$T={t_1=[v_1, v_2, v_3], t_2=[v_4, v_5, v_6], ßdots}$$
without shared vertices in the set and the information about the shared vertices of common edges as a separate set
$$E={[a,b,c,d], \dots|a\sim c, b\sim d}$$
where $\overline{ab}$ and $\overline{cd}$ are the same edges, when all triangles are re-assembled to a mesh.
Find affine transformations $A_i+t_i$ for all triangles, such that for two adjacent triangles $t_1=[a,b,x]$ and $t_2=[c,d,y]$ holds
$$ A_1 a + t_1 = A_2 c + t_2 \quad\wedge\quad A_1 c + t_1 = A_2 d + t_2 $$ and for all transformations $A_i$: $$ \text{trace} \frac{(A_i^T+A_i)}{2 \det A_i} \to \min $$
The first condition ensures that common edges are merged and the second condition minimizes the conformal deformation, what e.g. prevents the mesh form collapsing to a single vertex.
When I for example optimize the triangles of a cut cube, then the only transformations I need are rotation and translation to arrange the triangles in the plane. 490