ProtoSnap: Prototype Alignment For Cuneiform Signs
Rachel Mikulinsky Morris Alper Shai Gordin Enrique Jiménez Yoram Cohen Hadar Averbuch-Elor
The cuneiform writing system served as the medium for transmitting knowledge
in the ancient Near East for a period of over three thousand years. Cuneiform
signs have a complex internal structure which is the subject of expert paleographic
analysis, as variations in sign shapes bear witness to historical developments and
transmission of writing and culture over time. However, prior automated techniques
mostly treat sign types as categorical and do not explicitly model their highly varied
internal configurations. In this work, we present an unsupervised approach for
recovering the fine-grained internal configuration of cuneiform signs by leveraging
powerful generative models and the appearance and structure of prototype font
images as priors. Our approach, ProtoSnap, enforces structural consistency on
matches found with deep image features to estimate the diverse configurations
of cuneiform characters, snapping a skeleton-based template to photographed
cuneiform signs. We provide a new benchmark of expert annotations and evaluate
our method on this task. Our evaluation shows that our approach succeeds in
aligning prototype skeletons to a wide variety of cuneiform signs. Moreover, we
show that conditioning on structures produced by our method allows for generating
synthetic data with correct structural configurations, significantly boosting the
performance of cuneiform sign recognition beyond existing techniques, in particular
over rare signs. Our code, data, and trained models are available at the project page:
https://tau-vailab.github.io/ProtoSnap/
Machine learning for social sciences, Ancient character recognition, generative models
An unsupervised approach for recovering the fine-grained internal configuration of cuneiform signs using diffusion-based generative models.