text:
\section*{Acknowledgments}
We thank the Michael Stifel Center Jena for funding this work, which is part of the Carl Zeiss Foundation-funded project 'A Virtual Workshop for Digitization in the Sciences' (062017-02).

\section{Introduction}
\label{sec:introduction}
Recent advancements in pre-training large-scale language and vision (L\&V) models, such as CLIP~\cite{radford2021learning}, have led to exceptional performance on benchmarks and leaderboards in 2D image-text retrieval~\cite{shen2021much, fang2021clip2video, baldrati2022effective}.
However, the image-text data in these benchmarks have specific properties and biases~\cite{thomason2022language} that may limit the language grounding capabilities of existing L\&V models and their robustness in real-word scenarios~\cite{khandelwal2022simple, gadre2022clip}. A fundamental bias in existing L\&V data comes from the fact that images generally show single, human-centric views of \textit{different objects}.
This raises a simple but intriguing question:
to what extent can a model acquire knowledge about the concept of viewpoints and identify \textit{different} views on the \textit{same object}?
Figure \ref{fig:laion_search_example} illustrates this challenge, showing the top-3 images retrieved by CLIP for two basic viewpoint descriptions, \textit{car/airplane from the bottom}, in the LAION-5B~\cite{schuhmann2021laion}  data set:  the \textit{airplane} images mostly correspond to the correct view, but none of the \textit{car} images shows a bottom view.
It suggests that the model does not generalize the meaning of viewpoint descriptions across different objects,\footnote{When searching the LAION-5B dataset via image embeddings of cars from the bottom, dozens of relevant results can be provided, which shows that these views exist in the data.} and may fail to acquire visual-linguistic knowledge that would be needed in more realistic 3D scenarios, such as when instructing a drone to take a picture of an object from a specific viewpoint~\cite{thomason2020vision, fan2022aerial}.
This opens the door for a systematic examination of the capabilities of L\&V models for grounding viewpoint descriptions, delving into the question of why, despite their excellent zero-shot capabilities, a model like CLIP struggles when it comes to representing perspectives of the same object.

\begin{figure}
 \centering
 \includegraphics[width=\linewidth]{figures/intro_example_car_airplane_3_merged_spaced.png}
\caption{Top-3 retrieval results for \textit{car/airplane from the bottom} using CLIP on the LAION-5B dataset.\footnotemark}
 \label{fig:laion_search_example}
\end{figure}
\footnotetext{\url{https://rom1504.github.io/clip-retrieval/}}

In this paper, we investigate whether language understanding in pre-trained L\&V models generalizes to simple text-viewpoint descriptions of common objects.
We propose a new task -- text-viewpoint retrieval -- and a framework for analyzing and scaling image-text models with 3D data. We implement a \textbf{\texttt{Paparazzi}} agent that circles a  spherical camera around a 3D object, samples images, and scores pairs of image-viewpoint descriptions using a pre-trained image-text matching model. In this framework, we evaluate and analyze whether CLIP, as a representative image-text-matching model with excellent zero-shot capabilities, systematically retrieves images of views of 3D shapes, regardless of potential reporting biases in 2D L\&V data sets.

To successfully interpret viewpoint descriptions like \textit{car from the bottom}, models need to connect concepts in natural language to visual representations and basic knowledge of object geometry.
To investigate this, our approach is deliberately simple: we use 3D shapes from five categories of common objects in ShapeNet that have visually distinct canonical views \textit{(front, back, left, right, top, bottom)}.
Based on Goldberg polyhedrons~\cite{goldberg1937class}, that divide a sphere into hexagonal shapes, we analyze whether CLIP provides an adequate embedding for the viewpoint space around an object.
Our analysis suggests that basic viewpoint understanding is indeed a systematic gap in the pre-trained CLIP model, as it achieves very poor performance in scoring view-description pairs and even retrieves nonsensical, non-human-centric views.
Furthermore, we find that this problem is not fixed by standard fine-tuning.
Thus, we propose a procedure for fine-tuning CLIP that extends the contrastive learning approach to viewpoints and descriptions generated from 3D visualizations.
We find that a small amount of training data and extended fine-tuning is successful in scaling CLIP to basic viewpoint understanding in 3D.
\section{Related Work}
\label{sec:related_work}

\paragraph{Vision, View, and Language.}
To date, research on grounding language in vision focuses on connecting language to visual representations of 2D human-centric views of scenes and objects based on, e.g., large image-caption data sets~\cite{thomee2016yfcc100m, schuhmann2021laion}.
Retrieval models in L\&V usually rank a fixed set of images showing single views of different objects and scenes given a textual query or vice versa \cite{li2020unicoder, li2020oscar, baldrati2022effective}. Common understanding models process pairs of texts or questions and single-view images and predict labels for them, typical generation models process single-view images and generate descriptions for them~\cite{mokady2021clipcap, yu2022coca}. In this paper, we propose a new L\&V retrieval task where the model needs to search for a specific view, represented as an image, of a 3D object given a textual query. In our task, the space of possible view-images is not restricted to a human-centric view.

\paragraph{Language Grounding in 3D.}
\citet{achlioptas2019shapeglot} present pioneering work in this area, with a referring expression data set designed for learning the language of shape for \textit{chair} objects in ShapeNet, the most well-known resource for 3D object models~\cite{chang2015shapenet}. They build a neural resolution model that predicts which chair is referred to by a given shape description. Their encoder combines an autoencoder for point clouds of 3D shapes and a pre-trained image encoder for a single view of the object. As \citet{achlioptas2019shapeglot} collected descriptions of the 3D objects in a static environment with a fixed camera perspective, their approach does not account for dynamic viewpoints in 3D.  \citet{thomason2022language} present a larger data set for expressions referring to ShapeNet objects and build a model that relies on image-text matching via the CLIP architecture, similar to ours. Their model takes images of eight fixed viewpoints of the object as input and integrates a component that estimates the viewing angle of an image. They evaluate on resolution accuracy and do not explicitly test viewpoint understanding in the CLIP model. In contrast to these existing works, the input to our model does not specify a fixed set of camera positions, and the output is an explicit, specific viewpoint of an object represented as an image.

\paragraph{Camera Position Estimation.}
\label{sec:optimal_camera_position_estimation}
Viewpoint selection in a 3D environment is a well-known problem in other areas \cite{kamada1988simple,roberts1998viewpoint,arbel1999viewpoint,vazquez2001viewpoint,plemenos2006viewpoint, podolak2006planar,muhler2007viewpoint}. Work in photogrammetry investigates camera position estimation minimizing the error in 3D measurements and reconstruction~\cite{olague2002optimal}.
Systems in visualization aim to find an optimized viewpoint with the least possible occlusion and maximum information content for polygonal data~\cite{vazquez2001viewpoint, neugebauer2013amnivis, meuschke2017automatic}, volumetric data~\cite{bordoloi2005view} and vector fields~\cite{lee2011view, tao2012unified}.
A key challenge in these areas is the definition of what actually constitutes a good viewpoint \cite{bonaventura2018survey}.
Most algorithms aim to find a viewpoint that is of high interest to the user~\cite{leifman2016surface, neugebauer2013amnivis}, but do not yet incorporate textual descriptions of viewpoints.
In addition, most of these algorithms require expensive annotated mesh representations of 3D objects.
L\&V models pre-trained on raw image-text data constitute an extremely promising direction here, provided that they are capable of viewpoint understanding.

\section{Text-Viewpoint Retrieval Task}
\label{sec:text_viewpoint_retrieval_task}
We study viewpoint understanding from descriptions  and describe a framework for text-viewpoint retrieval.
We present a task definition, the set-up of the 3D environment and the camera, and our approach to evaluation and analysis.

\subsection{Task Definition}
\label{sec:task}
We define the input of our viewpoint retrieval task to consist of a 3D scene with a single object $O$, a search query describing a viewpoint $q$, and an orbital camera $\mathit{C}$ circling the object.
The camera returns single views of the object $v$ that are represented as RGB images.
The retrieval model's task is to find a viewpoint $v$ that matches the query $q$.
In this work, we implement retrieval via a scoring function $S$ that passes pairs of images $v$ (taken by the camera) and queries $q$ to a pre-trained text-image matching model.
The parameterization of the orbiting camera $\mathit{C}$ determines the space of possible viewpoints $\mathit{V}$ that the retrieval model has to search. The parameter setup we used in this work is explained in detail below.

This setting leverages the well-understood image-text matching in 2D for language grounding in 3D. Our retrieval model does not have a symbolic or explicit representation of the object's geometry but can perceive it by taking images from various perspectives. This framework is independent of different types of 3D data and only requires an engine that renders images of 3D environments.

\subsection{Camera Set-up}
\label{sec:camera}
For the purpose of this study, we restrict the viewpoint space $V$ to views that contain the object of interest.
We use a spherical camera system where the center of the object defines its center, as shown in Figure~\ref{fig:circling_camera_setup}.
The camera in orbit can be navigated around the desired object using polar coordinates.
The position of the camera towards the object is defined by $(r,\theta,\varphi)$ for the radial distance, the azimuthal angle, and the polar angle.
The center of the object is defined by the center of its bounding box.
The camera's local x and y axes are used to adjust the camera's viewing angles.
Rotation around the local z-axis of the camera is disabled in this work, as the results would be the same, only with a rotated output image. In summary, the exact camera position and rotation along the sphere can be described by five parameters: $(r,\theta,\varphi,x,y)$.

\begin{figure}[t]
 \centering
 \includegraphics[width=\linewidth]{figures/camera_and_car_cutoff_v003.pdf}
 \caption{The camera setup: the viewing angles $\theta$ and $\varphi$ describe the azimuthal and polar angle of the camera on the orbital sphere. The parameters x and y describe the camera's orientation at the given location.}
 \label{fig:circling_camera_setup}
\end{figure}

To create equidistant sample points for camera positions along the sphere, we use a Goldberg polyhedron \cite{goldberg1937class}. It divides a sphere into mostly hexagonal shapes, including a small finite number of pentagons, and creates a nearly equidistant sample space (see Figure~\ref{fig:circling_camera_setup}). The centers of the hexagons give us a discrete number of sample points, which reduce the possible configurations of our camera setup to a finite number. The hexagon centers can be approached for different radii $\mathit{r}$. The polyhedron used in this work initially yields 1002 sample points per radius. This discretization of the sample space is fine enough to allow benchmarking and analysis of viewpoint retrieval models.

The object $\mathit{O}$ lies at the origin of the Cartesian space $(0,0,0)$, which is also the center of the surrounding hypersphere.
The radius $\mathit{r}$ is clipped relatively to the size of the object. We estimate the extent of the object based on its bounding box. We determine the extent of the bounding box based on the minimum $r_{min}$ and maximum radius $r_{max}$ of the surrounding orbital spheres.
In our experiments, we set  $r_{min}$ to two times the edge length of the bounding box and $r_{max}$ to ten times the edge length of the bounding box.

\subsection{Evaluation and Analysis}
\label{sec:metrics}

\paragraph{Common Objects and Canonical Views.}
\label{sec:shapes}
To systematically evaluate language-view understanding in CLIP, we limit the set of viewpoint descriptions $\mathit{Q}$ in our experiments to the six \textbf{canonical views} \textit{front, back, right, left, top, bottom}  defined by~\citet{chang2015shapenet}.
We choose 3D models of common object categories in ShapeNet~\citep{chang2015shapenet}. From the available 55 categories, we selected five categories where all canonical views are visually distinct: \textit{cars, airplanes, motorbikes, mugs} and \textit{benches}.\footnote{Many object categories like \textit{bottle, ball, table, etc.} do not have this property. For instance, the \textit{front} and \textit{back} views of a bottle are not or much less distinct than the \textit{front} and \textit{back} views of a car.}
As ShapeNet provides an aligned representation of all 3D models,
these restrictions yield a fully controllable experimental setup where training and test data with pairs of queries and views can be generated automatically.
The experimental setup is general enough to be transferable to arbitrary object domains and various forms of textual viewpoint descriptions.

\paragraph{Viewpoint Quality Evaluation.}
To assess the quality of text-viewpoint retrieval, we use the KL divergence~\cite{kullback1951information} of a model's scoring function against a gold standard scoring distribution as well as the classical retrieval metrics \textit{precision@k} and \textit{retrieval@k}.
We use KL divergence in addition since retrieval metrics only reflect performance on gold standard viewpoints and do not allow us to infer the global performance needed to find out why models fail on certain queries, as discussed in Section~\ref{sec:experiments}.
We define the gold standard score distribution with respect to a particular viewpoint as a discrete normal distribution around the gold standard viewpoint, which is the mean of the distribution. The three polygonal rings around the mean are assigned the normalized score value at one, two, or three times the standard deviation of the normal distribution. The scores for all these viewpoints sum to 1. The scores for all other viewpoints around the sphere are set to zero. The setup is illustrated in Figure~\ref{fig:circling_camera_setup}.
To visually analyze the goodness of a scoring function over a sphere, we unfold the polyhedron and upsample it, as shown in the small map at the bottom right of Figure~\ref{fig:circling_camera_setup}. In this way, we can visualize the difference between the gold standard and the predicted score distribution for an object.

\paragraph{Search Performance Evaluation.}
When searching a 3D scene, there are many possible viewpoints to consider. A scoring function that works well on a subset of pre-selected viewpoints may yield a good result in retrieval metrics, but in practical usage, it may lead the search algorithm to an unexpected or nonsensical viewpoint. Therefore, to evaluate the performance of a model, we need to consider not only how well it performs on the gold standard viewpoint images, but also how well it can guide a search algorithm to find the right viewpoint in the scene.
We compare the performance of different search algorithms under different configurations of the scoring function to \textbf{understand the impact of the shape of the scoring function on search performance}.
We compute search performance as follows: a search is considered successfully completed if the found viewpoint is within a certain radius of the respective gold standard viewpoint. We define the radius discretely based on the hexagonal rings around a gold standard viewpoint on the Goldberg polyhedron. In our experiments, we consider a search to be solved if a viewpoint is found within the first two rings around the gold standard viewpoint (see Figure~\ref{fig:circling_camera_setup}). We compare performance in terms of the number $c$ of calls to the scoring function required by the search algorithm to solve the search problem described above. We restrict the search length to a maximum number $c_{max}$ of 300 viewpoints to visit. To obtain a robust comparison, we run the procedure $n$ times at randomly selected starting positions on the hypersphere around the object. In our experiments, we set $n$ to ten. Then, the number of calls $\frac{c}{n}$ is averaged.

\section{Model}
\label{sec:model_architecture}

\subsection{Scoring Function}
The heart of our retrieval model is a function $S$ that outputs matching scores for pairs of images and queries $(v,q)$.
Pre-trained L\&V models like CLIP~\cite{radford2021learning} embed $(v,q)$ pairs into a common subspace, resulting in latent vector representations $\mathbf{z_v}$ and $\mathbf{z_q}$, e.g., of size $512$ in the original CLIP.
The output of the scoring function $S$ is the cosine similarity of the latent representations of the viewpoint image and the search query:
\begin{equation}\resizebox{.99\hsize}{!}{$
  {S}(v,q) = cos(z_v,z_q) = \frac{\mathbf{z_v} \cdot \mathbf{z_q}}{|\mathbf{z_v}||\mathbf{z_q}|}
  =\frac{\sum_{i=1}^{N} z_{v_{i}} z_{q_{i}}}{\sqrt{\sum_{i=1}^{N} z_{v_{i}}^{2}} \sqrt{\sum_{i=1}^{N} z_{q_{i}}^{2}}}
  $}
\end{equation}
To evaluate a given viewpoint with respect to a query, both are encoded into their latent representations $\mathbf{z_v}$ and $\mathbf{z_q}$, and the cosine similarity of their latent representations is used as a \textbf{score} for how well the view matches the query.

\subsection{Objective Functions}
To achieve high similarity between associated texts and images, \citet{radford2021learning} apply a contrastive learning paradigm. In a training batch of N image-text pairs, a cosine similarity score is computed for each possible text-image combination. This leads to ${N \times N}$ scores over which a cross-entropy loss is calculated across the rows and columns. For corresponding text-image pairs, the maximum class score is expected, while for all other pairs, a minimum score is targeted.

We extend this contrastive learning paradigm for fine-tuning CLIP with 3D data by minimizing the combination of three different loss objectives: a) for negative examples, b) for random examples, and c) for hard negative examples.

\paragraph{Cross-Entropy Loss on Negative Examples} is calculated and summed for both queries $\mathbf{q}$ and viewpoints $\mathbf{v}$ as $\mathit{L_{\mathbf{v},\mathbf{q}}}$. The parameter $\tau$ is a learnable parameter for scaling the logits:
\begin{equation}
\mathit{L_{\mathbf{v},\mathbf{q}}}=-\frac{1}{N} \sum_{i=1}^{N} \log \frac{\exp \left(\cos\left(z_{v_{i}}, z_{q_{i}}\right) / \tau\right)}{\sum_{j=1}^{N} \exp \left(\cos\left(z_{v_{i}}, z_{q_{j}}\right) / \tau\right)}
\end{equation}

\paragraph{Cross-Entropy Loss on Random Examples} is denoted as $\mathit{L_{r}}$ and computed between annotated viewpoints and randomly generated viewpoints of the 3D scene. $\mathit{L_{r}}$ is computed exactly as in equation (2), but the contrastive examples are random images from the scene in this case.

\paragraph{Cross-Entropy Loss on Hard Negative Examples}
referenced as $\mathit{L_{h}}$ uses images that have a different annotation but appear to be similar in latent space~\cite{li2021align}.  ~\citet{robinson2020contrastive} present a sampling method that rescales the loss of negative examples based on their similarity to the gold standard sample. Following this, the loss $\mathit{L_{h}}$ is calculated as the weighted contrastive loss between the positive samples $x^{+}$ and the hard negative samples $x^{-}$ drawn from the modified negative sampling distribution $q$:
\begin{equation}
\resizebox{.999\hsize}{!}{
$\mathit{L_{h}}= \mathbb{E}_{x^{+}\sim p_{x}^{+}} \left [-\log \frac{e^{f(x)^{T} f\left(x^{+}\right)}}{e^{f(x)^{T} f\left(x^{+}\right)}+G \mathbb{E}_{x^{-} \sim q}\left[e^{f(x)^{T} f\left(x^{-}\right)}\right]}\right]$
}
\end{equation}
In notation, $p^{+}$ is the marginal distribution of positive examples in the overall distribution of samples $p$. $q$ is the distribution of negative samples. $x$ is a single sample, $x^{+}$ and $x^{-}$ are the respective positive and negative samples. $f$ is a similarity measure, in our case it is cosine similarity.
$\mathit{G}$ is a weighting parameter that can be used to adjust the hardness of the negative sampling.

The total loss is parameterized as the weighted sum of the three objectives:
\begin{equation}
    L_{total} = \alpha \mathit{L_{\mathbf{v},\mathbf{q}}} + \beta \mathit{L_{r}} + \gamma \mathit{L_{h}}
\end{equation}
The ablations resulting from the different combinations presented above are evaluated in Section~\ref{sec:analysis}.
The parameters $\alpha$, $\beta$, and $\gamma$ are chosen based on the respective experiment.

\subsection{Search Algorithms}
\label{sec:appendix_b}
At inference time, our retrieval model requires a search algorithm $\mathit{A}$, a function that optimizes the output of the scoring function $\mathit{S}$ given the space of viewpoints $V$ and a query $q$. We compare the performance of two search algorithms. \textbf{Greedy search} starts with a grid-based approach on the Goldberg polyhedron and tries to find the optimum by moving greedily in the direction of the neighboring region with the highest score in each iteration. \textbf{Bayesian search} samples positions on the hypersphere based on incrementally obtained function values, attempting to sample with higher probability in regions that contain optima~\cite{mockus1994application}. See appendix~\ref{sec:appendix_a} for implementation details.

\section{Experiments}
\label{sec:experiments}
\subsection{Experimental Setup}
\label{sec:problem_setup}
\paragraph{Training.} For each of the six canonical view query types and five object categories, we generate 1,000 training images in a Unity scene on randomly selected objects from the ShapeNet training set.
This results in 6,000 image and text pairs per object category, which is tiny as compared to the 15 million images in the YFCC100M~\cite{thomee2016yfcc100m} data set for training the original CLIP.

\begin{table}
\setlength{\tabcolsep}{3.8pt}
\small
\begin{tabular}{lccccccc}
 \toprule
 Model & \textbf{front} & \textbf{back} & \textbf{left} & \textbf{right} & \textbf{top} & \textbf{bottom} & \\
 \midrule

 PRE-TR & 4.12 &  4.09 &  4.12 &  4.12 &  4.09 & 4.15 & \\
 FT & 3.91 &  3.90 &  3.91 &  3.89 &  3.97 &   3.92 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{car}}} \\
 RC-HNS & 2.85 &  2.88 &  3.26 &  2.99 &  3.43 & 3.24 & \\
  \midrule

 PRE-TR & 4.12 &  4.10 &  4.13 &  4.15 &  4.08 & 4.08 & \\
 FT & 3.92 &  3.97 &  4.03 &  3.95 &  4.02 &   4.02 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{airpln}}} \\
 RC-HNS & 3.43 &  3.73 &  3.43 &  3.58 &  3.52 & 3.63 & \\
 \midrule

 PRE-TR & 4.08 &  4.09 &  4.12 &  4.12 &  4.21 & 4.20 & \\
 FT & 3.98 &  3.89 &  3.94 &  3.94 &  4.04 &   3.85 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{mbike}}}\\
 RC-HNS & 2.81 & 2.60 &  2.84 &  2.81 &  3.46 & 3.47 & \\
 \midrule

 PRE-TR & 4.15 &  4.14 &  4.07 &  4.05 &  4.21 & 4.21 & \\
 FT & 3.96 &  3.98 &  3.98 &  3.94 &  3.91 &   3.90 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{mug}}}\\
 RC-HNS & 3.34 &  3.10 &  3.19 &  2.52 &  2.52 & 2.11 & \\
 \midrule

 PRE-TR & 4.08 &  4.09 &  4.17 &  4.17 &  4.15 & 4.13 & \\
 FT & 3.94 &  3.90 &  4.00 &  4.04 &  3.98 &   3.93 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{bench}}} \\
 RC-HNS & 1.88 &  1.98 &  2.62 &  2.18 &  3.25 &  3.19 & \\
 \bottomrule
\end{tabular}
\caption{KL-Divergence between gold and predicted  viewpoint distribution for the models \textit{PRE-TR, FT, RC-HNS} on the objects \textit{car, airplane, motorbike, mug, bench} for \textit{front, back, left, right, top, bottom} viewpoints on synthetic images. Lower values are better.}
\label{tab:scoring_function_kullback_leibler_divergence}
\end{table}

\paragraph{Test Set.} For evaluating the retrieval quality for each object category we randomly select three 3D shapes from the ShapeNet test set.
Then we compute the normalized score distribution on synthetic images around the sphere with radius five for all selected objects of a category, compute the KL-Divergence and average the results per viewpoint query (see Table~\ref{tab:scoring_function_kullback_leibler_divergence}). To assess the performance on real-world data, we carefully curated a data set of 600 images (5 categories $\times$ 6 viewpoints $\times$ 20 images) by retrieving visually similar images for a seed image using image similarity on LAION-5B. Synthetic gold standard views are obtained from the sampled spheres (see Table~\ref{tab:precision_and_recall_metrics}).

\paragraph{Models.} From the official CLIP repository~\cite{openai}, we select ResNet-101~\cite{he2016deep} pre-trained on ImageNet~\cite{deng2009imagenet} as image encoder and pre-trained  BERT model~\cite{devlin2018bert} as query encoder. We compare the following models: (i) \textbf{PRE-TR}ained CLIP, without further fine-tuning, (ii) CLIP-\textbf{FT}, a version of CLIP fine-tuned on the training data with standard cross-entropy loss, (iii) CLIP-\textbf{RC-HNS}, fine-tuned with extended loss objectives explained in Section \ref{sec:model_architecture}.

\subsection{Viewpoint Quality Results}
\label{sec:quality}
Table~\ref{tab:scoring_function_kullback_leibler_divergence} shows the results for the quality of viewpoint retrieval with different models, objects, and viewpoints.
We find that a pre-trained CLIP model shows a high divergence from the gold standard distribution for all object categories under investigation.
The fine-tuned model performs slightly better, but still shows large differences from the gold standard.
The use of random contrasting and hard negative sampling brings the score distribution closer to the gold standard distribution.
This shows that standard CLIP pre-training and fine-tuning on human-centered 2D images do not produce a suitable scoring function for the viewpoint space around a 3D object.

Evaluating performance on real data using KL divergence is not possible in a similar way as on synthetic data because we do not have access to images from arbitrary viewpoints. Therefore, we compare precision@k and recall@k between synthetic images from ShapeNet and real images at the gold standard viewpoints in Table~\ref{tab:precision_and_recall_metrics}.
The results show that pre-trained CLIP performs poorly in grounding viewpoints on both synthetic data and real data. Fine-tuning the model on synthetic data greatly improves the retrieval metrics for both synthetic and real data. RC-HNS performs well on synthetic data that is within the distribution, however, it yields slightly lower scores on real-world data in comparison to FT.
This may result from the fact that
RC-HNS forces the model to generally score out-of-distribution data lower, thereby making the scoring function more sensitive to differences between synthetic and real-world images.
In traditional 2D benchmarks, this may seem like a disadvantage compared to FT, but it proves to be advantageous in 3D viewpoint search, as demonstrated in the following section. Here, the FT model loses performance due to unpredictable scoring behavior in regions far from the gold standard viewpoints.

\begin{table}
\small
\setlength{\tabcolsep}{3.2pt}
\begin{tabular}{l| ccc | cccc}
 \toprule
 Model &  \textbf{P@1} & \textbf{P@5} & \textbf{P@10} & \textbf{R@1} & \textbf{R@5} & \textbf{R@10}\\
 \midrule

 PRE-TR & 0.044 & 0.044 & 0.031 & 0.007 & 0.032 & 0.043 &\\
FT & 0.622 & 0.442 & 0.401 & 0.090 & 0.267 & 0.412 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{synth}}} \\
RC-HNS & 0.811 & 0.607 & 0.541 & 0.117 & 0.355 & 0.524 & \\
\midrule
 PRE-TR & 0.300 & 0.307 & 0.290 & 0.015 & 0.077 & 0.145 & \\
 FT & 0.867 & 0.787 & 0.710 & 0.043 & 0.197 & 0.356 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{real}}} \\
 RC-HNS & 0.733 & 0.673 & 0.633 & 0.036 & 0.168 & 0.317 & \\
 \midrule
\end{tabular}
\caption{Precision@K and Recall@K per model ablation split by synthetic data and real data measured across all object categories.}
\label{tab:precision_and_recall_metrics}
\end{table}

\begin{table}
\small
\setlength{\tabcolsep}{3.1pt}
\begin{tabular}{lccccccc}
 \toprule
 Model &  \textbf{front} & \textbf{back} & \textbf{left} & \textbf{right} & \textbf{top} & \textbf{bottom}\\
 \midrule

 PRE-TR & 171.6 & 168.3 & 165.7 & 159.8 & 174.1 & 165.0 &\\
FT & 135.1 & 137.1 & 189.1 & 130.1 & 142.2 & 127.4 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{Greedy}}} \\
 RC-HNS & 130.5 & 134.5 & 182.7 & 115.9 & 140.3 & 144.4 & \\
\midrule
 PRE-TR & 259.4 & 223.2 & 294.0 & 264.8 & 198.4 & 261.6 & \\
 FT & 82.4 & 79.1 & 133.0 & 101.1 & 29.7 & 21.5 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{Bayes}}} \\
 RC-HNS & 73.5 & 62.7 & 62.6 & 49.4 & 22.0 & 22.9 & \\
 \midrule
\end{tabular}
\caption{Average number of calls to the scoring function per search algorithm and viewpoint query.}
\label{tab:search_algorith_performance_comparison}
\end{table}

\begin{figure*}[ht]
 \centering
 \includegraphics[width=\linewidth]{figures/score_combi_clip.pdf}
 \caption{Score distribution on the six viewpoints per loss function combination on a car object. In a)  gold-standard viewpoints expected to have high scores are shown, b) pre-trained CLIP, c) fine-tuned CLIP, d) hard negative sampling, e) random contrasting, and f) random contrasting + hard negative sampling. For more, see appendix~\ref{sec:appendix_a}.}
 \label{fig:score_functions_overview}
\end{figure*}

\subsection{Search Performance Results}
\label{sec:search_performance_results}
We test search performance in 3D as described in Section~\ref{sec:metrics} for all six queries. Table~\ref{tab:search_algorith_performance_comparison} illustrates the performance for Greedy and Bayes search.
Both algorithms perform significantly better than an exhaustive search on the Goldberg polyhedron (= 1002 sample points, fixed radius).
Bayesian search is much faster than greedy search, when using a finetuned scoring function (FT, RC-HNS), and it is more affected by the shape of the scoring function since it samples it strategically: it is fastest with the smoothest scoring function RC-HNS and very slow with pretrained CLIP. This is in line with the viewpoint quality results in Section \ref{sec:quality}, showing that pretrained CLIP has a poor representation of the viewpoint space around an object.

\section{Analysis}
\label{sec:analysis}
This section takes a closer look at how well the text-viewpoint embeddings capture understanding of different viewpoints. Specifically, we will explore whether the scoring functions correctly identify viewpoints that align with the linguistic description, while providing lower scores for those that do not.

\subsection{Exhaustive Viewpoint Space Analysis}
Based on the polyhedron that defines the viewpoint space of the camera, we carry out an exhaustive analysis of the scoring function over this space for specific objects and queries. We select a car from the test set of the ShapeNet data set and plot the scores of the evenly distributed samples from the surface of the Goldberg polyhedron at a radius of five for the six canonical viewpoint queries.
We examine five different configurations of the loss objective shown in Equation (4). Figure~\ref{fig:score_functions_overview}a) illustrates the target region on the hexagon diagram, which contains the optimal viewpoint for a given query.
It can be seen in Figure~\ref{fig:score_functions_overview}b) that a pre-trained CLIP model even if trained on a large data set, is not able to discriminate between different viewpoints and that the scoring function has multiple optima.
Fine-tuning the CLIP model (\ref{fig:score_functions_overview}c) on synthetic images improves viewpoint discriminability. Nevertheless, apart from the absolute gold standard regions, the function shows problematic local optima and in particular the left and right side views of the car are difficult to distinguish.
In (d), we fine-tune the CLIP model by applying the hard negative sampling strategy proposed by~\citet{robinson2020contrastive}. The results show that the gold standard viewpoints can be distinguished much more effectively when compared to previous experiments. However, the transition between viewpoints is quite sudden, making it challenging for a search algorithm to reach the optimum.
In (e),
a combination of negative contrastive loss $L_{\mathbf{v},\mathbf{q}}$ and random contrastive loss $L_{r}$  is applied.
The results show that the additional objective makes the scoring function much more stable in regions farther away from known canonical viewpoints.
In experiment (f), we combine hard negative sampling $L_{h}$ with the idea of random contrasting.
The plot of the scoring function shows that for each canonical viewpoint, the function increases steadily toward the optimal view.

\subsection{Nonsensical Viewpoints}
\label{sec:nonsensical_viewpoints}
A further problem we noticed is that CLIP predicts high scores for nonsensical views that do not relate to  the query, but rather seem to activate certain features to drive up the score, similar to adversarial examples~\cite{goodfellow2014explaining}. Such behavior of models on unseen images has also been described by~\citet{du2022vos} and should be considered when using CLIP representations in continuous 3D environments, especially for vision-and-language navigation tasks, as in~\citet{khandelwal2022simple}. Figure~\ref{fig:nonsensical_viewpoints_figure} shows retrieved nonsensical viewpoint images among the top-5 for \textit{car from the front}.

\begin{figure}[h]
 \centering
 \includegraphics[width=\columnwidth]{figures/non_sensical_views_figure_5_clip_only.png}
 \caption{Retrieved nonsensical viewpoints in the top-5 scored images on CLIP for the query \textit{a picture of a car from the front}.}
 \label{fig:nonsensical_viewpoints_figure}
\end{figure}

\subsection{Data Set Size Ablations}
\label{sec:section_low_training_data_ablations}
To test how the scoring function is affected when only a small amount of training data is available, we gradually reduce the number of training samples from 1,000 to 1 for the best-performing model CLIP-RC-HNS.
Access to 1,000 training examples per viewpoint, as shown in \ref{fig:low_data_ablation_experiment_figure}a), leads to a smooth function.
Reducing the training data by 90 percent to 100 examples per viewpoint keeps  good performance for the target viewpoints. Compared to the full data set, smoothness suffers slightly.
Reducing the training data by 99 percent to ten samples per viewpoint still allows good results in the target regions. However, the surrounding regions become less smooth and drop more abruptly.
Surprisingly, when breaking down the training data to one example per viewpoint, the target viewpoint areas still lead to global optima in all search queries. However, the transitions are no longer smooth but rather abrupt, especially for the front and back.

\begin{figure}[h]
 \centering
 \includegraphics[width=\columnwidth]{figures/score_hard_negativ_v002.pdf}
 \caption{Overview of the effects of gradually reducing the number of training images per view from a) 1000 to b) 100 to c) 10 to d) 1 on CLIP-RC-HNS.}
\label{fig:low_data_ablation_experiment_figure}
\end{figure}
\section{Conclusion}
\label{sec:conclusion}
We developed a new framework to assess the capabilities of L\&V models to ground viewpoint descriptions.
Through our research, we discovered that a standard CLIP model struggles to distinguish between different viewpoints.
To address this, we explored a combination of different loss objectives on synthetic data to make it easier to retrieve viewpoints from language descriptions.
Our experiments revealed that incorporating random contrasting leads to a more accurate and seamless scoring function, as compared to using only text and human-centric images.
Our framework thus offers a promising approach to scale L\&V models trained on large-scale image-text datasets for applications that involve interaction in the 3D world.

\newpage

\subsection*{Limitations}
\label{sec:limitations}
We deliberately opted for a simple controllable setup in order to gain a precise understanding of viewpoint representation in CLIP.
Our experiments are restricted to canonical views and canned descriptions since they are easy to generate and evaluate automatically.
Extending the data to other views and to human-like descriptions is the obvious avenue for future research.
In particular, with the advent of NERF models in computer vision, we look forward to integrating these types of models into our framework, as this would allow the generation of near-realistic images in a controlled 3D setup, which would allow for even better evaluation of scoring functions in text viewpoint retrieval.
Varying the level of detail of the 3D shapes, especially in complex 3D scenes where large objects consist of smaller parts is another interesting direction.
Another restriction of our set-up is the fact that we consider context-free retrieval of viewpoints, whereas in many human-like descriptions such as the \textit{right front tire of a car}, the viewpoint may not be visually unique and depend on the context of the scene, such as the relative position of the viewpoint to other viewpoints.
The same applies to views that need to be delivered to a user in a task-oriented interaction, and are likely to be more complex and diverse than the canonical and synthetic ones used in this work.
In conclusion, we believe that our framework has the potential to provide a more comprehensive understanding of reporting biases in image-text data used for pre-training L\&V models. By conducting a 360-degree analysis of the scoring function, our framework allows for a more thorough examination of these biases, as everything is visible and nothing can be hidden from the investigator, unlike when evaluating against a set of gold-standard viewpoints.

\section*{Ethics Statement}
3D models from the ShapeNet dataset are available for research and non-commercial purposes as well as the LAION-5B data set.
We did not collect any personal information from any annotators.
We clearly state the intended use of our models, which is to support human-centric interaction with AI models in the 3D world.

\section{Experiment Details}
\label{sec:appendix_a}
This section provides additional details on our experimental setup.
Section~\ref{sec:appendix_scoring_function_analysis} contains further visualizations of the experiments discussed in section~\ref{sec:experiments}. Section~\ref{sec:search_algorithm_analysis_appendix} provides details about the implementation of the search algorithms used in our benchmark.

\subsection{Scoring Function Analysis}
\label{sec:appendix_scoring_function_analysis}
The following plots illustrate the score distributions obtained with the different model ablations CLIP-\textbf{PRE-TR}, CLIP-\textbf{FT}, and CLIP-\textbf{RC-HNS}.

\begin{figure*}[h]
     \centering
     \begin{subfigure}{1.0\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures/raw_clip_scoring_function_distribution.png}
         \caption{Scoring Function Distribution of CLIP PRE-TR model on cars, motorbikes, airplanes, benches, and mugs for the six canonical viewpoint queries.}
         \label{fig:clip_scoring_function_distribution}
     \end{subfigure}
     \hfill
     \begin{subfigure}{1.0\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures/clip_ft_scoring_function_distribution.png}
         \caption{Scoring Function Distribution of the CLIP-FT model on cars, motorbikes, airplanes, benches, and mugs for the six canonical viewpoint queries.}
         \label{fig:clip_ft_scoring_function_distribution}
     \end{subfigure}
     \caption{Scoring Function Distributions on CLIP PRE-TR and CLIP-FT.}
\end{figure*}

\begin{figure*}[h]
     \centering
     \begin{subfigure}{1.0\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures/clip_rc_hnm_scoring_function_distribution.png}
         \caption{Scoring Function Distribution of the CLIP-RC-HNS model on cars, motorbikes, airplanes, benches, and mugs for the six canonical viewpoint queries.}
         \label{fig:clip_rc_hnm_scoring_function_distribution}
     \end{subfigure}
      \hfill
     \begin{subfigure}{0.8\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures/comparison_optimal_viewpoints_all_objects.pdf}
         \caption{Optimal viewpoints of the six canonical views for a) cars, b) motorbikes, c) airplanes, d) benches, and e) mugs of the ShapeNet data set~\cite{chang2015shapenet} retrieved from the optima of the CLIP-RC-HNS scoring function.}
         \label{fig:optimal_viewpoint_images_overview}
     \end{subfigure}
     \caption{Scoring Function Distributions on CLIP-RC-HNS and retrieved viewpoint images.}
\end{figure*}

\paragraph{Scoring Function PRETR.}
Figure~\ref{fig:clip_scoring_function_distribution} shows the score distribution of the PRE-TRained CLIP model over 3D objects from the test set of the ShapeNet dataset.

\paragraph{Scoring Function FT.}
Figure~\ref{fig:clip_ft_scoring_function_distribution} depicts the scoring distribution of the CLIP-FT model over 3D objects from the test set of the ShapeNet dataset.

\paragraph{Scoring Function RC-HNS.}
Figure~\ref{fig:clip_rc_hnm_scoring_function_distribution} illustrates the score distribution of the CLIP-RC-HNS model over 3D objects from the test set of the ShapeNet dataset.

\paragraph{Comparison of Score Distributions for Object Only Queries.}
 To understand which viewpoints CLIP scores best on an object-only query such as \textit{a picture of a car}, we compare these object-only queries for all object categories tested on respective 3D objects from the test set. This tells us which viewpoints CLIP associates most with a given object category. Figure~\ref{fig:pure_object_queries_comparison} indicates that a PRE-TRained CLIP model is not able to distinguish specific viewpoint queries from pure object queries.

\paragraph{Comparison of Optimal Viewpoints.}
Figure~\ref{fig:clip_compare_screenshots_car} shows the viewpoint images obtained from the optima of the scoring distributions generated by a CLIP model and a CLIP-RC-HNS model. The images illustrate that descriptions of viewpoints are indeed a bias in CLIP.

Figure~\ref{fig:optimal_viewpoint_images_overview} illustrates the viewpoints resulting from the global optima of the scoring functions obtained from the CLIP-RC-HNS model.

\begin{figure*}[h]
     \centering
     \begin{subfigure}{0.9\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures/only_object_queries.png}
         \caption{Scoring function distribution on cars, motorbikes, airplanes, benches, and mugs given the query \textit{a picture of an X}, where X stands as a variable for \textit{car/motorbike/airplane/bench/mug}}
         \label{fig:pure_object_queries_comparison}
     \end{subfigure}

     \vspace{5cm}

     \begin{subfigure}{0.9\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures/comparison_clip_rchnm_optimal_viewpoints_car.pdf}
         \caption{Comparison of optimal viewpoints of the six canonical views between a) PRE-TRained CLIP and b) CLIP-RC-HNS.}
      \label{fig:clip_compare_screenshots_car}
     \end{subfigure}

     \vspace{5cm}

     \begin{subfigure}{0.9\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures/search_algorithm_comparison_greedy_bayes.png}
         \caption{A single run of the search for the respective search algorithms a) greedy, b) Bayesian, on a randomly selected car object from the ShapeNet data set~\cite{chang2015shapenet} given the search query \textit{a picture of a car from the left}.}
      \label{fig:search_trajectory_analysis}
     \end{subfigure}
     \caption{\textit{top:} Distribution on object-only queries, \textit{center:} retrieved optimal viewpoints on CLIP PRE-TR and RC-HNS, \textit{bottom:} Execution of search algorithms.}
\end{figure*}

\subsection{Search Algorithm Analysis}
\label{sec:search_algorithm_analysis_appendix}
In our work, we are particularly interested in the impact of the shape of the scoring function on the performance of various search algorithms. Section~\ref{sec:greedy_search_details} provides details on the implementation of greedy search. Section~\ref{sec:search_algorithm_behavior_on_sphere} illustrates how the search algorithms listed above perform their task on a sphere.

\subsubsection{Greedy Search Implementation Details}
\label{sec:greedy_search_details}
We implement a greedy search algorithm as a representative for gradient-based approaches.
The greedy search starts with a grid-based approach on the Goldberg polyhedron and always follows the region with the highest score. It tries to find the optimum by greedily selecting the highest scoring regions at each iteration and searching in their neighboring regions at the next iteration.
The search is initialized with $k$ randomly selected starting points (here $k=6$) from the Goldberg polyhedron.
In addition, a cutoff value $c$ must be chosen to determine how many grid points will be considered in the next iteration of the search. The cutoff value can be described as a relative percentage or as an absolute cutoff value.
After evaluating all viewpoints with respect to the given query, the next iteration is started by selecting the locations with the highest scores considering the selected cutoff.
All obtained scores and their neighboring sample points from the Goldberg polyhedron are added to the list of investigated viewpoints.
After that, the next iteration is started.
The neighborhood range $n$, which specifies the number of neighborhood grid points to be examined, can be adjusted.
The search can be terminated after $i$ iterations or when no new items have been added to the list of investigated viewpoints.
In summary, the greedy search is parameterized by: ($k,c,n,i$). We chose greedy search as a test algorithm for our benchmark to see how much gradient-based methods as candidate algorithms for the text-viewpoint retrieval task in a 3D environment depend on a smooth structure of the scoring function in their performance. We use a greedy nearest-neighbour heuristic, since the function is only defined at a fixed number of points due to the discretization of the search space.

\subsubsection{Bayesian Search Implementation Details}
\label{sec:bayesian_search_details}
Bayesian optimization~\cite{mockus1994application} is used to estimate the optimum of a black-box function that is costly to evaluate. The algorithm updates its Bayesian prior based on the stepwise function values obtained, increasing the certainty that the regions are likely to be optima and therefore more likely to be explored than other regions of the black box function. Then, the number of samples from the regions of interest is increased accordingly.
We construct the search problem as a Bayesian optimization as follows: The input of the search algorithm is a vector of size five describing the camera position on the hypersphere around the target object: $r,\theta ,\varphi, x, y $. In this parameterization, $\theta$ and $\varphi$ are spherical coordinates, $r$ is the distance to the center of the 3D object, and $x$ and $y$ are the orientations of the camera along the horizontal and vertical axes. The location of the optimum of the scoring function with respect to a query $\mathbf{q}$ depends on the rotation of the 3D object, which we only know is centered around $(0,0,0)$. Therefore, Bayesian search tries to find the optimum of the scoring function with respect to the properties of the 3D object at hand given the search query $\mathbf{q}$.
For our benchmarks, we use the implementation of the Bayesian optimization algorithm in~\citet{https://doi.org/10.5281/zenodo.1157319}.

\subsubsection{Search Algorithm Behavior on Sphere}
\label{sec:search_algorithm_behavior_on_sphere}
The experiments in Section~\ref{sec:experiments}
have shown that a smooth scoring function is advantageous for search algorithms in text-viewpoint retrieval. This section visually analyzes why this is the case by examining how the algorithms perform on a sphere around a target object.

\begin{table}
\setlength{\tabcolsep}{3.1pt}
\small
\begin{tabular}{l | ccc | cccc}
 \toprule
 Model & \textbf{P@1} & \textbf{P@5} & \textbf{P@10} & \textbf{R@1} & \textbf{R@5} & \textbf{R@10} & \\
 \midrule

 PRE-TR & 0.111 & 0.056 & 0.050 & 0.017 & 0.040 & 0.066 & \\
 FT & 0.778 & 0.567 & 0.500 & 0.113 & 0.330 & 0.485 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{car}}} \\
 RC-HNS & 0.944 & 0.778 & 0.644 & 0.136 & 0.432 & 0.592 & \\
  \midrule

 PRE-TR & 0.056 & 0.078 & 0.050 & 0.008 & 0.057 & 0.073 & \\
 FT & 0.778 & 0.500 & 0.433 & 0.112 & 0.297 & 0.424 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{airpln}}} \\
 RC-HNS & 0.833 & 0.522 & 0.439 & 0.119 & 0.310 & 0.441 & \\
 \midrule

 PRE-TR & 0.000 & 0.045 & 0.033 & 0.000 & 0.030 & 0.042 & \\
 FT & 0.500 & 0.322 & 0.339 & 0.074 & 0.217 & 0.400 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{mbike}}}\\
 RC-HNS & 0.667 & 0.500 & 0.450 & 0.098 & 0.312 & 0.462 & \\
 \midrule

 PRE-TR & 0.056 & 0.033 & 0.017 & 0.008 & 0.024 & 0.024 & \\
 FT & 0.389 & 0.311 & 0.294 & 0.056 & 0.179 & 0.286 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{mug}}}\\
 RC-HNS & 0.667 & 0.489 & 0.483 & 0.097 & 0.312 & 0.532 & \\
 \midrule

 PRE-TR & 0.000 & 0.011 & 0.006 & 0.000 & 0.008 & 0.008 & \\
 FT & 0.667 & 0.511 & 0.439 & 0.097 & 0.312 & 0.465 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{bench}}} \\
 RC-HNS & 0.944 & 0.744 & 0.689 & 0.136 & 0.411 & 0.592 & \\
 \bottomrule
\end{tabular}
\caption{Precision and recall metrics on synthetic data for the models \textit{PRE-TR, FT, RC-HNS} on the objects \textit{car, airplane, motorbike, mug, benchs} for\textit{front, back, left, right, top, bottom} viewpoints.}
\label{tab:precision_and_recall_on_synthetic_data_category_granularity}
\end{table}

Figure~\ref{fig:search_trajectory_analysis} illustrates how the different algorithms approach the regions with higher scores differently.
The greedy search with a low cutoff spreads across the sphere in waves, starting from the initial points. Once it touches a high point, it remains attached to it. In this respect, a good initialization is important, e.g., through a high number of random starting points.
Bayesian search also starts from randomly initialized starting points around the hypersphere. Compared to greedy search, it reaches the optimum much faster and more purposefully, since sampling is not bound to any local constraints, such as neighboring regions. Another advantage over greedy search is that random starting points have much lower cost than in greedy search, since they do not cause additional computations in the following iteration. The figure shows that the focus of sampling from random starting points across the sphere leads to small, concentrated regions with high scores. In terms of success rate, Bayesian search is less prone to confounding optima, since a certain number of samples are drawn randomly from different regions anyway. Therefore, the approach is more robust to cases with multiple optima, as is the case with the CLIP-FT model. Despite these obstacles, a solution is reached relatively quickly. However, if the scoring function has a ragged structure like the CLIP-PRETR model, even a sampling-based approach has difficulty identifying the optimal regions due to the raggedness and non-uniformity of the function.

\begin{table}
\setlength{\tabcolsep}{2.9pt}
\small
\begin{tabular}{l | ccc | cccc}
 \toprule
 Model & \textbf{P@1} & \textbf{P@5} & \textbf{P@10} & \textbf{R@1} & \textbf{R@5} & \textbf{R@10} & \\
 \midrule

 PRE-TR & 0.500 & 0.500 & 0.467 & 0.025 & 0.125 & 0.233 & \\
 FT & 1.000 & 1.000 & 0.967 & 0.050 & 0.250 & 0.483 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{car}}} \\
 RC-HNS & 1.000 & 0.933 & 0.950 & 0.050 & 0.233 & 0.475 & \\
  \midrule

 PRE-TR & 0.333 & 0.367 & 0.350 & 0.017 & 0.092 & 0.175 & \\
 FT & 1.000 & 1.000 & 0.917 & 0.050 & 0.250 & 0.458 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{airpln}}} \\
 RC-HNS & 1.000 & 0.833 & 0.750 & 0.050 & 0.208 & 0.375 & \\
 \midrule

 PRE-TR & 0.167 & 0.300 & 0.300 & 0.008 & 0.075 & 0.150 & \\
 FT & 0.667 & 0.633 & 0.650 & 0.033 & 0.159 & 0.325 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{mbike}}}\\
 RC-HNS & 0.833 & 0.733 & 0.783 & 0.0417 & 0.183 & 0.392 & \\
 \midrule

 PRE-TR & 0.167 & 0.167 & 0.167 & 0.008 & 0.042 & 0.08 & \\
 FT & 1.000 & 1.000 & 0.967 & 0.050 & 0.250 & 0.483 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{mug}}}\\
 RC-HNS & 0.833 & 0.933 & 0.933 & 0.042 & 0.233 & 0.467 & \\
 \midrule

 PRE-TR & 0.333 & 0.200 & 0.167 & 0.0167 & 0.050 & 0.083 & \\
 FT & 1.000 & 0.733 & 0.583 & 0.050 & 0.183 & 0.292 & \raisebox{-.05\normalbaselineskip}[0pt][0pt]{\rotatebox[origin=c]{90}{\textbf{bench}}} \\
 RC-HNS & 0.667 & 0.500 & 0.500 & 0.033 & 0.125 & 0.250 & \\
 \bottomrule
\end{tabular}
\caption{Precision and recall metrics on real data for the models \textit{PRE-TR, FT, RC-HNS} on the objects \textit{car, airplane, motorbike, mug, benchs} for\textit{front, back, left, right, top, bottom} viewpoints.}
\label{tab:precision_and_recall_on_real_data_category_granularity}
\end{table}

\subsection{Retrieval Metrics Analysis}
\label{sec:retrieval_metrics_analysis}
Table~\ref{tab:precision_and_recall_on_synthetic_data_category_granularity} shows the precision and recall metrics on \textbf{synthetic data} broken down by object category. Table~\ref{tab:precision_and_recall_on_real_data_category_granularity} shows the precision and recall metrics on \textbf{real data} obtained from the LAION-5B data set.

\section{Introduction}

These instructions are for authors submitting papers to EACL 2023 using \LaTeX. They are not self-contained. All authors must follow the general instructions for *ACL proceedings,\footnote{\url{http://acl-org.github.io/ACLPUB/formatting.html}} as well as guidelines set forth in the EACL 2023 call for papers.\footnote{\url{https://2023.eacl.org/calls/papers/}} This document contains additional instructions for the \LaTeX{} style files.
The templates include the \LaTeX{} source of this document (\texttt{eacl2023.tex}),
the \LaTeX{} style file used to format it (\texttt{eacl2023.sty}),
an ACL bibliography style (\texttt{acl\_natbib.bst}),
an example bibliography (\texttt{custom.bib}),
and the bibliography for the ACL Anthology (\texttt{anthology.bib}).

\section{Engines}

To produce a PDF file, pdf\LaTeX{} is strongly recommended (over original \LaTeX{} plus dvips+ps2pdf or dvipdf). Xe\LaTeX{} also produces PDF files, and is especially suitable for text in non-Latin scripts.
\begin{table}
\centering
\begin{tabular}{lc}
\hline
\textbf{Command} & \textbf{Output}\\
\hline
\verb|{\"a}| & {\"a} \\
\verb|{\^e}| & {\^e} \\
\verb|{\`i}| & {\`i} \\
\verb|{\.I}| & {\.I} \\
\verb|{\o}| & {\o} \\
\verb|{\'u}| & {\'u}  \\
\verb|{\aa}| & {\aa}  \\\hline
\end{tabular}
\begin{tabular}{lc}
\hline
\textbf{Command} & \textbf{Output}\\
\hline
\verb|{\c c}| & {\c c} \\
\verb|{\u g}| & {\u g} \\
\verb|{\l}| & {\l} \\
\verb|{\~n}| & {\~n} \\
\verb|{\H o}| & {\H o} \\
\verb|{\v r}| & {\v r} \\
\verb|{\ss}| & {\ss} \\
\hline
\end{tabular}
\caption{Example commands for accented characters, to be used in, \emph{e.g.}, Bib\TeX{} entries.}
\label{tab:accents}
\end{table}
\section{Preamble}
\begin{table*}
\centering
\begin{tabular}{lll}
\hline
\textbf{Output} & \textbf{natbib command} & \textbf{Old ACL-style command}\\
\hline
\citep{ct1965} & \verb|\citep| & \verb|\cite| \\
\citealp{ct1965} & \verb|\citealp| & no equivalent \\
\citet{ct1965} & \verb|\citet| & \verb|\newcite| \\
\citeyearpar{ct1965} & \verb|\citeyearpar| & \verb|\shortcite| \\
\citeposs{ct1965} & \verb|\citeposs| & no equivalent \\
\citep[FFT;][]{ct1965} &  \verb|\citep[FFT;][]| & no equivalent\\
\hline
\end{tabular}
\caption{\label{citation-guide}
Citation commands supported by the style file.
The style is based on the natbib package and supports all natbib citation commands.
It also supports commands defined in previous ACL style files for compatibility.
}
\end{table*}
The first line of the file must be
\begin{quote}
\begin{verbatim}
\documentclass[11pt]{article}
\end{verbatim}
\end{quote}
To load the style file in the review version:
\begin{quote}
\begin{verbatim}
\usepackage[review]{EACL2023}
\end{verbatim}
\end{quote}
For the final version, omit the \verb|review| option:
\begin{quote}
\begin{verbatim}
\usepackage{EACL2023}
\end{verbatim}
\end{quote}
To use Times Roman, put the following in the preamble:
\begin{quote}
\begin{verbatim}
\usepackage{times}
\end{verbatim}
\end{quote}
(Alternatives like txfonts or newtx are also acceptable.)
Please see the \LaTeX{} source of this document for comments on other packages that may be useful.
Set the title and author using \verb|\title| and \verb|\author|. Within the author list, format multiple authors using \verb|\and| and \verb|\And| and \verb|\AND|; please see the \LaTeX{} source for examples.
By default, the box containing the title and author names is set to the minimum of 5 cm. If you need more space, include the following in the preamble:
\begin{quote}
\begin{verbatim}
\setlength\titlebox{<dim>}
\end{verbatim}
\end{quote}
where \verb|<dim>| is replaced with a length. Do not set this length smaller than 5 cm.

\section{Document Body}

\subsection{Footnotes}

Footnotes are inserted with the \verb|\footnote| command.\footnote{This is a footnote.}

\subsection{Tables and figures}

See Table~\ref{tab:accents} for an example of a table and its caption.
\textbf{Do not override the default caption sizes.}

\subsection{Hyperlinks}

Users of older versions of \LaTeX{} may encounter the following error during compilation:
\begin{quote}
\tt\verb|\pdfendlink| ended up in different nesting level than \verb|\pdfstartlink|.
\end{quote}
This happens when pdf\LaTeX{} is used and a citation splits across a page boundary. The best way to fix this is to upgrade \LaTeX{} to 2018-12-01 or later.

\subsection{Citations}

Table~\ref{citation-guide} shows the syntax supported by the style files.
We encourage you to use the natbib styles.
You can use the command \verb|\citet| (cite in text) to get ``author (year)'' citations, like this citation to a paper by \citet{Gusfield:97}.
You can use the command \verb|\citep| (cite in parentheses) to get ``(author, year)'' citations \citep{Gusfield:97}.
You can use the command \verb|\citealp| (alternative cite without parentheses) to get ``author, year'' citations, which is useful for using citations within parentheses (e.g. \citealp{Gusfield:97}).

\subsection{References}

\nocite{Ando2005,augenstein-etal-2016-stance,andrew2007scalable,rasooli-tetrault-2015,goodman-etal-2016-noise,harper-2014-learning}

The \LaTeX{} and Bib\TeX{} style files provided roughly follow the American Psychological Association format.
If your own bib file is named \texttt{custom.bib}, then placing the following before any appendices in your \LaTeX{} file will generate the references section for you:
\begin{quote}
\begin{verbatim}
\bibliographystyle{acl_natbib}

redpajama_set_name: RedPajamaArXiv


You are visiting the placeholder page for Joel Aguilar. This page is here because someone used our placeholder utility to look for Joel Aguilar. We created this page automatically in hopes Joel Aguilar would find it. If you are not Joel Aguilar, but are an alumni of Nogales High School La Puente, CA, register on this site for free now.


text:
Home us Sher Bahadur Deuba
Sher Bahadur Deuba
KATHMANDU: President Bedia Devi Bhandari has appointed Congress President Sher Bahadur Deuba as Prime Minister on the orders of a Nepali court, using his powers under Article 76 (5) of the Constitution.
According to the international news agency, Nepali Congress President Sher Bahadur Deuba became the Prime Minister of the country for the fifth time. Nepal's President Bedia Devi Bhandari has appointed 72-year-old Sher Bahadur as the country's new prime minister under Article 76 (5) of the constitution.
Just yesterday, the country's Supreme Court quashed a long-running political crisis by Prime Minister Sharma Oli's May 21 move to dissolve parliament and ordered Prime Minister Sharma Oli to step down if he failed to prove a majority. Was
The Supreme Court had suggested to President Bedia Devi Bhandari the name of Sher Bahadur, a seasoned politician and four-time Prime Minister of the country, for the new Prime Minister, but it was not immediately known when the swearing-in ceremony would take place.
The new Prime Minister of the country will have to get a vote of confidence from the House within a month after taking oath. Sher Bahadur Divya was the Prime Minister of the country from 1995 to 1997, 2001 to 2002, 2004 to 2005 and 2017 to 2018.
Mia Khalifa New Photo Shoot
Los Angeles: Mia Khalifa's new photoshoot has caused a stir on the internet. The photos shared on Instagram have been viewed by millions of...
Jose Butler ruled out of the Ashes series after a broken...
Nintendo Gaming Platform: Switch Consoles, Switch Online Subscription and Exclusives
Angle and special adapters for custom water cooling
Modernization of the video card cooling system – down with noise...
Buy and Eat the Mona Lisa from Amazon
Make your own digital outfit with a 3D printer now!
More than just a TV: why Smart TV is cool
Joe Biden Meets Vladimir Putin

redpajama_set_name: RedPajamaCommonCrawl


text:
Mithion rubricoronatus är en spindelart som beskrevs av Embrik Strand 1911. Mithion rubricoronatus ingår i släktet Mithion och familjen hoppspindlar. Inga underarter finns listade i Catalogue of Life.

Källor

Hoppspindlar
rubricoronatus

redpajama_set_name: RedPajamaWikipedia


text:
Fahad Nasser Khamees Mubarak () (né le  à Dubaï, à l'époque dans les États de la Trêve, aujourd'hui aux Émirats arabes unis) est un joueur de football international émirati, qui évoluait au poste d'attaquant.

Son frère, Nasir, était également footballeur.

Biographie

Carrière en club

Carrière en sélection

Avec l'équipe des Émirats arabes unis, il joue 82 matchs (pour 37 buts inscrits) entre 1981 et 1993. Il figure notamment dans le groupe des sélectionnés lors de la Coupe du monde de 1990. Lors du mondial, il joue deux matchs : contre la Colombie et la Yougoslavie.

Il dispute également les Coupes d'Asie des nations de 1984 et de 1988.

Il joue enfin 12 matchs comptant pour les tours préliminaires des coupes du monde 1986, 1990 et 1994.

Palmarès
 Al Wasl

Notes et références

Liens externes

Naissance à Dubaï
Naissance en janvier 1962
Footballeur international émirati
Joueur de l'Al Wasl Dubaï


Volunteers will work within the Jacksonville Zoo and Gardens.
After 12 noon, volunteers are invited to stay and enjoy the zoo for the remainder of the day!
We are scheduled to work outside the Zoo's new pub location. Our tasks may include shoveling dirt & moving mulch, planting flowers, or preparing and painting. The Zoo serves all of Northeast Florida, and offers Jacksonville Spartans the opportunity to do the same.
Please wear appropriate clothing for the weather, and clothing that can get dirty! The work may be messy and physical. Wear long pants and closed-toe shoes. Bring work or garden gloves, sunblock, hat, boots, rain poncho, umbrella, so you are prepared for sun and/or rain. Also consider bringing extra water and a pre-packaged snack. If you require an accommodation, have a serious allergy, etc, please contact us.
This service project is organized by an alumni club: Jacksonville Spartans.
If you would like to volunteer, please contact Toni Chrabot at jaxspartans@gmail.com or 904-366-9993 .


Description: Experts and professionals from Careem, Protiviti, Entrepreneur Middle East, ASUS, IdeateLabs and Focus Social Academy were part of the exclusive Digital Conclave organized by S P Jain School of Global Management, Dubai Campus. The focus of the conclave was centered around the topic of "Era of Digital Disruption-Transforming the Middle East".
Thursday, 10th August, experts from both the academic and digital worlds converged to discuss and analyze how the Middle East is transforming with the digital disruption wave. Covering a wide range of digitalization subjects from media to the social media consumer, the event addressed the hottest challenges the Middle East and how Dubai is transforming to become the region's digital lab.
The half day forum was coordinated by Dr. Anna Tarabasz, Assistant Professor, Digital Marketing & Metrics Program - S P Jain School of Global Management. Prof. Christopher Abraham, Head of SP Jain School of Global Management, Dubai Campus and Sr. Vice-President, started the day's proceedings with a keynote address that highlighted the Digital Trends that are transforming the region. This was followed by thought provoking presentations from industry experts like Ahmed Gowda from ASUS, Anas Almarie from Focus Social Academy and Ideatelabs founder Amit Tripathi.
The session continued with a lively panel discussion moderated by Dr. Anna Tarabasz challenging the industry practitioners. Representing the digital media front was Entrepreneur Middle East Editor in Chief, Aby Sam Thomas while the financial industry was represented by Protiviti, along with Careem and Focus Social Academy to find the relevant answers to the digital challenges to the current professionals in the Middle East.
The digital forum discussed in-depth on the subject of the gap that is evident between the digital workforce and the market's demand for specialized digital skills.
Based on the latest research, Anas Almarie from Focus Social Academy presented numbers that showed the continuous growth of Middle East in the digital space. Compared to the United States, the research showed how the ME region is breaking down its own digital talent: UAE 2.0%, Egypt 1.9%, Bahrain 1.9%, Oman 1.6% and Saudi Arabia 0.9% compared with USA 3.8% and Europe 3.7%.
The Digital Disruption is changing the skill map of the region into a mix of skills, talents, and new jobs creation. Based on eminent market requirements, S P Jain is aligning its professional programs to respond to the skills gap with the launch of its Digital Marketing and Metrics program starting 8th September 2017. The Innovative and cutting-edge Digital Marketing & Metrics program aims to bridge the gap between theoretical knowledge and practical applications. This end-to-end, practice based, professional program, delivered by and under the mentorship of seasoned industry experts is designed to develop modern marketing, and market ready specialists, who are capable of thinking outside the box, and strategizing to face the overwhelming challenges of the digital wave.
The first edition of the conclave "Era of Digital Disruption-Transforming Middle East" has generated more than 1 million twitter impressions and an increased interest from a diverse audience covering UAE, Saudi Arabia, India, and out of the region countries such as UK, Germany, and Italy showing the interest of professionals across different regions in joining the digital development efforts of the UAE and the region.


text:
Арту́р Андре́евич Гачи́нский (род. 13 августа 1993, Москва) — российский фигурист, выступавший в одиночном катании. Бронзовый призёр чемпионата мира (2011), серебряный призёр чемпионата Европы (2012) и двукратный серебряный призёр чемпионата России (2011, 2012).

Мастер спорта России международного класса. По состоянию на апрель 2012 года занимал пятое место в рейтинге Международного союза конькобежцев.

Карьера
Артур катался на роликах до шести лет, но вскоре переключился на фигурное катание. Его семья переехала из Москвы в Санкт-Петербург, когда ему было девять лет. Тренировал его Алексей Мишин.

На дебютном чемпионате России среди юниоров, в 2006 году, стал бронзовым медалистом. В 2007 году Федерация фигурного катания России разрешила 13-летнему Артуру принять участие во «взрослом» чемпионате России. На тот момент это было беспрецедентное решение, ранее в чемпионатах России не выступали столь молодые спортсмены. Артур стал 14-м из 18 участников.

В своём первом международном сезоне, 2007—2008 годов, отобрался в финал юниорского Гран-при, где занял восьмое, последнее, место. На чемпионате России был 9-м.

В сезоне 2008—2009 снова был 8-м в финале Гран-при. На чемпионате России стал 10-м, а на юниорском чемпионате страны завоевал серебряную медаль. Был заявлен для участия в чемпионате мира среди юниоров 2009 года, но за один день до турнира снялся из-за простуды.

В олимпийский сезон 2013/2014 годов Артур поменял тренера. В начале 2015 года он выиграл бронзовую медаль на Универсиаде в Испании. В конце сентября он получил травму. Был в числе заявленных на очередной национальный чемпионат, но в последний момент снялся. Уже в ходе чемпионата 24 декабря 2015 года объявил о завершении карьеры.

Программы

Спортивные достижения

С сезона 2013 / 2014

До сезона 2013 / 2014

J = юниорский уровень

Примечания

Ссылки

Родившиеся в Москве
Фигуристы России


Work | gish&co. Work | gish&co.
We've also had the honor of working with: Ampers, Asure Software, Be the Match Foundation, Bush Foundation, C2 Communication + Design, Cole's Salon, Damage Help, Financial Navigation Group, GiveMN, Growth Mode Marketing, IFP Minnesota, JKR Consulting, Minneapolis Downtown Improvement District, Nonprofits Assistance Fund (now Propel Nonprofits), Pet Haven, Outlook Clinics, Pet Project Rescue, Slice Consulting, University of Minnesota Foundation, and more.
"gish&co. makes it easy to work with them. They understand both our goals and limitations and work to find creative ways to create exactly what we need. Plus, they are connected to the broader community and have connected us with many great partners we wouldn't have reached on our own.
Do you have an idea for a potential collaboration? Want to know if working with gish&co. might be right for you? Get in touch! We'd love to meet you.


text:
<?php

namespace app\models;

use Yii;

use app\models\general\GeneralLabel;
use app\models\general\GeneralMessage;

/**
 * This is the model class for table "tbl_ltbs_minit_mesyuarat_jawatankuasa_dokumen_muat_naik".
 *
 * @property integer $dokumen_muat_naik_id
 * @property integer $mesyuarat_id
 * @property string $nama_dokumen
 * @property string $muat_naik
 */
class LtbsMinitMesyuaratJawatankuasaDokumenMuatNaik extends \yii\db\ActiveRecord
{
    /**
     * @inheritdoc
     */
    public static function tableName()
    {
        return 'tbl_ltbs_minit_mesyuarat_jawatankuasa_dokumen_muat_naik';
    }

    public function behaviors()
    {
        return [
            'bedezign\yii2\audit\AuditTrailBehavior',
            [
                'class' => \yii\behaviors\BlameableBehavior::className(),
                'createdByAttribute' => 'created_by',
                'updatedByAttribute' => 'updated_by',
            ],
            [
                'class' => \yii\behaviors\TimestampBehavior::className(),
                'createdAtAttribute' => 'created',
                'updatedAtAttribute' => 'updated',
                'value' => new \yii\db\Expression('NOW()'),
            ],
        ];
    }

    /**
     * @inheritdoc
     */
    public function rules()
    {
        return [
            [['nama_dokumen'], 'required', 'skipOnEmpty' => true, 'message' => GeneralMessage::yii_validation_required],
            [['mesyuarat_id'], 'integer', 'message' => GeneralMessage::yii_validation_integer],
            [['nama_dokumen'], 'string', 'max' => 80, 'tooLong' => GeneralMessage::yii_validation_string_max],
            [['muat_naik'], 'string', 'max' => 100, 'tooLong' => GeneralMessage::yii_validation_string_max],
            [['muat_naik'],'validateFileUpload', 'skipOnEmpty' => false],
            [['nama_dokumen'], function ($attribute, $params) {
                if (!\common\models\general\GeneralFunction::validateXSS($this->$attribute)) {
                    $this->addError($attribute, GeneralMessage::yii_validation_xss);
                }
            }],
        ];
    }

    /**
     * @inheritdoc
     */
    public function attributeLabels()
    {
        return [
            'dokumen_muat_naik_id' => GeneralLabel::dokumen_muat_naik_id,
            'mesyuarat_id' => GeneralLabel::mesyuarat_id,
            'nama_dokumen' => GeneralLabel::nama_dokumen,
            'muat_naik' => GeneralLabel::muat_naik,

        ];
    }

    /**
     * Validate upload file cannot be empty
     */
    public function validateFileUpload($attribute, $params){
        $file = UploadedFile::getInstance($this, $attribute);

        if($file && $file->getHasError()){
            $this->addError($attribute, 'File error :' . Upload::getUploadErrorDesc($file->error));
        }

        if($file){
            if(!GeneralFunction::checkFileExtension($file->getExtension())){
                $this->addError($attribute, GeneralMessage::uploadFileTypeError);
            }
        }

        if(!$file && $this->$attribute==""){
            $this->addError($attribute, GeneralMessage::uploadEmptyError);
        }
    }
}

redpajama_set_name: RedPajamaGithub


Named for a famed hostess who entertained the Bloomsbury set, the Ottoline chandelier creates a fitting festive atmosphere whether placed above a dining room table or holding court in a formal living room. A frame in a polished brass finish sports blossoming flowers that surround the eight lights on this chandelier. The unfurling petals that bloom from the composition have been treated to a black bronze finish to make them gleam as they strike their pert poses. The Ottoline, which measures 36? in diameter by 24 ¾? high, also comes in a pendant.


You are such an idiot. Do you really believe half of the goody- two-shoes advice you give? Sometimes you don't even give them advice, you just gloss over the problem with pablum in print. And that (deleted) about telling you what people are grateful for on Thanksgiving! It is none of your business. I sure as (deleted) don't want anyone to know what is going on in my life. If I did, I would have a Facebook page. I am not some crazed teenager or a wacko like on Dr. Phil or Jerry Springer. I am a very private person who does not care what other people have to brag about. I am definitely not reading your Thanksgiving (Bovine by-product).
Grandpa says: Then shall I print you are grateful that The Voice gives you the opportunity to express yourself anonymously?
My wife and I spend our Winters in (a warmer state). I have a trusted friend from church who watches my house and gathers my mail. She has a key in case of emergency. This past Summer I had a camera system installed because of my wife's health conditions. I can see what is going on in other rooms or if I am not home and she is alone. Well, I saw my friend in our house snooping through our closets and pantry. My wife tells me to just let it be. I want to call her and demand an explanation. We have agreed to abide by your recommendation on how to address this.
Why start a situation that you cannot handle from afar? Any damage that might be done is already done. Relax and enjoy the Winter. In the Spring, when you are opening up the house, simply change the locks, even if the neighbor returns the key. Next Autumn, don't give the key to anyone. Go to a home improvement store and spend about $25 on a key box that realtors use. Put the key in that box and tell your house sitter that in an emergency she can call for the key code. If you don't trust her to not get the key copied, you can put on the door one of those locks that uses a code to open the lock.
Got something stuck in your craw? Ask Grandpa. Address your letters to Ask Grandpa c/o The Voice, 314 N. Lake Street, Suite 2, Aurora, IL 60506 or send an E-mail to askgrandpa@thevoice.us.


SEC. 8020. PROHIBITION REGARDING STATE AID.
``SEC. 9522. <<NOTE: 20 USC 7902.>> PROHIBITION REGARDING STATE AID.
State aid, with respect to free public education of children.
``SEC. 9523. <<NOTE: 20 USC 7903.>> PRIVACY OF ASSESSMENT RESULTS.


Did you know that we've been taking care of our community for 18 years?
The UW Neighborhood Shoreline Clinic provides a full range of healthcare services, including preventive medicine and treatment for existing problems. Our healthcare team consists of physicians specializing in family medicine, family medicine with obstetrics, internal medicine, pediatrics and obstetrics-gynecology, as well as advanced registered nurse practitioners, physician assistants and a registered dietitian.
Join our mailing list to receive future emails about events and updates associated with the UW Neighborhood Shoreline Clinic.
Dr. Atkins is a board certified family medicine physician who practices at the UW Neighborhood Shoreline Clinic.
Dr. Bagan is a board certified family medicine physician at the UW Neighborhood Shoreline Clinic.
Dr. Chilcote is a board certified pediatrician who sees patients at the UW Neighborhood Shoreline Clinic.
Dr. Chopra is a board certified internal medicine physician who sees patients at the UW Neighborhood Shoreline Clinic.
Dr. El-Attar is a board certified family medicine physician and the clinic chief at the UW Neighborhood Shoreline Clinic.
Dr. Jenkins is a board certified family medicine physician who sees patients at the UW Neighborhood Shoreline Clinic.
Neysa Koury is a certified physician assistant who sees patients at the UW Neighborhood Shoreline Clinic.
Dr. O'Kane is a board certified family medicine physician who sees patients at the UW Neighborhood Shoreline Clinic.
Dr. Roskam is board certified in internal medicine physician who sees patients at the UW Neighborhood Shoreline Clinic.
Dr. Shen is a board certified family medicine physician who sees patients at the UW Neighborhood Shoreline Clinic.
Dr. Volkman is a board eligible internal medicine physician who sees patients at the UW Neighborhood Shoreline Clinic.
Dr. Emmons is a board certified family medicine physician who provides urgent care services at the UW Neighborhood Shoreline Clinic.
Dr. Tubbs is a board certified family medicine physician who provides urgent care services at the UW Neighborhood Shoreline Clinic.
Dr. Bar-Joseph attended Brown University for undergraduate work and the University of Washington for medical school.
Dr. Shope is a UW clinical assistant professor in the Department of Obstetrics and Gynecology.


Let Chrome find Twitter users to follow on any website.
No more searching through webpages to find that @followme link. When Bird Spotter finds any Twitter users it will notify you with the little bird in your omnibar. Just click the bird to see them all.


I am a man, you a woman.
Your soul is built for diff'rent goals.
I am earth, you are air and breeze.
You glide above my planted feet.
you pin them, like dead butterflies.
old cages trap new emotions.
I write poetry, you write prose.
there were no space between us.
unmake the space between us.
This poem is in response to wewritepoems prompt 51, pairings, and stimulated by a line from DH Lawrence, Women in Love, which I was reading in the bath yesterday ("There is a space between us"). His one line is more expressive, in its context – Gudrun and Gerald in a boat, falling in love – than my entire poem, I feel.
and falling petals meet the rising scent of dew.
and lilac hands outstretched caress the watching air.
and buds open their hearts and cry 'begin'.
and lambs look up to watch your gentle dance.
his dappled shade stroking your snowdrop arms.
and fairy maids arrive to offer fruit.
creatures awake and humans call their names.
you say our time is now – it's come again.
This is a little twee and romantic, but I guess it's something to do with the weather, or the fairies. Submitted to Thursday Poets' Rally.
Unexpectedly, the room stopped moving.
She sat complete and waiting in her chair.
so he chose 'may I have a hug?' instead.
The welcome of her smile emptied his heart.
He felt unique, alone yet not apart.
"Yes," she said, again, and her eyes lit up.
He rose, in joy, and crossed the warming floor.
This poem is a second contribution to Poets United Thursday think tank – your first time… I intended a longer poem, but then this seemed just the right place to stop.
on signs along the shore.
and one toe touched bare rock.
and the earnest sparrows sang.
holding the world at bay.
his thighs embraced the air.
ospreys rose and scanned for prey.
and wide-eyed lambs looked on.
and plunged into the bay.
I've been walking and cycling around Rutland Water, and imagining a first act of rebellion maybe (for Poets United).
This is a response to wewritepoems prompt 49, Almost a poem, which asks us to write about the process of making a poem. It's also inspired by Elizabeth Crawfords's poem 1 thru 12 The true story, which in turn follows a structure from Kim Nelson. Instead of adding one syllable per line I've doubled the number each line, so 1 – 2 – 4 – 8 and then back again. The overall idea of this poem is the water cycle, of rain falling, forming pools, rivers flowing into oceans and then evaporating back to clouds and the whole process starting again.
and the screens in the kitchen all showed blue.
across the park to catch the bus, no fuss.
for several years, she found, eating her buns.
she had discovered something new and true.
But people died sometimes, and others flew.
quite tough, she sticks to poems, and has retired.
A tongue in cheek (?) response to wewritepoems prompt 48, the art of making fire, based on a real life discussion with my daughter about her history assignment -"What was the cause of the Cold War?". Her essay took a different tack.


A keystone program of the Impact Entrepreneurship Initiative, the Venture Lab is designed to support graduate students and upper undergraduates at The New School in developing financially sustainable ventures that address critical societal challenges and demonstrate the potential for scalable impact. Students accepted to the Venture Lab enroll in a semester-long course offered in the spring semester, for which they will receive academic credit. The Venture Lab course will be taught by Jen van der Meer, assistant professor of strategic design and management, Parsons School of Design, and the Impact Entrepreneurship Team, which includes Michele Kahane and Elizabeth Werbe. Please contact professor van der Meer at vandermj@newschool.edu to request permission to register.


text:
The Skyway Light is a German paramotor that was designed and produced by Skyway Products of Ettenheim for powered paragliding. Now out of production, when it was available the aircraft was supplied complete and ready-to-fly.

Design and development
The aircraft was designed to comply with the US FAR 103 Ultralight Vehicles rules as well as European regulations. It features a paraglider-style wing, single-place accommodation and a single  Skyway T170 engine in pusher configuration with a 2.8:1 ratio reduction drive and a  diameter three-bladed composite propeller. The fuel tank capacity is . The aircraft is built from a combination of bolted aluminium and composite material, with the engine mount and fuel tank made from composites.

As is the case with all paramotors, take-off and landing is accomplished by foot. Inflight steering is accomplished via handles that actuate the canopy brakes, creating roll and yaw.

Specifications (Light)

References

 Light
2000s German ultralight aircraft
Single-engined pusher aircraft
Paramotors


Mother Elmar Green was the 10th of thirteen children born to the union of John and Carrie Cooks on May 20, 1931 in Madison, FL. She attended public school in Madison County. When she reached adulthood, she moved to Jacksonville, FL, where she met and married Frank Green. To this union 7 children were born. They later settled in Daytona Beach, FL.
In 1955, Mother Green was a founding member of Madison Heights Church of God In Christ. She was well known for her service as the President of the Usher Board at Madison Heights COGIC and as President of the St. Augustine District Usher Board. Years later she was installed as Church Mother.
Mother Green worked as a cafeteria manager for Volusia County School Board for 29 years. She was also the first African American Cashier for Sears and Roebuck in Daytona Beach FL and worked there for 18 years. After retirement, she worked part-time at Publix grocery store for 8 years.
On the morning of November 18, 2017, Mother Elmar Green lifted her hand, waved and peacefully entered into her eternal rest. She leaves to cherish her memory; Children, Alveda Miller, Daytona Beach FL, Carrie Green, Pembroke Pines FL, Elder Frank Green (Roberta), Tallassee, AL, Constance Green, Daytona Beach FL, Anthony Green, Daytona Beach, FL and Elder Anita Wilson, Pembroke Pines FL; 15 grandchildren, 21 great grandchildren one great great grandchild; and a host of nieces and nephews and other relatives and friends.


text:
Ferrazzano ist eine italienische Gemeinde (comune) mit  Einwohnern (Stand ) in der Provinz Campobasso in der Region Molise. Die Gemeinde liegt etwa 3 Kilometer südsüdöstlich von Campobasso.

Geschichte
Das antike Ferentium wurde 296 v. Chr. gegründet.

Einzelnachweise

Weblinks

Ort in Molise
Weinbauort in Italien


Dr. George S. Tellam has provided foot and ankle care in north Florida and southern Georgia since 1986. He is an honors graduate of the University of Southern California and the California College of Podiatric Medicine in San Francisco. His practice offers 3 convenient locations throughout the Jacksonville area.
In addition, he is a proud partner in Center One Surgery Center of Jacksonville. Visit centeronejax.com for more information on one of the areas best physician owned multispecialty ambulatory surgery centers.
Dr. Tellam is always looking for new treatment options for his patients when traditional treatment plans are ineffective in relieving his patient's pain. Our three state of the art facilities offer the newest technologies in foot and ankle care including; on site digital x ray system, sophisticated and precise advanced orthotic treatment, visit sensormedicausa.com/ to learn more, diagnostic ultrasound, latest treatment using human connective tissue matrix (tissue regeneration and or replacement) see celularity.com/, EPAT (extracorporeal pulse activation technology) treatments, see www.curamedix.com for the latest, ACP (platelet rich plasma) injections visit arthrex.com/ as well as a host of traditional pain relief treatments and surgical options.
You will find our team friendly, accommodating, professional and most importantly looking forward to treating or eliminating your toe, foot, ankle or heel pain or injuries. We respectfully strive to exceed all of your expectations by offering same day appointments, and pledge to address all of your concerns in a timely, ethical, and cost-effective manner. We look forward to helping you live pain free.
Don't delay call us today 904-269-9595 for an appointment.
Copyright (c) 2013 Parson Technologies, Inc. All rights reserved.


Android's app market is growing by the minute. With Google churning out applications, it is focused on maintaining their market foothold. Android apps range from useful utilities, addons, games, entertainment and the list doesn't end there.
We offer a review to the 3 coolest apps.
Wapedia gives you access to Wikipedia and other wiki sites, claiming to have about 3 million articles and counting. The speed is incredible, even on simple 2G networks. Language support is provided for travellers.
This is not your basic camera application on your phone, far better than that! Fxcamera is loaded with features such as ToyCam, Polandroid, Fisheye, SymmetriCam, Warhol and normal modes. The application makes self-potraits a lot more fun and exciting and supports flash.
3. Evernote – Making Note Taking fun!
The note taking application is not included in the Android phones so they need to be downloaded from the Android App store. Evernote is still in Beta testing but it has the ability to snyc notes, pictures and even videos within and with other Androids. More to come when it is released.
More than 100,000 Android Apps available!


text:
Submarine force leaders: Expect a fight each year
Published October 24. 2013 12:01AM | Updated October 24. 2013 12:17AM
By Jennifer McDermott Day Staff Writer
Falls Church, Va. - Getting the money to buy two Virginia-class submarines per year will be "a constant battle," Vice Adm. Michael J. Connor said Wednesday.
"We win it every year," Connor, the commander of the submarine force, said. "Winning it one year does not guarantee that we win it the next year. That will go on as long as I'm in the Navy."
A second submarine is in the plans for 2014, Connor said, but the production rate could go down to one in future years.
Connor addressed Navy leaders, retired submariners and business executives at the Naval Submarine League's 31st annual symposium. Many of the speakers focused on the question of how to design, build and test submarines in an austere, unpredictable fiscal environment.
Connor told the group his first priority is building a new class of ballistic-missile submarines and missile systems to replace the aging Ohio-class boats. Next, he said, is buying two Virginia-class submarines per year, followed by developing new weapon and sensor payloads and unifying the undersea domain.
"I hope to not have to pick and choose among these but if we do, I'll pick from the top down," he said.
Electric Boat in Groton and its shipbuilding teammate Newport News Shipbuilding in Virginia build the attack submarines. EB is designing the new ballistic-missile submarines.
While some were optimistic these programs would stay on track, Rear Adm. Richard P. Breckenridge, director of undersea warfare, said as the "budget guy" in the group, he was going to "temper this attitude of optimism a little bit and talk about the real future."
"It's pretty bleak, I've got to let you know," he said. "… We've got some perilous times ahead. The bridge is out. The flashing signs are saying 'slow down.'"
Breckenridge said he recently submitted the next budget, and while the submarine force came through the process in "fairly good shape" compared to other areas of the Navy, the budget now needs approval from people who may have different priorities. The Ohio-replacement program was kept intact, even accounting for the cuts under sequestration, he said.
Breckenridge said he believes about $4 billion per year should be added to the Navy's shipbuilding account in the future, when the ballistic-missile submarines are under construction, and the program must not be delayed any longer to save money.
The submarine force waited a decade to increase the production rate of attack submarines to two per year, and now the number in the fleet will drop from 55 to as low as 40. Breckenridge said any further "belt tightening" would be "absurd."
Adm. John M. Richardson, the director of the naval nuclear propulsion program, said the uncertainty surrounding the budget and the cuts so far have been "a tremendous distraction" that has "almost consumed" senior managers.
When the requirements for new submarines are discussed, Richardson said, questions are asked, such as, "Does it need to go that fast, dive that deep, be that quiet," "How many torpedo tubes does it need or does it need a torpedo room," and "How many ships do we really need?"
Through the symposium, it is important to ensure that leaders in government and the private sector understand the decisions that are potentially trading away performance and the risk of doing so, Richardson said.
"We start with the technically correct answer, realistic schedule and realistic cost estimate. If we make a change from that starting point, we should be clear-minded about what we're giving up, the risks we're incurring, and be deliberate in our decision-making," he said.
Otherwise, the submarine force could end up with a broken program, Richardson said.
Connor said he believes the submarine force can maintain all of its priorities, if it "fights very hard."
j.mcdermott@theday.com
Fiscal impasse threatens engineering jobs at EB
Poitras to retire; Bath Iron Works president to lead EB
Courtney: House vote preserves two subs plan
Lawmakers, others looking at long-term sequestration effects
Navy to Congress: No budget, no second submarine in 2014
Congress supports building two submarines per year