after finishing a commission, i got down to my next. this is my start!
it's another Canada landscape painting- rocks and water with very little foliage. so i am going to get alot more practice on water.
for this one, i'm working from 5 different pics i took when in Canada. i'm also using some artistic license.
this is being done on Fabriano Artistico 300 lb rough and measures 22.5 X 22.5 inches.
i'm mostly doing washes right now using my Kolinsky sable, series 3 3/4 and Simmons A 83 12 brushes. i masked out a small area of grass that might become larger but i'm not sure yet.
colors so far are DS undersea green mixed with DS cobalt blue, DS cobalt blue, DS aureolin and mix with DS Naphthamide Maroon.
thanks for looking! i'll post an update again tomorrow.
I'll settle in quietly next to Lyn while we watch you do your lovely magic with the watercolors.
I am always facinated by Sarah's wips.
Am watching breathlessly, with all the others.
So am I. Looking good so far Sarah!
painting the rocks are going to be fun! this size painting is definitely small for me too.
it's moving along pretty well so far. for the rocks, i'm doing washes and waiting for alot of drying time. i layer like Doug does (i think he does) so it definitely is possible. you just have to wait alot.
for detail in the rocks, i will probably do some dry brush with my spotter.
hi Ellen and thanks for looking!
i double checked the horizon line and it's good now.
you definitely have a great eye!
hi Lyn (((hugs))) and thanks for looking!
get some tea! this is going to be a little while longer. my semester started and i'm off and running!
here's an update. i'm using the same colors as started with no new additions. i'm still doing washes for the most part.
I like the way you have done the water Sarah - it really looks liquid.
Ditto to what Doug just said.
I can see the rocks coming together already.
I am sittting on the floor right up close. Watching the way you are going to paint those rocks.
Looking forward to the next steps.
Really well done Sarah!! I'm enjoying this. First time I've been able to view it. Must have so many viewing at once that its hard to get in. This is my fourth try over three days. ;) :clap: Gooooo Girl!!
very nice, I can feel the movement of the water. thanks for sharing.
Wow, I will be following this too! Would you be able to post your 5 ref pics? I'm always interested to see how others combine pics & translate into a painting. Thanks!
Sarah, I am looking forward to seeing more of this as well. Your water is already looking great.
Can't wait for the rocks.
Your water already looks drinkable!!
hi Emilie ((hugs)) and thanks for looking!
i was away and i didn't bail out on this WIP. i wanted a weekend at the shore to de-stress. i did but i think i have more stress now that i'm back with all i need to do :eek: i'll get it done though.
i started the rocks yesterday, almost as soon as i got home. i'm doing washes but also adding some fine detail now too. i'm trying some more dry brush as well as it's working out well.
and if you can give me until tomorrow night, i can get them all uploaded here.
hi Becky and thanks for looking! i actually had trouble getting to the site Thur. night, before i left. weird!
hi Gail and thanks for looking! this is also Lake Rideau, right by our cabin! the sunset was beautiful and the rocks even looked more amazing! your country is awesome!
for the rocks, the colors i'm using are WN caput, DS naph maroon, DS aureolin, DS cobalt blue, DS sepia.
hi Sylvia ((hugs))) and thanks for looking again!
the horizon is definitely fixed!
hi Doug and thanks (((hugs))) the water came along well and i fixed that area in the center that was making me crazy!
here's an update and comments and crits appreciated.
Great start -- I'll be watching!
This is really coming along beautifully, Sarah! Your water is perfect... I especially like the transparency of it where it meets the rocks... the sandy bottom is really well done!
i finished the painting last night and am giving it to my friend tomorrow. this was a commission so it's all framed and ready to go!
thanks again everyone for watching.
This is really lovely. I'm sure your friend is going to be delighted with it.
I assume your photo is tilted (horizon).
Your friend will be delighted - the water is wonderfully captured.


Please use the active links below to get information for the group you are interested in. They are email links to the person that can give you the most current information. They will respond as soon as possible.


text:
\section{Introduction}
In recent years, contextualized embeddings have become increasingly important.
Embeddings created by the BERT model and its variants have been used to get state-of-the-art performance in many tasks \citep{devlin-etal-2019-bert, liu2019roberta, yang2019xlnet, radford2018gpt, radford2019gpt2,brown2020gpt3}.
Several multimodal-BERT models have been developed that learn multimodal contextual embeddings through training jointly on linguistic data and visual data \citep{lu2019vilbert, su2019vlbert, li2019visualbert, chen2020uniter}.
They achieve state-of-the-art results across many tasks and benchmarks, such as Visual Question Answering \citep{balanced_vqa_v2}, image and text retrieval \citep{lin2014microsoft}, and Visual Commonsense Reasoning \citep{suhr2018corpus}.\footnote{From here on we refer to the text-only BERT models as 'BERT' and the multimodal-BERT models as 'multimodal-BERTs'.}

BERT and multimodal-BERTs are blackbox models that are not easily interpretable.
It is not trivial to know what knowledge is encoded in the models and their embeddings.
A common method for getting insight into the embeddings of both textual and visual content is probing.

Language utterances have an inherent grammatical structure that contributes to their meaning.
Natural images have a characteristic spatial structure that likewise allows humans to interpret their meaning.
In this paper we hypothesize that the textual and visual embeddings learned from images that are paired with their descriptions encode structural knowledge of both the language and the visual data.
Our goal is to reveal this structural knowledge with the use of probing.
More specifically, in order to perform this probing, we first make the inherent structure of language and visuals explicit by a mapping between a dependency parse of the sentences that describe the image and by the dependency between the object regions in the image, respectively.
Because the language truthfully describes the image, and inspired by \citet{draschkow2017scene}, we define a visual structure that correlates with the dependency tree structure and that arranges object regions in the image in a tree structure.
We call this visual dependency tree the \textit{scene tree}.
An example of this mapping to the scene tree is visualized in Figure~\ref{fig:tree_mapping}.

\begin{figure}[t]
    \centering
    \includegraphics[width=\linewidth]{figures/visualize-1077546505_1.png}
    \caption{Example of the mapping from the linguistic dependency tree to the visual tree. The borders of the regions in the image have the same color as the phrase they are attached to. The rows below the image are the textual tree depth (in black), the visual tree depth (in red), the phrase index, and the words in the sentence.}
    \label{fig:tree_mapping}
\end{figure}

The aligned dependency tree and scene tree allow us to
conduct a large set of experiments aimed at discovering encoded structures in neural representations obtained from multimodal-BERTs.
By making use of the structural probes proposed by \citet{hewitt-manning-2019-structural}, we compare the dependency trees learned by models with or without provided image features.
Furthermore, we investigate if scene trees are learned in the object region embeddings.

\paragraph{Research Questions}
In this study, we aim to answer the following research questions.
\begin{itemize}
    \item \textbf{RQ 1:} Do the textual embeddings trained with a multimodal-BERT retain their structural knowledge? \\
    \textbf{Sub-RQ 1.1:} To what extent does the joint training in a multimodal-BERT influence the structures learned in the textual embeddings?
    \item \textbf{RQ 2:} Do the visual embeddings trained with a multimodal-BERT learn to encode a scene tree?
\end{itemize}

In a broader framework this study might contribute to better representation learning inspired by how humans acquire language in a perceptual context.
It stimulates the learning of representations that are compositional in nature and are jointly influenced by the structure of language and the corresponding structure of objects in visuals.

\section{Related Work}
\paragraph{Probing studies}
Several studies have been performed that aim at analyzing BERT and multimodal-BERTs. For BERT, probes are designed that explore gender bias \citep{bhardwaj2021probe_gender_bias},
relational knowledge \citep{singh-etal-2020-bertnesia},
linguistic knowledge for downstream tasks \citep{liu-etal-2019-linguistic},
part-of-speech knowledge \citep{hewitt2019designing,hewitt-etal-2021-conditional},
and for sentence and dependency structures \citep{tenney2019structure,hewitt-manning-2019-structural}.
These studies have shown that BERT latently learns to encode linguistic structures in its textual embeddings.
\citet{basaj2021explaining} made a first attempt at converting the probes to the visual modality and evaluated the information stored in the features created by visual models trained with self-supervision.

For multimodal-BERTs, one study by \citet{parcalabescu2020seeing} investigates how well these models learn to count objects in images and how well they generalize to new quantities.
They found that the multimodal-BERTs overfit the dataset bias and fail to generalize to out-of-distribution quantities.
\citet{frank-etal-2021-vision} found that visual information is much more used for textual tasks than textual information is used for visual tasks when using multimodal models.
These findings suggest more needed research into other capabilities of and knowledge in multimodal-BERT embeddings.
We build on this line of work but aim to discover structures encoded in the textual and visual embeddings learned with multimodal-BERTs. This is a first step towards finding an aligned structure between text and images. Future work could exploit this to make textual information more useful for visual tasks.

\paragraph{Structures in visual data}
There is large research interest in identifying structural properties of images e.g., scene graph annotation of the visual genome dataset \citep{krishnavisualgenome}.
In the field of psychology, research towards scene grammars \citep{draschkow2017scene} evidences that humans assign certain grammatical structures to the visual world.
Furthermore, some studies investigate the grounding of textual structures in images,
such as syntax learners \citep{shi_visually_2019} and visually grounded grammar inducers \cite{zhao_visually_2020}. Here the complete image is used, without considering object regions and their composing structure, to aid in predicting linguistic structures.

Closer to our work, \citet{Elliott2013ImageDU} introduced visual dependency relations (VDR), where spatial relations are created between object in the image. The VDR can also be created by locating the object and subject in a caption and matching it with object annotations in the image \citep{Elliott2015DescribingIU}. Our scene tree differs, since it makes use of the entire dependency tree of the caption to create the visual structure.

\section{Background}
\paragraph{Multimodal-BERT}
Many variations of the BERT model implement a transformer architecture to process both visual and linguistic data, e.g., images and sentences. These Multimodal-BERTs can be categorized into two groups: single-stream and dual-stream encoders. In the former, a regular BERT architecture processes the concatenated input of the textual description and the image through a transformer stack.
This allows for an "unconstrained fusion of cross-modal features" \citep{bugliarello2021volta}. Some examples of these models are ViL-BERT \citep{su2019vlbert},  VisualBERT \citep{li2019visualbert}, and UNITER \citep{chen2020uniter}.

In the dual-stream models, the visual and linguistic features are first processed separately by different transformer stacks, followed by several transformer layers with alternating \textit{intra-modal} and \textit{inter-modal} interactions.
For the \textit{inter-modal} interactions, the query-key-value matrices modeling the multi-head self-attention are computed, and then the key-value matrices are exchanged between the modalities.
This limits the interactions between the modalities but increases the expressive power with separate parameters. Examples of such dual-stream models are ViLBERT \citep{lu2019vilbert}, LXMERT \citep{tan2019lxmert}, and ERNIE-ViL \citep{yu2021ernie}.\footnote{The ERNIE-ViL model is trained with scene graphs of the visual genome dataset. We do not probe this model as there is an overlap between the training data of ERNIE-ViL and our evaluation data.}

\section{Method}
\subsection{Tree Structures}\label{sec:tree}
In the probing experiments we assume that the structural knowledge of a sentence is made explicit by its dependency tree structure and that likewise the structural knowledge of an image is represented by a tree featuring the dependencies between object regions.
Further, we assume that the nodes of a tree (words in the dependency tree of the sentence, phrase labels in the region dependency tree of the image) are represented as embeddings obtained from a layer in BERT or in a multimodal-BERT.

To generate the depths and distances values from the tree, we use properties of the embedding representation space \citep{mikolov2013distributed}.
For example, similar types of relations between embeddings have a similar distance between them, such as counties and their capital city.
The properties we use are that the length (the norm) of a vector which describes the depth in a tree and the distance between nodes that can be translated as the distance between vectors.

\paragraph{Generating distance values}
For the distance labels, a matrix ${\bm{D}} \in {\mathbb{N}}^{n\times n}$ is required, with each ${\bm{D}}_{ij}$ describing the distance between nodes $i$ and $j$.
To fill the matrix, we iterate over all possible pairs of nodes.
For nodes $i$ and $j$, it is computed by starting at node $i$ in the tree and traverse it until node $j$ is reached while ensuring a minimum distance.
This is achieved by using the breadth-first search algorithm.

\paragraph{Generating depth values}
For the depth labels, we generate a vector ${\bm{d}} \in {\mathbb{N}}^n$, with $n$ the number of nodes in the tree.
There is a single node that is the root of the tree, to which we assign a depth of zero.
The depth increases at every level below.

\subsection{Constructing the Trees}\label{sec:data_processing}
\begin{algorithm}[pt]
\caption{$ConstructSceneTree(T_t, P, I)$}
\label{alg:tree_mapping}
\begin{algorithmic}[1]
\REQUIRE Language dependency tree $T_t = \{E_t,V_t\}$, with $V_t$ the set of $TextIDs$ for words in a sentence and $E_t$ the set of edges such that each $e_{t} = (v_{t,j}, v_{t,k})$, where $v_{t,k}$ is a child node of $v_{t,j}$
\REQUIRE Set of phrases $P$, each $p_i$ describes one or more regions and covers multiple words
\REQUIRE Image $I$
\ENSURE Scene tree $T_s$
\STATE $V_s = \{\}$, set of Nodes in Scene Tree $T_s$
\STATE $E_s= \{\}$, set of Edges in Scene Tree $T_s$
\STATE $v_{s,0} = I$, set Image as root node
\STATE $D_0 = 0$, set root node depth as 0
\STATE $add(V_s, v_{s,0})$
\STATE $v_{t,0} = FindRootNode(T_t)$
\STATE $PhraseID2TextID(0) = v_{t,0}$
\FOR{$p_i \in P$}
    \STATE $v_{t,k} = FindHighestNode(p_i)$
    \STATE $PhraseID2TextID(p_i) = v_{t,k}$
    \STATE $D_i = DepthInTree(T_t, v_{t,k})$
\ENDFOR
\FOR{$p_i \in P$ \textbf{ordered by} $D$}
    \STATE $v_{t,k} = PhraseID2TextID(p_i)$
    \WHILE{\textbf{True}}
        \STATE $e_{t} = EdgeWithChildNode(E, v_{t,k})$
        \STATE $v_{t,j} = SelectParentNode(e_{t})$
        \STATE $p_p = TextID2PhraseID(v_{t,j})$
        \IF{$p_p \in V_s$}
            \STATE $add(V_s, p_i), \quad add(E_s, (p_p, p_i))$
            \STATE $D_i = D_p + 1$
            \STATE \textbf{break while loop}
        \ELSE
            \STATE $v_{t,k} = v_{t,j}$
        \ENDIF
    \ENDWHILE
\ENDFOR
\RETURN $T_s$
\end{algorithmic}
\end{algorithm}
\paragraph{Language dependency tree}
We use the dependency tree as linguistic structure. The tree annotations are according to the Stanford dependency guidelines \citep{de2008stanford}.
They can either be provided as gold-standard in the dataset, or generated using the spacy dependency parser \citep{spacy}.

\paragraph{Scene tree}
\citet{draschkow2017scene} found that there are commonalities between words in language and objects in scenes, allowing to construct a scene grammar.
Furthermore, \citet{zhao_visually_2020} have shown that an image provides clues that improve grammar induction.
In line with these works, we want a visual structure that aligns with a linguistic representation like the dependency tree.

As visual structure, a scene graph could be used for the relations between regions \citep{krishnavisualgenome}.
However, the unconstrained graph
is difficult to align with the dependency tree.
Therefore, we propose a novel visual structure, the \textit{scene tree}, that is created by mapping a textual dependency tree to the object regions of an image.
An example of such a mapping for an image-sentence pair is given in Figure~\ref{fig:tree_mapping}.
This process requires a tree for the sentence and paired data for images and sentences.

Each node in the scene tree directly matches one or more visual regions.
The node description is a phrase that covers multiple words in the sentence (or nodes in the dependency tree).
The output of this method is a tree that contains the phrase trees that directly correspond to the regions.
The algorithm is completely described as pseudo-code in Algorithm~\ref{alg:tree_mapping}.

The algorithm starts by initializing the scene tree.
We set the full image as the root node.
For each phrase that describes an image region, we select the dependency tree node (or word with a $TextID$) that is closest to the root and assign this a phrase ID. This creates a mapping between the phrases (Phrase IDs) and dependency tree nodes (Text IDs) $PhraseID2TextID$, and its reverse $TextID2PhraseID$.
We assign each phrase an initial depth, based on the word it maps to in $PhraseID2TextID$.
On line 12, the loop over the phrases that describe the object regions starts, to find the direct parent for each phrase so it can be added to the new scene tree.
For each phrase $p_i$, we select the matching dependency tree node the $v_{t,k}$ from $PhraseID2TextID$. From $v_{t,k}$ we follow the chain of parent nodes, until an ancestor $v_{t,l}$ is found that points back to a phrase $p_j$ (using $TextID2PhraseID$) that is already a member of the scene tree. Phrase $p_i$ is added to the tree as child of $p_j$.
The completed tree of phrases is our \textit{scene tree}.

\subsection{Embeddings}\label{sec:embeddings}
\paragraph{Textual embeddings}
For each sentence $l$, every word becomes a node $n_i$ in the tree, such that we have a sequence of $s$ nodes $n_{1:s}^l$. To obtain the textual embeddings ${\bm{h}}_{1:s}^l \in {\mathbb{R}}^{m}$, we do a wordpiece tokenization \citep{wu2016google} and pass the sentence into BERT. Depending on the requested layer, we take the output of that BERT layer as the embeddings. For nodes with multiple embeddings because of the wordpiece tokenization, we take the average of those embeddings.

To obtain the textual embeddings ${\bm{h}}_{1:s}^l$ for a multimodal-BERT, we use the same process but also provide visual features.
When an image is present, we enter the visual features (as described in the next paragraph), otherwise, a single masked all-zero feature is entered.

\paragraph{Visual embeddings}
For sentence with image $l$, the sequence of $s$ nodes $n_{1:s}^l$ consists of the number of regions plus the full image.
The visual embeddings ${\bm{h}}_{1:s}^l \in {\mathbb{R}}^{m}$ are obtained by passing the raw Faster R-CNN features \citep{ren2015faster} into the multimodal-BERT.
Depending on the requested layer, we take the output of that multimodal-BERT layer as the embeddings.

\subsection{Structural Probes}\label{sec:probes}
Here we shortly describe the structural probes as defined by \citet{hewitt-manning-2019-structural}.
Originally designed for text, we use these probes to map from an embedding space (either textual embeddings or visual embeddings) to depth or distance values as defined in Section~\ref{sec:tree}.

\paragraph{Distance probe}
Given a sequence of $s$ nodes $n_{1:s}^l$ (words or objects) and their embeddings ${\bm{h}}_{1:s}^l \in {\mathbb{R}}^{m}$, where $l$ identifies the sequence and $m$ the embedding size, we predict a matrix of $s\times s$ distances.
First, we define a linear transformation ${\bm{B}} \in \mathbb{R}^{k\times m}$ with $k$ the probe rank, such that ${\bm{B}}^T{\bm{B}}$ is a positive semi-definite, symmetric matrix.
By first transforming a vector ${\bm{h}}$ with matrix ${\bm{B}}$, we get its norm like this: $({\bm{B}}{\bm{h}})^T({\bm{B}}{\bm{h}})$.
To get the squared distance between two nodes $i$ and $j$ in sequence $l$, we compute the difference between node embeddings ${\bm{h}}_i$ and ${\bm{h}}_j$ and take the norm following equation~\ref{eq:distance}:
\begin{equation}
{\bm{D}}_{ij} = ({\bm{B}}({\bm{h}}_i^l-{\bm{h}}_j^l))^T({\bm{B}}({\bm{h}}_i^l-{\bm{h}}_j^l))\label{eq:distance}
\end{equation}
The only parameters of the distance probe are now the transformation matrix ${\bm{B}}$, which can easily be implemented as a fully connected linear layer.
Identical to the work by \citet{hewitt-manning-2019-structural}, the probe is trained through stochastic gradient descent.

\paragraph{Depth probe}
For the depth probe, we transform the embedding of each node $n_i$ to their norm, so we can construct the vector ${\bm{d}}$.
This imposes a total order on the elements and results in the depths.
We compute the squared vector norm $\norm{{\bm{h}}_i}_{\bm{B}}^2$ with the following equation:
\begin{equation}
{\bm{d}}_i = \norm{{\bm{h}}_i}_{\bm{B}}^2 = ({\bm{B}}{\bm{h}}_i^l)^T({\bm{B}}{\bm{h}}_i^l)\label{eq:depth}
\end{equation}

\section{Experimental Setup}
\subsection{Data}\label{sec:data}
By using a text-only dataset, we can test how the textual embeddings of the multimodal-BERTs perform compared to the BERT model, without the interference from the visual embeddings.
This allows us to see how much information the multimodal-BERTs encode in the visual embeddings.

Therefore, we use the Penn Treebank (PTB3) \citep{marcus1999treebank}.
It is commonly used for dependency parsing (also by \citet{hewitt-manning-2019-structural} from whom we borrow the probes) and consists of gold-standard dependency tree annotations according to the Stanford dependency guidelines \citep{de2008stanford}.
We use the default training/validation/testing split, that is, the subsets 2-21 for training, 22 for validation and 23 for testing
of the Wall Street Journal sentences.
This provides us with 39.8k/1.7k/2.4k sentences for the splits, respectively.

The second dataset is the Flickr30k dataset \citep{young2014flickr30k}, which consists of multimodal image captioning data.
It has five caption annotations for each of the 30k images. An additional benefit of this dataset are the existing extensions, specifically the Flickr30k-Entities (F30E) \citep{plummer2015flickr30k_entities}.
In F30E all the phrases in the captions are annotated and match with region annotations in the image.
This paired dataset is used to create the scene trees proposed in Section~\ref{sec:data_processing}.

The Flickr30k dataset does not provide gold-standard dependency trees.
Therefore, the transformer based Spacy dependency parser \citep{spacy} is used to generate silver-standard dependency trees according to the Stanford dependency guidelines \citep{de2008stanford}.
The dataset consists of 30k images, with (mostly) 5 captions each, resulting in 148.9k/5k/5k sentences for the training/validation/testing splits, respectively.

\subsection{Models}\label{sec:models}
We use two different multimodal-BERTs, one \textbf{single-stream} and one \textbf{dual-stream} model.
As implementation for the multimodal-BERTs, we make use of the \textsc{Volta} library \citep{bugliarello2021volta}.
Here, all the models are implemented and trained under a controlled and unified setup with regard to hyperparameters and training data.
Based on the performance under this unified setup on the Flickr30k image-sentence matching task, we have chosen the best performing models: ViLBERT \citep{lu2019vilbert} as single-stream model and UNITER \citep{chen2020uniter} as dual-stream model.

When probing the textual embeddings, we also use a text-only \textbf{BERT-base model} (from here on referred to as BERT) \citep{devlin-etal-2019-bert}.
\citet{hewitt-manning-2019-structural} use the same model, allowing for easy comparability.
The implementation used is from the HuggingFace Transformer library \citep{wolf-etal-2020-transformers}.

\paragraph{Hyperparameters}
For our setup and metrics, we follow the setup from \citet{hewitt-manning-2019-structural}.
The batch size is set to 32 and we train for a maximum of 40 epochs.
Early stopping is used to terminate training after no improvement on the validation L1-loss for 5 epochs.

\subsection{Metrics}\label{sec:metrics}
\begin{figure*}[t]
    \centering
    \begin{subfigure}[t]{0.9\linewidth}
        \centering
        \includegraphics[scale=0.4]{figures/results/ptb3/BERT/select_layer_ParseDepthTask_rank128.test.legend.pdf}
    \end{subfigure}
    \\
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.48]{figures/results/ptb3/BERT/select_layer_ParseDepthTask_rank128.test.pdf}
        \caption{BERT}
        \label{fig:ptb_depth_bert}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.48]{figures/results/ptb3/UNITER/select_layer_ParseDepthTask_rank128.test.pdf}
        \caption{UNITER}
        \label{fig:ptb_depth_unit}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.48]{figures/results/ptb3/ViLBERT/select_layer_ParseDepthTask_rank128.test.pdf}
        \caption{ViLBERT}
        \label{fig:ptb_depth_vil}
    \end{subfigure}
    \caption{Comparison for the depth probe on the PTB3 test set, with textual embeddings.}
    \label{fig:ptb3_depth}
\end{figure*}
\begin{figure*}[t]
    \centering
    \begin{subfigure}[t]{0.9\linewidth}
        \centering
        \includegraphics[scale=0.4]{figures/results/ptb3/BERT/select_layer_ParseDistanceTask_rank128.test.legend.pdf}
    \end{subfigure}
    \\
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/ptb3/BERT/select_layer_ParseDistanceTask_rank128.test.pdf}
        \caption{BERT}
        \label{fig:ptb_dist_bert}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/ptb3/UNITER/select_layer_ParseDistanceTask_rank128.test.pdf}
        \caption{UNITER}
        \label{fig:ptb_dist_unit}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/ptb3/ViLBERT/select_layer_ParseDistanceTask_rank128.test.pdf}
        \caption{ViLBERT}
        \label{fig:ptb_dist_vil}
    \end{subfigure}
    \caption{Comparison for the distance probe on the PTB3 test set, with textual embeddings.}
\label{fig:ptb3_distance}
\end{figure*}
\begin{figure*}[t]
    \centering
    \begin{subfigure}[t]{0.9\linewidth}
        \centering
        \includegraphics[scale=0.4]{figures/results/flickr30k/BERT/select_layer_ParseDepthTask_rank128.test.legend.pdf}
    \end{subfigure}
    \\
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/BERT/select_layer_ParseDepthTask_rank128.test.pdf}
        \caption{BERT}
        \label{fig:layer_flickr_dep_bert}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/UNITER/select_layer_ParseDepthTask_rank128.test.pdf}
        \caption{UNITER}
        \label{fig:layer_flickr_dep_unit}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/ViLBERT/select_layer_ParseDepthTask_rank128.test.pdf}
        \caption{ViLBERT}
        \label{fig:layer_flickr_dep_vil}
    \end{subfigure}
    \caption{Comparison for the depth probe on the Flickr30k test set, with textual embeddings.}
    \label{fig:layer_flickr_dep}
\end{figure*}
\begin{figure*}[t]
    \centering
    \begin{subfigure}[t]{0.9\linewidth}
        \centering
        \includegraphics[scale=0.4]{figures/results/flickr30k/BERT/select_layer_ParseDistanceTask_rank128.test.legend.pdf}
    \end{subfigure}
    \\
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/BERT/select_layer_ParseDistanceTask_rank128.test.pdf}
        \caption{BERT}
        \label{fig:layer_flickr_dist_bert}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/UNITER/select_layer_ParseDistanceTask_rank128.test.pdf}
        \caption{UNITER}
        \label{fig:layer_flickr_dist_unit}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/ViLBERT/select_layer_ParseDistanceTask_rank128.test.pdf}
        \caption{ViLBERT}
        \label{fig:layer_flickr_dist_vil}
    \end{subfigure}
    \caption{Comparison for the distance probe on the Flickr30k test set, with textual embeddings.}
    \label{fig:layer_flickr_dist}
\end{figure*}
\begin{figure*}[t]
    \centering
    \begin{subfigure}[t]{0.9\linewidth}
        \centering
        \includegraphics[scale=0.4]{figures/results/flickr30k/BERT/select_layer_ParseDepthTask_rank128.test.legend.pdf}
    \end{subfigure}
    \\
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/BERT/select_layer_ParseDepthTask_rank128.test.pdf}
        \caption{BERT}
        \label{fig:layer_flickr_dep_bert_just_text}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/UNITER/just_text/select_layer_ParseDepthTask_rank128.test.pdf}
        \caption{UNITER - only text}
        \label{fig:layer_flickr_dep_unit_just_text}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/ViLBERT/just_text/select_layer_ParseDepthTask_rank128.test.pdf}
        \caption{ViLBERT - only text}
        \label{fig:layer_flickr_dep_vil_just_text}
    \end{subfigure}
    \caption{Ablation comparison for the depth probe on the Flickr30k test set while just providing textual embeddings to the multimodal-BERTs.}
    \label{fig:layer_flickr_dep_just_text}
\end{figure*}
\begin{figure*}[t]
    \centering
    \begin{subfigure}[t]{0.9\linewidth}
        \centering
        \includegraphics[scale=0.4]{figures/results/flickr30k/BERT/select_layer_ParseDistanceTask_rank128.test.legend.pdf}
    \end{subfigure}
    \\
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/BERT/select_layer_ParseDistanceTask_rank128.test.pdf}
        \caption{BERT}
        \label{fig:layer_flickr_dist_bert_just_text}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/UNITER/just_text/select_layer_ParseDistanceTask_rank128.test.pdf}
        \caption{UNITER - only text}
        \label{fig:layer_flickr_dist_unit_just_text}
    \end{subfigure}
    \begin{subfigure}[t]{0.32\linewidth}
        \centering
        \includegraphics[scale=0.47]{figures/results/flickr30k/ViLBERT/just_text/select_layer_ParseDistanceTask_rank128.test.pdf}
        \caption{ViLBERT - only text}
        \label{fig:layer_flickr_dist_vil_just_text}
    \end{subfigure}
    \caption{Ablation comparison for the distance probe on the Flickr30k test set while just providing textual embeddings to the multimodal-BERTs.}
    \label{fig:layer_flickr_dist_just_text}
\end{figure*}
\begin{figure*}[t]
\begin{minipage}{.49\textwidth}
\flushleft
    \begin{subfigure}[t]{\linewidth}
        \centering
        \includegraphics[scale=0.4]{figures/results/flickr30k/UNITER/select_layer_ParseVisualDepthTask_rank128.test.legend.pdf}
    \end{subfigure}\\
    \begin{subfigure}[t]{0.48\linewidth}
        \flushleft
        \includegraphics[scale=0.4]{figures/results/flickr30k/UNITER/select_layer_ParseVisualDepthTask_rank128.test.pdf}
        \caption{UNITER}
        \label{fig:layer_flickr_visdep_unit}
    \end{subfigure}
    ~
    \begin{subfigure}[t]{0.48\linewidth}
        \flushright
        \includegraphics[scale=0.4]{figures/results/flickr30k/ViLBERT/select_layer_ParseVisualDepthTask_rank128.test.pdf}
        \caption{ViLBERT}
        \label{fig:layer_flickr_visdep_vil}
    \end{subfigure}
    \caption{Comparison for the depth probe on the Flickr30k test set, with visual embeddings. Note that the scale is different in this Figure.}
    \label{fig:layer_flickr_visdep}
\end{minipage}
~
\begin{minipage}{.49\textwidth}
    \flushright
    \begin{subfigure}[t]{\linewidth}
        \centering
        \includegraphics[trim={0 10 0 3},clip,scale=0.4]{figures/results/flickr30k/UNITER/select_layer_ParseVisualDistanceTask_rank128.test.legend.pdf}
    \end{subfigure}
    \\
    \begin{subfigure}[t]{0.47\linewidth}
        \flushleft
        \includegraphics[trim={0 0 30 0},clip,scale=0.4]{figures/results/flickr30k/UNITER/select_layer_ParseVisualDistanceTask_rank128.test.pdf}
        \caption{UNITER}
        \label{fig:layer_flickr_visdist_unit}
    \end{subfigure}
    ~
    \begin{subfigure}[t]{0.47\linewidth}
        \flushright
        \includegraphics[trim={30 0 0 0},clip,scale=0.4]{figures/results/flickr30k/ViLBERT/select_layer_ParseVisualDistanceTask_rank128.test.pdf}
        \caption{ViLBERT}
        \label{fig:layer_flickr_visdist_vil}
    \end{subfigure}
    \caption{Comparison for the distance probe on the Flickr30k test set, with visual embeddings. Note that the scale is different in this Figure.}
    \label{fig:layer_flickr_visdist}
\end{minipage}
\end{figure*}
The main metric used for both the distance and the depth probes is the Spearman rank coefficient correlation.
This indicates if the predicted depth vector of the nodes, or the predicted distance matrix of the nodes, correlate with the gold-standard (or silver) depths and distances generated according to the method in Section~\ref{sec:probes}.
The Spearman correlation is computed for each length sequence separately.
We take the average over the scores of the lengths between 5 and 50 and call this the Distance Spearman (DSpr.) for the distance probe and the Norm Spearman (NSpr.) for the depth probe.\footnote{Just as done by \citet{hewitt-manning-2019-structural}.}

For the depth probes, we also use the root accuracy (root\_acc). This computes the accuracy of predicting the root of the sequence.
This metric is only applicable for the textual embeddings, due to our method of generating the visual tree, where the root is always the full image at the start of the sequence.

For the distance probe, we make use of the undirected unlabelled attachment score (UUAS). This directly tests how accurate the predicted tree is compared to the ground-truth (or silver) tree by computing the accuracy of predicted connections between nodes in the tree. It does not consider the label for the connection or the direction of the connection \citep{jurafsky2021speech}.

\paragraph{Baseline comparisons}
We design one baseline for the textual data and two for the visual data. For the textual baseline, we use the initial word piece textual embeddings (from either BERT or a multimodal-BERT) before inserting them into the transformer stack.
We simply refer to it as \textbf{baseline}.

The first visual baseline implements the raw Faster R-CNN features \citep{ren2015faster} of each object region. However, they have a larger dimension than the BERT embeddings.
We refer to it as \textbf{R-CNN baseline}.
The second baseline uses the visual embeddings before they are fed to the transformer stack. This is a mapping from the Faster R-CNN features to the BERT embedding size. We refer to it as \textbf{baseline}.

\subsection{Hypotheses}\label{sec:hypotheses}
First, we want to determine the probe rank of the linear transformation used on the textual or the visual embeddings.
Based on results by \citet{hewitt-manning-2019-structural}, we set the probe rank for BERT to 128. We run a comparison with several probe ranks on UNITER and ViLBERT to find the optimal setting for the textual and visual embeddings. The results are shown and discussed in Appendix~\ref{app:tune_rank}. We use a rank of 128 for all our following experiments.

\paragraph{RQ 1}
The multimodal-BERT models are pre-trained on language data. We assume that the resulting embeddings integrate structural grammatical knowledge and hypothesize that this knowledge will not be forgotten during multimodal training.

To determine if training on multimodal data affects the quality of predicting the dependency tree when trained solely with textual data, we train the probes with BERT and both multimodal-BERTs and evaluate on the PTB3 dataset \citep{marcus1999treebank}.

\paragraph{Sub-RQ 1.1}
We expect that more interaction between the regions and the text will have a stronger impact. Some dependency attachments that are hard to predict might require visual knowledge.
Next to the effect on the linguistic knowledge, we also want to discover if the multimodal data helps the multimodal-BERTs in learning structural knowledge.
We run the probes on Flickr30k dataset \citep{young2014flickr30k} with the textual embeddings for all our models. Furthermore, we compare these to the difference in scores on the PTB3 dataset \citep{marcus1999treebank}.

\paragraph{RQ 2}
The Multimodal-BERTs learn highly contextualized embeddings. Therefore, we hypothesize that a model should be able to discover important interactions between object regions in the image.
To see if the model has learned to encode the scene tree in the visual region embeddings, we run the probes on the Flickr30k dataset \citep{young2014flickr30k} with the visual embeddings.
Furthermore, to see if the scene tree is learned mainly through joint interaction with the textual embeddings, we compare the scores between the single-stream model UNITER (with many cross-modal interactions) and the dual-stream model ViLBERT (with limited cross-modal interactions).

\section{Results and Discussion}\label{sec:results}
This discussion is based on the results from the test split. The results on the validation split (see Appendix~\ref{sec:val_results}), lead to the same observations.

\paragraph{RQ 1: Do the textual embeddings trained with a multimodal-BERT retain their structural knowledge?}
To answer RQ 1, we report the results for both structural probes on the PTB3 dataset. Here we only use the textual embeddings, since no visual features are available. The results for the depth probe are in Figure~\ref{fig:ptb3_depth}, and for the distance probe in Figure~\ref{fig:ptb3_distance}.

The results of both multimodal-BERTs (Figures~\ref{fig:ptb_depth_vil} and \ref{fig:ptb_dist_vil} for ViLBERT and Figures~\ref{fig:ptb_depth_unit} and \ref{fig:ptb_dist_unit} for UNITER) in terms of NSpr. and Root Acc are very comparable showing similar curves and scores.
For both, the seventh layer is the best performing one. The shape of the curves across the layers is similar to those for the BERT model in Figures~\ref{fig:ptb_depth_bert} and \ref{fig:ptb_dist_bert}.
However, the scores of the multimodal-BERTs drop significantly.
While the multimodal-BERTs were initialized with weights from BERT, they were trained longer on additional multimodal data with a different multimodal objective. This shows that the multimodal training hampers the storing of grammatical structural knowledge in the resulting embeddings.

\paragraph{Sub-RQ 1.1: To what extent does the joint training in a multimodal-BERT influence the structures learned in the textual embeddings?}
For this experiment, we compare the effect of having visual features present when using the structural probes on the textual embeddings. We run the probes on Flickr30k. The results for the depth probe are in Figure~\ref{fig:layer_flickr_dep}, and for the distance probe in Figure~\ref{fig:layer_flickr_dist}.

First, we see that for all models (BERT and multimodal-BERTs) the scores increase compared to the results on the PTB3 dataset (see discussion of RQ 1), but still follow a similar trend across the layers.
The latter is most likely due to the complexity of the sentences and language of the PTB3 dataset, which is simpler for the captions.
For ViLBERT, there is a drop in performance for the earlier layers. We believe this is caused by the early stopping method firing early with these settings. Another explanation is that it is more difficult for the dual-stream model to use the additional parameters.

BERT outperforms the multimodal-BERTs on PTB3, however, this is not the case on Flickr30k.
For the depth probe (Figure~\ref{fig:layer_flickr_dep}) and the UUAS metric on the distance probe (Figure~\ref{fig:layer_flickr_dist}), the results obtained on these two datasets are almost equal.
This can be due to the additional pretraining of the multimodal-BERTs on similar captioning sentences.
Another explanation is that, during such pretraining, the models learned to store relevant information in the visual embeddings.

We run an additional experiment where we use the pretrained multimodal-BERT, but while probing we only provide the sentence to the model, and mask out the image. The results for the depth probe are in Figure~\ref{fig:layer_flickr_dep_just_text}, and for the distance probe in Figure~\ref{fig:layer_flickr_dist_just_text}. Here we can see that the results are almost identical to when we provide the model with the visual embeddings. This indicates that the model does not have any benefit from the visual data when predicting the structures for textual embeddings, and it seems that the model uses the extra parameters of the vision layers to store knowledge about the text.

\paragraph{RQ 2: Do the visual embeddings trained with a multimodal-BERT learn to encode a scene tree?}
We aim to find the layer with the most structural knowledge learned when applied to multimodal data. See the results in Figures~\ref{fig:layer_flickr_visdep} and \ref{fig:layer_flickr_visdist}.

Regarding the results for the depth probe (Figure~\ref{fig:layer_flickr_visdep}), the scores between layers fluctuate inconsistently. The scores do improve slightly over the baselines, indicating that the multimodal-BERT encodes some knowledge of depth in the layers.

With regard to the distance probe (Figure~\ref{fig:layer_flickr_visdist}), the trend in the curves across the layers indicate that this is a type of knowledge that can be learned for the regions.
The multimodal-BERTs seem to disregard scene trees. There is a strong downward trend across the layers.
Furthermore, all the scores are much lower than the baseline and the R-CNN baseline scores.
This lack of learning of the scene tree can be caused by the chosen training objective of the multimodal-BERTs.
These objectives require an abstract type of information, where only basic features are needed to predict the masked items.

For the distance probe, there is a noticeable difference between the single-stream  (Figure~\ref{fig:rank_flickr_visdist_unit}) and the dual-stream (Figure~\ref{fig:rank_flickr_visdist_vil}) models, where single stream models benefit from the multimodal interactions to retain structural knowledge.
For UNITER, the scores in the first layers are very close to the baseline, showing that the single stream interaction benefits the memorizing of the scene tree structure.

\section{Conclusion and Future Work}
We made a first attempt at investigating whether the current Multimodal-BERT models encode structural grammatical knowledge in their textual embeddings, in a similar way as text-only BERT models encode this knowledge.
Furthermore, we were the first to investigate the existence of encoded structural compositional knowledge of the object regions in image embeddings.
For this purpose, we created a novel scene tree structure that is mapped from the textual dependency tree of the paired caption.
We discovered that the multimodal-BERTs encode less structural grammatical knowledge than BERT. However, with image features present, it is still possible to achieve similar results. The cause for this requires more research.

While tree depths from the scene tree are not natively present in the features, we found that this could be a potential method of finding connections and distances between regions, already decently predicted with the Faster R-CNN features.
The Multimodal-BERT models are currently trained with an objective
that does not enforce the learning or storing of these types of structural information. Hence we assume that the models learn to encode more abstract knowledge in their features.

Our work opens possibilities to further research on scene trees as a joint representation of object compositions in an image and the grammatical structure of its caption.
Furthermore, we recommend investigating the training of multimodal-BERTs with objectives that enforce the encoding of structural knowledge.

\section*{Acknowledgments}
We would like to thank Desmond Elliott, Djam\'e Seddah, and Liesbeth Allein for feedback on the paper.
Victor Milewski and Marie-Francine Moens were funded by the European Research Council (ERC) Advanced Grant CALCULUS (grant agreement No. 788506).
Miryam de Lhoneux was funded by the Swedish Research Council (grant 2020-00437).
\bibliographystyle{acl_natbib}

redpajama_set_name: RedPajamaArXiv


text:
Home » Dunn-Edwards Director Receives ACA Industry Achievement Award
Dunn-Edwards Director Receives ACA Industry Achievement Award
KEYWORDS awards
LOS ANGELES — Robert Wendoll, Director of Environmental Affairs for Dunn-Edwards Paint, recently received an Industry Achievement Award by the American Coatings Association (ACA), in conjunction with the organization's 125th anniversary celebration. The award honors individuals for their distinguished achievement in service to the industry.
With close to 40 years experience in the paint and coatings industry, Wendoll is considered one of the leading authorities on the atmospheric impact of architectural coatings. His leadership in various industry organizations has led to more self-regulation by many paint manufacturers and an increase in policy-relevant scientific research. He frequently interacts with regulatory agencies involved in the development of new or amended regulations affecting the paint industry.
During his 31-year tenure at Dunn-Edwards, he has focused on the company's environmental responsibility – not only ensuring its full compliance with all environmental, health and safety laws and regulations, but going a step beyond to lead the industry in minimizing and/or eliminating the use of potentially hazardous ingredients from Dunn-Edwards' paints.
Wendoll has been an active member of ACA's committees, including its Environmental Management Committee and its Architectural and Industrial Maintenance (AIM) VOC Committee, for many years. Wendoll has been a leader in identifying vital AIM VOC issues for the association to work on, also providing critical data and support to staff so that ACA could successfully work through these matters. Chief among these is the Paints and Architectural Coatings Environmental Study (PACES) project, for which he provided key information and direction that helped the successful completion of this multi-year research project. With his guidance, the project oversaw among many things, a Life Cycle Analysis to assess impacts of stringent, low-VOC regulations and the relationship between coating performance and environmental impacts and work in "atmospheric availability" that should advance the scientific understanding of whether or how much of the VOCs in coatings actually become available in the atmosphere for ground-level ozone formation.
The three other inductees honored were: Nelson Augé, President of Kelley Technical Coatings Inc.; Steve Dearborn, President and CEO of Miller Paint Co. Inc.; and Lewis Manring, Vice President of Portfolio Optimization for DuPont Science & Technology.
Sun Chemical Executive Receives Industry Achievement Award
Dunn-Edwards Wins Award for Its Manufacturing Plant
Dunn-Edwards Wins Manufacturing Leadership 100 Award
Drew Vogel Receives ACA's Heckel Award

redpajama_set_name: RedPajamaCommonCrawl


text:
CHLOE FOX
Chloe Fox is a journalist and author, whose work regularly appears in the pages of VOGUE, PORTER, THE TELGRAPH, and THE FINANCIAL TIMES to name but a few.
Her magazine profiles, drawn mainly from the worlds of film and theatre, are some of the most standout of their kind. Over the years, the likes of Judi Dench, Jude Law, Kate Winslet, Tom Hanks, Sienna Miller and Leonardo di Caprio have all told Chloe what makes them tick.
A regular on the pages of VOGUE, Chloe has also authored two biographies in Conde Nast's popular 'Vogue On' series - Vogue On: Alexander McQueen and the forthcoming Vogue On: Manolo Blahnik.
She also writes a regular fictional social stereotype column in the Financial Times' How To Spend It magazine.
Chloe Fox was born in London in the heatwave of 1976. Despite having grown up in a theatrical family, she resisted the strong genetic urge to take to the stage and chose writing instead.
In 1999, after graduating from Edinburgh University with a Master's Degree in English Literature, she went to work as a researcher at the Telegraph Magazine. When she left four years later, it was as a features writer. Over the years, her interview subjects there have included Tom Hanks, Jude Law, Judi Dench, Gerard Butler and Kate Winslet.
©2017 Chloe Fox | Website by Sketchanet


Steven (Chicagoland, United States) on 13 October 2014 in Landscape & Rural and Portfolio.
Photo captured at the Van Patten Woods Forest Preserve on October 6, 2012.
How clean and well-maintained it all looks : like a living model !
Excellent !! Is an authentic autumn photo !
The road gives a wonderful sense of the journey as well as seasons change.
excellent compo! the winding road, the colour scheme with the autumn leaves and the blue sky ... all very attractive !
What a lovely photo of fall. Great tree colors, and greenery.
Gorgeous colours starting to come through.
Beautifully composed. The curve is placed ideally to lead the eye to the colorful foliage.
such beautiful composition, perspective, clouds, colors & lighting!
That is gorgeous. The curve of the road leads the eye right into the spectacular colour.
A beautiful image! I love the curve in the road and the gorgeous autumn color. This is just fantastic!
love the sweep of that curve.


Where do I enter my state or province code?
If your State or Province does not appear in the drop down menu when you are filling out your ship-to address, shipment is not available to your location. At this time OfficeSuperSavers.com does not offer shipment to APO/FPO, Alaska, Hawaii, Puerto Rico, Guam, Virgin Islands and International addresses.
However, we do have a separate website that features the products available for shipment in Canada. The web address for our Canadian division is www.BuyOnlineNow.ca. Not all items available in the United States are available in Canada. If you have any questions regarding a product's availability in Canada, please contact our Canadian division via telephone at 888-231-9493 or email customercare@buyonlinenow.ca.


text:
package models

// This file was generated by the swagger tool.
// Editing this file might prove futile when you re-run the swagger generate command

import (
	"context"

	"github.com/go-openapi/errors"
	"github.com/go-openapi/strfmt"
	"github.com/go-openapi/swag"
)

// CapacityPoolReference Reference to a capacity pool object.
//
// swagger:model capacity_pool_reference
type CapacityPoolReference struct {

	// links
	Links *SelfLink `json:"_links,omitempty"`

	// serial number
	// Example: 390000100
	SerialNumber string `json:"serial_number,omitempty"`
}

// Validate validates this capacity pool reference
func (m *CapacityPoolReference) Validate(formats strfmt.Registry) error {
	var res []error

	if err := m.validateLinks(formats); err != nil {
		res = append(res, err)
	}

	if len(res) > 0 {
		return errors.CompositeValidationError(res...)
	}
	return nil
}

func (m *CapacityPoolReference) validateLinks(formats strfmt.Registry) error {
	if swag.IsZero(m.Links) { // not required
		return nil
	}

	if m.Links != nil {
		if err := m.Links.Validate(formats); err != nil {
			if ve, ok := err.(*errors.Validation); ok {
				return ve.ValidateName("_links")
			}
			return err
		}
	}

	return nil
}

// ContextValidate validate this capacity pool reference based on the context it is used
func (m *CapacityPoolReference) ContextValidate(ctx context.Context, formats strfmt.Registry) error {
	var res []error

	if err := m.contextValidateLinks(ctx, formats); err != nil {
		res = append(res, err)
	}

	if len(res) > 0 {
		return errors.CompositeValidationError(res...)
	}
	return nil
}

func (m *CapacityPoolReference) contextValidateLinks(ctx context.Context, formats strfmt.Registry) error {

	if m.Links != nil {
		if err := m.Links.ContextValidate(ctx, formats); err != nil {
			if ve, ok := err.(*errors.Validation); ok {
				return ve.ValidateName("_links")
			}
			return err
		}
	}

	return nil
}

// MarshalBinary interface implementation
func (m *CapacityPoolReference) MarshalBinary() ([]byte, error) {
	if m == nil {
		return nil, nil
	}
	return swag.WriteJSON(m)
}

// UnmarshalBinary interface implementation
func (m *CapacityPoolReference) UnmarshalBinary(b []byte) error {
	var res CapacityPoolReference
	if err := swag.ReadJSON(b, &res); err != nil {
		return err
	}
	*m = res
	return nil
}

redpajama_set_name: RedPajamaGithub


text:
Q: Problema com LEFT JOIN no PHP Olá, amigos.
Tenho 3 tabelas no meu banco mysqli para registrar o acesso a determinado local:

*

*visitante, cujos dados sempre serão preenchidos.

*corporativo, cujos dados serão preenchidos apenas se o visitante representar uma empresa.

*veiculo, cujos dados serão preenchidos se o visitante vier de carro.
Ao fazer uma busca para retornar todos os resultados, utilizo a seguinte query:
SELECT *
FROM `visitante`
LEFT JOIN `corporativo` ON `corporativo`.`id_visitante` = `visitante`.`id_visitante`
LEFT JOIN `veiculo` ON `veiculo`.`id_visitante` = `visitante`.`id_visitante`
WHERE `visitante`.`entrada` LIKE '%$data%' OR `visitante`.`status` LIKE '%Aberto%'
GROUP BY `visitante`.`id_visitante`
ORDER BY `visitante`.`entrada`;

O que acontece é que nos registros onde não tenho dados do visitante corporativo ou dados do veículo, ele retorna minha id.visitante como NULL.
Acho que é algum problema do PHP pois ao fazer essa query no phpmyadmin, ele retorna todos os dados normalmente.
Alguém tem alguma sugestão???
Obrigado.

A: Tente criar aliases para a coluna id na tabela visitante, ou então mudar o nome. Se nas tabelas envolvidas existem várias colunas com o mesmo nome (id_visitante) ele pode estar 'escolhendo' as últimas (corporativo.id_visitane, veiculo.id_visitante) que de fato estão vazias e acabam sobrescrevendo o valor da primeira (visitante.id_visitante).
Lembre-se que você não precisa trazer três vezes o valor do id_visitante
O ideal seria algo assim:
(Apenas um exemplo)
SELECT
    visitante.id as id_visitante
    , visitante.nome
    , corpoartivo.empresa
    , veiculo.placa
FROM
    visitante
    LEFT JOIN corporativo ON corporativo.id_visitante = visitante.id
    LEFT JOIN veiculo ON veiculo.id_visitante = visitante.id

redpajama_set_name: RedPajamaStackExchange


text:
Star Wars: Tom Veitch's 10 Biggest Impacts On The Franchise
Tom Veitch, a fantastic comic book author and poet, passed away on February 18th, 2022. While he worked for the comic industry in general, it's in Star Wars his impact is most clearly felt. For those unaware, Tom Veitch was the author of both the Tales of the Jedi and Dark Empire comics, the names of which have likely brought up specific memories for fans old and new.
RELATED: Star Wars: 5 Things We Know About The KOTOR Remake (& 5 Questions We Have)
The following list has been compiled for those unaware of what Tom Veitch's work meant, not just for comic book fans but Star Wars fans as well. Many may not realize some of them came directly from his scripts.
10 The Old Republic Era
As mentioned, Tom Veitch wrote the original Tales of the Jedi comics for Dark Horse. These '90s comics may not be well known to casual fans; However, their impact certainly is. They were the first major dive the Expanded Universe took into the era of The Old Republic. Much of the time before the films was kept off-limits by Lucasfilm. However, by setting the story thousands of years in the past, it provided the freedom not just for Veitch's story, but for little-known companies Bioware and Obisidian to develop three of the most cherished Star Wars games of all time.
9 The Fall Of Freedon Nadd
Freedon Nadd was a Dark Jedi who lived and died well before the time of the Tales of the Jedi comics. He ruled with an iron fist on the savage planet of Onderon, and his ghostly specter would prove more dangerous than his living body ever was. The first few tales involving Ulic-Qel Droma (who will be spoken of later) involved the espionage and intrigue of the planet's politics, as well as its ancient cult-like worship for the fallen man. Freedon Nadd's ghostly machinations would set the stage for much Star Wars lore in the future.
8 Ancient Sith Society, Culture, And Lore
During the writing of the Hair to the Empire by Timothy Zahn, the Noghri (the book's race of alien assassins) were at first intended to be the Sith. It may be hard to think of a time before the term was so well explained and explored, but outside of Darth Vader's title as "Dark Lord of the Sith," there was next to no info.
RELATED: Star Wars: 5 Best Legends Sith (& 5 Worst)
Lucasfilm had Zahn change the name of his race, meaning it was Tom Veitch who got to show readers the ancient Sith people and their Egyptian-like culture.
7 The Great Sith Lord Naga Sadow
Speaking of ancient Sith, there is no Dark Lord more infamous than Naga Sadow. Claiming the throne of his people through manipulation and intrigue, the dark sorcerer would start a galactic conflict known as "The Great Hyperspace War," which would devastate much of the galaxy.
RELATED: The 10 Craziest Force Abilities From Star Wars Legends
Before returning to the world of Yavin IV to have his Massassi constructing grand temples for him, he'd go out with a literal bang by igniting twin suns and causing a supernova.
6 The Great Hyperspace War Conflict
The Great Hyperspace War itself was one of the largest conflicts in the Star Wars galaxy back during Legends continuity. A war so brutal and widespread that not even the galactic capital of Coruscant was safe from its wrath. Naga Sadow was the manipulator and instigator by claiming two innocent hyperspace explorers from Republic space were actually scouts for an invasion force. Many characters within the Tales of the Jedi series were swept up in the conflict as the battles raged and caused possibly the greatest fall from grace the Old Republic had seen up until that point.
5 The Double Blade Weilder Exar Kun
Exar Kun's double-bladed lightsaber likely looks familiar to some due to how commonplace the weapon is in modern Star Wars lore. However, Exar Kun was actually the first character in the mythos to use such a weapon, and it's a picture of him that George Lucas described as giving inspiration for Darth Maul's iconic weapon in Episode I: The Phantom Menace. Exar Kun was a great warrior who slaughtered all before him and would eventually team with Ulic-Qel Droma to take control of the galaxy for the dark side.
4 Heroine Of The Old Republic, Nomi Sundrider
Nomi Sunrider was the wife of a Jedi Knight shot down before her eyes due to the machinations of the criminal underworld. With their child in tow, she left to learn the ways of the force and honor her husband's pledge to defend the galaxy from evil. She would become one of the primary heroes during The Great Hyperspace War and even due some of the strongest dark side warriors of her era to their own submission. The Sunrider name and legacy would continue throughout the Old Republic era and leave many impacts across the galaxy.
3 The Tragedy Of Ulic-Qel Droma
Ulic-Qel Droma was a brave Jedi and the initial hero of the Tales of the Jedi books. However, after multiple interactions with Darkside users, it would be Droma who would fall. Becoming an armored juggernaut intentionally resembling the future Darth Vader, Droma teamed with the evil Exar Kun and ended up killing his own brother in a fit of rage. Droma eventually accepted defeat and, before dying, trained a young force user for the side of good. In a small way, this was his attempt to make up for his past sins.
2 The Return of Boba Fett
Boba Fett had already escaped the Sarlacc pit in a Marvel Star Wars Comic, but was quickly thrown back in. (This was due to the need for spin-off media at the time and putting everything back in its place to avoid conflicting with future films). However, once Tom Veitch began writing Dark Empire, he was given free rein with the cast and characters. This allowed him to once again and this time permanently return Boba Fett to his former glory. He also sported a very silly new Slave-II ship, but it was thankfully very short-lived.
1 The Ressurection of Palpatine
Anyone who's read Dark Empire knows that it's the most important and singular influence on Episode IX: The Rise of Skywalker. While Tom Veitch was hopefully paid well for his contribution, considering how Disney has handled other authors such as Alan Dean Foster, he doesn't seem likely. Clone Palpatine and Imperial forces from unknown space all come from this series of comics by Veitch and, if anything, should be credited as such – along with the screenwriters of the film. Here's hoping Disney is more respectful of the people it pulls clear influence from for its billion-dollar properties in the future.
NEXT: 10 Star Wars Legends Storylines That Aged Poorly
10 Strongest Marvel Heroes Shazam Could Defeat
Billie Sparkle
(61 Articles Published)
Born in Little Rock and raised in Texarkana, Billie Sparkle has only just begun writing professionally within the last few months. She's a graduate of Texas A&M – Texarkana with a bachelor's degree in history and holds a fascination for the behind-the-scenes stories of her favorite media. A fan of all things comics, kaiju, and fighting games, she hopes to provide an enthusiastic and peppy vibe to her work.
More From Billie Sparkle
Young performers ready for Sydney Eisteddfod
Borderlands Producer PictureStart Joins Scout's Script2Comic Contest
That '70s Show Co-Creator Cried on That '90s Show Set
The Conjuring 4 Might Be the Franchise's Last
The Nice House on the Lake Sets Up a Sequel
Tanjiro's Most Important Fights in the Anime
Netflix's White Noise Works Better as Psychedelic Horror
Merrily We Roll Along Follows Lindsay Mendez's All Rise Role
WEBTOON Announces 2022 CANVAS Award Winners (Exclusive)
Persona 3 Remake Reportedly In the Works


text:
‹ All Press
LucidWorks Tackles the Problem of Dark Data
LucidWorks Big Data is Industry's First Complete Cloud-based Open Source Software Stack for Prototyping Big Data Applications
San Mateo, Calif. – May 9, 2012 – Organizations today have little to no idea how much lost opportunity is hidden in the vast amounts of data they've collected and stored. They have entered the age of total data overload driven by the sheer amount of unstructured information, also called "dark" data, which is contained in their stored audio files, text messages, e-mail repositories, log files, transaction applications, and various other content stores. And this dark data is continuing to grow, far outpacing the ability of the organization to track, manage and make sense of it.
LucidWorks [formerly Lucid Imagination], a developer of search, discovery and analytics software based on Apache Lucene and Apache Solr technology, today unveiled LucidWorks Big Data™. LucidWorks Big Data is the industry's first fully integrated development stack that combines the power of multiple open source projects including Hadoop, Mahout, R and Lucene/Solr to provide search, machine learning, recommendation engines and analytics for structured and unstructured content in one complete solution available in the cloud.
With LucidWorks Big Data, LucidWorks equips technologists and business users with the ability to initially pilot Big Data projects utilizing technologies such as Apache Lucene/Solr, Mahout and Hadoop, in a cloud sandbox. Once satisfied, the project can remain in the cloud, be moved on premise or executed within a hybrid configuration. This means they can avoid the staggering overhead costs and long lead times associated with infrastructure and application development lifecycles prior to placing their Big Data solution into production.
The product is now available in beta. To sign up for inclusion in the beta program, visit lucidworks.com/products/lucidworks-big-data
Dark Data Problem Is Real
How big is the problem of dark data? The total amount of digital data in the world will reach 2.7 zettabytes in 2012, a 48 percent increase from 2011.* 90 percent of this data will be unstructured or "dark" data. Worldwide, 7.5 quintillion bytes of data, enough to fill over 100,000 Libraries of Congress get generated every day. Conversely, that deep volume of data can serve to help predict the weather, uncover consumer buying patterns or even ease traffic problems – if discovered and analyzed proactively.
"We see a strong opportunity for search to play a key role in the future of data management and analytics," said Matthew Aslett, research manager, data management and analytics, 451 Research. "LucidWork's Big Data offering, and its combination of large-scale data storage in Hadoop with Lucene/Solr-based indexing and machine-learning capabilities, provides a platform for developing new applications to tackle emerging data management challenges."
LucidWorks Big Data
Data analytics has traditionally been the domain of business intelligence technologies. Most of these tools, however, have been designed to handle structured data such as SQL, and cannot easily tap into the broad range of data types that can be used in a Big Data application. With the announcement of LucidWorks Big Data, organizations will be able to utilize a single platform for their Big Data search, discovery and analytics needs. LucidWorks Big Data is the only complete platform that:
Combines the real time, ad hoc data accessibility of LucidWorks (Lucene/Solr) with compute and storage capabilities of Hadoop
Delivers commonly used analytic capabilities along with Mahout's proven, scalable machine learning algorithms for deeper insight into both content and users
Tackles data, both big and small with ease, seamlessly scaling while minimizing the impact of provisioning Hadoop, LucidWorks and other components
Supplies a single, coherent, secure and well documented REST API for both application integration and administration
Offers fault tolerance with data safety baked in
Provides choice and flexibility, via on premise, cloud hosted or hybrid deployment solutions
Is tested, integrated and fully supported by the world's leading experts in open source search.
Includes powerful tools for configuration, deployment, content acquisition, security, and search experience that is packaged in a convenient, well-organized application
LucidWorks's Open Search Platform uncovers real-time insights from any enterprise data, whether structured in databases, unstructured in formats such as emails or social channels, or semi-structured from sources such as websites. The company's rich portfolio of enterprise-grade solutions is based on the same proven open source Apache Lucene/Solr technology that powers many of the world's largest e-commerce sites. LucidWorks's on-premise and cloud platforms are quicker to deploy, cost less than competing products and are more easily tailored to specific needs than business intelligence solutions because they leverage innovation from the open source community.
"We're allowing a broad set of enterprises to test and implement data discovery and analysis projects that have historically been the province of large multinationals with large data centers. Cloud computing and LucidWorks Big Data finally level the field," said Paul Doscher, CEO of LucidWorks. "Large companies, meanwhile, can use our Big Data stack to reduce the time and cost associated with evaluating and ultimately implementing big data search, discovery and analysis. It's their data – now they can actually benefit from it."
Watch the future of open search unfold on the LucidWorks blog
Follow LucidWorks on Twitter and Facebook.
*IDC Predicts 2012 Will Be the Year of Mobile and Cloud Platform Wars as IT Vendors Vie for Leadership While the Industry Redefines Itself, Dec.1, 2011


It isn't possible to reverse the hands of time and prevent yourself from aging. It is more important for you to ensure that the time ahead is enjoyable and healthy than to try to recapture lost youth. Keep reading this article for insightful tips on caring for yourself as you age.
Don't worry about numbers when thinking about aging. You will drive yourself crazy obsessing over your age, height and weight. You most likely pay someone who focuses on your numbers so stop worrying yourself about them.
Resveratrol will aid in aging gracefully. Studies being conducted about the anti-aging effects of calorie-restricted diets are promising. Resveratrol is found naturally in both grapes and nuts. Resveratrol in supplements is often derived from Japanese knotweed roots (the Latin name for which is Polygonum cuspidatum or Fallopia japonica). Another place to find it is the South American shrub called Senna quinquangulata.
Make sure you are sleeping the number of hours you need. Sleeping seven or nine hours can both relax you and help you retain a healthy hormonal balance. The less you sleep, the more irritable you will be in the morning.
Perform random acts of kindness and spread joy. Taking that sunny disposition and spreading it around makes the world a better place and makes you feel better as well. Joy is a free, infinite resource, so spread it around freely and experience the benefits.
Make your home your haven. As we age, health problems may have us moving from rehab centers to assisted living centers and not in our homes. Do your best to create a welcoming environment with objects you like or even modifications if you have the necessary skills.
Be sure to get an ample amount of sleep every day. Most people need between seven and nine hours of sleep each night. Depression and heart-related problems have been connected to people not getting enough sleep every night.
Take the time to stop and smell the roses, and enjoy every day of your life. If you take the time to set milestones like you did for your kids as they were growing, you will find that when you reach those milestones you will feel the same pride as you did all of those years ago.
Avoid environmental extremes. Every season and all types of weather have the potential to damage your skin. It can increase your risk of premature aging of the skin as well as more serious problems, including skin cancer.
Hormone balance is of the utmost importance to all men and women as they age. If you are gaining weight, cannot sleep or feel depressed, you might be suffering from a hormone imbalance. If you find yourself suffering from any of these symptoms, please see a doctor in order to find the best treatment to balance out your hormones, so the aging process is a much more pleasant experience.
Make sure that you are drinking enough water. As people age, dehydration becomes more prevalent. To prevent this, it is suggested that 8-10 glasses of water be drunk every day.
Be sure to check your blood pressure on a regular basis if it is high. If you have high blood pressure you may not even know it, that is why they call it the 'silent killer'. As you age, it is vitally important to monitor your blood pressure. You will have an easy time noticing a problem when you are monitoring your blood pressure, and will be able to address it quickly.
If you want to start taking anti-aging supplements, it is a good idea that your first talk to your doctor. Ask about his recommendations on anti-inflammatory medicines, antioxidants and daily vitamins for you. With the right balance of these supplements as you age, your activity levels will increase and your down time, decrease. Ensure that each of these elements becomes a part of your everyday routine.
Make sure you visit your doctor for testing that will help you better manage your health. If you follow through with regular testing, your doctor may be able to detect potential health problems before they become severe. When it comes to acne problems, the earlier you seek help, the easier it is to manage and treat the condition.
It is good to get as many anti-oxidants into your diet as you can. Antioxidants will help you get rid of free radicals. Most antioxidants can be found in dark-colored veggies and fruits such as spinach, carrots, tomatoes, squash, blackberries and blueberries.
Taking care of the skin is integral to the aging process. Protect your skin from the sun's rays, which can cause cancer or blemishes. Prolonged sun exposure causes premature aging and creates a much greater risk of developing skin cancer.
Stress levels can lead to aging prematurely. A good exercise routine of at least twenty minutes every day is great for maintaining proper health and making you feel more at ease.
Think of age as just a number. Your chronological age is just a number. Your state of mind and physical condition are way more important. If you feel old, then you are old. Your chronological age is just a number; you may feel like you are much younger. The most important thing is not to let your age rule your life.
Look to the foods that you eat as a source of good health. Whole foods contain many vitamins and minerals that are effective at keeping your body feeling fit and young. Eating healthier foods instead of junk foods can help your body, even as you get older.
When you are putting on your makeup, use a primer. Primers are a fairly recent development in the makeup arsenal, and the main ingredient in primers is silicone. These products work to plump up creased areas of the face, add smoothness and facilitate easier application of other makeup.
It's possible to have fun during your retirement. Keep your mind and body in tip-top shape in order to make the most of the time you have left. You will find many sources of information to help you. By learning new ways to stay healthy and strong, your sunset years can be the most rewarding of your life.


GB/T 7714 Wang, Can,Zhang, Shihua. Reveal cell type-specific regulatory elements and their characterized histone code classes via a hidden Markov model[J]. BMC GENOMICS,2018,19.
APA Wang, Can,&Zhang, Shihua.(2018).Reveal cell type-specific regulatory elements and their characterized histone code classes via a hidden Markov model.BMC GENOMICS,19.
MLA Wang, Can,et al."Reveal cell type-specific regulatory elements and their characterized histone code classes via a hidden Markov model".BMC GENOMICS 19(2018).


text:
Q: How add space in between box Hi i'm create website for mobile phone,
but i have a problem
i want add space in between box css
this is screenshot

and this is my code
<?php
include "connection.php";
?>

Sekolah Rekomendasi

            <?php
            $dataskl = mysqli_query($con,"SELECT alamat,diskon,link,logo,kode,nama_sekolah,vvip from sekolah where vvip='Ya' order by id_sekolah desc limit 0,10");
            while ($skl = mysqli_fetch_object($dataskl))
            {
            $jd = $skl->link;
            ?>

                <a data-ajax="false" href='<?php echo $jd ?>'>
                    <img style="float:left; margin-right:8px;width:70px;height:85px;"   src='images/sekolah/logo/<?php echo $skl->logo ?>' class='c_image_1' alt=''>

                <?php if ($skl->diskon=="Ya") { ?>

                <?php } else { } ?>

                <?php
                $ptng_nama =  substr($skl->nama_sekolah,0,15);
                $ptng_alamat =  substr($skl->alamat,0,34);
                ?>

                        <a style="text-decoration:none" data-ajax="false" href='<?php echo $jd ?>'><?php echo $ptng_nama ?>

                        <?php echo $ptng_alamat ?>

                        <?php
                        $datac_pre = mysql_query("select count(id_kunjungan) as hitung from sekolah_kunjungan where kode='$skl->kode' and tgl_kunjungan between '$web->range_awal' and '$web->range_akhir'");
                        $cpre = mysql_fetch_object($datac_pre); ?>
                        Dilihat : <?php echo $cpre->hitung ?>

              <?php  } ?>

How to add space in between box ?
helpme thank's

A: You can use margin or padding property in the css rules for those blocks.
CSS:
body, html {
    margin:0;
    padding:0;
}
div {
    width:100px;
    display:inline-block;
    border:1px solid black;
    height:300px;
}
.a {
    margin-right:20px;
}

HTML:


Nelson Symphony Orchestra (NSO) collects and uses information through our website nso.nz or our online communications (such as email lists) for the purposes set forth within this policy.
NSO complies with the New Zealand Privacy Act 1993 (the Act) when dealing with personal information. Personal information is information about an identifiable individual (a natural person).
When you visit this website we collect anonymous information such as your IP address or domain name to analyse site traffic for "visits" but this information is not tied to a given user's personal information. This information is collected and stored via Google Analytics.
When you make a purchase or donation via our website, certain types of sensitive information, such as financial information, may be collected. We do not receive or retain copies of this information on our server. We redirect visitors to a secure server to complete these transactions and all data is both collected and stored by these processors. Current payment gateways in user are maintained by PayPal and Stripe.
Current payment gateways in user are maintained by PayPal and Stripe.
If you want to exercise either of the above rights, email us at secretarynso@gmail.com. Your email should provide evidence of who you are and set out the details of your request (e.g. the personal information, or the correction, that you are requesting).
If you post your personal information on the website's blog posts, message board or chat room, you acknowledge and agree that the information you post is publicly available.


I have a series of emails dating from the winter of 2015-2016 that I cannot delete, yet I do not reread them either. My inability to do something with those emails is a reminder that grief impacts us in funny ways, especially when it occurs in an unexpected place: at work.
When I woke up that morning and read the email on my phone, I felt hollowed out. The tears started. When I tried to speak to anyone the day we received the news it was impossible to hide my feelings so I did not bother trying. It had been six months since I had seen my friend and coworker. Such is the slow march of cancer. He left for Christmas holidays and never returned. Tears had been spilled, anger had been expressed, the need to keep the team supported took over all else. A sense of loss had already crept in long before the news actually broke.
There are a lot of wonderful memories stored up from that time. In life, we are not prepared to bury coworkers the way we are conditioned to the passing of family. The hole they leave in our lives when we lose them is not fleeting. The people we work with are the number one reason we are afraid to move to another company. The relationships we build with the people we work alongside every day grow strong due to familiarity. Because we see them every day, many of our coworkers come to know about our families, our struggles, and our achievements. They may help us build a deck one weekend; go for a beer after work to share our frustrations, or lend us an extra bit of help at work to get us over a slump.
My friend was one of those. He was one of the first people to interview me when I was hired on. We bonded quickly over my dry sense of humor, and a shared love of learning. Soon we were trading podcasts and book titles and discussing the merits or lack thereof of what we were learning. He grew to appreciate the dark beer we served when we hosted him at our house. He was an optimist who was quick with a laugh and a realist who told it like it was.
He was my friend and I miss him.
I wasn't there to see someone else sit at his desk. I had moved on. They say time heals all wounds though I find that to be a load of crap. I still cry now and then when talking about him or whenever The Tragically Hip come on the radio. (The two are now forever entwined in my world). These things just sort of sneak up on me and I am caught off guard by a quick and powerful wave of emotion that thankfully subsides before most people notice.
The truth is I don't know how to grieve for him. When to close the book. Which brings me to those emails. The majority of the emails I have saved between us occur after he knew he was ill. On the surface, there is really not much inside. They contain a lot of positivity: jokes, favourite lines from Monty Python and an unhealthy amount of cat memes. I imagine his logical self, watching me hover over the delete button and saying "they are just emails of kittens - don't be so sentimental". Did I mention he could be very blunt?
He would be right of course. Compared to five years of memories, what are a bunch of silly emails? I'd like to think he would understand my hesitancy. I don't delete those emails because part of me feels it would be letting go, and I'm just not ready to do that.


text:
In brief: MedPAC recommends CMS shift more products into bid program, AmerisourceBergen buys H.D. Smith
WASHINGTON – The Medicare Payment Advisory Committee made payment policy recommendations for non-competitively bid DMEPOS during a meeting earlier this month, according to Epstein Becker & Green, a healthcare law firm. MedPAC plans to recommend that CMS shift more products away from the "excessive" fee schedule to bid rates. It also plans to call for "immediate reduced payment rates for certain non-CBP products while CMS works on incorporating them into the CBP," the law firm reports in a post for The National Review. Alternatively, MedPAC plans to recommend a policy option that would have CMS consider capping balance billing at a percentage of the fee schedule rate and reduce the allowed amount by 5% for non-participating providers, the law firm reported.
AmerisourceBergen buys H.D. Smith
VALLEY FORGE, Pa., and SPRINGFIELD, Ill. – AmerisourceBergen will purchase H.D. Smith for $815 million in cash, the two companies have announced.
H.D. Smith is among the largest wholesalers in the country, providing a full line of brand name, generic and specialty drugs from 10 distribution centers across the U.S.
"The acquisition of H. D. Smith strengthens our core business and expands and enhances our strategic scale in U.S. pharmaceutical distribution," said Steven Collis, chairman, president and CEO of AmerisourceBergen, a provider of pharmaceutical products that generates more than $150 billion in annual revenues.
H. D. Smith subsidiaries Triplefin, a pharmaceutical brand support provider, and Arete Pharmacy Network, a pharmacy services administrative organization, are not part of the acquisition.
AmerisourceBergen plans to fund the acquisition through the issuance of new long-term debt.
The acquisition is expected to be slightly accretive to AmerisourceBergen's adjusted diluted earnings per share in fiscal year 2018, and be about $0.15 accretive to adjusted EPS in fiscal year 2021.
For fiscal year 2018, AmerisourceBergen now expects revenue growth to be in the range of 8% to 11%, adjusted operating income growth to be in the range of 4% to 7%, and operating growth income for its Pharmaceutical Distribution Services segment to be in the range of 4% to 7%.
The acquisition is subject to regulatory review and other closing conditions, and is expected to close in early calendar 2018.
Part B premiums, deductibles remain stable for 2018
WASHINGTON – The standard monthly premium for Medicare Part B enrollees will be $134 for 2018, the same as 2017, CMS has announced.
Some beneficiaries who were protected from Part B premium increases in prior years will have increases in 2018, but they will be offset by the increases in their Social Security benefits next year, CMS says.
"Medicare's top priority is to ensure that beneficiaries have choices for affordable, high-quality care that fit their needs," said CMS Administrator Seema Verma. "Next year, no beneficiary protected by the hold-harmless provision will see a Part B premium increase that is greater than the increase in their Social Security benefits. We encourage Medicare beneficiaries to explore their options to make an informed choice between original Medicare and Medicare Advantage before Open Enrollment ends on Dec. 7."
The average monthly premium for Medicare Advantage enrollees will drop to about $30 in 2018, about a 6% decrease compared to 2017, CMS estimates. More than 77% of Medicare Advantage enrollees remaining on their current plans will have the same or lower premiums for 2018, the agency says.
The annual deductible for all Medicare Part B beneficiaries will be $183 in 2018, the same as 2017.
Providers meet with Verma
MODESTO, Calif. – Representatives of Home Oxygen Company attended a Nov. 20 meeting with Rep. Jeff Dunham, R-Calif., and CMS Administrator Seema Verma to discuss the challenges of rural providers in California. "Administrator Verma was very attentive to our needs and knowledgeable of our challenges," said Andrea Ewert, president and CEO of Home Oxygen Company. "Our representatives were able to articulate the challenges of running a business and serving Medicare beneficiaries in this financially compressed market." Denham is a co-sponsor of H.R. 4229, which seeks to extend a retroactive delay of a second round of reimbursement cuts in non-bid areas from Jan. 1, 2017, to Jan. 1, 2019. The bill currently has 71 co-sponsors.
Aeroflow partners with Stratice to digitize orders
ASHEVILLE, N.C. – Aeroflow Breastpumps & Urology has partnered with Stratice Healthcare to use eOrdersPlus to digitize orders for medical equipment. eOrdersPlus, developed and operated by Stratice Healthcare, allows Aeroflow to capture documentation, physician signatures and other order-related information using a cloud-based solution. "We are always looking for additional avenues that ease the process for our partners and provide higher quality patient care," said Amanda Baethke, strategic partnerships manager at Aeroflow Healthcare. "Their system links to all the major EHR platforms in our area, and provides an easy way for physicians to prescribe to us directly."eOrdersPlus eliminates multiple phone calls, duplicate faxes and time-consuming data entry, creating a better experience from start to finish, Aeroflow says.
ResMed study results published in CHEST
SAN DIEGO – Patient engagement apps significantly improve adherence to CPAP therapy, according to a ResMed study now published in CHEST. Patients whose nightly use was remotely and self-monitored via ResMed's Airview and myAir platforms were more than 87% adherent on therapy, compared to 70% of those only remotely monitored, according to the study. "CHEST's publication is a significant call for clinicians to recommend self-monitoring tools for PAP patients," said Carlos Nunez, M.D., ResMed's chief medical officer. "I also believe these tools could help patients on therapies beyond sleep as well, since half of all patients don't take their medications as prescribed." ResMed has billed the study, first presented at the CHEST 2016 conference in Los Angeles, as the world's largest to look at sleep apnea and self-monitoring, with 128,000 patients participating.
Toyota challenge to benefit people with lower-limb paralysis
LOS ANGELES – The Toyota Mobility Foundation, in partnership with Nesta's Challenge Prize Centre, has launched a $4 million global challenge to change the lives of people with lower-limb paralysis, culminating with the unveiling of winners in Tokyo in 2020. The challenge seeks teams around the world to create game-changing technology that will help radically improve the mobility and independence of paralyzed people. Technology could include anything from exoskeletons, to artificial intelligence and machine learning, to cloud computing. A panel of expert judges will pick five finalists who will each receive $500,000 to take their concepts from an intelligent insight to a prototype. The winner will receive $1 million to help get their product to market. Among the ambassadors supporting the challenge is Rory Cooper, director of the Human Engineering Research Laboratories at the University of Pittsburgh.
NSM names new CIO
NASHVILLE, Tenn. – National Seating & Mobility has named Mark Marschke its new CIO. The national complex rehab provider recruited Marschke from a unit of Warren Buffett's Berkshire Hathaway. He will oversee NSM's technology strategies and solutions. Most recently, Marschke was CIO of Atlanta-based Larson-Juhl, which designs and makes picture frames. During his time at Larson-Juhl, he was a member of the Berkshire-Hathaway CIO Council.
PHM seeks to finalize restructuring
LAFAYETTE, La. – Patient Home Monitoring on Dec. 15 will ask security holders to vote on splitting the company in two. PHM will become a DME company specializing in delivering and servicing oxygen therapy, sleep apnea treatment and mobility equipment; and Viemed will become a respiratory company specializing in non-invasive vents for patients with COPD, CHF and neuromuscular conditions. PHM announced plans to split into two different companies last year. PHM's board of directors has unanimously approved the arrangement and recommends that security holders also vote in favor of it. Shares of PHM will continue to be listed for trading on the Toronto Stock Exchange under PHM, and shares of Viemed will be listed under VMD.
BraunAbility names new leader
WINAMAC, Ind. – The board of directors of BraunAbility, a manufacturer of wheelchair accessible vehicles and lifts, has selected Staci Kroon as the company's new president and CEO effective Nov. 27. She will also serve as the board's director. The announcement follows the retirement of Nick Gutwein, who has led BraunAbility since 2008. Kroon comes to BraunAbility from Eaton Corp., a $20 billion global technology leader in power management solutions, where she was most recently executive vice president of Eaton Business Systems. Previously, she was president of Eaton's automotive business in North America. Gutwein, who has elected to pursue his work in the ministry full time, will remain a member of BraunAbility's board of directors.
RESNA updates position paper on pediatrics
ARLINGTON, Va. – RESNA has published an updated position paper on the application of power mobility devices for pediatric users. The goal of the paper is to share typical clinical applications, as well as provide evidence from literature, supporting the application of power mobility for young children and to assist practitioners in decision-making and justification. The paper, approved by RESNA's board of directors on Nov. 2, has now been updated with more current and additional scientific literature. The paper, which was worked on by a working group of three or more experts and underwent a 30-day public comment period, is valid for five years.
MedPac,
competitive bidding,
Seema Verma,
Aeroflow,
ResMed,
National Seating & Mobility


of grace from a Savior.
here and above . . .
than the one for me God's got.
but real love like God's . . .
wasn't to be found anywhere.
was love cold and sterile.
for my one true love . . .
Jesus' love is for me.
with all the fulness of God.
God loves us . . .
on me will be shone.
then all you have to do . . .
of what a dying Savior brings.
it took to end mankind's strife.
flowing throughout the Good News.
why Jesus . . . had to die!
to give Him their trust.
everlasting it is . . .
is to make everyone His.
for man to embrace . .
and the gift of His grace!
no other I can find.
Jesus is a perfect fit.
is hard to dispel . . .
of all who let Him dwell.
so to God we'll stay near.
so with it we'll be blessed.
for, love that is perfect . . .
for, God's perfect love . . .
all the stars that shine above.
over the universe can span.
all the water in every sea.
all the sand upon, every shore!
we wish upon a star . . .
hoping our dreams are caught.
we kiss a toad in hopes . . .
it turns into a prince one day.
our heart-strings it will strum.
hoping true love we will score.
but, we know God loves us . . .
for our lives, Jesus bought!
there He, would surely be.
to put faith, in my stride.
beckoning, for me to near.
His perfect love will tame.
do you see Jesus too?
He is waiting there for you!
my soul mate to the end.
bring to it hope and cheer.
in heaven, one day above.
only lasting love like this . . .
can bring my spirit such a lift.
I know I am Yours too . . .
then the one I found in You!
I cannot grasp it . . .
I cannot fathom it . . .
I cannot clarify it . . .
though I can feel it here.
and yet I am amazed . . .
to infinity ~ You love me.
but, I know I cannot live . . .


Buying insurance is kind of a funny thing. We buy it and yet we hope we'll never get to use it. But accidents happen to anyone at any moment no matter how safe or unsafe a driver you are. And when they do, you'll be glad you kept your finances safe.
Whoever is at fault, your car insurance policy can save you. However, the amount of help it gives is entirely up to you. That is determined by the options laid out by your insurer. Despite what the ads say, the best insurance package isn't the same for all. And choosing the package that fits your needs may confuse you.


I was inspired to make this Loquat Upside Down Cake after I stopped by the Mira Mesa Farmers Market. I was browsing around and I ended up picking up some loquats!
Well, that loquat tree died and it no longer there and when I was at the farmers market I saw the fruit and got inspired. I had no idea what I would make when I initially bought them. But I saw a suggestion for using a Rhubarb Upside Down Cake but swapping it for Loquats, which I did!
This is the bottom of the cake. That looks like the top. Cause I still need to flip it.That's how it becomes Loquat Upside Down Cake after all!
I liked that the bottom had a nice crispy crust on it.
And here's how the Loquat Upside Down Cake looks after it's flipped over!
The loquat looks gorgeous on top of the cake, don't you think?
The loquat fruit is cooked in a butter-sugar-lemon mixture and a dough is placed on top of that. The dough puffs up and shapes into a cake while it bakes in the oven and forms a golden crust so the cake is a little crunchy on the bottom.
It's a tender, flaky cake and the cooked loquats are soaked with the lemony butter on top. It's also not toooooooo super sweet – just right if you ask me. This recipe is definitely a keeper! I'd love to try it with other fruits, like nectarines, peaches, pears, and the classic pineapple.
Hope you try it with loquats or another favorite fruit!
Make sure that your loquats are an orange/yellow color. Best to get them off of the vine or still attached to the vine! Cut the loquats in half and take the seeds out. Peel off the outer skin. You should be able to easily remove the skin when it's ripe by just pulling it off with your fingers. Rinse loquats and set aside.
In a 10 inch cast iron skillet, melt 1 cup of the sugar, 4 tablespoons butter, lemon zest and juice, vanilla, and 1/4 teaspoon of salt over medium heat. When the butter and sugar have melted together into a nice gooey mess, add your loquat and cook, stirring occasionally, until the loquat is tender and slightly caramelized, about 8 minutes. For a prettier presentation, flip all loquats so they are facing the same way (I flipped them so the "hole" side was up). Turn off heat and prepare the cake!
Whisk together remaining sugar, salt, flour and baking powder in a bowl. Add remaining butter, and using a pastry cutter (or your fingers), rub into flour mixture to form coarse pea-size pieces (kind of like you're making biscuits). Add milk and beaten egg and stir until a soft, sticky dough forms. If your egg is on the small side, you may need an extra splash of milk for the dough to come together. Stir until a dough fully forms and all flour is incorporated.
Place pieces of dough over the hot loquat butter mixture, covering the whole surface of the pan. The dough will rise up and transform into a luscious cake! Place cast iron skillet onto a baking sheet and bake until the cake is golden and shining and totally cooked through, about 25 to 30 minutes.
Remove from the oven and let the cake rest for 10 minutes. Place a large, flat serving platter on top of the skillet and invert quickly and carefully. Serve warm or at room temperature (warm is best, if you ask me).