text:
\section{Introduction}

Smart cities have been increasing around the world especially with the increasing importance of sustainable development and the effect of AI on our everyday life. It is projected also that the market for such cities will keep on growing~\cite{smartCities_2020}. For a successful implementation of smart cities, a large amount of data is required to better understand its different aspects such as congestion and other transportation network conditions. Visual information have long been collected through the use of many static cameras placed at different locations. Installing such cameras with the required infrastructure is costly and fails to monitor a very large area. Moreover, certain unexpected scenarios require visual information at un-monitored location, such as in the case of huge congestions caused by an accident. As a result, data collected from unmanned aerial vehicles (UAV) or drones have been increasing in popularity for urban traffic monitoring. One such example is the introduction of a new drone-collected large-scale dataset called pNEUMA
~\cite{barmpounakis2020new} which aims to help researchers better study and model traffic congestion. Collecting such large datasets requires analyzing large amounts of objects across many images. This work was traditionally outsourced to manual labellers such as Amazon Mechanical Turk (MTurk)
~\cite{amazon_turk}. With the huge success of deep learning, especially in the field of computer vision, most of the labelling work has become automated. In addition to using UAVs for dataset collection, many people believe that drones is the solution to increase the efficiency of last-mile deliveries~\cite{wang2019vehicle,cheng2020drone,gonzalez2020truck,SALAMA2020620,MURRAY2020368}. This requires the drone to be able to visually understand its surrounding using different sensors. Thus, paradigms for different vision tasks are still challenged with an arms race of methods every year. One of the most prominent vision tasks is \textit{object detection}. From traditional methods such as sliding window and deformable parts model (DPM)~\cite{felzenszwalb2009object} to deep neural networks with different components
~\cite{Fan_2020_CVPR}, the object detection race is still on.

\begin{figure}[t]
\begin{center}
\includegraphics[width=0.55\linewidth]{assets/butterflypull.pdf}

\end{center}
\caption{
Butterfly fields outputted by our method overlayed on top of an aerial image (bottom). The purple vector components are pointing towards the center (orange circle). Using the different components of our butterfly fields allows us to better detect and localize the vehicles in the image}
\label{fig:pull}
\end{figure}

Although general object detectors have achieved promising results on front facing cameras \cite{law2018cornernet,zhou2019objects}, they fail to perform as well when used from a drone/UAV point-of-view. Aerial images capture a large scene made of hundreds of objects occupying a tiny portion of the image with very limited number of pixels (Figure~\ref{fig:pull}). Hence, this adds an exciting challenge to the problem of object detection. In this work, we propose a method tailored to detect objects in aerial images. Our method targets applications such as traffic analysis, smart cities, and more broadly digital twins to name a few.

With the prevalence of UAVs with high resolution cameras, two new datasets were recently released called UAVDT~\cite{du2018unmanned} and VisDrone2019~\cite{zhuvisdrone2018}. A common practice is to re-train and adapt some of the design choices of existing object detectors (\textit{e.g.}, Yolo \cite{redmon2016you}) to the new distribution of the data. For instance, Wu et al.~\cite{wu2019delving} adapted the sizes of the anchor boxes of Faster R-CNN \cite{ren2015faster} to improve its results on aerial images. Ding et al.~\cite{Ding_2019_CVPR} also introduced an ROI transformer to replace the conventional ROI and deal with a common challenge in UAV images, variations in objects' orientations. Yet, state-of-the-art methods still suffer from detecting tiny objects of any rotation.

Recent object detectors, using deep neural networks, can be categorized as anchor-based approaches and anchor-free approaches. On one hand, anchor-based methods leverage strong priors on scale and aspect ratio by pre-defining specific bounding boxes as initial detections. Rather than directly detecting the bounding boxes, the objects are located relative to pre-defined boxes obtained by analyzing the training dataset. Hence, they do not generalize well to objects of different scales or to images of different resolutions, especially when used with different datasets. On the other hand, anchor-free methods focus on locating corners and centers of objects directly with the use of prior knowledge. They are very challenging to train due to the sparsity of their output feature map. This is especially true in the case of objects in high-resolution images, a common characteristic of aerial images.

To overcome the limitations of both anchor-based and anchor-free approaches, we introduce a novel detector,  called Butterfly Detector (BD), that extends and adapts field theory, which has been gaining interest in the 2D human pose estimation task \cite{cao2018openpose,kreiss2019pifpaf,papandreou2018personlab}. In previous anchor-free models, only a specific feature in the output feature map of a model is responsible to localize the center or corner of an object. In our work, we make use of fields, which we call \textit{butterfly fields}, to localize the center and predict the width and height of an object. Each output feature that contains information about the object has a set of field components spatially pointing towards the center. Rather than implementing a naive approach of using all predicted bounding boxes at every output feature, we implement a voting mechanism leveraging the direction and information of related points in a field to improve localization as well as obtain a significant reduction in false positives. We also improved our precision, especially for small objects, by relying on a sub-pixel localization. Our experiments show that we outperform previous state-of-the-art methods on UAVDT and VisDrone2019 dataset. We especially achieve a boost in performance on images that are recorded at high altitudes (Table~\ref{table:sot_uavdt}) showcasing the importance of our method on aerial images. The code will be made available\footnote[1]{https://github.com/vita-epfl/butterflydetector}. Our method can also be tested using the video-labelling tool,
UltimateLabeling\footnote[2]{https://github.com/alexandre01/UltimateLabeling}, where it is used to detect vehicles.

\section{Related Work}

Object detection can be done at 2D scale~\cite{ren2015faster,He_2017_ICCV,redmon2016you,liu2016ssd} or 3D scale~\cite{Meyer_2019_CVPR,Qi_2019_ICCV,Li_2019_CVPR,Liu_2019_CVPR}. Since we are dealing with aerial images, we focus on 2D detection and further divide these methods into traditional detectors, anchor-based and anchor-free deep learning detectors.

\subsection{Traditional detectors for aerial images}
During the rise of satellites and aerial imagery, a lot of effort was put in developing methods to detect different objects~\cite{gupte2002detection}. This was quite challenging because of many different variation such as the small object size and different weather conditions. One of the earliest form of object detection relied on using a hand-crafted template per class and measuring its similarity at every location in the image using the sliding window technique. A lot of advancements have been been done to improve the templates used, from rigid templates
~\cite{chaudhuri2012semi,weber2012spatial} to deformable ones
~\cite{lin2014rotation,ahmadi2010automatic}. However, these methods are difficult to generalities ze to different shapes and orientations. To solve this, researchers moved toward extracting different features that help in localization and classification, such as color intensities and color gradients. One of the most interesting work in detection was done by ~\cite{dalal2005histograms}
 where they introduced histogram of oriented gradients (HoG) for human detection. The features extracted from the image are the distribution of oriented gradients, which allows the detection of edges. Many work since then extended this idea and further improved it to deal with different variations that occur while detecting objects. For instance, rotation-invariant HoG features~\cite{liu2014rotation} were developed to make these features resilient to rotational changes while deformable HoG~\cite{almazan2013deformable} were introduced to be resilient against occlusion and shape changes. Although pioneering works relied on handcrafted features and often dealt with satellite images \cite{cheng2013object,cheng2014multi,eikvil2009classification,grabner2008line,zhang2015generic,zhang2014nonlinear}, we will focus on reviewing recent deep learning based methods tailored for UAV images.

\subsection{Anchor-based detector for UAV images}

When AlexNet~\cite{krizhevsky2012imagenet} was first used for vehicle detection in aerial images, it was able to surpass all traditional methods. Since then, deep convolutional neural networks have dominated object detection with the release of Faster R-CNN~\cite{ren2015faster}, Mask R-CNN~\cite{He_2017_ICCV}, YOLO~\cite{redmon2016you}, and SSD~\cite{liu2016ssd}. Although these methods achieved good performance on natural images (MS COCO~\cite{lin2014microsoft} and PASCAL VOC~\cite{everingham2010pascal}), they perform poorly on high-resolution aerial images. Since most of these methods rely on anchor boxes and objects that are rather big, they require a lot of modifications to improve their performance on a different task such as UAV object detection.

Rather than re-inventing the wheel, many works have extended Faster R-CNN~\cite{ren2015faster}, R-FCN~\cite{dai2016r}, SSD~\cite{liu2016ssd}, and YOLO~\cite{redmon2016you} for vehicle detection~\cite{sommers2018,sommer2017fast,Li2019CCIGT,sommer2018multi,yang2018deep,ringwald2019uav} by proposing many adaptations to overcome challenges characteristic of aerial images. These adaptations include changing the number and size of anchor boxes, increasing the resolution, as well as making use of skip connections in the network. Tang et al.~\cite{tang2017vehicle} proposed replacing the RPN module of Faster R-CNN with a hyper region proposal network (HRPN) that makes use of hierarchical feature maps. Deng et al.~\cite{deng2017toward} extended this work by learning vehicle attributes as well as location and type.

Wu et al.~\cite{wu2019delving} proposed augmenting an off-the-shelf detector to also learn different attributes for every input such as altitude, weather, and viewing angle. This is done in an adverserial manner to disentangle these variations from the extracted features. Another work by Cai et al.~\cite{cai2019guided} uses an anchor-free architecture with background and foreground attention modules as well as multi-level features to improve object detection. In order to deal with small objects, Duan et al.~\cite{duan2019detecting} introduced a channel-aware de-convolution layer to exploit features from multiple channels of different layers. They also developed a Multi-RPN module that performs multiple detection at different layers simultaneously. In contrast, Li et al.~\cite{li2019simultaneously} combined bottom-up and top-down attention mechanisms to extract more discriminative features. By also counting the number of objects in a scene, they were able to show that these two tasks aid each other and lead to a boost in performance. Other methods utilized clustering~\cite{abs_1904_08008}, feature fusion network~\cite{long2019object}, or rotation-invariant cascaded forest~\cite{ma2019vehicle} to perform vehicle detection. All these methods, make use of either extra annotations, attention, or the fusion of multiple features from different layers to deal with different scales. This makes both the training and inference more complicated and slow.

\subsection{Anchor-free Object Detectors}
Due to the drawbacks of anchor-based object detectors, there has been recent work focusing on developing anchor-free detectors. Such methods detect the corners and centers of bounding boxes by outputting a heatmap over the image highlighting their location~\cite{tychsen2017denet,wang2017point,law2018cornernet,law2019cornernet,zhou2019bottom,zhou2019objects}. Point Linking Network (PLN)~\cite{wang2017point} detects the four corners and centers of bounding boxes. Then each corner predicts which output feature cell has a high probability of being the center. Then a bounding box is formed from each pair of center and corner. PLN is different then our Butterfly detector since we do not only make use of corners but rather every point inside the bounding box. Moreover, instead of predicting a probability over all feature cells, each point predicts a vector pointing to the center, allowing us to have a floating point precision of the center location. Law et al.~\cite{law2018cornernet,law2019cornernet}, on the contrary, predict only two opposite corners and use associative embedding to group them together. Since corners usually lie outside an object, they might not contain a lot of appearance features. In order to remedy this problem, Zhou et al.~\cite{zhou2019bottom} suggested using points that lie at the extreme edges of an object. They were able to show promising results, but this limits the use of this method on datasets that contain object masks. CenterNet~\cite{zhou2019objects} showed performance improvement by detecting the center as well as the size of the object. This method also shares many similarities with DenseBox~\cite{HuangYDY15}, GuidedAnchoring~\cite{Wang_2019_CVPR}, FAFS~\cite{Zhu_2019_CVPR}, FCOS~\cite{Tian_2019_ICCV}, and FoveaBox~\cite{kong2019foveabox} where they output two types of maps: (i) a confidence (heatmap) per object class predicting the center of objects, and (ii) a shape map predicting the bounding boxes. Instead of directly predicting a heatmap of centers, we propose to utilize the field vectors to point to object centers. Objects in aerial images experience huge variations in orientation compared to general object datasets (MS-COCO~\cite{lin2014microsoft}). We solve this problem by having many output feature cells point towards the center, decreasing the reliance on specific features such as only the center of an object. To take into account the consensus of all vectors, we develop a voting mechanism to locate the center of an object. This mechanism provides us with a two fold advantage: we not only suppress overlapping predictions, but also accumulate the information thereby providing a more refined output.

Instead of directly predicting a heatmap of centers, we use vectors from different output cells to point to object centers. A similar idea is used for instance segmentation~\cite{Kendall_2018_CVPR,8500621,Neven_2019_CVPR,Cheng_2020_CVPR} where the vectors predicted from different output cells is used to cluster related pixels. In contrast, rather than using these vectors to group feature cells, we make explicit use of where these vectors are pointing to localize the center of an object. We thus develop a voting mechanism to refine our predictions by allowing different vectors to agree to a final prediction, allowing us to deal with occlusion as well as scale variations.

\section{Method}

The goal of our method is to detect objects (such as cars, buses, and pedestrians) in aerial images. We propose an anchor-free method that overcomes the limitations of current anchor-based and anchor-free methods. Existing anchor-free detectors output a sparse heatmap wherein each cell is in charge of only locally detecting either a corner or a center of a bounding box. This is typically challenging for a network due to the fact that it needs to output a low confidence even for features that contain information about the object but do not lie at the specific corner of interest. Thus, we extend fields from 2D pose estimation to object detection and introduce \textbf{Butterfly Fields} which are named based on the shape they form when drawn over an image (Figure \ref{fig:pull}).

Figure~\ref{fig:systemfigure} illustrates the overall method. A base network, such as HRNet~\cite{Sun_2019_CVPR}, is used to extract features from an aerial image. These features are then processed by the head responsible in outputting the composite fields, made up of a confidence, width and height of the bounding box, as well as vectors pointing towards the center of every object. In the following sections, we highlight the main parts of our method and the advantage of using fields for object detection compared to previous methods.

\begin{figure*}[t!]
\begin{center}
 \includegraphics[width=\linewidth]{assets/butterflymethod3_updated.pdf}
\end{center}
\caption{Model architecture showing the different components of our butterfly fields. The features extracted by the base-network are used by the head responsible to output the butterfly fields. The top component, represented by a heatmap over the input, indicates the confidence scalar component. The middle component is made up of vectors where each activated feature cell is pointing towards the center (orange circle). The bottom component shows the width and height predicted by each activated output cell. The resulting fields are then decoded to output the final bounding box predictions (right image)}
\label{fig:systemfigure}

\end{figure*}

\subsection{Butterfly Fields}

General anchor-free object detectors usually assign a single feature cell to be responsible to detect the center of the object by predicting a confidence as well as the offset of the center relative to this cell~\cite{law2019cornernet,wang2017point}. Some methods also detect other attributes such as the width and height~\cite{zhou2019objects}. This forces the neural network to output a sparse feature map with very few and specific activated location while reducing all cells around them. This task is extremely challenging in high-resolution images, where many cells may contain similar information. To overcome this drawback, we introduce \textbf{Butterfly Fields}. The goal of these fields is to localize the center of the object, width, and height from every feature cell that contains information about the object of interest. The difference between other anchor-free methods is explained in Figure~\ref{fig:comparison}. Predicting the center, height, and width, instead of corners does not require association between the corners, which would make inference slower. For instance, if N top-left corners and N bottom-right corners were detected, this requires N$^2$ computations to cluster the corners and form the bounding boxes.

Butterfly fields are composite fields made up of a scalar component for confidence, a vector component that points toward the closest center of an object, and two scalar components indicating the width and height of the predicted object. This can be represented over the feature map \textit{f} as:
\begin{equation}
f^{c}_{ij} = \{p^{c}_{ij}, x^{c}_{ij}, y^{c}_{ij}, w^{c}_{ij}, h^{c}_{ij}\}
\label{eq:onefield}
\end{equation}
where $i$ and $j$ index the output feature cell, and \textit{c} represents the object class. $p$ is the confidence of the cell in its prediction, $x$ and $y$ form the vector from the center of the cell to the center of the object. $w$ and $h$ are the the width and height respectively of the referred object. In the case of overlapping objects of same class, the vector components will point to the closest center since the datasets used do not have any information about relative object positions.

As can be seen from Equation~\ref{eq:onefield}, a composite field is predicted for every object category. This allows the network to output different fields that are specialized for specific objects with specific aspect ratios. For instance, a model can learn that a pedestrian has a specific width to height ratio and thus the output feature map specific for this class would capture this characteristic. On the contrary, previous keypoint-based object detectors predict a common width, height, and even offset for all categories. Our modification allows us to obtain specialized predictors for different classes.

\subsection{Loss functions}
In order to train the model, a ground-truth feature map is built based on the bounding box annotations provided by the dataset. At every feature cell inside a bounding box, the butterfly field components are set such that the confidence $p$ is 1. $w$ and $h$ are set to be the logarithm of the down-scaled width and height respectively in Eq~\ref{eq:onefield}. The vector component ($x$ and $y$) are set to point to the center of the object from the center of the corresponding feature cell. All other feature cells that do not contain any object have their confidence component $p$ set to zero. We also compare the performance of our model when different number of cells around the center are chosen to predict the bounding box. For instance, we test on our model when using a 4x4 window around the center
Figure~\ref{fig:sub-cbbox16} and when using all feature cells inside the bounding box Figure~\ref{fig:sub-cbboxfullfields}.

Two main losses are used in order to train our network. The scalar components that represent the confidence are trained with independent binary cross-entropy, while the components of the field related to the width and height are trained using an L1 Loss. To train the vector components of the butterfly field, we make use of the Laplace Loss~\cite{kendall2017uncertainties,Bertoni_2019_ICCV}:
\begin{equation}
    L = \frac{|x-\mu|}{b} + \log{2b}
\end{equation}
The model learns $b$ without any supervision and is used to attenuate the gradients based on the model's confidence in its predictions. It is important to note that we do not deal with the imbalance of negative and positive samples compared to other anchor-free object detectors that make use of a modified focal loss~\cite{abs_1708_02002}. The focal loss was tested and did not show any improvement compared to using the normal binary cross-entropy.

\subsection{Decoding: Consensus of every point in a field}\label{sec:decoding}

During inference, given the fields over the output feature, we extract the set of bounding boxes with their confidences and appropriate classes. In order to deal with the huge variation of objects, we develop a decoding procedure that scales according to the height and width of every object.

For every class-specific feature output, we build a high resolution confidence map using both the confidence and vector components of the butterfly fields. This produces a heatmap over the input image where the intensity of a region corresponds to its probability of being the center of an object.
We propose the following adaptive function to build our heatmap given the output feature map:
 \begin{equation}
 \small
    f(x,y) =
    \sum_{ij}\frac{p_{ij}}{\chi_{ij}}
    \exp \left(
        -\frac{1}{2}
            \left(\frac{x - x_{ij}}{\sigma^{x}_{ij}}\right)^{2}
        -\frac{1}{2}
            \left(\frac{y - y_{ij}}{\sigma^{y}_{ij}}\right)^{2}
    \right)

\label{eq:gaussian}
\end{equation}
where $x$ and $y$ index a pixel in the high-resolution confidence map, and $i$ and $j$ vary over the output feature cells with confidences greater than $0.1$. Each output cell predicts a center of an object ($x_{ij}$, $y_{ij}$) and confidence $p_{ij}$. Instead of adding the confidences only at the predicted center, Equation~\ref{eq:gaussian} uses an unnormalized 2D Gaussian to spread this confidence in a neighborhood of size $\sigma_{ij}$. A variable number of output cells can predict the same object center and thus we need to deal with the variable normalization of the accumulated confidence. Since summing all the
$p_{ij}$ in Equation~\ref{eq:gaussian} will result in confidences greater than 1, we use $\chi_{ij}$ to normalize the confidence based on the number of expected vectors voting for the same center. This value scales with the number of cells localizing an object center (related to the number of output cells inside the bounding box). In the case of using 16 cells around the center (4x4 window), $\chi_{ij}=16 \ \forall i,j$. When using all the feature cells inside the bounding box to predict the center, $\chi_{ij}$ is the area of the corresponding box. We define $\sigma^{x}_{ij}$ and $\sigma^{y}_{ij}$ as functions of the width and height respectively:

\noindent\begin{minipage}{.5\linewidth}
\begin{equation}
    \sigma^{x}_{ij} = \max\left(2, \frac{w_{ij}}{\rho}\right)
\label{eq:sigma_x}
\end{equation}
\end{minipage}%
\noindent\begin{minipage}{.5\linewidth}
\begin{equation}
    \sigma^{y}_{ij} = \max\left(2, \frac{h_{ij}}{\rho}\right)
\label{eq:sigma_y}
\end{equation}
\end{minipage}\\
These equations can be interpreted as the maximum number of allowed error pixels for predicted bounding box center. This depends on the size of the object since the error in prediction for small objects is more costly than the prediction for big objects. $\rho$ is a hyperparameter that defines the amount of error allowed relative to the size of the object. It can be a single number or a distinct number for each class. We set the minimum value for $\sigma$ to be 2 pixels (shown in Equation~\ref{eq:sigma_x} and ~\ref{eq:sigma_y}).

Based on this voting mechanism, the resulting high-resolution confidence map contains peaks highlighting the probability of the existence of object centers. In addition, another consensus occurs when calculating both the width and height. They are calculated using a weighted average of all predictions following this equation:
\\
\noindent\begin{minipage}{.5\linewidth}
\begin{equation}
    w(x,y) = \frac{\sum_{ij}{p_{ij}w_{ij}}}{\sum_{ij}{p_{ij}}}
\end{equation}
\end{minipage}%
\noindent\begin{minipage}{.5\linewidth}
\begin{equation}
    h(x,y) = \frac{\sum_{ij}{p_{ij}h_{ij}}}{\sum_{ij}{p_{ij}}}
\end{equation}
\end{minipage}\\
where $i$ and $j$ are the indices of all feature cells that have their vector fields pointing towards a specific feature located at $x$ and $y$.

Figure~\ref{fig:comparison} illustrates the two key differences of our method with respect to previous works.
Firstly, the confidence map of previous methods is usually dense only around the center of the object (Figure~\ref{fig:sub-previous}). Such a sparse feature map results in difficulty in training. To tackle this, our method utilizes a greater number of feature cells within each bounding box.  Each feature cell falling inside the bounding box predicts a vector pointing toward the object center (Figure~\ref{fig:sub-cbbox16}-\ref{fig:sub-cbboxfullfields}). Moreover, the feature cell confidence $p$ indicates the confidence of the pointed vector rather than the probability of this cell being the center of an object. A map is also built where each feature cell predicts the height and width at the location pointed by the vector. Secondly, previous methods rely on Non-Maximum Suppression to filter their predictions. Instead, we propose our voting mechanism that allows the network to overcome many challenges such as occlusion. Consider the case in which a confidence map detecting the center is used, such as in CenterNet~\cite{zhou2019objects}; if more than half of the object of interest is occluded, the method would find difficulties detecting the center. Since we leverage cues from different feature cells to detect the center, we are able to overcome such shortcomings. For instance, by detecting only the trunk of a car, our method is able to correctly output a tight bounding box as shown in the qualitative results (Figure~\ref{fig:qualitative}).

\begin{figure*}[t]

\begin{subfigure}{0.3\linewidth}
  \centering
   \includegraphics[width=\linewidth]{assets/example_previousMethods.png}
  \caption{ }
  \label{fig:sub-previous}
 \end{subfigure}%
 \hspace{4mm}
 \begin{subfigure}{0.3\linewidth}
  \centering
  \includegraphics[width=\linewidth]{assets/example_cbbox16.png}
  \caption{ }
   \label{fig:sub-cbbox16}
 \end{subfigure}%
 \hspace{4mm}
 \begin{subfigure}{0.3\linewidth}
  \centering
  \includegraphics[width=\linewidth]{assets/example_cbboxfull.png}
  \caption{ }
   \label{fig:sub-cbboxfullfields}
 \end{subfigure}
\caption{A visualization of the difference between anchor-free methods (a)~\cite{HuangYDY15,Wang_2019_CVPR,Zhu_2019_CVPR,Tian_2019_ICCV,kong2019foveabox} and our buttefly detector (b-c). (b) shows our method using a 4x4 window around to point towards the center. (c) shows our method using all grid cells inside the bounding box. Instead of having a map indicating the center, we make use of fields over the image where each vector is pointing towards the center to localize it. The voting mechanism aggregates information from multiple cells to obtain a refined prediction}
\label{fig:comparison}

\end{figure*}

\subsection{Decoding: Sub-pixel Localization}

Upon building a high-resolution confidence map, cells with high confidence represent the center of objects. A loss of spatial precision occurs when using the center of cells as the centers. As can be seen in Figure~\ref{fig:sub-normalpixel}, using the center of the cell with the most Gaussian overlap might lead to an average between two predictions. This will lead to a loss of precision and will affect the detection of small objects. To remedy this issue, we make use of the floating-precision of our vector components. Instead of taking the center of the highest confident cell as the center of an object (red dot in Figure
~\ref{fig:sub-normalpixel}), we take the exact location the vector is pointing to inside this cell (red dot in Figure
~\ref{fig:sub-subpixel}). This method leads to a boost in performance, especially for the VisDrone2019 since it contains many small objects, such as people and bicycles (see Section \ref{sec:exp}).

\begin{figure*}[t]
\begin{subfigure}{0.5\linewidth}
  \centering
   \includegraphics[width=0.55\linewidth]{assets/normalPixel.png}
  \caption{ }
  \label{fig:sub-normalpixel}
 \end{subfigure}%
 \hspace{4mm}
 \begin{subfigure}{0.5\linewidth}
  \centering
  \includegraphics[width=0.55\linewidth]{assets/subPixel.png}
  \caption{ }
   \label{fig:sub-subpixel}
 \end{subfigure}%
\caption{A visualization comparing using sub-pixel decoding (b) to using directly the high resolution map (a). The yellow circles represent the Gaussian used to fill the high-confidence resolution map (Eq.~\ref{eq:gaussian}). The red circle is the resulting object center since it should be located in the cell with a high confidence}
\label{fig:comparison_subvspixel}
\end{figure*}

\section{Experiments}\label{sec:exp}
To evaluate our method, we study the performance of the butterfly detector on two publicly available UAV datasets: UAVDT~\cite{du2018unmanned} and VisDrone2019~\cite{zhuvisdrone2018}. In order to showcase the importance of the different parts of our method, we also perform a detailed ablation study.

Our experiments show the advantage of using composite fields for object detection. Combined with a high resolution confidence accumulation procedure and a sub-pixel localization, we demonstrate state-of-the-art-performance for object detection in aerial images.
\subsection{Datasets}

\paragraph{UAVDT} is a dataset collected using a UAV platform at different locations, altitudes, weather conditions, as well as viewing angle. This makes it a challenging dataset for both tracking and detection. The dataset also includes annotations for the previously listed variations in addition to the amount of occlusion. It contains three different classes: car, truck, bus. The detection aspect of this dataset is made up of around 41k frames from 50 distinct videos, 20 of which are used for testing.
\paragraph{VisDrone 2019} is a new dataset that is also collected using a UAV platform. Compared to UAVDT, the detection aspect of this dataset is relatively small but contains much more categories. The 8.5k images include 10 classes: pedestrian, people, bicycle, car, van, truck, tricycle, awning-tricycle, bus, and motor. The small number of images and big number of categories introduces another challenge for our method. Moreover, the difference between pedestrian and people category is that pedestrians are people who are walking or standing still.

\subsection{Implementation and Training Details}
Our model uses an ImageNet pre-trained HRNet as a base-network since it recently showed good performance for keypoint estimation, a task that shares some similarities to our method.  We augment the base-network with a head composed of 1x1 convolutional layers responsible to output the different components of our butterfly fields.

During training, we apply different standard data augmentation techniques. After normalizing the images, we randomly flip the input horizontally. The resulting image is then rescaled by a scale between 0.4 and 2.0. We subsequently extract a random 512x512 crop that also undergoes a random $90^{\circ}$ rotation.

As mentioned previously, the confidence map is trained using a binary cross entropy loss. The vector components of our fields are trained using a Laplace loss while all other scalar components, such as width and height, are trained using an L1 loss. In order to find the best prefactor dictating the contribution of each loss on the gradient, we make use of the work by Kendall et al.~\cite{Kendall_2018_CVPR} that weighs the different losses based on their homoscedastic uncertainty, thereby reducing the number of hyperparameters. The model is trained using the SGD optimizer, a momentum of 0.95 and a learning rate of $2x10^{-3}$. The batch size is set to 5, and the training runs for 70 epochs for UAVDT and 150 epochs for VisDrone2019. In the cases where not all the cells inside the bounding box are used to localize the center, a region defined as an "ignore" region, with a size set to 20\% that of the object in question, is created around the used cells

\subsection{Testing Details}
During testing, the image is rescaled to have the width and the height as multiples of 32, a requirement based on the architecture of HRNet. The confidence threshold for selecting centers is 0.05, similar to previous methods. We set the $\rho$ to 10 for the UAVDT Dataset and 5 for Visdrone. The latter requires a smaller $\rho$ because it contains small objects. Once the butterfly fields are extracted, we use the voting mechanism introduced in Section~\ref{sec:decoding} to aggregate and obtain the predicted bounding boxes. The resulting bounding boxes are then refined using soft non-maximum suppression~\cite{Bodla_2017_ICCV}.

Both datasets are evaluated using average precision (AP). UAVDT benchmark requires the AP to be calculated at a specific IOU threshold of 0.7. On the contrary, VisDrone2019 requires an evaluation protocol similar to MS-COCO~\cite{lin2014microsoft} where the average precision as well as average recall are calculated at different IOU thresholds. These metrics are shown in Table~\ref{table:sota_visdrone} and are explained in detail in~\cite{lin2014microsoft}.
\section{Results}

\begin{table}[t]
\caption{Comparison of AP (\%), False Positives (FP), and Recall (\%) with state-of-the-art methods on UAVDT datasets. We outperform previous methods on medium to high altitude images as well as bird-view images. These are significant challenges that general object detectors face on aerial images. BD$_{NV}$ is our method with no voting mechanism and using all predicted bounding boxes. The subscript refers to the number of feature cells used to localize the center. BD$_{full}$ makes use of all feature cell inside the ground truth bounding box. *: trained and tested by us. $fps$: frames per second}
\centering
\ra{1.3}
\resizebox{\textwidth}{!}{\begin{tabular}{@{}llrrcrcrrrcrrrcrrr@{}}\toprule
&& \multicolumn{1}{c}{}&\multicolumn{1}{c}{}& \phantom{a}&\multicolumn{1}{c}{} & \phantom{a}& \multicolumn{3}{c}{Flying Altitude} & \phantom{a}& \multicolumn{3}{c}{Camera View} &
\phantom{a} & \multicolumn{2}{c}{Time} &\\

\cmidrule{8-10} \cmidrule{12-14} \cmidrule{16-17}
& Backbone&FP&Recall&&$avg.$ &&$low $ & $medium$ & $high$ && $front$ & $side$ & $bird$ && $day$ & $night$ & $fps$\\ \midrule
 YOLOv3\cite{redmon2016you}  & DarkNet-53 +FF&--&--&&20.6 && -- & -- &  -- && -- & -- & -- && -- & --&--\\
 SSD~\cite{liu2016ssd} & VGG-16&2,111k&49.28&& 33.6 && -- & -- & -- && -- & -- & -- && -- & --& 42\\
 CADNet~\cite{duan2019detecting} &VGG-16 +FF&--&--&& 43.6 && -- & -- & -- && -- & -- & -- && -- & --&--\\
Faster R-CNN~\cite{ren2015faster} &ResNet-101 &--&--&& 45.64 && 68.14 & 49.71 & 18.70 && 53.34 & 68.02 & 27.05 && 45.63 & 52.14& 2.8\\
GANet~\cite{cai2019guided} &ResNet-50 +FF&--&--&& 47.2 && -- & -- & -- && -- & -- & -- && -- & --&--\\
 NDFT F-RCNN~\cite{wu2019delving} &ResNet-101 &--&--&& 47.91 && \textbf{74.84} & 56.24 & 20.55 && \textbf{64.88} & \textbf{67.50} & 28.79 && 45.91 & 64.16&--\\
 CenterNet$^{*}$[47]& Hourglass-104 &1,204k&66.0&& 53.95 && 70.21 & 60.22 & 27.65 && 57.13 & 66.29 & 38.85 && 62.63 & 67.24&6\\
\hline

BD$_{16}$ & ResNet-50 &453k&62.6&& 52.74&& 68.32 & 61.87 & 21.44 && 57.44 & 65.15 & 34.55 && 62.58 & 69.27&12.2\\

BD$_{NV}$ &HRNet-W32& 2,881$k$ & \textbf{71.28}&& 45.02 &&58.21 & 51.79 & 21.52 && 48.83 & 50.62 & 32 && 50.29 & 64.61&--\\

  BD$_{1}$ & HRNet-W32& 412$k$  & 67.46 &&51.44&& 63.19 & 56.3 & 29.57 && 49.91 & 62.13 & 42.14 && 58.07 & 62.94&18.4\\

BD$_{full}$ &HRNet-W32&  224$k$ & 63.74 &&54.28 && 67.6 & 59.77 & 31.26 && 55.16 & 63.88 & 42.79 && 60.84 & 70.88&15.4\\

BD$_{16}$ &HRNet-W32& 313$k$ &69.46&& \textbf{58.1} && 67.84& \textbf{61.93} & \textbf{40.6} && 56.28 & 66.81 & \textbf{51.57} && \textbf{62.97} & \textbf{69.59}& \textbf{18.4}\\

  \bottomrule
\end{tabular}}
\label{table:sot_uavdt}

\end{table}

\subsection{Comparison to State-of-the-art}

Our method is able to significantly outperform previous state-of-the-art methods on UAVDT dataset. As seen in Table~\ref{table:sot_uavdt} and comparing it to published state-of-the art method(NDFT F-RCNN), our butterfly detector reaches an average precision of 58.1\%, exceeding it by $\sim 10\%$. We also validate our motivation for the use of fields in aerial images. This is shown by the drastic improvement of $\sim 14\%$ in average precision on medium to high altitude images as well as an improvement of $\sim 17\%$ on images taken from bird view. We also train a state-of-the-art anchor-free object detector, CenterNet, on UAVDT. It uses an Hourglass architecture with input size 512x512. Testing is done on the full resolution. Our method outperforms CenterNet by $\sim 4\%$ (Table~\ref{table:sot_uavdt}) while obtaining similar results using ResNet-50. Compared to CenterNet, we especially achieve better results on medium and high altitude images (+1.7\% and +13\% respectively) as well as on bird-view images ($\sim +13\%$)

\begin{table*}[t]
\caption{Comparison of AP (average precision), AR (average recall), True Positives (TP), and False Positives (FP) with state-of-the-art methods on VisDrone dataset. As shown, we outperform previous methods on all metrics. $det.$: specifies the maximum detections chosen. **: trained and tested by us. $\star$: use of sub-pixel localization}
\centering
\ra{1.3}
\resizebox{\textwidth}{!}{\begin{tabular}{@{}lrrrcrrrrcrcr@{}}\toprule
& \multicolumn{3}{c}{AP(\%, det.=500)} &
\phantom{a} & \multicolumn{4}{c}{AR(\%, IoU$_{0.5:0.95}$)} &\phantom{a}& \multicolumn{1}{c}{TP} &\phantom{a}& \multicolumn{1}{c}{FP} \\

\cmidrule{2-4} \cmidrule{6-9} \cmidrule{11-11} \cmidrule{13-13}
& IoU$_{0.5:0.95}$ & IoU$_{0.5}$ & IoU$_{0.75}$ && $det.=1$ &$det.=10$ & $det.=100$ & $det.=500$&& IoU$_{0.75}$&& IoU$_{0.75}$\\
\midrule

  FRCNN\cite{ren2015faster} +FPN\cite{lin2017feature} & 21.4 & 40.7 & 19.9 && -- & -- & -- & --&& --&& --\\
  CenterNet~\cite{law2018cornernet}* & 22.96 & 42.05 & 21.81 && 0.68 & 6.14 & 32.56 & 32.56 && 15.9k&& 36k\\
  ClusDet~\cite{abs_1904_08008} & 28.4 & 53.2 & 26.4 && -- & -- & -- & -- && --&& --\\% 1.3fps
   \hline

   BD & 28.93 & 54.92 & 26.15 && 0.98 & 7.06 & 36.1 & 40.93&& 17.8k&& 46.7k\\

   BD$^{\star}$ & \textbf{30.15} & \textbf{54.67} & \textbf{28.66} && \textbf{0.98} & \textbf{7.11} & \textbf{37.44} & \textbf{41.41}&& 19.3k && 31.7k\\

  \hline
\end{tabular}}

\label{table:sota_visdrone}

\end{table*}

We also obtain state-of-the-art results on the VisDrone2019 dataset (Table~\ref{table:sota_visdrone}). We report our results on the validation set. Our detector is able to attain better performance than all previous methods. ClusDet~\cite{abs_1904_08008}, a two-stage detector, achieves good results since it runs inference on multiple small patches of the image as well as the full image. Although this allows it to be more fine-grained, we outperfom it by $\sim 2.2\%$. Comparing it to CenterNet~\cite{zhou2019objects}, an anchor-free object detector that suffer from the limitations discussed previously, our method outperforms it by $\sim 7\%$ when averaging over the different IoUs and by $\sim 12\%$ at IoU=0.5. The improvement of the AP at IoU=0.75 indicates that our method is able to correctly predict the scale of the objects.

\subsection{Inference Speed}

We also focus on the inference time of previous methods and compare them to our butterfly detector (Table~\ref{table:sot_uavdt}). Most methods did not provide their inference time for comparison. As observed, Faster R-CNN has a low fps due to it being a two-stage detector that relies on a region-proposal network(RPN). Since the architecture and base network of NDFT F-RCNN is based on Faster R-CNN, it is expected that it also does not run in real-time reaching around $~3\ fps$. This is also the case for CADNet that also makes use of a more complicated RPN. Our method performs better than all previous state-of-the-art while running at a speed of 18.4 fps, a real-time performance. All reported inference speeds were obtained using a GTX 1080 Ti.

\section{Ablation Study}

\paragraph{Effect of Fields.} In order to showcase the benefits of fields, we compare in Table~\ref{table:sot_uavdt} our  method (BD$_{16}$) to the same method but using only one feature cell responsible for the center (BD$_{1}$). As can be observed, a gain of $\sim$ 7\% is achieved when different locations in a field are used to vote for the localization of the center instead of only using the center cell. We observe that we achieve a boost of $2\%$ in recall as well. This is due to the fact that different bounding boxes are being proposed based on the extracted features at different output cells.

\paragraph{Effect of Voting Mechanism.}
To study the advantages of voting, we measure the performance of the model when all the bounding boxes predicted by the activated output feature cells are used. Our method without the voting mechanism achieves an AP of 45.02\% (Table~\ref{table:sot_uavdt}) which is almost on-par with Faster R-CNN, a slower two stage detector. As expected, we observe a large difference between true positives and false positives leading to a high recall but low average precision. When the voting mechanism described in section~\ref{sec:decoding} is applied, a significant decrease in the false positives is achieved while preserving the recall. As a result, a significant boost in the AP is observed of around 13\% increasing from 45.02\% to 58.1\%. This improvement is also shown over all the different attributes of the UAVDT dataset.

\paragraph{Effect of Backbone.} In order to better verify the positive effect of our method, we replace the HRNet-W32 with a commonly-used backbone, ResNet-50. Our method with a ResNet-50 architecture is able to outperform previous methods while achieving similar results to CenterNet. These previous methods make use of larger architectures as well as fusion between different feature scales (Table~\ref{table:sot_uavdt}). HRNet-W32 is used in our method since it allows learning from different feature scales similar to CenterNet's backbone, Hourglass-104.

\paragraph{Number of Localizing Cells in a Field.} We also study the effect of the number of points in a field localizing the center of an object on the performance. We evaluate the performance of our method while using one feature cell which is the center (BD$_1$), 16 feature cells around the center (BD$_{16}$), and all feature cells lying inside the bounding box (BD$_{full}$). Table~\ref{table:sot_uavdt} shows the numbers of true and false positives as well as the average precision per attribute for these three configurations. We notice that the best settings is using 16 cells in a 4x4 window to localize the center and predict the width and the height of the bounding box. BD$_{full}$ works by utilizing all features inside the bounding box. Since ground truth boxes are axis-aligned, a bounding box of a car moving diagonally relative to the image will contain areas of the road. In such cases, using the full boxes requires classifying some roads with high confidence leading to a difficult training process. Thus, this can explain the slightly lower performance compared to BD$_{16}$. Nonetheless, our detector with the full bounding box still outperforms previous methods (Table
~\ref{table:sot_uavdt}). As for BD$_{1}$, it makes use of only the center feature to localize the object which makes it challenging to train and decode as explained previously.

\paragraph{Effect of Sub-pixel Localization.} To better observe the effect of the sub-pixel localization, we perform this ablation study on VisDrone2019. As observed in Table~\ref{table:sota_visdrone}, using the sub-pixel technique improves the performance of the model by $\sim 1.2\%$. This method also helped in drastically reducing the false positives while also increasing the true positives.
\begin{figure*}[t]

\begin{subfigure}{0.5\linewidth}
  \centering
   \includegraphics[width=\linewidth]{assets/uavdt_cropped.png}
   \caption{UAVDT}
  \label{fig:sub-first}
 \end{subfigure}%
 \hspace{5mm}
 \begin{subfigure}{0.47\linewidth}
  \centering
  \includegraphics[width=\linewidth]{assets/visdrone_cropped.png}
  \caption{VisDrone2019}
   \label{fig:sub-second}
 \end{subfigure}
\caption{Qualitative results of the Butterfly Detector on UAVDT and VisDrone dataset. Our method was able to perfectly detect partly occluded objects (\textit{e.g.}, images from UAVDT). This is because our Butterfly Detector leverages several locations in a field pointing to the center through a voting mechanism}
\label{fig:qualitative}

\end{figure*}

\section{Conclusions}

In this paper, we propose a new object detector, Butterfly Detector, that utilizes composite fields over the spatial feature map to localize the center of objects and predict their sizes. Using the extracted fields for an object center, we apply a voting mechanism to further improve our detector especially in cases of occlusion or viewing angle variations. We evaluated our approach on UAVDT and VisDrone and show that it outperforms the previous state-of-the-art methods. We further perform an ablation study to demonstrate the benefits of butterfly fields, coupled with a voting process and sub-pixel localization, for object detection in aerial images.

\input{main.bbl}
\end{document}

redpajama_set_name: RedPajamaArXiv


text:
News»Press Releases»Hopson Road Closure Beginning Wednesday
Hopson Road Closure Beginning Wednesday
RALEIGH -A railroad improvement and safety project on Hopson Road in Durham will require the closure of the road between Davis Drive and Miami Boulevard for 10 days, effective Wednesday, Feb. 26 at 9 a.m. The closure is needed so crews can do utility work along the road.
Traffic will be diverted onto N.C. 54 as an alternate route between Davis Drive and Miami Boulevard.
The project, underway since early 2013, is designed to improve safety on Hopson Road, as well as speed up train travel along the Charlotte -Raleigh corridor by eliminating railroad crossings at Hopson Road and Church Street. A bridge wide enough to accommodate any future widening of Hopson Road is being built to carry train tracks over the road, replacing the existing street level crossing.
In addition, Church Street is being realigned to close its railroad crossing, creating an intersection with Hopson Road.
The Hopson Road crossing was the site of a triple fatality in 1992. This event prompted NCDOT to establish its Sealed Corridor Program in partnership with CSXT and Norfolk Southern. The goal of the program is to "seal" the rail corridor between Raleigh and Charlotte by eliminating or improving rail-highway street level crossings.
There is also 3.8 miles of parallel tracks being built from north of I-40 to north of McCrimmon Parkway in Morrisville, and the straightening of existing railroad curves. The passing siding will improve efficiency for both passenger and freight trains by allowing them to both travel through that area at the same time. That will also mean increased track speeds, which will help reduce travel times and improve efficiency.
The project is expected to wrap up by late December 2015.
The $27 million project is a partnership effort. The NCDOT is paying $10.9 million for the railroad bridge, road realignment and grading work for the parallel track. Norfolk Southern is building the additional track line, and North Carolina Railroad purchased the land for the improvements.
The project is part of the Piedmont Improvement Program to modernize railroad track, roads and bridges along the corridor between Raleigh and Charlotte.
Durham/Morrisville Railroad Improvements and Grade Separation
STIP number U-4716ABC

redpajama_set_name: RedPajamaCommonCrawl


This is the structure that was built furthest to the north when Tojo-tei House was first completed in 1884. The outward facing section is to the south, however half of this is blocked by the Naka-zashiki (Spare Room) Building, so sun exposure is not so good.
Yae, the second wife of Akitake Tokugawa, lived in this building. She was the daughter of a lower class samurai who served the House of Tokugawa during the Edo Period (1603 to 1868). At first, she served Akitake as his maid, but he later married her.
At that time, the position of women was lower than that of men in Japan. Furthermore, there was a significant difference in the social status of Akitake, a former candidate to be shogun, and Yae. The configuration of this building reflects this. The living spaces of Akitake and Yae were very different. A samurai with a high social status in the Edo Period would enter into the building where his wife and children sleep directly from the building where he spent his day. It is possible to see from this building that Akitake maintained this kind of lifestyle even in the Meiji Period (1868 to 1912).


Les Anciennes Indes (The Ancient Indies) was a set of eight tapestries created by the skilled artists at the Gobelins manufactory. One of the eight is "Le Point D'eau", which added elements of fantasy to otherwise realistic depictions of plants and animals. These were based on the work of Albert Eckhout (1610-1665), a well-regarded Dutch painter who drew upon his expedition to South America for subject matter. These detailed images were used throughout the eight tapestries. In this particular tapestry, the emphasis is on the horse which is pictured on the left of the larger work. The horse emerges from a thickly wooded forest with exquisite detailing. The colors and vibrancy of the original are reproduced faithfully by the weavers of "Le Point D'eau". This tapestry is lined and has a tunnel for easy hanging.
Dimensions: W 30" x H 60"


Private investigators and private detectives often have to take into account their distance, this means that in conducting their surveillance of an individual or group of persons the private investigators or private detectives should take care in order not to be detected by the really individuals he or she is surveying. Private investigators and private detectives ought to often utilise diverse covers in order not to compromise his or her position while conducting surveillance, which could ultimately compromise his or her investigation.
The life of a private investigator and private detective is constantly on the line and is always at risk when conducting surveillance because once discovered the individuals that he or she is following could harm him or her. Nonetheless, such risk is essential to be able to gather details and data that would prove to be beneficial in solving the case that is why it truly is imperative that the private investigator or private detective need to be equipped with the most recent in surveillance technologies.
1 of the so-called cutting edge technologies inside the field of surveillance and investigation will be the hybrid of binoculars which are incorporated with superior optics with up to eight times magnification. But it's very best function is that it really is combined or integrated using a built-in digital camera which also comes with an LCD screen that users such as private investigators and private detectives can flip-up to evaluation previously captured images. The built-in digital camera has a five-mega pixel resolution, which would therefore give clear images and even records videos of as much as sixty seconds. The Binocular Digital Camera hybrid would allow private investigators and private detectives to conduct surveillance from far away and would still be able to gather information via taking videos and photos. The hybrid of a digital camera with binoculars indeed is regarded as as the private investigators' ideal friend.


We haven't sent any newsletter for quite a long time. We were simply too busy to find time to put it together 🙂 We've decided that we'll definitely send one to all our customers once we launch our brand new AitThemes.club website. We didn't want to be boring, full of text and buttons.
Once we were in the pub discussing what kind of newsletters we are actually reading. Newsletter definitely have to be interesting, it has to include nice or funny pictures. So instead of going to bigstock, grabbing a photo of a girl with a laptop we decided to put ourselves there.
What's the result? It's the most successful newsletter we ever created!


text:
Why GCs Make Ace Candidates For Board Service
Lee Udelsman
Paul S. Williams knows a few things about finding boardroom talent. From 2005 to 2018, he was a partner and managing director of the recruitment firm Major, Lindsey & Africa, where he led searches for executives and board members. A lawyer by education, Williams worked as corporate secretary and chief legal officer at Cardinal Health, and currently serves as a director of Compass Minerals International, a public company, and of the NACD Chicago Chapter. Williams has a keen perspective on how the general counsel (GC) can play an active role in helping to shape the structure of boards, including their bylaws and composition.
At the annual Strategic-Asset General Counsel event—hosted this year in New York City by NACD in partnership with Major, Lindsey & Africa; Sard Verbinnen & Co.; and Weil, Gotshal & Manges—Williams offered his views on the ways in which GCs can think about expanding their roles as board advisors and why they should consider plying their talents from a seat at the boardroom table. Highlights of the conversation, led by Lee Udelsman, managing partner of Major, Lindsey & Africa's New Jersey office, follow.
What are the top trends you see facing boards today?
The first is increased scrutiny. If you compare board dynamics now to those of 20 or 30 years ago, there is a vast difference in the level of scrutiny we are exposed to. On one of my boards, we had an activist situation that culminated in a proxy fight. That scrutiny is indicative of the heightened visibility of the function of the board.
Second is the ascension of diversity and inclusion on the board agenda. The way boards look today is different than it was before both from gender and ethnic perspectives. That's not to say things are moving as quickly as advocates of diversity would like. Nonetheless, I do see change and evolution.
Third is the rise of a more rigorous evaluation process for boards. It used to be that boards were assessing their performance in aggregate rather than on an individual director level. Today, a sizable number of boards evaluate how individual directors are performing. I personally think that's a positive trend because we should be held accountable individually for our performance. Unfortunately, you have situations where a director who once provided a lot of value now needs to consider stepping off. I had a situation in my capacity as lead independent director where a director who had been a strong contributor historically developed earlyonset dementia. Through the board evaluation process, I got feedback from fellow directors and was able to have a fraught conversation with that director. It was difficult, but needed to happen.
Lastly, there's the increased emphasis on board composition. Boards are looking in a more methodical way at skills gaps and filling those gaps, whether it's experience with digital business or crisis management. I think boards are much more purposeful in terms of what they're doing from a composition perspective.
How can the general counsel actively shape board composition?
This is the age of the general counsel. If you look at the evolution of the role, what GCs did 30 years ago is very different from what they do today, and the stature of and respect for the role are different. When you think about a proxy fight, for instance, what really prompts examination is the role that a board is playing, and the GC is right there.
When you think about board recruitment and potential skills gaps on the board, in many instances, the GC is the person who is responsible for spearheading the recruitment of new board members, working hand in hand with the nominating and governance committee to identify board candidates. From a diversity perspective, you have cases where it's the GC who very diplomatically might have a conversation behind closed doors with the nominating and governance committee chair and suggest that it might be helpful to have additional diversity on the board. Maybe it's a situation where the shareholders are making comments about a dearth of diversity. So the GC definitely has input into almost all of these trends. From a board evaluation perspective, the GC often prepares the written instruments that are used for board evaluations. The GC is on the front lines.
Should the general counsel participate in the interview process?
In the vast majority of instances, the GC has a chance to meet with a board candidate. I've served on four boards, and in every instance as a board candidate, I interviewed with the GC. Sometimes I met that person as part of a group, sometimes it was a one-on-one meeting.
Having a conversation with the GC is critical for any potential board member. It's part of your due diligence because the GC is going to have insight into the risk exposures and the character dynamics of the board. I can get a sense of what's to come from the GC were I to join that board.
Are more GCs serving on public company boards today?
I think there's a misconception that most directors don't want lawyers on their boards and that lawyers, specifically GCs, are not serving on boards. That is categorically not the case. Roughly half of all public company boards have at least one lawyer serving as a director.
And lawyers provide value from a tangible skill set perspective, what we know about governance and how to handle a sensitive investigation. Our jobs have exposed us to a lot of information and experiences that are relevant to board service. We bring communication skills. Most lawyers are adept at listening to other people and understanding what the dynamics and concerns are. My personal view is that it really is helpful to have directors from backgrounds outside the C-suite on boards.
What should the dynamic be when a lawyer and a GC are both at the same boardroom table?
Many GCs are skeptical about having another lawyer in the boardroom, but I believe that it's helpful because there are issues that another lawyer is going to understand clearly that you as a GC will need to bring to the attention of your board. In many ways you have a kindred spirit, and as long as the lawyer who is serving on the board knows he or she is not there to also serve as the GC, it can be a helpful and healthy relationship.
I've always had positive working relationships with the GCs of the boards I've served on. And I've been able to navigate issues on their behalf. There might be something the GC is concerned about but, for political reasons, feels awkward approaching in the boardroom. If that's the case, maybe I can broach that issue and be on the front lines, asking particular questions. I see a lawyer on the board as a natural ally.
When should a GC commence the search for a board seat?
Definitely when you're still in the GC chair. Think about board service before you retire. It's preferable to have your first board role while you're still working.
What's the benefit of board service?
Once you get that first directorship, you've already demonstrated you have credibility and other board appointments tend to flow more easily. You're often perceived as having knowledge and skills that are current. For me, the way I got my first directorship was that a member of our board had reached a mandatory retirement age on all of his boards, and he had seen me in my capacity as GC and said, "Paul, would you have an interest in replacing me?" I was skeptical at first about how I would juggle the time commitments, but I'm glad I pursued it.
When seeking a directorship, should a GC stay in the same industry as his or her day job?
It doesn't have to be the same industry. If you're still working and it's your first directorship, in some ways it's better that it not be the same industry from a conflict perspective. It's amazing how often we join boards in industries that are pretty far afield from where we started. When I went on an insurance company board, I'd never had experience working in financial services. Over time, I learned the business and ultimately I became the lead independent director. Think beyond your own industry.
How can you manage the time commitments of a day job and a directorship?
Get the board calendar. If there's a direct conflict, you can't realistically take on that directorship. Assuming there's no conflict, then you may have intermittent conflicts, but every board member has that challenge to deal with. You can't get yourself in a situation where you are going to consistently miss regularly scheduled board meetings due to your role as GC.
If offered a board opportunity, what should the candidate focus on?
First, it's time. The average directorship demands 200 to 300 hours a year, so you need to make sure you're realistically able to do that. Second, I'd make sure you're aligned with the culture and the risk calibration of that board. My experience is that every company has its own level of risk calibration. If you are someone who's highly risk averse, you probably don't want to go on the board where there's a very aggressive CEO or that hasn't seen a risk they haven't been afraid of.
Lee Udelsman is Managing Partner in the New Jersey office of legal recruiting firm Major, Lindsey & Africa. Lee recruits general counsel, compliance leaders and other high-level, specialized legal professionals for a range of organizations and industries.
Paul Williams is Partner Emeritus and Advisory for the In-House Counsel Recruiting of legal recruiting firm Major, Lindsey & Africa.


text:
Islanders' playoff run ends in heartbreaking Game 7 loss to Lightning
Mollie Walker
Social Links for Mollie Walker
More On: new york islanders
Islanders rookie gets first NHL goal after false positive
Islanders hit key milestone with marathon shootout win
Islanders' key trio finally starting to live up to expectations
Barry Trotz surprises Islanders in return from COVID list
TAMPA — They say that Lightning isn't supposed to strike in the same place twice, but the Islanders evidently are the exception.
For the second straight season, the Islanders fell just short of a trip to the Stanley Cup Final, as the Lightning beat them 1-0 Friday night at Amalie Arena to close out the semifinal series and tee up a chance to defend their title against the Canadiens.
One win, and one goal, separated the Islanders from the franchise's first championship series since 1984.
"This group believed that we could do this," a somber head coach Barry Trotz said. "We still believe we could. It's just a lot of pain because they gave their all. There's some guys that are playing that are beat up, worn out, hurt and they just keep going.
"There's a bond between all of the players that is really strong and the disappointing thing is we didn't get this game, and that group in that room won't be together again. That's just the lay of the land in the National Hockey League."
The Lightning jump the boards to celebrate their Game 7 win.
Tampa Bay fed off its noisemaker-bearing crowd and dictated the pace of play for a majority of a scoreless first period, outshooting the Islanders 15-5. But when the Islanders were supposed to be the ones in control of the game, on the lone power play of the night in the second period, that was the beginning of the end for their season.
The power play has been a weak spot for the Islanders all season long, but it was disastrous in Friday's loss.
When Barclay Goodrow was penalized for cross-checking Adam Pelech 50 seconds into the second period, with the game scoreless, the Islanders actually started off strong. Once the Lightning cleared their zone and Anthony Cirelli carried the puck to the other end, however, three Islanders skaters came to challenge Tampa Bay's second-line center — but they lost sight of the rest of the ice.
Yanni Gourde then hopped off the Lightning bench, skated in uncontested and buried a shorthanded goal — the only one of the game — at 1:49 of the middle frame. It was the first Game 7 in the history of the NHL to end in a 1-0 score off a shorthanded tally.
"It hurts no matter what," Josh Bailey said of the Gourde goal, his voice quivering. "That's a situation we certainly would like to have back. At the end of the day, it stings to lose. So hard to just get to this point."
Still, the Islanders made the franchise's longest playoff run since 1993, when they lost in five games in the conference finals to the Canadiens.
Matt Martin and the Islanders couldn't generate much offense in Game 7.
With Friday's loss, not only did the Islanders' season come to an end, but also Nassau Coliseum's run as the team's home officially expired.
Sometimes, Trotz pointed out, teams need to suffer some heartache before being able to win a Stanley Cup. After two agonizing semifinal losses to the Lightning, the Islanders have certainly had their fill of playoff pain.
"You're down one goal going into the third period of Game 7 in the conference final, you'll take that," a choked up Mathew Barzal said. "Credit to them, they locked us up pretty good in the third period. Couldn't find one, couple good chances.
"It's not our time. It was right there. Hopefully, back next year."
2021 nhl playoffs
Read Next Islanders hoping this do-or-die chance ends as happily as ...
Brian Costello
The time for Joe Douglas to turn the Jets into contenders is now
Mark Cannizzaro
This underrated team will make noise in NFL playoffs
Pursuing Zion Williamson to reunite Duke's Big 3 is as risky as it is tantalizing
Sudden Olympics opportunity gives goalie Strauss Mann chance to prove he belongs in NHL
Postseason shows younger coaches are staking their claim in NFL rat race
Enjoy Member-Exclusive Stories and More!
The best insights from the ultimate insiders
Jalen Rose: Renaissance Man
Listen on Apple PodcastsSpotify
Pinstripe Pod: A NY Yankees Baseball Podcast
Amazin' But True: A NY Mets Baseball Podcast
Blue Rush: NY Giants Football Podcast
Gang's All Here: A NY Jets Football Podcast


text:
Go to My Account Center
Orders Pending Approval
View Your Account Team
Browse Product Categories
Tech Solutions Library > New Data Centers Support Growth Potential
New Data Centers Support Growth Potential
By increasing capacity and decreasing downtime, Hydromax USA builds a foundation for a strong future.
There's Big Data, and then there's really Big Data. It was the latter that prompted Hydromax USA to scrap its elderly, maxed-out data center and replace it with two data centers capable of handling the company's aggressive growth.
Based in Chandler, Ind., Hydromax USA is a professional services firm that helps utility companies and contractors locate and assess the condition of underground water, sewer and natural gas infrastructure. For example, using satellite technologies originally developed to find water on other planets, Hydromax USA can scour thousands of square kilometers for leaks and then pinpoint the location of each within an area the size of a tennis court. The company also uses sonar and remote cameras to assess the condition of sewer systems.
You don't have to be in the utility business to appreciate that this level of scrutiny generates a lot of data. In fact, that's the takeaway of this story: Companies in other verticals that store and analyze tens of terabytes of video, images and other media (whether X-rays or archived video conferences) can learn from the way Hydromax USA stocked its new data centers.
The number of hours of unplanned downtime at Hydromax USA in 2016, compared to 70 hours (at $10,000 per hour) in 2014
It was all about the service: having somebody who can get on the phone and help me keep everything straight. The whole project, start to finish, CDW was invaluable.
Nick Schapker, System Administrator, Hydromax USA
Keep Pace with Growth
Between the time Hydromax USA was founded in 2003 and 2014, it relied on a single Dell server: initially, a PowerEdge 1900, later swapped for a 2950. When Nick Schapker was hired as system administrator in October 2014, the company had replaced the 2950 with an R510 and implemented a 25TB Dell PowerVault MD1000 storage array.
"Hydromax was very near 100 percent capacity," Schapker says. "They were unable to pursue additional projects or customers due to lack of storage space."
Hydromax USA bought some time with a few secondhand servers and storage devices purchased via eBay.
"We attempted to build very cheap commodity storage using storage technology that was already phased out of use in most environments," he says. "It became clear to ownership that the cost to get and maintain this older technology, with its modest performance gains, was just not sustainable."
"In the history of Hydromax USA, very little had been spent on the core infrastructure. In 2015, we invested the most the company had ever spent on storage and compute resources, about $10,000, to prove to ownership that this was sorely needed."
Yet even this expansion — from 25TB of capacity to 60TB — couldn't match the need, says Schapker. "The capacity expansion gave the company a huge growth potential, which was utilized in the same year, resulting in the biggest year to date for the company."
During this period, the company's IT burden wasn't the only part of the business that was growing. Gross revenue doubled, illustrating the relationship between the company's bottom line and its data center capabilities. It also highlights why Hydromax USA needed major upgrades ASAP.
Select the Right Solutions
Schapker joined Hydromax USA from a local computer consultancy, where he often worked with CDW. That positive experience put CDW on his short list when it came time to rip and replace Hydromax USA's data center.
"There, I was introduced to what CDW can do for its customers. Having the ability to get a representative on the phone and connect with other engineers or consultants was invaluable. Many times it made the difference for our customers. I brought that relationship with me to Hydromax USA. The relationship has really augmented the IT department in a very positive way."
Dan Lah, CDW data center solution architect, visited Hydromax USA to understand its challenges, recommend options and work with Schapker to test potential gear. One of Lah's suggestions, which Hydromax USA agreed with, was deploying the EMC Isilon for video storage.
"Isilon really is the market leader for scale-out network-attached storage repositories," Lah says. "Scale-out NAS is a good fit for large amounts of video. It makes it easy to manage if it can be one pool. On their previous solution, they had to store video in a bunch of different logical buckets — run out of space, and throw it on another storage area network."
A single logical pool of storage is also easier to maintain, Lah says. Unlike a traditional SAN, technicians don't have to manage a redundant array of independent disks or logical unit numbers. And Isilon makes it easy to do a technology refresh.
"As you add nodes to the cluster, and some other nodes get three to five years old and ready to retire, you just push a button," says Lah. "The data evacuates automatically from the old node, and you can just pull them out. So you can have a constant, rolling refresh cycle, so to speak, and avoid future data migrations."
Hydromax USA also picked the EMC VNXe, an entry-level yet reliable SAN to support the virtual machine workload.
EMC Isilon's scale-out network-attached storage was a good fit for Hydromax USA's video data.
A Smooth Transition
CDW shipped all the EMC gear, along with APC racks and uninterruptible power supplies and Dell servers and switches, to Hydromax USA's Louisville, Ky., data center.
"It was like Christmas morning," Schapker says. "You have these racks and boxes that you really want to get into and put to work."
CDW assembled everything, seeded all the data, tested the equipment for about four months, then divided out one rack's worth of data for shipment to the second new data center. The team spent a weekend breaking the clusters apart and moving them, another step where Schapker says CDW's involvement was critical.
"CDW's assistance with the project was invaluable. To do this without them would have been very tough. Moving this rack of equipment to a new location involved setting up a completely new infrastructure in that office. This was completely new compute, storage, networking and Active Directory environments. Without CDW's assistance we would not have been able to complete this migration in the time we had allotted."
"Working with CDW made this whole migration project really smooth. There could have been unforeseen obstacles along the way, but with support for every step in the process — initial setup, testing, power down, physical relocation to another state, power on and reconfiguration — this project went off without any issues. It was smoother than bringing online a new mail server."
CDW also helped Hydromax USA migrate about 82TB of data. "That was a painless process," Schapker says. "They gave me the white-glove treatment."
The company now has the scalability necessary to maintain its aggressive growth.
"By adding these storage and compute resources we are able to support the growing needs of the company. It also allowed the company itself to pursue larger customers and contracts. It has allowed the company to really expand in our market space."
Maximize Uptime and Redundancy
The new data centers also enable the company to make good on its reliability goals.
"In 2014 we made a commitment to our customers, that we would hold all of the data we collect in-house, on our own servers," Schapker says. "Prior to these upgrades, were there to be an event that affected the data center, a natural disaster for example, that would have been the business at stake. Now we have full redundancy in geographically dispersed locations. We are ready to handle a natural or man-made disaster and continue operations with minimal interruptions."
CDW implemented VMware Site Recovery Manager, which gives Hydromax USA the necessary automation to feel secure in the event of a disaster. Should one occur, Schapker's team can simply push a button to fail over to the disaster recovery site.
"SRM also provides the ability to do routine testing without requiring production system outages and special configurations," says Herb Numrich, CDW senior consulting engineer for virtualization. "Management of the replication and the disaster recovery plans are all integrated into the vSphere Web Client, providing Hydromax with a single pane of glass to manage its disaster recovery solution."
The new infrastructure has also minimized downtime and its associated costs. In 2014, the company had 70 hours of unplanned downtime at $10,000 per hour.
By 2016 the unplanned downtime had been reduced by 96 percent, to just three hours. "Implementing this new infrastructure has really changed our business," Schapker says.
"We are ready to handle a natural or man-made disaster and continue operations with minimal interruptions."
On the Move: Relocating the Data Center
When CDW helps clients migrate facilities, it sometimes ships data over a dedicated, high-bandwidth virtual private network leased for that task. It's a smart option for major migrations, particularly in markets where broadband providers offer big pipes on a short-term basis.
That type of connectivity isn't always available, however, and it wasn't an option for Hydromax USA's data centers. As an alternative, CDW worked with a third-party data center relocation company to transport a rack of equipment.
"If it's not something we do in-house, we have vetted partners we can bring in to do a physical relocation," says CDW's Dan Lah.
CDW built out two vSphere environments at the company's primary location, staging the equipment for the disaster recovery site and seeding the replicated data.
"We implemented vSphere replication to provide low-cost replication services for the solution, which is integrated with vSphere," says CDW's Herb Numrich. "Once the data was replicated locally to the disaster recovery environment, the equipment was physically moved to the actual disaster recovery site, and we assisted with the reconfiguration of components to reflect their new location."
The relocation is another reason why Nick Schapker and his team at Hydromax USA are thrilled with the project's outcome.
"It was all about the service: having somebody who can get on the phone and help me keep everything straight," he says. "The whole project, start to finish, CDW was invaluable."
Photography by Patrick Cavan Brown
Call us at 800.800.4239 to set up a consultation with a data center expert.
MKT21171
CDW®, CDW•G® and PEOPLE WHO GET IT® are registered trademarks of CDW LLC. All other trademarks and registered trademarks are the sole property of their respective owners.
Copyright © 2007 - 2018 CDW.


As the Patient Protection and Affordable Care Act continue to evolve, widespread regulatory updates are constantly changing the healthcare landscape. For this reason, it is essential to retain a CPA firm who understands this ever-changing tax and health insurance legislation and the impact it will have on your business. One of Hall & Company's main priorities is to strengthen and maintain your organization's financial health and keep your business in compliance with shifting federal and state laws and regulations.
Our expertise in the healthcare industry allows us to assist a wide range of clients including single or multiple physicians' groups, dental practices, radiology practices, and diagnostic service providers, among others.
At Hall & Company our healthcare team has extensive experience in managing increased cash flow, maximizing tax savings, minimizing compliance risks and achieving faster reimbursements from insurance providers.


Willis in collaboration with the Chartered Quality Institute - Ensuring, Assuring & Insuring Responsible Business Practices: Why aren't quality accreditations given direct recognition by insurers?
Share Willis in collaboration with the Chartered Quality Institute - Ensuring, Assuring & Insuring Responsible Business Practices: Why aren't quality accreditations given direct recognition by insurers?
Voluntary Standards & the Insurance Industry: Why aren't quality accreditations given direct recognition by insurers in the form of premium reductions or enhanced cover?
Is the insurance industry not valuing voluntary standards and certification as part of clients' risk assessment and management? If not, then should these standards constitute a recognised variable for mitigating business, operational and societal risk by insurers in the form of premium reductions or enhanced cover? The challenges and opportunities raised by these questions will be addressed through an informed debate presented by industry experts including bsiand panel discussion with questions from the audience.
Wednesday 3rd February 2016: 8.00-10.30am.
Sponsored by Willis in collaboration with the CQI (North London Branch) Key note speaker Maureen Sumner Smith Managing Director BSI, Chaired by Alderman Professor Michael Mainelli FCCA FCSI FBCS, Gresham Professor of Finance, Executive Chairman Z/Yen.
Is the insurance industry not valuing voluntary standards and certification as part of clients' risk assessment and management? If not, then should these standards constitute a recognised variable for mitigating business, operational and societal risk by insurers in the form of premium reductions or enhanced cover? The challenges and opportunities raised by these questions will be addressed through an informed debate presented by industry experts including bsi and panel discussion with questions from the audience.
Voluntary standards are embedded in a wide range of industries as part of businesses' strategic culture. This includes over 100 ISO standards in buildings and civil engineering, 177 in nuclear energy, 285 in shipping and marine, over 600 in plastics, and more than 800 in the food industry. These standards assure customers of consistent quality in products, services, processes, systems and people. They are a means by which organisations can assure key stakeholders that risk is managed effectively. Compliance to a voluntary standard is independently assessed through third party verification and auditing. In this respect, ISO 14001 certification can be used by organisations in annual financial accounts as evidence of effective environmental management required by the Companies Act 1976 (Strategic Reports and Directors Reports) Regulations 2013.
The recently revised ISO 9001:2015 is a standard that applies across all industries providing organisations with an explicit framework and systematic approach for risk assessment and management. Top management including Directors and shareholders is accountable. Top management is required to demonstrate leadership and commitment by ensuring risks, that can affect the conformity of a product or serviceand well-being of the business as a whole, are determined and addressed. Statutory and regulatory requirements need to be identified, understood and complied with.
The insurance industry itself has adopted voluntary standards but not to the same extent as other industries. The insurance industry may also acknowledge clients' use of voluntary standards including; the topical ISO 27001:2013 (Information security management) relevant to any industry sector that needs to have appropriate measures in place to manage monetary and data security; ISO 22301:2012 (business continuity management), BS8533:2011 (flood risk assessment), and appears to like BS OHSAS 18001 (Occupational health and safety management). However, the adoption of voluntary standards by the insurance industry itself and direct recognition of clients embedding these standards in their organisation is not a variable that forms part of consistent insurance industry practice and culture. Should it be?
Dr Louise Boulter is a Senior Lecturer in the Department of International Management and Innovation at Middlesex University Business School. She recently chaired the CQI's National Strategy for Quality committee and is the academic advisor to the Advisory Council of the CQI. Louise is an author of several books for the Financial Times and numerous papers including research on the relationship between Quality and improved shareholder value. Recent research concerns appetitive risk and personality types.
Hilary Smith-Milne is a chartered quality professional with over 20 years expertise in the oil and gas industry (the first national and international expert PAS99:2006 and PAS99:2012), Hilary is on the Advistory Council of the CQI.


Here is our review of Teen Wolf Season 6 Episode 12 "Raw Talent," in which I talk about Scott and Malia getting close, Gerard's arrival and more.
Teen Wolf airs on Sundays at 8pm MTV. You can find all our Teen Wolf coverage here.


text:
Allison L. Westfahl Kong
Improving the Protection of Species Endangered in the United States by Revising the Distinct Population Segment Policy
While one primary goal of the Endangered Species Act is to prevent the global extinction of species, it is less clear whether the Act is intended, and can be used, to protect species that are endangered solely within the United States. Although the global preservation of species may be sufficient to achieve many of the goals of the Endangered Species Act, some goals may only be completely served by ensuring that certain populations of species occur within the United States, even if the animals are abundant elsewhere. The current Distinct Population Segment Policy being used by the Fish and Wildlife Service and the National Marine Fisheries Service to determine whether to list domestic populations of species as threatened or endangered only allows the agencies to protect these population segments if they are significant to the species' taxon as a whole. This Note argues that this policy should be changed because there are many compelling reasons to protect domestic populations of particular species, even if these species are abundant elsewhere, and suggests criteria that should be used to determine whether a particular population segment should be protected, including the species' conservation status and importance to the American people. It also demonstrates that this proposal would be consistent with the goals of the Endangered Species Act.


text:
INSIDER Law
FDA Supplement Director Seeking Collaboration in Developing Pre-DSHEA List

Steven Tave, director of FDA’s Office of Dietary Supplement Programs (ODSP), acknowledged creating such a list of pre-DSHEA, or grandfathered, ingredients poses challenges.

Josh Long | Mar 13, 2017
An FDA official said in an interview his office is seeking to collaborate in developing an authoritative list of ingredients exempt from a notification requirement in the 1994 Dietary Supplement Health and Education Act (DSHEA).
"That's an area where I think there's a lot of opportunity for us to work collaboratively with stakeholders going forward," said Steven Tave, director of FDA's Office of Dietary Supplement Programs (ODSP), who acknowledged the potential benefits of an authoritative list for the industry and his agency. "To the extent that we don't have to review notifications because the law doesn't require it, it preserves our resources to work on things that we need to."
In a 102-page draft guidance document published in August, FDA disclosed it is "prepared to develop an authoritative list of pre-DSHEA ingredients, based on independent and verifiable data." The list would reflect ingredients exempt from a notification requirement in the law because they were marketed before Oct. 15, 1994, and presumed safe.
Although dietary supplements aren't subject to premarket approval, companies must notify FDA at least 75 days before introducing a new dietary ingredient (NDI)—providing evidence the NDI is reasonably expected to be safe. FDA issued draft NDI guidance for the industry in efforts to improve compliance with the notification requirement and improve the quality of submissions.
Tave described the benefits of the NDI notification process for the estimated 170 million Americans who take supplements, saying it reflects FDA's only premarket opportunity to review the safety of dietary ingredients.
"And from a public health perspective, having a robust, functional, meaningful NDI review process has clear benefits to consumers," he said in a phone interview.
FDA often objects to NDI submissions for various reasons, such as a finding that the notification didn't provide sufficient evidence of the ingredient's safety. However, Tave said the notices indicate companies "are trying to do the right thing."
"They're trying to notify us," Tave said, "and they're trying to give us the information that we need to see in order to know that they've demonstrated a reasonable assurance of safety.
"We want to give them an understanding of what kind of information we need to see," he added. "We want to improve that success rate, and we want companies also to know when they do need to file a notification…."
Pre-DSHEA List Poses Challenges
The development of an authoritative list of pre-DSHEA ingredients is a work in progress.
Tave acknowledged creating such a list poses challenges. For example, what is the standard for determining whether an ingredient was marketed before 1994, and what are the variations of the ingredient that would be covered, he asked.
"And then what's going to be the process for developing that list," Tave said. "Who is involved? How do we apply it? How do we do it consistently and fairly and in a way that's meaningfully quick, so that it can be useful to industry and consumers and healthcare providers who are looking at that information as well?"
Tave said he hopes to make progress in this area in the coming months, and he underscored the endeavor would be a collaborative and transparent one involving industry and consumer interests.
Master File: 'A Win-Win'
The concept of an NDI master file, Tave noted, is one of the areas in the draft guidance in which the agency "absolutely" needs feedback from the industry.
In the draft guidance, FDA cited circumstances in which a company filing an NDI notification could rely on data from a master file, provided the company obtained written permission from the submitter of the master file. The concept could make it easier for several finished supplement manufacturers to take an ingredient to market, while safeguarding an ingredient company's proprietary information.
Master files would enable companies to "protect and capitalize on their intellectual property," said Tave.
Herbalife Ltd., the multi-level marketer of supplements, proposed the master file concept in 2011 draft NDI guidance. The company sought to eliminate the practice of copycatting or piggybacking in which companies are marketing a substance that is identical to, or resembles, an ingredient that has been the subject of a successful NDI notification to FDA.
Last year in an interview, Herbalife senior executive Vasilios "Bill" Frankos said his company was seeking to avoid "me-too ingredients that are knockoffs of NDIs that were successfully submitted" to FDA.
Herbalife's master file proposal also reflected an effort to eliminate duplicative NDI submissions to FDA for the same ingredient sold by different supplement manufacturers. The idea seemed to resonate with Tave.
He described the master file as an avenue for his agency to meaningfully review NDIs "without the burden of, No. 1, unnecessarily duplicative submissions on industry's part and, No. 2, [FDA] having to review unnecessarily duplicative submissions.
"Since we have limited resources, I think that's a win-win," Tave explained.
But FDA must balance its support of the master file concept with its duty to protect public health. And Tave signaled the agency hasn't ironed out all the details, such as those related to intellectual property protections.
FDA to 'Carefully' Review Comments
FDA received hundreds of comments from the industry and others on the 2016 draft NDI guidance. Larissa Pavlick, VP of Global Regulatory and Compliance with the United Natural Products Alliance (UNPA), met in person with Tave and his colleague, Cara Welch, Ph.D., ODSP's senior advisor, to discuss the guidance. Pavlick indicated the agency was open to hearing criticisms from the industry.
For example, UNPA expressed concern the draft guidance interprets various manufacturing changes as triggering the NDI notification requirement. According to Pavlick, Tave said the agency needed to clarify the part of the guidance related to manufacturing changes.
"He (Tave) directly told me he didn't want to be flooded with notifications related to manufacturing changes, and that wasn't their intent," she said in a phone interview.
Pavlick said FDA officials planned to hold additional industry meetings, while finalizing the guidance.
"They didn't know if those meetings would be public, or if they would be just with trade organizations, but they did need additional feedback to kind of finalize some of those areas, and they suggested that manufacturing changes would be one of those," she explained.
In the interview with INSIDER, Tave did not address Pavlick's comments regarding her discussion with him about manufacturing changes. However, he said the agency is open to meeting with representatives from the industry and other third parties.
Tave observed FDA received many substantive comments that addressed various aspects of the draft document. "We're going to read every single one of them carefully, and we're going to think about every single point that's raised in there," he said.
'It's a Complex Statute'
Tave also addressed criticisms that the guidance—at 102 pages—may be too complicated given the seemingly straightforward notification requirement in DSHEA.
"It's always a balance between simplifying and being comprehensive and responsive," he said, explaining much of the document is filled with Q&As for industry. "We want to look at this from a firm's perspective. Here's my question. How would I answer it?"
Commenting on DSHEA itself, Tave said, "The statute is short, but it's a complex marketplace, and it's a complex statute … speaking as a lawyer and as a human being. It's not easy to make sense of all the different provisions that are packed into DSHEA.
"We're trying to give useful guidance," Tave continued, "so that nobody goes down the wrong path."
Publishing Final Guidance is 'Priority'
Tave couldn't provide a specific timeline for publishing a final NDI guidance. He noted the many layers of clearance and review that are outside his control.
Industry sources said the final NDI guidance will be subject to review by the White House's Office of Management and Budget (OMB). Complicating matters, President Donald Trump's policies have raised the specter that the publication of such documents could be delayed or scrapped entirely by the administration.
Tave said he hoped it wouldn't take five years to publish a final guidance—the time that elapsed between the 2011 NDI draft and the publication of the 2016 document.
"It's a priority for the agency to get this out because I think that firms want to see it," he said. "I think industry expects it, and reasonably so."


Once the basement of a small fortified temple, Lockhart's Delve has been home to a variety of creatures and groups since the destruction of the temple proper. Some maps of the old temple of the moon god show two entrances into the lower levels – one into the basements and another into the connected crypts. But some fifty years ago the crypt entrance was completely filled with stone and earth under the orders of Short Terog, an ogre adventurer who made this her home for a time.
Most recently, the Blackguard Lockhart has used the structure as barracks while assembling their forces to cross the dark barrens and assault Oceansong Harbor. Since the fall and subsequent recapture of Oceansong Harbor, the dungeons have seen little use except by the shadows of the kobolds who came here looking for their clanmates.
The blame for this dungeon falls squarely on the shoulders of Evey Lockhart. Four years ago she challenged a few of us over on Google+ to try doing our art in a medium we normally didn't use. As I was poor at the time, instead of buying other art supplies, I tried drawing a dungeon map using software instead of my usual penwork.
But finally, four years later, I've gone back and redrawn the original design in my current style.


This week's Smart City features discussions with Harvard economist Ed Glaeser about shrinking cities, and with urban design critic Randy Gragg about interactive art and urban design projects.
Glaeser talks about what makes cities work and questions the intentions of shrinking cities that are trying to grow back to their previous economic power.
Gragg is the former Architecture and Urban design critic for Portland's daily newspaper, The Oregonian. He has been a close observer of that city's evolution to what is widely considered to be one of the nation's most successful cities. He is collaborating on a fantastic mash-up of art, architecture and urban design called City Dance in downtown Portland's public fountains designed by Lawrence Halprin.