text:
Promising Budget 2019 Instills Hope in India's Renewable Industry and E-Mobility Segment
The new budget is here and it sounds promising for the country's fledgling non-conventional energy sector.
This year's budget shows a marked increase in the budgetary allocations for renewable energy sources. The budget has an outlay of ₹9.20 billion (~$130 million) for wind power, ₹30.05 billion (~$440 million) for solar (both grid-connected and off-grid) and ₹5 billion (~$73 million) for green energy corridor that aims at synchronising electricity produced from renewable sources with conventional power stations in the grid.
A budgetary outlay of ₹40.66 billion (~$590 million) has been fixed for the power supply program of Deendayal Upadhyay Gram Jyoti Yojana (DDUGJY), and ₹10.35 billion (~$150 million) for the power system development fund.
In her speech, Finance Minister Nirmala Sitharaman said, "We will use the approach of mission LED bulb to promote the use of solar stoves and battery chargers in the country."
PIB India
�"
@PIB_India
India's 10- point 'Vision for the Decade' flagged in #Budget2019
India to become a 3 trillion Dollar Economy this year and a 5 trillion Dollar Economy by 2024-25
Details ➡http://pib.nic.in/PressReleseDetail.aspx?PRID=1577411 …#BudgetForNewIndia
2:12 PM - Jul 5, 2019
The budget also underlines the relief that will be provided to solar manufacturers in India. The minister stated that to boost economic growth and the Make in India program, the government will launch a program to invite global companies through a transparent competitive bidding process to set up mega-manufacturing plants in advanced technology areas such as semi-conductor fabrication (FAB), solar photovoltaic cells, lithium storage batteries, solar electric charging infrastructure, computer servers, and laptops. The government plans to provide them investment-linked income tax exemptions under Section 35 AD of the Income Tax Act, among other indirect tax benefits.
In its comment, the National Solar Federation of India chairman Pranav R Mehta said, "NSEFI is very happy and welcomes the proposal in the Union Budget for sunrise and advanced technology in the manufacturing areas for semiconductor fabrication, lithium storage batteries, solar electric charging infrastructure, and solar PV cells, by offering them investment-linked income tax exemptions and other indirect tax benefits. We are particularly elated by the incentives for the important future areas of lithium storage batteries and solar electric charging infrastructure." However, he was quick to add a note of caution saying that the specific details of these proposals should be looked out for.
"I would like to congratulate Hon'ble FM, Smt Nirmala Seetharaman for presenting her maiden budget. The industry was expecting a policy direction from the government to promote manufacturing through Make In India program, especially in the renewable energy sector. The government announcement to launch a program aimed at encouraging global companies (through competitive bidding process) to set up mega-manufacturing plants in sunrise and advanced technology sectors appears to be a step forward in that direction. We welcome this move as it will boost the Make in India Program. Having said that, setting up mega manufacturing plants of solar photovoltaic cells and modules require a larger support structure in the form of soft loans, export credits in order to compete globally," said Rajendra Kumar Parakh, Vikram Solar, Chief Financial Officer.
In a significantly positive development for the mobility segment, the budget puts the impetus on electric vehicle (EV) adoption in India. To make EVs affordable to consumers, the government will provide additional income tax deduction of ₹150,000 (~$2,190.97) on the interest paid on loans taken to purchase EVs. This amounts to a benefit of around ₹250,000 (~$3651.63) over the loan period to the taxpayers who take loans to purchase EVs. Moreover, in order to promote e-mobility, customs duty on certain parts of electric vehicles will also be exempted.
Shailesh Chandra, President – Electric Mobility Business and Corporate Strategy, Tata Motors, commented on the budget saying, "The incentives announced today by the Finance Minister, in terms of additional interventions and steps to support the EV adoption, reinforces a strong commitment by the government to steer electrification on a faster trajectory. The proposal to lower the GST rate for EVs to 5% and reduction in duties of EV components, which we are studying, is a welcome step. It will help in further narrowing down the cost of ownership gap against engine vehicles. Additionally, private buyers, who were earlier not considered for a subsidy through FAME II, will now have a reason to seriously consider an EV with the tax exemption of up to ₹150,000."
Nishchal Chaudhary, Founder & CEO- BattRE Electric Mobility Pvt Ltd, said, "I think the budget has clearly indicated the positive intent of the government in pushing the e-mobility revolution in India. The question of when is not that important as the how. The 'how' is now getting addressed. Consumer adoption of this technology not only requires education but also an attractive and feasible overall proposition. I think with the government proposing lowering of GST and providing a deduction of tax on loan interest, the outlay for infrastructure for battery charging stations, push towards Make in India initiative are just the measures required to kickstart the process."
Moreover, the government has announced that it will support private entrepreneurship in driving value-addition to farmers' produce from the field and for those from allied activities, like bamboo and timber from the hedges and for generating renewable energy.
In her maiden speech as the finance minister, Sitharaman stated that the government is examining the performance of the UDAY program and it will be further improved. The center will work with the state governments to remove barriers like cross subsidy surcharges, undesirable duties on open access sales or captive generation for industrial and other bulk power consumers. Besides these structural reforms, considerable reforms are also needed in tariff policy. A package of power sector tariff and structural reforms will soon be announced, according to Sitharaman.
On the topic of rural development, Sitharaman announced that all willing households in the country will have electricity by 2022 along with clean energy fuel. It is important to note here that the government's rural electrification program SAUBHAGYA was launched in 2017 with a deadline of December 2018. However, the government is yet to achieve universal electrification in the country. The program's dashboard shows that though 100% electrification target has been achieved in all the states except Chhattisgarh — which has a 99.67% electricity penetration.
This budget demonstrates the government's intent to boost programs that have been announced for the growth of solar, wind, EV, and energy infrastructure sectors. A budgetary allocation of ₹123.5 billion (~$1.80 billion) has been made for the Ministry of New and Renewable Energy (MNRE).
Guru Inder Mohan Singh, Director, and Chief Operating Officer, Amplus Solar commented on the budget, " The focus on attracting investment in advanced technology area is a welcome move. Technology is typically born in universities in Europe and the U.S. and then commercialized in China before it reaches India. By promoting advanced technology manufacturing here, India can short circuit this process and attract FDI, especially in the background of the U.S.-China trade war. This promotion will also generate employment and slow down India's brain drain. The vision of India being a global hub for manufacturing of electric vehicles is a pleasant move too. In the past, India has lost to China in the solar manufacturing space. A concerted effort on battery manufacturing will ensure that India does not lag far behind China, despite them having already secured the pole position."
Sunil Rathi, Director, Waaree Energies, commented on the budget announcement saying, "While we appreciate the government's focus on environmental reforms, it is imperative to focus on the solar segment as a key contributor for clean energy, which is missing from the Union Budget 2019-20. With the economic viability of the solar power coupled with the fact that conventional energy sources now have to match solar parity, it would have been heartening to see more focus on the solar segment to promote ecological stability. The infusion of ₹700 billion (~$10 billion) in the public sector banks to stabilize the economy will in-turn benefit the NBFCs, which, in the absence of a recognized banking unit to support small to mid-scale solar financing, will provide an impetus to the solar project financing. However, the invitation to foreign PV manufactures to set up shop in India, without prior stabilization of the domestic manufacturing market, is premature and may prove to be counterproductive for the demand in the sector, which will render the NBFC financial support redundant."
"The government has been indicating some changes in the solar segment for a while. However, we believe that there are critical gaps that need to be plugged. On one hand, while the safeguard duty provided the industry with interim relief, the short-sighted implementation of a year holds the proverbial sword of uncertainty in the industry. Moreover, the lack of tangible movement on the anti-dumping policy has dampened the business projections in the segment. The only silver lining in the budget is the progressive movement towards adoption of EVs and the incentives being offered to the end consumer. The promotion of clean energy through the use of EVs is likely to boost the demand in the segment, thus providing impetus to achieve economies of scale and in-turn create a viable ecosystem.," he added.
Budgetary allocation for green energy corridor, off-grid solar, FAME, power system development fund shows that the government is working on all problem areas.
News Source: https://mercomindia.com/budget-2019-renewable-e-mobility/


text:
<?php
namespace persiang\Grids\Components;
use Illuminate\Foundation\Application;

/**
 * Class TFoot
 *
 * The component for rendering TFOOT html tag inside grid.
 *
 * @package persiang\Grids\Components
 */
class TFoot extends HtmlTag
{
    const NAME = 'tfoot';

    /**
     * Returns default set of child components.
     *
     * @return \persiang\Grids\Components\Base\ComponentInterface[]
     */
    protected function getDefaultComponents()
    {
        if (version_compare(Application::VERSION, '5', '<')) {
            $pagerClass = 'persiang\Grids\Components\Pager';
        } else {
            $pagerClass = 'persiang\Grids\Components\Laravel5\Pager';
        }
        return [
            (new OneCellRow)
                ->addComponent(new $pagerClass)
        ];
    }
}

redpajama_set_name: RedPajamaGithub


On 23 August the Minister for Indigenous Affairs, Nigel Scullion introduced the Social Security Legislation Amendment (Community Development Program) Bill 2018. The Bill will amend the Social Security Act 1991 and the Social Security (Administration) Act 1999.
As announced in the 2018–19 Budget, the reforms will build on the success of CDP and focus on strengthening remote communities.
enable more people in remote Australia to have greater choices for their employment pathways.


text:
Q: How can I pass array of pointers between my functions in C? I am biginner in C and PRO*C and need some help. I have a structure as below:
  typedef struct pt_st{
   char (*s_no)[100];
   char (*s)[100];
    } pt_st;

I have a function like c_info which calls post function:
int c_info(pt_st ir_st)
{
int   li_result = 0;
li_result = post(ir_st.s_no)
}

and post function is :
int post(char *is_st)
{
//do something
}

When I compile my program , I get three errors:
warning: passing arguments post from incompatible pointer type
warning: passing arguments post make integer from ponter without cast
warning: passing arguments post make ponter from integer without cast

Does anyone kow how can I fix this?
Thank you!

A: pt_st.s_no as well a pt_st.s both declare a pointer to an array of char.
So the function post() needs to expect such, like:
int post(char (*s_no)[100]);

If the shown definition of int post(char * is_st) cannot be changed, then call it like this:
pt_st s = ... /* some initialisation */

int result = post(*(s.s_no));

redpajama_set_name: RedPajamaStackExchange


text:
\section{Additional Data Analysis}
\subsection{Statement Length Distribution}
The length distribution for positive and negative statements are presented in Figure~\ref{fig:real_len} and Figure~\ref{fig:fake_len}, respectively. There is no significant bias in statement lengths for positive and negative statements.
\begin{figure}[h]
\begin{center}
   \includegraphics[width=\linewidth]{img_app/real_len_dist.jpg}
\end{center}
   \caption{Distribution of positive statement lengths.}
\label{fig:real_len}
\end{figure}
\begin{figure}[h]
\begin{center}
   \includegraphics[width=\linewidth]{img_app/fake_len_dist.jpg}
\end{center}
   \caption{Distribution of negative statement lengths.}
\label{fig:fake_len}
\end{figure}
\subsection{Statement Content}
Table~\ref{table:common_words} shows the most common nouns, verbs and adjectives in positive statements, respectively.
\begin{table}[t!]
\begin{center}
\small
\begin{adjustbox}{scale=0.9,center}
\begin{tabular}{c|c}
\hline
Type & Most Common Words \\
\hline
     & man, woman, shirt, suit, hair, jacket, girl, lady, boy, dress,\\
Noun & sweater, friend, brunette, room, guy, people, tie, glass, table,\\
     & car, coat, door, hat, phone, hand, top, bed, house, couch, group\\
\hline
     & tell, wear, ask, want, sit, try, say, talk, go, explain,\\
Verb & walk, get, make, look, see, think, take, give, will, hold,\\
     & can, stand, know, come, leave, feel, have, find, put, like\\
\hline
     & black, blue, blonde, red, white, brown, green, haired, young, dark,\\
Adj  & grey, old, other, pink, purple, upset, plaid, gray, yellow, long, little\\
     & blond, happy, good, excited, surprised, striped, light, angry, short\\
\hline
\end{tabular}
\end{adjustbox}
\end{center}
\caption{Most common words in positive statements.}
\label{table:common_words}
\end{table}
\subsection{Video Length Distribution}
The video clips collected from MovieClips are all 40 seconds long. For video clips collected from TV shows, their lengths vary from 15 to 40 seconds, shown in Figure~\ref{fig:vid_len}.
\begin{figure}[h]
\begin{center}
\hspace{-0.5cm}
   \includegraphics[width=\linewidth]{img_app/video_len_dist.jpg}
\end{center}
   \caption{Distribution of video lengths from TV shows.}
\label{fig:vid_len}
\end{figure}

\section{Instructions for Human Annotators}
Figure~\ref{fig:ins1} through \ref{fig:ins4} show the detailed instructions and user interface for human annotators.

\section{More Examples}
Figure~\ref{fig:movie} and Figure~\ref{fig:tv} show some more examples of predictions from our model on movie and TV show clips. The model used in these examples is Stmt+Subtt+Vis with BERT and Img features.
\begin{figure*}[h]
\begin{center}
\includegraphics[width=\linewidth]{img_app/instruction1.png}
\end{center}
   \caption{Overall instructions for human annotators.}
\label{fig:ins1}
\end{figure*}
\begin{figure*}[h]
\begin{center}
\includegraphics[width=\linewidth]{img_app/instruction2.png}
\end{center}
   \caption{Instructions for writing real statements.}
\label{fig:ins2}
\end{figure*}
\begin{figure*}[h]
\begin{center}
\includegraphics[width=0.9\linewidth]{img_app/instruction3.png}
\end{center}
   \caption{Instructions for writing fake statements.}
\label{fig:ins3}
\end{figure*}
\begin{figure*}[h]
\begin{center}
\includegraphics[width=0.9\linewidth]{img_app/instruction4.jpg}
\end{center}
   \caption{User interface for workers to cut an interval from the video and write three pairs of real and fake statements.}
\label{fig:ins4}
\end{figure*}

\begin{figure*}[h]
\begin{center}
\includegraphics[width=\linewidth]{img_app/qualitative_study_movie.pdf}
\end{center}
   \caption{Examples on movie clips. The pos/neg at the beginning of each statement indicates its ground truth. The {\cmark} or {\xmark} at the end of each statement indicates the system's prediction. {\cmark} means the system judges the statement as positive, and {\xmark} means negative.}
\label{fig:movie}
\end{figure*}
\begin{figure*}[h]
\begin{center}
\includegraphics[width=\linewidth]{img_app/qualitative_study_tv.pdf}
\end{center}
   \caption{Examples on TV show clips. The pos/neg at the beginning of each statement indicates its ground truth. The {\cmark} or {\xmark} at the end of each statement indicates the system's prediction. {\cmark} means the system judges the statement as positive, and {\xmark} means negative.}
\label{fig:tv}
\end{figure*}

\section{Introduction}
Joint vision-and-language understanding sits at the nexus of computer vision and natural language processing (NLP), and has attracted rapidly growing attention from both communities. Popular tasks include visual question answering~\cite{antol2015vqa,goyal2017making}, referring expression comprehension~\cite{yu2016modeling,yu2018mattnet}, visual dialog~\cite{das2017visual}, visual reasoning~\cite{johnson2017clevr,suhr2018corpus,hudson2019gqa}, visual commonsense reasoning~\cite{zellers2019recognition}, NLVR$^2$~\cite{suhr2018corpus}, and visual entailment~\cite{xie2019visual}. The emergence of these diverse Vision+Language tasks, benchmarked over large-scale human annotated datasets~\cite{lin2014microsoft,krishna2017visual}, has driven tremendous progress in joint multimodal embedding learning~\cite{tan2019lxmert,lu2019vilbert,chen2019uniter,su2019vl}.
However, most of these datasets and models were centered on static images,
leaving the joint modeling of video and its aligned textual information (\emph{e.g.}, video-and-language understanding) a relatively under-explored territory.

Video Question Answering (Video QA) is one of the most popular tasks in current studies for video-and-language understanding.
Video QA model aims to answer a natural language question given a video clip.
Existing Video QA datasets include MovieFIB~\cite{maharaj2017dataset}, MovieQA~\cite{tapaswi2016movieqa}, TGIF-QA~\cite{jang2017tgif}, PororoQA~\cite{kim2017deepstory}, and TVQA~\cite{lei2018tvqa,lei2019tvqaplus}.
While these datasets have covered a rich pool of video content (\emph{e.g.}, cartoons, short GIFs and TV shows), they are limited to QA task only.
On the other hand, in NLP field, one important benchmark for natural language understanding is natural language inference (NLI)~\cite{bowman2015large,williams2017broad}, where a model is presented with a pair of sentences (premise and hypothesis), and judges the relationship between the pair (\emph{e.g.}, \emph{Contradiction}, \emph{Neutral}, and \emph{Entailment}).

Inspired by NLI,
we present a novel task, Video-and-Language Inference, to foster deeper investigations in video-and-language understanding. Specifically, given a video clip with aligned subtitles as premise, and a natural language statement as a hypothesis describing the video content, a model is expected to infer whether the statement is entailed or contradicted by the given video clip.
This new task is easy to evaluate, since only binary classification is measured; but also challenging to solve, as a thorough interpretation of both visual and textual clues is required in order to achieve in-depth understanding and inference for a complex video scenario.

We introduce a large-scale dataset for this new task, \textbf{VI}de\textbf{O}-and-\textbf{L}anguage \textbf{IN}ference (\textsc{Violin})\footnote{Project page: \url{https://github.com/jimmy646/violin}.}, built upon natural video content with rich temporal dynamics and social interactions. Video clips are collected from diverse sources to cover realistic visual scenes, and statements are collected from crowdsource workers via Amazon Mechanical Turk (AMT)\footnote{\url{https://www.mturk.com/}}, who watched the videos accompanied by subtitles (dialogue, scene description, etc). Our goal is to provide a dataset that can test a model's cross-modality reasoning skills over both video and textual signals. To this end, we require AMT workers to write statements based on joint understanding of both video and subtitles, which not only describe explicit information in the video (\emph{e.g.}, objects, locations, characters, social activity), but also reveal in-depth comprehension of complex plots (\emph{e.g.}, interpreting human emotions and relations, understanding the events, inferring causal relations of events throughout the video). This distinguishes our collected statements from the straightforward captions in video/image captioning dataset~\cite{lin2014microsoft,krishna2017dense,wang2019vatex}, which are dominated by explicit factual descriptions without deeper inference.

Writing negative statements for an inference task is challenging~\cite{bowman2015large,zellers2019recognition}. To gather high-quality negative statements without artificial cues or biased priors, we employed two strategies in the data collection: ($i$) requiring annotators to write negative statements by changing just a few words or phrases in a positive statement, to ensure that the style and length of the statement remain unchanged; ($ii$) performing adversarial matching~\cite{zellers2019recognition}: for each video, select challenging and confusing statements from the statement pool of other videos as the negative ones. The first strategy ensures the collected statements can test a model's in-depth inference ability, since only a small fraction of a positive statement is modified, which requires the model to distinguish highly similar statements with different meanings. The second strategy focuses more on testing a model's global understanding of the video, to distinguish statements with high-level scene difference between videos. When combined together, these two strategies produce a dataset with minimal visual or textual bias. Through this effort, we collected 95,322 video-statement pairs, containing 15,887 video clips spanning over 582 hours. Each video is paired with 6 statements and is 35.2 seconds long on average.

The main contributions of this paper are three-fold. ($i$) We propose a new task, Video-and-Language Inference, which requires a model to draw inference on whether a written statement entails or contradicts a given video clip. ($ii$) We introduce a new dataset \textsc{Violin} for this task, providing a reliable benchmark for measuring joint video-and-language understanding models.
($iii$) We provide a detailed analysis of the \textsc{Violin} dataset with evaluation over strong baselines, and suggest future directions for this new task.

\begin{table*}
\begin{center}
\small
\scalebox{0.9}{
\begin{tabular}{l c c c c c c }
\hline
source & \# episodes & \# clips & avg clip len & avg pos. statement len & avg neg. statement len & avg subtitle len\\
\hline
Friends & 234 & 2,676 & 32.89s & 17.94 & 17.85 & 72.80\\
Desperate Housewives & 180 & 3,466 & 32.56s & 17.79 & 17.81 & 69.19\\
How I Met Your Mother & 207 & 1,944 & 31.64s & 18.08 & 18.06 & 76.78\\
Modern Family & 210 & 1,917 & 32.04s & 18.52 & 18.20 & 98.50\\
MovieClips & 5,885 & 5,885 & 40.00s & 17.79 & 17.81 & 69.20\\
\hline
All & 6,716 & 15,887 & 35.20s & 18.10 & 18.04 & 76.40\\
\hline
\end{tabular}}
\end{center}
\vspace{-2mm}
\caption{Statistics of different video sources used to create our dataset.}
\label{tab:data-stat}
\vspace{-2mm}
\end{table*}

\section{Related Work}
\paragraph{Natural Language Inference (NLI)}
Understanding entailment and contradiction relations between sentences (\emph{i.e.}, Natural Language Inference) is fundamental to natural language understanding. Several large-scale datasets have been developed as NLI benchmarks, such as SNLI~\cite{bowman2015large} and MultiNLI~\cite{williams2017broad}. NLI is also included in the GLUE benchmark for evaluating general language understanding~\cite{wang2018glue}. Recent introduction of large-scale pre-trained language models, such as BERT~\cite{devlin2018bert}, XLNet~\cite{yang2019xlnet}, and RoBERTa~\cite{liu2019roberta}, has propelled significant progress in NLI. Multi-task learning and adversarial training~\cite{liu2019multi,zhu2019freelb} also prove to be helpful in improving model performance.

Inspired by NLI, we propose the task of Video-and-Language Inference to evaluate a system's multimodal reasoning ability. However, different from NLI, our task is more challenging in the sense that both video and text (subtitles) are provided; thus, a thorough joint understanding of both modalities is required for inference.

\vspace{5pt}
\noindent\textbf{Visual Entailment} \,
Visual Entailment (VE)~\cite{xie2019visual} is a recently proposed task that extends NLI to the visual domain. In this task, a natural image premise and a natural language hypothesis are given, and the goal is to judge whether the textual hypothesis can be confirmed based on the visual content in the image. Three labels are assigned: \emph{Entailment, Neutral}, and \emph{Contradiction}. The dataset is created based on Flickr30k image captions~\cite{young2014image} and SNLI~\cite{bowman2015large}. Similarly, NLVR$^2$~\cite{suhr2018corpus} is proposed to investigate the grounding relationship between given images and a natural language description.

Our proposed task is different from VE in the following aspects. ($i$) VE considers images as input, while our task focuses on videos instead. Compared with static images, videos contain complex temporal dynamics, making the video-and-language inference task  more challenging as the model needs to understand the relationship between different visual scenes to draw inference. ($ii$) Our proposed task requires deeper visual understanding. Images in the VE task are mostly natural images, while the videos in \textsc{Violin} were collected from popular TV shows and movie clips, which contain rich social interactions and diverse scenes. This requires a model to not only understand explicit visual cues, but also infer in-depth rationale behind the scene. ($iii$) Our task requires more sophisticated language understanding. VE is a combination of Flickr30k~\cite{young2014image} and SNLI~\cite{bowman2015large}, with no crowdsouring involved. The hypotheses in VE task are composed of captions only, containing factual descriptions that can be explicitly derived from the visual content in the image. On the other hand, \textsc{Violin} mainly consists of implicit statements that cannot be solved without in-depth understanding of the video and text, designed specifically to evaluate a model's multimodal reasoning skills.

\vspace{5pt}
\noindent\textbf{Video-and-Language Research} \,
With the emergence of large-scale video datasets~\cite{caba2015activitynet,abu2016youtube,kay2017kinetics,videvent,Wang_2016_CVPR}, several video-and-language tasks have been proposed, such as video captioning~\cite{guadarrama2013youtube2text,venugopalan2015sequence,xu2016msr,gan2017semantic,krishna2017dense,gan2017stylenet,pu2018adaptive,wang2019vatex}, localizing video segments from natural language queries~\cite{gao2017tall,anne2017localizing,chen2018temporally,lei2020tvr}, video reasoning~\cite{yi2019clevrer}, and video question answering \cite{tapaswi2016movieqa,lei2018tvqa}. Video captioning is a conditional text generation task, while the other three belong to video-and-language understanding. In particular, MovieQA~\cite{tapaswi2016movieqa}, TGIF-QA~\cite{jang2017tgif} and TVQA~\cite{lei2018tvqa,lei2019tvqaplus}, which contain real-world videos and human-generated questions, are recently proposed for video question answering.

Our \textsc{Violin} dataset also uses TV shows as one of the video sources, similar to TVQA~\cite{lei2018tvqa}. The main differences are summarized as: ($i$) Our dataset contains richer video content, including 5,885 movie clips in additional to TV shows used in TVQA.
($ii$) Our dataset requires more sophisticated reasoning skills from a model, such as inferring reasons and interpreting human emotions, while most QA pairs in TVQA are focused on identifying explicit information.

\vspace{5pt}
\noindent\textbf{Visual Question Answering} \,
Our proposed task is also related to Visual Question Answering (VQA)~\cite{antol2015vqa,goyal2017making}. The CLEVR dataset~\cite{johnson2017clevr} serves as a popular synthetic diagnosis dataset that tests a model's compositional reasoning skills. Recently, GQA~\cite{hudson2019gqa} was introduced to benchmark real-world visual reasoning, and VCR~\cite{zellers2019recognition} for visual commonsense reasoning.

Many neural network models have been proposed for these tasks, such as more advanced attention mechanisms~\cite{yang2016stacked,lu2016hierarchical,yu2019deep}, better multimodal fusion methods~\cite{fukui2016multimodal,yu2017multi,kim2016hadamard,kim2018bilinear}, the use of multi-step reasoning~\cite{hudson2018compositional,gan2019multi,cadene2019murel}, the incorporation of relations~\cite{santoro2017simple,li2019relation,norcliffe2018learning}, and neural module networks for compositional reasoning~\cite{andreas2016neural,johnson2017inferring,hu2017learning,chen2019meta}.
Our proposed task can provide a new perspective for benchmarking these models.

\section{Video-and-Language Inference Dataset}

\begin{table*}
\begin{center}
\small
\scalebox{0.9}{
\begin{tabular}{l c c c c c c c }
\hline
Dataset & Visual Domain & Source & Subtitles & Inference & Task & \# images/videos & \# samples\\
\hline
Movie-QA~\cite{tapaswi2016movieqa} & video & movie & \cmark & \xmark & QA & 6.8K & 6.5K\\
MovieFIB~\cite{maharaj2017dataset} & video & movie& \xmark & \xmark & QA & 118.5K & 349K\\
TVQA~\cite{lei2018tvqa} & video & TV show & \cmark & \xmark & QA & 21.8K & 152.5K\\
VCR~\cite{zellers2019recognition} & image & movie & \xmark & \cmark & QA & 110K & 290K\\
GQA~\cite{hudson2019gqa} & image & indoor & \xmark & \cmark & QA & 113K & 22M \\
SNLI-VE~\cite{xie2019visual} & image & natural & \xmark & \cmark & Entailment & 31.8K & 565.3K\\
NLVR$^2$~\cite{suhr2018corpus} & image& natural & \xmark & \cmark & Entailment & 127.5K & 107.3K\\
\hline
\textsc{Violin} (ours) & video & TV show/movie & \cmark & \cmark & Entailment & 15.9K & 95.3K\\
\hline
\end{tabular}}
\end{center}
\vspace{-2mm}
\caption{Comparison between \textsc{Violin} and other existing vision-and-language datasets.}
\label{tab:data-comp}
\vspace{-2mm}
\end{table*}
In our \textsc{Violin} dataset for video-and-language inference, the input is a video clip $V$ consisting of a sequence of video frames $\{v_i\}_{i=1}^T$, paired with its aligned text $S=\{s_i, t_i^{(0)},t_i^{(1)}\}^n_{i=1}$ ($s_i$ is the subtitle within time span $(t_i^{(0)}\rightarrow t_i^{(1)})$ in the video) and a natural language statement $H$ as the hypothesis aiming to describe the video clip. For every $(V,S,H)$ triplet, a system needs to perform binary classification: $f(V,S,H)\rightarrow\{0,1\}$, deciding whether the statement $H$ is entailed (label $1$) from or contradicts (label $0$) the given video clip. In order to increase the coverage and versatility, we collect the videos from diverse sources, including 4 popular TV shows of different genres and YouTube movie clips from thousands of movies. To ensure high video quality, we also provide carefully-designed protocols to guide crowdsource workers to select representative video segments for which to write positive/negative statements. The procedure of dataset collection is detailed in Sec.~\ref{sec: dataset}, and Sec.~\ref{sec: analysis} provides a thorough analysis on the dataset.

\subsection{Dataset Collection} \label{sec: dataset}
We collect videos from two sources: ($i$) 4 popular TV shows,
and ($ii$) movie clips from YouTube channels\footnote{\url{https://www.youtube.com/user/movieclips}} covering thousands of movies. Both sources contain rich human interactions and activities. Each episode of the TV shows is 20-40 minutes long, which we split into clips of 90 seconds long (while avoiding splitting dialogues in the middle). These 90 second-long clips may contain more than one scene, which are then presented to crowdworkers to select a video segment containing a single, self-contained scene for which they can write the statements. Additionally, we restrict the length of the selected interval to 15-40 seconds long, to maintain a reasonable difficulty level for the task. For movie clips from YouTube channels, the original lengths are around two minutes, which by nature usually contain only one scene of the movie. Thus, there is no need for the workers to manually select a video segment from the provided movie clips. We just select the first 40 seconds from every movie clip for annotation, to keep it consistent with TV show clips. Figure~\ref{fig:amt-ui} shows the interface for AMT workers. By dragging the slider below the video player, users can adjust the start and end timestamps of the segment they want to select (for movie clips the slider is disabled).

\begin{figure}[t!]
\begin{center}
\hspace{0cm}
   \includegraphics[width=0.85\linewidth]{img/amt-ui.pdf}
\end{center}
\vspace{-6mm}
\caption{User interface for annotators. Each annotator is provided with a video clip and required to first drag the slider below the video player to select a single-scene clip from the video, then write three pairs of positive/negative statements in the text boxes}
\label{fig:amt-ui}
\vspace{-3mm}
\end{figure}

After video segments are selected, they are presented to another group of annotators to write positive/negative statements. Each worker is assigned with one video clip, and is required to write three pairs of positive/negative statements describing the video (in the text boxes in Figure~\ref{fig:amt-ui}). We do not require AMT workers to follow any templates, as our goal is to collect diversified and natural expressions. We do have several rules/guidelines for writing positive statements: ($i$) We do not allow annotators to refer to characters in the video by name. Instead, they should use grounded referring expressions (\emph{e.g.}, ``the man with blonde hair wearing grey shirt", ``the girl sitting in the sofa holding a cup of coffee"). The purpose of this is to keep the dataset consistent across different video sources (not all video clips have character names), and to reduce potential bias (in TV shows, the number of character names is very small). ($ii$) We ask workers to keep to a minimum level of copying from subtitles (\emph{e.g.}, ``somebody says ...") or describing explicit visual information (\emph{e.g.}, object, color), and encourage them to write statements combining information from both the video clip and subtitles. ($iii$) We encourage workers to write about different aspects of the given video clip in different statement pairs, which may require different types of reasoning, such as inferring character emotions/relations/intentions and inferring causal relations in complex events.

In practice, we observe that when letting human annotators write negative statements without any constraint, the resulting statements show serious bias (\emph{i.e.}, models can learn to classify positive/negative statements without even absorbing information from the video or subtitles). When intentionally writing fake content without any reference, humans tend to use subtle patterns that statistical models can easily pick up. Therefore, when collecting negative statements, we propose two strategies to alleviate the bias issue. First, we ask annotators to use a positive statement as reference, and only modify a small portion of it to make it negative. In this case, most part of the statement remains true to the video content, and human-introduced bias is kept to minimum. This rigorous setting makes the statements more challenging to distinguish by the model, and in-depth reasoning is required to identify the fake content.
For quality control, only workers located in English-speaking countries with a lifetime task approval rate greater than 98\% can participate in our study. Also, during data collection, we manually check every worker's submissions to ensure the quality of the video segments and statements.

VCR~\cite{zellers2019recognition} proposes adversarial matching to construct wrong answers for multiple-choice QA, by selecting a correct answer (from another question) that is most similar to the current question. In our task, we use a similar strategy. For a human-generated positive statement $H_i$ for video $V_i$, we select a positive statement $H_j$ collected for another video $V_j$, which is most similar to $H_i$, and use $(H_i, H_j)$ as a pair of positive/negative statements for video $V_i$. Using this strategy, a portion of the collected statements serve as both positive and negative samples, which helps removing artificial bias. Unlike the first strategy aforementioned, statement pairs constructed this way focus more on the global understanding of the video. For example, in Figure~\ref{fig:example}, the first two negative statements are written by modifying positive statements (the modified part is marked in red), and the third negative statement is obtained by adversarial matching. In the final dataset, $2/3$ of the negative statements are constructed following the first strategy, and the remaining $1/3$ with the second strategy.

\subsection{Dataset Analysis}
\label{sec: analysis}

The \textsc{Violin} dataset contains 15,887 video clips, and each video clip is annotated with 3 pairs of positive/negative statements, resulting in 95,322 $(V,S,H)$ triplets in total. Statistics on the full dataset is provided in Table~\ref{tab:data-stat}. Each statement has 18 words on average, and the lengths of positive and negative statements are almost the same, showing no significant bias in length.

As discussed in Sec.~\ref{sec: dataset}, we use two strategies to collect negative statements: one is adversarial matching that tests a model's ability of global video understanding; the other is modifying a small part of a positive statement for the video clip, which requires in-depth reasoning skills for a model to distinguish between positive and negative statements. To investigate in more detail, for each pair of positive and negative statements, we categorize it into 6 types of reasoning skills required, as shown in Figure~\ref{fig:category}. The types of ``visual recognition", ``identifying character", and ``action recognition" are more focused on explicit information and require relatively low-level reasoning. ``Human dynamics" includes inferring human emotions/relations/intentions, etc. ``Conversation reasoning" requires performing inference over characters' dialogues and other forms of interactions (body language, hand gestures, etc.). And ``inferring reasons" is about inferring causal relations in complex events. These 3 types of statement require in-depth understanding and commonsense reasoning. Overall, ``explicit information recognition" makes up 54\% of the dataset, and ``commonsense reasoning" makes up the remaining 46\%, making our dataset a balanced one, imposing new challenges on multi-facet video-and-language understanding. Compared to other datasets, our \textsc{Violin} dataset is more focused on reasoning rather than surface-level grounding (\emph{e.g.}, in TVQA~\cite{lei2018tvqa}, only 8.5\% of the questions require reasoning).

\begin{figure}[t!]
\begin{center}
\hspace{-0.5cm}
   \includegraphics[width=\linewidth]{img/categories_stat_coarse_new.jpg}
\end{center}
\vspace{-3mm}
   \caption{Distribution of reasoning types. ``Visual recognition", ``identifying character" and ``action recognition" focus on explicit visual information; the other three require high-level inference.}
\label{fig:category}
\end{figure}

\section{Model}
\label{sec:model}

\begin{figure*}[t!]
\begin{center}
\includegraphics[width=0.9\linewidth]{img/model_flat.pdf}
\end{center}
\vspace{-3mm}
   \caption{Overview of the proposed model for the Video-and-Language Inference task. The model takes a video (a sequence of frames), its aligned subtitles and a statement hypothesis as input, and produces a scalar measuring the probability of the input statement being positive.}
\label{fig:model}
\end{figure*}

In this section, we introduce our baseline model used for benchmarking the \textsc{Violin} dataset and evaluating the effectiveness of different feature choices. An overview of the model is illustrated in Figure~\ref{fig:model}.

\subsection{Video and Text Encoders}
We first extract a sequence of visual features from video frames as $\mathbf{V}\in\mathbb{R}^{T\times d_v}$, where $T$ is the number of time steps, and $d_v$ is the dimension of each feature. Choices of visual features will later be discussed in Sec.~\ref{sec:exp-baselines}. The video encoder is implemented by a bi-directional LSTM, to capture the temporal correlation among consecutive frames. By passing video features into the video encoder and stacking hidden states from both directions, we obtain the video representations as $\mathbf{H}_V\in\mathbb{R}^{T\times2d}$, where $d$ is the hidden-state dimension of the LSTM encoder.

Statements and subtitles share the same text encoder. Statements are tokenized into a word sequence $\{w_i\}_{i=1}^{n_{stmt}}$. Each line in the subtitle is tokenized, and all the lines are concatenated together into one single word sequence $\{u_i\}_{i=1}^{n_{subtt}}$. Here, $n_{stmt}$ and $n_{subtt}$ are the lengths of statement and subtitle, respectively. We experiment with two types of text encoder: LSTM encoder and BERT~\cite{devlin2018bert} encoder. For LSTM encoder, every word token is converted to its word embedding and then fed to the LSTM encoder, producing text representations $\mathbf{H}_{stmt}\in\mathbb{R}^{n_{stmt}\times2d}$ and $\mathbf{H}_{subtt}\in\mathbb{R}^{n_{subtt}\times2d}$. For BERT encoder, we use pre-trained BERT-base model, finetuned on \textsc{Violin} training statements and subtitles. The output of BERT encoder at each position is 768-dimensional, which is then projected to $2d$ dimensions, also denoted as $\mathbf{H}_{stmt}$ and $\mathbf{H}_{subtt}$.

\subsection{Combining Multimodality Streams}
The model takes three streams of information as input: video, subtitles and statement. The goal is to determine whether the statement entails or contradicts with the video and subtitles. In our model, statement representations are jointly modeled with video and subtitles via a shared fusion module. The fusion module is implemented with bidirectional attention, adopted from \cite{seo2016bidirectional,yu2018qanet,lei2018tvqa}, where it is used for query-context matching. For simplicity, we only describe the process of combining the video and the statement streams. Subtitles and statement are fused in a similar way. Statement representations $\mathbf{H}_{stmt}\in\mathbb{R}^{n_{stmt}\times2d}$ are used as context, and video representations $\mathbf{H}_V\in\mathbb{R}^{T\times2d}$ as query. Each word in the statement thus attends to every time step in the video representations. Let $\mathbf{a}_i\in\mathbb{R}^T$ be attention weights for the $i$-th word in the statement, $\sum_{j=1}^T\mathbf{a}_{i,j}=1$ for all $i=1,\ldots,n_{stmt}$, $\mathbf{a}\in\mathbb{R}^{n_{stmt}\times T}$. The output is a video-aware statement representation: $\mathbf{M}_{stmt}^V=\mathbf{a}\mathbf{H}_V\in\mathbb{R}^{n_{stmt}\times2d}$. Similarly, we combine subtitles and statement streams to obtain a subtitle-aware statement representation $\mathbf{M}_{stmt}^{subtt}\in\mathbb{R}^{n_{stmt}\times2d}$. These two sets of representations are further fused via:
\begin{equation}\nonumber
\resizebox{.48 \textwidth}{!}{
$
\mathbf{M}_{stmt}^{all}=[\mathbf{H}_{stmt};\mathbf{M}_{stmt}^V;\mathbf{M}_{stmt}^{subtt};\mathbf{H}_{stmt}\odot \mathbf{M}_{stmt}^V;\mathbf{H}_{stmt}\odot \mathbf{M}_{stmt}^{subtt}],
$
}
\end{equation}
where $\odot$ stands for element-wise product. The resulting matrix $\mathbf{M}_{stmt}^{all}\in\mathbb{R}^{n_{stmt}\times10d}$ combines information from all three modality streams, which is then fed into another bidirectional LSTM. The last hidden states from both directions are concatenated and passed through a fully-connected layer with 1-dimensional output followed by a sigmoid activation function, predicting the probability of the input statement being positive.

The proposed baseline model is similar to the one in \cite{lei2018tvqa}. The main difference is that our model uses statement representations as context and video/subtitle representations as query in the fusion module. The intuition is that, in our video-and-language inference task, the full statement needs to be supported by evidence from either the video or subtitles, in order to judge the statement to be positive/negative, instead of just locating the position in the video/subtitles that is most relevant to the query (as in TVQA~\cite{lei2018tvqa}). Thus, in our model, every word in the statement is attended to the video and subtitles in the fusion module, then combined and fed to the final bi-LSTM to make the prediction.

\section{Experiments}
\label{sec:exp}
For evaluation, we compare our model with several baselines on the dataset and provide detailed analysis on the results.
In all the experiments, we split the \textsc{Violin} dataset into 80\% for training (76,122 $(V,S,H)$ triplets), 10\% for validation (9,600 triplets) and 10\% for testing (9,600 triplets). Model performance is evaluated via binary classification accuracy.

\subsection{Compared Models}
\label{sec:exp-baselines}
First, we define the following combinations of input sources, to evaluate the importance of different modality streams:
\vspace{3px}\\
\textbf{Statements Only}: Using statements only, without absorbing information from video or subtitles. This option is to test the innate bias of positive/negative statements.\vspace{3px}\\
\textbf{Video}: Using video features only.\vspace{3px}\\
\textbf{Subtitles}: Using subtitles only.\vspace{3px}\\
\textbf{Video+Subtitles}: Using both video and subtitle features, which is the full setting for the task.\vspace{3px}\\
\textbf{Single Frame+Subtitles}: Using subtitle features plus only one middle frame from the video. This option is to test the usefulness of temporal information in the video.

Different visual features are also evaluated on the \textsc{Violin} task: ($i$) Image feature: we use ResNet101 \cite{He2015} trained on ImageNet \cite{imagenet} to extract the global image feature for each frame;
($ii$) C3D feature: we use 3-dimensional convolutional neural network (C3D) \cite{c3d} to extract video features;
($iii$) Detection feature: we run Faster R-CNN \cite{fasterrcnn} trained on Visual Genome~\cite{krishna2017visual} to detect objects in each frame and use their regional features as the input.
For image features, we first down-sample each video to 3 frames per second, and then extract the 2048-dim feature for each frame.
Similarly, for detection features, we use the same sampling rate and extract features followed by a pooling layer outputting the 2048-dim feature for each frame.
For C3D features, we extract 4096-dim features for every 16 frames on the original video (without down-sampling).
To encode text input as features, we use ($i$) pre-trained BERT-base model~\cite{devlin2018bert} finetuned on \textsc{Violin} statements and subtitles in the training set, and ($ii$) GloVe \cite{glove} embeddings. For thorough evaluation, we also test a large-scale pre-trained model LXMERT~\cite{tan2019lxmert} that jointly learns multimodal features.

\subsection{Experimental Results}
Table \ref{tab:acc} summarizes results from baseline methods and our proposed model (using full-length video clips, subtitles and statements). We also run a set of experiments with different visual/text features and compare the results in Table \ref{tab:acc}.

\begin{table}[t!]
\begin{center}
\small
\begin{tabular}{c|c|c|c|c}
\hline
\# & Method & Visual & Text & Accuracy \\
\hline
0 & Random & - & - & 50.00 \\
\hline
1 & Stmt & - & GloVe & 53.94\\
2 & Stmt & - & BERT & \textbf{54.20}\\
\hline
3 & Stmt+Subtt & - & GloVe & 60.10 \\
4 & Stmt+Subtt & - & BERT & \textbf{66.05} \\
\hline
5 & Stmt+Vis & Img & GloVe & 55.30\\
6 & Stmt+Vis & Img & BERT & 59.26\\
7 & Stmt+Vis & C3D & GloVe & 55.91 \\
8 & Stmt+Vis & C3D & BERT & 58.34\\
9 & Stmt+Vis & Det & GloVe & 56.15 \\
10 & Stmt+Vis & Det & BERT & \textbf{59.45} \\
\hline
11 & Stmt+Subtt+SglFrm & Img & BERT & 66.60 \\
\hline
12 & Stmt+Subtt+Vis & Img & GloVe & 60.33\\
13 & Stmt+Subtt+Vis & Img & BERT & 67.60\\
14 & Stmt+Subtt+Vis & C3D & GloVe & 60.68\\
15 & Stmt+Subtt+Vis & C3D & BERT & 67.23\\
16 & Stmt+Subtt+Vis & Det & GloVe & 61.31 \\
17 & Stmt+Subtt+Vis & Det & BERT & \textbf{67.84}\\
\hline
18 & Stmt+Subtt+Vis &\multicolumn{2}{c|}{LXMERT} & 66.25 \\
\hline
\end{tabular}
\end{center}
\vspace{-2mm}
\caption{Accuracy of different methods on \textsc{Violin} test set. Subtt = Subtitle, Vis = Video, Stmt = Statement, SglFrm = single frame,
Img = Image features, Det = Detection features, C3D = C3D features, BERT = BERT features, LXMERT = LXMERT features.}
\label{tab:acc}
\end{table}

\vspace{5pt}
\noindent\textbf{Baseline Comparison}\, Row 0 is the random guess baseline with an accuracy of 50\%. When using only the statement to decide whether itself is positive or negative, the best model with BERT features only achieves 54.20, presenting little bias in the dataset. By adding subtitles or video,
all the models obtain significant gains over the ``statement only" versions. Notably, Stmt+Subtt with BERT and Stmt+Vis with Det+BERT achieve 66.05 (row 4) and 59.45 (row 10), respectively.
From row 3-4 and 12-17, we can observe that adding subtitles improves the performance significantly. However, the gain of adding video (row 5-10 and 12-17) is not as significant as adding subtitles. This might be due to visual features not capturing video information well. Using only one frame as video features (row 11) is worse than using full video (row 13), showing the importance of exploiting temporal information in the video. Overall, the best performance is achieved by using all the sources, with BERT and Detection features (row 17).

\vspace{5pt}
\noindent\textbf{Model Variants} \, We first evaluate the effectiveness of different visual features. In most settings, Detection features work better than Image and C3D features, indicating that the extracted regional information and external knowledge from Visual Genome are useful for this task.
Among all the textual features, BERT~\cite{devlin2018bert} is the strongest as expected. In all the settings, BERT-based versions generally improve the accuracy by 3\% to 6\%, compared with non-contextualized embedding such as GloVe~\cite{glove}. Joint multimodal embedding (LXMERT, row 18) achieves 66.25, which is slightly worse than the best baseline model (row 17), showing that \textsc{Violin} imposes more challenges on existing single-image-based joint pre-trained models.

\vspace{5pt}
\noindent\textbf{Human Evaluation} \, Human performance via AMT is presented in Table \ref{tab:human}. As expected, humans achieve the best performance when provided with both video and subtitles (85.20)\footnote{We repeated the human evaluation ourselves, and the accuracy is 93\%.}. Without context (video and subtitles), humans only achieve 51.38\% accuracy. Interestingly, we find that adding video brings in more gain than adding subtitles, showing the importance of visual information in \textsc{Violin} task.

\begin{table}[t]
\begin{center}
\small
\begin{tabular}{c|c}
\hline
Source & Test Accuracy (\%)  \\
\hline
Statement & 51.38 \\
Subtitle + Statement & 73.85\\
Video + Statement & 77.19\\
Video+Subtitle+Statement & {85.20} \\
\hline
\end{tabular}
\end{center}
\vspace{-2mm}
\caption{Accuracy in human evaluation on test set over
different input sources.}
\label{tab:human}
\end{table}

\begin{table}[t]
\begin{center}
\small
\begin{tabular}{c|c|c}
\hline
Method & Annotated & Adversarial matching \\
\hline
Stmt+Subtt & 61.05 & 66.05\\
Stmt+Vis & 57.08 & 59.26 \\
Stmt+Subtt+Vis & 61.99 & 67.60\\
\hline
\end{tabular}
\end{center}
\vspace{-2mm}
\caption{Accuracy (\%) on test set containing negative statements collected via different strategies. Image and BERT features are used in this experiment.}
\label{tab:neg-sacc}
\end{table}

\begin{figure*}
\begin{center}
\includegraphics[width=\linewidth]{img/qualitative_study_two.pdf}
\end{center}
\vspace{-3mm}
\caption{Qualitative analysis results. Pos/neg at the beginning of each statement indicates ground truth. {\cmark} or {\xmark} at the end of each statement represents model prediction. {\cmark} means the system judges the statement as positive, and {\xmark} means negative.}
\label{fig:qualitative}
\end{figure*}

\begin{table}[t!]
\begin{center}
\small
\begin{adjustbox}{scale=0.88,center}
\begin{tabular}{c|c|c|c|c|c}
\hline
Statement & Stmt+ &\multicolumn{2}{c|}{Stmt+Vis} & \multicolumn{2}{c}{Stmt+Subtt+Vis}\\
\cline{3-6}
Reasoning Type & Subtt & Img & Det & Img & Det \\
\hline \hline
Visual recognition & 67.19 & 67.41 & 67.41 & 67.97 & 67.97 \\
\hline
Identify character & 57.78 & 64.44 & 65.18 & 62.22 & 62.22 \\
\hline
Action recognition & 70.75 & 66.04 & 66.04 & 73.58 & 73.58 \\
\hline
Human dynamics & 63.39 & 58.04 & 58.04 & 60.71 & 61.48 \\
\hline
Conversation reasoning & 76.23 & 58.20 & 58.20 & 76.23 & 76.23 \\
\hline
Inferring reasons & 59.52 & 50.00 & 50.31 & 59.52 & 60.18 \\
\hline
\end{tabular}
\end{adjustbox}
\end{center}
\vspace{-2mm}
\caption{Accuracy (\%) on each statement type in \textsc{Violin} test set. All the methods use BERT feature. }\label{table:compared_scores}
\end{table}

\subsection{Further Analysis}

\noindent\textbf{Accuracy on Different Question Types} \, To have a better understanding of the dataset, we examine the accuracy of models on different statement types on test set in Table \ref{table:compared_scores}. Compared to Stmt+Subtt, Stmt+Subtt+Vis models improve mostly on ``visual recognition" and ``action recognition". For categories such as ``inferring reasons" and ``identify character",  including video gains some improvement. On ``conversation reasoning" and ``human dynamics", adding video features does not help.

\vspace{5pt}
\noindent\textbf{Human-Written vs. Adversarially-Sampled Negatives} \, For comparison, we create a new statement set by replacing the adversarially-sampled negative statements with original human-written negative statements. Results are presented in Table \ref{tab:neg-sacc}. Performance on the sampled negatives is higher than that on human-written ones. Our interpretation is that human-written content has higher propensity for intent understanding and in-depth reasoning, which makes the statements more challenging to the model.

\vspace{5pt}
\noindent\textbf{Qualitative Analysis} \, Figure \ref{fig:qualitative} presents some prediction examples from our model using statement, video and subtitles. The correct cases in Figure \ref{fig:qualitative} (a) demonstrate the model's ability to recognize action, infer emotion, identify referred person, and understand temporal dynamics in the video. In (b), the error cases show that our model does not work well on inferring reasons and human relations.

\section{Conclusion}
We introduce a new task, video-and-language inference (\textsc{Violin}), which requires intelligent systems to capture rich temporal signals about activities/events in video and text, in order to acquire reasoning skills for multimodal inference. We provide thorough baseline experiments for benchmarking different models on the large-scale dataset, as well as a comprehensive analysis of the dataset.
The gap between the baseline models and human performance is significant. We encourage the community to participate in this task and invent stronger methods to push the state of the art on multimodal inference. Possible future directions include developing models to localize key frames, as well as better utilizing the alignment between video and subtitles to improve reasoning ability.

\vspace{5pt}
\noindent\textbf{Acknowledgement} \,
We would like to thank Yandong Li, Liqun Chen, Shuyang Dai, Linjie Li, Chen Zhu, Jiacheng Xu and Boyi Li for providing useful feedback on the project and their help in collecting and annotating data. We thank all the reviewers for their helpful comments. The first author is supported in part by NSF under grant IIS-1546329.

\clearpage
{\small
\bibliographystyle{ieee_fullname}

redpajama_set_name: RedPajamaArXiv


Ontario's hospitals are always working toward better care, and that includes preventing complications at all stages of surgery. Our latest report, Quality Surgery: Improving Surgical Care in Ontario, looks at how an international quality improvement program is helping hospitals here make surgeries safer, and how well our hospitals compare to their international peers on surgical quality.
If you have a question about this report, email SystemPerformance@HQOntario.ca.


text:
Newsense
Web site: http://www.newsense.tk
In March of 2003, Newsense was officially born when they played their first show ever. The original concept of the band was conceived by Marc Walther and Tobias Mishel who originally started the band. Marc was set to sing and play bass, Tobias would play guitar. The first drummer used for Newsense was Ryan Brunty. This set-up began to lay the foundation for what is now Newsense. The original sound of Newsense was very pop-punk and formulaic, with songs written primarily by Marc, Tobias, and in many cases Eric Walther, who would later join the band. By late 2003, Newsense lost Ryan on drums and turned to someone who had been jamming and writing with the band for years. Eric Walther became the drummer of Newsense. At this time the band began to expand and more in its sound and added more variety into different forms of rock music. The sound was a success and they continued to improve their songs as well as their live performance. Now Newsense continues writing more songs that are complex in structure, lyrically divine, and still maintain that catchy vibe that has always been their backbone. Tobias Mishel moved to California during the summer of 2006 and now Steven Johnson plays guitar, as well as bass for Newsense. Toward the latter end of 2006, another member would be added to also play the guitar, Jello Nishimoto, who has known the band and its members for several years. But after over a year with his work with the band, Jello had to leave the band due to an issue with his responsibilities with his career as a teacher at a local high school. Coming in to play bass with the band would be Declan Desart. With the new line-up, the band is able to be more flexible by adding multiple guitar parts, more keyboards, and thrive off of the energy each of the four members have. So with a few demos and a live DVD under their belt, Newsense is ready to release a new album, "Bittersweet."
Like Newsense? Tell us!
Backs Turned
Mark Walther – Bass, Guitar, Keyboards, Vocals
Eric Walther – Bass, Drums, Guitar, Keyboards, Vocals
Steve Johnson – Bass, Guitar, Vocals
Declan Desart – Bass, Vocals
The Beatles, The Killers, The Strokes, Modest Mouse
Touring, recording, writing, playing local shows
Seventeen In Chains
Concubines
Make It Snappy
Blind to Reality
Minivan E.P.

redpajama_set_name: RedPajamaCommonCrawl


text:
Home > Entertainment > The Magpie Salute to Debaser Strand
The Magpie Salute to Debaser Strand
By Helena Olofsson on August 23, 2018
With the guitarist Rich Robinson, known from The Black Crowes, the highly topical rock band THE MAGPIE SALUTE returns to Sweden for a gig in this fall. The concert takes over where the acoustic trio show on Obaren ended this summer – but this time with a full stand and band. Prepare for an evening in the world of classical psychedelic blues rock characters and storytelling! See you at Debaser Strand on November 24th!
Tickets will be released on Friday, August 24 at 10 am via LiveNation.se
THE MAGPIE SALUTE is the rock band formed by former The Black Crowes guitarist Rich Robinson in front of a Woodstock Concert in 2016. The premise of collecting a ten-man band before the game quickly evolved into an entire tour. The band spent most of 2017 traveling and performed more than 77 concerts worldwide, including four sold-out evenings at The Gramercy in New York. In addition to Robinson, the band today consists of former Black Crowes members Marc Ford and Sven Pipien as well as the musicians Matt Slocum, Joe Magistro and John Hogg.
THE MAGPIE SALUTE has acquired a strong fan base and, besides its psychedelic blues rock, has become popular for its unique storytelling. On August 10, the debut album "High Water I" was released with "Send Me An Omen" as the first single. The album is produced by Rich Robinson and recorded in the Dark Horse Studios in Nashville, Tennessee.
"'Send Me An Omen,' to me, wraps up all of the elements of this band. There's pure rock 'n roll juxtaposed with these pop melodies sung with a melancholy that creates this beautiful balance of surreal dark and light". – Rich Robinson
THE MAGPIE SALUTE has during the summer toured as a headline act and festival band around the world. In early August they performed a successful acoustic performance at Obaren, then as a trio in Stockholm. This fall they will finally return to Sweden, and then as a full band. See you at Debaser Strand on November 24th!
Helena Olofsson, Entertainment | Södermalm
ConcertDebaserMagpieSaluteSliderStockholmStrandSwedenTicker


text:
Q: Python open exe as seperate Process I am looking for a way for my application to start its background logging application as a completely separate process.
The program is 90% user input so crashes happen, I need to see the logs but when using subprocess it spawns as a child process which means when the main application crashes so does the logger.
I have tried using os.system but that launches it as part of the current process stopping all execution until the process has completed.
The Logger accepts 3 arguments, a temp folder path, a port and a process id. Theses are required to enable the program to communicate to the main application.
Both applications are frozen and build on windows explicitly.
There is not much code I can show but here is how everything is generated.
if getattr(sys, 'frozen', False):
    debug = False
    exe_path = os.path.dirname(sys.executable)
elif __file__:
    debug = True
    exe_path = os.path.dirname(os.path.abspath(__file__))

def get_open_port():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind(('', 0))
    addr = s.getsockname()
    open_port = addr[1]
    s.close()
    return open_port
proc = [p.info for p in psutil.process_iter(attrs=['pid', 'name']) if str(os.getpid()) in str(p.info['pid'])][0]
proc = proc['name']

tmp = tempfile.mkdtemp()
tmp = str(tmp)+"\\"

port = get_open_port()
path = os.path.join(exe_path, "logger", "Logger.exe")
#os.startfile('''%s", "%s", "%s", "%s"''' %(path, tmp, proc, port)) #failes to get past here.
#subprocess.Popen([path, tmp, proc, port]) #creates as a child process which crashes with the main.
time.sleep(2) # waits for the application to start


text:
eAccess Ltd.
eAccess Announces 9-10 billion Third-Party Private Equity Placement Japan Telecom, Carlyle Group, and Goldman Sachs to lead one of Japan's largest ever private equity financing
TOKYO, August 30, 2001- eAccess Limited ('eAccess' or the 'company'), a broadband data communications company based in Tokyo, will raise an aggregate of 9-10 billion upon the completion of this third-party private equity placement in September 2001. The transaction, one of the largest ever third-party allocation of equity by a privately held company in Japan, is eAccess' new round of funding, following previous rounds in February 2000 (4.5 billion) and February 2001 (5 billion). With these three transactions, eAccess will have raised a combined total amount between 18.5-19.5 billion through third-party private equity placements. The current round of funding brings eAccess' capital to 10 billion, and gives the company total paid-in capital of 18.7-19.7 billion. (The exact amount will be confirmed upon the completion of the payment.)
This round of funding is characterized by three significant factors:
1. eAccess has secured the funding required to further develop its broadband operations
eAccess currently provides ADSL service through its own equipment installed in 473 local telephone exchanges in eight prefectures, including Tokyo, Osaka, Aichi and others. The company's current goals for its dynamic broadband services are to expand the geographic coverage of its ADSL services; construct a dedicated IP backbone using dark fiber; develop a range of value-added services, such as a content infrastructure; and promote fiber-to-the-home. The funding puts the company in a strong position to achieve these goals.
2. Investments based on strategic partnerships with telecommunications carriers
As part of this round of funding, eAccess is receiving 4 billion from Japan Telecom Co. Ltd. ("Japan Telecom"), one of the largest telecommunications carriers in Japan. Japan Telecom becomes the single largest shareholder (owning about 15%) in the privately held company. This financial relationship will form a stepping-stone for creating a strategic partnership between the two companies in the area of broadband services, especially DSL. (Please refer to today's press release, jointly announced with Japan Telecom, "eAccess and Japan Telecom to form Strategic Partnership")
3. Investments from major global investment firms including Carlyle Group's first major equity investment in Japan and continued commitment from Goldman Sachs
The Carlyle Group ("Carlyle"), one of the world's largest private equity firms, has recently established separate buyout and venture capital arms in Japan. This investment, Carlyle's first significant one in Japan, is a collaborative effort of both of these arms. Carlyle currently manages $14 billion of private equity capital in 15 funds operating in the United States, Asia, Europe and Japan. Since its founding in Washington, D.C., in 1987, Carlyle has been a significant telecommunications investor, often partnering with leading global telecommunciations companies and entrepreneurs. Goldman Sachs Group and its affiliated funds have invested more than $10 billion in 480 companies worldwide since 1986 and is one of the largest private equity investors in the world. Goldman Sachs Group and its affiliated funds have been a lead investor in eAccess since the Company's initial Series A round in February 2000, providing not only cash fund but also value added support on a continuous basis. Investments from Carlyle, Goldman Sachs, and others in the midst of the current highly competitive climate for venture capital can be taken as an indication of investors' positive evaluation of eAccess' business fundamentals.
In addition to Japan Telecom, Carlyle, and Goldman Sachs, this round's third party private equity placement includes Morgan Stanley, Premier Ventures, Sycamore Ventures as existing shareholders with additional investments; and Asahi Mutual Life Insurance, CDIB, HSBC Technology Fund, Nissho Electronics Co., Sumitomo Trading Co., and others as new equity investors (Above companies are listed in alphabetical orders).
eAccess was established in November 1999 as a broadband data communications company, focusing on DSL, Internet Protocol (IP) services and solutions to customers.
pr@eaccess.net


STACK's Brandon Hall trained in a gym on the Chicago River and played flag football as part of Nike's 'Get Out Here' campaign.
Winter in Chicago is not for the faint of heart. Temperatures often dip into the single digits, and snowfall routinely pounds the city for days on end. As if that weren't wicked enough, high winds off Lake Michigan bombard the streets and turn exposed skin raw. It hardly sounds like an ideal vacation spot, but it was the perfect location for Nike to promote its "Get Out Here" campaign.
"Get Out Here" is the slogan Nike chose to encourage athletes to train and play outdoors this winter. The campaign, which kicked off with a cool "Snow Day" ad in late October, coincides with the promotion of a line of new winter gear designed to help athletes battle the elements and perform at a high level in cold conditions.Along with other media members, STACK was invited to a #GetOutHere Challenge in Chicago to embrace the cold and put Nike's new products to the test. Here's what went down.
My day started at 7 a.m., when I got suited up in clothing from Nike's Cold Gear Collection. Stepping out of my hotel room, I felt more than ready to battle the elements, considering my outfit included a jacket, tights and a compression shirt.
After everyone met up in the lobby, it was time to jog to the gym for a morning workout. We did a quick stretch on the bustling sidewalk and were on our way, led by a pair of Nike Run Club coaches. It was a mild December day for Chicago, as temperatures were in the high 40s—not exactly the best conditions to test cold gear, especially if you grew up with Midwest winters. After a six-block jaunt through Chicago's Loop district, we arrived at our destination. But this was no run-of-the-mill gym. Nike had erected an open-air training facility on top of a barge in the Chicago River. The floating white exoskeleton emblazoned with a giant silver swoosh would serve as our training center for the morning.
The unique location made conditions a bit more chilly. Once we got inside the structure (I'm using the word inside rather loosely), we met up with two Nike Master Trainers who would lead us through our workout. The group performed a dynamic warm-up before breaking into six small teams.
The workout plan was simple. Six stations were set up around the perimeter of the gym. On the whistle, the teams would rotate among the stations, performing as many reps as possible in an allotted time. After one full circuit, we'd break for water and then perform another round, albeit with shorter sets and shorter breaks. We would perform four total rounds—one-minute sets, 45-second sets, 30-second sets and 15-second sets—to complete the workout.
Stations included Speed Ladders, Box Jumps, Rope Slams and Goblet Squats, and things got pretty challenging after just a few stations. It had been a while since I'd included agility and plyometric drills in my circuit training, and I had forgotten how those type of exercises challenge your cardio endurance.
My face was drenched in sweat by the completion of the second circuit (I've always been a heavy sweater), but the Nike gear did a great job of keeping my body dry. I never felt cold, especially once the workout started heating up, so it was hard to gauge how well the gear protected against the elements, but all the clothing was very functional.
The setting can get lost on you pretty quickly when you're in the middle of an intense training session. I was more focused on perfecting Rope Slams than admiring the scenery around me, but there was one moment when I found myself contemplating the absurdity of the situation.
Toward the end of the workout, I was doing Overhead Med Ball Slams, and the trainers were encouraging me to give it my all. Following their advice, I took a 20-pound med ball over my head, raised onto my toes and smashed it into the carpeted floor as hard as I could. I felt the ground shake beneath my feet and momentarily panicked, imagining the ball burrowing through the bottom of the barge and sinking the entire gym in the process. I'm not Superman so that didn't happen, but it certainly wasn't for lack of trying.
The Shield Flash Max is a running jacket that features a reflective print that shines in low light conditions. Since the sun sets so early, many people run at night during the winter, which forces them to don cumbersome vests or ridiculous flashing lights to stay visible. I dig the fact that this jacket looks trendy during the day but has built-in safety at night.
The Nike Therma-Sphere Max training jacket is ridiculously comfortable. I wore it most of the weekend and it's cozy enough for lounging around but very functional during training.
The SneakerBoots are exactly what they sound like—hybrid footwear that's part sneaker, part boot. I thought they looked awesome and I could appreciate the purpose they serve. If you're wearing actual sneakers in the snow, they're going to get trashed pretty quickly and won't give you much traction. If you're wearing traditional boots in the snow, they'll handle the elements but feel clunky when you get inside. SneakerBoots are in a middle ground. They're durable enough to stand up to the elements but stylish enough to look fresh when you get inside.
After grabbing lunch at Moto, an ultra-trendy restaurant in Chicago's Fulton River District that serves meticulously deconstructed southern cuisine, the group headed to a city park on the shores of Lake Michigan for some flag football.
Former NFL Defensive lineman Anthony "Spice" Adams was on hand to talk about his experiences playing in cold weather. Spice was a member of the Chicago Bears from 2007 to 2011, and he knows all about performing in frigid environments.
"Nobody cares that it's snowing out, nobody cares that it's 20 below, they just want you to perform. So you have to block out everything and go out and do what you're supposed to do," Adams said. "[We'd] always be out there with no sleeves on, especially if we were playing a team like Atlanta or Miami. You want to let them know that the elements might change, but we're not going to."
Luckily for us, we did get to wear sleeves. The unusually mild morning had given way to chillier weather as dark clouds swirled around the skyline and a stiff wind blew off the lake. Ever try to throw a football downfield and see it get blown six yards sideways? Now I know what Jay Cutler has to deal with.
Although the conditions weren't conducive for pinpoint passing, they were perfect for trying out Nike's cold gear. The Therma-Sphere Max training jacket was effective at stopping the wind, but light enough to keep me comfortable running routes. The Pro Hyperwarm shirt kept me warm without feeling uncomfortably tight.
As for the game itself, I captained a team called "We Are The Champions" (super creative name, I know) to a close victory. Everyone seemed to be more focused on the game than on the atmospheric conditions, so in that sense we heeded Adams's advice.
I definitely enjoyed training outside, but I was more than ready to crawl into my hotel bed by the end of the day. Getting out there took a lot out of me.


Conservation Biology is a new, multidisciplinary science born of crisis: the unprecedented threat to biological diversity in modern times. This course has two basic goals.
First, it examines the magnitude and nature of the problem - human impacts, ranging from habitat destruction to introductions of exotic species - on genetic diversity, species, communities, and ecosystems.
Second, the course explores solutions, including both theoretical and practical approaches, to the protection and maintenance of biodiversity. These efforts range from maintaining genetic diversity to preventing species extinctions to restoring entire functioning ecosystems.
Identify and describe the key environmental issues facing humankind and outline the role that humans play in environmental degradation and species extinction.
Evaluate and compare the conservation techniques that can be used to restore habitats and ecosystem function.
Appraise the special problems involved in attempting to conserve endangered species.
Integrate complex political, economic and social issues to formulate effective conservation outcomes.
Prescribed Practical Conservation Biology, Lindenmayer D, Burgman M, CSIRO Publishing, 2005.
Authorised by: Chief Marketing Officer, Strategic Marketing and Communications. Maintained by: Science Webmaster Team. Last updated: May 2018.