text:
The state of CSI in SA
Tags: csi, mining, anglo american, nick rockey, trialogue, cathy duff
Corporate social investment (CSI) expenditure in South Africa for the 2014/15 financial year is estimated to be R8.1 billion. This estimate, according to Trialogue's 2015 CSI research, represents a year-on-year decline in expenditure of 6% in real terms, following a real decline of 2% in 2013/14.
The primary reason for the decline given by companies participating in the research is a reduction in corporate profits. Companies typically commit a percentage of profits to CSI initiatives, but it appears that the growth the CSI sector experienced between 2001 and 2013 has entered a period of decline.
The Trialogue CSI Handbook, now in its 18th edition, contains a breakdown of where this CSI expenditure goes, as well as the trends in social investment.
From its first issue in 1998, Trialogue's CSI Handbook has mapped South Africa's corporate social investment landscape and helped establish the standards for best practice in CSI, becoming the go-to reference guide on every CSI practitioner's desk. The 18th edition of Trialogue's CSI Handbook extends this strong tradition of research to understand what corporates are funding and how they are doing it, as well as how non-profits are using the funds.
This year the two questionnaires (one each for corporates and non-profits) were updated, some sections were streamlined and questions on community trusts, the revised Broad-based Economic Empowerment (BBBEE) Codes and corporate/grantee relationships were added.
How does it work? Professional researchers conducted one-on-one interviews with representatives from large South African companies. For the first time, companies were allowed to complete the questionnaire themselves and all self-completed responses were verified by the researchers. Of the 81 companies included in this year's sample (in 2014 there were 99), 63 (77%) also participated in the previous year's study. In addition, 123 NPOs responded to Trialogue's online survey (there were 171 in 2014) and 28% of these were repeat participants from 2014.
According to the research, corporate respondents continue to represent the diversity of the South African business sector. Financial services was again the best-represented sector in the sample, with 21% of the total response (2014: 22%), followed by retail and wholesale (14%, up from 13% in 2014). As in 2014, more than half (54%) of the sample was characterised by a combination of business-to-consumer (B2C) and business-to-business (B2B) elements. Thirty-one percent followed a B2B model, while 15% followed a B2C model. The research sample included companies of various sizes, as measured by workforce and total annual income. In 2015, more than half (54%) of the respondent companies employed less than 5 000 people, whereas 12% had staff complements greater than 20 000. More than three-quarters (78%) of companies in the sample had annual income of over R1 billion in their latest full financial year.
"NPOs were also measured by staff complement and annual income and asked to provide two years of data for both metrics. There were no significant shifts in staff numbers since 2014. More than 50 people were employed by 34% (2014: 34%), 26% employed 11 to 30 and 27% employed one to 10 people. Similarly, annual income has held relatively constant with the greatest proportion (27%) raising between R5 and R20 million in both years.
"In 2015, we expanded the reputation component of our research to include corporates' and NPOs' impressions of the good work that the NPO sector is doing. Similar to the corporate rankings, we asked respondents to list three organisations they feel are having the greatest impact and we ranked top companies/NPOs by the number of mentions they received. This section describes the findings," according to the handbook.
Mining companies tops
Non-profit organisations and corporates alike agreed that Anglo American continues to have the greatest developmental impact. Anglo American's global spend amounted to R1 474 billion and Anglo America Platinum's annual published spend was R236 million. The annual public spend of African Rainbow Minerals was R282.5 million, that of Kumba Iron Ore was R202.3 million and Royal Bafokeng spent R133. Looking at CSI spend across sectors, the mining, retail and financial services sectors together accounted for just over 70% of the total R8.1 billion of CSI expenditure, with mining alone accounting for over 40% of total CSI expenditure. The distribution by sector remains broadly consistent with 2014.
There was a great deal of consensus, as most companies were named on both lists. The exceptions were Pick n Pay, which appeared only on the NPO list, and MTN, Transnet, Telkom and MultiChoice, which did not feature on the NPO list. Nearly all of the names ranking highly for reputation are regular features of this annual exercise; however, 2015 marks MultiChoice's first time on the corporate list. With a smaller sample of corporates participating in this year's research, the number of mentions received by top companies declined accordingly. Despite a similar decrease in NPO participation, the number of mentions for top companies on the NPO list remained similar to previous years, suggesting a greater degree of consensus in 2015.
Total CSI expenditure declining
Over the past year the total CSI expenditure in South Africa was estimated to amount to R8.1 billion. This estimate is based on Trialogue's analysis of the CSI expenditure of large South African companies and state-owned enterprises: a broader sample than its primary research
component. This means that for the first time since Trialogue started tracking CSI expenditure in 1998, there has been a decline in both real and nominal terms. Until 2013, Trialogue consistently found that the total annual estimated CSI expenditure was growing in real terms and 2014 was the first time that CSI expenditure experienced a negative growth of 2% in real terms, but a slight increase in nominal terms. In 2015, the CSI estimate revealed a 1% decline in nominal growth and a real decline of 6% from 2014. This suggests that the growth that the CSI sector experienced between 2001 and 2013 is beginning to decline.
"Despite a small decline in total CSI expenditure, social investment continued to grow within Trialogue's primary research sample of 81 companies. This increased spend from companies within the primary research sample seems to contradict the total expenditure findings above. However, it should be noted that the primary research group is a smaller, select sample, while the estimated overall CSI spend is extrapolated from a broader range of companies, using publicly available data. The disparity is explained by notably lower expenditure by a few companies with large CSI budgets that did not participate in our research.
"CSI expenditure remains concentrated among larger companies. The top 100 companies (by CSI spend) invested R5.4 billion, or 67% of the total CSI expenditure. The distribution of expenditure by the top 100 companies is shown in the chart below. Just 15 companies accounted for close to half (48%) of the total amount spent by top 100 companies, while 30 companies accounted for more than two-thirds (68%)," the handbook reveals.
Nick Rockey, Trialogue's MD, says within the corporate primary research sample, giving was on the rise. The majority (59%) of companies reported increased expenditure in 2015, slightly down from the 64% that reported increases in 2014. The most common reason given for increased expenditure was an increase in corporate profits (41%, up from 37% in 2014), followed by growing requirements of the projects funded (26%, up from 25% in 2014). Least common was a more inclusive definition of CSI or policy change, with 7% each. Similarly, decreasing profits were the most often-cited reason for declining budgets (55%), with policy changes (6%) the least common cause.
"The average total CSI spend rose from R51.0 million in 2014 to R57.1 million in 2015. The median---a figure less influenced by outlier companies (for example, the one company claiming to spend over R1 billion on CSI)---rose from R14.4 million to R16.6 million.
"In South Africa, the BBBEE Scorecard requires large companies to spend 1% of net profit after tax (NPAT) on socio-economic development (SED). In 2015, our sample again reported giving more than this target, with expenditure averaging 1.7% of the previous year's NPAT, up from 1.2% in 2014," says Rockey.
Non-cash giving stabilising
Research shows that in 2015, the trend seen in recent years of increased giving of non-cash donations reversed. Approximately one-third (34%) reported non-cash giving, down from 40% in 2014. In line with this decline, the overall proportion of non-cash giving decreased marginally to 10% (2014: 12%). Product and service donations accounted for the vast majority of non-cash giving. Twenty-nine companies reported figures for these donations, equating to 5% of total giving and 95% of non-cash giving. By comparison, 14 companies quantified the value of their employees' volunteering time (less than 1% of total giving and 5% of non-cash donations).
Furthermore, the breakdown of corporates' CSI expenditure by cost type remained largely unchanged from the previous year. Nearly all (97%) respondents reported some of their expenditure going directly to the projects they supported (2014: 100%). This direct project expenditure accounted for an average of 87% of companies' total spend, down slightly from 89% in 2014. The remaining 13% of companies' CSI expenditure went towards management and support costs. Aside from direct project expenditure, no cost type accounted for more than 3% of total expenditure or was included in the budgets of more than half the respondents.
As for geographic distribution of CSI funding and development focus areas:
Corporates are most likely to be involved in projects in Gauteng (70%), but gave the greatest proportion of CSI funding to national causes (34%);
Education continues to be supported by most corporates (92%) and received the greatest share of corporate funding (47%). Social and community development overtook education as the most common development sector among NPOs (73%); and
Companies in the sample gave to an average of 4.6 developmental sectors in 2015.
Rockey says national projects continue to receive the largest portion of CSI expenditure Companies again gave to an average of four provinces in addition to their national and/or international projects.
"While projects in Gauteng were the most commonly supported among corporate respondents (70%), the province's relative share of corporate funding declined slightly from 27% in 2014 to 24% in 2015. By comparison, projects with a national focus were supported by fewer respondents (62%) but garnered a greater portion of budgets (34%, up from 29% in 2014). Corporate support for projects in the Western Cape, Free State, North West Province and Limpopo fell during the year. Nine respondents (11% of the sample) reported international giving in 2015, with the lion's share (91%) of their giving going to projects in Africa.
"The NPO funding footprint of our sample is concentrated in Gauteng and the Western Cape. Whereas the proportion of NPOs operating in Gauteng declined marginally from 50% in 2014 to 49% in 2015, 30% of NPO resources went to the province (2014: 26%), with only 15% allocated to national projects (2014: 13%) (see figure 15). Comparing NPO funding with corporate funding, a significantly larger portion of NPO funding was directed to projects in the Western Cape (21% as compared to 11% of corporate funding), but this is most likely due to our NPO sample being drawn predominantly from Gauteng and the Western Cape. NPOs in the 2015 sample reported supporting an average of two provinces, down from three in 2014," he says.
Looking at the sectors which were supported best, education continues to receive the greatest share of corporate support. Each year Trialogue asks corporate participants which development sectors they support and what proportion of their CSI funding is allocated to each sector. Education was again the most popular sector, with 92% of respondents supporting this sector, followed by social and community development (74%) and health (58%). For the first time, Trialogue added disaster relief as a sector to understand how companies have responded to high-profile events. While a significant 17% of companies reported giving to these types of causes, they accounted for less than 1% of total giving.
Causes supported include floods in Cape Town, Mozambique and Malawi, as well as xenophobic attacks, the Ebola crisis and fires in Kya Sands and Cape Town. Education again received the greatest share of total expenditure, however by a slightly smaller margin (47%, down from 49% in 2014). Social and community development and health continued to hold second and third positions, receiving 17% and 12% of CSI spend, respectively. Despite the addition of disaster relief to the research, there was little shift in the distribution of funding across the development sectors; no sector gained or lost more than 5% of funding from 2014. Companies reported giving to an average of 4.6 different development sectors, up slightly from 4.5 in 2014.
Cathy Duff, Trialogue Director, says NPO respondents follow a similar, but not identical, development footprint to their corporate counterparts. "Social and community development overtook education as the most common area of focus, with 73% and 71% of NPO respondents, respectively, supporting these sectors. (2014: 57% and 62%). In 2015, NPO respondents were involved in an average of 4.5 development sectors, more than double the 2014 sample's average of two sectors."
Corporate relationships with NPOs:
The vast majority of corporate respondents (90% of the sample) channelled a significant proportion of CSI funding (52%) to NPOs.
While NPOs are most likely to have corporate donors (66% of the sample), government surpassed companies as the largest source of funding (20%) to NPOs.
NPOs are accessing funding allocated to BBBEE Scorecard requirements, most commonly socio-economic development, cited by 82% of respondents.
Companies are giving multiple grants (70% gave more than 20 grants during the year) to many organisations (58% gave to more than 20 organisations).
Government's involvement
Within the sample, the NPO funding landscape has changed notably since 2014. Corporates, the most common source of NPO funding in 2014, have been surpassed by the South African Government as the largest source of NPO income. The former dropped from 22% in 2014 to 18% in 2015, whereas the latter nearly doubled from 11% to 20% during the same period. This was likely fuelled by a sharp increase in the proportion of respondents that received funding from government (63%, up from 45% in 2014). It is important to note that these changes may be due to the different sample of NPOs responding to the survey in 2015. NPO income from private individuals and self-generated funding declined by more than 5%, whereas income from trusts and foundations increased from 9% in 2014 to 13% in 2015. Investment income, while only accounting for 2% of total NPO income, was nevertheless relatively common among NPO respondents, with 38% citing it as a source of income. Other sources of income included churches and proceeds from the sale of property.
As for the newly introduced section on BBBee Scorecard funding, NPOs are capitalising on BBBEE Scorecard requirements in securing their funding from companies. Socio-economic development (SED) was the most common element of the Scorecard for accessing funding, with 82% of NPO respondents; however, nearly a third (31%) of respondents were drawing on skills development funds and a quarter (25%) drew on enterprise and supplier development funding.
Duff says there is a weak link between CSI and skills development, enterprise and supplier development, with little change expected. The Department of Trade and Industry's (dti) BBBEE Codes of Good Practice are designed to encourage transformation of the country's business sector through a number of practices, including investment in socio-economic development (SED). In their most recent verifications, 48% of corporate respondents received the full five points for SED, down from 63% in the previous year. Giving among these respondents averaged 1.1% of NPAT.
"We asked corporate respondents how their CSI activities relate to skills development and enterprise and supplier development, and how changes in the codes will affect their practices. While the greatest proportion (47%) of companies reported no linkage between their CSI and skills development programmes, nearly the same amount of respondents (42%) used their CSI programmes as a feed-in to their skills programmes. This afforded them a more targeted development of a pipeline of talent from which to ultimately draw employees. Most companies (60%) foresaw no change to the relationship between skills development and CSI as a result of the revised BBBEE Codes. Greater integration between the functions was the most frequently anticipated change (19% of respondents), while a mere 7% expected expenditure on skills development through CSI to increase (see figure 33).
"The relationship between CSI and enterprise development closely mirrors that of CSI with skills development. Most respondents (60%) reported no linkage between the programmes; however, the most common linkage was for CSI to feed into companies' ED programmes (23% of respondents). Two-thirds of companies (66%) did not expect the revised BBBEE Codes to impact this relationship. Among those who did anticipate change, it was most often in the form of greater integration (26% of the sample) and increased ED expenditure (15%)," Duff concludes.
Lindsay King


Box Of Birthday Cards box of birthday cards stampin up birthday central box template. box of birthday cards big box fundraising home welcome ideas. box of birthday cards handmade card in a box unique birthday greeting cardbox card 3d printable….


All device operational procedures must be learned on the ground.
Received traffic information is to be used as an aid to situational awareness and is merely supplemental and advisory in nature.
uAvionix is not liable for damages arising from the use or misuse of this product.
uAvionix products are warranted to be free from defects in material and workmanship for one year from purchase. For the duration of the warranty period, uAvionix, at its sole option, will repair or replace any product which fails in normal use. Such repairs or replacement will be made at no charge to the customer for parts or labor, provided that the customer shall be responsible for any transportation cost.
This warranty does not apply to cosmetic damage, consumable parts, damage caused by accident, abuse, misuse, water, fire or flood, damage caused by unauthorized servicing, or product that has been modified or altered.
IN NO EVENT SHALL UAVIONIX BE LIABLE FOR ANY INCIDENTAL, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES, WHETHER RESULTING FROM THE USE, MISUSE OR INABILITY TO USE THE PRODUCT OR FROM DEFECTS IN THE PRODUCT. SOME STATES DO NOT ALLOW THE EXCLUSION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THE ABOVE LIMITATIONS MAY NOT APPLY TO YOU.
Warranty repair service shall be provided directly by uAvionix.
For inquiries related to the Scout Warranty, please contact uAvionix.


- Evaluating state fiscal strength now, amid an expansion, is relevant because larger federal deficits and a still-low federal funds rate imply there could be less countercyclical support in the next downturn.
Sep. 17 2018 — It has now been 10 years since the start of what former Federal Reserve Chairman Ben Bernanke called the "worst financial crisis in global history." What ensued in its wake was also the most severe economic downturn since the Great Depression. In retrospect, U.S. states weathered the historic turmoil remarkably well from a credit perspective. No state saw its credit rating fall to below the 'A' category. Curiously, however, the distribution of state credit ratings now—more than nine years into an economic recovery—is somewhat lower and extends further down the rating scale than before.
In a recent study, S&P Global Ratings evaluated the states' level of fiscal preparedness for another downturn and found that, overall, the picture is mixed. On the one hand, most states are in an adequate position to manage through a hypothetical recession that causes moderate budget stress. On the other, doing so would require making fiscal adjustments that go beyond drawing from accumulated budget reserves alone, because in most states, budget reserves remain insufficient to absorb even the first-year fiscal effects of a downturn.
While credit quality across the state sector remains strong, it has become more uneven, reflecting that states face a range of long-term structural headwinds. Reaching lower on the ratings' scale, the current spectrum of state ratings also implicitly acknowledges that in contrast to prior cycles, the capacity to assemble a counteractive response, both on the monetary and (federal) fiscal policy fronts—even after the extended expansion—has diminished. Widening federal budget deficits and a still-low federal funds rate imply that the economic and fiscal brunt of the next recession is likely to fall more squarely on the states. And given that states are structurally predisposed to the countervailing effects of pro-cyclical revenue trends and countercyclical expenditure pressures, they could be challenged as never before by the next recession.


text:
City Under Siege
Director: Benny Chan
Stars: Aaron Kwok, Shu Qi, Jacky Wu Jing, Zhang Jing-Chu, Yuen Wah, Collin Chou, Tie Nan, Karen Cheung Bo-Man, Gao Yan-Chao, So Ming-Hei, Ben Wong Chi-Yin, Gary Chiu Cheun-Sing, Terence Yin Chi-Wai, Chrissie Chau
Genre: Comedy, Action, Martial Arts, Science Fiction
Rating: 6 (from 1 vote)
Review: Sonny (Aaron Kwok) is a dumb but lovable circus clown whose dream of becoming a famous knife-thrower just like his dear-old dad are forever frustrated by nasty knife expert, Chu (Collin Chou). On tour in Malaysia, Sonny tags along when Chu and his gang discover a cave occupied by the Japanese army during the Second World War that holds a cache of gold. Unfortunately, it also contains an experimental bio-chemical toxin that transforms them all into super-powered mutants. A body-bloated Sunny washes ashore in Hong Kong where he is rescued by beautiful and ambitious TV news reporter, Angel (Shu Qi). She spies a scoop when Sunny reverts back to his handsome self and uses his newfound superpowers to foil a bank robbery. Acupuncture wielding kung fu cop Sun Hao (Jacky Wu Jing) and his partner, both professional and personal, Tai (Zhang Jing-Chu) enlist Sunny as a super-powered crime-fighter against mutant criminals led by his old adversary, Chu.
Until relatively recently there weren't that many superhero movies made in Hong Kong. In fact the colourfully chaotic Shaw Brothers classic Super Infra-Man (1975) remained the sole effort in this genre for a great many years followed only by the ignominious Invincible Space Streaker (1988). However, Bryan Singer's first X-Men movie was surprisingly popular in Hong Kong. Coinciding with the boom in CGI effects spectacles such as Storm Riders (1998) its influence has been evident in recent Cantonese superhero outings like City of Masks (2002), Future X-Cops (2010) and most prominently this wacky comic book styled adventure from veteran action hand, Benny Chan. City Under Siege even mimics the World War Two-set prologue from X-Men, albeit substituting a Japanese military lab for a concentration camp. Also certain scenes seem to have been closely modelled on specific sequences from Hollywood superhero epics including Spider-Man (2002) and The Dark Knight (2008) though also, inevitably The Matrix (1999). In fact the film was promoted in France, under the alternate title of Blast, as a Chinese variation on The Matrix although the tone established by Aaron Kwok's remarkably manic lead performance is a little closer to The Mask (1994).
A typically schizophrenic Hong Kong production, City Under Siege flip-flops from frantic farce to tear-jerking tragedy with a carefree abandon liable to infuriate those unaccustomed to Cantonese genre filmmaking. The plot is undeniably inconsistent and borderline incoherent and yet manages to remain wildly entertaining not only on the strength of its accomplished special effects, neon-vibrant comic book colours and rousing action sequences choreographed by Nicky Lee and Ma Yuk Sing (who worked on Storm Riders) but the oddly engaging mugging of its gleefully uninhibited cast. The film goes out of its way to place handsome Cantopop idol and award-winning actor Aaron Kwok in the zaniest situations imaginable, whether goofing around in crazy clown makeup or a surprisingly convincing latex fat suit.
Much like Terence Hill in Supersnooper a.k.a. Super Fuzz (1981) there is an inconsistency to Sunny's superpowers as they tend to switch from super-strength or speed to some form of psychokinesis depending on the situation. Nevertheless, Kwok's accomplished acting leaves Sunny a lovable doofus even though his particular plotline proves less compelling than the one assigned to co-stars Jacky Wu Jing and Zhang Jing-Chu. Their likeable crime-fighting couple have a satisfying story arc that proves rather affecting. Elsewhere, the ever-lovely Shu Qi is charismatic and likeable as a more proactive love interest than normally showcased in superhero films, not above getting stuck into a fight. Rather than go the Lois Lane route, the plot has Angel cynically manipulate Sunny into a media phenomenon to further her career. This, coupled with Ho's equally cynical use of him as bait to lure out the hideously deformed Chu and his gang, plays to an underlining theme about the pursuit of fortune and glory making monsters of men and women. Sunny remains an incorruptible innocent and Angel grows to regret her cavalier abuse of his trust and affection. This prefigures a third act shift into total seriousness, introspection and of course kick ass martial arts action. Naturally, Aaron also sings several sugary Cantopop songs including the closing theme.
Reviewer: Andrew Pragasam

redpajama_set_name: RedPajamaCommonCrawl


Over the next year, her hit singles "Saving All My Love for You" and "How Will I Know" helped the album reach the top of the charts, where it stayed for 14 non-consecutive weeks.
After almost a decade of struggling with her personal life, Houston seemed to be pulling herself together.She soon became an extremely sought-after teenage model, one of the first African-American women to appear on the cover of magazine. When she was 19, Whitney Houston was discovered in a nightclub by the renowned Clive Davis of Arista Records, who signed her immediately and took the helm of her career as she navigated from gospel to pop stardom.In 1983, Houston made her debut on national television, appearing on The Merv Griffin Show to sing "Home" from the musical and almost immediately became a smash pop sensation.Houston followed the monumental success of her first album with a second release, , in 1987.That record, too, went platinum many times over and won a Grammy for the single "I Wanna Dance With Somebody (Who Loves Me)," with a successful world tour following.According to a report released by the Los Angeles County coroner's office on March 22, 2012, the official cause of her death was accidental drowning.
The effects of heart disease and cocaine found in her system were contributing factors as well.
In spite of these growing personal troubles, Houston continued to progress in her career, crossing over successfully into acting in 1992 by starring opposite Kevin Costner in the wildly popular , a cover of Dolly Parton's 1974 "I Will Always Love You," proved to be Houston's biggest hit ever, spending a record-breaking 14 weeks atop the U. Later in the 1990s, Houston also starred in , her first non-soundtrack studio album in many years, and it earned her another Grammy for the single "It's Not Right But It's Okay." But the album could not top the chart performance of her previous full-length releases.
However, her collaboration with Mariah Carey in the animated film produced a hit single, "When You Believe," which won an Academy Award.
During this time, the singer also appeared at a concert for Nelson Mandela's birthday and founded the Whitney Houston Foundation for Children, a nonprofit organization that funds projects to help needy children over the world.
By 1992, Whitney Houston was on top of the world, but her life was about to get very complicated very quickly.
The show aired during the worst years of the couple's crumbling marriage; drug use, lifestyle excess and bad behavior were all caught on tape and Houston's reputation sunk to new lows.


text:
SET QUOTED_IDENTIFIER ON
GO
SET ANSI_NULLS ON
GO
---BUILD+
CREATE PROCEDURE [tSQLt].[Private_CreateResultTableForCompareTables]
 @ResultTable NVARCHAR(MAX),
 @ResultColumn NVARCHAR(MAX),
 @BaseTable NVARCHAR(MAX)
AS
BEGIN
  DECLARE @Cmd NVARCHAR(MAX);
  SET @Cmd = '
     SELECT ''='' AS ' + @ResultColumn + ', Expected.* INTO ' + @ResultTable + '
       FROM tSQLt.Private_NullCellTable N
       LEFT JOIN ' + @BaseTable + ' AS Expected ON N.I <> N.I
     TRUNCATE TABLE ' + @ResultTable + ';' --Need to insert an actual row to prevent IDENTITY property from transfering (IDENTITY_COL can't be NULLable);
  EXEC(@Cmd);
END
---BUILD-
GO

redpajama_set_name: RedPajamaGithub


text:
Posted on May 31, 2019 by Record Herald
US holds CBD hearing as fans, sellers await legal clarity
By Candice Choi - AP Food & Health Writer
SILVER SPRING, Md. (AP) — CBD products have surged in popularity despite confusion around their legal status . Now U.S. regulators are exploring ways the hemp ingredient might officially be allowed in food, drinks and dietary supplements.
The U.S. Food and Drug Administration held a hearing Friday to collect information about cannabis compounds such as CBD, which is already available in candy, syrups, oils, drinks, skin patches and dog food.
No decisions are expected immediately, but the hearing is seen as an important step toward clarifying regulations around the ingredient.
"There is mass confusion in the marketplace," said Peter Matz of the Food Marketing Institute, one of dozens of speakers who addressed the FDA panel.
Other speakers including academic researchers, businesses and consumer advocates urged the FDA to move quickly, noting that the industry is growing rapidly with little oversight. That is raising concerns about the accuracy of product labels and people not realizing how much they may be consuming through various products.
Before the hearing began, acting FDA Commissioner Ned Sharpless noted critical questions remain about the safety of CBD, such as how much is safe to use daily and the compound's effects on pregnant women and children.
CBD often comes from a cannabis plant known as hemp, which is defined by the U.S. government as having less than 0.3% THC, the compound that causes marijuana's mind-altering effect. CBD doesn't cause that high, but fans of the products claim benefits including relief for pain and anxiety.
For now, the FDA has said CBD is not allowed in food, drinks and supplements. But given the agency's limited resources, many do not expect the agency to enforce the position unless products make explicit health claims that could endanger people.
Adding to the confusion, some states like Colorado allow it in food and drinks. In New York City, where officials have warned it's not allowed in food and drinks, restaurants and stores have continued selling it.
By Candice Choi
AP Food & Health Writer
Hi! A visitor to our site felt the following article might be of interest to you: US holds CBD hearing as fans, sellers await legal clarity. Here is a link to that story: http://www.recordherald.com/features/health/40228/us-holds-cbd-hearing-as-fans-sellers-await-legal-clarity


text:
\section{Introduction}
\vspace{-1pt}
Video object segmentation (VOS)
has two common settings, zero-shot and one-shot.
Zero-shot VOS (Z-VOS)\footnote{Some conventions~\!\cite{perazzi2016benchmark,DBLP:conf/cvpr/WangSP15} also use `unsupervised VOS' and `semi-supervised VOS' to name the Z-VOS and O-VOS settings~\!\cite{Caelles_2017_CVPR}. In this work, for notational clarity, the terms `supervised', `weakly supervised' and `unsupervised' are only used to address the different learning paradigms. \label{notation}} is to automatically segment out the primary foreground objects, without any test-time human supervision; whereas one-shot VOS (O-VOS) focuses on extracting the human determined foreground objects, typically assuming the first-frame annotations are given ahead inference\textsuperscript{~\!\ref{notation}}. Current leading methods for both Z-VOS and O-VOS are \textit{supervised} deep learning models that require
extensive amounts of elaborately annotated data to improve the performance and avoid over-fitting. However, obtaining pixel-wise segmentation labels is labor-intensive and expensive (Fig.~\!\ref{fig:top-right}~\!(a)).

It is thus attractive to design VOS models that can learn from unlabeled videos.
With this aim in mind, we develop a unified, \textit{unsupervised/weakly supervised} VOS method that mines \textit{multi-granularity} cues to facilitate video object pattern learning (Fig.~\!\ref{fig:top-right}~\!(b)). This allows us to take advantage of nearly infinite amounts of video data.
Below we give a more
formal description of our problem setup and main idea.

\begin{figure}[t]

	\centering
	\includegraphics[width=0.99\linewidth]{figs/top-right}
	\vspace{-9pt}
	\captionsetup{font=small}
	\caption{\small{$_{\!}$(a) Current$_{\!}$ leading$_{\!}$ VOS methods$_{\!}$ are$_{\!}$  learned in a supervised manner,  requiring large-scale elaborately labeled data. (b) Our model, MuG, provides an unsupervised/weakly-supervised framework that learns video object patterns from unlabeled videos. (c) Once trained, MuG can be applied to diverse VOS settings, with strong modeling ability and high generability.
		}
	}
	\label{fig:top-right}
	\vspace{-13pt}
\end{figure}

\noindent\textbf{Problem Setup and Main Idea.}
Let \bm{$\mathcal{X}$} and \bm{$\mathcal{Y}$} denote the input video space and output VOS space, respectively. Deep learning based VOS solutions seek to learn a differentiable, \textit{ideal} video-to-segment mapping $g^*\!\!\!:\!$ \bm{$\mathcal{X}$}$\mapsto$\bm{$\mathcal{Y}$}.
To approximate $g^{*\!}$, recent leading VOS models typically work in a \textit{supervised learning} manner, requiring $N$ input
samples and their desired outputs $y_n\!\!:=\!g^{*\!}(x_n)$, where
$\{(x_n,
y_n)\}_{n\!}\!\subset$\bm{$\mathcal{X}$}$\times$\bm{$\mathcal{Y}$}. In contrast, we address the problem in settings with much less supervision: (1) the \textit{unsupervised} case, when we only have samples drawn from  \bm{$\mathcal{X}$}, $\{x_n\}_{n\!}\!\subset$\bm{$\mathcal{X}$}, and want to approximate $g^*$, and (2) the
\textit{weakly supervised learning} setting, in which  we have annotations for
\bm{$\mathcal{K}$}, which  is a related output domain for which obtaining
annotations is easier than \bm{$\mathcal{Y}$}, and we approximate $g^{*}$ using samples from
\bm{$\mathcal{X}$}$\times$\bm{$\mathcal{K}$}.

The standard way of evaluating learning outcomes follows an \textit{empirical} risk/loss minimization formulation~\!\cite{Shen_2018_CVPR
:
\vspace{-5pt}
\begin{equation}\small
\tilde{g}\in\mathop{\arg\min}_{g\in\bm{\mathcal{G}}} \frac{1}{N}\sum\nolimits_n\varepsilon(g(x_n),z(x_n)),
\vspace{-0pt}
\end{equation}
where \bm{$\mathcal{G}$} denotes the hypothesis (solution) space, and $\varepsilon\!\!:$\bm{$\mathcal{X}$}$_{\!}$$\times$\bm{$\mathcal{Y}$}$\mapsto\!\!\mathbb{R}$ is an error function that evaluates the estimate $g(x_n)$ against VOS-related prior knowledge $z(x_n)\!\!\in$\bm{$\mathcal{Z}$}. To make $\tilde{g}$  a good approximation of $g^{*\!}$, current supervised VOS methods directly use the desired output $y_n$, \ie, $z(x_n)\!\!:=\!\!g^{*\!}(x_n)$, as the prior knowledge, with the price of vast amounts of well-annotated data.

In our method, the prior knowledge \bm{$\mathcal{Z}$}, in the unsupervised learning setting, is built upon several heuristics and intrinsic properties of VOS itself, while in the weakly supervised learning setting, it additionally considers a related, easily-annotated output domain \bm{$\mathcal{K}$}. For example, part of the fore-background knowledge could be from a saliency model~\!\cite{DBLP:conf/cvpr/YangZLRY13} (Fig.~\!\ref{fig:top-right}~\!(b)), or in a form of CAM maps~\!\cite{zeng2019multi,zhou2016learning} from a pre-trained image classifier~\!\cite{Huang_2017_CVPR} (\ie, a related image classification domain \bm{$\mathcal{K}$})\footnote{\scriptsize{Note that any unsupervised or weakly supervised object segmentation/saliency model can be used; saliency~\!\cite{DBLP:conf/cvpr/YangZLRY13}, and CAM~\!\cite{zeng2019multi,zhou2016learning} are just chosen due to their popularity and relatively high performance.}}. Exploring VOS in an unsupervised or weakly supervised
setting is appealing not only because it  alleviates the annotation burden of
\bm{$\mathcal{Y}$}, but also because it inspires  an in-depth
understanding of the nature of VOS by exploring
\bm{$\mathcal{Z}$}. Specifically, we analyze several different types of cues at multiple granularities, which are crucial for video object pattern modeling:
\vspace*{-0pt}
\begin{itemize}[leftmargin=*]
	\setlength{\itemsep}{0pt}
	\setlength{\parsep}{-2pt}
	\setlength{\parskip}{-0pt}
	\setlength{\leftmargin}{-13pt}
	\vspace{-5pt}
	\item At the \textit{frame} granularity, we leverage information from an unsupervised saliency method~\!\cite{DBLP:conf/cvpr/YangZLRY13} or CAM~\!\cite{zeng2019multi,zhou2016learning} activation maps to enhance the foreground and background discriminability of our intra-frame representation.
	\item At the \textit{short-term} granularity, we impose the local consistency within the representations of short video clips, to describe the continuous and coherent visual patterns within a few seconds.
	\item At the \textit{long-range} granularity, we address semantic correspondence among distant frames, which makes the cross-frame representations robust to local occlusions, appearance variations and shape deformations.
	\item At the \textit{whole-video} granularity, we encourage the video representation to capture global and compact video content, by learning to aggregate multi-frame information and be discriminative to other videos' representations.
	\vspace*{-2pt}
\end{itemize}

All these constraints are formulated under a unified, multi-granularity VOS (MuG) framework, which is fully differentiable and allows unsupervised/weakly supervised video object pattern learning, from unlabeled videos. Our extensive experiments over various VOS settings, \ie, object-level Z-VOS, instance-level Z-VOS, and O-VOS, show that MuG outperforms other unsupervised and weakly supervised methods by a large margin, and continuously improves its performance with more unlabeled data.

\vspace{-3pt}
\section{Related Work}
\vspace{-2pt}

\subsection{Video Object Segmentation}
\vspace{-2pt}
\noindent\textbf{Z-VOS.} As there is no indication for objects to be segmented, conventional ZVOS methods resorted to certain heuristics, such as saliency~\!\cite{DBLP:conf/cvpr/WangSP15,wang2017video,wang2015consistent,DBLP:conf/bmvc/FaktorI14}, object proposals~\!\cite{DBLP:conf/cvpr/KohK17, DBLP:conf/iccv/PerazziWGS15, lee2011key}, and discriminative motion patterns~\!\cite{DBLP:conf/iccv/OchsB11,DBLP:conf/cvpr/FragkiadakiZS12,DBLP:conf/iccv/PapazoglouF13}.
Recent advances have been driven by deep learning techniques. Various effective networks have been explored, from some early, relatively simple architectures, such as
recurrent network~\!\cite{Song_2018_ECCV,pang2019deep,wang2019learning}, and two-stream network~\!\cite{cheng2017segflow,DBLP:conf/iccv/TokmakovAS17,zhou2020motion}, to recent, more powerful designs, such as teacher-student adaption~\!\cite{siam2018video}, neural co-attention~\!\cite{Lu_2019_CVPR} and graph neural network~\!\cite{wang2019zero,Yan_2019_CVPR}.

\begin{figure*}[b]
	\renewcommand\thefigure{3}
	\vspace{-10pt}
	\centering
	\includegraphics[width=0.99\linewidth]{figs/short-term}
	\put(-486,7){\scriptsize {$X_{\!t}$}}
	\put(-432,7){\scriptsize {$X_{\!t\!+\!1}$}}
	\put(-374,7){\scriptsize {$X_{\!t\!+\!2}$}}
	\put(-456,47){\small {\textcolor{reda}{$p$}}}
	\put(-396,48){\small {\textcolor{reda}{$p'$}}}
	\put(-333,48){\small {\textcolor{reda}{$p''$}}}
	\put(-283,16.5){\scriptsize {$X_{\!t\!+\!1}$}}
	\put(-279,46.5){\small {\textcolor{reda}{$p$}}}
	\put(-242,26){\scriptsize {$\varphi$}}
	\put(-242,48.5){\scriptsize {$\varphi$}}
	\put(-229.1,33.5){\small {$\star$}}
	\put(-210,21){\scriptsize {$S_{\Rightarrow}$}}
	\put(-265,6){\scriptsize {$X_{\!t}$}}
	\put(-221,6){\scriptsize {$X_{\!t\!+\!1}$}}
	\put(-169,6){\scriptsize {$X_{\!t\!+\!2}$}}
	\put(-119,6){\scriptsize {$X_{\!t\!+\!1}$}}
	\put(-69,6){\scriptsize {$X_{\!t}$}}
	\put(-124,16.5){\scriptsize {$X_{\!t}$}}
	\put(-82,26){\scriptsize {$\varphi$}}
	\put(-82,48.5){\scriptsize {$\varphi$}}
	\put(-69.4,33.5){\small {$\star$}}
	\put(-29,19){\scriptsize {$S_{\Leftarrow}$}}
	\put(-29,48){\scriptsize {$G_p$}}
	\put(-20,34){\scriptsize {\textcolor{reda}{$\mathcal{L}_{\text{short}}$}}}
	\vspace{-8pt}
	\captionsetup{font=small}
	\caption{\small Left: Main idea of short-term granularity analysis. Right: Training details for intra-clip coherence modeling.}
	\label{fig:short-term}
	\vspace{-1pt}
\end{figure*}

\noindent\textbf{O-VOS.} As the annotations for the first frame are assumed available at the test phase, O-VOS focuses on how to accurately propagate the initial labels to subsequent frames. Traditional methods typically used optical flow based propagation strategy~\!\cite{DBLP:conf/cvpr/HicksonBEC14,DBLP:journals/tog/FanZLCC15,wang2018semi,DBLP:conf/cvpr/MarkiPWS16}.
Now, deep learning based solutions become the main stream, which can be broadly classified into three categories, \ie, \textit{online learning}, \textit{propagation} and \textit{matching} based methods. Online learning based methods~\!\cite{Caelles_2017_CVPR,voigtlaender2017online,DBLP:conf/cvpr/PerazziKBSS17}  fine-tune the segmentation network for each test video on the first-frame annotations. Propagation based methods~\!\cite{DBLP:conf/cvpr/JampaniGG17, wug2018fast,yang2018efficient} rely on the segments of the previous frames and work in a frame-by-frame manner. Matching based methods~\!\cite{wang2019ranet,Voigtlaender_2019_CVPR,luiten2018premvos} segment each frame according to its correspondence/matching relation to the first frame.

Typically, current deep learning based VOS solutions, under either Z-VOS or O-VOS setting, are trained using a large amount of elaborately annotated data for supervised learning.
In contrast, the proposed method trains a VOS network from scratch using unlabeled videos.  This is essential
for understanding how visual recognition works in VOS and for
narrowing down the annotation budget.

\vspace{-0pt}
\subsection{VOS with Unlabeled Training Videos}
\label{sec:wvos}
\vspace{-1pt}
Learning VOS from unlabeled videos is an essential, yet rarely touched avenue. Among a few efforts,~\cite{pathak2017learning} represents an early attempt in this direction, which uses a modified, purely unsupervised version of~\cite{DBLP:conf/bmvc/FaktorI14} to generate proxy masks as pseudo annotations.
With a similar spirit, some methods use heuristic segmentation masks~\cite{croitoru2017unsupervised} or weakly supervised location maps~\!\cite{Lee2019FrametoFrameAO} as supervisory signals. With a broader view, some works~\!\cite{Tang_2013_CVPR,hartmann2012weakly,Zhang_2017_CVPR} capitalized on untrimmed videos tagged with semantic labels. In addition to increased annotation efforts, they are hard to handle such a class-agnostic VOS setting. Recently, self-supervised video learning has been applied for O-VOS~\!\cite{vondrick2018tracking,CVPR2019_CycleTime}, which imposes the learned features to capture certain constraints on local coherence, such as cross-frame color consistency~\!\cite{vondrick2018tracking} and temporal cycle-correspondence~\!\cite{CVPR2019_CycleTime}

Our method is distinctive for two aspects. First, it explores various intrinsic properties of videos as well as class-agnostic fore-background knowledge in a unified, multi-granularity framework, bringing a more comprehensive understanding of visual patterns in VOS. Second, it shows strong video object representation learning ability and, for the first time, it is applied to diverse VOS settings after only being trained once. This gives a new glimpse into the connections between the two most influential VOS settings.

\vspace{-2pt}
\section{Proposed Algorithm}
\vspace{-2pt}
\subsection{Multi-Granularity VOS Network}
\vspace{-1pt}
\label{sec:multi-gran-network}
For a training video $\mathcal{X}\!\!\in$\bm{$\mathcal{X}$} containing $T$ frames: $\mathcal{X}\!=\!\{X_t\}_{t=1}^T$, its features are specified as $\{\textbf{\textit{x}}_{t}\}_{t=1}^T$, obtained from a fully convolutional feature extractor $\varphi$: $\textbf{\textit{x}}_{t\!}\!=_{\!}\!\varphi(X_t)\!\in_{\!}\!\mathbb{R}^{W\!\times\! H\!\times\!C}$. Four-granularity
characteristics are explored to guide the learning of $\varphi$ (Fig.~\!\ref{fig:overview}), described as follows.

\noindent\textbf{Frame Granularity Analysis: Fore-background Knowledge Understanding.} As $\varphi$ is VOS-aware, basic fore-background knowledge is desired to be encoded. In our method, such knowledge (Fig.~\!\ref{fig:top-right}~\!(b)) is initially from a background prior based saliency model~\!\cite{DBLP:conf/cvpr/YangZLRY13} (in an unsupervised learning setting), or in a form of CAM maps~\!\cite{zeng2019multi,zhou2016learning} (in a weakly supervised learning setting).

Formally, for each frame $X_t$, let us denote its corresponding initial fore-background mask as $Q_t\!\in\!\{0,1\}^{W\!\times\!H}$ (\ie, a binarized saliency or CAM activation map). In our frame granularity analysis, the learning of $\varphi$ is guided by the supervision signals of $\{Q_t\}_{t=1}^T$, \ie, utilizing the intra-frame information $\textbf{\textit{x}}_{t\!}\!=_{\!}\!\varphi(X_t)$ to regress $Q_t$:
\vspace{-2pt}
\begin{equation}\small
\begin{aligned}
\mathcal{L}_{\text{frame}}=\mathcal{L}_{\text{CE}}(P_t, Q_t).
\end{aligned}
\vspace{-2pt}
\label{eq:frame}
\end{equation}
Here $\mathcal{L}_{\text{CE\!}}$ is the \textit{cross-entropy} loss, and $P_t\!=\!\rho(\textbf{\textit{x}}_{t})$ where $\rho_{\!}\!:_{\!}\!\mathbb{R}^{\!W\!\times\!H\!\times\!C}\!\!\mapsto\!\![0,1]^{\!W\!\times\!H\!}$ maps the input single-frame feature $\textbf{\textit{x}}_{t}$ into a fore-background prediction map $P_t$. $\rho$ is implemented by a $1\!\times\!1$ convolutional layer with \textit{sigmoid} activation.

\begin{figure}[t]

	\renewcommand\thefigure{2}
	\centering
	\includegraphics[width=0.99\linewidth]{figs/overview}
	\put(-45,106){\footnotesize {\textcolor{reda}{$\mathcal{L}_{\text{long}}$}}}
	\put(-128,36){\footnotesize {\textcolor{reda}{$\mathcal{L}_{\text{global}}$}}}
	\put(-219,86){\footnotesize {\textcolor{reda}{$\mathcal{L}_{\text{short}}$}}}
	\put(-147,157){\footnotesize {\textcolor{reda}{$\mathcal{L}_{\text{frame}}$}}}
	\vspace{-8pt}
	\captionsetup{font=small}
	\caption{\small Overview of our approach. Intrinsic properties over {\color{reda}\textbf{frame}}, {\color{myyellow}\textbf{short-term}}, {\color{mygreen}\textbf{long-term}} and {\color{myblue}\textbf{whole video}} granularities are explored to guide the video object pattern learning.}
	\label{fig:overview}
	\vspace{-12pt}
\end{figure}

\noindent\textbf{Short-Term Granularity Analysis: Intra-Clip Coherence Modeling.} Short-term coherence is an essential property in videos, as temporally close frames typically exhibit continuous visual content changes~\!\cite{hurri2003simple}. To capture this property, we apply a forward-backward patch tracking mechanism~\cite{wang2019unsupervised}. It learns  $\varphi$ by
tracking a sampled patch forwards in a few successive frames and then backwards until the start frame, and penalizing the distance between the initial and final backwards tracked positions of that patch.

Formally, given two consecutive frames $X_{t}$ and $X_{t+1}$, we first crop a patch $p$ from $X_t$ and apply $\varphi$ on $p$ and $X_{t+1}$, separately. Then we get two feature embeddings: $\varphi(p)\!\in\!\mathbb{R}^{w\!\times\!h\!\times\!C\!}$  and $\textbf{\textit{x}}_{t+1\!}\!=_{\!}\!\varphi(X_{t+1})\!\in\!\mathbb{R}^{W\!\times\!H\!\times\!C\!}$. With a design similar to the classic Siamese tracker~\!\cite{bertinetto2016fully}, we forward track the patch $p$ on the next frame $X_{t+1}$ by conducting a cross-correlation operation `$\star$' on $\varphi(p)$ and $\varphi(X_{t+1})$:
\vspace{-2pt}
\begin{equation}\small
S_{\Rightarrow} = \varphi(p)\star\varphi(X_{t+1}) \in [0, 1]^{W\!\times\!H},
\label{equ:fortrack}
\vspace{-1pt}
\end{equation}
where$_{\!}$ $S_{\Rightarrow\!}$ is a \textit{sigmoid}-normalized response map whose size is rescaled into$_{\!}$ $(H_{\!}, W)$. The new location of $p$ in$_{\!}$ $X_{t+1\!}$ is then inferred according to the peak value on$_{\!}$ $S_{\Rightarrow}$. After obtaining the forward tracked patch $p'$ in$_{\!}$ $X_{t+1}$, we backward track$_{\!}$  $p'_{\!}$ to$_{\!}$ $X_{t\!}$ and$_{\!}$  get$_{\!}$  a$_{\!}$  backward tracking response map$_{\!}$ $S_{\Leftarrow}$:
\vspace{-8pt}
\begin{equation}\small
S_{\Leftarrow} = \varphi(p')\star\varphi(X_{t}) \in [0, 1]^{W\!\times\!H}.
\label{equ:backtrack-correlation}
\vspace{-0pt}
\end{equation}
Ideally, the peak of $S_{\Leftarrow\!}$ should correspond to the location of $p$ in the initial frame $X_t$. Thus we build a consistency loss that measures the alignment error
between the initial and forward-backward tracked positions of $p$:
\vspace{-2pt}
\begin{equation}\small
\mathcal{L}_{\text{short}} = \|S_{\Leftarrow}- G_p\|_2^2,
\label{equ:clip-level}
\vspace{-1pt}
\end{equation}
where $G_p\!\in\![0, 1]^{W\!\times\!H}$ is a $(H, W)$-dimensional Gaussian-shape map with the same center of $p$ and variance proportional to the size of $p$. As in~\!\cite{wang2019unsupervised}, the above forward-backward tracking mechanism is extended to a multi-frame setting (Fig.~\!\ref{fig:short-term}). Specifically, after obtaining the forward tracked patch $p{'\!}$ in $X_{t+1}$, $p{'\!}$ is further tracked to the next frame$_{\!}$ $X_{t+2}$, and$_{\!}$ a$_{\!}$ new$_{\!}$ tracked$_{\!}$ patch $p''$ is obtained. Then $p''\!$ is reversely tracked$_{\!}$ to$_{\!}$ $X_{t+1\!}$ and$_{\!}$ further$_{\!}$ to$_{\!}$ the$_{\!}$ initial$_{\!}$ frame $X_{t}$, and the local consistency loss in Eq.~\!\ref{equ:clip-level} is computed. Moreover, during training, we first random sample a short video clip consisting of six successive frames. Then we perform above forward-backward tracking based learning strategy over three frames random drawn from the six-frame video clip. With above designs, $\varphi$ captures the spatiotemporally local correspondence and is content-discriminative (due to its cross-frame target re-identification nature)

\begin{figure}[t]

	\centering
	\includegraphics[width=0.99\linewidth]{figs/long-term}
	\put(-230,6){\scriptsize {$\mathcal{X}$}}
	\put(-183,32){\scriptsize {$X_{\!i}$}}
	\put(-183,5){\scriptsize {$X_{\!j}$}}
	\put(-131,14){\scriptsize {$\varphi$}}
	\put(-131,40){\scriptsize {$\varphi$}}
	\put(-87,49){\scriptsize {$\textbf{\textit{A}}_{i,j}$}}
	\put(-87,4){\scriptsize {$\textbf{\textit{A}}_{j,i}$}}
	\put(-77,40){\footnotesize {$\kappa$}}
	\put(-77,14){\footnotesize {$\kappa$}}
	\put(-46.2,37){\scriptsize {$\tau_{i,j}$}}
	\put(-46.,15){\scriptsize {$\tau_{j,i}$}}

	\put(-21,26){\scriptsize {\textcolor{reda}{$\mathcal{L}_{\text{long}}$}}}
	\vspace{-10pt}
	\captionsetup{font=small}
	\caption{\small Illustration of our long-term granularity analysis.}
	\label{fig:long-term}
	\vspace{-12pt}
\end{figure}

\noindent\textbf{Long-Term Granularity Analysis: Cross-Frame Semantic Matching.} In addition to the local consistency among adjacent frames, there also exist strong semantic correlations among distant frames, as frames from the same video typically contain similar
content~\!\cite{Hossein2009,yan2019fine}. Capturing this property is essential for $\varphi$, as it makes $\varphi$ robust to many challenges, such as appearance variants, shape deformations, object occlusions, \etc. To address this issue, we conduct a long-term granularity analysis, which casts cross-frame correspondence learning as a dual-frame semantic matching problem (Fig.~\!\ref{fig:long-term}). Specifically, given a training pair of two \textit{disordered} frames $(X_i, X_j)$ randomly sampled from $\mathcal{X}$, we compute a similarity affinity $\textbf{\textit{A}}_{i,j}$ between their embeddings: $(\varphi(X_i), \varphi(X_j))$ by a co-attention operation~\!\cite{DBLP:conf/nips/VaswaniSPUJGKP17}:
\vspace{-1pt}
\begin{equation}\small
\textbf{\textit{A}}_{i,j}=\text{softmax}({\textbf{\textit{x}}_i}^{\!\!\!\top}\textbf{\textit{x}}_j)\in [0, 1]^{(W\!H)\times (W\!H)},
\label{equ:long-coattention}
\vspace{-0pt}
\end{equation}
where ${\textbf{\textit{x}}_{i\!}}\!\in_{\!}\!\mathbb{R}^{C\!\times\!(W\!H)\!\!}$ and$_{\!}$ ${\textbf{\textit{x}}_{j\!}}\!\!\in_{\!}\!\mathbb{R}^{C\!\times\!(W\!H)\!}$ are$_{\!}$ flat$_{\!}$ matrix formats of $\varphi(X_i)$ and $\varphi(X_j)$, respectively. `\text{softmax}' indicates \textit{column-wise softmax} normalization.
Given the normalized cross-correlation $\textbf{\textit{A}}_{i,j}$, in line with~\!\cite{rocco2018end}, we use a small neural network $\kappa_{\!}\!:\!\mathbb{R}^{(W\!\times\!H)\!\times\!(W\!\times\!H)\!\!}\!\mapsto\!\mathbb{R}^{6\!}$ to regress the parameters of a geometric transformation $\tau_{i,j}$, \ie, six-degree of freedom (translation, rotation and scale). $\tau_{i,j\!}\!:\!\mathbb{R}^{2}\!\mapsto\!\mathbb{R}^{2}$ gives the relations between the spatial coordinates in $X_i$ and $X_j$ considering the corresponding semantic similarity:
\vspace{-2pt}
\begin{equation}\small
\textbf{\textit{m}}_i=\tau_{i,j}(\textbf{\textit{m}}_j),
\vspace{-1pt}
\end{equation}
where $\textbf{\textit{m}}_i$ is a 2-D spatial coordinate of $X_i$, and $\textbf{\textit{m}}_j$ the corresponding sampling coordinates in $X_j$. Using $\tau_{i,j}$, we can warp $X_i$ to $X_j$.
Similarly, we can also compute $\tau_{j,i}$, \ie, a 2-D warping from $X_j$ to $X_i$. Let us consider two sampling coordinates $\textbf{\textit{m}}_i$ and $\textbf{\textit{n}}_j$ in $X_j$ and $X_i$, respectively, we introduce a semantic matching loss~\!\cite{rocco2018end}:
\vspace{-1pt}
\begin{equation}\small
\begin{aligned}
\mathcal{L}_{\text{long}}=-\big(&\sum\nolimits_{\textbf{\textit{m}}_i\in\Omega}\sum\nolimits_{\textbf{\textit{o}}_j\in\Omega}\textbf{\textit{A}}_{i,j}(\textbf{\textit{m}}_i, \textbf{\textit{o}}_j)\iota(\textbf{\textit{m}}_i, \textbf{\textit{o}}_j)+\\[-2mm] &\sum\nolimits_{\textbf{\textit{n}}_j\in\Omega}\sum\nolimits_{\textbf{\textit{o}}_i\in\Omega}\textbf{\textit{A}}_{j,i}(\textbf{\textit{n}}_j, \textbf{\textit{o}}_i)\iota(\textbf{\textit{m}}_i, \textbf{\textit{o}}_i)\big),
\end{aligned}
\label{equ:semantic-matching}
\vspace{-1pt}
\end{equation}
where $\Omega$ refers to the image lattice, $\textbf{\textit{A}}_{i,j}(\textbf{\textit{m}}_i, \textbf{\textit{o}}_j)\!\in_{\!}\![0,1]$ gives the similarity value between the positions $\textbf{\textit{m}}_i$ and $\textbf{\textit{o}}_j$ in $X_i$ and $X_j$, and $\iota(\textbf{\textit{m}}_i, \textbf{\textit{o}}_j)$ determines if the correspondence between $\textbf{\textit{m}}_i$ and $\textbf{\textit{o}}_j$ is geometrically consistent.$_{\!}$ If$_{\!}$ $||\textbf{\textit{m}}_i, \tau_{i,j}(\textbf{\textit{o}}_j)||_{\!}\!\leq_{\!}\!1$, $\iota_{\!}\!=_{\!}\!1$; otherwise $\iota_{\!}\!=_{\!}\!0$.

\noindent\textbf{Video Granularity Analysis: Global and Discriminative Representation Learning.}
So far, we have used the pairwise cross-frame information in local and long terms to boost the learning of $\varphi$. $\varphi$ is also desired to learn a compact and globally discriminative video representation. To achieve this, with a \textit{global information aggregation} module, we perform a video granularity analysis within an unsupervised video embedding learning framework~\!\cite{dosovitskiy2014discriminative}, which leverages supervision signals from different videos.

Starting$_{\!}$ with$_{\!}$ our$_{\!}$ global$_{\!}$ information$_{\!}$ aggregation$_{\!}$ module, we split
$\mathcal{X}_{\!}\!=_{\!}\!\{_{\!}X_{t\!}\}_{t=1\!}^T$ into $K$ segments of equal durations:$_{\!}$ \!$\mathcal{X}_{\!}\!=\!\cup^K_{k=1}\mathcal{X}_k$. For each segment  $\mathcal{X}_{k}$, we randomly sample a single frame, resulting in a $K_{\!}$-frame abstract  $\mathcal{X}'_{\!}\!\!=_{\!}\!\{X_{t_k}\}_{k=1\!}^K$ of $\mathcal{X}$. $\mathcal{X}'_{\!}$ reduces the redundancy among successive frames while preserving global information.

With a similar spirit of \textit{key}-\textit{value} retrieval networks~\!\cite{sukhbaatar2015end}, for each $X_{t_k}\!\in\!\mathcal{X}'$, we set it as a \textit{query} and the rest frames $\mathcal{X}'/X_{t_k}$ as \textit{reference}. Then we compute the normalized cross-correlation between the query and reference:
\vspace{-1pt}
\begin{equation}\small
\!\!\!\!\textbf{\textit{A}}_{t_k\!}\!\!=\!\text{softmax}({\textbf{\textit{x}}_{t_{k}}}^{\!\!\!\!\!\!\top}[\{\textbf{\textit{x}}_{t_{k'}}\}_{t_{k'}}])\in [0,1]^{(W\!H)\times(W\!H\!(K-1))},
\label{equ:global-coattention}
\vspace{-1pt}
\end{equation}
where $k'\!\in\!\{1,\cdots,K\}/k$, and `[$\cdot$]' denotes the concatenation operation. ${\textbf{\textit{x}}_{t_k}}\!\in_{\!}\!\mathbb{R}^{C\!\times\!(W\!H)\!}$
and $[\{\textbf{\textit{x}}_{t_{k'}}\}_{t_{k'}\in\{1,\cdots,K\}/k}]\!\in_{\!}\!\mathbb{R}^{C\!\times\!(W\!H\!(K-1))}$ are flat feature matrices  of the query and reference, respectively. Subsequently, $\textbf{\textit{A}}_{t_k}$ is used as a weight matrix for global information summarization:
\vspace{-1pt}
\begin{equation}\small
\!\!\!\!\textbf{\textit{x}}'_{t_k}\!=\![\{\textbf{\textit{x}}_{t_{k'\!}}\}_{t_{k'\!}}]~\textbf{\textit{A}}_{t_k}^{\!\top}\!\in \! \mathbb{R}^{(W\!H)\!\times\!C\!},~~\text{where~} k'\!\in\!\{1,\cdots\!,K\}/k.\!\!
\label{equ:aggregation}
\vspace{-1pt}
\end{equation}
Our global information aggregation module gathers information from the reference set by a correlation-based feature summarization procedure. For the query frame $X_{t_{k}}$, we obtain its global information augmented representation by:
\vspace{-1pt}
\begin{equation}\small
\textbf{\textit{r}}_{t_k} = [\textbf{\textit{x}}'_{t_k}, \textbf{\textit{x}}_{t_k}] \in \! \mathbb{R}^{W\! \times \!H\!\times \!2C}.
\vspace{-1pt}
\label{eq:global}
\end{equation}

During training, the video granularity analysis essentially discriminates between a set of surrogate video classes~\!\cite{dosovitskiy2014discriminative}. Specifically, given $N$ training videos, we randomly sample a single frame from each video, leading to $N$ training \textit{instances}: $\{X^{n}\}_{n=1}^N$. The core idea is that, for a query frame $X^n_{t_k}$ in the $n$-th video, its global feature embedding is close to the instance $X^{n}$ from the same $n$-th video, and far from other unrelated instances $\{X^{n'}\}_{n'\neq n}$ (from the other$_{\!}$ $N_{\!}\!-_{\!}\!1$ videos). We$_{\!}$ solve$_{\!}$ this$_{\!}$ as$_{\!}$ a$_{\!}$ binary$_{\!}$ classification
problem via maximum likelihood estimation (MLE). In particular, for $X^n_{t_k}$,  instance $X^n$ should be classified into $n$, while other instances $\{X^{n'}\}_{n'\neq n}$ shouldn't be. The probability of
$X^n$ being recognized as instance $n$ is:
\vspace{-3pt}
\begin{equation}\small
P(n|X^n) = \frac{\exp(\text{GAP}(\textbf{\textit{r}}^{n\!\top}_{t_k} \textbf{\textit{r}}^{n}))}{\sum_{i=1}^{N}\exp(\text{GAP}(\textbf{\textit{r}}^{n\!\top}_{t_{k}} \textbf{\textit{r}}^{i}))}.
\vspace{-2pt}
\end{equation}
where `\text{GAP}' stands for \textit{global average pooling}. Similarly, given $X^n_{t_k}$, the probability of other instances $X^{n'}$ be recognized as instance $n$ is:
\vspace{-6pt}
\begin{equation}\small
P(n|X^{n'}) = \frac{\exp(\text{GAP}(\textbf{\textit{r}}^{n\!\top}_{t_k} \textbf{\textit{r}}^{n'}))}{\sum_{i=1}^{N}\exp(\text{GAP}(\textbf{\textit{r}}^{n\!\top}_{t_{k}} \textbf{\textit{r}}^{i}))}.
\vspace{-3pt}
\end{equation}
Correspondingly, the probability of $X^{n'\!}$ not being recognized as instance$_{\!}$ $n$ is$_{\!}$ $1_{\!}\!-_{\!}\!P(n|X^{n'\!})$. The$_{\!}$ joint$_{\!}$ probability of$_{\!}$ $X^{n\!}$ being$_{\!}$ recognized$_{\!}$ as$_{\!}$ instance$_{\!}$ $n_{\!}$ and$_{\!}$ $X^{n'\!}$ not$_{\!}$ being is: $P(n|X^{n})\prod_{n'\neq n}(1\!-\!P(n|X^{n'\!}))$, under the assumption that different instances being recognized as $n$ are independent.

Then the loss function is defined as the negative log likelihood over $N$ query frames from $N$ videos:
\vspace{-1pt}
\begin{equation} \small
\mathcal{L}_{\text{global}}\!=\!-\!\sum_n\log P(n|X^{n})-\!\sum_n\!\sum_{n'\neq n\!}\log(1\!-\!P(n|X^{n'})).
\label{equ:MLE}
\vspace{-3pt}
\end{equation}
Next we will describe the network architecture during the training and inference phases. An appealing advantage of our multi-granularity VOS network is that, after being trained in a unified mode, it can be directly applied to both Z-VOS and O-VOS settings with only slight adaption.

\vspace{-2pt}
\subsection{\!One$_{\!}$ Training$_{\!}$ Phase$_{\!}$ for$_{\!}$ both$_{\!}$ Z-VOS$_{\!}$ and$_{\!}$ O-VOS}
\label{sec:architecture}
\vspace{-1pt}
\noindent\textbf{Network Architecture.} Our whole module is end-to-end trainable.
The video representation space $\varphi$ is learned by a fully convolutional network, whose design is inspired by ResNet-50~\!\cite{he2016deep}. In particular, the first four groups of convolutional layers in ResNet are preserved and dilated convolutional layer~\!\cite{yu2015multi} is used to maintain enough spatial details as well as ensure a large receptive field, resulting in a 512-channel feature representation $\textbf{\textit{x}}$ whose spatial dimensions are $1/4$ of an input video frame $X$.

During training, we use a mini-batch of $N_{\!}\!=_{\!}\!16$ videos and scale all the training frames into $256_{\!}\!\times_{\!}\!256$ pixels.
For frame granularity analysis,
all the frames access to the supervision signal from the loss $\mathcal{L}_{\text{frame}}$ in Eq.~\!\ref{eq:frame}.

For short-term granularity analysis, six successive video frames are first randomly sampled from each training video, resulting a six-frame video clip. For each video clip, we further sample three video frames orderly and randomly crop a $64\!\times\!64$ patch as $p$. With the feature embedding $\varphi(p)\!\in\!\mathbb{R}^{16\!\times\!16\!\times64}$ of $p$, we forward-backward track $p$ and get its final backward tracking response map ${S}_{\Leftarrow\!}\!\in\![0,1]^{64\!\times\!64}$ via Eq.~\!\ref{equ:backtrack-correlation}. For computing the loss in Eq.~\!\ref{equ:clip-level}, the Gaussian-shape map $G_p\!\in\![0,1]^{64\!\times\!64}$ is obtained by convolving the center position of $p$ with a two-dimension Gaussian map with a kernel width proportional (0.1) to the patch size.

For long-term granularity analysis, after randomly sampling two \textit{disordered} frames $(X_i, X_j)$ ($|i\!-\!j|\!\!\geq\!\!6$) from a training video $\mathcal{X}$, we compute the correlation map $\textbf{\textit{A}}_{i,j}\!\in\![0,1]^{(64\!\times\!64)\!\times\!(64\!\times\!64)}$ by the normalized inner production operation in Eq.~\!\ref{equ:long-coattention}. For the geometric transformation parameter estimator $\kappa_{\!}\!:\!\mathbb{R}^{(64\!\times\!64)\!\times\!(64\!\times\!64)}\!\mapsto\!\mathbb{R}^{6\!}$, it is achieved by two convolutional layers and one linear layer, as in~\!\cite{rocco2018end}. Then the semantic matching loss in Eq.~\!\ref{equ:semantic-matching} is computed.

For video granularity analysis, we split each training video $\mathcal{X}$ into $K\!\!=\!\!8$ segments, and get the global information augmented representation $\textbf{\textit{r}}_{t_k}\!\in\!\mathbb{R}^{64\!\times\!64\!\times\!256}$  for each query frame $X_{t_{k}}$ by Eq.~\!\ref{eq:global}. Then, we compute the soft-max embedding learning loss using Eq.~\!\ref{equ:MLE}, which leverages supervision signals from the $N$ training videos.

\noindent\textbf{Iterative Training by  Bootstrapping.} As seen  in Fig.~\!\ref{fig:top-right}~\!(b), the fore-background knowledge from the saliency~\!\cite{DBLP:conf/cvpr/YangZLRY13} or CAM~\!\cite{zeng2019multi,zhou2016learning} is ambiguous and noisy. Inspired by Bootstrapping~\!\cite{reed2014training}, we apply an iterative training strategy: after training with the initial fore-background maps, we use our trained model to re-label the training data. With each iteration, the learner bootstraps itself by mining better fore-background knowledge and then leading a better model.
Specifically, for each training frame $X$, given the initial fore-background mask $Q\!\in\!\{0,1\}^{64\!\times\!64}$ and current prediction $\bar{P}^i\!\in\!\{0,1\}^{64\!\times\!64}$ of the model in $i$-th training iteration, the loss in Eq.~\!\ref{eq:frame} in $(i\!+\!1)$-th iteration is formulated in a bootstrapping format:
\vspace{-2pt}
\begin{equation}\small
\begin{aligned}
\!\!\!\!\mathcal{L}^{(i+1)}_{\text{frame}}\!\!=\!\!\sum\nolimits_{\textbf{\textit{m}}\in\Omega}&[\alpha Q_{\textbf{\textit{m}}}\!+\!(1\!-\!\alpha)\bar{P}_{\textbf{\textit{m}}}^{i}]\log(P_{\textbf{\textit{m}}}^{i+1})+\\[-0.7mm] &[\alpha(1\!-\!Q_{\textbf{\textit{m}}})\!+\!(1\!-\!\alpha)(1\!-\!\bar{P}_{\textbf{\textit{m}}}^{i})]\!\log(1\!-\!P_{\textbf{\textit{m}}}^{i+1}),\!\!\!
\end{aligned}
\vspace{-1pt}
\label{eq:framenew}
\end{equation}
where $\alpha\!=\!0.05$ and $Q_{\textbf{\textit{m}}}$ gives the value in position $\textbf{\textit{m}}$. In such a design, the `confident' fore-background knowledge is generated as a convex combination of the initial fore-background information $Q$ and model prediction $P$.

In the $i$-th training iteration, the overall loss to optimize the whole network parameters is the combination of the losses in Eq.~\!\ref{eq:framenew},~\!\ref{equ:backtrack-correlation},~\!\ref{equ:semantic-matching}
and~\!\ref{equ:MLE}:
\vspace{-2pt}
\begin{equation}\small
\mathcal{L}^{(h)}\!=\!\mathcal{L}^{(h)}_{\text{frame}}\!+\!\beta_1\mathcal{L}_{\text{short}}\!+\! \beta_2\mathcal{L}_{\text{long}}\!+\!\beta_3\mathcal{L}_{\text{global}}
\label{equ:total}
\vspace{-1pt}
\end{equation}
where $\beta$s are coefficients: $\beta_1\!=\!0.1, \beta_2\!=\!0.02$ and $\beta_3\!=\!0.5$.

The above designs enable a unified un-/weakly supervised feature learning framework. Once the model is trained, the learned representations $\varphi$ can be used for Z-VOS and O-VOS, with slight modifications. In practice, we find that our model can perform well after being trained with 2 iterations; please see~\!\S\ref{sec:ablation} for related experiments.

\vspace{-2pt}
\subsection{Inference Modes for Z-VOS and O-VOS}
\label{sec:inference}
\vspace{-1pt}
Now we detail our inference modes for object-level Z-VOS, instance-level Z-VOS, and O-VOS settings.

\noindent\textbf{Object-Level Z-VOS Setting.} For each test frame, object-level Z-VOS aims to predict a binary segmentation mask where the primary foreground objects are separated from the background while the identities of different foreground objects are not distinguished. In the classic VOS setting, since there is no any test-time human intervention, how to discover the primary video objects is the central problem. Considering the fact that interested objects frequently appear throughout the video sequence, we readout the segmentation results from the global information augmented feature $\textbf{\textit{r}}$, instead of directly using intra-frame information to predict the fore-background mask (\ie, $\rho(\textbf{\textit{x}})$).
This is achieved by an extra segmentation readout layer $\upsilon_{\!}\!:\!\mathbb{R}^{64\!\times\!64\!\times\!256}\!\mapsto\![0,1]^{64\!\times\!64}$, which takes the global frame embedding $\textbf{\textit{r}}$ as the input and produces the final object-level segmentation prediction. $\upsilon$ is also trained by the \textit{cross-entropy} loss, as in Eq.~\!\ref{eq:framenew}. For notation clarity, we omit this term in the overall training loss in Eq.~\!\ref{equ:total}.  Please note that $\upsilon$ is only used in Z-VOS setting; for O-VOS setting, the segmentation masks are generated with a different strategy.

\noindent\textbf{Instance-Level Z-VOS Setting.} Our model can also be
adapted for the instance-level Z-VOS setting, in which different
object instances must be discriminated, in addition to separating the
primary video objects from the background without test-time human
supervision.  For each test frame, we first apply mask-RCNN~\!\cite{he2017mask} to produce a set of category agnostic object proposals
Then we apply our trained model for producing a binary foreground-background mask per frame.
After combining object bounding-box proposals with binary object-level segmentation masks,
we can filter out the background proposals and obtain pixel-wise, instance-level object candidates for each frame. Finally, to link those object candidates across different frames, similar to~\!\cite{luiten2018premvos}, we use overlap ratio and optical flow as the cross-frame candidate-association metric. Note that, mask-RCNN can be replaced with non-learning Edgebox~\cite{ZitnickECCV14edgeBoxes} and GrabCut, resulting a purely unsupervised/weakly-supervised protocol.

\noindent\textbf{O-VOS Setting.} In O-VOS, for each test video sequence, instance-level annotations regarding multiple general foreground objects in the first frame are given.  In such a setting, our trained network works in a per-frame matching based mask propagation fashion. Concretely, assume there are a total of $L$ object instances (including the background) in the first-frame annotation, each spatial position $\textbf{\textit{n}}_{\!}\!\in_{\!}\!\Omega$ will be associated with a one-hot class vector $\hat{\textbf{\textit{y}}}_{\textbf{\textit{n}}\!}\!\in_{\!}\!\{0,1\}^L$, whose element $\hat{y}^l_{\textbf{\textit{n}}\!}$ indicates whether pixel $\textbf{\textit{n}}_{\!}$ belong to $l$-th object instance. Starting from the second frame, we use both the last segmented frame $X_{t-1}$ as well as current under-segmented frame $X_{t}$ to build an input pair for our model. Then we compute their similarity affinity $\textbf{\textit{A}}_{t-1,t\!}\!\in_{\!}\![0, 1]^{(64\!\times\!64)\!\times\!(64\!\times\!64)}$ in the feature space: $\textbf{\textit{A}}_{t-1,t\!}\!=_{\!}\!\text{softmax}({\textbf{\textit{x}}_{t-1}}^{\!\!\!\top}\textbf{\textit{x}}_{t})$. After that, for each pixel $\textbf{\textit{m}}$ in $X_{t}$, we compute its probability distribution $\textbf{\textit{v}}_{\textbf{\textit{m}}}\!\in\![0,1]^L$ over the $L$ object instances as:
\vspace{-2pt}
\begin{equation}\small
\textbf{\textit{v}}_{\textbf{\textit{m}}}\!=\!\sum\nolimits_{\textbf{\textit{n}}\in\Omega}\textbf{\textit{A}}_{t-1,t}(\textbf{\textit{n}},\textbf{\textit{m}})~~\hat{\textbf{\textit{y}}}_{\textbf{\textit{m}}},
\vspace{-1pt}
\end{equation}
where $\textbf{\textit{A}}_{t-1,t}(\textbf{\textit{n}}, \textbf{\textit{m}})\!\in_{\!}\![0,1]_{\!}$ is the affinity value between pixel $\textbf{\textit{n}}$ in $X_{t-1}$ and $\textbf{\textit{m}}$ in $X_t$.  For $\textbf{\textit{m}}$, it is assigned to $l^{*\!}$-th instance: $l^{*\!}\!=_{\!}\!\mathop{\arg\max}_l(\{v^l_{\textbf{\textit{m}}}\}_{l=1}^L)$, where $\textbf{\textit{v}}_{\textbf{\textit{m}}\!}\!=\![v^l_{\textbf{\textit{m}}}]_{l=1}^L$. Then we get its label vector $\hat{\textbf{\textit{y}}}_{\textbf{\textit{m}}}$. In this way, from the segmented frame $X_{t}$, we move to the next input frame pair $(X_{t},X_{t+1})$ and get the segmentation result for $X_{t+1}$. As our method does not use any first-frame fine-tuning~\!\cite{cheng2017segflow,DBLP:conf/cvpr/PerazziKBSS17} or online learning~\cite{voigtlaender2017online} technique, it is fast for inference

\begin{table}[t]
	\centering
	\resizebox{0.49\textwidth}{!}{
		\setlength\tabcolsep{1pt}
		\renewcommand\arraystretch{1.0}
		\begin{tabular}{c|c||cc||cc}
			\hline\thickhline
			\rowcolor{mygray}
			&	 &\multicolumn{2}{c||}{Unsuper.} &\multicolumn{2}{c}{Weakly-super.}\\
			\rowcolor{mygray}
			\multirow{-2}{*}{Aspects} &\multirow{-2}{*}{Module}  &mean $\mathcal{J}$ & $\Delta$$\mathcal{J}$ &mean $\mathcal{J}$ & $\Delta$$\mathcal{J}$\\
			\hline
			\hline
			Reference&\textbf{Full model} (2 iterations)  & 58.0 &- & 61.2 &-\\
			\hline
			\hline
			\multirow{2}{*}{\tabincell{c}{Initial Fore-/Background\\Knowledge}}
			&Heuristic Saliency~\!\cite{DBLP:conf/cvpr/YangZLRY13} & 37.2&-20.8 &- &-\\
			&CAM~\!\cite{zeng2019multi} &- &- &45.3 &-15.9\\
			\hline
			\multirow{4}{*}{\tabincell{c}{Multi-Granularity\\Analysis}}
			&\textit{w/o.}  Frame Granularity  & 40.2 & -17.8 & 40.2  & -21.0\\
			&\textit{w/o.}  Short-term Granularity  & 51.3 & -6.7 & 57.1 & -4.1\\
			&\textit{w/o.}  Long-term Granularity  & 52.8 & -5.2 & 56.0 &-5.2 \\
			&\textit{w/o.}  Video  Granularity  & 56.4 & -1.6 & 60.4 & -0.8\\

			\hline
			\multirow{3}{*}{\tabincell{c}{Iterative Training\\ via Bootstrapping}} & 1 iteration~ & 50.8 & -7.2 & 54.9 & -6.3\\
			& 3 iterations & 58.0 & 0.0 & 61.2 & 0.0\\
			& 4 iterations & 58.0 & 0.0 & 61.2 & 0.0\\
			\hline
			More Data& 	+ LaSOT dataset~\!\cite{Fan_2019_CVPR}  & 59.5 & +1.5& 62.3 & +1.1\\
			\hline
			Post-Process& 	\textit{w/o.} CRF  & 55.3 & -2.7 & 58.7 & -2.5\\
			\hline
	\end{tabular}}
	\vspace*{-8pt}
	\captionsetup{font=small}
	\caption{\small \textbf{Ablation study on DAVIS$_{16}$~\!\cite{perazzi2016benchmark} \texttt{val} set}, under the object-level Z-VOS setting. Please see~\!\S\ref{sec:ablation} for details.}
	\label{table:abl}
	\vspace*{-12pt}
\end{table}

\vspace{-3pt}
\section{Experiment}
\label{sec:exp}
\vspace{-2pt}
\subsection{Common Setup}
\vspace{-1pt}
\noindent\textbf{Implementation Details.}  We train the whole network from scratch on the OxUvA~\!\cite{valmadre2018long} tracking dataset, as in~\!\cite{Lai19}. OxUvA comprises 366 video sequences with more than 1.5 million frames in total. We train our model with  SGD optimizer. For our bootstrapping based iterative training, two iterations are used and each takes about 8 hours

\begin{table*}[t]
	\centering
	\begin{threeparttable}
		\resizebox{0.99\textwidth}{!}{
			\setlength\tabcolsep{6pt}
			\renewcommand\arraystretch{1.0}
			\begin{tabular}{lc||cccccc||ccc||cccccc}
				\hline\thickhline
				\rowcolor{mygray}
				\multicolumn{2}{c||}{Supervision}&\multicolumn{6}{c||}{Non Learning} & \multicolumn{3}{c||}{Unsupervised Learning} & \multicolumn{2}{c}{Weakly-supervised}\\
				\rowcolor{mygray}
				\multicolumn{2}{c||}{Method}&TRC~\!\cite{DBLP:conf/cvpr/FragkiadakiZS12}&CVOS~\!\cite{DBLP:conf/cvpr/TaylorKS15}&KEY~\!\cite{lee2011key} &MSG~\!\cite{DBLP:conf/iccv/OchsB11} &NLC~\!\cite{DBLP:conf/bmvc/FaktorI14} &FST~\!\cite{DBLP:conf/iccv/PapazoglouF13}  &Motion Masks~\!\cite{pathak2017learning} &TSN~\!\cite{croitoru2017unsupervised}  &\textbf{Ours}&COSEG~\!\cite{tsai2016semantic} &\textbf{Ours}\\
				\hline
				\hline
				\multirow{3}{*}{ $\mathcal{J}$ } & Mean~ $\uparrow$ & 47.3&48.2   &  49.8& 53.3& 55.1 & \textbf{55.8} &{48.9}&31.2&\textbf{58.0}& 52.8 & \textbf{61.2}\\
				& Recall $\uparrow$  & 49.3&54.0& 59.1& 61.6& 55.8 &\textbf{64.7} &44.7 &18.7 &\textbf{65.3} &50.0 & \textbf{65.9}  \\
				& Decay~$\downarrow$    & 8.3&10.5&14.1& 2.4 & 12.6 &\textbf{0.0} &19.2& \textbf{-0.4}&{2.0} &\textbf{10.7} &11.6 \\\hline
				\multirow{3}{*}{ $\mathcal{F}$ } & Mean~ $\uparrow$ &44.1 & 44.7 &  42.7 & 50.8 & \textbf{52.3}  & 51.1   &39.1 & 18.4 &  \textbf{51.5}&49.3 &   \textbf{56.1} \\
				& Recall $\uparrow$  &43.6&52.6& 37.5&60.0 &  \textbf{51.9}&51.6&28.6 &5.6 &\textbf{53.2} &52.7 & \textbf{54.6} \\
				& Decay~$\downarrow$     &12.9& 11.7&10.6 & 5.1& 11.4 &\textbf{2.9}&17.9&\textbf{1.9}	&2.1& \textbf{10.5} & 20.3\\ \hline
				$\mathcal{T}$  & Mean~ $\downarrow$   & {39.1}& \textbf{25.0}& 26.9 &30.1  &42.5& 36.6 &36.4& 37.5& \textbf{30.1} & 28.2&\textbf{23.6}\\
				\hline
			\end{tabular}
		}
	\end{threeparttable}
	\vspace*{-8pt}
	\captionsetup{font=small}
	\caption{\small \textbf{Evaluation of object-level Z-VOS on DAVIS$_{16}$ \texttt{val} set~\!\cite{perazzi2016benchmark} (~\!\S\ref{sec:OZVOS})}, with region similarity $\mathcal{J}$, boundary accuracy $\mathcal{F}$ and time stability $\mathcal{T}$.
		(The best scores in each supervision setting are marked in \textbf{bold}. These notes are the same to other tables.) }
	\label{OZVOSdavis16}
	\vspace*{-8pt}
\end{table*}

\begin{table*}[t]
	\centering
	\begin{threeparttable}
		\resizebox{0.99\textwidth}{!}{
			\setlength\tabcolsep{5pt}
			\renewcommand\arraystretch{1.0}
			\begin{tabular}{lc||cccc||ccc||cccccc}
				\hline\thickhline
				\rowcolor{mygray}
				\multicolumn{2}{c||}{Supervision}&\multicolumn{4}{c||}{Non Learning} & \multicolumn{3}{c||}{Unsupervised Learning} & \multicolumn{4}{c}{Weakly-supervised Learning}\\
				\rowcolor{mygray}
				\multicolumn{2}{c||}{Method}&CRANE~\!\cite{Tang_2013_CVPR} &NLC~\!\cite{DBLP:conf/bmvc/FaktorI14}&FST~\!\cite{DBLP:conf/iccv/PapazoglouF13} &ARP~\!\cite{DBLP:conf/cvpr/KohK17}   &Motion Masks~\!\cite{pathak2017learning} &TSN~\!\cite{croitoru2017unsupervised}  &\textbf{Ours}&SOSD~\!\cite{zhang2015semantic}&BBF~\!\cite{Fatemeh2017semantic}&COSEG~\!\cite{tsai2016semantic} &\textbf{Ours}\\
				\hline
				\hline
				{ $\mathcal{J}$ } Mean~$\uparrow$& & 23.9&27.7 &  \textbf{53.8}& 46.2& 32.1& 52.2 & \textbf{57.7} &54.1&53.3&58.1& \textbf{62.4}\\

				\hline
			\end{tabular}
		}
	\end{threeparttable}
	\vspace*{-8pt}
	\captionsetup{font=small}
	\caption{\small \textbf{Evaluation of object-level Z-VOS on Youtube-Objects~\!\cite{prest2012learning} (\S\ref{sec:OZVOS})}, with mean $\mathcal{J}$. See the supplementary for more details.	}
	\label{OZVOSYoutube}
	\vspace*{-16pt}
\end{table*}

\noindent\textbf{Configuration and Reproducibility.}  MuG is implemented on PyTorch. All experiments are conducted on an Nvidia TITAN Xp GPU and an Intel (R) Xeon E5 CPU. All our implementations, trained models, and segmentation results will be released to provide the full details of our approach.

\vspace{-3pt}
\subsection{Diagnostic Experiments}
\label{sec:ablation}
\vspace{-1pt}
A series of ablation studies are performed for assessing the effectiveness of each essential component of MuG.

\noindent\textbf{Initial Fore-Background Knowledge.}
Baselines \textit{Heuristic Saliency} and \textit{CAM} give the scores of initial fore-background knowledge, based on their CRF-binarized outputs. As seen, with the low-quality initial knowledge, our MuG gains huge performance improvements ($+20.8\%$ and $+15.9\%$ promotions), showing the significance of our multi-granularity video object pattern learning scheme.

\noindent\textbf{Multi-Granularity Analysis.}
Next we  investigate the contributions of multi-granularity cues in depth. As shown in Table~\!\ref{table:abl}, the intrinsic, multi-granularity properties are indeed meaningful, as disabling any granularity analysis component causes performance to erode. For instance, removing the frame granularity analysis during learning hurts performance (mean $\mathcal{J}$: $\!58.0\!\rightarrow\!40.2$, $\!61.2\!\rightarrow\!40.2$), due to the lack of fore-/background information. Similarly, performance drops when excluding short- or long-term granularity analysis, suggesting the importance of capturing local consistency and semantic correspondence. Moreover, considering video granularity information also improves the final performance, proving the meaning of comprehensive video content understanding in video object pattern modeling.

\noindent\textbf{Iterative Training Strategy.}
From Table~\!\ref{table:abl}, we can see that with more iterations of our bootstrapping training strategy
($1\!\rightarrow\!2$), better performance can be obtained. However, further iterations ($2\!\rightarrow\!4$) give only marginal performance change. We thus use two iterations in all the experiments.

\noindent\textbf{More Training Data.} To show the potential of our unsupervised/weakly supervised VOS learning scheme, we probe the upper bound by training on additional videos. With more training data (1400 videos) from LaSOT dataset~\!\cite{Fan_2019_CVPR}, performance boosts can be observed in both two settings

\vspace{-2pt}
\subsection{Performance for Object-Level Z-VOS}
\label{sec:OZVOS}
\vspace{-1pt}
\noindent\textbf{Datasets.} Experiments are conducted on two famous Z-VOS datasets: DAVIS~\!\cite{perazzi2016benchmark} and Youtube-Objects~\!\cite{prest2012learning}, which have pixel-wise, object-level annotations. DAVIS$_{16}$ has 50 videos (3,455 frames), covering a wide range of challenges, such as fast motion, occlusion, dynamic background, \etc. It is split into a \texttt{train} set (30 videos) and a \texttt{val} set (20 videos).
Youtube-Objects contains 126 video sequences that belong to 10 categories (such as \textit{cat}, \textit{dog}, \etc) and has 25,673 frames in total. The \texttt{val} set of DAVIS$_{16}$ and whole Youtube-Objects are used for evaluation.

\noindent\textbf{Evaluation Criteria.}  For fair comparison, we follow the official
evaluation protocols of each dataset. For DAVIS$_{16}$, we report region similarity $\mathcal{J}$, boundary accuracy $\mathcal{F}$ and time stability $\mathcal{T}$.  For Youtube-Objects,  the performance is evaluated in terms of region similarity $\mathcal{J}$.

\noindent\textbf{Post-processing.} Following the common protocol in this area~\!\cite{DBLP:conf/iccv/TokmakovAS17,Song_2018_ECCV,cheng2017segflow},  the final segmentation results are optimized by CRF~\!\cite{krahenbuhl2011efficient}  (about 0.3s per frame).

\noindent\textbf{Quantitative Results.} 	Table~\!\ref{OZVOSdavis16} presents the comparison results with several  non-learning, unsupervised or weakly supervised learning competitors in DAVIS$_{16}$ dataset. In particular, MuG exceeds current leading unsupervised learning-based methods (\ie, Motion Masks~\!\cite{pathak2017learning} and TSN~\!\cite{croitoru2017unsupervised} ) in large margins (58.0 \textit{vs} 48.9 and 58.0 \textit{vs} 31.2). MuG also outperforms classical weakly-supervised Z-VOS method COSEG~\!\cite{tsai2016semantic}, and all the previous heuristic methods.  Table~\!\ref{OZVOSYoutube} summarizes comparison results on Youtube-Objects dataset, showing again our superior performance in both unsupervised and weakly supervised learning settings.

\noindent\textbf{Runtime Comparison.} The inference time of MuG is about 0.6s per frame, which is faster than most deep learning based competitors (\eg, MotionMask~\!\cite{pathak2017learning} (1.1s), TSN~\!\cite{croitoru2017unsupervised} (0.9s)). This is because, except CRF~\!\cite{krahenbuhl2011efficient}, there is no other pre-/post-processing step (\eg, superpixel~\!\cite{tsai2016semantic}, optical flow~\!\cite{DBLP:conf/iccv/PapazoglouF13}, \etc) and online fine-tuning~\!\cite{DBLP:conf/cvpr/KohK17}.

\begin{table}[t]
	\centering
	\vspace*{5pt}
	\begin{threeparttable}
		\resizebox{0.49\textwidth}{!}{
			\setlength\tabcolsep{4pt}
			\renewcommand\arraystretch{1.0}
			\begin{tabular}{cc||ccc||cc||cc}
				\hline\thickhline
				\rowcolor{mygray}
				\multicolumn{2}{c||}{Supervision}&\multicolumn{3}{c||}{Fully Supervised} & \multicolumn{2}{c||}{Unsupervised} & \multicolumn{2}{c}{Weakly-super.}\\
				\rowcolor{mygray}
				\multicolumn{2}{c||}{}&AGS&{PDB}& RVOS &&&&\\
			\rowcolor{mygray}
				\multicolumn{2}{c||}{\multirow{-2}{*}{Method}}& ~\!\cite{wang2019learning} &~\!\cite{Song_2018_ECCV} & ~\!\cite{ventura2019rvos} &  \multirow{-2}{*}{\textbf{Ours*}}& \multirow{-2}{*}{\textbf{Ours}} &\multirow{-2}{*}{\textbf{Ours*}}& \multirow{-2}{*}{\textbf{Ours}} \\
				\hline
				\hline
				$\mathcal{J}\&\mathcal{F}$ & Mean~ $\uparrow$  & 45.6& 40.4&22.5&36.5 &37.3& 40.6&41.7 \\
				\hline
				\multirow{3}{*}{ $\mathcal{J}$ } & Mean~ $\uparrow$    &  42.1 & 37.7& 17.7& 33.8&35.0& 37.7&38.9  \\
				& Recall $\uparrow$   & 48.5& 42.6& 16.2&38.2 &39.3&42.5&44.3  \\

				& Decay~$\downarrow$    & 2.6& 4.0 & 1.6&2.1 &3.8&1.9 &2.7  \\\hline
				\multirow{3}{*}{ $\mathcal{F}$ } & Mean~ $\uparrow$     &49.0 & 43.0 & 27.3&38.0&39.6&43.5&44.5 \\
				& Recall $\uparrow$  & 51.5&44.6& 24.8 & 38.6 &41.1 &44.9&46.6\\
				& Decay~$\downarrow$     &2.6 & 3.7 &1.8& 3.2&4.6&1.0&1.7  \\
				\hline
			\end{tabular}
		}
	\end{threeparttable}
	\captionsetup{font=small}
	\vspace*{-8pt}
	\caption{\small \textbf{Evaluation of instance-level Z-VOS on DAVIS$_{17}$ \texttt{test-dev} set~\!\cite{Caelles_arXiv_2019} (\S\ref{sec:IZVOS})}, $*$ denotes purely unsupervised/weakly-supervised protocol with non-learning Edgebox~\cite{ZitnickECCV14edgeBoxes} and GrabCut
	}
	\label{IZVOSdavis17}
	\vspace*{-13pt}
\end{table}

\begin{table*}[t]
	\centering
	\begin{threeparttable}
		\resizebox{0.99\textwidth}{!}{
			\setlength\tabcolsep{2pt}
			\renewcommand\arraystretch{1.0}
			\begin{tabular}{lc||ccccc||ccccc||ccc}
				\hline\thickhline
				\rowcolor{mygray}
				\multicolumn{2}{c||}{Supervision}&\multicolumn{5}{c||}{Non Learning} & \multicolumn{5}{c||}{Unsupervised Learning} & \multicolumn{2}{c}{Weakly-supervised}\\
				\rowcolor{mygray}
				\multicolumn{2}{c||}{Method}&HVS~\!\cite{DBLP:conf/cvpr/HicksonBEC14}&JMP~\!\cite{DBLP:journals/tog/FanZLCC15} &FCP~\!\cite{DBLP:conf/iccv/PerazziWGS15} &SIFT Flow~\!\cite{liu2010sift}&BVS~\!\cite{DBLP:conf/cvpr/MarkiPWS16}  &Vondrick \etal~\!\cite{vondrick2018tracking}  &mgPFF~\!\cite{kong2019multigrid} &TimeCycle~\!\cite{CVPR2019_CycleTime} &CorrFlow~\!\cite{Lai19}  &\textbf{Ours}&FlowNet2~\!\cite{DBLP:conf/cvpr/IlgMSKDB17}&\textbf{Ours}\\
				\hline
				\hline
				\multirow{3}{*}{ $\mathcal{J}$ } & Mean~ $\uparrow$ &54.6  &  57.0& 58.4&51.1 & \textbf{60.0} & 38.9&40.5 & 55.8 &{48.9}&\textbf{63.1}& 41.6&\textbf{65.7}\\
				& Recall $\uparrow$  & 61.4& 62.6& \textbf{71.5}&58.6 &66.9 & 37.1& 34.9&64.9&44.7 &\textbf{71.9}& 45.7 &\textbf{77.6}  \\
				& Decay~$\downarrow$    &23.6& 39.4&\textbf{ -2.0} &18.8& 28.9& 22.4& 18.8&\textbf{0.0}&19.2& 28.1 & \textbf{19.9}& 26.4 \\\hline
				\multirow{3}{*}{ $\mathcal{F}$ } & Mean~ $\uparrow$ & 52.9 &  53.1 & 49.2 & 44.0&\textbf{58.8} & 30.8& 34.0& 51.1  & 39.1 &\textbf{61.8}& 40.1 &\textbf{63.5}  \\
				& Recall $\uparrow$&61.0& 54.2&49.5 &50.3&\textbf{67.9}&21.7 & 24.2&51.6 &28.6 &\textbf{64.2}& 38.3& \textbf{67.7} \\
				& Decay~$\downarrow$     & 22.7 & 38.4& \textbf{-1.1} &20.0& 21.3 &16.7 &13.8 &\textbf{2.9}&17.9&30.5 &\textbf{26.6} &27.2	\\ \hline
				$\mathcal{T}$  & Mean~ $\downarrow$   & 36.0& \textbf{15.9} &30.6&16.4&34.7&45.9&53.1&36.6&\textbf{36.4}&43.0& \textbf{29.8} &44.4 \\
				\hline
			\end{tabular}
		}
	\end{threeparttable}
	\captionsetup{font=small}
	\vspace*{-8pt}
	\caption{\small \textbf{Evaluation of O-VOS on DAVIS$_{16}$ \texttt{val} set~\!\cite{perazzi2016benchmark} (\S\ref{sec:OVOS})}, with region similarity $\mathcal{J}$, boundary accuracy $\mathcal{F}$ and time stability $\mathcal{T}$.
	\label{OVOSdavis16}
	\vspace*{-8pt}
\end{table*}

\begin{table*}[t]\small
	\centering
	\begin{threeparttable}
		\resizebox{0.99\textwidth}{!}{
			\setlength\tabcolsep{5pt}
			\renewcommand\arraystretch{1.0}
			\begin{tabular}{cc||cc||ccccccc||cccc}
				\hline\thickhline
				\rowcolor{mygray}
				\rowcolor{mygray}
				\multicolumn{2}{c||}{Supervision}&\multicolumn{2}{c||}{Non Learning} & \multicolumn{7}{c||}{Unsupervised Learning} & \multicolumn{2}{c}{Weakly-supervised}\\
				\rowcolor{mygray}
				\multicolumn{2}{c||}{} &SIFT Flow&BVS&DeepCluster&Transitive Inv &Vondrick \etal &mgPFF &TimeCycle &CorrFlow & &FlowNet2& \\
				\rowcolor{mygray}
				\multicolumn{2}{c||}{\multirow{-2}{*}{Method}}&~\!\cite{liu2010sift}&~\!\cite{DBLP:conf/cvpr/MarkiPWS16}& ~\!\cite{caron2018deep}&~\!\cite{wang2017transitive}& ~\!\cite{vondrick2018tracking} &~\!\cite{kong2019multigrid} &~\!\cite{CVPR2019_CycleTime}&~\!\cite{Lai19}&\multirow{-2}{*}{\textbf{Ours}}&~\!\cite{DBLP:conf/cvpr/IlgMSKDB17}&\multirow{-2}{*}{\textbf{Ours}} \\
				\hline
				\hline
				$\mathcal{J}\&\mathcal{F}$ & Mean~ $\uparrow$  &34.0& \textbf{37.3} &35.4 &29.4&34.0& 44.6 &42.8 &50.3 &\textbf{54.3}&26.0&\textbf{56.1}\\
				\hline
				\multirow{2}{*}{ $\mathcal{J}$ } & Mean~ $\uparrow$ & \textbf{33.0}&32.9&37.5 & 32.0& 34.6& 42.2& 43.0 & 48.4&\textbf{52.6}&26.7&\textbf{54.0}  \\
				& Recall $\uparrow$   & -& 31.8 &- &-&34.1 &41.8 &43.7&53.2&\textbf{57.4}&  23.9&\textbf{60.7} \\
				\hline
				\multirow{3}{*}{ $\mathcal{F}$ } & Mean~ $\uparrow$ &35.0&\textbf{41.7}&33.2 & 26.8&32.7&46.9 &
				42.6& 52.2&\textbf{56.1}&25.2& \textbf{58.2} \\
				& Recall $\uparrow$  &-&41.4&- &-&26.8 & 44.4& 41.3
				&56.0&\textbf{58.1}&24.6& \textbf{62.2}
				\\
				\hline
			\end{tabular}
		}
	\end{threeparttable}
	\captionsetup{font=small}
	\vspace*{-8pt}
	\caption{\small \textbf{Evaluation of O-VOS on DAVIS$_{17}$ \texttt{val} set~\!\cite{pont20172017} (\S\ref{sec:OVOS})}, with region similarity $\mathcal{J}$, boundary accuracy $\mathcal{F}$ and average of $\mathcal{J}\&\mathcal{F}$.
	}
	\label{OVOSdavis17}
	\vspace*{-15pt}
\end{table*}

\vspace{-2pt}
\subsection{Performance for Instance-Level Z-VOS}
\label{sec:IZVOS}
\vspace{-1pt}
\noindent\textbf{Datasets.} We test the performance for instance-level Z-VOS on DAVIS$_{17}$~\!\cite{Caelles_arXiv_2019} dataset, which has 120 videos and 8,502 frames in total. It has three subsets, namely, \texttt{train},
\texttt{val}, and \texttt{test-dev}, containing 60, 30, and 30 video sequences, respectively. We use the ground-truth masks provided by the newest DAVIS challenge~\!\cite{Caelles_arXiv_2019}, as the original annotations are biased towards the O-VOS scenario.

\noindent\textbf{Evaluation Criteria.}
Three standard evaluation metrics, provided by DAVIS$_{17}$, are used, \ie, region similarity $\mathcal{J}$, boundary accuracy $\mathcal{F}$ and the average value of $\mathcal{T}\&\mathcal{F}$.

\noindent\textbf{Quantitative Results.} Three top-performing ZVOS methods from the DAVIS$_{17}$ benchmark are included. As shown in Table~\!\ref{IZVOSdavis17},  our model achieves comparable performance with the fully supervised methods (\ie, AGS~\!\cite{wang2019learning} and PDB~\!\cite{Song_2018_ECCV}). Notably, it significantly outperforms recent RVOS~\!\cite{ventura2019rvos}  (mean $\mathcal{T}\&\mathcal{F}$: $+14.8\%$ and $+19.2\%$ in unsupervised and weakly-supervised learning setting, respectively).

\noindent\textbf{Runtime Comparison.} The processing time for each frame is about 0.7s which is comparable to AGS~\!\cite{wang2019learning} and PDB~\!\cite{Song_2018_ECCV}, and slightly slower than RVOS~\cite{ventura2019rvos} (0.3s)

\vspace{-3pt}
\subsection{Performance for O-VOS}
\label{sec:OVOS}
\vspace{-3pt}
\noindent\textbf{Datasets.} DAVIS$_{16}$~\!\cite{perazzi2016benchmark} and DAVIS$_{17}$~\!\cite{pont20172017} datasets are used for performance evaluation under the O-VOS setting.

\noindent\textbf{Evaluation Criteria.}
Three standard evaluation criteria are reported: region similarity $\mathcal{J}$, boundary accuracy $\mathcal{F}$ and the average value of $\mathcal{T}\&\mathcal{F}$. For DAVIS$_{16}$ dataset, we further report the time stability $\mathcal{T}$.

\noindent\textbf{Quantitative Results.} Table~\!\ref{OVOSdavis16} and Table~\!\ref{OVOSdavis17} give evaluation results on DAVIS$_{16}$ and DAVIS$_{17}
, respectively. Table~\!\ref{OVOSdavis16} shows that our unsupervised method exceeds  representative self-supervised methods (\ie, TimeCyle~\!\cite{CVPR2019_CycleTime} and CorrFlow~\!\cite{CVPR2019_CycleTime}) and the best non-learning method (\ie, BVS~\!\cite{DBLP:conf/cvpr/MarkiPWS16}) across most metrics. In particular, with the learned CAM as supervision, our weakly supervised method further improves the performance, \eg, mean $\mathcal{J}$ of 65.7. Table~\ref{OVOSdavis17} verifies again our method performs favorably against the current best unsupervised method, CorrFlow, according to mean $\mathcal{T}\&\mathcal{F}$ (54.3 \textit{vs} 50.3). Note  that CorrFlow and our method use the same training data. This demonstrates our MuG is able to learn more powerful video object patterns, compared to previous self-learning counterparts.

\begin{figure}[t]
	\centering
	\includegraphics[width=.48\textwidth]{figs/visualization.pdf}
	\vspace*{-18pt}
	\captionsetup{font=small}
	\caption{\small Visual results on three videos (top: \textit{blackswan}, middle: \textit{tram}, bottom: \textit{scooter-black}) under object-level Z-VOS, instance-level Z-VOS and O-VOS setting,  respectively (see \!\S\ref{sec:visualization}). For \textit{scooter-black}, its first-frame annotation is also depicted.}
	\label{fig:visualization}
	\vspace*{-18pt}
\end{figure}

\noindent\textbf{Runtime Comparison.}
In instance-level Z-VOS setting, MuG runs about 0.4s per frame. This is faster than matching based methods (\eg, SIFT Flow~\!\cite{liu2010sift} (5.1s) and mgPFF~\!\cite{kong2019multigrid} (1.3s)), and favorably against self-supervised learning methods, \eg, TimeCycle~\!\cite{CVPR2019_CycleTime} and CorrFlow~\!\cite{Lai19}.
\vspace{-3pt}
\subsection{Qualitative Results}
\label{sec:visualization}
\vspace{-2pt}
Fig.~\!\ref{fig:visualization} presents some visual results for object-level ZVOS (top row), instance-level Z-VOS (middle row) and O-VOS (bottom row).
For \textit{blackswan} in DAVIS$_{16}$~\!\cite{perazzi2016benchmark}, the primary objects undergo  view changes and background clutter, but our MuG still generates accurate foreground segments. The effectiveness of instance-level Z-VOS can be observed in \textit{tram} of DAVIS$_{17}$~\!\cite{Caelles_arXiv_2019}. In addition, MuG can produce high-quality results with the given first-frame annotations in O-VOS setting (see the results on the last row for \textit{scooter-black} in DAVIS$_{17}$~\!\cite{pont20172017}), although the different instances suffer from fast motion and scale variation. More results can be found in supplementary materials.

\vspace{-4pt}
\section{Conclusion}
\vspace{-4pt}
We proposed MuG -- an end-to-end trainable, unsupervised/weakly supervised learning approach for segmenting objects from the videos. Different from current popular supervised VOS solutions  requiring extensive amounts of elaborately annotated training samples, our MuG models video object patterns by comprehensively exploring supervision signals from different granularities of unlabeled videos. Our model sets new state-of-the-arts over diverse VOS settings, including object-level Z-VOS, instance-level Z-VOS, and O-VOS. Our model opens up the probability of learning VOS from nearly infinite amount of unlabeled videos and unifying different VOS settings from a single view of video object pattern understanding.

{\small
	\bibliographystyle{ieee_fullname}

redpajama_set_name: RedPajamaArXiv


Crisp antique circa 1880 Victorian wire bustle with original ties and buckle. Label inside: "Taylor's Flexo * Woven Wire Bustle * Patented March 20 1880 (?) * Size No 2". This bustle was used to create the angular 1880s bustle silhouette. Excellent condition. Minimal signs of use and age.
Measurements: ~ 5" x 8 3/4"


text:
Home > Videos > Sinead O'Connor
Arsenio Hall Reveals The TRUTH About Sinead O'Connor's Shocking Prince Claims
After Sinead O'Connor accused Arsenio Hall of providing the late Prince with drugs, insinuating a connection to the icon's sudden death, the comedian fired back, revealing the truth and dropping a $5 million lawsuit!
Last year, Hall, 61, took legal action against O'Connor, who not only suggested he supplied Prince with narcotics, but even "spiked her drink" during a party at Eddie Murphy's house years prior.
Calling her a "desperate attention-seeker" spewing "despicable, fabricated lies," Hall's attorneys claimed that "as a direct and proximate result" of her conduct, Hall had suffered at least five million dollars in damages.
On Wednesday, O'Connor sheepishly apologized for her allegations via Facebook.
"I apologize for my Facebook posts about Arsenio Hall to the extent that anyone thought I was accusing him of acting as Prince's drug dealer and supplying him with illegal hard drugs, or insinuating that Arsenio had something to do with Prince's death," she said.
Sinead O'Connor
Sinead O'Connor Hospitalized Less Than One Week After Son's Death, 'I've Decided To Follow My Son'
Sinead O'Connor Claims Teenage Son Is Missing: 'We Are Frantic'
'They Are Disgusting!' Sinead O'Connor Says She Despises White People In Bizarre Rant
"I sincerely apologize because those statements would be false, and I retract them unequivocally," O'Connor added.
As RadarOnline.com readers know, this isn't the "Nothing Compares 2 U" singer's first public feud, and likely not the last either. Last June, shortly after Hall's lawsuit, O'Connor even possibly threatened to commit suicide.
The troubled songstress had a tumultuous relationship with Prince over the years, whereas Hall argued he had minimal contact with the pop icon.
Hall's lawyers plan to dismiss the lawsuit following O'Connor's official apology.
We pay for juicy info! Do you have a story for RadarOnline.com? Email us at tips@radaronline.com, or call us at (866) ON-RADAR (667-2327) any time, day or night.
Kourtney's Baby Cover-Up Foiled! See Her MAJOR Bump
Christina El Moussa Attends Church With Daughter Solo After Tarek Files Joint Custody


Vincent Chase Evil Eye Black Women's Eyeglasses don a bewitching frame style which will bring compliments wherever you go. Spring hinges impart better mobility and flexibility and helps in the initial adjustment of the frame. These impart unparalleled zeal, confidence and a feminine appeal to your personality. Its cat-eye frame shape is ideal to be worn with both formal and informal attires.
Its sleek appeal brings more attention on the eyes making the frame look like new for a long-time. This style is donned by Rekha, Sonam Kapoor, Katy Perry, Adele and many other celebs. Get the similar feminine zeal like these leading ladies in your own cat-eye frames. Get hands on these Violet cat-eye eyeglasses and turn your rivals green with envy!
Evil Eye collection is designed for people who are fashionable and love simplicity at the same time. Throw away your regular, clich© looking spectacles and switch to Evil Eye eyeglasses to uplift your zeal and style quotient.