text:
Posted on March 26, 2015 March 26, 2015 by John Beckwith basketball, degenerate, DiRTy DiMEs, draftkings, Entertainment, Fanduel, fantasy, gambler, Games, how to, lineups, make money, money, nba, optimizer, projections, Rankings, review, rotocanon, sports
DiRTy Plays – ThREesDay
***No Games on FanDuel tonight so we have posted a list of new info for Draftkings on our daily DiRTy Plays just click here to be taken right there.
Shots Up
To recap Mega Slam Wednesday, it can be said in one word, Thriller! There were 26 teams in action last night and there was no shortage of close games, game winning attempts, blowouts, playoffs implications and injuries. It makes you wonder why we sit here on Thursday morning and there is only one game of action tonight. For all of you avid NBA Fantasy players you get the day off today. I don't know about you but I was surely disappointed to find out that Milwaukee and Indiana is the only game I will get to see tonight. Especially considering the games that went down last night. We started off the night with the Bulls and Raptors and the Bulls showed why they are still in the hunt in the Eastern Conference and why the Raptors are looking at another first round exit. The Raptors have been reeling of late and given the fact that Kyle Lowry can't seem to stay on the court over a bruised back, it doesn't surprise me that Raptors find themselves falling hard down the stretch. The Bulls however are starting to get healthy and with Derrick Rose seemingly ready to return to action in two weeks, they are a team to look out for moving forward. We quickly moved to the Miami Heat game and I fully expected them to tank that game given the fact that Dwayne Wade was (resting) and Whiteside had to get ten stitches in his hand the night before. They unpredictably came out on fire and got huge contributions from Haslem and Dragic. While they did let the game get close they were able to pull it out and remain on top of Boston, there was a moment with about three minutes left that I thought, are they going to lose on a buzzer beater again! The Atlanta Hawks got back on track with a victory against the Magic last night. For all you fantasy players, this was the game to watch. If you had Millsap, Payton and Korver, then you kicked off the night with a great start. If you added Carroll, Harris, Teague and Vucevic then you were sitting around 275 on Fan Duel with two guys to play. George Hill hit a shot late to secure victory over the Wizards, who just like the Raptors, are looking at falling into the playoffs. What happened to this team that started out so hot? The only thing I can tell you is I believe it all comes down to Gortat for the Wiz. I wrote an article three weeks ago, highlighting Gortat, where I was able to determine that if he plays 30 minutes in the game then the Wiz are 19 – 3. Unfortunately, it appears that they haven't figured this out just yet and so he continues to play around 28 minutes and they continue to lose. Some other thrilling games were the Nets against the Hornets and I can't believe I am saying this, the Lakers versus the Timberwolves. Brook Lopez continued his fantasy hot streak as he lit up the Hornets for 34 points and 10 rebounds, accumulating 54+ fantasy points for only the second time all year. The Lakers pulled out an overtime victory against Minnesota and the star of the game was Chase Budinger. If you follow our DiRTy Plays then you would have known, he was one of our key Sleepers last night and he did not disappoint. What I took away from these games is that if Thad Young plays, Lopez is unstoppable, if Young misses time as he got injured last night, Lopez will be an over priced center for your daily lineup. As for the Lakers and T-Wolves, what can you say, the T-Wolves give it all they have each night. The Lakers are quietly tanking in hopes of top 3 draft pick this summer to make one last run with Kobe. Nothing would please me more than to see the Lakers add Kevin Love this summer, get Randle back and add a strong small forward to the mix, who can shoot threes. You keep Lin and Clarkson as your point guards and now the Lakers might compete for the 7th or 8th seed next year. We finished the night with Rockets holding off a late surge by the Pelicans, 95 – 93. What was crazy about this game was the fact that the Pelicans were up 16 early in the first half, only to relinquish it in the second half. They were down 12 when Anthony Davis jumped up and stole a full court lob. As he came down, he tapped shoulders with Cory Brewer and went to the ground. I immediately thought to myself, there he goes again, he can't stay on the court and will he be another tragedy of big men who can't stay healthy. He did return and almost lead a come back but clearly his shoulder is going to require surgery. I welcome the thought of an NBA player, playing through the pain, attempting to make the playoffs. We just don't see that these days and instead we get a bunch of guys complaining about chartered flights, back to back games and an otherwise grueling schedule #INeedRest. Don't get me started on that again but thank you Anthony, you put a glimmer of hope back into my mind that not everyone is as soft as these guys are today. That said, I fully expect Davis to require surgery this summer and hope that he will be back at it next season because the Pelicans are not going to make the playoffs. The last game was the Spurs and Thunder. A marquee matchup, highlighted by ESPN, we got what the Thunder really are, HOPELESS! They got blown off the court and no MVP that I can remember every took a beat down like that so good-bye to this notion that Westbrook deserves to be in the MVP race. No one put Kevin Love in the MVP race last year, while he put up 26 points and 11 rebounds per game, every night. The reason for this is simple, they didn't WIN and neither are the Thunder. Good thing there is only 10 games left and they have a three game lead.
It's On The Way
Let's have a quick breakdown of the NBA playoff picture and highlight some key games coming up over the next few days. In the East we have a crazy race to the 8th spot. It's going to come down to who can win games against each other down the stretch. Boston, Indiana, Charlotte and Brooklyn will probably all finish 8 or 9 games under .500 and yet they will be rewarded with a first round playoff birth against the Atlanta Hawks. Most might think that they would get blown off the court but come playoff time a team that I think can beat the Hawks is the Indiana Pacers. It appears that Paul George is going to return here shortly and if the Pacers can squeak in, while George gets game ready, they would be a team that has the ability to compete with the Hawks. That said only one game separates these four teams, with the Heat and Bucks, only two and four games ahead. If one of those teams hits a cold spell or catches the injury plague then we might get two of these bottom feeders in. It does however, appear that we are going to get three teams under .500 to make the playoffs in the East and that makes me sick!
The Western Conference is an entirely different situation. If you watch or listen to any sports radio or talk shows on ESPN, they are constantly talking about the Thunder and Warriors potential matchup. I certainly hope that last night's beat down of the Thunder will quiet these folks. I don't care how well Westbrook appears to be playing. If you actually watch his games, like I do, then you would quickly see that he loses just as many games for his team as he wins. Yes, he is playing lights out when you look at the stat sheet and for guys like me that are constantly after fantasy value, he is your guy. If you want to win games in the West, you cannot afford to have a turnover machine and a guy that jacks up 15 foot jumpshots with a one on four break. That team NEEDS Durant if they are going to have any shot and that has seemingly been shot down. Look for the Suns to make a push down the stretch and squeak in this year and leave three All-Stars watching from home. It's not that I don't like the Pelicans chances but as I continue to say, Davis is hurt, he is giving it everything he can but he is not enough. Tyreke Evans is not a point guard and until the Pelicans have one that can actually take the court. I do believe that we will have ten, yes count them, ten teams that go over .500 in the West.
Teams like the Jazz, who have been playing very well during the back half of the season are going to finish a few games under .500 for the season with a bright future. The talks of just taking the top sixteen teams to the NBA playoffs in the future is something that I embrace. Embrace change and think outside of the box. It is visionary people that lead our country, fortune 500 companies and I would like to think lead our major sports. Do we really need to watch teams that can barely beat the Nuggets on a night in a night basis for a 7 game series? I say NO! Make this change while at the same time rewarding the team that makes the last spot in the draft with a top 5 lottery pick. You reward the teams that want to win and take away from the teams that seemingly want ten centers on their team, Philadelphia. I can't even tell any of your players apart with the exception of Noel and Covington. Noel is your big man and Covington should be your shooting guard. The rest should be sent to the D-League along with your management team for building a team of centers in a league full of guards.
Score It
Let's breakdown how we are doing in our daily NBA Fantasy leagues. It has been a journey but after two months of full-time, NBA work, we are proud to produce a $2,840 profit on Fan Duel and almost $700 on DraftKings. We have finalized our lineup analyzer, salary breakdown analysis and highlighted Must Haves, Sleepers and Notable players for each night. While we do highlight some of these features, for one day only, I will be providing you the full breath of this offering tomorrow. We get a return of 26 teams into action for a Friday night frenzy. I will show you all the science and math it takes in order to truly be profitable on these sites, while providing you the end result you are after, winning. Come back tomorrow and check out my DiRTy Plays!
Fan Duel Earnings:
Started with $100 deposit on January 16th, 2015.
Earnings through March 25th, 2015 are $2,840
Profit for the week of March 23rd is a total of $223.50 as we have looked to increase our overall number of entries and entry fee contests to validate results on all levels of players.
Fan Duel Top Tournament Results:
We post results on Thursday!
Get in now while we are running our promotion these slots are filling up quickly and be sure to follow us on Twitter @TheDiRTCanon.
DraftKings Earnings:
Started with a $50 deposit on February 15th, 2015.
Earnings through March 25th, 2015 are $693.57
Profit for the week of March 23rd is $87.00
Results from Draftkings
We post results on Thursday
***There are no games today so why not try your hand at the NCAA***
Just click on the one of the links above and you will be directed to these contests and you can start making a profit.
#Fanduel #heat #ManInTheIronMask #nuggets #skynet absolute absolute must have absolute notable absolute sleeper basketball bball cavs conferene degenerate dollar bills DraftKings eastern espn fanduel fantasy flying frenchman gamble gambler gambling go home gobert how to information irving jazz Johnny Football kentucky kobe lakers lebron james make money Mardi Gras Michael Jordan money must haves nba nba basketball nfl northmen notables okc playoffs profit rudy scored scoring sleepers solutions standings thunder utah western


I have some exciting news to share with you, which you might have guessed already if you've been following the instagram or twitter - I am the UK ambassador of Olympus as a part of the PEN Generation. I feel so extremely honoured, as my childhood memories go way back with them having one of their film cameras document my childhood, and me now shooting with the same film camera here in London. Another thing is, that I've now been playing around with the PEN for a little while, and I'm so thrilled to be representing something as an ambassador, that I'm genuinely excited by!
I've been documenting my fashion week here in London with the PEN, and if there's something I particularly like about fashion, it's those details that make the difference. They catch your eye, and I'm not talking about the unidentified stain on my jummies, but the power of accessories just reels you in and have free space to play during LFW!
I'M SO EXCITED TO SHOW YOU THE REST OF MY PHOTOS LATER THIS WEEK!
Great photos, I love streetstyle!
Looks like so much fun! Great photos!
if it's the PEN camera you used for these shots, it really takes stunning, vibrant pictures!!!
These wine tones shoes are fantastic. The shape and color both feel like they from several centuries ago. Amazing.
You made some amazing pictures with your Pen camera. So cool you were the ambassador!
Great Photos :D & OMG to that jacket on the last photo- wonder where it`s from. Do you know? It`s amazing!!


Well I'd like to tell you that since my last posting here I've been living the life of Riley, when Riley was enjoying a particularly hedonstic period in his fun-packed life, but alas chronic fatigue syndrome can kinda put the kybosh on general fun and larks. Fortunately I'm back on my feet now, just in time for the work grindstone to pull my not inconsiderble nose towards it with harpie-like efficiency.
On Monday I did my first talk about Attention All Shipping for a while, to the massed ranks of employees at the RNLI's frankly splendid Lifeboat College in Poole. It was a lunchtime event, and probably one of the best I've done in quite a while as it turned out. They laughed in all the right places, luckily for my crippling feelings of self-doubt.
In addition, the fine folk at Time Warner are now starting to openly wonder about what the next book will be, so it's time to get the thinking cap on. Yikes.
In the meantime, a couple of sneak preview pics from the Elvis book. Here I am with Ontario's Elvis Priestley, singing an a cappella version of Blue Moon of Kentucky, while here I'm in Norway meeting the Norwegian Elvis Kjell Bjornestad. Giving my startling photogenicity, these are pictures that will surely whet your appetite for the book itself, eh? Eh..?
The eagle-eyed amongst you may have noticed there's been no North Atlantic Drifter newsletter this month. There are lots of reasons for this, not least the rather time-consuming matter of writing more than 92,000 words in a vaguely understandable and agreeable manner. However, the June edition will be a bumper summer special, just like the Beezer used to produce for the holiday season. Rest assured though, there will be no pictures of me in swimming trunks. If you've not done so already, sign up for the free, allegedly monthly newsletter by clicking here. Well, you start by clicking there, there's a small bit of typing involved. But not much.


text:
25 Seasons 25 Games
Wednesday, Apr 10, 2019 09:26 AM
O-Zone: Minor detail
John Oehser
JACKSONVILLE – Let's get to it …
John from Cape May Courthouse, NJ
It's fair to say that new Jaguars quarterback Nick Foles is a big question mark going into this season. Nobody knows whether we are going to get Super Bowl MVP Foles or St. Louis Rams Foles. Hell, we don't even know if we're going to get 16 games out of Foles. So, to me, quarterback still needs to be addressed. If Kyler Murray or Dwayne Haskins are available at No. 7, we have to take one of them. Besides the economic benefits of having a young, cost-controlled quarterback on your roster, who knows when you're going to be in this position again? This team has got to get the quarterback position figured out, and the more we invest in the position the better off we will be.
This concept/concern clearly isn't going away, and I will repeat what I have said multiple times in recent weeks – that I understand that those observers who want the Jaguars to take a quarterback in the Top 10 of the 2019 NFL Draft are unlikely to be swayed from that stance. But people wanting that approach simply aren't seeing how the Jaguars almost certainly see their quarterback situation. You call Foles a "big question mark" and you say, "nobody knows whether we are going to get Super Bowl MVP Foles or St. Louis Rams Foles." That's not how the Jaguars see Foles. They don't see him as a short-term fix or a stopgap. They see him as capable of being a franchise quarterback for at least four years – most likely five or six years. For that reason, I would be stunned if they select Ohio State quarterback Dwayne Haskins or Oklahoma quarterback Kyler Murray – or any other quarterback – No. 7 overall. Your last statement began: "This team has got to get the quarterback position figured out." When predicting the Jaguars' direction in the '19 draft, you must remember that the Jaguars believe they already have done so.
Chad from Orange Park, FL
John, I think people are having a hard time grasping the concept that you would take Haskins, but the Jaguars probably will not. Just sayin'.
Well, that I can't help.
Gabe from Chapel Hill, NC
Did you suggest that Jaguars defensive end Yannick Ngakoue would not get paid more than Dallas Cowboys defensive end Demarcus Lawrence because he had slightly fewer sacks? Because a wise man once (twice, several times) told me that sacks far from tell the whole story about pass rush productivity.
That wise(?) man was me, and he was right that sacks aren't always the most accurate measure of pass-rush productivity. But that doesn't mean players and teams don't use them in some form/fashion when finding a starting place in negotiations. Remember: Teams and players don't always operate logically.
Alejandro from Mexico City, Mexico
Do you think that a Jaguars training camp, preseason game or another kind of activity in Mexico could be possible in the future?
Realistically, probably not. At least not soon. Jaguars Owner Shad Khan has made clear his international interest is London, so I would expect most team efforts to focus there.
Dwayne from Jacksonville
Shouldn't you at least have given Spazman credit for getting the part about math errors correct?
Scott from Fernandina Beach, FL
Hi John: Bill Belichick, 44 years, 7 teams. Who knew we could lump him in with Norv Turner & Dom Capers?
This is an aftereffect of the aforementioned Spaz question. He apparently believes a coach changing teams often over a long career makes him less of a coach. This in turn obviously means no one in the NFL can coach. That can't possibly be true because it's always coaching in the NFL.
Josiah from Fargo, ND
I know there are a couple of us. I just wanted to say how cool it is that there are even Jags fans up here in North Dakota. DUUUUUUVAAAAAL
David from Richmond, VA
O, just not understanding why we are stockpiling right tackles, but I will leave that for another day. So, tackle, safety, tight end quarterback? Cody Kessler is a nice kid, but I think he is third-stringer material. Tanner Lee I know from watching many Nebraska games, and I was amazed he got drafted at all. Strong arm but accuracy problems, poor and slow decisions – i.e. interception machine. Sound familiar? So, who is my candidate for a Round 3-7 candidate? Tyree Jackson, University of Buffalo. He is big and raw, but he can do it all. Do you have any picks for a quarterback?
The Jaguars are stockpiling right tackles because they're signing them inexpensively with no real commitment. If they don't draft one, they can open competition with little risk. If they do draft one, they can move forward from most players involved with little consequence. As far as Jackson goes, he has potential. If the Jaguars are seeking a developmental quarterback behind Foles, he makes sense. That would surprise me.
Mike from Bridgeport, CT
As the draft gets closer, the more I like the idea of drafting a defensive lineman – particularly an edge rusher – if the Jags stay in the 7 hole. It feels like value can meet need at that spot. You can't have too many pass rushers. The Mayor of Sacksonville won't be around forever and I love the idea of moving Calais Campbell inside on obvious passing downs to push the pocket, allowing Ngakoue and a stud rookie to tee off on opposing quarterbacks. For me, depth along the D-Line is a concern and the Jags could use more talent for the rotation. Where is this defense if they lose Calais or Yannick?
Not as good as it is now.
Chuck from Ponte Vedra Beach, FL
Recently, the Jags were projected to finish last in division. Is that because our football operation is viewed as old school or due to big talent gaps?
I haven't the foggiest idea why whoever projected the Jaguars to finish last did so. Perhaps it's because the Jaguars are awful, or perhaps it's because the rest of the AFC South is good or perhaps the people doing the projecting haven't the foggiest idea of what they're doing.
Jason from Thousand Oaks, CA
With all this draft talk, I'm ready for a golf break. Who you got in the Masters this weekend? I'd like to see Rory get his head on straight but if I were a betting man my money would be on one of the Justin's.
Is Lanny playing?
Chris from Mandarin, FL
I re-watched the 1996 AFC Wild Card Game between the Jacksonville Jaguars and Buffalo Bills recently. Here is what stood out more than running back Natrone Means' and left tackle Tony Boselli's performances. On the play where Bills quarterback Jim Kelly fumbled the ball, his knee was clearly down prior to fumbling and the wrong call was made on the field. The Bills were driving for a late go-ahead score and were around midfield at the time. It was actually a pretty disappointing game to go back and watch. There were a number of highlight plays, but if not for a blown call by the refs the Jags likely would have and probably should have lost. Perception of events is usually different from reality and hindsight can be an underwhelming thing. We complained for a year that Myles Jack Wasn't Down. Hell, I did too. Stuff happens. I really wonder if the new officiating rules will keep something like this from happening again.
Really? A Jaguars fan is disappointed in one of the most memorable games in franchise history? And one of the most exciting? The Jaguars that season were a remarkably resilient team. They twice overcame seven-point leads in that game. Why couldn't they have stopped the Bills on that late drive? Why couldn't they have won in overtime? Look, the history of the NFL is littered with games that would have been different if not for a call. A controversial call on a would-be touchdown reception by wide receiver Mike Renfro in the AFC Championship Game following the 1979 season prevented Houston from tying the game late and perhaps led to the Pittsburgh Steelers winning a fourth Super Bowl. That doesn't diminish the Steelers' accomplishments of that era, just as a multitude of other missed calls don't change or diminish history. Missed calls are as much part of the game as mistakes by players. And no … the new officiating rules won't prevent missed calls from being part of the game. They might prevent some mistakes, but they won't remove all controversy. The game is played by large, fast, strong men and there always will be close plays that only judgement can decide. And where there is judgement there is disagreement. And where there is disagreement there is controversy.
Jason from St. Augustine, FL
Why haven't we acquired Marcedes Lewis?
Because he plays for the Green Bay Packers.
O-Zone Late Night: Jaguars 38, Colts 20
Senior Writer John Oehser answer's your questions
O-Zone Late Night: Falcons 24, Jaguars 12
Senior Writer John Oehser answers your questions
O-Zone: Wearing a vest
Senior writer John Oehser answers your questions.
O-Zone: "Expert" analysis
O-Zone Late Night: Jaguars 20, Raiders 16
O-Zone: Plan B
O-Zone: All good
O-Zone: Yeah, probably
O-Zone: Where are the strong?
O-Zone: Mutually beneficial
O-Zone: Facts only
O-Zone Late Night: Chargers 45, Jaguars 10

redpajama_set_name: RedPajamaCommonCrawl


August is normally a time when reactionary megadonors put politics aside to spend some quality time with their families and/or capital gains. But this is not a normal year.
As Donald Trump's poll numbers nosedived last month, conservative plutocrats spent millions trying to insulate the GOP Senate from wild garbage fires. Politico reports that two outside groups designed to elect Republican Senate candidates — the Senate Leadership Fund and One Nation — took in $42 million in August, the largest one-month haul either group has received this cycle.
At present, the GOP holds a four-seat majority in the Senate. But the 2016 map is favorable to Democrats, and Trump's deep unpopularity among college-educated and nonwhite voters exacerbates the challenges facing down-ballot Republicans in purple states. As of this writing, the New York Times gives Democrats a 53 percent chance of taking back the upper chamber, while the Princeton Election Consortium gives Team Blue a 74 percent chance of seizing Mitch McConnell's gavel.
Still, the race for the Senate remains close. And the Senate Leadership Fund has just begun its air campaign, which is poised to broadcast $60 million worth of ads in Indiana, Missouri, New Hampshire, North Carolina, Nevada, Ohio, and Pennsylvania (all of which boast competitive Senate races). If those ads reach their target voters — and Trump's numbers continue to tick upward — Democratic Senate candidates will have their work cut out for them.
For now, the single most likely seat count in next year's Senate, according to the Times, is 50 Democrats (including Democratically aligned independents) and 50 Republicans.


Alamo Heights is surrounded by beauty, character and charm. The homes you'll find here are diverse, from lovely cottages to extraordinary manors. This unique city is a home-rule municipality, and is dedicated to creating and maintaining its infrastructure and excellent city services. Alamo Heights is in a perfect location, just 4.5 miles north of downtown San Antonio and near many shopping amenities.
Find yourself close to the Alamo Quarry Market, Quarry Village, Lincoln Heights Shopping Center, HEB's Central Market, adorable specialty boutiques and more for you and your family. Alamo Heights is also known for its top-rated school district.


text:
\subsection{Anomaly Detection for Temporal Data}
\label{subsec:related_ad}
Most recent studies on anomaly detection for temporal data (e.g., multivariate time-series) utilize deep neural networks to learn the temporal normality (or regularity) from training data~\cite{ahmad2017unsupervised,hundman2018detecting,munir2018deepant}.
In particular, VAE-based models~\cite{hasan2016learning,luo2017remembering,park2018multimodal,su2019robust,xu2018unsupervised} have gained much attention because of their capability to capture the normal patterns in an unsupervised manner.
Their high-level idea is to compute the point-level anomaly scores by measuring how well a temporal context for each time point (i.e., a sliding window-based temporal input) can be reconstructed using the VAE.
They are able to localize the temporal anomalies within an input instance to some extent by the help of this point-level detection (or alert) approach.
However, they do not utilize information about the anomaly (e.g., anomaly labels) for model training, which makes it difficult to accurately identify the anomalies.
In general, as the anomalies are not limited to simple outliers or extreme values, modeling the \textit{abnormality} as well is helpful to learn useful features for discrimination between normal and anomalous time points~\cite{gornitz2013toward, pang2019deep}.

Several studies make use of anomaly labels by adopting U-Net architectures~\cite{wen2019time,zhang2018human} which are known to be effective for spatial segmentation~\cite{long2015fully,ronneberger2015u}.
However, their models need to be trained with full supervision, which means that the model training is guided by anomaly labels for every time point.
It makes the detector impractical because labeling every point in each instance or obtaining such point-level labels is infeasible or costs too much in practice.

\subsection{Weakly Supervised Temporal Segmentation}
\label{subsec:related_weakseg}
To address the limitation of fully supervised temporal segmentation, weakly supervised approaches have been actively researched for video action segmentation (or detection) tasks~\cite{chang2019d3tw, richard2018neuralnetwork, yu2019temporal, yuan2017temporal}.
They aim to learn weakly annotated data whose labels are not given for every time point (frame) in an instance (video).
To this end, dynamic program-based approaches are employed to search for the best results of segmentation over the time axis under the weak supervision, including the Viterbi algorithm~\cite{richard2018neuralnetwork} and dynamic time warping~\cite{chang2019d3tw, yu2019temporal, yuan2017temporal}.
Nevertheless, they cannot be directly applied to anomaly segmentation, because they require instance-level \textit{sequential} labels being used as rough segmentation masks, or they impose strong constraints on the occurrence of target events (actions).\footnote{They assume that each action occurs following a specific transition diagram~\cite{yu2019temporal} or occurs at most once in a single instance~\cite{yuan2017temporal}.}

Without the help of sequential labels and constraints, the multiple-instance learning (MIL) approach~\cite{sultani2018real} showed promising results in detecting anomalous events by leveraging weakly labeled videos.
It considers each video as a bag after dividing the video into a fixed number of same-length segments, then trains an anomaly detection model by using both positive (i.e., anomalous) and negative (i.e., normal) bags.
The trained model is able to predict the label of each video segment as well as a bag, thus the results can be utilized for anomaly segmentation.
However, due to its statically-segmented inputs, there exists a trade-off between accurate anomaly detection and precise segmentation with respect to the number of segments in a single bag.
The more (and shorter) segments make it harder to capture the long-term contexts within the segments, whereas the fewer (and longer) segments generate coarse-grained segmentation results which could much differ from the actual observation.

\subsection{Problem Formulation}
\label{subsubsec:problem}
The goal of temporal anomaly segmentation is to specify variable-length anomalous segments (i.e., the starting and end points) within an input instance of temporal data.
Formally, given a $D$-dimensional input instance $\mathbf{X} = [\vecx{1}, \vecx{2}, \ldots, \vecx{T}] \in \mathbb{R}^{D\times T}$ of temporal length $T$, we aim to
produce segment-level anomaly predictions $\hat{\mathbf{y}}=[\hat{y}_{t_0+1:t_1}, \hat{y}_{t_1+1:t_2}, \ldots, \hat{y}_{t_{L-1}+1:t_L}] \in \{0,1\}^{L}$ where $t_l$ denotes the end point of the $l$-th segment (i.e., $t_0=0$ and $t_L=T$).
This task can be treated similarly to the point-level anomaly detection from the perspective of its final output $\hat{\mathbf{y}}$, but differs in that it guarantees a one-to-one mapping between identified segments and anomalous events.

In this work, we focus on \textit{weakly supervised learning}, in order to bypass the challenge of obtaining densely labeled temporal data for model training.
Specifically, a training set consists of $N$ instances\footnote{Each instance is collected for the fixed period of time $T$, or obtained by splitting data streams into fixed-length temporal data.} with their instance-level binary labels, $\{(\mathbf{X}^{(1)}, y^{(1)}), (\mathbf{X}^{(2)}, y^{(2)}), \ldots, (\mathbf{X}^{(N)}, y^{(N)})\}$.
In this context, the weak supervision means that each label simply indicates whether any anomalous events are observed or not in the instance.
Only with the weakly annotated dataset, we train the model for temporal anomaly segmentation, where temporal data are given without any labels at test time.

\subsection{Dynamic Time Warping (DTW) Alignment for Temporal Anomaly Segmentation}
\label{subsec:dtwseg}
We first present how to obtain the segmentation result of an input instance by using a rough segmentation mask.
To this end, we define an additional type of anomaly labels which can serve as the segmentation mask, referred to as sequential anomaly label.\footnote{The details about synthesizing this label are discussed in Section~\ref{subsec:weaklearn}.}
This label indicates the existence (and order) of normal and anomalous events for each input instance; $\mathbf{z}=[\aslab{1},\aslab{2},\ldots,\aslab{L}]\in\{0, 1\}^L$ where $L$ is the length of the sequential label.
For example, $\mathbf{z} = [0, 1, 0, 1, 0]$ in case of $L=5$ means that two anomalous events are observed within the instance.
Note that the sequential label $\mathbf{z}$ itself cannot be the final output of our framework, because it does not contain the information about the starting and end points of each segment.

For temporal anomaly segmentation, we utilize dynamic time warping (DTW)~\cite{sakoe1978dynamic} which outputs the optimal alignment between a target instance and the sequential anomaly label.
The goal of DTW is to find the optimal alignment (i.e., point-to-point matching) between $\mathbf{z}$ and $\mathbf{X}$, which minimizes their total alignment cost with the time consistency among the aligned point pairs.
To be precise, the total alignment cost is defined by the inner product of the cost matrix $\Delta(\mathbf{z}, \mathbf{X}) \in \mathbb{R}^{L \times T}$ and the binary alignment matrix $A\in\{0,1\}^{L \times T}$.
Each entry of the cost matrix, $[\Delta(\mathbf{z}, \mathbf{X})]_{lt} := \delta(\aslab{l}, \vecx{t})$, encodes the cost for aligning $\vecx{t}$ with $\aslab{l}$ (i.e., the penalty for labeling $\vecx{t}$ as $\aslab{l}$), and similarly,
the entry of $A$ indicates the alignment between $\mathbf{z}$ and $\mathbf{X}$;
that is, $A_{lt}=1$ if $\vecx{t}$ is aligned with $\aslab{l}$ and $A_{lt}=0$ otherwise.
The optimal alignment matrix $A^*$ is obtained by
\begin{equation}
\small
    A^* = \argmin_{A\in\mathcal{A}}\ \langle A, \Delta(\mathbf{z}, \mathbf{X})\rangle,
\end{equation}
where $\mathcal{A}$ is the set of possible binary alignment matrices.

\begin{figure*}[t]
	\centering
	\includegraphics[width=\linewidth]{./FIG/overall_framework.png}
	\caption{The overall framework of WETAS\xspace, optimized by both the classification loss $\mathcal{L}_{c}$ and alignment loss $\mathcal{L}_{a}$. The red arrows in the cost matrix $\Delta(\widetilde{\mathbf{z}}, \mathbf{X})$ are the desired alignment between a temporal input (column) and its sequential label (row) with time consistencies. Based on the optimal alignment, WETAS\xspace can produce the segment-level anomaly predictions.}
	\label{fig:framework}
\end{figure*}

In order to align the two series ($\mathbf{z}$ and $\mathbf{X}$) in a temporal order, each alignment matrix is enforced to represent a single path on a $L \times T$ matrix that connects the upper-left $(1, 1)$-th entry to the lower-right $(L, T)$-th entry using $\rightarrow, \searrow$ moves.
Based on the obtained alignment $A^*$, the starting and end points of the $l$-th temporal segment and its anomaly label are determined by $\hat{y}_{t_{l-1}+1:t_l}=\aslab{l}$ such that $A^*_{lt}=1$ for all $t_{l-1}< t\leq t_l$.
For example, in Figure~\ref{fig:framework} (Left), the red arrows represent non-zero entries in the optimal alignment matrix, and we can easily identify the segments whose segment-label is 1 (i.e., $z_l=1$) as the anomaly.
Unlike the conventional DTW\xspace, $\downarrow$ moves are not available in our warping paths because each time point should be aligned with only a single label.

The two key challenges are raised here as follows.
1) We need to carefully model the cost matrix based on the anomaly score so that it can effectively capture the \textit{normality} and \textit{abnormality} at each point, while considering the long-term dependency.
In addition, 2) we need to obtain sequential labels used for the DTW alignment by distilling the information from given instance-level labels.

\subsection{Anomaly Score Modeling for Cost Matrix}
\label{subsec:model}
To model the cost function $\delta(\aslab{l}, \vecx{t})$ that is required to define the cost matrix $\Delta(\mathbf{z}, \mathbf{X})$, we parameterize the anomaly score at time $t$ (i.e., the probability that $\vecx{t}$ comes from an anomalous event) by using deep neural networks.
Among various types of autoregressive networks designed for temporal data, we employ the basic architecture of dilated CNN (DiCNN\xspace) introduced by WaveNet~\cite{oord2016wavenet}.
As illustrated in Figure~\ref{fig:framework} (Right), DiCNN\xspace is basically stacks of dilated \textit{causal} convolutions, which is the convolution operation not considering future inputs when computing the output at each point.
Specifically, the $n$-th layer has a convolution with filter size $k$ and dilation rate $k^{n-1}$, thus
the output vector from layer $n$ at time $t$ (denoted by $\vech{t}$) depends on its previous $k^n$ points in the end;
i.e., $[\vecx{t-k^n+1}, \ldots, \vecx{t}]$ becomes its receptive field.

Using the inner product of the output vector at each point $\vech{t}\in\mathbb{R}^{d}$ and the anomaly weight vector $\mathbf{w}\in\mathbb{R}^{d}$, whose parameters are trainable, we compute the series of local anomaly scores $\mathbf{s} \in \mathbb{R}^T$ by
\begin{equation}
\small
    \mathbf{s} = [\score{1}, \score{2}, \ldots, \score{T}], \text{\ where\ \  } \score{t} = \sigma(\mathbf{w}{}^\top\vech{t}).
\label{eq:lscore}
\end{equation}
We apply the sigmoid function $\sigma$ to make the score range in $[0, 1]$.
Finally, we define the cost $\delta(\aslab{l}, \vecx{t})$ by the negative log-posterior probability using the anomaly score $\score{t}$:
\begin{equation}
\small
\label{eq:distfunc}
\begin{split}
\delta(\aslab{l}, \vecx{t})
&= -\log p \left(\aslab{l}|\vecx{t} \right) \\
&= -\left\{\aslab{l}\cdot\log\score{t} + (1-\aslab{l})\cdot\log(1-\score{t}) \right\}.
\end{split}
\end{equation}

\noindent
The cost becomes large when the computed score $\score{t}$ is far from a target label $\aslab{l}$.
In this sense, DTW can identify the anomalous segments by finding the optimal point-to-point matching between the series of anomaly scores and its sequential label, which minimizes the total alignment cost.

\subsection{Learning with Weak Supervision}
\label{subsec:weaklearn}
Our proposed framework, termed as WETAS\xspace, optimizes the DiCNN\xspace model by leveraging only weak supervision for the anomaly segmentation.
To fully utilize the given instance-level labels, two different types of losses are considered:
1) the classification loss for correctly classifying an input instance as its instance-level anomaly label, and 2) the alignment loss for matching the input instance well with the sequential label, which is synthesized by the model by distilling the instance-level label.

\smallsection{Learning from Instance-level Label}
\label{subsubsec:learn_weaklabel}
Similar to the \textit{local} (or point-level) anomaly score in Equation~\eqref{eq:lscore}, we define the \textit{global} (or instance-level) anomaly score for its binary classification.
The global anomaly score $\score{*}\in\mathbb{R}$ is computed by using the anomaly weight $\mathbf{w}\in\mathbb{R}^d$ and the global output vector $\vech{*}\in\mathbb{R}^d$, obtained by global max (or average) pooling on all the output vectors along the time axis,
\begin{equation}
\small
\begin{split}
    \label{eq:gmp}
    \vech{*} &= \text{global-pooling}\ (\vech{1}, \vech{2}, \ldots, \vech{T}) \\
    \score{*} &= \sigma(\mathbf{w}{}^\top\vech{*}).
\end{split}
\end{equation}

To optimize our model based on the instance-level labels, we define the classification loss by the binary cross entropy between $y$ and $\score{*}$.
Thereby, we can predict the anomaly label for an input instance based on its global anomaly score:
\begin{equation}
\small
\label{eq:closs}
        \mathcal{L}_{c}

        = -\sum_{i=1}^{N}\left\{y^{(i)} \cdot \log \score{*}^{(i)} + (1-y^{(i)})\cdot\log(1-\score{*}^{(i)})\right\}.
\end{equation}
The classification loss optimizes the model to differentiate between normal patterns and anomalous patterns that are commonly observed in weakly labeled temporal data.

\smallsection{Learning from Sequential Pseudo-Label}
\label{subsubsec:learn_seqlabel}
To begin with, we propose the technique to infer the sequential anomaly label (i.e., pseudo-label generation) of an input instance, then utilize the generated sequential pseudo-label to further improve our DiCNN\xspace model so that it can compute more accurate local anomaly scores.
The pseudo-labeling technique is motivated by temporal class activation map (CAM)~\cite{lee2021learnable, wang2017time} which helps to analyze temporal regions that most influence the instance-level classification.
We obtain the anomaly activation map $\mathbf{m}=[\actmap{1}, \actmap{2}, \ldots, \actmap{T}] \in \mathbb{R}^{T}$ by multiplicating each output vector $\vech{t}$ with the weight vector $\mathbf{w}$ that is used for the instance-level anomaly classification.
That is, $\actmap{t}$ becomes proportional to how strongly the time point $t$ contributes to classifying the input instance as the anomaly.
For the condition $0\leq\actmap{t}\leq1$, their values are min-max normalized along the time axis within each instance,
\begin{equation}
\small
\label{eq:actmap}
    \actmap{t} = \frac{\mathbf{w}^\top\vech{t} - \min_{t'} \mathbf{w}^\top\vech{t'}}{\max_{t'} \mathbf{w}^\top\vech{t'} - \min_{t'} \mathbf{w}^\top\vech{t'}}.
\end{equation}

Then, we introduce a pseudo-labeling function $\bm{\phi}_L:\mathbb{R}^T \rightarrow \{0, 1\}^L$ which partitions a whole anomaly activation map into $L$ segments (or disjoint intervals) of the same length $\lceil T/L \rceil$ and generates a binary label for each segment, i.e., $\widetilde{\mathbf{z}} =\bm{\phi}_L(\mathbf{m})\in\{0, 1\}^L$.
The pseudo-label $\slab{l}$ is determined by whether the maximum activation value in the $l$-th segment is larger than the anomaly threshold $\tau$ or not:
\begin{equation}
\small
\label{eq:seqlabel}
    \slab{l} = \mathbb{I}\left[\left(\max_{(l-1) \cdot \lceil T/L \rceil \leq t \leq l \cdot \lceil T/L \rceil} \actmap{t}\right) \geq \tau \right].
\end{equation}
This sequential pseudo-label $\widetilde{\mathbf{z}}$ offers information on how many anomalous segments do exist and also their relative locations.
Note that it is used as the rough segmentation mask for the DTW alignment, explained in Section~\ref{subsec:dtwseg}.

The alignment loss is designed to reduce the DTW discrepancy between an input $\mathbf{X}$ and its sequential pseudo-label $\widetilde{\mathbf{z}}$ for their better alignment.
However, the gradient of DTW is not very well defined (i.e., not differentiable) with respect to the cost matrix, due to its discontinuous hard-min operation taking only the minimum value.
Thus, we instead adopt the soft-DTW~\cite{cuturi2017soft} that provides the continuous relaxation of DTW by combining it with global alignment kernels~\cite{cuturi2007kernel}.
The soft-DTW utilizes the soft-min operation,
\begin{equation}
\small
\label{eq:softdtw}
    \sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X}) =  \text{min}_{\gamma}\left\{ \langle A, \Delta(\widetilde{\mathbf{z}}, \mathbf{X})\rangle, \forall A\in \mathcal{A} \right\}
\end{equation}
where $\text{min}_{\gamma}{}$
with a smoothing parameter $\gamma$ is defined by
\begin{equation*}
\small
\label{eq:softmin}
    \text{min}_{\gamma}\{a_1, a_2, \ldots, a_n\} =
    \begin{cases}
    \; \min_{i\leq n} a_i, &\gamma = 0\\
    \; -\gamma \log \sum_{i=1}^{n}e^{-a_i/\gamma}, &\gamma > 0.
    \end{cases}
\end{equation*}
The soft-DTW distance in Equation~\eqref{eq:softdtw} can be obtained by solving a dynamic program based on Bellman's recursions.
Please refer to~\cite{cuturi2017soft, lee2021learnable} for its forward and backward recursions to compute $\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X})$ and $\nabla_{\Delta(\widetilde{\mathbf{z}}, \mathbf{X})}\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X})$.

We remark that only minimizing $\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X})$ causes the problem of \textit{degenerated alignment}~\cite{chang2019d3tw} that assigns each label to a single time point.
To obtain more precise boundaries of segmentation results, we use the discriminative modeling under the supervision of the obtained pseudo-label.
In detail, we optimize the model so that the alignment with a positive pseudo-label $\widetilde{\mathbf{z}}_+$ costs less than that with a negative pseudo-label $\widetilde{\mathbf{z}}_-$ by a margin $\beta$ as follows.
\begin{equation}
\small
\label{eq:aloss}
    \mathcal{L}_{a} =
    \sum_{i=1}^{N} \left[\frac{1}{T}\sdtw{\gamma}(\widetilde{\mathbf{z}}_+^{(i)}, \mathbf{X}^{(i)})- \frac{1}{T}\sdtw{\gamma}(\widetilde{\mathbf{z}}_-^{(i)}, \mathbf{X}^{(i)})  +\beta \right]_{+},
\end{equation}
where $[z]_{+} = \max(z, 0)$.
The positive and negative pseudo-labels are respectively obtained by $\widetilde{\mathbf{z}}_+ = y\cdot\bm{\phi}_L(\mathbf{m})$ and $\widetilde{\mathbf{z}}_- =(1-y)\cdot\bm{\phi}_L(\mathbf{m})$, thereby the instance-level label $y$ plays the role of a binary mask.
In other words, we regard the sequential pseudo-label corrupted by the wrong instance-level label (i.e., $1-y$) as negative.
This loss is helpful to produce more accurate local anomaly scores
by making them align better with its sequential pseudo-label.
Consequently, the model is capable of exploiting richer supervision than only using the instance-level labels.

\subsection{Optimization and Inference}
\label{subsec:optimization}
The final loss is described by the sum of the two losses, $\mathcal{L}=\mathcal{L}_{c}+\mathcal{L}_{a}$, where we can control the importance of the alignment loss by adjusting the margin size $\beta$.
Note that the anomaly weight vector $\mathbf{w}$ and the network parameters in the DiCNN\xspace model are effectively optimized by minimizing the final loss, whereas $L$, $\tau$, and $\beta$ are the hyperparameters of WETAS\xspace.
Figure~\ref{fig:framework} shows the overall framework of WETAS\xspace.

The segmentation result for a test instance is obtained by the DTW alignment with its sequential pseudo-label $\widetilde{\mathbf{z}}=\hat{y}_*\cdot\bm{\phi}_L(\mathbf{m})$, where $\hat{y}_*=\mathbb{I}[\score{*} \geq \tau_{*}]$.
Since the instance-level label is not given for the test input, we instead impose the predicted label $\hat{y}_*$ as a binary mask;
that is, we filter out normal instances based on the global anomaly score $\score{*}$.
Once the model is trained, $\tau_{*}$ is automatically determined by its optimal value that achieves the best instance-level classification performance on the validation set.
\subsection{Experimental Settings}
\label{subsec:expsetting}
\smallsection{Dataset}
For our experiments, we use four real-world temporal datasets collected from a range of tasks for detecting anomalous events, including multivariate time-series (MTS) and surveillance videos (Table~\ref{tbl:dataset}).
We split the set of all the instances by 5:2:3 ratio into a training set, a validation set, and a test set.
Note that only the instance-level labels are given for the training and validation set.
\begin{itemize}

    \setlength\itemsep{-0.2em}

    \item \textbf{Electromyography Dataset\footnote{http://archive.ics.uci.edu/ml/datasets/EMG+data+for+gestures} (EMG\xspace)}~\cite{lobov2018latent}:
    The 8-channel myographic signals recorded by bracelets worn on subjects' forearms.
    Among several types of gestures, ulnar deviation is considered as anomalous events.
    Each time-series instance contains the signals for 5 seconds, which are downsampled to 500 points.

    \item \textbf{Gasoil Plant Heating Loop Dataset\footnote{https://kas.pr/ics-research/dataset\_ghl\_1} (GHL\xspace)}~\cite{filonov2016multivariate}:
    The control sequences of a gasoil plant heating loop, which suffered cyber-attacks.
    As done in~\cite{wen2019time}, we crop 10 different instances of length 50,000 from each time-series, then downsample each of them to 1,000 points.

    \item \textbf{Server Machine Dataset\footnote{https://github.com/smallcowbaby/OmniAnomaly} (SMD\xspace)}~\cite{su2019robust}:
    The 38-metric multivariate time-series from server machines over 5 weeks, collected from an internet company.
    We split them every 720 points (i.e., 12 hours) to build the set of time-series instances.

    \item \textbf{Subway Exit Dataset  (Subway\xspace)}~\cite{adam2008robust}: The surveillance video for a subway exit gate, where each anomalous event corresponds to the passenger walking towards a wrong direction.
    We extract the visual features of each frame by using the pre-trained ResNet-34~\cite{he2016deep}, and make a single video instance include 450 frames.
\end{itemize}

\smallsection{Baselines}
We compare the performance of WETAS\xspace with that of anomaly detection methods for temporal data, which are based on three different approaches.
\begin{itemize}
\setlength\itemsep{-0.2em}
    \item \textbf{Unsupervised learning}: VAE-based methods that compute the point-level anomaly scores from the reconstruction of temporal inputs --- Donut\xspace~\cite{xu2018unsupervised}, LSTM-VAE\xspace~\cite{park2018multimodal}, LSTM-NDT\xspace~\cite{hundman2018detecting}, and OmniAnomaly\xspace~\cite{su2019robust}.

    \item \textbf{Semi-supervised learning}: The variants of the unsupervised methods, whose models are trained by using only normal instances in order to make them utilize given instance-level labels\footnote{These methods are categorized as semi-supervised learning~\cite{chandola2009anomaly} in that they only utilize densely-labeled (normal) points from normal instances.} ---  Donut\textsuperscript{++}\xspace, LSTM-VAE\textsuperscript{++}\xspace, LSTM-NDT\textsuperscript{++}\xspace, and OmniAnomaly\textsuperscript{++}\xspace.

    \item \textbf{Weakly supervised learning}: The multiple-instance learning method that can produce the anomaly prediction for each fixed-length segment --- DeepMIL\xspace~\cite{sultani2018real}.
\end{itemize}

For fair comparisons, DeepMIL\xspace employs the same model architecture with WETAS\xspace (i.e., DiCNN\xspace).
Moreover, we consider different numbers of the segments in a single instance, denoted by DeepMIL\xspace-4, 8, 16.
In case of the video dataset (i.e., Subway\xspace), the additional comparison with other frame-level video anomaly detection methods~\cite{ionescu2017unmasking, ionescu2019detecting, liu2018classifier, pang2020self} is presented in the supplementary material.

\begin{table}[t]
    \caption{Statistics of real-world temporal datasets.}
    \label{tbl:dataset}
    \centering
    \resizebox{.99\linewidth}{!} {%
    \begin{tabular}{rcccc}
    \toprule
    Dataset & EMG\xspace & GHL\xspace & SMD\xspace & Subway\xspace \\ \midrule
    Data Type & MTS & MTS & MTS & Video \\
    \#Variables & 8 & 19 & 38 & 1024 \\
    \#Points (Train) & 211,892 & 240,000 & 354,200 & 32,051 \\
    \#Points (Valid) & 84,734 & 96,000 & 141,670 & 13,050 \\
    \#Points (Test) & 127,199 & 144,000 & 212,550 & 19,800 \\
    Anomaly Ratio (\%) & 5.97 & 0.49 & 4.16 & 2.60 \\
    Instance Length & 500 & 1,000 & 720 & 450 \\
    \bottomrule
    \end{tabular}
    }
\end{table}

\smallsection{Evaluation Metrics}
For quantitative evaluation of the anomaly detection (and segmentation) results, we measure the F1-score (denoted by F1) and the intersection over union (denoted by IoU) by using the point-level ground truth:
F1 = $\frac{2\times \text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}}$ where Precision = $\frac{TP}{TP+FP}$ and Recall = $\frac{TP}{TP+FN}$, and IoU = $\frac{TP}{TP+FP+FN}$.
Several previous work~\cite{park2018multimodal, sultani2018real, xu2018unsupervised} mainly reported the area under the ROC curve (AUROC) as their evaluation metric, but it is known to produce misleading results especially for severely imbalanced classification with few samples of the minority class~\cite{saito2015precision}, such as anomaly detection tasks.

To compute the precision and recall, the weakly supervised methods (i.e., DeepMIL\xspace and WETAS\xspace) can find the optimal anomaly threshold (applied to instance-level or segment-level scores) by utilizing the instance-level labels from the validation set.
However, in cases of the unsupervised and semi-supervised methods, they require point-level anomaly labels to tune the anomaly threshold (applied to point-level scores), thus we report their F1 and IoU by using the anomaly threshold that yields the best F1 among all possible thresholds (denoted by F1-best and IoU-best, respectively).
This can be interpreted as a measure of the discrimination power between normal and anomalous points.
For some of the baselines (i.e., LSTM-NDT\xspace and OmniAnomaly\xspace) that have their own techniques to automatically determine the anomaly threshold, we additionally report their F1 and IoU by using the selected threshold.

\begin{table*}[t]
\caption{Performances of WETAS\xspace and other baselines in detecting anomalous points from real-world temporal data. All the results support statistically significant improvement of WETAS\xspace over the best baseline ($p \leq 0.05$ from the paired $t$-test).}
\label{tbl:mainresults}
\centering
\resizebox{0.99\linewidth}{!}{%
\begin{tabular}{rcccccccc}
    \toprule
    \multirow{2.5}{*}{Methods} & \multicolumn{4}{c}{EMG\xspace} & \multicolumn{4}{c}{GHL\xspace}  \\ \cmidrule{2-9}
    & F1 & IoU & F1-best & IoU-best & F1 & IoU & F1-best & IoU-best \\ \midrule
    Donut\xspace~\cite{xu2018unsupervised} & - & - & 0.1748(0.002) & 0.0958(0.001) & - & - & 0.0363(0.013) & 0.0185(0.007) \\
    LSTM-VAE\xspace~\cite{park2018multimodal} & - & - & 0.1728(0.000) & 0.0946(0.000) & - & - & 0.0746(0.015) & 0.0388(0.008) \\
    LSTM-NDT\xspace~\cite{hundman2018detecting} & 0.1317(0.016) & 0.0705(0.009) & 0.1880(0.010) & 0.1199(0.001) & 0.0640(0.030) & 0.0332(0.016) & 0.1025(0.023) & 0.0541(0.013) \\
    OmniAnomaly\xspace~\cite{su2019robust} & 0.1574(0.003) & 0.0854(0.002) & 0.1793(0.001) & 0.0985(0.001) & 0.0611(0.033) & 0.0318(0.018) & 0.0743(0.026) & 0.0387(0.014)  \\
    \midrule

    Donut\textsuperscript{++}\xspace~\cite{xu2018unsupervised} & - & - & 0.1784(0.001) & 0.0980(0.001) & - & - & 0.0850(0.036) & 0.0447(0.019) \\
    LSTM-VAE\textsuperscript{++}\xspace~\cite{park2018multimodal} & - & - & 0.1745(0.000) & 0.0956(0.000) & - & - & 0.0828(0.023) & 0.0433(0.013) \\
    LSTM-NDT\textsuperscript{++}\xspace~\cite{hundman2018detecting} & 0.1344(0.005) & 0.0720(0.003) & 0.1916(0.011) & 0.1201(0.002) & 0.0984(0.091) & 0.0538(0.053) & 0.1043(0.106) & 0.0578(0.064) \\
    OmniAnomaly\textsuperscript{++}\xspace~\cite{su2019robust} & 0.1536(0.002) & 0.0832(0.001) & 0.1807(0.002) & 0.0993(0.001) & 0.1209(0.101) & 0.0668(0.058) & 0.2096(0.143) & 0.1231(0.094) \\
    \midrule

    DeepMIL\xspace-4~\cite{sultani2018real} & 0.4699(0.022) & 0.3073(0.019) & - & - & 0.0690(0.035) & 0.0359(0.098) & - & - \\
    DeepMIL\xspace-8~\cite{sultani2018real} & 0.4317(0.029) & 0.2755(0.023) & - & - & 0.0571(0.023) & 0.0323(0.015) & - & - \\
    DeepMIL\xspace-16~\cite{sultani2018real} & 0.3182(0.056) & 0.1902(0.039) & - & - & 0.1497(0.040) & 0.0813(0.023) & - & - \\
    \textbf{WETAS\xspace (ours)} & \textbf{0.5803(0.068)} & \textbf{0.4118(0.064)} & - & - & \textbf{0.2295(0.028)} & \textbf{0.1298(0.018)} & - & - \\
    \bottomrule
    \toprule
    \multirow{2.5}{*}{Methods} & \multicolumn{4}{c}{SMD\xspace} & \multicolumn{4}{c}{Subway\xspace} \\ \cmidrule{2-9}
    & F1 & IoU & F1-best & IoU-best & F1 & IoU & F1-best & IoU-best  \\ \midrule
    Donut\xspace~\cite{xu2018unsupervised} & - & - & 0.3206(0.011) & 0.1909(0.008) & - & - & 0.5080(0.018) & 0.3406(0.016) \\
    LSTM-VAE\xspace~\cite{park2018multimodal} & - & - & 0.2671(0.018) & 0.1542(0.012) & - & - & 0.5329(0.024) & 0.3635(0.022)  \\
    LSTM-NDT\xspace~\cite{hundman2018detecting} & 0.1145(0.018) & 0.0608(0.010) & 0.3588(0.019) & 0.2187(0.014) & 0.0000(0.000) & 0.0000(0.000) & 0.5658(0.005) & 0.3945(0.005) \\
    OmniAnomaly\xspace~\cite{su2019robust} & 0.1176(0.002) & 0.0625(0.001) & 0.1223(0.006) & 0.0651(0.004) & 0.5367(0.043) & 0.3704(0.038) & 0.6065(0.020) & 0.4355(0.020) \\
    \midrule

    Donut\textsuperscript{++}\xspace~\cite{xu2018unsupervised} & - & - & 0.2875(0.067) & 0.1693(0.047) & - & - & 0.5100(0.026) & 0.3449(0.021) \\
    LSTM-VAE\textsuperscript{++}\xspace~\cite{park2018multimodal} & - & - & 0.2477(0.015) & 0.1414(0.010) & - & - & 0.5452(0.016) & 0.3749(0.016) \\
    LSTM-NDT\textsuperscript{++}\xspace~\cite{hundman2018detecting} & 0.1211(0.010) & 0.0645(0.006) & 0.3819(0.020) & 0.2361(0.015) & 0.0000(0.000) & 0.0000(0.000) & 0.5723(0.004) & 0.4009(0.004) \\
    OmniAnomaly\textsuperscript{++}\xspace~\cite{su2019robust} & 0.1435(0.081) & 0.0790(0.049) & 0.1750(0.077) & 0.0974(0.047) & 0.5479(0.028) & 0.3790(0.026) & 0.6198(0.023) & 0.4494(0.024) \\
    \midrule

    DeepMIL\xspace-4~\cite{sultani2018real} & 0.3561(0.052) & 0.2176(0.038) & - & - & 0.5138(0.081) & 0.3738(0.073) & - & - \\
    DeepMIL\xspace-8~\cite{sultani2018real} & 0.3450(0.032) & 0.2088(0.023) & - & - & 0.6471(0.066) & 0.4885(0.064) & - & - \\
    DeepMIL\xspace-16~\cite{sultani2018real} & 0.3568(0.016) & 0.2173(0.012) & - & - & 0.6102(0.077) & 0.4391(0.072) & - & -\\
    \textbf{WETAS\xspace (ours)} & \textbf{0.4358(0.046)} & \textbf{0.2795(0.037)} & - & - & \textbf{0.7414(0.023)} & \textbf{0.5907(0.028)}  & - & - \\
    \bottomrule
\end{tabular}
}
\end{table*}

\begin{table}[t]
\caption{F1-scores of WETAS\xspace that ablates each component, Dataset: SMD\xspace. C and R denote the ``corruption'' and ``random sampling'' for negative pseudo-labels, respectively.}
\label{tbl:ablation}
\small
\centering
\resizebox{0.99\linewidth}{!}{%
\begin{tabular}{PccPPPP}
    \toprule
    Model Arch. & $\mathcal{L}_{c}$ & $\mathcal{L}_{a}$ & DTW Seg. & Global Pool. & Neg. Label & F1 \\\midrule
    LSTM\xspace
    & \checkmark & \checkmark & \checkmark & MAX & C & 0.3685 \\ \midrule
    \multirow{6.5}{*}{DiCNN\xspace}
    & \checkmark & - & - & MAX & C & 0.1225 \\
    & \checkmark & \checkmark & - & MAX & C & 0.1265 \\
    & \checkmark & - & \checkmark & MAX & C & 0.3384 \\ \cmidrule{2-7}
    & \checkmark & \checkmark & \checkmark & AVG & C & 0.3046 \\
    & \checkmark & \checkmark & \checkmark & MAX & R & 0.4272 \\
    & \checkmark & \checkmark & \checkmark & MAX & C & \textbf{0.4358} \\
    \bottomrule
\end{tabular}
}
\end{table}

\smallsection{Implementation Details}
We implement our WETAS\xspace and all the baselines using PyTorch, and train them with the Adam optimizer~\cite{kingma2014adam}.
For the unsupervised methods, we tune their hyperparameters in the ranges suggested by the previous work~\cite{su2019robust} that considered the same baselines.
In case of VAE\xspace-based methods, we set the size of a temporal context for each point (i.e., sliding window) to 128 (for MTS) and 16 (for video).
For DiCNN\xspace-based methods, we stack 7 (for MTS) and 4 (for video) layers of dilated convolutions with filter size 2 to keep the size of its receptive field (=$2^7$, $2^4$) the same with the others'.
The dimensionality $d$ of hidden (and output) vectors in DiCNN\xspace and the smoothing factor $\gamma$ of soft-DTW are set to 128 and 0.01, respectively.
We provide the in-depth sensitivity analysis on the hyperparameters (i.e., $L$, $\tau$, and $\beta$) in our supplementary material.

\subsection{Experimental Results}
\label{subsec:expresults}
\smallsection{Performances on Anomaly Detection}
\label{subsubsec:performance}
We first measure the detection performance of WETAS\xspace and the other baselines.
In this experiment, we do not consider the point-adjust approach~\cite{su2019robust,xu2018unsupervised} for our evaluation strategy:
if any point in a ground truth anomalous segment is detected as the anomaly, all points in the segment are considered to be correctly detected as the anomalies.\footnote{This approach skews the results by excessively increasing the true positives, i.e., the F1 and IoU are overestimated.}
We repeat to train each model five times using different random seeds, and report the averaged results with their standard deviations.

\begin{figure*}[t]
	\centering
	\includegraphics[width=\linewidth]{./FIG/qresults_smd.png}
	\vspace{-20pt}
	\caption{Anomaly detection results of WETAS\xspace and other baselines on 4 consecutive instances, Dataset: SMD\xspace. WETAS\xspace correctly detects anomalous points in terms of the number of ground truth anomalous segments and their boundaries. }
	\label{fig:qresults}
\end{figure*}

\begin{figure}[t]
	\centering
	\includegraphics[width=\linewidth]{./FIG/qresults_training.png}
	\vspace{-15pt}
	\caption{Test instances (upper) and the local anomaly scores computed by WETAS\xspace every 10 epoch (lower), Dataset: SMD\xspace. The ground truth and identified anomalous segments are colored in yellow and red, respectively.}
	\label{fig:trainingprocess}
\end{figure}

In Table~\ref{tbl:mainresults}, the unsupervised and semi-supervised methods show considerably worse performance than the weakly supervised methods in terms of both F1 and IoU.
Even considering F1-best and IoU-best, their performance is at most comparable to that of the weakly supervised methods.
In other words, though they additionally use the point-level labels for finding the best anomaly threshold that clearly distinguishes anomalous points from normal ones, they cannot achieve the detection performance as high as the weakly supervised methods due to the lack of supervision.

Compared to the unsupervised methods that use all training instances regardless of their instance-level labels, the semi-supervised methods that selectively use them (i.e., only normal instances) sometimes perform better and sometimes worse.
That is, learning the normality of temporal data from anomalous instances can degrade the detection performance due to anomalous patterns that reside in the instances;
but in some cases, the increasing number of training data points is rather helpful to improve the generalization power despite the existence of anomalous points.
This strongly indicates that how to utilize the instance-level labels does matter and largely affects the final performance.

Our WETAS\xspace achieves the best performance for all the datasets, and notably, it significantly beats DeepMIL\xspace whose performance highly depends on the number of segments.
For the EMG\xspace dataset whose instance length is only 500, DeepMIL\xspace-16 is not able to effectively detect anomalies because each input segment is too short to accurately compute the segment-level anomaly score;
for the GHL\xspace dataset, DeepMIL\xspace-4 suffers from coarse-grained segmentation, which leads to the limited F1 and IoU.
On the contrary, WETAS\xspace successfully finds anomalous segments by using dynamic alignment with the sequential pseudo-label.

\smallsection{Ablation Study}
\label{subsubsec:ablation}
To further investigate the effectiveness of WETAS\xspace, we measure the F1-score of several variants that ablate each of the following components:
1) DiCNN\xspace (vs. LSTM\xspace) for capturing the temporal dependency, 2) DTW-based alignment loss, 3) DTW-based segmentation,
4) global max (vs. average) pooling for the classification loss,
and 5) corruption by the instance-level label (vs. random sampling) for the negative pseudo-label $\widetilde{\mathbf{z}}_-$ in Equation~\eqref{eq:aloss}.
In case of WETAS\xspace without the DTW-based segmentation, we measure both the F1\footnote{For this measure, we set the anomaly threshold (applied to our point-level anomaly scores) to $\tau$ that is used for pseudo-labeling in Equation~\eqref{eq:seqlabel}.} and F1-best using the series of local (or point-level) anomaly scores.

Table~\ref{tbl:ablation} summarizes the results of our ablation study on the SMD\xspace dataset.
It is worth noting that the DTW-based segmentation considerably improves the performance compared to the case simply using the obtained local anomaly scores.
This is because the alignment with the pseudo-label filters out normal instances based on the global anomaly score, and also considers the continuity of anomalous points while limiting the number of anomalous segments.
Nevertheless, their F1-best scores (for Rows 2 and 3) are respectively 0.4472 and 0.4590, which means that WETAS\xspace outperforms other point-level detection methods even without the DTW-based segmentation.
In addition, it is obvious that DiCNN\xspace is more effective to capture the long-term dependency of temporal data than LSTM\xspace.
The global max pooling and the corrupted pseudo-labels also turn out to help the DiCNN\xspace model to learn useful features from the instance-level labels, compared to their alternatives.

\smallsection{Qualitative Analyses}
\label{subsubsec:qualitative}
We also qualitatively compare the anomaly detection results
for test instances.
Figures~\ref{fig:qresults} and~\ref{fig:trainingprocess} present the results on consecutive instances for the SMD\xspace dataset.
We plot only the two series of input variables that are directly related to observed anomalous events.

Compared to the baselines, WETAS\xspace more accurately finds
anomalous segments in terms of their number as well as the boundaries.
To be specific, DeepMIL\xspace-4 with fewer segments correctly identifies the segments that include anomalous events, but the boundaries are far from the ground truth due to their fixed length and location.
In contrast, DeepMIL\xspace-16 with more segments makes unreliable predictions, because each input segment is not long enough that the normality and abnormality of the segment are effectively captured.
The point-level detection method, LSTM-VAE\textsuperscript{++}\xspace, finds out non-consecutive anomalous points, which results in a large number of separate segments.
It is obvious that such discontinuous prediction makes it difficult to investigate the occurrence of anomalous events.

\section{Pseudo-code of WETAS\xspace Framework}

Algorithm~\ref{alg:pseudocode} shows the pseudo-code of training the framework.
In practice, the model parameters are updated by using minibatch SGD with the Adam optimizer~\cite{kingma2014adam}.

\begin{algorithm}[h]
	\small
    \DontPrintSemicolon

    \SetKwComment{Comment}{$\triangleright$\ }{}
	\KwIn{A training set containing $N$ instances of temporal data $\{(\mathbf{X}^{(1)},y^{(1)}),\ldots,(\mathbf{X}^{(N)},y^{(N)})\}$}
	\KwOut{Updated DiCNN\xspace model $f(\cdot;\Theta)$ and the anomaly weight vector $\mathbf{w}$}

	\While{convergence}{
    \For{$i=1,\ldots,N$}{
    \vspace{5pt}
    \Comment*[r]{Compute the local anomaly scores}
    $\vech{1}^{(i)},\ldots,\vech{T}^{(i)} = f(\vecx{1}^{(i)},\ldots,\vecx{T}^{(i)};\Theta)$ \;
    $s_1^{(i)},\ldots,s_T^{(i)} = \sigma(\mathbf{w}^\top\vech{1}^{(i)}),\ldots,\sigma(\mathbf{w}^\top\vech{T}^{(i)})$ \;

    \vspace{5pt}
    \Comment*[r]{Compute the global anomaly score}
    $\vech{*}^{(i)} = \text{global-pooling}(\vech{1}^{(i)},\ldots,\vech{T}^{(i)})$ \;
    $s_*^{(i)} = \sigma(\mathbf{w}^\top \vech{*}^{(i)})$\;

    \vspace{5pt}
    \Comment*[r]{Obtain the sequential pseudo-label}
    $m_1^{(i)},\ldots,m_T^{(i)} = $ \;
    \renewcommand{\nl}{\let\nl\oldnl}\hspace{30pt} $\text{min-max-norm}(\mathbf{w}^\top\vech{1}^{(i)},\ldots,\mathbf{w}^\top\vech{T}^{(i)})$\;
    $\widetilde{\mathbf{z}}_+^{(i)} = y^{(i)}\cdot\phi_L(m_1^{(i)},\ldots,m_T^{(i)})$\;
    $\widetilde{\mathbf{z}}_-^{(i)} = (1-y^{(i)})\cdot\phi_L(m_1^{(i)},\ldots,m_T^{(i)})$\;

    \vspace{5pt}
    \Comment*[r]{Calculate the two losses}
    $\mathcal{L}_{c} = -y^{(i)}\log s_*^{(i)} - (1-y^{(i)})\cdot\log(1-s_*^{(i)})$ \;
    $\mathcal{L}_{a} = \max(0, (1/T)\cdot\sdtw{\gamma}(\widetilde{\mathbf{z}}_+^{(i)}, \mathbf{X}^{(i)})$ \;
    \renewcommand{\nl}{\let\nl\oldnl}\hspace{44pt} $- (1/T)\cdot\sdtw{\gamma}(\widetilde{\mathbf{z}}_-^{(i)}, \mathbf{X}^{(i)}) +\beta)$ \;
    $\mathcal{L} = \mathcal{L}_{c} + \mathcal{L}_{a}$ \;

    \vspace{5pt}
    \Comment*[r]{Update all the model parameters}
    $\Theta = \Theta - \eta\cdot {\partial\mathcal{L}}/{\partial\Theta}$\;
    $\mathbf{w} = \mathbf{w} - \eta\cdot {\partial\mathcal{L}}/{\partial\mathbf{w}}$\;
    }}
\caption{Optimizing the WETAS\xspace framework}
\label{alg:pseudocode}
\end{algorithm}

\section{Computation of Dynamic Time Warping}

The dynamic time warping (DTW) discrepancy (and the optimal alignment matrix) between two time-series of lengths $M$ and $N$ is usually computed by solving a dynamic program based on Bellman recursion, which takes a quadratic $O(MN)$ cost.
The continuous relaxation of DTW (i.e., soft-DTW), which enables to calculate the gradient of DTW with respect to its input, also can be computed in a similar way to the original DTW~\cite{cuturi2017soft}.
Algorithm~\ref{alg:softdtw} presents the detailed algorithms for computing $\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X})$ and $\nabla_{\Delta(\widetilde{\mathbf{z}}, \mathbf{X})}\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X})$ based on Bellman recursion.

In order to obtain the eligible segmentation result, we need to enforce the constraint that a single time point should not be aligned with multiple consecutive labels.
To this end, our forward recursion which computes $\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X})$ does not consider the $\downarrow$ relation in its recurrence; i.e., $R_{l,t}$ depends on only $R_{l-1,t-1}$ and $R_{l,t-1}$, excluding $R_{l-1,t}$ (Line 6 in Algorithm~\ref{alg:softdtw}).
Accordingly, the backward recursion which computes $\nabla_{\Delta(\widetilde{\mathbf{z}}, \mathbf{X})}\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X})$ also does not allow the $\uparrow$ relation; i.e., $E_{l,t}$ is obtained from $E_{l,t+1}$ and $E_{l+1, t+1}$, excluding $E_{l+1, t}$ (Line 16 in Algorithm~\ref{alg:softdtw}).

Note that the gradient of soft-DTW with respect to the cost matrix, $\nabla_{\Delta(\widetilde{\mathbf{z}}, \mathbf{X})}\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X})$, can effectively update the anomaly weight vector $\mathbf{w}$ as well as the model parameters of DiCNN\xspace $\Theta$ by the help of the gradient back-propagation.
This is possible because each entry of the cost matrix $[\Delta(\widetilde{\mathbf{z}}, \mathbf{X})]_{lt} = \delta(\slab{l}, \vecx{t})$ is defined by the binary cross entropy between a pseudo-label $\slab{l}$ and a local anomaly score $s_t=\sigma(\mathbf{w}^\top \vech{t})$,
\begin{equation*}
    \delta(\slab{l}, \vecx{t}) = -\left\{\slab{l}\cdot\log s_t + (1-\slab{l})\cdot\log(1-s_t)\right\}.
\end{equation*}
In the end, the gradient optimizes each local anomaly score to be closer to the pseudo-labels that are softly-aligned with the time point.

\begin{algorithm}[h]
\small
    \DontPrintSemicolon
    \SetKwProg{Fn}{Function}{:}{}
    \SetKwComment{Comment}{$\triangleright$\ }{}

    \Fn{$\text{forward} \left(\widetilde{\mathbf{z}}, \mathbf{X} \right)$}{
    \Comment*[r]{Fill the alignment cost matrix $R\in\mathbb{R}^{L\times T}$}
    $R_{0,0} = 0$ \;
    $R_{:,0} = R_{0,:} = \infty$ \;
    \For{$l=1,\ldots,L$}{
        \For{$t=1,\ldots,T$}{
        \vspace{2pt}
            \small
            $R_{l,t} = \delta(\slab{l}, \vecx{t}) + \text{min}_\gamma\{R_{l-1,t-1}, R_{l,t-1} \}$
        }
    }
    \Return $\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X}) = R_{L,T}$
    }
    \vspace{5pt}
    \Fn{$\text{backward} \left(\widetilde{\mathbf{z}}, \mathbf{X} \right)$}{
    \Comment*[r]{Fill the soft alignment matrix $E\in\mathbb{R}^{L\times T}$}
    $E_{L+1,T+1} = 1$ \Comment*[r]{$E_{l,t}:=\partial R_{L,T}/\partial R_{l,t}$}
    $E_{:,T+1} = E_{L+1,:} = 0$ \;
    $R_{:,T+1} = R_{L+1,:} = -\infty$ \;
    \For{$l=L,\ldots,1$}{
        \For{$t=T,\ldots,1$}{
        \vspace{2pt}
        $a = \exp\frac{1}{\gamma}(R_{l,t+1}-R_{l,t}-\delta(\slab{l},\vecx{t+1}))$ \;
        $b = \exp\frac{1}{\gamma}(R_{l+1,t+1}-R_{l,t}-\delta(\slab{l+1},\vecx{t+1}))$ \;
        \vspace{-10pt}
        $E_{l,t} = a \cdot E_{l,t+1} + b \cdot E_{l+1,t+1}$ \;
        }
    }
    \Return $\nabla_{\Delta(\widetilde{\mathbf{z}}, \mathbf{X})}\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X}) = E$
    }
	\caption{Forward recursion and backward recursion to compute $\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X})$ and  $\nabla_{\Delta(\widetilde{\mathbf{z}}, \mathbf{X})}\sdtw{\gamma}(\widetilde{\mathbf{z}}, \mathbf{X})$}
	\label{alg:softdtw}
\end{algorithm}

\smallsection{Complexity Analysis}
In terms of efficiency, we analyze that our framework additionally takes the computational cost of $O(LT)$ for DTW alignment between $\widetilde{\mathbf{z}}$ and $\mathbf{X}$ (per CNN inference and its gradient back-propagation), as described in Algorithm~\ref{alg:softdtw}.
Furthermore, because of 1) the small $L$ value ($L \ll T$), 2) batch-wise computations in the PyTorch framework, and 3) GPU parallel computations for DTW recursions based on the numba library,
it does not raise any severe efficiency issue.

\section{Reproducibility}
For reproducibility, our implementation is publicly available\footnote{https://github.com/donalee/wetas}, and Table~\ref{tbl:optim} presents the optimization details of WETAS\xspace.
We empirically found that the performances are hardly affected by these hyperparameters for the optimization.

\begin{table}[htbp]
    \caption{Details for the optimization of WETAS\xspace.}
    \label{tbl:optim}
    \centering
    \resizebox{.99\linewidth}{!} {%
    \begin{tabular}{rl}
    \toprule
    \multirow{2}{*}{Batch size} & 32 (for EMG\xspace, GHL\xspace, SMD\xspace) \\
    & \ \ 8 (for Subway\xspace) \\
    Optimizer & Adam optimizer  \\
    Initial learning rate & 0.0001 \\
    Max \# epochs & 200 \\
    Stopping criterion & Instance-level F1 (on validation) \\ \bottomrule
    \end{tabular}
    }
\end{table}

\section{Hyperparaemter Search}

For our WETAS\xspace framework, we search the optimal values of the following hyperparameters:
the length of a sequential pseudo-label $L\in$ \{4, 8, 12, 16\}, the anomaly threshold for pseudo-labeling $\tau\in$ \{0.1, 0.3, 0.5, 0.7\}, and the margin size for the alignment loss $\beta\in$ \{0.1, 0.5, 1.0, 2.0\}.
The optimal values are selected based on instance-level F1-scores on the validation set.
The selected hyperparameter values for reporting the final performance are listed in Table~\ref{tbl:hyperparameters}.
In case of the smoothing parameter $\gamma$ used for soft-DTW, we fix its value to 0.1 without further tuning.

\begin{table}[htbp]
    \caption{Selected hyperparameter values for WETAS\xspace.}
    \label{tbl:hyperparameters}
    \centering

    \begin{tabular}{rHHHH}
    \toprule
    Datasets & EMG\xspace  & GHL\xspace & SMD\xspace & Subway\xspace  \\ \midrule
    global-pooling & avg & max & max & avg  \\

    $L$ & 4 & 16 & 12 & 12 \\
    $\tau$ & 0.3 & 0.1 & 0.5 & 0.5 \\
    $\beta$ & 0.1 & 1.0 & 0.5 & 0.5 \\ \bottomrule
    \end{tabular}

\end{table}

\section{Baseline Methods}
We describe the details of the anomaly detection methods which are used as the baseline in our experiments.
All of them employ their own deep neural networks to effectively model the temporal dependency among time points, and compute the anomaly score for each point or segment.
\begin{itemize}
    \setlength\itemsep{-0.1em}
    \item \textbf{Donut\xspace}~\cite{xu2018unsupervised}:
    A simple VAE model optimized by the modified evidence lower bound (M-ELBO).
    It also uses a sampling-based imputation technique for missing points, in order to effectively deal with anomalous points during the detection.

    \item \textbf{LSTM-VAE\xspace}~\cite{park2018multimodal}:
    A VAE model that employs long short term memory (LSTM) for its encoder and decoder.
    It is trained to reconstruct the non-anomalous training data well, and defines the anomaly score by the reconstruction error.

    \item \textbf{LSTM-NDT\xspace}~\cite{hundman2018detecting}:

     A LSTM network that is trained to predict the next input (i.e., sequence modeling).
     It additionally adopts the non-parametric dynamic thresholding (NDT) technique to automatically determine the optimal anomaly threshold.

    \item \textbf{OmniAnomaly\xspace}~\cite{su2019robust}:
    The state-of-the-art point-level anomaly detection model that uses gated recurrent units (GRU\xspace) as the encoder and decoder of VAE.
    It incorporates advanced techniques into VAE, including normalizing flows and a linear Gaussian space model, to consider stochasticity and temporal dependency among the time points.

    \item \textbf{DeepMIL\xspace}~\cite{sultani2018real}: The multiple-instance learning method

    that learns from weakly labeled temporal data.
    It produces the anomaly prediction for each fixed-length segment, thus the result can be used for temporal anomaly segmentation.
    We consider different numbers of the segments in a single instance, denoted by DeepMIL\xspace-4, 8, 16.

\end{itemize}
Note that Donut\xspace, LSTM-VAE\xspace, LSTM-NDT\xspace, and OmniAnomaly\xspace fall into the category of unsupervised learning as they do not utilize any anomaly labels for training, and their variants that use only normal instances are categorized as semi-supervised learning.
DeepMIL\xspace is the only existing method that is based on weakly supervised learning.

\section{Additional Experiments}
\smallsection{Comparison with frame-level video anomaly detectors}
\label{subsec:videoperf}
In case of the video dataset (i.e., Subway\xspace), we additionally report the AUROC scores of unsupervised video anomaly detection methods~\cite{tudor2017unmasking, ionescu2019detecting, liu2018classifier, pang2020self} as a benchmark.
They aim to train the networks that take spatio-temporal (or spatial) inputs to compute the anomaly score of each video frame.
Note that all these methods produce the frame-level (or point-level) anomaly predictions, without utilizing the instance-level anomaly labels for training.
In Table~\ref{tbl:videoperf}, the weakly supervised methods\footnote{To compute AUROC for the weakly supervised methods, we regard the anomaly score of each segment as the score for all points in the segment (for DeepMIL\xspace), and use the local (or point-level) anomaly scores without the DTW-based segmentation (for WETAS\xspace).}
(i.e., DeepMIL\xspace and WETAS\xspace) show better performance than the unsupervised methods.
Even though DeepMIL\xspace and WETAS\xspace simply use the pre-computed visual features of each frame (extracted by the pre-trained ResNet), it outperforms the other domain-specific baseline methods (optimized in an end-to-end manner) by the help of the instance-level anomaly labels.
This indicates that leveraging the weak supervision can be more effective to discriminate normal and anomalous video frames compared to fine-tuning the networks for visual feature extraction.

\begin{table}[t]
    \caption{Comparison with several recent frame-level video anomaly detection methods, Dataset: Subway\xspace.}
    \label{tbl:videoperf}
    \centering
    \resizebox{.99\linewidth}{!} {%
    \begin{tabular}{ABAB}
    \toprule
    \multicolumn{1}{c}{Methods} & AUROC & \multicolumn{1}{c}{Methods} & AUROC  \\ \midrule
    Ionescu et al.~\cite{tudor2017unmasking} & 85.7\% & DeepMIL\xspace-4~\cite{sultani2018real} & 95.7\% \\
    Ionescu et al.~\cite{ionescu2019detecting} & 95.1\% & DeepMIL\xspace-8~\cite{sultani2018real} & 96.9\% \\
    Liu et al.~\cite{liu2018classifier} & 93.1\% & DeepMIL\xspace-16~\cite{sultani2018real} & 96.4\% \\
    Pang et al.~\cite{pang2020self} & 92.7\% & WETAS\xspace (ours) & 97.8\% \\ \bottomrule
    \end{tabular}
    }
\end{table}

\begin{figure}[t]
	\centering
	\includegraphics[width=\linewidth]{./FIG/learning_curves.png}
	\caption{Learning curves of WETAS\xspace in terms of training/validation loss and test/validation F1-score, Datasets: EMG\xspace, GHL\xspace, SMD\xspace, and Subway\xspace (from the first row).}
	\label{fig:traincurves}
\end{figure}

\smallsection{Learning curves}
\label{subsubsec:learncurve}
We plot the learning curves of WETAS\xspace by using the EMG\xspace, GHL\xspace, SMD\xspace, and Subway\xspace datasets.
In Figure~\ref{fig:traincurves}, as the number of epochs increases, the classification loss and alignment loss consistently decrease for both the training and validation sets.
This implies that WETAS\xspace can infer more accurate sequential pseudo-labels as the training progresses, and the alignment loss better guides its model to output anomaly scores that are well aligned with the pseudo-label.
Similarly, Figure~\ref{fig:trainingprocess} illustrates that the pseudo-labeling and the DTW-based segmentation collaboratively improve with each other.
Consequently, the instance-level and point-level F1-scores for the test set increase as well.
The results empirically show that the instance-level F1-score (or the total loss) on the validation set can be a good termination criterion for the optimization of WETAS\xspace, thus we adopt it in our experiments.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.99\linewidth]{./FIG/sensitivity_small.png}
	\vspace{10pt}
	\caption{F1-scores of WETAS\xspace with different hyperparameter values, Dataset: GHL\xspace.}
	\label{fig:sensitivity}
\end{figure}

\smallsection{Sensitivity Analyses}
\label{subsubsec:sensitivity}
We finally examine the performance changes of WETAS\xspace with respect to the three hyperparameters (i.e., $L$, $\tau$, and $\beta$).
In Figure~\ref{fig:sensitivity}, we observe that the final performance of WETAS\xspace is not sensitive to $\beta$, and it shows higher F1-score when using a smaller $\tau$ and a larger $L$.
Specifically,
\begin{itemize}
    \item $\beta$ does not much affect the final performance of WETAS, because it simply controls the margin size in the alignment loss.
    \item A smaller $\tau$ encourages to find out more anomalous segments, which leads to a high recall for anomaly detection, by making the sequential pseudo-label have more number of 1s.
    \item A larger $L$ allows a finer-grained segmentation by aligning the time points with more number of 0s and 1s in each sequential anomaly pseudo-label.
\end{itemize}
Nevertheless, unlike the DeepMIL\xspace, the granularity of pseudo-labeling is not a critical factor for WETAS\xspace because the DTW-based segmentation is capable of dynamically aligning an input instance with its pseudo-label.
In conclusion, the best performing hyperparameter values successfully optimize WETAS\xspace under the weak supervision so that it can identify variable-length anomalous segments.

\section{Introduction}
\label{sec:intro}
\input{010introduction}

\section{Related Work}
\label{sec:related}
\input{020relatedwork}

\section{Temporal Anomaly Segmentation}
\label{sec:proposed}
\input{030proposed}

\section{Experiments}
\label{sec:exp}
\input{040experiment}

\section{Conclusion}
\label{sec:concl}
\input{050conclusion}

\smallsection{Acknowledgement}
This work was supported by the NRF grant (No. 2020R1A2B5B03097210) and the IITP grant (No. 2018-0-00584, 2019-0-01906) funded by the MSIT.

\balance
{\small
\bibliographystyle{ieee_fullname}

redpajama_set_name: RedPajamaArXiv


"Habitat Group" has revealed its 2019 business plan which will see the company strive for continuous growth, with new developments planned for Bangkok and Pattaya that will double its current portfolio and with a total project value of over THB 8BN in total. In a move against the current market trend, the group is confident in its upper market target for low-rise condominiums in the central business district of Bangkok and resort destination of Pattaya. Habitat Group will also strengthen the group's overall position by increasing its recurring income portfolio including two new hotel developments as well as an expansion of the group's international client base from the new markets, namely Japan, Taiwan, Singapore, India and the Middle East countries. This widening strategy of prospective clients from previously identified China and Hong Kong will support the company's risk diversification.
The company still anticipates 2019 to be a year of growth and will go ahead with its expansion plan and more developments. The company is preparing to launch five new projects worth a total of over 8 billion baht. These projects remain true to the group's core as being property-for-investment projects located in Bangkok's central business district along the BTS line in Thonglor, Phrom Phong and Asoke areas, as well as in Pattaya's central business district. This year, the company has targeted its sales to be 3 billion baht, a 50% growth compared to its actual 2018's sales of 2 billion baht.
"We are committed to continuous growth and our targets as previously projected. This will be supported by new projects in the pipeline that will double in both the quantity and investment size when compared to those of 2018. At the same time, we are also managing our risk diversification by expanding our customer base and aim to increase our recurring income through a widening of the company's portfolio," declared Mr. Chanin.
Three new luxury low-rise condominium developments in Bangkok will be developed under the "Walden Brand". All are located in the CBD and together are worth approximately THB 3.5 BN. There will also be two more projects in Pattaya, which will be lifestyle investment projects that are managed by global hotel management brand partners. The total project value of these two developments is approximately THB 4.5 BN. The first project in Pattaya will be located in North Pattaya and will open for sales in the first quarter of 2019. Another project will be a condominium in Na Jomtien Beach, situated on a beach front plot of 8 rai. The project is undergoing architectural design and is expected to be launched in mid-2019. For Pattaya, the company has planned for larger development projects in order to offer more units to serve a growing demand. Last year, Habitat was able to close sales in all its Pattaya projects.
The company has forecasted a healthy growth for its condominiums which target upper market buyers, and expects them to continue to appeal to Thai and international residents and investors, especially those that are located in prime locations that have limited land available but remain high in demand. This trend matches with the company's strategy that prioritizes prime locations such as areas close to Bangkok's mass transit system. The company also offers affordable unit prices to its targeted customers. As for Pattaya, the company has recognized that the city is ripe for expansion on the back of a strong tourism sector and that it is part of the Eastern Economic Corridor Development Project (EEC).
"Based on these factors, we are confident to move forward with our growth plans and double the value of our properties for investment. We anticipate a positive growth in the property market this year as it will be driven by the government's continuous infrastructure development projects, further supported by by various private sector investments, especially in the EEC region. However, the company has realized it is essential to develop competitive products for the market as well and we will ensure our designs distinguish ourselves from others and make us stand out from our competitors," said Mr. Chanin.
Mr. Chanin also added that in terms of risk diversification, Habitat Group will implement a strategy to expand its customer base. The company will look for opportunities to open up new markets in addition to the Thai, Chinese and Hong Kong markets in which Habitat Group are strong. The company will reach new markets such as the Middle East as well as Japan, Taiwan, Singapore, and India for Asia. Last year, the company reported total sales of 600 million baht from international buyers. This was 30% of total sales, and Habitat Group expects it will grow to be 40% of total sales this year.
Moreover, the company has targeted to increase its recurring income portfolio by 10% this year. It has also targeted the recurring income portfolio to grow to 30% within three years in order to create a stable income and strengthen the business. Currently, two of its hotels are already being under operation in Pattaya, including the Ville Jomtien and X2 Vibe Pattaya Sephere. These hotels are managed by its sister company Habitat Hospitality. In 2019, the company plans to open two new hotels; X2 Pattaya Oceanphere in the second quarter and the Best Western Premier Bayphere Pattaya in the third quarter.
"A vital factor for strong and sustainable growth is risk management. Our decisions are based on our market evaluation as well as our experience in these core markets. Launching more new projects in 2019 is another big step for Habitat Group and one we will manage successfully," Mr. Chanin concluded.


text:
La Batalla de San Miguel de Chimbo fue un enfrentamiento militar en la Revolución liberal del Ecuador o Revolución liberal alfarista, ocurrida en Ecuador, , enfrentandose las fuerzas del gobierno interino de Vicente Lucio Salazar y las fuerzas revolucionarias liberales alfaristas bajo el mando del militar Cornelio Vernaza.

Antecedentes
Ante la situación del país varias fuerzas liberales se sublevaron en varias partes del país como Manuel Serrano Renda en el Oro, Juan Manuel Triviño y León Valles Francisco en Guayas, así las fuerzas liberales se apoderaban de la costa, el gobierno había enviado refuerzos a varias provincias en el río Chimbo el batallón Babahoyo al mando del general Manuel J. Castillo. encontrarian a los enemigos.</ref>

Orden de batalla

Desarrollo de la Batalla
La batalla fue feroz y ambos bandos lucharon con ardor casi hasta la noche. Inicialmente las tropas revolucionarias sufrieron un descalabro ya que las fuerzas gobiernistas dispararon con una pieza de artillería al centro del poblado de inmediato ordenó a los batallones «Pichincha», «Vencedores Nº 1», «Artillería de Montaña>> que atacasen el poblado durante 11 horas las tropas se enfrascaron en una lucha sin cuartel pero finalmente las tropas gobiernistas tomaron el control, la victoria parecía asegurada pero un en un hecho inesperado el ejército regular – cuando la victoria le parecía segura- abandonó el campo de batalla para pasar a Gatazo -en las cercanías de Riobamba-, y unirse a las tropas del Gral. José María Sarasti para enfrentar a las fuerzas que Alfaro comandaba personalmente.

Consecuencias
Las fuerzas liberales partieron rumbo a unirse a Alfaro que con su ejército marchaba a combatir José María Sarasti en Gatazo, mientras que los conservadores se unieron a Sarasti donde finalmente serían derrotados.

Fuentes

Referencias

Fuentes externas

Ecuador en 1895
Guerras de Ecuador
Batallas de Ecuador
Revolucion Liberal Ecuatoriana
Historia del liberalismo
Liberales de Ecuador

redpajama_set_name: RedPajamaWikipedia


Why Do I Have Roaches in My House?
No one likes feeling as if he is being watched in his own home, especially when those doing the watching are roaches. You must know why you are experiencing a roach infestation before ridding your home of roaches and to prevent future infestations.
All homes are susceptible to a roach infestation.
Every crack in an exterior wall, every crevice that might exist, every door jamb that doesn't seal as well as it used to can look like a giant "welcome" sign to roaches and other pests.
Anything you bring into your home can be a vehicle for roaches, from grocery bags to a box from the storage unit.
The climate in your home is preferable to the outdoors for roaches. The outdoors can be wet and cold, but your home is dry and warm. Roaches find the comfortable climate of your home attractive, just as you do.
The tiniest crumb can make a good snack for a roach and it doesn't take much to create a smörgåsbord in the eyes of a roach. If you consider that your kitchen is an entire room devoted to food, you can easily see why a roach would want to be there.
Outside, roaches can become the prey of other kinds of wildlife. In your home there are most likely no predators for roaches to worry about. Your home is safer than the outdoors so it becomes the natural choice for roaches and other pests.
To prevent infestations, be sure to correct any cracks in walls or doors that don't seal. Inspect everything you bring into your home from an outside source to ensure you're not carrying roaches with you. Keep food messes to a minimum in your kitchen to keep it less attractive to roaches. Set traps in dark corners, behind your refrigerator and in the back of cupboards where roaches tend to lurk.
Cathy Williams is a high school history and English teacher with degrees in education and history. She contributes articles to eHow.


The bull shark, Carcharhinus leucas, also known as the Zambezi shark or unofficially Zambi in Africa and Nicaragua shark in Nicaragua, is a shark commonly found worldwide in warm, shallow waters along coasts and in rivers. The bull shark is known for its aggressive nature, predilection for warm shallow water, and presence in brackish and freshwater systems including estuaries and rivers. The bull shark can thrive in both saltwater and freshwater and can travel far up rivers.


text:
Clinton: Congress must act on Zika
By By Hillary Clinton
Hillary clinton wants Congress to act fast on funding to aid in fighting the Zika crisis. (Mark Lennihan / AP)
Five months ago, I was alarmed to read the first stories about a new virus called Zika that was spreading through South and Central America. I was particularly concerned by reports of the disease's devastating health consequences for pregnant women and their babies. Since then, we've all learned a great deal about the threat posed by the Zika virus, and it's become clear that we have to take urgent action to stop it from becoming a full-on epidemic.
As of today, Florida has experienced 154 travel-related cases, including 36 pregnant women whose children are now at risk of developing microcephaly – a dangerous developmental condition where babies are born with heads that are too small.
So far, all of these infections have been related to travel – people went to Central and South America, were bitten by infected mosquitos there, and came home with the virus. But as summer temperatures begin to rise, we can expect more of the mosquitos that carry the virus to reach American shores. And we now have evidence that Zika can be sexually transmitted as well.
To spur action, my campaign has launched a petition on Change.org that has collected thousands of signatures, and I sent two of my top aides to Puerto Rico to see first-hand what more can be done to combat Zika's spread. They spent time meeting with government officials and health experts, visiting affected neighborhoods, and observing Zika-education efforts at a local Women, Infants, and Children clinic. They saw children playing around a flooded canal – a high-risk area for contracting mosquito-borne illnesses – in an area where most families cannot afford effective repellent. And even though there is an effort underway to rid the island of stagnant water pools where mosquitos can breed, Puerto Rican officials simply do not have the resources to contain the problem.
What they saw on the ground made it crystal clear that more emergency funding will be required to stop the spread of the Zika virus in Puerto Rico and the health crisis looming over the United States. Florida has already declared a public health emergency, and is taking important steps to increase Zika awareness. But much more needs to be done at the federal level as well.
Florida's anti-democratic 'poll tax' will cost voters — and hundreds of millions of dollars | Opinion
Legislature's Amendment 4 bill is a return to Jim Crow voting practices | Opinion
The GOP's nightmare scenario is playing out in its Obamacare lawsuit | Opinion
First, as Congress returns to Washington, it needs to act quickly to provide federal emergency funding. Before the recess, the Senate passed a $1.1 billion package to fund Zika research and prevention. Now, it's time for Republicans in the House to stop playing politics with people's health, and send this bill to President Obama's desk. Only then can we finally get to work on creating a functional rapid diagnostic test for Zika, developing a vaccine, preventing further infections through mosquito control and abatement, and expanding our research into the connection between Zika and microcephaly.
Second, we need to ensure that all Americans – and especially pregnant women – know how to protect themselves. Zika is especially dangerous but it often exhibits no symptoms, which can make it harder to convince people of the need to protect themselves and their families. The CDC Foundation's campaign to increase Zika awareness education in the affected areas, as well as expanded access to contraception, deserves strong support.
Zika is real. It's dangerous. And if we're serious about stopping this epidemic in its tracks, then there's no time to waste.
Hillary Clinton, the presumptive Democratic candidate for President of the United States, wrote this exclusively for the Sun Sentinel.
Diseases and Illnesses


Contains (2) Pro Form Leather Sealing Rings, (2) microphone covers and (2) sets of Interior Foam for battery covers. Easy to use 'Peel & Stick' backing. Special soft foam provides the ultimate in comfort and sound deadening qualities. Conforms for a complete fit for glasses wear. Replacement recommended every 9 to 15 months.


text:
Q: Is there a way to change the CSS of someone's website every time you go to the page? I'm a little bit of a noob so I apologize if this is a dumb question but I'm wondering if there's a way to edit the CSS of a website every time you go to automatically.
Many kids in my programming class find the dark theme on their eyes to be easier. Of course our school website updates their site to a sleek all white design.
Obviously I could just inspect and then change the CSS but I'm wondering if there's a way so that the CSS I change stays or is permanent and will automatically load like what it was changed to.
Is there possibly a way you could load a javascript file automatically when a page is reloaded or something?
Is there some kind of file or program that could possibly do this? Or anyone have any ideas how I could make this possible?
Thanks everyone!

A:
Is there possibly a way you could load a javascript file automatically when a page is reloaded or something?

If you really want to go the Javascript route, yes, it's pretty easy. Install a userscript manager of your choice, such as Tampermonkey, and insert a customized <style>. For example:
// ==UserScript==
// @name        Stack Input Height
// @include     https://stackoverflow.com/questions/*
// @grant       none
// ==/UserScript==

document.head.appendChild(document.createElement('style')).textContent = `
  textarea.wmd-input, textarea#wmd-input {
    height: 800px;
  }
`;

You can also change/remove style attributes directly on page load.
There are also add-ons specifically designed to customize styles, for example, Stylish.

redpajama_set_name: RedPajamaStackExchange