text:
Alex Hudson
Editor, Newsweek International
https://www.facebook.com/alexjhuds Twitter https://www.linkedin.com/in/alexjhudson/
Editor of Newsweek International, Alex is responsible for international audiences. Writing predominantly about politics and technology, he is an industry leader in AI and finding out what happens next.
10 Years In, Tim Cook's Apple Still Has One More Thing
The Apple CEO has been in the job for 10 years but still hasn't really introduced any game-changing products. Could the "Apple Car" change all that?
On 8/24/21 at 8:38 AM EDT
I Was Saved in the London Riots by the Hackney Heroine
"10 years on and you've brought me tears," Pauline Pearce says. "I'll never forget that moment and I'm overwhelmed. I'm really happy to find out that you're alive after all this time."
On 8/6/21 at 9:18 AM EDT
After Five Years of Brexit Meaning Brexit, It Remains Difficult to Define
Everyone know that "Brexit means Brexit" but, five years after the vote, many Brits aren't any clearer on what that really means in practice.
On 6/23/21 at 10:39 AM EDT
Brexit Could Cost Talent. A U.K. Educator on What Universities Can Do Now
"Universities are felt by many people not to be quite as relevant as they could be. They have become a preserve of the left-wing elite," says Professor Sir Ed Byrne. "Their thinking isn't as fine-tuned as it might be with the needs of ordinary people and their communities."
On 1/21/21 at 3:03 PM EST
New British COVID Lockdown Could Last Until Anniversary on March 23
British Prime Minister Boris Johnson is rumored to be looking at the end of COVID restrictions to be a year to the day since the start of them.
On 1/5/21 at 9:13 AM EST
British Man Receives World's First Dose of Oxford COVID Vaccine
Prime Minister Boris Johnson has promised "tens of millions" of vaccinations in the next three months.
England to Ban Animal Exports for Slaughter in European First
Britain will continue to export millions of poultry but animal export from England and Wales for slaughter will end.
On 12/3/20 at 6:37 AM EST
Former Hospital Worker Charged With Dozens of Child Sexual Offenses
The 84 charges include rape, attempted rape, sexual assault of a child under 13 and indecent assault on a male.
On 11/26/20 at 7:54 AM EST
Joe Biden: Hard Border on the Island of Ireland 'Just Not Right'
President-elect Joe Biden has reaffirmed his belief in the Good Friday Agreement and that nothing should put Irish peace in jeopardy.
The World Is $277 Trillion in Debt So Why Aren't Economists More Worried?
Economists are keeping calm, carrying on and using war analogies to assure the public that everything will be ok.
Brexit Architect Dominic Cummings Going Leaves 'Power Vacuum' in Government
"The Conservative Party is not a happy place at the moment," Sir Craig Oliver said.
Britain Attempts to Test Whole Population for COVID
Officials are purchasing over 60 million tests a month with the hope to be the biggest country in the world to be testing everybody.
On 11/12/20 at 10:18 AM EST
'Major Incident' Declared After Car Crashed Into London Police Station
There are a number of videos of the incident, showing a man getting out of the car after crashing into the police station, pouring a substance onto the road before setting it alight as a police vehicle arrives.
On 11/11/20 at 3:58 PM EST
British Politicians Rebel Against Any Further COVID Lockdowns
England is currently under tight lockdown restrictions, with all non-essential shops and pubs, gyms and non-elite sport shut down.
What Happens to Brexit Now Joe Biden Is President-Elect?
Biden has Irish heritage and would gain more economically from agreeing trade deals with the European Union first rather than the U.K.
British Prime Minister Refuses to Say Donald Trump Should Concede
Boris Johnson refused to say Donald Trump had lost despite being willing to say that Joe Biden had won.
On 11/9/20 at 1:18 PM EST
Neil Kinnock Happy to Write New Biden Speeches, Even Without Attribution
Kinnock could not be happier for the president-elect and is even willing to reprise his role as Biden's "greatest ever speechwriter."
On 11/9/20 at 10:26 AM EST
Will Donald Trump's 'Fraud' Claims End up in the Supreme Court?
President Trump has promised that the Supreme Court will have the final judgment on the 2020 election.
Nigel Farage Says Vote Fraud 'There for All to See' Despite No Evidence
Trump's friend, British politician Farage, said Trump made comments about victory "out of frustration."
Four Killed in Vienna 'Terror Attack' With Shootings in Six Locations
The incident is still ongoing with at least one shooter still believed to be at large.
More Terror Attacks Inevitable in France as 'War on Islamists' Continues
It was 1971 when France last went a calendar year without a terror attack.
On 10/30/20 at 2:53 PM EDT
Health Worker Thrown From Train After Confronting Maskless Group
The victim's eye socket was broken in three places after he was thrown from the train after being repeatedly punched.
On 10/28/20 at 8:49 AM EDT
Africa's Largest Nation Backs Trump Despite Black Lives Matter Hostility
While Trump wanted to stop immigrants coming to the U.S. from countries like Nigeria in a "clearly racist" move, his support hasn't gone anywhere, despite Black Lives Matter and the "End SARS" crisis.
Boris Johnson Tells Britain to 'Get Ready for No-Deal Brexit'
"We will prosper mightily as an independent free-trading nation, controlling our own borders, our fisheries and setting our own laws," the U.K. prime minister said.
Russia and China Biggest Threats to U.K., New MI5 Intelligence Boss Says
Moscow delivered "bursts of bad weather" but China was responsible for "changing the climate" of security and intelligence, the new head of MI5 Ken McCallum said.
Google Accused of Not Fixing 'Illegal Advantage' Competition Problems
"Google is still in the mood that Microsoft once was, they think [they are] above the law," lawyer Thomas Höppner tells Newsweek.
On 9/28/20 at 3:25 PM EDT
Why Does Jeremy Corbyn's Brother Still Believe COVID-19 Is a Hoax?
COVID-19 is definitely real but millions of people around the world believe it isn't. Is it just a perfect storm of conspiracy theorists congregating around one issue?
U.K. Government to Top Up Wages for Six Months As COVID Measures Continue
Rishi Sunak said that the British economy is going through a "more permanent adjustment" and what happens now "must be different from what came before."
Johnson Tells U.K. to 'Summon Discipline' as COVID 'Second Wave' Grows
Boris Johnson said that these new restrictions could last six months through winter until a vaccine is approved and deployed across the country.
U.K. Could Hit 50,000 COVID-19 Cases-a-Day by Mid-October, Scientists Warn
"The trend in the U.K. is heading in the wrong direction and we are at a critical point in the pandemic," Britain's top medical advisers have warned.
Personalized daily newsletter
Daily news headlines & detailed briefings enjoyed by half a million readers.


text:
package org.scalactic

import org.scalatest._
import scala.collection.GenSeq
import scala.collection.GenMap
import scala.collection.GenSet
import scala.collection.GenIterable
import scala.collection.GenTraversable
import scala.collection.GenTraversableOnce
import scala.collection.{mutable,immutable}

class FreshTypeCheckedMapEqualityConstraintsSpec extends Spec with NonImplicitAssertions with CheckedEquality {

  case class Super(size: Int)
  class Sub(sz: Int) extends Super(sz)

  val super1: Super = new Super(1)
  val sub1: Sub = new Sub(1)
  val super2: Super = new Super(2)
  val sub2: Sub = new Sub(2)
  val nullSuper: Super = null

  case class Fruit(name: String)
  class Apple extends Fruit("apple")
  class Orange extends Fruit("orange")

  implicit class IntWrapper(val value: Int) {
    override def equals(o: Any): Boolean =
      o match {
        case that: IntWrapper => this.value == that.value
        case _ => false
      }
    override def hashCode: Int = value.hashCode
  }

  object `the MapEqualityConstraints trait` {

    def `should allow any Map to be compared with any other Map, so long as the key and value types of the two Maps have respective recursive EqualityConstraints` {
      assert(mutable.HashMap('a' -> 1, 'b' -> 2, 'c' -> 3) === immutable.HashMap('a' -> 1, 'b' -> 2, 'c' -> 3))
      assert(mutable.HashMap('a' -> 1, 'b' -> 2, 'c' -> 3) === immutable.HashMap('a' -> 1L, 'b' -> 2L, 'c' -> 3L))
      assert(mutable.HashMap('a' -> 1L, 'b' -> 2L, 'c' -> 3L) === immutable.HashMap('a' -> 1, 'b' -> 2, 'c' -> 3))
      assert(immutable.HashMap('a' -> 1, 'b' -> 2, 'c' -> 3) === mutable.HashMap('a' -> 1L, 'b' -> 2L, 'c' -> 3L))
      assert(immutable.HashMap('a' -> 1L, 'b' -> 2L, 'c' -> 3L) === mutable.HashMap('a' -> 1, 'b' -> 2, 'c' -> 3))

      // Test for something convertible
      assertTypeError("mutable.HashMap('a' -> new IntWrapper(1), 'b' -> new IntWrapper(2), 'c' -> new IntWrapper(3)) === immutable.HashMap('a' -> 1, 'b' -> 2, 'c' -> 3)")
      assertTypeError("mutable.HashMap('a' -> 1, 'b' -> 2, 'c' -> 3) === immutable.HashMap('a' -> new IntWrapper(1), 'b' -> new IntWrapper(2), 'c' -> new IntWrapper(3))")

      assert(mutable.HashMap('a' -> new Apple, 'b' -> new Apple) === immutable.HashMap('a' -> new Fruit("apple"), 'b' -> new Fruit("apple")))
      assert(immutable.HashMap('a' -> new Fruit("apple"), 'b' -> new Fruit("apple")) === mutable.HashMap('a' -> new Apple, 'b' -> new Apple))
      assertTypeError("mutable.HashMap('a' -> new Apple, 'b' -> new Apple) === immutable.HashMap('a' -> new Orange, 'b' -> new Orange)")
      assertTypeError("immutable.HashMap('a' -> new Apple, 'b' -> new Apple) === mutable.HashMap('a' -> new Orange, 'b' -> new Orange)")
      assertTypeError("immutable.HashMap('a' -> new Orange, 'b' -> new Orange) === mutable.HashMap('a' -> new Apple, 'b' -> new Apple)")
      assertTypeError("mutable.HashMap('a' -> new Orange, 'b' -> new Orange) === immutable.HashMap('a' -> new Apple, 'b' -> new Apple)")
    }
  }
}

redpajama_set_name: RedPajamaGithub


text:
Crassispira rubidofusca é uma espécie de gastrópode do gênero Crassispira, pertencente à família Pseudomelatomidae.

 Schepman, 1913. The prosobranchia of the Siboga expedition. Part IV -V - VI: Toxoglossa

Pseudomelatomidae
Gastrópodes descritos em 1913

redpajama_set_name: RedPajamaWikipedia


Wine can be found at virtually any event or gathering where there are people. However, wine can also be frustrating and confusing. If you want to jump into the world of wine, the following piece is for you.
Get familiar with your wine shop. That's because each shop is different. The various shops have their own preferences, structures, and themes. If you're new to the world of wine, having a store full of expensive labels may not be a great fit. Find a wine shop that fits you well.
Storing your wine properly preserves its flavor. Any wine can be negatively affected by temperatures outside its optimal range. The best temperature for storing your wine is around 55 degrees. Either purchase a wine cooler or store them in your basement for best results.
Wine that's cheap isn't as bad as many people would think. Consider a wine from Chile if you want to save money without sacrificing quality. Wine from Chile is a very affordable option when you are on a budget. Specifically, look toward their Sauvignon Blancs and Cabernet Sauvignons. Other great regions for their great-tasting, yet economical wines are Argentina, New Zealand, and South Africa.
Choose a wine because you enjoy it not because some critic recommends it. While professional wine tasters have strong opinion over which wines are best, studies have been done that show that the average person actually prefers less expensive wine if they do not know the price. Even if you like cheap wine, stick with it. The end goal, after all, is to enjoy what you're drinking.
The largest difference between red and white wines is the type and color of grape. Red wine is made of purple grapes that have a fuller body. Green grapes are used to produce the light and crisp white wines. There is more than just color differences in these wines, so go out and experiment to find the perfect wine for you.
Young white wines are your best choice. Chardonnay, is a notable exception to this rule. The difference is due to the fact that Chardonnay is aged in oak barrels, unlike other white wines. On the other hand, wines stored in oak barrels will taste better as they age.
Now that you have more wine knowledge, you should find it easier to locate the wine you need much easier. You now have sound knowledgable about the wine and will not be confused. You'll soon be a wine connoisseur with your new knowledge.


text:
Home > Portuguese president hosts Mawlana Hazar Imam at Presidential Palace
The.Ismaili | 09 July 2018 | Portugal
Portuguese president hosts Mawlana Hazar Imam at Presidential Palace
President Marcelo Rebelo de Sousa welcomes Mawlana Hazar Imam to Palácio de Belém. Akbar Hakim
The Portugal Army Band performs the Portuguese national anthem and the Nashid al-Imamah to welcome Mawlana Hazar Imam to Palácio de Belém. Akbar Hakim
Mawlana Hazar Imam in conversation with President Marcelo Rebelo de Sousa inside Palácio de Belém. Akbar Hakim
Mawlana Hazar Imam shares a light moment with President Marcelo Rebelo de Sousa during their meeting on 9 July 2018. Akbar Hakim
President Marcelo Rebelo de Sousa bids farewell to Mawlana Hazar Imam, upon his departure from Palácio de Belém. Akbar Hakim
President Marcelo Rebelo de Sousa officially welcomed Mawlana Hazar Imam to Portugal on 9 July with full state honours at Palácio de Belém, the official residence of the President.
Before entering the presidential palace on Monday, Mawlana Hazar Imam and President Marcelo Rebelo de Sousa paused to observe the Portugal Army Band's rendition of the Portuguese National Anthem and the Nashid al-Imamah.
During a one-to-one meeting, the two parties discussed areas of mutual cooperation.
President de Sousa will host a dinner in honour of Hazar Imam on Monday night. Portuguese government officials and leaders of the Jamat and the Aga Khan Development Network are expected to attend.
At their last meeting in July 2017, President Marcelo Rebelo de Sousa decorated Mawlana Hazar Imam with the Grã-Cruz da Ordem da Liberdade — or Grand Cross of the Order of Liberty — in recognition of his service to uplifting lives around the world.
Hazar Imam arrived in Portugal on 6 July to attend events marking the conclusion of the Diamond Jubilee; a celebration of 60-years of his Imamat. While in Lisbon, he is also expected to address the Portuguese parliament on 10 July, which will be streamed live on https://the.ismaili/live at 11:45 AM Lisbon time.
This is a historic time for the Ismaili Muslim community. Thousands of Ismailis have travelled to Lisbon from far and wide to take part in festivities related to the end of the Diamond Jubilee.
The Ismaili | 14 January 2020 | Global
President of the Republic of Portugal visits Aga Khan Academy Maputo
Shirin Malik, Nazim Karim | 12 January 2020 | Global
Ismailis at forefront of advances in science and technology in the United States

redpajama_set_name: RedPajamaCommonCrawl


text:
Semifinal, Final Ticket Packs Sold Out For Women's World Cup
By Travis Yoesting October 31, 2018
If you didn't act fast, you might be out of luck.
Did you get your tickets for the Women's World Cup semifinals and final in France? If not, you might be out of luck. Women's World Cup ticket sales have been through the roof, with the pack including the semifinals and final having already sold out shortly after going on sale.
Ticket packages for the Women's World Cup went on sale on Oct. 19, with some shockingly affordable deals. If you acted fast, you could score tickets to the two semifinal matches and the final (all in Lyon) for just $223 total — $74 per ticket.
But those packages are now sold out, according to FIFA.
TICKETING INFO:
Dear fans. All Lyon #FIFAWWC packages in the current sales phase, have now been sold. Single tickets will be available after the official draw, during the VISA pre-sale.
Tickets in other packages are still availablehttps://t.co/8TFwgUEGPA pic.twitter.com/r9kr6xXP2l
— FIFA Women's World Cup (@FIFAWWC) October 31, 2018
Back in February when FIFA released its schedule for the Women's World Cup, we fretted that FIFA was selling the women's game short by not including more large stadiums.
The semifinals and final will be played at Stade de Lyon, which holds 58,000. That's more than the 53,341 that saw the 2015 final between the U.S. and Japan, but far below the record of 90,185 set in the 1999 final between the U.S. and China. It seemed FIFA was taking the safe route instead of betting big on women's football.
This has proven to be the case as the ticket packages for the final and semifinal have sold out already — aided by their affordable prices.
While FIFA hasn't released details of who all bought the tickets, you can be sure plenty of Americans bought tickets. The U.S. has never failed to reach the semifinals of a World Cup. And the host nation, France, is among the favorites for next summer, so you know French fans copped their share, especially with the possibility of becoming the first country to hold the men's and women's World Cup trophies at the same time.
If you want to go to the Women's World Cup final next summer, you're not completely out of luck just yet. A limited number of single-match tickets will be available for the semifinals and final after the official draw, which will take place on Dec. 8.
In the meantime, you can still buy ticket packages for the other host cities outside of Lyon. Paris, while personally my least favorite French city, has an incredible deal where you can watch seven matches for as little as $82.
But if you want to wait to see where your favorite team is drawn first, that's completely understandable. Just understand that, based on the current pace of Women's World Cup ticket sales, matches may sell out quickly.


text:
export enum ErrorNotificationState{
  ERROR=<any>"ERROR",WARNING=<any>"WARNING",INFO=<any>"INFO"
}


Emo legends Dashboard Confessional and hometown heroes Tigers Jaw led the bill at the inaugural Alt 92.1 Snow Show on Sunday, Jan. 28 at the FM Kirby Center in Wilkes-Barre, Pa. In addition to the Chris Carrabba-led Dashboard and TJ, the bill also featured AJR and SYML.
Dashboard will release a new album, "Crooked Shadows," on Feb. Feb. 9. It's available for pre-order. On Feb. 21, the band will embark on an extensive tour, starting in Canada and winding through most of the U.S., including March 29 at Brooklyn Steel and April 2 at The Stone Pony in Asbury Park, N.J. You can find the full tour dates here.
Tigers Jaw also has a lively touring schedule, with its next show set for Feb. 22 in New Hampshire. The band will hit Harrisburg's The Capitol Room on Feb. 24 and hit markets throughout most of the country before traveling to Europe. The dates can be viewed here.


text:
As Nephila hits $10 billion, its contribution to KKR grows
7th February 2014 - Author: Artemis
Nephila Capital, the insurance-linked securities and reinsurance-linked investment markets largest asset manager, passed $10 billion of assets under management at the end of 2013, according to KKR & Co. L.P. (Kohlberg Kravis Roberts) executives.
Nephila Capital has not been raising capital as quickly as it could in the last year, with a number of its insurance linked investment strategies closed to new investors at times, but still it has managed to grow its assets under management by 25%, from around $8 billion at the end of 2012 to finish 2013 with over $10 billion under management.
The data was revealed by KKR & Co. L.P. executives during the private equity and buyout giants fourth-quarter earnings call yesterday. KKR bought a 25% stake in Nephila Capital just over a year ago and also revealed that its relationship with the ILS manager is now paying a growing contribution to the firm as well.
KKR co-chairman and co-CEO George Roberts said recently that the Nephila Capital deal was something that worked both ways, in that both KKR and Nephila could find benefits by working together to increase Nephila's assets under management.
Roberts said that KKR will benefit by helping Nephila Capital to grow and that it can take what Nephila Capital has achieved and, if the capital deployment opportunities can be found, introduce Nephila to some key investors that KKR has. Roberts also said that the relationship with Nephila is already generating meaningful earnings for KKR.
Some idea of the level of earnings contributed by Nephila Capital to KKR was also revealed in the earnings call yesterday. William Janetschek, CFO of KKR Management LLC, said that KKR saw a meaningful increase in incentive fees in the fourth-quarter, with Nephila Capital making a contribution to this.
Economic net income at KKR's public markets division was $73.3m during the fourth quarter, primarily driven by fee related earnings which includes the incentive fees earned from its stake in Nephila Capital. Total incentive fees in the fourth-quarter of 2012 were $12.35m which almost tripled to $36.695m in the fourth-quarter of 2013.
Scott Nuttall, Head of Global Capital & Asset Management Group and Principal at KKR, said that Nephila Capital accounted for about 25% of the firms fourth-quarter incentive fees, which suggests just over $9m for the quarter.
Nuttall said of Nephila and KKR's other similar initiatives; "Profits grew significantly, integration went smoothly and the businesses contributed to the origination and investment capabilities of the firm as a whole."
Nuttall is positive about the future of KKR's stake in Nephila Capital and clearly sees the opportunities to benefit each other increasing over time.
He explained; "The integration has gone really very well. The fund-raising teams have been working together. We're starting to see the pipelines continue to build in terms of jointly sourced opportunities. So I'd say it's still quite early days. And the good news is when you look at the backlog of potential fund-raising opportunities and you look at the product development ideas, I think there's a lot of good things ahead."
Janetschek explained how KKR expects to realise its incentive fees from its stake in Nephila; "Generally speaking, Nephila is going to realize its incentive fee once a year. So you might see small amount come through, through the first three-quarters if there are redemptions, but you're not going to see any significant activity. Again, you'll only see that pop if performance is there in the fourth-quarter."
Nuttall commented on Nephila's assets under management, saying that the ILS manager finished the year with 'a bit over $10 billion', adding that the firm had a very strong incentive fee quarter and year.
Then commenting on Nephila Capital alongside another similar investment KKR made (Prisma), Nuttall said; "Overall, frankly both are well ahead of our expectations in terms of their first year financial results, so the performance has been strong. Virtually all the assets across the strategies are above the high watermarks and off to a great start."
So, clearly KKR is realising real benefits from the stake it has in Nephila Capital and the beneficial mutual relationship between the firms. With Nephila Capital having passed the $10 billion of assets under management mark, the first ILS manager to reach that milestone, these real benefits are only going to increase and it will be interesting to see how the contribution KKR's stake in Nephila Capital makes grows over time.
Read our other recent article on KKR and Nephila Capital:
– KKR & Co. L.P. and Nephila Capital, the benefits flow both ways.
Tagged as: ILS funds, Insurance linked securities, investment manager, KKR, Kohlberg Kravis Roberts, nephila capital, Reinsurance linked investment
← Older article Cincinnati Financial on its expanded Skyline Re 2014 catastrophe bond
Cincinnati Financial on its expanded Skyline Re 2014 catastrophe bond
Cincinnati Financial Corporation, parent of Cincinnati Insurance Company the ceding insurer and sponsor of the recent Skyline Re Ltd. (Series 2014-1) catastrophe...


text:
\section{Introduction}
\label{sec:intro}
\vspace{-1mm}

Detecting interest points in images and matching them across views is a fundamental capability of many robotic systems. Tasks such as Structure-from-Motion (SfM)~\citep{agarwal2010bundle}, Visual Odometry (VO), and visual Simultaneous Localization and Mapping (SLAM)~\citep{cadena2016past} require salient keypoints to be detected and re-identified in diverse settings with strong invariance to lighting, viewpoint changes, and scale. Until recently, these tasks have relied on hand-engineered keypoint features~\citep{lowe1999object,rublee2011orb} with limited performance~\citep{sarlin2019coarse}. Deep learning has recently revolutionized many computer vision applications in the supervised setting~\citep{he2016deep,alp2018densepose,kirillov2019panoptic}. However, these methods rely on strong supervision in the form of ground-truth labels that are often expensive to acquire. Moreover, supervising interest point detection is challenging, as a human annotator cannot trivially identify salient regions in images that would allow their re-identification in diverse scenarios.
Inspired by recent approaches to keypoint learning~\citep{detone2018superpoint,christiansen2019unsuperpoint,revaud2019r2d2,bhowmik2020reinforced}, we propose an approach that exploits the temporal context in videos to learn an accurate and repeatable monocular keypoint detector, descriptor, and 2D-3D keypoint lifting function in a fully \textit{self-supervised} manner. We focus on the task of \textit{ego-motion estimation} and show that by combining geometry and appearance in a joint optimization framework we can identify depth-aware monocular keypoints that are particularly well suited for pose estimation. In addition, we demonstrate that our 3D keypoint estimator can be effectively integrated into a state-of-the-art visual tracking framework for accurate, long-range monocular visual odometry (see Figure~\ref{fig:intro-fig}).

\begin{figure}[t]
    \centering
     \includegraphics[width=0.95\columnwidth]{imgs/teaser2.pdf}
    \caption{\textbf{Self-Supervised 3D Keypoints for Robust Visual-Odometry.}~\textbf{Left:}~The illustration shows the proposed self-supervised 3D keypoints learned \textit{purely} from unlabeled monocular videos together with matched sparse 3D scene flow. Our method can effectively handle dynamic objects via outlier rejection thanks to structured pose estimation with our 3D keypoints.
    \textbf{Right:}~The proposed 3D keypoint estimator is integrated into a state-of-the-art visual tracking framework for accurate, scale-aware, long-range monocular visual odometry.}
    \label{fig:intro-fig}
\vspace{-3.5mm}
\end{figure}

Our \textbf{main contribution} is a fully self-supervised framework for the learning of depth-aware keypoint detection and description purely from unlabeled videos. Our novel formulation allows us to simultaneously learn keypoint detection, matching, and 3D lifting for robust visual ego-motion. \rares{Our \textbf{second contribution} lies in our principled use of the Orthogonal Procrustes algorithm to differentially regress an SE3 pose that is tightly coupled with the joint estimation of depth and keypoints from videos}. We show that by enforcing strong regularization in the form of sparse multi-view geometric constraints, the keypoint and depth networks strongly benefit from joint optimization in an end-to-end framework. Finally, in our \textbf{third contribution}, we show results comparable to state-of-the-art stereo long-term tracking by integrating our self-supervised, monocular and depth-aware keypoints into existing visual tracking frameworks such as Direct Sparse Odometry (DSO)~\citep{engel2017direct}.

\section{Related Work}
\label{sec:related}
\vspace{-2mm}

\textbf{Learning-based Methods for Keypoint Estimation.} Over the past two decades, handcrafted image features such as SIFT~\citep{lowe1999object} and ORB~\citep{rublee2011orb} have been the key enabler of visual SLAM~\citep{cadena2016past} and SfM applications~\citep{agarwal2010bundle}. More recently, learning-based keypoint detectors and descriptors have advanced the state-of-the-art performance on challenging benchmarks. While generating ground truth data is a tedious and expensive process,~\citet{detone2018superpoint} has shown that synthetic data can be used for supervising keypoints which can be transferred to real world scenes. Alternatively, SfM~\citep{yi2016lift} or two-view consistency~\citep{ono2018lf,suwajanakorn2018discovery} can been used for keypoint learning without any additional labels. Single images have also been successfully used to train generalizable keypoints, with explicit descriptor losses by~\citet{christiansen2019unsuperpoint} or without by~\citet{tang2019neural} . Additionally, descriptor discriminativeness can be learned, thus identifying high confidence matching regions~\citep{revaud2019r2d2}. Keypoints can also be learned in an end-to-end optimization for downstream tasks such as localization~\citep{Sarlin:etal:CVPR2019}, relative pose estimation~\citep{bhowmik2020reinforced,sarlin2020superglue} and 3D matching~\citep{bai2020d3feat}. Our work extends previous work~\citep{christiansen2019unsuperpoint,tang2019neural} by combining self-supervised keypoint learning with depth estimation in monocular videos and by using the temporal consistency and scene geometry to regress a robust and repeatable keypoint estimator.

\textbf{Learning-based Methods for Visual Odometry.} While many learning-based methods exist, we will focus our discussion on self-supervised methods. Recently,~\citet{godard2017unsupervised}  used stereo imagery to derive a photometric loss as proxy supervision to self-supervise a monocular depth network. This was extended to the generalized multi-view case by~\citet{zhou2017unsupervised}, leveraging SfM constraints to simultaneously learn depth and camera ego-motion from monocular image sequences. A number of end-to-end self-supervised methods have been proposed, using super-resolution~\citep{pillai2018superdepth}, learned~\citep{casser2018depth} or self-discovered object detectors~\citep{bian2019unsupervised}, two-stream rgb and depth networks~\citep{ambrus2019two}, optical flow~\citep{luo2019every,zhan2019visual,zhao2020towards}, modeling camera intrinsics~\citep{gordon2019depth}, using adversarial networks~\citep{feng2019sganvo}, etc. Additionally, a number of hybrid methods have been proposed that combine self-supervised learning of the depth network with traditional methods, for example using a robust estimator to compute the fundamental matrix and eliminate outliers~\citep{zhan2019visual,zhao2020towards} or by integrating learning~\citep{yang2018deep,yang2020d3vo}  with a state-of-the-art tracking method~\citep{engel2017direct}. Similar to~\citet{zhan2019visual,zhao2020towards}, we use traditional methods to take advantage of the model-based PnP solution and the inliers established to outfit a differentiable pose estimation module within the self-supervised 3D keypoint learning framework. Differentiable Procrustes for point cloud alignment has been used previously by~\citet{wang2019deep,choy2020deep}, with a focus on global registration as a supervised learning problem. \rares{In~\citep{murthy2019gradslam} RGB-D data is registered through differentiable nonlinear least squares, allowing gradient flow and enabling end-to-end differentiable SLAM.} In this paper we instead focus on self-supervised sparse keypoint detection and optimization for the task of ego-motion estimation, and show that our appearance-based depth-aware keypoints achieve superior results compared to other learning-based methods.
\section{Self-Supervised Depth-Aware Keypoint Learning}
\label{sec:ss_3d_keypoint}
\vspace{-1mm}

\input{latex/031procedure_notation.tex}

\input{latex/033procedure_pnp_pose.tex}

\input{latex/032procedure_learning.tex}
\subsection{Notation}
\label{subsec:notation}
\vspace{-2mm}

Our method works on sequences of monocular images, and we refer to $I_t$ as the target image and $I_C$ as the set of temporally adjacent context images, with $I_c \in I_C$.
We jointly learn two networks. First the \textbf{KeypointNet}~ $f_{k}: I \to  \left(\bm{p}, \bm{f}, \bm{s}\right)$ regresses $N$ \textit{image keypoints} consisting of positions $\bm{p} \in \mathbb{R}^{2\times N}$, descriptors $\bm{f} \in \mathbb{R}^{256\times N}$ and scores $\bm{s} \in \mathbb{R}^{N}$. Second, the \textbf{DepthNet}~ $f_D: I \to D$ predicts the \textit{scale-ambiguous} dense depth map $D$. We use $\bm{d} = D(\bm{p}) \in \mathbb{R}^{N}$ to denote the values of $D$ associated with the keypoint positions $\bm{p}=[\bm{u},\bm{v}]^{T}$. The \textbf{ego-motion estimator}~ $f_{x}(I_c,I_t)=X_{t \to c} = \begin{psmallmatrix}R & t\\ 0 & 1\end{psmallmatrix} \in \mathbb{SE}(3)$ predicts the relative 6-DoF rigid-body transformation between the target and context image. Throughout this manuscript, we use $\bm{p}$ to refer to sets of keypoints, while $p$ is used to refer to a single keypoint. Given two sets of detected keypoints, we define $\phi\left(\bm{f}_t,\bm{f}_c\right)$ as the \textbf{keypoint matching} function that computes keypoint correspondences via reciprocal matching in descriptor space:
\begin{equation} \label{eq:descriptor_matching}
\begin{gathered}
\phi\left(\bm{f}_t,\bm{f}_c\right) = \{\left(f_i,f_j\right) | \argmin_{j'}{\|f^2_i - f^2_{j'} \|_2} = j \land \argmin_{i'}{\|f^2_{i'} - f^2_j \|_2} = i \}
\end{gathered}
\end{equation}

with $f_i \in \bm{f}_t$ and $f_j \in \bm{f}_c$.
$\phi$ defines an association between the keypoint positions in the two images, which we denote by $\left[\bm{p}^{\phi}_t, \bm{p}^{\phi}_{c}\right]$. We use the correspondences $\left[\bm{p}^{\phi}_t, \bm{p}^{\phi}_c\right]$ and their corresponding predicted depths $\left[\bm{d}^{\phi}_t, \bm{d}^{\phi}_c\right]$ to estimate the rigid-body transformation $X_{t \to c}$ as described in detail in Section~\ref{subsec:pnp_pose}. Through $X_{t \to c}$, we define a joint optimization framework described in Section~\ref{subsec:learning}.
\subsection{Joint Self-Supervised Learning of Depth and Keypoints}
\label{subsec:learning}

We self-supervise the joint end-to-end learning of the keypoint and depth networks using a combination of a geometric keypoint loss $\mathcal{L}_{kpn}$, based on keypoint reprojection error, and a dense photometric loss $\mathcal{L}_{depth}$, based on the warped projection of $D_t$ in $I_c$: $\mathcal{L} = \mathcal{L}_{depth} + \alpha\mathcal{L}_{kpn}$.

\subsubsection{Keypoint Loss}

The total keypoint loss is composed of three terms: $\mathcal{L}_{kpn} = \mathcal{L}_{geom} + \beta_1\mathcal{L}_{desc} + \beta_2\mathcal{L}_{score}$.

\textbf{Geometric Loss.}~
Using $X_{t \to c}$ and $\bm{P}^{\phi}_t$, we compute the \emph{warped} keypoints from image $I_t$ to $I_c$ as:
\begin{equation} \label{eq:warped_keypoints}
\begin{gathered}
\mathbf{p}^{\phi}_{t\to c} = \pi\left( X_{t \to c}\bm{P}^{\phi}_t\right) = \pi(R\cdot \bm{P}^{\phi}_t + t)
\end{gathered}
\end{equation}
At training time, we aim to minimize the distance between the set of warped keypoints $\bm{p}^{\phi}_{t \to c}$ and the set of corresponding keypoints $\bm{p}^{\phi}_c$ obtained via descriptor matching (Eq.~\ref{eq:descriptor_matching}):
\begin{equation} \label{eq:geometric_loss}
\begin{gathered}
\mathcal{L}_{geom} = || \bm{p}^{\phi}_c - \bm{p}^{\phi}_{t \to c} ||_2
\end{gathered}
\end{equation}

\textbf{Descriptor Loss.}~
Following~\citet{tang2019neural}, we use nested hardest sample mining to self-supervise the keypoint descriptors between the two views. Given \textit{anchor} descriptors $\bm{f}_t$ from the target frame and their associated \textit{positive} descriptors $\bm{f}_{+}=\bm{f}^{\phi}_t$ in the source frame, we define the triplet loss:
\begin{equation} \label{eq:descriptor_loss}
\begin{gathered}
\mathcal{L}_{desc} = \max (0, \|\bm{f}, \bm{f}_{+}\|_2 - \|\bm{f}, \bm{f}_{-}\|_2 + m)~,
\end{gathered}
\end{equation}
where $\bm{f}_{-}$ is the hardest descriptor sample mined from $\bm{f}_s$ with margin $m$.

\textbf{Score Loss.}~
The score loss is introduced to identify reliable and repeatable keypoints in the matching process. In particular, we want to ensure that (i) the feature-pairs have consistent scores across matching views; and (ii) the network learns to predict high scores for good keypoints with low geometric error and strong repeatability. Following~\citet{christiansen2019unsuperpoint}, we achieve this by minimizing the squared distance between scores for each matched keypoint-pair, and minimizing (respectively maximizing) the average score of a matched keypoint-pair if the distance between the paired keypoints is greater (respectively smaller) than the average distance:
\begin{equation} \label{eq:score_loss}
\begin{gathered}
\mathcal{L}_{score} = \left[ \frac{(\bm{s}^{\phi}_t + \bm{s}^{\phi}_c)}{2}\cdot (\|(\bm{p}^{\phi}_{t \to c},\bm{p}^{\phi}_c\|_2- \bar{\bm{d}}) + (\bm{s}^{\phi}_t -\bm{s}^{\phi}_c)^2\right]~,
\end{gathered}
\end{equation}
where $\bm{s}^{\phi}_t$ and $\bm{s}^{\phi}_c$ are the scores of the target and context keypoints respectively, and $\bar{\bm{d}}$ is the average reprojection error of corresponding keypoints given by $\bar{\bm{d}}=\sum_{i}^{L}\frac{\|\left(p^{\phi}_{i \to c}\right)^2 - \left(p^{\phi}_i\right)^2\|_2}{L}$, and $L$ denotes the total number of keypoint pairs.

\subsubsection{Depth Loss}

The total depth loss is also composed of three terms: $\mathcal{L}_{depth} = \mathcal{L}_{photo} + \beta_3\mathcal{L}_{smooth} + \beta_4\mathcal{L}_{const}$.

\textbf{Photometric loss.}
Following~\citep{zhou2017unsupervised,godard2018digging,guizilini2019packnet}, we warp the estimated dense depth of the target image $D_t$ via the predicted ego-motion estimate $X_{t \to c}$ to the context frame $I_c$. Using~\citep{jaderberg2015spatial}, we synthesize $\hat{I}_t = I_c\left(q_{t\to c} \right)$ for all pixels $q_t \in I_t$, where $q_{t \to c}$ is the context pixel computed by warping the target pixel $q_t$ via Equation \ref{eq:warped_keypoints}. This operation is done via grid sampling with bilinear interpolation~\citep{jaderberg2015spatial} and thus is differentiable.
Following~\citep{zhou2017unsupervised,godard2018digging,jaderberg2015spatial}, we impose a dense photometric loss which consists of a structural similarity (SSIM) loss~\citep{wang2004image} (defined in Appendix~\ref{appendix:sec:SSIM}) and an L1 pixel-wise loss term:
\begin{equation} \label{eq:photo_loss}
\begin{gathered}
 \mathcal{L}_{photo}\left(I_t,\hat{I_t}\right) = \gamma~\frac{1 - \text{SSIM}\left(I_t,\hat{I_t}\right)}{2} + \left(1-\gamma\right)~| I_t - \hat{I_t} |~.
\end{gathered}
\end{equation}
In addition, we mask out static pixels which have a \textit{warped} photometric loss $\mathcal{L}_{photo}(I_t, \hat{I}_t)$ higher than their corresponding \textit{unwarped} photometric loss $\mathcal{L}_{photo}(I_t, I_{c})$, calculated using the original source image without view-synthesis as described in~\citep{godard2018digging}. Additionally, we employ a smoothness loss term $\mathcal{L}_{smooth}$ following~\citep{godard2017unsupervised} which we describe in detail in Appendix~\ref{appendinx:sec:depth_losses}.

\textbf{Depth Consistency.}~
While recovering scale-consistent depth is not a strict requirement for the proposed framework to learn 3D keypoints, scale-consistency is important for tasks that involve accurate ego-motion estimation~\cite{bian2019unsupervised,gordon2019depth}. To this end, we incorporate a depth consistency term that discourages scale-drift between dense depth predictions in adjacent frames:
\begin{equation}\label{eq:consistency_loss}
\begin{gathered}
\mathcal{L}_{const} = \frac{\|D_t(\bm{p}^{\phi}_t) - D_c(\bm{p}^{\phi}_c)\|}{D_t(\bm{p}^{\phi}_t) + D_c(\bm{p}^{\phi}_t)}
\end{gathered}
\end{equation}
Note that $\mathcal{L}_{const}$ is a sparse loss defined based on the correspondences $\left[\mathbf{p}^{\phi}_t, \mathbf{p}^{\phi}_c\right]$.

\subsection{Differentiable Pose Estimation from Depth-Aware Keypoints}
\label{subsec:pnp_pose}

\textbf{Pose Estimation via Perspective-n-Point.}
Using the estimated dense depth $D_t$ of the target image, we can compute the set $\bm{P}^{\phi}_t$ of 3D \emph{lifted} keypoints through the operation: $\bm{P}^{\phi}_t = \pi^{-1} (\bm{p}^{\phi}_t, \bm{d}^{\phi}_t)$ where $\bm{P}_t = [\bm{X}, \bm{Y}, \bm{Z}]^T \in \mathbb{R}^{3\times N}$ and $\pi(\cdot)$ is the standard pinhole camera projection model.
Using the keypoint correspondences $\left[\bm{p}^{\phi}_t, \bm{p}^{\phi}_c\right]$, we then have a 3D-2D correspondence set and can use the PnP algorithm~\cite{lepetit2009epnp} to compute the initial relative pose transformation $X^0_{t \to c}  = \begin{psmallmatrix}R_0 & t_0\\ 0 & 1\end{psmallmatrix}$ to geometrically match the keypoints in the target image to those in the context image. Specifically, we minimize:
\begin{equation} \label{eq:pnp}
E_{\psi}(X^{0}_{t \to c}) = \left\lVert \bm{p}^{\phi}_c - \pi\left( X^0_{t \to c} \cdot \bm{P}^{\phi}_t  \right)   \right\rVert_2~.
\end{equation}

The estimated relative pose $X^0_{t \to c}$ is obtained by minimizing the residual error in Equation~(\ref{eq:pnp}) using the Gauss-Newton (GN) method with RANSAC to ensure robustness to outliers (see Appendix~\ref{appendix:sec:pose_pnp} for details regarding this operation). This step allows us to compute the pose robustly, and yields an inlier set of correspondences $\left[\bm{p}^{\psi}_t, \bm{p}^{\psi}_c\right]$. However, this procedure is not differentiable with respect to the keypoint set used to estimate it.

\textbf{Residual Pose Correction via Orthogonal Procrustes.} To alleviate the limitation of traditional PnP and allow end-to-end learning, we show how the initial pose estimate can be used to derive a 3D loss based on 3D-3D correspondences. We note that~\citet{sheffer2020pnp} have recently proposed a differentiable version of the traditional PnP algorithm which combines deep learning with a model-based fine-tuning step. While this method has shown good results when trained with noisy correspondences, we follow instead the monocular direct method of~\citet{engel2017direct} that performs frame-to-keyframe tracking. Specifically, we lift the context keypoints $\bm{p}^{\psi}_c$ to 3D using the re-projected depth of the source keypoints via the initial pose estimate $X^{0}_{t \to c}$. This allows us to form a 3D residual that can be used to recover the pose in closed-form (for convenience we omit the $\psi$ superscript below):
\begin{equation} \label{eq:3d_icp_1}
\begin{gathered}
E_{OP}(X_{t \to c}) = {\| \bm{P}_c - X_{t \to c} \cdot \bm{P}_t \|}_2~, \\
\text{where}~\mathbf{P}_t=\pi^{-1}(\bm{p}_t, D_t(\bm{p}_t))~,~\bm{P}_c=\pi^{-1}(\bm{p}_c, \bm{d}_c)~,~\text{and}~\bm{d}_c = \left[ X^0_{t \to c} \cdot \bm{P}_t \right]_{z}~. \\
\end{gathered}
\end{equation}
\noindent The 3D residual above can be effectively minimized by estimating the rotation and translation separately using a closed-form solution on the established inlier set.
We first estimate the rotation by subtracting the means of the points and minimizing Eq.~\ref{eq:orthogonal_residual} by solving an SVD in closed-form (otherwise known as the Orthogonal Procrustes problem~\cite{zhang2000flexible}):
\begin{equation} \label{eq:orthogonal_residual}
E(R) = \| \bm{P}^*_c - R \cdot \bm{P}^*_t \|_2~, \quad
\text{where} \enskip ~\bm{P}^*_i = \bm{P}_i - \bm{\overline{P}}_i~,
\end{equation}
\vspace{-4mm}
\begin{equation} \label{eq:3d_icp_2}
U\Sigma V = \mathtt{SVD}\left(\sum {(\bm{P}^*_c)}^T (\bm{P}^*_t)\right)~, \enskip \text{where} \quad R = V U^T~.
\end{equation}
Once the rotation $R$ is computed, the translation $t$ can be directly recovered by minimizing $t = \bm{P}^*_c - R \bm{\cdot} \bm{P}^*_t~$. Thus, the gradients for the pose rotation and translation can be effectively propagated with respect to the lifted 3D keypoint locations, making the overall pose estimation fully-differentiable. The differentiable pose estimated using the 2D keypoints from the source image and 3D keypoints from the target image tightly couples keypoint and depth estimation, thereby allowing both predictions to be further optimized using the overall keypoint learning objective described next.
\section{Experiments}
\label{sec:experiments}

We evaluate our system on the \textit{KITTI}~\citep{geiger2013vision} dataset and report $t_{rel}$ - average translational RMSE drift (\%) on trajectories of length 100-800m, and $r_{rel}$ - average rotational RMSE drift (deg /100m) on trajectories of length 100-800m. All our $t_{rel}$ results are obtained after performing a single Sim(3) alignment step~\citep{grupp2017evo} w.r.t. the ground truth trajectories. We report results for the two main protocols used in related works. Most end-to-end learning based methods train on the KITTI sequences 00-08 and evaluate on seq. 09 and 10. These results are summarized in Table~\ref{table:kitti-odom-results-horizontal}, with more details in Appendices~\ref{appendix:sec:pose_ablation} and~\ref{appendix:sec:qualitative_odometry}. A number of methods including DVSO~\citep{yang2018deep} and D3VO~\citep{yang2020d3vo} train on the \textit{Eigen}~\citep{eigen2014depth} train split, which includes sequences 01, 02, 06, 08, 09 and 10; in this case the test sequences are 00, 03, 04, 05 and 07. We summarize our results for this protocol in Table~\ref{table:kitti-eigen-traj}. Additionally, we report in the standard depth evaluation metrics on the \textit{Eigen}~\citep{eigen2014depth} test split in Appendix~\ref{appendix:sec:depth_evaluation}.

To evaluate the performance of our kepoint detector and decriptor we use the \textit{HPatches}~\citep{balntas2017hpatches} dataset, which contains a set of 116 image sequences (illumination and viewpoint) for a total of 580 image pairs. We quantify detector performance through the \textit{Repeatability} and \textit{Localization Error} metrics and descriptor performance through the \textit{Correctness} and \textit{Matching Score} metrics (the exact definition of these metrics can be found in Appendix~\ref{appendix:keypoint_metrics}). For a fair comparison, we evaluate the results generated without applying Non-Maxima Suppression (NMS). Following related work~\citep{tang2019neural,christiansen2019unsuperpoint,detone2018superpoint}, we pre-train our \textit{KeypointNet} on the \textit{COCO}~\citep{lin2014microsoft} dataset, which contains $118k$ training images. We note that pretraining on COCO is self-supervised, as described in Sec.~\ref{subsec:training}.

\vspace{-2mm}
\subsection{Training}
\label{subsec:training}

We implement our networks in PyTorch~\citep{paszke2017automatic} using the ADAM optimizer~\citep{kingma2014adam}. We use $10^{-4}$ as the learning rate and train KeypointNet and DepthNet jointly for $50$ epochs with a batch size of $8$. We implement KeypointNet following~\citep{tang2019neural}, with the mention that we use an ImageNet pre-trained ResNet-18 backbone, which we find performs better than the reference architecture. We follow~\citep{godard2018digging} and implement DepthNet using an ImageNet~\citep{Deng09imagenet} pretrained ResNet-18 backbone along with a depth decoder that outputs inverse depth at 4 different resolution scales. However, at test-time, only the highest resolution scale is used for 2D-to-3D keypoint lifting. We describe our networks in detail in Appendix~\ref{appendix:network_diagrams}. We train on snippets of 3 images $\left(I_{t-\Delta t}, I_t, I_{t+ \Delta t}\right)$, for $\Delta t\in[1,2,4]$ with target image $I_t$ and images $\left(I_{t-\Delta t}, I_{t+\Delta t}\right)\in I_C$ as context images. Using the pair of target and context images we compute the losses as defined in Section~\ref{subsec:learning}. The dense \textit{photometric} loss is computed over the context $I_C$ as shown in Equation~\ref{eq:photo_loss}. In all our experiments we set $\alpha=0.1$, $\beta_1=\beta_2=1.0$, $\beta_3=\beta_4=0.1$, and $\gamma=0.85$. Additionally, starting from the target image $I_t$, we also perform Homography Adaptation similar to~\citep{tang2019neural}, e.g., translation, rotation, scaling, cropping and symmetric perspective transform and we apply per-pixel Gaussian noise, color jitter, and Gaussian blur to the images. \rares{Our method runs at 30fps on images of resolution 640x192 on a V100 GPU}.

\textbf{Pretrained baseline.}~We pretrain KeypointNet on \textit{COCO} using Homography Adaptation for $50$ epochs using a learning rate of $5 \cdot 10^{-4}$ which is halved after $40$ epochs. We refer to this as our baseline KeypointNet, and evaluate its performance in Table~\ref{table:results_descriptor}. To speed up convergence, we pretrain our DepthNet on the KITTI training sequences using the method described in~\citep{godard2018digging}. We train for 200 epochs with a learning rate of $10^{-4}$ which is decayed by $0.5$ every $40$ epochs. We refer to this as our baseline DepthNet, and we evaluate its performance in the ablative evaluations in Table~\ref{table:ablative}.
\input{latex/041dso}

\subsection{Direct Sparse Odometry (DSO) Integration}
\label{sec:ds-dso}

To evaluate tracking performance, we integrate our self-supervised, depth-aware keypoints into the DSO framework of~\citet{engel2017direct}, and we show that we are able to achieve long-term tracking results which are especially on par with stereo methods such as DVSO~\citep{yang2018deep} or ORB-SLAM2~\citep{mur2017orb}. Unlike other monocular visual odometry approaches, the superior keypoint matching and stable 3D lifting performance of our proposed method allows us to bootstrap the tracking system, rejecting false matches and outliers and avoiding significant scale-drift. Our integration is built on top of the windowed sparse direct bundle adjustment formulation of DSO. Specifically, we improve depth-initialization of keyframes in the original DSO implmenetation by using the depth estimated through our proposed self-supervised 3D keypoints. In addition, we modify the hand-engineered direct semi-dense tracking component to use the proposed sparse and robust learned keypoint-based method introduced in this work. We describe the DSO integration in more details in Appendix~\ref{appendix:sec:ds-dso}.

\subsection{ORB-SLAM Integration}
\label{sec:orb-slam}

\subsection{Visual Odometry Performance}
\label{subsec:vo-results}

\input{latex/results/kitti_eigen_traj_results.tex}

\rares{We summarize our visual odometry results and comparisons with state-of-the-art methods in Table~\ref{table:kitti-eigen-traj}, and we note that our method outperforms all other monocular-trained methods, particularly in the $t_{rel}$ metric. For $r_{rel}$ we note that~\citep{ambrus2019two} achieves better results on some sequences, which we attribute to difficulty in matching keypoints in those settings, however our $r_{rel}$ is superior on average over the test sequences.} Our method also outperforms stereo-trained methods except for DVSO~\citep{yang2018deep} and D3VO~\citep{yang2020d3vo}. However, we emphasize that while both~\citep{yang2018deep,yang2020d3vo} are trained from a wide-baseline stereo setup which provides a strong prior for outlier rejection, our system is trained in a self-supervised manner purely relying on monocular videos - a significantly harder problem. The results indicate that when integrated with tracking frameworks such as DSO our depth-aware keypoints provide superior matching performance that even rivals state-of-the-art methods trained on stereo imagery.

\input{latex/results/kitti_odom_results.tex}

Additionally, we report our results when training on KITTI sequences 00-08 in  Table~\ref{table:kitti-odom-results-horizontal}. We note that our method outperforms all other monocular methods except for DF-VO~\citep{zhan2019visual} and~\citep{zhao2020towards} for $r_{rel}$. Both~\citep{zhao2020towards,zhan2019visual} heavily rely on optical-flow and RANSAC-based essential/fundamental matrix computation and scale-factor recovery. Recall that our method uses PnP (as described in Section~\ref{subsec:pnp_pose}) for pose estimation. When comparing with the PnP-based version DF-VO~\citep{zhan2019visual} (DF-VO PnP in Table~\ref{table:kitti-odom-results-horizontal}), we observe that our method performs much better, which we attribute to the direct optimization of sparse 2D-3D keypoints, as opposed to~\citep{zhan2019visual} which relies purely on dense optical flow. \rares{Finally, we record that our method performs worse on sequence 01 compared to the other sequences. In this sequence the vehicle is driving at high speeds on the highway and the image contains a high number of sky pixels, making this a challenging environment for feature detection and matching; we note that a number of other methods also perform poorly in this setting~\citep{ambrus2019two,li2017undeepvo,pillai2018superdepth,zhan2019visual}.}

\subsection{Ablation Study}
\label{subsec:ablative_study}
We summarize our ablative analysis in Table~\ref{table:ablative}. Our baseline - KeypointNet pre-trained on COCO and DepthNet trained on KITTI, but the two are not optimized together - shows superior results compared to most monocular methods (see Tables~\ref{table:kitti-eigen-traj} and~\ref{table:kitti-odom-results-horizontal}), thus motivating our approach of combining keypoints and depth in a self-supervised learning framework.  We notice a significant improvement when training the two networks together (\textit{Row 2: Ours $-$ Diff Pose}). Adding the differential pose estimation (\textit{Row 5: Ours }) further improves the performance of our system for the $t_{rel}$ metric; we note that the $r_{rel}$ test metric does not improve, mostly due to an error in Sequence $04$ (please refer to Appendix~\ref{appendix:sec:pose_ablation} for detailed results). We further ablate the KeypointNet (\textit{Row 3: Ours $-$ KPN trained}) - i.e., we estimate the ego-motion using the DepthNet after training together with the KeypointNet, but we use the original KeypointNet trained only on COCO. We perform a similar experiment ablating the trained DepthNet (\textit{Row 4: Ours $-$ DN trained}). Finally, when integrated in DSO (\textit{Row 6: Ours DSO}) our results improve significantly, which we attribute to the robustness of our features to scene appearance and geometry. We provide more details in Appendix~\ref{appendix:sec:pose_ablation}, and emphasize that all our experiments, including pretraining, are done in a self-supervised fashion, without any supervision.

\input{latex/results/KITTI_traj_ablation.tex}

\subsection{Keypoint Detector and Descriptor Performance}
\label{subsec:keypoint-results}

Table~\ref{table:results_descriptor} shows the performance of our method on HPatches~\citep{balntas2017hpatches}. We note that by improving the network capacity and using a \textit{ResNet-18} architecture, our baseline outperforms all classical as well as learning-based methods (recall that our KeypointNet baseline is trained only on COCO, and can thus be directly compared to these methods). Further, we note similar metrics between our baseline method and our method after training on KITTI (\textit{KeypointNet baseline vs KeypointNet}). We conclude that by training with the proposed losses in a joint self-supervised framework, our keypoints have gained geometric understanding and can be used for state-of-the-art ego-motion estimation while at the same time retaining their robustness to illumination and viewpoint changes.

\input{latex/results/hpatches.tex}

\section{Conclusion}
\label{sec:conclusion}
\vspace{-2mm}
In this paper, we proposed a fully self-supervised framework for depth-aware keypoint learning from unlabeled monocular videos by incorporating a novel differentiable pose estimation module that simultaneously optimizes the keypoints and their depths in a Structure-from-Motion setting.
Unlike existing learned keypoint methods that employ only homography adaptation, we exploit the temporal context in videos to further boost the repeatability and matching performance of our proposed keypoint network. The resulting 3D keypoints and associated descriptors exhibit superior performance compared to all other traditional and learned methods, and are also able to learn from realistic non-planar 3D scenes.
Finally, we show how our proposed network can be integrated with a monocular visual odometry system to achieve accurate, scale-aware, long-term tracking results that are on par with state-of-the-art stereo-methods.
\section{Supplementary Materials}
\section{Architecture Diagram}
\label{appendix:network_diagrams}
\textbf{ResNet18-DepthNet.} we provide a detailed description of our DepthNet architecture in Table~\ref{table:depthnet-arch}. Note that we follow~\citet{godard2017unsupervised} and use a \textit{ResNet18} encoder followed by a decoder which outputs inverse depth at 4 scales.

\begin{table}[!h]
\small
\renewcommand{\arraystretch}{1.1}
\centering
\begin{tabular}{l|l|c|c}
\toprule
& \textbf{Layer Description} & \textbf{K} & \textbf{Output Tensor Dim.} \\
\toprule
\#0 & Input RGB image & & 3$\times$H$\times$W \\
\midrule
\midrule
\multicolumn{4}{c}{\textbf{ResidualBlock}} \\ \hline
& Conv2d + BatchNorm + ReLU & 3 &  \\
& Conv2d + BatchNorm        & 3 &  \\
\midrule
\multicolumn{4}{c}{\textbf{Depth Encoder}} \\ \hline
\#1  & Conv2d (S2) + BatchNorm + ReLU & 7 &  64$\times$H/2$\times$W/2 \\
\#2  & Conv2d + BatchNorm + ReLU            & 3 &  64$\times$H/2$\times$W/2 \\
\#3  & ResidualBlock (\#2)  x2              & - &  64$\times$H/2$\times$W/2 \\
\#4  & Max. Pooling ($\times$1/2)          & 3 &  64$\times$H/4$\times$W/4 \\
\#5  & ResidualBlock (\#3 + \$2) x2         & - & 128$\times$H/4$\times$W/4 \\
\#6  & Max. Pooling ($\times$1/2)          & 3 & 128$\times$H/8$\times$W/8 \\
\#7  & ResidualBlock (\#4 + \#3) x2         & - & 256$\times$H/8$\times$W/8 \\
\#8  & Max. Pooling ($\times$1/2)          & 3 & 256$\times$H/16$\times$W/16 \\
\#9  & ResidualBlock (\#5 + \#4) x2         & - & 512$\times$H/16$\times$W/16 \\
\midrule
\multicolumn{4}{c}{\textbf{Depth Decoder}} \\
\midrule
\#10 & Conv2D + ELU (\#9)                                   & 3 & 128$\times$H/16$\times$W/16 \\
\#11 & Conv2D + Upsample (\#10)                             & 3 & 128$\times$H/8$\times$W/8  \\
\textbf{\#12} & Conv2D + Sigmoid                            & 3 & 1$\times$H/8$\times$W/8 \\
\#13 & Conv2D + ELU                                         & 3 & 64$\times$H/8$\times$W/8 \\
\#14 & Conv2D + Upsample(\#7 $\oplus$ \#13)                 & 3 & 64$\times$H/4$\times$W/4  \\
\textbf{\#15} & Conv2D + Sigmoid                            & 3 & 1$\times$H/8$\times$W/8 \\
\#16 & Conv2D + ELU                                         & 3 & 32$\times$H/4$\times$W/4 \\
\#17 & Conv2D + Upsample (\#5 $\oplus$ \#16)                & 3 & 32$\times$H/2$\times$W/2  \\
\textbf{\#18} & Conv2D + Sigmoid                            & 3 & 1$\times$H/8$\times$W/8 \\
\#19 & Conv2D + ELU                                         & 3 & 16$\times$H/2$\times$W/2 \\
\#20 & Conv2D + Upsample (\#3 $\oplus$ \#19)                & 3 & 16$\times$H$\times$W  \\
\textbf{\#21} & Conv2D + Sigmoid                            & 3 & 1$\times$H$\times$W \\
\bottomrule
\end{tabular}
\caption{DepthNet diagram. Line numbers in bold indicate output inverse depth layer scales. \emph{Upsample} is a nearest-neighbor interpolation operation that doubles the spatial dimensions of the input tensor. $\oplus$ denotes feature concatenation for skip connections.}
\label{table:depthnet-arch}
\end{table}

\textbf{ResNet18 KeypointNet.} Table~\ref{table:keypointnet_arch} details the network architecture of our KeypointNet. We follow~\cite{tang2019neural} but change the network encoder and use a \textit{ResNet18} architecture instead, which we found to perform better.

\begin{table}[!h]
\small
\renewcommand{\arraystretch}{1.1}
\centering
\begin{tabular}{l|l|c|c}
\toprule
& \textbf{Layer Description} & \textbf{K} & \textbf{Output Tensor Dim.} \\
\toprule
\#0 & Input RGB image & & 3$\times$H$\times$W \\
\midrule
\midrule
\multicolumn{4}{c}{\textbf{ResidualBlock}} \\ \hline
& Conv2d + BatchNorm + ReLU & 3 &  \\
& Conv2d + BatchNorm        & 3 &  \\
\midrule
\multicolumn{4}{c}{\textbf{KeyPoint Encoder}} \\ \hline
\#1  & Conv2d (S2) + BatchNorm + ReLU & 7 &  64$\times$H/2$\times$W/2 \\
\#2  & Conv2d + BatchNorm + ReLU            & 3 &  64$\times$H/2$\times$W/2 \\
\#3  & ResidualBlock (\#2)  x2              & - &  64$\times$H/2$\times$W/2 \\
\#4  & Max. Pooling ($\times$1/2)          & 3 &  64$\times$H/4$\times$W/4 \\
\#5  & ResidualBlock (\#3 + \$2) x2         & - & 128$\times$H/4$\times$W/4 \\
\#6  & Max. Pooling ($\times$1/2)          & 3 & 128$\times$H/8$\times$W/8 \\
\#7  & ResidualBlock (\#4 + \#3) x2         & - & 256$\times$H/8$\times$W/8 \\
\#8  & Max. Pooling ($\times$1/2)          & 3 & 256$\times$H/16$\times$W/16 \\
\#9  & ResidualBlock (\#5 + \#4) x2         & - & 512$\times$H/16$\times$W/16 \\
\midrule
\multicolumn{4}{c}{\textbf{KeyPoint Decoder}} \\
\midrule
\#10 & Conv2D + BatchNorm + LReLU (\#9)                     & 3 & 256$\times$H/16$\times$W/16 \\
\#11 & Conv2D + Upsample (\#10)                             & 3 & 256$\times$H/8$\times$W/8  \\
\#12 & Conv2D + BatchNorm + LReLU                           & 3 & 256$\times$H/8$\times$W/8 \\
\#13 & Conv2D + Upsample(\#7 $\oplus$ \#12)                 & 3 & 128$\times$H/4$\times$W/4  \\
\#14 & Conv2D + BatchNorm + LReLU                           & 3 & 128$\times$H/4$\times$W/4 \\
\#15 & Conv2D + Upsample (\#5 $\oplus$ \#14)                & 3 & 64$\times$H/2$\times$W/2  \\
\#16 & Conv2D + BatchNorm + LReLU                           & 3 & 64$\times$H/2$\times$W/2 \\
\midrule
\midrule
\multicolumn{4}{c}{\textbf{Score Head}} \\
\midrule
\#12 & Conv2d + BatchNorm + LReLU (\#12) & 3 & 256$\times$H/8$\times$W/8 \\
\#13 & Conv2d + Sigmoid                    & 3 & 1$\times$H/8$\times$W/8 \\
\midrule
\multicolumn{4}{c}{\textbf{Location Head}} \\
\midrule
\#14 & Conv2d + BatchNorm + LReLU (\#12) & 3 & 256$\times$H/8$\times$W/8 \\
\#15 & Conv2d + Tan. Harmonic              & 3 & 2$\times$H/8$\times$W/8 \\
\midrule
\multicolumn{4}{c}{\textbf{Descriptor Head}} \\
\midrule
\#16  & Conv2d + BatchNorm + LReLU (\#16) & 3 & 64$\times$H/2$\times$W/2 \\
\#17  & Conv2d                          & 3 & 64$\times$H/2$\times$W/2 \\
\bottomrule
\end{tabular}
\caption{KeypointNet diagram. \emph{Upsample} is a nearest-neighbor interpolation operation that doubles the spatial dimensions of the input tensor. $\oplus$ denotes feature concatenation for skip connections.}
\label{table:keypointnet_arch}
\end{table}

\section{Video}
\label{sec:video}

The accompanying video presents our contributions and demonstrates the long-term visual odometry accuracy obtained by our method on a video sequence. The main panel (top, center) shows our real-time semi-dense reconstruction along with the vehicle trajectory. The bottom panel shows (from left to right): the flow of inlier keypoints, color-coded based on depth; the estimated monocular depth; and finally, the number of matched keypoints versus the number tracked inliers.

\section{Dense Depth Evaluation}
\label{appendix:sec:depth_evaluation}
\input{latex/results/depth_results.tex}

We perform a qualitative evaluation of our DepthNet on the KITTI dataset, specifically on the Eigen~\cite{eigen2014depth} test split, and report the numbers in Table~\ref{table:depth-accuracy}. We also include the numbers reported by~\cite{godard2018digging} and note that our \textit{DepthNet baseline} numbers are on par with those of~\cite{godard2018digging} (which correspond to row \textit{1. Baseline} of Table 3 in the main text). Table~\ref{table:depth-accuracy} also shows our numbers after fine-tuning the DepthNet and KeypointNet through the proposed \textit{Multi-View Adaptation} method (which correspond to row \textit{5. Ours} of Table 3 in the main text). We note a slight decrease in the \textit{Abs Rel} and \textit{Sq Rel} metrics, but otherwise the numbers are within error margin with respect to our baseline. These results provide an important sanity check: as the main focus of this work is sparse, depth-aware keypoint learning, we don't expect to see much variation when performing dense depth evaluation. We mention that sparsely evaluating the depth using the keypoints regressed by our method is not feasible using the depth available in the KITTI dataset: even using the denser depth maps provided by~\cite{uhrig2017sparsity}, only about $10\%$ of our keypoints have valid depths in the ground truth maps, which amounts to a very small number of points ($<50$) per image.

\input{latex/073suppmat_dso}

\section{Depth losses}
\label{appendinx:sec:depth_losses}
\textbf{Depth Smoothness Loss.}~
In order to regularize the depth in texture-less low-image gradient regions, we use an edge-aware loss term similar to~\cite{godard2017unsupervised}:
\begin{align}
\mathcal{L}_{smooth} = | \delta_x \hat{D}_t | e^{-|\delta_x I_t|} + | \delta_y \hat{D}_t | e^{-|\delta_y I_t|}~.
\label{eq:loss-disp-smoothness}
\end{align}

\textbf{Structural Similarity (SSIM) loss.}
\label{appendix:sec:SSIM}
\input{latex/071suppmat_ssim.tex}

\section{Pose Estimation}
\label{appendix:sec:pose_pnp}
\input{latex/072suppmat_pnp_pose.tex}

\section{Detailed Results for the Pose Ablation Study }
\label{appendix:sec:pose_ablation}

Table~\ref{table:kitti-eigen-traj-appendix} provides detailed results on all the KITTI odometry sequences for each entry  of our ablation study (Table 3 of the main text). We note that (i) the proposed contributions - i.e. row 2 vs row 1 and row 5 vs row 1 consistently improve over the baseline; and that (ii) by swapping out the KeypointNet or DepthNet trained using the proposed method with their baseline counterparts (rows 3 and 4) results in worse performance for both the $t_{rel}$ and $r_{rel}$ metrics. Thus we conclude that the proposed training procedure along with the differentiable pose component improves both the DepthNet and KeypointNet for the task of Visual Odometry.

\input{latex/results/appendix_traj_ablation_results.tex}

\section{Additional KITTI odometry results}
\label{appendix:sec:pose_kitti_odom}

We show additional comparisons with state-of-the-art methods in Table~\ref{table:appendix-kitti-odom-results-horizontal}, complementing our results from Table 2 of the main text. Additionally, we present detailed results of our method on all the KITTI odometry sequences in Table~\ref{table:supp-kitti-odom-results-ours-detailed}.

\input{latex/results/supp_kitti_odom_results}

\input{latex/results/supp_kitti_odom_ours_detailed}

\section{Qualitative results on KITTI odometry sequences}
\label{appendix:sec:qualitative_odometry}

We show qualitative results of our method in Figure~\ref{fig:trajectory_evaluation}, noting that our DSO results accurately follow the ground truth trajectory with minimal scale drift.

\begin{figure}[!t]
\centering
\resizebox{\hsize}{!}{
	\includegraphics[width=0.45\columnwidth]{imgs/seqs/03.png} \includegraphics[width=0.45\columnwidth]{imgs/seqs/04.png}}
\resizebox{\hsize}{!}{
    \includegraphics[width=0.45\columnwidth]{imgs/seqs/05.png}
    \includegraphics[width=0.45\columnwidth]{imgs/seqs/07.png}
}
\caption{\textbf{Qualitative trajectory estimation results on the KITTI Odometry Seq.~03,~04,~05 and 07}. We compare trajectory estimation results obtained via hand-engineered keypoint matching methods against our depth-aware learned keypoint matching, with a common visual odometry backend such as DSO. As illustrated in the figure, our self-supervised method is able to accurately and robustly track stable keypoints for the task of long-term trajectory estimation.}
\label{fig:trajectory_evaluation}
\end{figure}

\section{Keypoint Detector and Descriptor Evaluation Metrics}
\label{appendix:keypoint_metrics}

We follow~\cite{detone2018superpoint} and use the \textit{Repeatability} and \textit{Localization Error} metrics to estimate keypoint performance and \textit{Homography Accuracy} and \textit{Matching Score} matrics to estimate descriptor performance. We note that for all metrics we used a distance threshold of $3$. For the Homography estimation, consistent with other reported methods, we used $300$ keypoints with the highest scores. Similarly, for the frame to frame tracking we selected $480$ keypoints to estimate the relative pose.

\textbf{Repeatability} is computed as the ratio of correctly associated keypoints after warping onto the target frame. We consider a warped keypoint correctly associated if the nearest keypoint in the target frame (based on Euclidean distance) is below a certain threshold.

\textbf{Localization Error} is computed as the average Euclidean distance between warped and associated keypoints.

\textbf{Homography Accuracy} To compute the homography between two images we perform reciprocal descriptor matching and we used OpenCV's \textit{findHomography} method with RANSAC, with a maximum of 5000 iterations and error threshold 3. To compute the Homography Accuracy we compare the estimated homography with the ground truth homography. Specifically we warp the image corners of the original image onto the target image using both the estimated homography and the ground truth homography, and we compute the average distance between the two sets of warped image corners, noting whether the average distance is below a certain threshold.

\textbf{Matching Score} is computed as the ratio between successful keypoint associations between the two images, with the association being performed using Euclidean distance in descriptor space.

\section{Qualitative Results on HPatches}
\label{sec:qualitative_hpatches}

Figure~\ref{fig:qualitative_hpatches} shows qualitative examples of our keypoints and matches on image pairs from the HPatches dataset~\cite{balntas2017hpatches}.

\begin{figure}[!h]
	\centering
	\includegraphics[width=0.9\textwidth]{imgs/hpatches/q1.png}\\
	\includegraphics[width=0.9\textwidth]{imgs/hpatches/q2.png}\\
	\includegraphics[width=0.9\textwidth]{imgs/hpatches/q3.png}

	\caption{Qualitative matching results of our method on the HPatches dataset~\cite{balntas2017hpatches}.}
	\label{fig:qualitative_hpatches}
\end{figure}

\section{Direct Sparse Odometry (DSO) Integration}
\label{appendix:sec:ds-dso}

In this section, we explain how the fully self-supervised depth-aware keypoint network can be incorporated as the front-end into a visual SLAM framework.
We show that by integrating our method into a state-of-the-art monocular visual tracking framework such as DSO~\cite{engel2017direct}, we are able to achieve high accuracy long-term tracking results as reported in Table 1 of the main paper.

Unlike other monocular visual odometry approaches, the superior keypoint matching and stable 3D lifting performance of our proposed method allows us to bootstrap the tracking system, rejecting  false  matches  and outliers and avoiding significant scale-drift.

Figure~\ref{fig:ds_dso} shows the whole pipeline of our Deep Semi-Direct Sparse Odometry (DS-DSO) system, which is built on top of the windowed sparse direct bundle adjustment formulation of DSO.
As illustrated, we improve depth-initialization of keyframes in the original DSO implmenetation by using the depth estimated through our proposed self-supervised 3D keypoint network. In addition, we modify the hand-engineered direct semi-dense tracking component
to our proposed sparse and robust learned keypoint-based method introduced in this work.

 \begin{comment} {
In the windowed-optimization module,
as in~\cite{engel2017direct}, we define the photometric error of a \textit{DSO point} $\mathbf{q}$ in an reference frame $\mathtt{I}_i$,
observed in a target frame $\mathtt{I}_j$,
as the weighted Sum of Squared Difference (SSD) over a small neighborhood of pixels $N_\mathbf{q}$:

\begin{equation}
E_{\mathbf{q}_j} :=  {\sum_{\mathbf{q} \in N_\mathbf{q}}} w_{\mathbf{q}} {\left\lVert (\mathtt{I}_j[\mathbf{q}_j] - b_j) -  \frac{t_j e^{a_j}}{t_i e^{a_i}} (\mathtt{I}_i[\mathbf{q}] - b_i)  \right\rVert}_{\gamma}~,
\label{eq:pnp}
\end{equation}

\noindent
where $\mathbf{q}_j$ is the projected point in the target frame;
$t_i$ and $t_j$ are the exposure times;
$a_i$, $a_j$, $b_i$, $b_j$ are the photometric parameters of the frame $i$ and $j$;
and $||.||_{\gamma}$ is the Huber norm.
We also apply the gradient-dependent weight $w_{\mathbf{q}}$ given in~\cite{engel2017direct}.

The full photometric error over all frames and points in the local window is defined as:

\begin{equation}
E_{DSO} :=  {\sum_{i \in F}} {\sum_{\mathbf{q} \in Q_i}} {\sum_{j \in obs(\mathbf{q})}} E_{\mathbf{q}_j}~,
\label{eq:pnp}
\end{equation}

\noindent
where $i$ runs over all frames $F$, $\mathbf{q}$ over all points $Q_i$, and $j$ over all frames where $\mathbf{q}$ is observed.
}
\end{comment}

redpajama_set_name: RedPajamaArXiv


text:
Connect with Us facebook twitter RSS
In the Church
Global Nature
Practicing Faith
UMC Future
Advocating Justice
Love Justice Directory
Views from a Ridge
Christy Thomas
Perspectives Archive
GC Archive
Be Part of Our Mission
Home In the Church Superintendents Discuss Legislation Affecting Clergy Appointments, Pensions
Superintendents Discuss Legislation Affecting Clergy Appointments, Pensions
by Vicki Brown
GBHEM Photo
Legislation Discussed
Shown are Julie Love from Kentucky Annual Conference and Dale Jones from the General Board of Pension and Health Benefits.
Bishops or district superintendents can initiate part-time appointments for clergy, and annual conferences can choose not to pay pension benefits for part-time clergy under legislation passed by General Conference 2012.
"Our proposal to General Conference removed pension benefits for quarter-time clergy. General Conference apparently felt that we did not go far enough and made pension benefits mandatory only for full-time clergy," said Dale Jones, managing director of Education and Plan Sponsor Relations, General Board of Pension and Health Benefits.
Staff from United Methodist boards and agencies discussed General Conference 2012 legislation that affects the work of district superintendents and directors of connectional ministries Wednesday morning during the orientation for new DSes and DCMs at Lake Junaluska, N.C. The orientation is sponsored each year by the Council of Bishops, the General Board of Higher Education and Ministry, and the General Board of Discipleship. This year, 64 new district superintendents and four new directors of connectional ministry are attending.
The Rev. Gwen Purushotham, associate general secretary of GBHEM's Division of Ordained Ministry, and Jones both spoke about legislation revising the specific responsibilities for DSes listed in The Book of Discipline.
The legislation, which came from the Church Systems Task Force, condensed six paragraphs into one, retaining the same basic functions but removing many specific detailed requirements, with the goal of making the DS the chief missional strategist.
However, Purushotham cautioned that many of the specific responsibilities no longer listed in the Discipline must still be carried out by someone in the district -- for instance, making sure that local churches have required audits and evaluations. She said those responsibilities still fall to the DS even if not explicitly spelled out.
The Rev. Meg Lassiat, GBHEM's director of Candidacy, Mentoring, and Conference Relations, said that in conjunction with the removal of security of appointment for elders and associate members, General Conference approved legislation adding a new condition for the status of transitional leave. This would apply to associate members, provisional or full connection elders for whom "a missional appointment is not available."
She noted that the Judicial Council is considering the constitutionality of the removal of security of appointment in their October session.
There is also a new voluntary transition program for clergy choosing to leave ordained ministry. Jones said that legislation provides for severance pay and health insurance for up to six months, depending on length of service.
The panel was questioned about what changes the boards and agencies are making that reflect deep and adaptive change in the wake of Judicial Council's ruling that the church restructuring passed by General Conference was not constitutional.
Darryl Stephens of the General Council on the Status and Role of Women, said COSROW is now responsible for addressing inequities globally, not just in the United States, but with the same staff and a lower budget. He said that was adaptive change.
Karen Greenwaldt, GBOD's general secretary, said that before General Conference, the general secretaries had been finding ways to work together and streamline services.
"We have ramped that up," she said. One example is an effort to create a UM portal where everything churches need from the boards and agencies will be available in one place. And she said the role of agency staff now is to "find the bright spots of ministry and shine a light on that."
The Rev. Janice Virtue, executive director for Leadership Development at the United Methodist Foundation of Western North Carolina, told the group that change does not mean wholesale change in the DNA of United Methodists.
"The key is that small changes lead to incredible adaptions. Keep disturbing the status quo. . . . Change will never take place because somebody or some conference says it should. Real change happens when hearts and minds and habits are changed," she said, adding that it takes time for those changes to really take root.
Bishop Elaine Stanovsky, resident bishop of the new Mountain Sky Area created from the Rocky Mountain and Yellowstone Conferences, said some bishops are going to try to initiate conversations about how the bishops who serve on the boards of directors of general agencies can use those positions to carry forward the work from the Call to Action report.
"That doesn't require legislation, it just requires cooperation," she said.
The Rev. Anita Wood, GBHEM's director of Provisional Membership, Deacon Support and Certification in Specialized Ministries, said General Conference passed legislation that deacons will now be ordained to "Word, Service, Compassion and Justice" and there will be a single Disciplinary paragraph (¶331) for everything related to appointments for deacons (new paragraph 331.) Formerly, deacons were ordained to Word and Service.
Other legislation of note included:
Administrative and judicial complaint revisons
The $7 million Young Clergy Initiative Fund
Legislation that any changes in a church incorporation must be approved in writing by the pastor and the DS. Rick Rettberg, general counsel of the General Council on Finance and Administration, said that stemmed from situations in which some churches had modified their incorporation papers to remove references to The United Methodist Church, then attempted to leave the denomination and take the property with them.
Julie Love, director of Connectional Ministries for the Kentucky Annual Conference, reported that a change in ¶610.1 will allow annual conferences the flexibility to design their conference and district structures in ways that best support the mission of making disciples of Jesus Christ in an increasingly diverse global community and that place secondary any prescribed structure, except for the mandated entities.
Vicki Brown is associate editor and writer, Office of Interpretation, General Board of Higher Education and Ministry.
United Methodist clergy 2012 General Conference
proper use of words
When blogging please learn the difference between affect and effect.
Will more than 8 years ago
As we face challenging times, United Methodist Insight welcomes your financial support to bring you news and views of The United Methodist Church in a world seeking a new future.
TEXT your contribution to 84-321 with "(amt) Insight" via your smartphone or go to https://ststephenumctx.churchcenter.com/giving/to/um-insight Your additional contribution of the 2.2% processing fee will make your donation go farther.
Make CHECKS payable to our sponsoring congregation, St. Stephen UMC, and write "UM Insight" on the memo line. Then mail to United Methodist Insight, c/o St. Stephen United Methodist Church, 2520 Oates Drive, Mesquite, TX 75150. Thank you!
A forum for discerning God's future for The United Methodist Church, hosted by St. Stephen UMC, Mesquite, TX. We do not use cookies to collect user data.


text:
NISEKO Grand HIRAFU & Friends
Ryunosuke Ohkoshi
Gen Sasaki
Yoichi Watanabe
Takanori Nagata
Hiromi Tatsumi
Tsutomu Nakata
Tenra Katsuno
Koutaro Minamiyama
Wakana Hama
Yamasemi-kun & Kawasemi-kun
Born in Kutchan, Hokkaido, he is currently affiliated with Tokyu Resort Service.
As a young boy, Ohkoshi went to watch the Ski Mogul contest held at his local resort (Niseko Grand HIRAFU) and was stunned by the magnificent skiing of the medalists, which led to his ambition to become a professional skier. Although he was captivated by mogul skiers up until his early elementary school years, he joined "Niseko Hanazono Racing", a competitive alpine skiing club, in third grade at the invitation of a friend. There he discovered the excitement of competing against other people in terms of speed, and began to devote himself to competing.
Ohkoshi's win at the All-Japan Junior High School Ski Tournament in second grade of junior high school was the start of his successes, followed by victories and prize-winning rankings at numerous competitions. From the ski season of his first year of high school, he was designated a junior member of Japan's Alpine National Team, and his stage shifted to international tournaments. During high school, he achieved a total of four victories in two events in two consecutive years at the All-Japan High School Ski Tournament (Inter-High), setting a new record.
As a college student, Ohkoshi was designated a senior member of the national team, and was able to appear on a global stage (the World Championship and World Cup), as he had longed to as a young boy. He reached 28th place in the World Championship that he entered at the age of 21. Since then, centering his activities in Europe, he stands at the peak of his abilities, breaking down global barriers, while constantly striving to be an athlete befitting such a position.
Ohkoshi Ryunosuke Official Website
http://www.ohkoshiryunosuke.com/
Born in Niseko, Hokkaido, he is a freestyle skier active on the world stage with Niseko Grand HIRAFU as his home.
Sasaki took up freestyle skiing due to the influence of his elder brother, and developed his skiing technique while skiing across the various mountains of Niseko with his brother and his friends. He began to dedicate himself fully to freestyle skiing during junior high school, and started training at ski parks with multiple jumps and jib items.
In second grade of high school, he received an invitation from the Shirakawa Skiing School, where famous freestyle skiers from across Japan get together, to start studying freestyle skiing at Shirakawa, right from relearning the basics. The following year, his achievements at domestic competitions were recognized with selection as a member of the junior national team, and he set out to compete in the World Junior Championship.
Sasaki has now developed to the level where he is victorious even at tournaments that determine the Japanese number one, and also competes in the World Cup.
He is currently active with the aim to become a global top skier, with his skiing skills earned amid the magnificent nature of Niseko and his personal motto of enjoying himself and tackling every task with feelings of gratitude.
Gen Sasaki Official Website
http://www.gen-sasaki.com/
Yoichi Watanabe is the first person that comes to mind when you think of a Niseko photographer.
He's one of the photographers who represent Japan in the field of ski photography.
Yoichi is a skier himself, and travels the world's mountains with Niseko as his base, in addition to presenting his numerous works.
He has also revealed his own personal photo collections, featuring incredible snow-covered mountains across the world including those in Alaska, Canada and Europe, as well as countless slopes that hold a powerful attraction for skiers.
In particular, his collection entitled "Yukiyama o Suberu Hito" is an absolute must-see for all those who love the mountains.
http://www.j-n.co.jp/ski/YW_pb.html
Through his photos, Yoichi sends out images of the skiing, snowboarding and scenery of Niseko, his base of activities, around Japan and overseas.
While outdoor sports is the main subject matter of his photos, he also continues to capture the topography and lifestyles of snowy regions, and has made a huge contribution to the current international development of the Niseko area.
He values the fact that he is a skier above being a photographer, and his charismatic personality means there is always a crowd of people around him.
Yoichi's activities are not only limited to ski photography - he also holds his own photo exhibitions as an artist.
Recently, his photography featured in "Expressive Photos, Hokkaido", a special exhibition by the City of Sapporo Photography Culture Promotion Project, a project run in coordination with Sapporo International Art Festival 2014.
He's active in a range of other different fields, too.
Yoichi Watanabe Official Website
http://yoichiwatanabe.jp/
This photo was taken by Joe Kobashi.
He's photographer of various sports with a focus on skiing, and is regularly featured in magazines and publicity materials.
http://joe-jk.com/index.html
Here's a guy you've been waiting for – the snowboarder Takanori Nagata.
He qualified as a pro in 2007, and is active in multiple JSBA All Japan snowboard cross competitions.
As well as his appearances in multiple forms of media, he's busy with diverse activities as a rider, a guide, and more.
Takanori serves as the director of Pleasure Associates, the organizer of Niseko Pleasure Camp held every season in late March.
His friendly, open personality attracts a lot of like-minded people, and the Pleasure Camp, with its top-class coaching team, always has a wide range of participants from kids to adults.
Takanori Nagata Official Blog
http://nagata-saimon.jugem.jp
Niseko Pleasure Camp Official Website
http://nisekopleasurecamp.jimdo.com
Hiromi became paralyzed in his lower body in a snowboarding accident in 2008 and has since adapted to life in a wheelchair.
Despite this, his incredibly positive attitude has led him to keep pushing himself at sports including kayaking and chair skiing.
It was not long before he received the support of GENTEMSTICK to become a chair snowboarder.
Hiromi energetically keeps up his activities in summer as a paracanoe athlete and rafting guide.
"Youtei Outdoor", official web site.
http://www.youteioutdoor.com
Hiromi Tatsumi Official Web Site.
http://hiromitatsumi.com
The distinctively-shaped snowboard is the GENTEMSTICK.
It's pretty famous among snowboarders.
The tools Hiromi is holding in both hands are called "outriggers", which help him keep his balance in all directions.
GENTEMSTICK
http://gentemstick.com/index.html
Born in Chitose, Hokkaido.
With Hokkaido as his base, Nakata is active as a photographer, with skiing and snowboarding as his subjects.
In his late teens he commuted to Niseko as a park digger, and later started working at a pro shop. Nakata considered staying in his job at the shop, but found that he really wanted to be up in the mountains. When he noticed that doing paperwork for the shop and standing at the counter took up most of each day, he decided to start a new path in life as a photographer. He was 27 years old.
Now active as a photographer, his aim is to convey the things he has seen and felt through photos, and he finds his feet leading him up to the mountains.
With Hokkaido as his current base, he is expanding the sphere of his activities to the rest of the world.
Nakata works energetically, launching his Hokkaido-themed free magazine "Room Snowboard Magazine" in 2014. His photos are often used on the covers of various Japanese and North American specialist snowboarding publications.
Katsuno is a freestyle skier who learned to ski at his home ground of Niseko Grand HIRAFU from childhood, and now aims to enter and stand on the podium of the Freeride World Tour.
Utilizing his skills cultivated through various types of skiing – the Niseko Grand HIRAFU Ski School, moguls, slopestyle, and more – he is current mainly active in freeriding.
At the age of 17, Katsuno demonstrated his natural dynamism by travelling to New Zealand on his own to enter a freeriding competing. His capabilities were recognized with an invitation to an even higher-level competition held in Europe, the Freeride Junior World Championship 2018. With his win of the MVP award at the FJWC 2018, he is firmly showcasing his skills worldwide.
In 2018 he made his second trip to New Zealand, where he achieved podium places at all the competitions he entered, and received an invitation to the Freeride Junior World Championship 2019 in Europe in succession to the previous season.
Born in Muroran, Hokkaido.
The leader of the Niseko-based snowboarding crew "in crew", he also manages the snowboard and skateboard shop "Lowstar" in Kutchan.
During the snow season, Minamiyama creates and uploads web movies at a pace of around one per month. In spring, he hosts his own event, "Mad Dog Jam", at Niseko Grand HIRAFU. He also works hard to train up the next generation of junior snowboarders.
Born in Tomakomai, Hokkaido.
Hama took up snowboarding in 2004. She has set up base in Niseko, Hokkaido when in Japan, and in Whistler, Canada when overseas, and brushes up her technique to handle all-round snowboarding.
As well as training with the aim of competing in international snowboard cross, while she has yet been unable to reach the global stage, she works not only on improving her own skills but also is engaged in the position of instructing others.
In 2018, she challenged herself to a new field and entered freeriding competitions. She was victorious at the "Freeride Qualifier Hakuba" qualifying event, and earned the right to compete in the main tournament. Despite it being her first appearance in the event, she became the first ever Japanese champion.
Since then, her main activities have centered on her efforts to enter the Freeride World Tour.
Niseko Grand HIRAFU Brothers: Yamasemi-kun & Kawasemi-kun
The gray one is "Yamasemi-kun", and the blue one is "Kawasemi-kun"
[Origin]
One of the former staff here really loved fishing, and when he was fishing nearby he chanced upon the sight of a real "yamasemi" (crested kingfisher).
It was so adorable and extremely rare to see one in the wild, so we started using the "yamasemi" as our main character (back in the days when Grand HIRAFU was called Niseko HIRAFU International Ski Resort).
After that, when the resort's name changed to Grand HIRAFU, the "kawasemi" (crested kingfisher), which hadn't been seen around here for a while, came back to Niseko - so he joined his brother to become our two mascot characters!
[Favorite Things]
Powder snow, potatoes, corn, salmon
[Least Favorite Things]
Hot places (they don't appear in summer), crunchy snow on the slopes, bears
[Hobbies]
English conversation, Yamasemi likes skiing while Kawasemi likes snowboarding.
Outside of winter they go fishing together secretly, apparently.
Yamasemi is 180cm, and Kawasemi is 170cm. But they get smaller in a blizzard!
NISEKO Mt RESORT Grand HIRAFU
204 Aza Yamada, Kutchan-cho, Abuta-gun, Hokkaido JAPAN
Tel. +81-136-22-1105 Fax. +81-136-22-2821


Agriculture is the major occupation of the people of Pakistan. Even today agriculture is a major sector and occupies a very important place in the economy of Pakistan. It not only provides food for our growing population but is a source of raw materials for our principal industries as well as a source of foreign exchange for our government.
24% of our gross domestic products belong to this sector. It engages 53% of our employed labour .About 75% of our people are engaged directly or indirectly with this profession. A large part (about 10%) of Pakistan's exports is made up of primary commodities, including the major cash crops. In addition to this major large scale industries like cotton textile and sugar etc. as well as medium and small scale agro-based and cottage industries are directly dependent on this sector for their raw materials.
1. The farmers of our country are mostly un-educated and lack technical knowledge .They are unable to unde4rstand the modern scientific methods of agriculture and often remain ignorant of good means to protect and increase their yield. Their production is therefore low.
2. The majority of our farmers are poor and they often live in a hand to mouth position. Most of them are always under heavy burdens of debts. So due to lack of capital they cannot afford to purchase modern scientific implements, chemical manures, improved types of seeds etc. Hence they cannot attain the required standards.
3. Most of our farmers are still stuck to the old traditions of their fore fathers. The circumstances have compelled them to use the crude implements, because due to small holdings of land and poverty they are unable to acquire and use modern scientific methods. That is why their standard is lower than that of cultivators in developed countries.
4. Due to excessive use of canal water, most of our cultivated lands have become victims of these two dangerous diseases. Every year, salinity alone, is turning about 1, 00,000 acres of arable land into marches and salt lands. Water-logging is no less injurious. On the other hand the measures taken so far are quite inadequate for such issues.
5. Due to our law of inheritance, our farmers command very small pieces of hands which prove to be costly. A farmer cannot afford tractors and other machineries for a small piece of land and those incomes are low. Hence his interest is converted to some other sector.
6. Our farmers have to face many problems due to scarcity of water which is one our major problems. Large tracts of land estimated to be about 22 million acres is lying uncultivated due to shortage of water. Rainfall is uncertain and the existing irrigational facilities in our country are quite insufficient and need to be extended.
7. The land tenure system of Pakistan has creates a chain of intermediaries in between the state and the tenants. The system, instead of being conductive to agricultural development, stands in the way of its progress.
8. In some areas of our country, especially in hilly regions, the upper fertile soil is being eroded by different natural agents of change .Thus the fertility is poor and soils are becoming less productive.
9. The marketing facilities for agricultural products in Pakistan are still far from satisfactory level. Our cultivators cannot get just prices for their produce due to defective marketing organization. Moreover the chain of middlemen between the producers and ultimate consumers take a heavy share of their produce .Thus the cultivators do not take much real interest in increasing their product too.
10. Due to lack of agricultural education and methods of modern research, our farmers cannot control the various diseases of crops and attacks of pests and insects. The result is low yields.
11. The agricultural activities in our country are performed in rural areas, but most of our villages have no road or railway links with our markets. So farmers have to face innumerable hardships to sell their products. Hence the farmers take very little interest in their profession and production suffers.
12. The resources required for agricultural operation are land, layout, livestock, farm equipment, seeds, fertilizers, irrigation, transport etc. For the convenient and timely procurement of these resources the farmers must have easy access to credit. Zarai taraqiati and commercial banks provide loans to the farmers which are insufficient because our farmers are very poor.
13. Due to the above mentioned facts, it is clear that the yield per acre of various crops in our country is comparatively low than of the other countries. Hence we are backward agriculturally.
1. The land which has been declared useless due to salinity and water-logging should be reclaimed. Tube-wells should be installed in the affected areas to decrease the salinity. Beds of new canals should be made of concrete to avoid water-logging. These measures should be taken on priority basis to avoid further deterioration of land .
2. The major problem of agriculture in Pakistan is scarcity of water. Most of the land is lying unused due to this problem. Therefore, it is necessary that the irrigation facilities be extended for increasing agricultural output.
3. The agricultural yield can also be increased to a great extent by using fertilizers. The use of artificial manure should be introduced throughout the country. Due to poverty and illiteracy our farmers hesitate to purchase the fertilizers.
4. The use of better seeds, fertilizers and modern implements is not possible without adequate credit facilities for the farmers. The government has extended the existing credit facilities to a large extent. The commercial banks also grant loans to the farmers, but still there is a need for more facilities as our farmers are very poor.
5. The farmers should be provided better quality seeds at the lowest price and at the right time. Better seeds will ultimately give better yield.
6. Various plant diseases damage a large part of our crops. But our farmers have no effective control over them. Therefore, preventive and narrative measures should be taken throughout the country.
7. Mechanization of agriculture refers to the use of various equipments the different stages of cultivation. By the use of modern equipments better results can be achieved in the shortest time.
8. Co-operative farming refers to the farming done on the basis of mutual help and co-operation. Under this method the small holdings are consolidated .In this way the owners of small holdings can also use fertilizers and modern equipments jointly. After harvesting, the yield can be distributed among the owners according to their ownership.
9. Increase of literacy ratio in rural areas especially in agree-education is the need of the day. The more educated the farmers will be the better will be the results achieved.
10. Marketing procedures should be simplified and various marketing facilities should be provided to the cultivators.
11. The rural areas of the country must be provided with road and railway links with marketing centers and better means of transport and communication should be provided.