text:
Pakistan Navy ships visit Morocco: A step towards revamping Pak-Africa ties
By Ali Basit As per IMF forecast, Africa will be second-fastest growing region in the world between 2016 and 2020 with annual growth of 4.5%. Seven African countries are posting annual growth rates well above 5.5%. Similarly, among the…
Tabish Khan expresses his desire to represent Pakistan in Test cricket
KARACHI: Pakistan's discarded first class player Tabish Khan expressed his confidence to play Test cricket for Pakistan. While addressing a presser in Karachi, the pacer said ""Representing Pakistan in Test cricket is the biggest dream…
Australia's Tim Paine has said the upcoming Test series against Pakistan and New Zealand may be the last time he captains the side at home as he enters the twilight of his career. Paine, who turns 35 next month, was named captain after…
Pak's Imaad Ali crowned Jr World Scrabble champion
Pakistan's 13-year-old Syed Imaad Ali became the youngest player ever to win the Junior World Scrabble Championship at Torquay, England. According to details, Imaad remained in the lead right throughout the championship and was declared…
Aisam says he won't play Davis Cup tie against india if the event not shifted back to Pakistan
Pakistan tennis superstar Aisam-ul-Haq Qureshi has written a letter to the President of the Pakistan Tennis Federation (PTF), Saleem Saifullah and said that he will opt out of the Davis Cup tie against arch-rivals India if the match is…
Pakistan beat Oman, to face India in Emerging Asia Cup 2019 semis
Opener Haider Ali's sensational ton guided Pakistan to a 147-run win over Oman in the Asian Cricket Council (ACC) Emerging Teams Asia Cup 2019 match played at the Sheikh Kamal International Cricket Stadium, Cox's Bazar on Monday.…
Edwards, Mpofu among more than 100 foreign cricketers register in PSL-5 Silver category
The Pakistan Cricket Board (PCB) announced the names of overseas players registered in the Silver category for the Pakistan Super League (PSL) 2020. The six franchises will be able to pick 182 foreign players from the category.…
Senator Faisal Javed urges Pak cricketers to not participate in India-owned T10 league
ISLAMABAD: Senator Faisal Javed Khan said that the ongoing Abu Dhabi T10 league a private league with major share of Indian businessmen and discouraged Pakistani players to participate in the tournament. Faisal Javed took to his twitter…
SBC Green beat Grey in Sindh Bar Council cricket tourney
KARACHI: SBC Green cricket team defeated SBC Grey by six wickets in the ongoing Sindh Bar Council T20 Tournament at the TMC Ground. According to details, SBC Grey managed to score 131 runs on allotted 20 overs quota. Furqan Siddiqui top…
Shahid Afridi among seven Pak cricketers picked in BPL 2020 Draft
DHAKA: Former Pakistan star all-rounder Shahid Khan Afridi among seven Pakistani cricketers have been picked in the upcoming Bangladesh Premier League (BPL) Draft. According to information available here, Dhaka Platoon picked up former…
Cricketers Hafeez, Yousuf react to LHC verdict on Nawaz Sharif's ECL case
LAHORE: Pakistan veteran all-rounder Mohammad Hafeez has reacted to the Lahore High Court's (LHC) decision of allowing fomer PM Nawaz Sharif to travel abroad for his medical treatment. Mohammad Hafeez took to his Twitter and wrote:…
CPEC update: PM inaugurates Havelian-Thakot Section
ISLAMABAD: Prime Minister Imran Khan will perform the inauguration of Havelian-Thakot section of Hazara motorway project on Monday. The motorway is part of the China Pakistan Economic Corridor project. The project will serve as the main…
CA bans James Pattinson from Pakistan Test for 'personal abuse'
Australia fast bowler James Pattinson was suspended on Sunday for player abuse, ruling him out of the first Test against Pakistan this week. The paceman was found guilty of breaching Cricket Australia's code of conduct during Victoria's…
Pak Army wins 33rd National Games with 380 medals
PESHAWAR: Pakistan Army won the 33rd National Games with a tally of 380 medals in Peshawar. With 150 Gold, 134 Silver and 96 Bronze, Army ruled the medal table and won their 24th title. WAPDA bagged the second position with 320 medals…
Mushtaq Ahmed all set to join Pakistan squad as spin bowling consultant: report
Pakistan's former leg-spinner Mushtaq Ahmed is reportedly all set to become Pakistan's spin bowling consultant. According to Cricket Pakistan website, He was kept in mind while doling out the advertisement for the position. It is…


IPOW Metal Bike & Motorcycle Cell Phone Mount, with Unbreakable imparts most of the ingredients with a wonderful discount price of 17.99$ which is a deal. It features countless factors including ANY, Cell Phone, Accessories and gps motorcycle handlebar mount.
Ipow ABS+ & Cell Mount, & 100% Satisfaction Please Feel Free Shop.


text:
Military.com Authors
Thomas Novelly
Thomas Novelly is a reporter for Military.com focusing on coverage of the Air Force and Space Force. He previously covered veterans, military bases and federal politics in South Carolina for The Post and Courier as well as breaking news for the Courier-Journal in Louisville, Kentucky. He lives in Charleston, South Carolina.
The community of missileers who have spent years operating America's nuclear arsenal are voicing concern about a presentation...
Some Space Force Guardians Can Skip Fitness Test Until New Program Is Unveiled
Space Force Guardians will not be required to take a fitness assessment in 2023 if they took one in 2022, at least until the...
Now You Can Send Recruiters After Almost Anyone Using Updated Air Force App
Air Force recruiters want anyone, civilian or service member, who has a smartphone in their pocket to to recruit people to...
Air Force Accepted Dozens of New Recruits Who Tested Positive for THC Under New Program
In late September, the Air Force and Space Force announced a pilot program that would grant certain applicants who test...
The Old Air Force Tape Test Is Officially Dead. Long Live the New Tape Test.
Airmen and Guardians will no longer have to go through the dreaded abdominal tape test, but will have their waist and height...
Air Force Moves F-16s from Europe to Japan as Ukraine War Lingers and Pacific Tensions Simmer
The move comes after the Pentagon announced it was pulling older F-15 Eagles from Kadena Air Base in an attempt to bring more...
Military Families Stressed by Delay in Parental Leave Rules from the Army, Navy and Marine Corps
The services need to issue specific guidance to their formations before the rules can be used by the rank and file.
Attacks on Satellites May Trigger Military Response, US and Japanese Officials Say
With additional treaty obligations between the U.S. and Japan, American forces could be drawn into a skirmish if Japanese...
Air Force Academy Cadet Dies Suddenly While Walking to Class
Cadet 3rd Class Hunter Brown of Lake Charles, Louisiana, died after he "experienced a medical emergency while leaving his...
Applications to Air Force Academy Bounce Back After Plummeting During Pandemic
More than 10,000 students had applied to be part of the class of 2027 as of the Dec. 31 deadline.
Get Ready for Fort Liberty: The Pentagon Begins Changing Confederate Base Names
The Pentagon has started the process of renaming Fort Bragg and other bases, as well as ships and hundreds of signs and roads...
Deliveries of F-35s and Engines on Hold in Latest Fallout from Texas Crash
The halt in deliveries comes three weeks after an F-35B Lightning II crashed during a vertical landing at a Naval Air Station...
Get the latest on pay updates, benefit changes and award-winning military content. Right in your inbox.
WWII Rocket Unearthed by Man Using Excavator in Florida Yard, Cops Say. He Got Lucky
The fate of the rocket was not revealed, and it's still a mystery how it came to be buried in someone's yard.
Latest Military Videos
US Weighs in on Government Shake-up in Kyiv
The State Department is weighing in on developments in Kyiv as several senior Ukrainian officials, including front-line governors, lost...
French Soldiers Hold Combat Exercise in Romania
Around 600 French soldiers deployed as part of a NATO battle group in Romania held a combat exercise on...
Biden Says Supporting Ukraine a 'Worldwide Commitment'
President Joe Biden thanked western allies providing military support to Kyiv as he announced on Wednesday that the United...
Biden Announces US Will Send Abrams Tanks to Ukraine
President Joe Biden announce Wednesday the U.S. will send 31 M1 Abrams battle tanks to Ukraine, reversing months of...
They Said the Rise in Military Suicide Is a Mystery. Traumatic Brain Injury May Be an Answer.
Veterans who have had a head injury are at higher risk for suicide, but these deaths can be prevented, experts say.
What Happened to Members of the Military Accused of Storming the Capitol on January 6?
At least five service members allegedly were part of the deadly pro-Trump mob that assaulted the U.S. Capitol on Jan. 6.
Command Sgt. Maj. Michael Weimer is set to become the Army's next top enlisted leader in August, replacing Michael Grinston.
Navy News
A Navy Veteran Built New Technology to Aid Wildfire Prevention and Disaster Relief
Harbinger is software that combines a 1950s idea with the latest in sensors and communications technology.
In 1942, Phil Bucklew and nine other naval warfare pioneers trained to perform amphibious assaults before the U.S. had any...
Air Force News
Space Force News
Military Probing Whether Cancers Linked to Nuclear Silo Work
Nine military officers who had worked decades ago at a nuclear missile base in Montana have been diagnosed with blood cancer.
Marine Corps News
Marine Corps Base Camp Blaz, which is on Guam's northern plateau, is still in the process of being built.
Taylor Sheridan now has eight series airing or in production for Paramount+.
Coast Guard News
The hearing, led by Coast Guard and National Transportation Safety Board officials, revealed new details about issues the...
As hundreds of Cuban migrants washed ashore on New Year's Day in the Florida Keys, local officials complained to the state...

redpajama_set_name: RedPajamaCommonCrawl


The death of Arthur Tudor gives us one of the great what if's in History, and it changed England irrevocably.
The young man of only 15 died leaving his wife Katharine of Aragon, the great Spanish Infanta and prize of Henry VII a widow, seemingly never to fulfill her destiny to be Queen of England.
After Arthur's death, she left Ludlow for London in a litter dressed in mourning black.
She was housed at Durham House, built in the 14th century and which belonged to the Bishop of Durham on The Strand. It was a beautiful house, with gardens that stretched down to The Thames.
This wasn't to be a happy time for Katharine. She was in limbo, and a political pawn in a foriegn country where she was still learning the language.
Her father, Ferdinand of Aragon had only paid 50% of her dowry from her marriage to Arthur, and this infuriated Henry VII who continued to wrangle and negotiate with Ferdinand.
By 1503, the two wily King's had agreed on how to move forward – agreeing to the marriage of Katharine to Henry, Arthur's younger brother and the newly created Prince of Wales.
A papal dispensation was issued due to the fact that Katharine had been married to Henry's brother.
Katharine swore that her marriage to Arthur had not been consummated, which was also confirmed by her Duenna. The dispensation was granted even in the case of affinity and if the marriage had been consummated.
However it wasn't going to be a quick wedding.
In 1504, Katharine's formidable mother Isabella of Castille died, leaving Katharine as a less attractive match for Henry seeing she was no longer the daughter of the monarch of Castille, just of Aragon.
Henry VII knew that this put him in control and he was less enthusiastic about the match. In 1505, Henry VII convinced his son to repudiate the match.
Katharine was once again in limbo and lived in poverty, unable to pay her staff due to the squabbling monarchs. Both Henry and Ferdinand expected the other to give her an allowance.
Ferdinand did not want Katharine to return to Spain, and made her his ambassador to the royal court for which she conducted herself admirably.
Shr was a match for Henry VII and his councillors and was thought to have been the first female ambassador in European history.
However her luck was about to change (if you call it that) Henry VII died at Richmond on the 21st of April, 1509 possibly of Tuberculosis. If he hadn't have died at this point, who knows who the marriage negotiations would have fallen upon?
Henry, Prince of Wales was now King Henry VIII and free for the first time in his life to make his own decisions, having growing up coddled and hidden from the public by his father and grandmother.
One of his first decisions was to marry Katharine as soon as possible.
There are many thoughts as to why Henry made this decision, and I believe it was a combination of factors.
I believe it was a result of his ardent desire to be the chivalrous Knight, a gentleman coming to a ladies rescue. He saw himself as a man versed in courtly love and took an interest in chivalry.
Also, Katharine at 23 was beautiful, intelligent and pious, everything a 16th Century Queen should be.
On the 11th of June, 1509, Henry married Katharine at the Palace of Greenwich, London in a quiet ceremony, the opposite to Katharine's grand marriage to Arthur 8 years before.
There was no public bedding ceremony, and all celebration was channeled into their joint coronation which occurred 13 days later on the 24th of June.
It was a joint coronation, which was unusual as it raised Katharine to be an anointed Queen, not a simple consort.
In true fashion there were many celebrations and a banquet that followed the coronation at Westminster Hall, with days of feasting and jousting.
Not even the death of the Tudor matriarch, Henry's Grandmother Lady Margaret Beaufort on the 29th of June would dampen the celebrations.
The coronation of Henry and Katharine was seen as a new start, a new dawn for England, with the Wars of The Roses firmly behind them.
There was a young King on the throne who would bring about a golden age and the country could prosper and move on from the nervous reign of Henry VII.
I like to think that Katharine felt pure joy on this day. She had spent the last 8 years in poverty and uncertainty, after loosing the husband she had expected her entire life. She showed strength and fortitude beyond her years and as her life didn't end happily I take comfort in the thought that one day in 1509, she truly felt happy.
Who could have known on that summer's day how things would progress?


text:
That name is familiar: Blue Jays claim Justin Smoak
Chad Jennings
cjennings@lohud.com
1. It signals at least a slight shift in the American League East
Smoak is a first baseman, and so his move to Toronto suggests Adam Lind is on his way out. For several days now, rumors have suggested the Blue Jays are interested in trading Lind. He's been a pretty powerful bat the past couple of years -- kind of a return to form after a few disappointing seasons -- and given the offensive slide throughout baseball, a bat like Lind's could bring the Blue Jays a fairly significant return. Smoak has been mostly a disappointment in the big leagues, but the Blue Jays still have Edwin Encarnacion to play first base, and acquiring Smoak could make it easier to trade Lind in order to upgrade elsewhere.
2. It's a flashback to the Yankees failed bid to acquire Cliff Lee in 2010
Far more than the impact in Toronto, this waiver claim is probably most significant because of what it means in Seattle. It's essentially an acknowledgment -- not that they were ever denying it -- that the Mariners did not get nearly what they expected out of their high-profile Cliff Lee trade that brought Smoak over in the first place in 2010. At the time, the Yankees basically had a Lee deal in place -- it was centered on Jesus Montero -- but the Mariners backed out in favor of a deal built around Smoak, who's been just a .224/.309/.380 hitter in the big leagues. Smoak's slash line is actually quite similar to the .258/.302/.396 slash line for Montero, who's had far more limited big league chances.


text:
Q: Angular2 Auth With Auth0 Redirecting Before Gravatar Exist Background
In my angular2 application I am using Auth0 to authenticate. After the redirect back to my website I display some of the user's data to them in the component. When the page renders it does not show the user's email or gravatar / image. But at that point if I refresh the page the gravatar and email shows.
Setup
In app.component.html I am adding the breadcrumb component directly under the navigation but directly on top of the router-outlet.
The breadcrumb is shown on every page. If someone is not logged in it shows a Login / Register link. If someone is logged in it is supposed to show a picture of the user and their email which is returned in the auth0 profile saved to the localStorage.
Inside the component that is rendered to the router-outlet all of the data that is in localStorage is show in the page. This page renders perfectly. But the breadcrumb component does not update with the correct data from localStorage.
It is obvious that the problem here is that the breadcrumb component is being rendered while the profile in localStorage is still null.
Code Example
I am displaying the data like this,
breadcrumb.component.html

    Login / SignUp

      {{auth.userProfile.nickname}}
      Account

      5

Please notice I have tried verifying and assigning the data in multiple different ways using built in NG2 features.
breadcrumb.component.ts
import { Component, OnInit, AfterViewInit, OnChanges } from '@angular/core';
import { Router, ActivatedRoute }    from '@angular/router';
import { Auth }                      from './../services/auth.service';

@Component({
    selector: 'breadcrumbs',
    templateUrl: './breadcrumb.component.html'
})
export class BreadcrumbsComponent implements OnInit, AfterViewInit, OnChanges {
  private gravatar: string
  private email: string;

  constructor(private router:Router, private route:ActivatedRoute, private auth: Auth) {
      if (this.auth.authenticated()) {
        var userProfile = JSON.parse(localStorage.getItem('profile'));
        this.gravatar = userProfile.picture;
        this.email = userProfile.email;
      }
    }

  ngOnChanges() {
    if (this.auth.authenticated()) {
      var userProfile = JSON.parse(localStorage.getItem('profile'));
      this.gravatar = userProfile.picture;
      this.email = userProfile.email;
    }
  }

  ngOnInit() {
    if (this.auth.authenticated()) {
      var userProfile = JSON.parse(localStorage.getItem('profile'));
      this.gravatar = userProfile.picture;
      this.email = userProfile.email;
    }
  }

  ngAfterViewInit() {
    if (this.auth.authenticated()) {
      var userProfile = JSON.parse(localStorage.getItem('profile'));
      this.gravatar = userProfile.picture;
      this.email = userProfile.email;
    }
  }
}

When I check the local storage the data does exist. So this leads me to believe that the page is rendering before the data is able to be placed in the local storage.
This is the majority of my auth0 service.
authorize.service.ts
@Injectable()
export class Auth {
  lock = new Auth0Lock(myConfig.clientID, myConfig.domain, options, {});
  userProfile: Object;
  logreg: LogReg;
  user: User;

  constructor(private router: Router, private http: Http ) {

    this.userProfile = JSON.parse(localStorage.getItem('profile'));
    this.user = JSON.parse(localStorage.getItem('user'));
    this.lock.on('authenticated', (authResult: any) => {
      localStorage.setItem('access_token', authResult.idToken);
      this.lock.getProfile(authResult.idToken, (error: any, profile: any) => {
        if (error) {
          console.log(error);
          return;
        }

          // Login Or Register User On Our Server
        this.logreg = new LogReg(profile.email_verified, profile.email);

        this.checkRegister(this.logreg).subscribe(
            (res)=>{
              console.log("Hey this runs");
              console.log(res);
                if (res.email_verified === false) {
                 localStorage.removeItem('profile');
                 localStorage.removeItem('api_key');
                 localStorage.removeItem('access_token');
                 localStorage.removeItem('user');
                 this.userProfile = null;
                 this.user = null;
                 this.router.navigate(['/verify-email']);

                }
            else if (res.api_key_exist === false) {
                  console.log("Hey this works")
                localStorage.setItem('profile', JSON.stringify(profile));
                this.userProfile = profile;
                console.log(this.userProfile);
                this.user = new User(profile.email, '', '', '', '', '', '', '', '', '', '', res.api_key_exist, '')
                localStorage.setItem('user', JSON.stringify(this.user));
                this.router.navigate(['/profile']);

            } else if (res.api_key_exist === true) {
                this.user = new User(res.user.email,
                    res.user.first_name,
                    res.user.middle_name,
                    res.user.last_name,
                    res.user.dob,
                    res.user.phone,
                    res.user.street_address,
                    res.user.city_address,
                    res.user.state_address,
                    res.user.zip_address,
                    res.user.client_ss,
                    res.api_key_exist,
                    res.api_key);
                console.log(this.user);
                localStorage.setItem('api_key', JSON.stringify(res.api_key));
                localStorage.setItem('user', JSON.stringify(this.user));
                localStorage.setItem('profile', JSON.stringify(profile));
                this.router.navigate(['/overview']);
            }
        },
            (err)=>{ console.log(err);}

        );
      });
      this.lock.hide();
    });
  }

  public checkRegister(model: LogReg) {
      // Parameters obj-
      let params: URLSearchParams = new URLSearchParams();
      params.set('email', model.email);
      params.set('email_verified', model.email_verified);

      return this.http.get(STATICS.API_BASE + STATICS.API_LOGIN,
          { search: params }).map((res:Response) =>  res.json());
      }

  public login() {
    this.lock.show();
  }
}

Question
How would I go about making sure the data exist before completing the rendering of the page when someone logs in? This way I can make sure the correct data is displayed to the user.

A: I cannot comment I will write this as an answer.
I have the same issue and I resolved it doing this:
edit
try this:
first I use this custom event emitter that makes life easy:
import { Injectable, EventEmitter } from '@angular/core';
@Injectable()
export class EmitterService {
   private static _emitters: { [ID: string]: EventEmitter<any> } = {};

   static get(ID: string): EventEmitter<any> {
      if (!this._emitters[ID])
        this._emitters[ID] = new EventEmitter();
      return this._emitters[ID];
    }

 }

inside the auth.service:
constructor(private auth: Auth, private issuesService: IssuesService) {
if (this.auth.authenticated()){
    this.userProfile = this.auth.getProfile()
    this.username = this.userProfile.name
}
else {
  this.loadUserDetails().subscribe(_ => {
    this.username= this.userProfile.name
  })
  }
 }

 ngOnInit() {
  if (this.auth.authenticated()){
    this.userProfile = this.auth.getProfile()
    this.username = this.userProfile.name
    console.log(this.username)
 }
 else {
  this.loadUserDetails().subscribe(_ => {
    this.username= this.userProfile.name
  })
}
}

loadUserDetails(){
 return EmitterService.get('USER_LOGGED_IN').map(userProfile => {
    this.userProfile= userProfile
  });
}

and in my html, I have:

Welcome, {{username}}

hope this helps!

redpajama_set_name: RedPajamaStackExchange


Non-contact radar continuously measures a variety of liquids in all conditions, including challenging environments, aggressive materials, and sanitary applications. Non-contact radars are easy to setup and configure using the BinDisc interface and are compatible with the BinView® web application and Binventory® software for simple access to inventory data from a phone, tablet, or desktop PC.
Ultrasonic sensors provide continuous, non-contact level measurement in bins, tanks, and silos or can detect material on conveyors. They emit sound above the human hearing range to detect echoes off a material surface and convert it to a distance using "time of flight." They offer built-in temperature compensation and self-cleaning operation in liquids up to 90 feet and solids up to 40 feet.
A pressure transmitters is used for liquid level measurement in both process and storage environments. It measures the amount of force on the sensor at a given depth and translates it to a level. Diverse models are available for external or submersible mounting, with a wide range of hazardous location approvals for use in water, wastewater, chemicals, and a variety of fuels.
BinMaster's magnetostrictive level sensors are continuous float level transmitters used in a wide variety of liquids. They are an ideal fit in rigorous applications requiring a heavy duty sensor with a simple setup and long lifespan. These sensors feature Class 1, Div 1 hazardous location certifications, and the sensor design allows monitoring of both top and interface levels with a single sensor, making it easy to measure oil and water interface levels inside production tanks.
Float switches can be used to help monitor liquid levels in small tanks or constrained spaces. These point level switches are commonly used for controlling pumps and activating alarms when a liquid level must be maintained to prevent overfilling or damage to the pump. They can also be used for simple level monitoring such as indicating when a tank is a certain percentage full.


text:
Neymar hits Barcelona with fresh lawsuit
Brazilian forward Neymar has filed a fresh lawsuit against Barcelona to demand some three million euros of unpaid wages from his previous stint at the Spanish club, a judicial source said Friday.
The lawsuit was filed "a few weeks ago", the source told AFP, confirming a report published in Spanish daily newspaper El Mundo which said the Paris Saint-Germain player was suing the Catalan giants to collect "3.2 million euros ($3.6 million) relating to his July 2017 salary".
According to the newspaper, Neymar alleges Barcelona withheld part of his salary when he quit the club to join the French side in 2017.
Neymar's shock transfer to PSG for a then-world record 222 million euros ($250 million at the time) came less than a year after he signed a contract that would have kept him at the Spanish club until 2021.
The 27-year-old player argues Barcelona "did not pay him his final salary as a 'reprisal' for having claimed his extension bonus", El Mundo reported.
Neymar has already sued Barcelona to demand the paymenet of 64.4 million euros which he negotiated with the club in 2016 in exchange for the extension of his contract until 2021.
A Barcelona court in September heard both sides in the dispute, but is yet to issue its ruling.
Barcelona say they paid Neymar a first tranche of 20.75 million euros before he left for PSG, but did not pay him the rest.
The player then sued to collect the rest of the sum. Barcelona counter-sued to demand that Neymar return the 20.75m euros he had already been paid.
La Liga: Barcelona secure late win over Elche, Sevilla beat Atletico
Boca beat Barcelona to win inaugural Maradona Cup
Barcelona to play Napoli in Europa knockout play-offs
Osasuna vs Barcelona: Osasuna strike late to snatch draw
Agent lists four top clubs Dortmund's Haaland may join in 2022
Barcelona to start from scratch after Champions League exit -Xavi


It's not what you earn it's what you keep!
Could My Family Benefit from a Family Limited Partnership?
Effective estate planning should address wealth transfer from a practical and cost-effective approach. One estate planning strategy that families with closely held businesses should consider is the family limited partnership.
What Is a Family Limited Partnership?
A family limited partnership is a partnership agreement that exists between family members who are actively involved in a trade or business. The partnership divides rights to income, appreciation, and control among the family members, according to the family's overall objectives. Under family partnership rules, the "family business" can include real estate or investments.
How Is This Arrangement Achieved?
Under the most common form of family partnership, you would begin by creating general and limited partnership interests in your business. Once the partnership is established, you then gift the limited partnership interests to your children.
By holding the general partnership interest, you are considered the "general partner" and maintain control over the enterprise. Your children are the "limited partners," and the limited partnership interest lets them share in the ownership of your business as well.
A family limited partnership enables you to provide your children with an interest in your business while achieving many goals. First, you can gauge whether or not they possess suitable ownership abilities by involving them in the business. Second, it removes the asset from the parents' estate, thus lowering the estate tax liability, if properly executed. In addition, you can transfer the limited partnership interests in increments over time, resulting in a gradual, systematic transfer of ownership. Finally, and perhaps most importantly, there may be immediate income tax benefits.
The interests transferred to your children, including all appreciation since the transfer, escape inclusion in your estate when you die. Only the value of the taxable gift(s) will be included. This can result in estate tax savings down the road.
By giving the partnership interests in increments over time, you can take maximum advantage of the annual gift tax exclusion. The exclusion duobles if you're married and if each spouse elects to give the maximum amount. The gift tax exclusion is indexed for inflation.
In addition, "minority discounts" — allowable reductions to the value of the gift because it is a minority interest — can lead to greater leverage of the annual exclusion and the unified credit. For instance, you may be able to discount the value of the gift up to 30 percent or more. However, in order for the discount to be valid, there must be a legitimate business reason for the partnership.
Generally, your wish to keep the business in the family is a legitimate reason to set up a partnership agreement — as long as you are joined together for the purpose of enterprise and not just to avoid taxes.
Aside from the estate planning advantages, the family limited partnership can result in substantial income tax savings. By including your children as partners and sharing partnership income with them, total family taxes may be reduced.
You should be aware, however, that if the income is unearned and the recipient is under age 14, "kiddie tax" rules will apply.
Family partnerships are arrangements under which each partner must play a role in the management and day-to-day operations of the business. Many of the benefits are similar to that of a limited partnership, but the family members accept more liability and will be more involved in the business. As managing partner, however, you must always receive a minimum income share that is proportional to the value of your services.
In addition, minors typically cannot be partners unless there is someone who controls the interest for the minor.
Investment partnerships are partnerships that hold nonbusiness assets such as securities and real estate that are likely to grow in value. Families can base a limited partnership on an investment partnership. In some cases, however, the arrangement would be considered an investment company, and gains and losses will be realized on the transfer of property to the partnership. Normally, under partnership rules, gains and losses are not realized when transferred to the partnership.
The benefits of the family limited partnership can be significant. But they can only be realized if the arrangement is valid under the requirements of the IRS. There are costs and expenses associated with the creation of these legal instruments. Consult a qualified legal or tax advisor if you think your family could benefit from a family limited partnership.
Matthew Devers is a registered representative registered to transact business with clients in AR TN SC. Matthew Devers is also is an insurance agent licensed in AR. If you are not a resident of the states noted above, all investment-related information on this site is for informational purposes only and does not constitute a solicitation or offer to sell securities or insurance services over the internet. Securities offered through H.D. Vest Investment ServicesSM, Member: SIPC, 6333 North State Highway 161, Fourth Floor, Irving, Texas 75038, (972) 870-6000.
The Ledger Company is not a registered broker/dealer or independent investment advisory firm.


text:
Philanthropy : The New Gospel of Wealth
In many regards, free market is at the centre of inequalities. In fact liberalism tends to tilt the balance in favor of they who work and earn money. As a result liberalism results in rewarding they who win and disadvantaging they who lose. But this debate is a horse of different pattern.
Do the so-called winners have to lend a hand to the "losers"? They have to, according to Andrew Carnegie, one of the most famous industrialists known for its philanthropy. Indeed M.Canergie acknowledged that the wealthiest had to assume a certain responsibility concerning the situation of the poorest groups. Like he described it in his article "Gospel of Wealth", philanthropy purports to ease the social wrongs which poison our society. From then on, high-income groups have to provide food, water, a shelter, health to society.
In the 20th century several businessmen created foundations which aimed at wiping out those social wrongs. We thus saw rich and powerful men like Mac Arthur, Knight or even Rockefeller build schools, libraries, invest in new vaccines and so on. Philanthropy is still on the agenda of well-off people: Zuckerberg (Facebook), Hoffman (Linkedin), Bezos (Amazon) and other CEOs created the Breakthrough Energy Coalition during the COP 21. This foundation purports to thwart the most harmful environmental damages caused on Earth. In this case philanthropy becomes a universal stake, which is so providential that it could overshadow the role of the States.
No wonder that this kind of foundation is praiseworthy. But according to Darren Walker, president of the philanthropic Ford foundation, philanthropy has to grapple with the "why and how" of the human misery rather than the "what". In a nutshell, injustice itself has to be swept out. Structural politics are cruelly needed. Investments are useless if they are not along with a real change of mindset and of the distribution of national wealth. A New Gospel of Wealth has to be written. Just like M.L.King stated:" Philanthropy is commendable, but it must not cause the philanthropists to overlook the circumstances of economic injustice which make philanthropy necessary".
Text inspired by the Guardian
Guns in classrooms : Bang Bang !
Does TV atomize societies ?
Shooping Days : Bang Bang


text:
Caceres (Extremadura), Spain February 2022
Still more than a month behind. We arrived in Caceres on 22 February and as I type this blog it is 27 March.
Caceres has much going for it and is now listed among my favourite Spanish towns. We stopped just outside the town at Camping Caceres near the old football ground, Estadio Principe Felipe. At first glance Camping Caceres appears a fairly basic site in a somewhat remote location but, no, it offers everything we require (most especially a popular bar restaurant) and; each plot has it's own bathroom/toilet and; while the city is a fair walk away, it's a pleasant walk through and around a series of olive groves.
It was a long but pleasant walk through through the countryside to Caceres
Vanya elected to stay by the Van and catch up on some Spanish homework which was set by Varndean College before we left England. I left her to it. I exercised the dogs and then set off on a further 10 mile walk to from and around Caceres.
One of the first buildings I passed as I entered the city was the local bull ring
Caceres is a large town by Extremadura standards. It is a university town with some 90,000+ inhabitants and it is split into two very distinct parts, the new town and the old town. Except for Plaza Mayor, which is filled with lively bars and restaurants (and perfect for people watching), the new town does little for me but the old town more than compensates.
I sat for a while outside a bar on Plaza Mayor with a glass of the local wine and planned a route around the old town. Also known as Monument City, Caceres is considered to be one of the most beautiful cities in Spain and in 1986 was the first city in Spain to be declared a UNESCO World Heritage Site. It comprises a mix of early Middle Ages and Renaissance buildings all enclosed within ancient Roman and/or Moorish stone walls and it is full of character and quite beautiful.
Photos on Plaza Mayor. The first photo includes a view of the 12th century Torre de Bujaco which sits alongside the Arco de la Estrella
Entering La Ciudad Monumental from Plaza Mayor through the Arco de la Estrella is like being transported back in time. As you walk the narrow cobbled lanes between the palaces, mansions and churches there is no sign of anything modern in the old town. Little wonder that the place has featured in countless historical dramas (and, more recently, in Game of Thrones although I'm thinking that almost everywhere worth visiting has featured in Game of Thrones).
My earlier route planning over a glass of wine went to pot the moment I entered the old town. It's a captivating tangle of cobbled streets, small squares and palatial buildings. You go where the fancy takes you and all the better if you end up retracing your steps. You'll see things you missed the first time around.
Entry into the Torre de Bujaco and along the town walls made for some interesting photos of the town's skyline with it's turrets, spires, gargoyles and the ubiquitous storks nests. The first of these two photos captured the Palacio de Toledo-Moctezuma; the second is of the white baroque San Antonio Church and was taken from the bell tower of the Santa Maria Cathedral
This particular blog would stretch into pages if I were to wax lyrical about all of the buildings I visited during my time in Caceres so, I'll focus on just two of the more interesting places namely, the Santa Maria Cathedral and the Palace de los Golfines de Abajo.
Built as a church fortress, the outside of the Santa Maria looks a very modest 15th century gothic cathedral (with just a small statue of San Pedro de Alcantata outside to set it apart from countless other large medieval buildings in the city) but inside, it is something else. It has three naves each of which hold some wonderful pieces of romanesque art (including a crucifix with a black Christ) but the most remarkable is the central nave with it's 16th century cedar altarpiece sculpted by Roque Balduque. It is breathtakingly beautiful. The cathedral has many other interesting features, including flagstones depicting the coats of arms of the region's most influential families and a small museum of religious artifacts both of which I paused to admire but, it is the walk up the spiral staircase to the bell tower with it's amazing panoramic views over the old town that hooked me.
The Palace de los Golfines de Abajo is the largest and most impressive of the city's palaces. It was built piecemeal between the 14th and 20th centuries and has long been home to the Golfin family. I visited Caceres out of season and so couldn't join one the theatrical guided tours which are supposedly quite enthralling but it was still worth going inside. The palace is crammed with historical treasures.
Front right of the first photo is the Palace de los Golfines de Abajo and on the left is the Casa de los Duques de Valencia (with the bell tower of the Santa Maria Cathedral behind. The second photo is of a typical street scene.
One place I missed during our visit to Caceres is the Cave of Maltravieso which can be found at the edge of the town. Caceres was developed by the Romans at much the same time as they built Merida in 25BC but, there is evidence of 'human' life in the area some 350,000 years before then. The Cave holds many ancient painting/stencils dating back to Neanderthal times.
Author DJmotortouringPosted on March 27, 2022 September 6, 2022 Categories SpainTags Extremadura
Previous Previous post: Merida (Extremadura), Spain February 2022
Next Next post: Burgos (Castilla y Leon), Spain February 2022


text:
ACCEPTED

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
Repert. bot. syst. 4:108. 1845 "<I>buchanani</I>"

#### Original name
null

### Remarks
null

redpajama_set_name: RedPajamaGithub


text:
Q: How do I select records that are older than 5 minutes from oracle using unix dates/times I am trying to bring back records that are older than 5 minutes from the current time. Here is what I have so far, but it seems to bring back all the records not just ones which are older than 5 minutes:
SELECT COUNT(DISTINCT "DOCUMENT_QUEUE"."DOCUMENT_NUMBER") as "Records"
FROM   "MAGINUS"."DOCUMENT_QUEUE" "DOCUMENT_QUEUE"
WHERE  to_date('01-JAN-1970', 'dd-MON-yyyy HH:MI:SS') + (DOCUMENT_QUEUE.CREATION_DATE/86400) > sysdate - 38/1440
ORDER BY "DOCUMENT_QUEUE"."DOCUMENT_NUMBER"

A: Use interval literals:
SELECT COUNT(DISTINCT "DOCUMENT_QUEUE"."DOCUMENT_NUMBER") as "Records"
FROM   "MAGINUS"."DOCUMENT_QUEUE" "DOCUMENT_QUEUE"
WHERE  DOCUMENT_QUEUE.CREATION_DATE < SYSDATE - INTERVAL '5' MINUTE
ORDER BY "DOCUMENT_QUEUE"."DOCUMENT_NUMBER";

A: This should work:
SELECT COUNT(DISTINCT "DOCUMENT_QUEUE"."DOCUMENT_NUMBER") as "Records"
FROM   "MAGINUS"."DOCUMENT_QUEUE" "DOCUMENT_QUEUE"
WHERE  round(to_number(DOCUMENT_QUEUE.CREATION_DATE-sysdate)*1440) > 15
ORDER BY "DOCUMENT_QUEUE"."DOCUMENT_NUMBER"

A: Use NOW() and  (date_to_unix_ts(DOCUMENT_QUEUE.CREATION_DATE) - 86400 ) for defining range.


text:
\section{Relation Extraction with Knowledge-Enhanced Generative Model}\label{sec:model}

\paragraph{Overview}

Figure \ref{fig:1} shows the workflow of our method.
The relational fact extraction is modeled as a seq2seq generative model, where knowledge-enhanced text is passed to the encoder, and the relational facts are generated sequentially from the decoder.
We use text $\mathcal{S} = $\textit{``Peter Parker, from Avengers, says \ldots''} and relation set $\mathcal{R} = $  \{\textit{member\_of}, \textit{nationality}, \ldots\} as a running example.
First, the plain text $\mathcal{S}$ is grounded to the public Knowledge Base (KB) Wikidata\footnote{\url{https://www.wikidata.org/}} to obtain relevant background knowledge $\mathcal{K}_{E}$, where the subscript $E$ refers to external knowledge.
In this example, we ground \textit{``Peter Parker''} to \url{https://www.wikidata.org/wiki/Q23991129} and \textit{``Avengers''} to \url{https://www.wikidata.org/wiki/Q322646}.
We use the ``instance of'' property in Wikidata to retrieve the background knowledge, e.g. <\textit{Peter Parker}, instance of, \textit{fictional human}> and <\textit{Avengers}, instance of, \textit{fictional organization}>.
Then, we combine the plain text $\mathcal{S}$, knowledge from the dataset $\mathcal{K}_{\mathcal{D}}$, and external knowledge $\mathcal{K}_{E}$ via a pre-defined template $\mathcal{T}$.
Finally, the combined text is passed into a generative language model, such as BART \cite{lewis2019bart} or T5 \cite{raffel2019exploring}, and the model directly generates the target relational triples token by token, e.g. \textit{``Peter Parker member\_of Avengers''}.

\paragraph{Knowledge-Enhancement} We leverage an entity linking model to ground texts to knowledge bases.
In the Relation Classification task, the span of an entity $\mathcal{E}_{pos}$ is given.
Therefore, we apply the bi-encoder entity linking method (BLINK) for entity linking \cite{wu2019scalable}.
BLINK uses a two-stage method that links entities to Wikipedia, based on fine-tuned BERT architectures.
First, BLINK performs the retrieval step in a dense space defined by a bi-encoder that independently embeds the entity $\mathbf{e} \in \mathcal{E}_{pos}$ and the text $\mathcal{S}$.
In the second stage, each candidate in the KG is examined with a cross-encoder ranking model by concatenating the entity and the text.
An approximate nearest neighbor search method FAISS \cite{johnson2019billion} is applied to search the large-scale KG efficiently.
In the task of Joint Entity and Relation Extraction, the span of an entity is absent. We apply an efficient one-pass end-to-end Entity Linking method (ELQ) \cite{li-etal-2020-efficient}.
ELQ follows a similar architecture as BLINK, but the bi-encoder jointly performs mention detection and linking in one pass.

After we obtain the grounded Wikipedia entries for entities in the text, we use the public-source pywikibot\footnote{https://github.com/wikimedia/pywikibot} to retrieve the entity knowledge from Wikidata, i.e. the knowledge provided by the "instance of" property.
If multiple "instances of" properties exist, we select the one with the highest frequency over the training data.
Formally, for the entity position-aware case, entity linking is formulated as:
\begin{equation} \label{eq:el1}
    \mathcal{K}_E = \{\mathcal{E}_{type}\} = EL_{BLINK}(\mathcal{S} | \{\mathcal{E}_{pos}\}).
\end{equation}
The entity position-absent case is formulated as:
\begin{equation} \label{eq:el2}
    \mathcal{K}_E = \{\mathcal{E}_{pos}, \mathcal{E}_{type}\} = EL_{ELQ}(\mathcal{S}).
\end{equation}
More implementation details will be discussed in Section \ref{subsec:model_implement}.

\paragraph{Template} We consider two templates to incorporate entity-related knowledge for both entity position-aware and entity position-absent cases in our application.
For text $\mathcal{S}$, we denote the joint knowledge from dataset and external knowledge as $\mathcal{K}_\mathcal{S} = \mathcal{K}_{\mathcal{D}} \bigcup \mathcal{K}_E$.
For the entity position-aware case, we construct the template as
\begin{equation*}
    \begin{split}
        & \mathcal{T}_1(\mathcal{S} | \mathcal{K}_\mathcal{S}) =\\ & \{w_1, .., w_{i-1}, [es], w^{s_t}_{i},.., w^{s_t}_{j}, [gr], \mathbf{t}_{s_t}, w_{j + 1}, .., \\
        &  w_{k - 1} , [es], w^{o_t}_{k},.., w^{o_t}_{l}, [gr], \mathbf{t}_{o_t}, w_{l + 1}, .., w_{N}\},
    \end{split}
\end{equation*}
where $[es]$ and $[gr]$ are special tokens to denote entity mention start and grounded entity information position, respectively. In the above case, the token span for subject $\mathbf{s}_t$ ranges from $i$-th to $j$-th token in original text, similar notation is applied to object $\mathbf{o}_t$.
Entity type for entity $\mathbf{e}$ is denoted as $\mathbf{t}_{e}$.

For the entity position-absent case, we construct the template by concatenating the entity knowledge information at the end of the original text as follows,
\begin{equation*}
    \begin{split}
        & \mathcal{T}_2(\mathcal{S} | \mathcal{K}_\mathcal{S}) =\\ & \{w_1, w_2, .., w_{N}, [gr], \mathbf{e_1} is \ an \  instance \ of \ \mathbf{t}_{e_1},  \\
        & ... , [gr], \mathbf{e}_M \ is \ an \ instance \ of \ \mathbf{t}_{e_M} ..\},
    \end{split}
\end{equation*}
where $\mathbf{e}_i, i = 1, 2,.., M$ are the retrieved entities via the entity linking system,
and $\mathbf{t}_{e_i}$ are corresponding entity knowledge from public  KB\footnote{In the T5 paper, authors added a prefix to distinguish different tasks during the pre-training stage.
Therefore, if T5 pre-trained model is adopted, we add the prefix ``\texttt{summary:}" in the template for our fine-tuning stage.}.

\paragraph{Data Augmentation}
If multiple triples exist in a single input text, the order of target relational triples may affect the performance in the seq2seq model, which is known as \textit{exposure bias} \cite{ranzato2015sequence}.
In previous relation extraction work, researchers attempted to alleviate exposure bias by replacing maximum likelihood estimation (MLE) loss with reinforcement learning-based loss \cite{zeng-etal-2019-learning} or unordered multi-tree structure-based loss \cite{zhang2020minimize}.
In our work, we adapt the data augmentation strategy by shuffling the order of target relational triples and adding the shuffled triples to the original dataset during the training phase.
Our ablation study in Section \ref{subsec:result} shows that the data augmentation strategy can significantly improve the performance.

\paragraph{Loss Function}
We apply a generative model, such as BART \cite{lewis2019bart} and T5 \cite{raffel2019exploring}, to input text $\mathbf{x} = \mathcal{T}(\mathcal{S}| \mathcal{K}_\mathcal{S})$.
The output of triples $\mathbf{y} = \{ \mathbf{s}_1\ \mathbf{r}_1\ \mathbf{o}_1; \mathbf{s}_2\ \mathbf{r}_2\ \mathbf{o}_2;  ... ;\mathbf{s}_T\ \mathbf{r}_T\ \mathbf{o}_T\}$ is generated in the text format in a single pass, where token ``;'' is used to separate different triples in the output.
The generative model needs to model the conditional distribution $p(\mathbf{y} | \mathbf{x})$.
The output tokens come from a fixed size vocabulary $V$.
The distribution $p(\mathbf{y} | \mathbf{x})$ can be represented by the conditional probability of the next token given the previous tokens and input text:
$$
p(\mathbf{y} | \mathbf{x}) = \prod_{i = 1}^{|y|} p(y_i | y_{<i}, \mathbf{x}),
$$
where $y_i$ denotes the $i$-th token in $\mathbf{y}$.
The subject $\mathbf{s}_t$ and object $\mathbf{o}_t$ are tokens shown in the text $\mathcal{S}$, and $\mathbf{r}_t$ is a relation from the relation set $\mathcal{R}$, i.e. $\mathbf{s}_t, \mathbf{o}_t \subset \mathcal{S}$, $\mathbf{r}_t \in \mathcal{R}$ for $t = 1, 2,..,T$.
For training set $D = \{(\mathbf{x}^i, \mathbf{y}^i)_{i = 1, 2.., |D|}\}$, we train the generative model with parameter $\theta$ using MLE by
minimizing the negative log-likelihood over $D$:
$$
\mathcal{L}(D) = -\sum_{i = 1}^{|D|}\log p_\theta(\mathbf{y}^i | \mathbf{x}^i).
$$

\paragraph{Post-processing}

The generated output $\mathbf{y}$ is in the text format, which should be transformed to standard triples format $\{<\mathbf{s}_i,\mathbf{r}_i,\mathbf{o}_i>\}_{i = 1,2..,T}$ for evaluation.
Therefore, we apply some post-processing steps, including \textit{split}, \textit{delete} and \textit{replace}, to transform the generated output.
Token ``;'' is used to separate triples in the training stage, hence the generated output can be \textit{split} by token ``;'' to obtain different triples in text format.
For each generated triple, the relation should belong to the relation set $\mathcal{R}$.
If no valid relation is found in the generated triple, we will \textit{delete} the corresponding triple.
In Relation Classification task, since the gold entities $\mathcal{E}$ are given, we match the generated entities to the gold entities using Levenshtein similarity \citep{navarro2001guided}. In the Relation Classification task, we would \textit{delete} the generated triple if the generated entities show low similarity to all golden entities,
i.e. $\max_{\mathbf{e}_i \in \mathcal{E}} L_{sim}( \hat{\mathbf{e}}, \mathbf{e}_i) < \epsilon$
where $L_{sim}$ denotes Levenshtein similarity, $\hat{\mathbf{e}}$ is the generated entity, and $\epsilon$ is a threshold\footnote{We use $\epsilon = 0.85$ in our experiments}.
Otherwise, we \textit{replace} the generated entity with the gold entity with the highest Levenshtein similarity.
In the Joint Entity and Relation Classification task, the generated entity should be shown in the original text.
Therefore, we match the generated entities to the set of sub-spans in the text, which is motivated by span-level prediction in Name Entity Recognition (NER) research \cite{li2019unified, yu2020named, fu2021spanner}.
We \textit{replace} a generated entity with the sub-span of text in the text with the highest Levenshtein similarity.

\section{Problem Definition } \label{sec:problem_def}

\begin{figure*}[ht]
  \centering
  \includegraphics[width=0.9\textwidth]{overview.png}
 \caption{Overall REKnow framework. See Section \ref{sec:model} for more details about each component.
}
 \label{fig:1}
\vspace{-1em}
\end{figure*}

All tasks in Table \ref{table:1} are formally formulated as follows.
We denote the plain text as $\mathcal{S} = \{w_i\}_{i = 1,2,..,N}$, where $w_i$ represents a word, and $N$ is the text length.
The knowledge provided in a dataset $\mathcal{D}$ is denoted as $\mathcal{K}_{\mathcal{D}}$, such as pre-defined relation set $\mathcal{R}$, entity positions $\mathcal{E}_{pos}$, and entity types $\mathcal{E}_{type}$.
The goal is to find all relation triples $\{<\mathbf{s}_i,\mathbf{r}_i,\mathbf{o}_i>\}_{i = 1,2..,T}$ in text $\mathcal{S}$, where
$\mathbf{s}_i$ and $\mathbf{o}_i$ are subject and object entities in $\mathcal{S}$,
$\mathbf{r}_i$ belonging to $\mathcal{R}$ is the relation between subject and object, and $T$ is the number of relation triples.
The RE problem can be formulated as follows:
\begin{equation}\label{eq:general_format}
   f(\mathcal{S} | \mathcal{K}_{\mathcal{D}}) = \{<\mathbf{s}_i,\mathbf{r}_i,\mathbf{o}_i>\}_{i = 1,2..,T}.
\end{equation}

For different tasks, $\mathcal{K}_{\mathcal{D}}$ may contain a different set of knowledge.
We consider three tasks: (1)~Entity Type aware Relation Classification, where $\mathcal{K}^{ETRC}_{\mathcal{D}} = \{\mathcal{R}, \mathcal{E}_{pos}, \mathcal{E}_{type}\}$; (2)~Relation Classification, where $\mathcal{K}^{RC}_{\mathcal{D}} = \{\mathcal{R}, \mathcal{E}_{pos}\}$; and (3)~Joint Relation and Entity Extraction $\mathcal{K}^{JREE}_{\mathcal{D}} = \{\mathcal{R}\}$.
We denote tasks (1)~and (2)~as entity position-aware case and task (3)~as entity position-absent case for simplicity in the following sections.

\section{Related Work}\label{sec:related}

\paragraph{Relational Fact Extraction}

We group the relational fact extraction tasks with pre-defined relation set into three categories based on different levels of information provided in the data and introduce the related methods accordingly.
In the first category, for data with entity span as well as corresponding entity types, SpanBERT \cite{joshi2020spanbert} built the relation classification model by replacing the subject and object entities with their NER tags such as \textit{``[CLS][SUBJ-PER] was born in [OBJ-LOC], Michigan, \ldots''}.
A linear classifier was added on top of the [CLS] token to predict the relation type in their applications.
LUKE \cite{yamada2020luke} constructed an additional Entity Type Embedding layer in their model construction to utilize the entity type information.

In the second category, data come with an entity span.
It is the most popular and standard format in the RE task.
Traditional methods \cite{kambhatla2004combining, zhou2005exploring} extracted the relation between entities based on feature engineering, which incorporated semantic, syntactic, or lexical features in the text.
After the fast development of the deep language model, like BERT \cite{devlin2018bert}, BERT-Entity \cite{soares2019matching} utilized the entity span information by adding a mention pooling layer on top of the BERT model.

In the third category, only text is provided in the data.
In such a case, models were developed to extract both entity and relation.
For pipeline-based methods,
NER techniques \cite{chiu2016named, huang2015bidirectional, fu2021spanner} were used to detect the span of entities, then relation classification models are applied based on the detected entities.
Researchers \cite{li2014incremental, miwa2014modeling} also noticed that sharing parameters between NER and RE models are important in the pipeline model.
For example, Pure \cite{zhong2020frustratingly} built separate encoders for entities and relations, where the side product of the entity model is fed into the relation model as input.
For joint extraction methods, Copy-RE \cite{zeng2018extracting} built the relation extraction model based on the copy mechanism. Tplinker \cite{wang2020tplinker} jointly extracted the relation and entity in novel handshaking tagging schema, which are constructed to deal with the issue of overlapping entities.
In this work, we propose a unified framework for the relation extraction task for all cases above.
Compared to a unified framework for NER \cite{yan2021unified} tasks, our work focuses on relational triplet extraction.

\paragraph{Knowledge-Enhanced Information Extraction}

With the development of KG construction, many NLP tasks benefit from the external well-organized knowledge in KG.
We group knowledge-enhanced methods in the RE task into two categories based on the knowledge-ingestion stage.
The first type is to use the KG in the pre-training stage. The key challenges for such knowledge-ingestion type are Heterogeneous Embedding Space (HES), which means the embedding spaces for KG and text would be heterogeneous, and Knowledge Noise (KN), which represents information in KG would divert the text from its correct meaning \cite{liu2020k}.
ERNIE \cite{zhang2019ernie} solved the HES by proposing a pre-training task that requires the model to predict entities based on the given entity sequence rather than all entities in KGs.
K-Adapter \cite{wang2020k} constructed a neural adapter to infuse different kinds of knowledge.
REBEL \cite{cabot2021rebel} used BART-large as the base model to generate relation triplets, which is pretrained on Wikidata. However, since the method was trained on the Wikidata with 220 different relation types, it may introduce knowledge noise implicitly.
MIUK \cite{li2021multi} leveraged the uncertainty characteristic in ProBase \cite{wu2012probase} across three views: mention, entity and concept view.
Numerical studies about integrating knowledge into NLU tasks \cite{xu2021does} showed that KG information can increase the performance by a significant amount.

Another type is to use the KG in downstream tasks directly.
One of the popular implementations is distantly supervised relation extractors \citealp{mintz2009distant}, which leveraged the information in Freebase \cite{bollacker2008freebase} to extract relations.
\citealp{liu2020knowledge} utilized KG to conduct information extraction via RL framework.
Our method uses entities linking to automatically ground entities in the text to public KG.
The experiment results show that the entity type in KG helps boost the performance in most cases.

\section{Introduction}

Numerous relational facts are hidden in the emerging corpus. The extraction of relational facts is beneficial to a diverse set of downstream problems.
For example, relational facts facilitate the development of dialogue systems with lower labor cost \cite{wen2016network, dhingra2016towards}. Knowledge Graph (KG) construction leverages relational fact extraction of everyday news to keep its KGs up-to-date \cite{shi2018open}. Even in the field of e-commerce, relation extraction of product descriptions helps build category information for products \cite{dong2020autoknow}.

\begin{figure}[ht]

\resizebox{1\columnwidth}{!}{%
 \centering
  \includegraphics[width=0.9\textwidth]{illu.png}
  }
 \caption{The three examples have identical syntax structures, but their corresponding relational facts
 are different. Therefore, background knowledge is very crucial to correctly extract relational facts.
}
 \label{fig:0}
\vspace{-1em}
\end{figure}

\begin{table}
\resizebox{1\columnwidth}{!}{%
\begin{tabular}{@{}lp{7.5cm}@{}}
\toprule
Task name                           & \textbf{Entity Type aware Relation Classification}                                                                                \\ \cmidrule(l){2-2}
Text $\dagger$  & Born in London in 1939 the son of a Greek   tycoon, Negroponte grew up in Britain, Switzerland and the United   States                                                                                \\ \cmidrule(l){2-2}
Entity Positions                   &  Negroponte: (51, 61) \\
                                 & United States: (103, 116) \\
Entity Types                                    &  Negroponte: PERSON \\
                                & United States: LOCATION  \\
                                    \midrule
Task   name                         & \textbf{Relation Classification}                                                                                \\ \cmidrule(l){2-2}

Text $\ddagger$ & The   cutting machine contains 13 circular blades mounted on a cutting axis.                                                                                \\ \cmidrule(l){2-2}
Entity Positions                                    & machine: (12, 19)   \\
                                                    & blades: (41, 47)                                                                                \\ \midrule
Task   name                         & \textbf{Joint Entity and Relation Extraction}                                                                                \\ \cmidrule(l){2-2}
Text $\mathsection$    & It is Japan's second-biggest automaker,   behind Toyota and ahead of Nissan.
\\ \bottomrule
\end{tabular}}
\caption{Different type of relational fact extraction tasks. $\dagger$  example is obtained from TACRED \citep{zhang2017tacred}; $\ddagger$ from SemEval2010 \cite{hendrickx2019semeval}; $\mathsection$ from NYT10 \cite{riedel2010modeling}.
See Section \ref{sec:problem_def} for more details.}
\label{table:1}
\vspace{-1em}
\end{table}

Although relation extraction (RE) has been widely studied, we observe two shortcomings of previous work.
First, there is no unified framework for different settings of relation extraction tasks.
Table \ref{table:1} summarizes the popular relational fact extraction tasks.
The formats of input and output vary by task.
In the Entity Type aware Relation Classification task, models take the plain text and entity positions and entity types as input and predict relation types for all entity pairs.
For the Relation Classification task, entity types are not provided.
In the Joint Entity and Relation Extraction task, models only take plain texts as input and predict all possible entities and relations among them.
Previous work usually designs different model frameworks to handle these tasks separately \citep{joshi2020spanbert, yamada2020luke, soares2019matching, wei2019novel, cabot2021rebel}.
Also, directly applying Joint Entity and Relation Extraction models to Relation Classification tasks would lead to a bad performance due to the lack of fully usage of entity information compared with task-specific models.

Second, conventional relation extraction models do not utilize external knowledge to resolve relational ambiguities.
However, background knowledge of entities can be very crucial for models to figure out the right relations.
For the three examples in Figure \ref{fig:0},
they have the same syntax structure: "\texttt{[entity1] from the [entity2], says}".
The main information for models to predict different relation types for them is the background knowledge of these involved entities.
For example, the facts of "Joseph Biden is a human" and "the United States is a country" determine the "nationality" relation. The facts of "Elon Musk is a human" and "Tesla is an enterprise" determine the "company" relation. The facts of "Peter Parker is a fictional human" and "Avengers is a fictional organization" determine the "member\_of" relation.
Lack of the background knowledge, it is very difficult for models to disambiguate.

Our solution to the first issue is to utilize a generative model framework.
The popular sequence-to-sequence frameworks are naturally equipped to solve multiple tasks with variations in input and output formats.
Moreover, previous studies \citet{ye2020contrastive} and \citet{josifoski2021genie} have shown the effectiveness of the generative framework for each specific RE task.
In this work, we aim to unify all types of RE tasks with a generative model.
To address the second issue, we propose to leverage the well-organized information from the knowledge graph (KG).
We employ an entity linking model to connect all entities within the plain text with their corresponding entries in KG, then explicitly feed the knowledge of these entities into the RE model for relation prediction.
Previous methods \citep{zhang2019ernie, wang2020k, liu2020k} have shown that injecting KG into models during pre-training can boost the performance in downstream tasks.
However, involving KG during pre-training can be very expensive, and the model may suffer the catastrophic forgetting issue.
Differently, our method directly feeds relevant knowledge into the downstream model, thus can be more efficient and effective.

In this paper, we propose a unified generative framework for \textbf{R}elation \textbf{E}xtraction with \textbf{Know}ledge enhancement (\textbf{REKnow}). Our generative framework is trained to sequentially generate relational facts token-by-token.
Also, our framework enhances the input plain text with relevant KG information.
Our experiments show that the proposed framework significantly boosts the extraction performance on multiple benchmarks and task settings.
To our best knowledge, this is the first work to integrate KG into a generative framework for relational fact extraction.
In summary, our contributions are threefold:
\begin{itemize}
    \item We construct a novel knowledge-enhanced generative framework for relational fact extraction task.
    \item We provide KG grounding predictions for public relation extraction benchmark datasets.\footnote{Code, models and grounded KG will be available at GitHub.}
    \item Our generative model achieves superior performance on multiple benchmarks and settings, including ACE2005, NYT10, and TACRED.
\end{itemize}

\section{Conclusion}

We proposed a knowledge-enhanced unified generative model for a relational fact extraction task that generates triples sequentially with one pass.
We showed that it outperforms previous state-of-the-art models
on multiple relation extraction benchmarks.
\section{Experiments}\label{sec:exp}
In this section, we compare our proposed method to the state-of-the-art of relational fact extraction methods --- demonstrating its effectiveness and ability to extract relational facts under multiple tasks.
Our codes will be made publicly available.

\subsection{Data}

\begin{table}[]
\resizebox{1\columnwidth}{!}{%
\tiny
\begin{tabular}{@{}lll@{}}
\toprule
\multirow{2}{*}{Dataset} & Data Size          & Ave. Triple Size \\
                         & (train/val/test)   & (train/val/test) \\ \midrule
TACRED                   & 10,021/3,894/2,307 & 1.30/1.40/1.44   \\
SemEval2010              & 6,507/1,493/2,717  & 1.00/1.00/1.00      \\
NYT10                    & 56,196/5,000/5,000 & 2.01/2.02/2.03   \\
Webnlg                   & 5,019/500/703      & 2.74/3.11/2.82   \\
ACE2005                  & 2,619/648/590      & 1.83/1.82/1.95   \\ \bottomrule
\end{tabular}
}
\caption{Statistics of relational fact extraction benchmark datasets. Avg Triple Size represents the average size of triples in each input data. }
\label{table:2}
\vspace{-1em}
\end{table}

Several public available RE datasets are evaluated in our numerical experiments.
\textbf{TACRED} \citep{zhang2017tacred} is a large supervised RE dataset,
which is derived from the TAC-KBP relation set, with labels obtained via crowdsourcing.
It contains 41 valid relation types as well as one null relation type and 12 entity types.
Although alternate versions of TACRED have been published recently \cite{alt2020tacred, stoica2021re}, we use the original version of TACRED, with which the state of the art is mainly tested.
\textbf{SemEval 2010 Task 8} \citep{hendrickx2019semeval} is a benchmark dataset for evaluating classification of semantic relations between pairs of nominal entities, such as ``part-whole'', ``cause-effect'', etc.
It defines 9 valid relation types as well as one "other" relation type and no entity type is provided.
We follow the split setting as in \cite{soares2019matching} for numerical study.
\textbf{NYT10} dataset \citep{riedel2010modeling} was originally produced by distant supervision method from 1987-2007 New York Times news articles.
Original data are adapted by \cite{zeng2018extracting} for complicated relational triple extraction task, where EntityPairOverlap (EPO) and SingleEntityOverlap (SEO) widely existed in the dataset.
\textbf{WebNLG} dataset \citep{gardent2017creating} was originally created for Natural Language Generation task.
\citealp{zeng2018extracting} adapted WebNLG for relational triple extraction task as well.
In our experiment, we evaluate the proposed method on NYT10 and WebNLG datasets with whole entity span as in prior works \cite{ wang2020tplinker, ye2020contrastive} other than datasets with the last word of the entities \cite{zeng2018extracting, wei2019novel}.
\textbf{ACE2005} \cite{ace2005} is developed by the Linguistic Data Consortium (LDC) containing a collection of documents from a variety of domains including news and online forums.
In the dataset, 6 relation types between the entities are provided.
In our implementation, we focus on the RE task in ACE2005, hence we only keep the sentences with valid triples.
The detailed statistics about all datasets are summarized in Table \ref{table:2}.
For the evaluation, we report the standard micro Precision, Recall, and F1-score.

\begin{table}[]
\resizebox{1\columnwidth}{!}{%
\begin{tabular}{@{}p{1.6cm}llll@{}}
\toprule
Dataset                      & Method                                  & Prec. & Rec. & F1            \\ \midrule
\multirow{8}{*}{TACRED}      & ERNIE \cite{zhang2019ernie}             & 80.0  & 66.1 & 68.0          \\
                             & SpanBERT \cite{joshi2020spanbert}       & 70.8  & 70.9 & 70.8          \\
                             & K-Adapter \cite{wang2020k}              & 68.9  & 75.4 & 72.0          \\
                             & RoBERTa \cite{wang2020k}                & 70.2  & 72.4 & 71.3          \\
                             & LUKE \cite{yamada2020luke}              & 70.4  & 75.1 & \textbf{72.7} \\ \cmidrule(l){2-5}
                             & REKnow (ours)                           & 75.7  & 73.6 & \textbf{74.6} \\
                             & REKnow    w/o KG                & 75.4  & 72.4 & 73.9          \\
                             & REKnow using orig. entity type            & 75.6  & 73.0 & 74.3          \\ \midrule
\multirow{5}{*}{Semeval} & CR-CNN    \cite{santos2015classifying}  & -     & -    & 84.1          \\
                             & BERT-Entity   \cite{soares2019matching} & -     & -    & 89.2          \\
                             & BERT-MTB \cite{soares2019matching}      & -     & -    & \textbf{89.5} \\ \cmidrule(l){2-5}
                             & REKnow (ours)                           & 87.1  & 92.3 & \textbf{89.6} \\
                             & REKnow    w/o KG                & 86.0  & 91.5 & 88.7          \\ \bottomrule
\end{tabular}
}
\caption{Main result for entity position-aware case. The top two performing models per dataset are marked in bold. In REKnow, the pre-trained model is T5-large and entity type is obtained by Equation \eqref{eq:el1}.}
\label{table:main_1}
\vspace{-1em}
\end{table}

\subsection{Model Implementation Details} \label{subsec:model_implement}
All experiments are conducted with 8-cores NVIDIA Tesla V100 GPUs with 2.5 GHz (base) and 3.1 GHz (sustained all-core turbo) Intel Xeon 8175M processors.
\paragraph{Decoder} For the BART model, \citealp{shakeri2020end} found that using a variant of nucleus sampling \citep{holtzman2019curious} can increase the diversity of generated output compared to beam search.
We observe similar findings in our preliminary experiments; therefore, we adopt the \textit{Topk+Nucleus} decoding strategy in our decoding step for the BART model.
To be specific, we pick top $k = 20$ tokens, and within top k, tokens with top 95\% probability mass are picked in each generated step.
For the T5 model, we simply adopt the greedy decoding strategy, which can generate triples well.
\paragraph{Entity Linking} We use the pre-trained BLINK and ELQ model\footnote{\url{https://github.com/facebookresearch/BLINK} (MIT License)} to align Wikipedia entity for entity span aware and absent cases, respectively.
The entity linking models are trained on 5.9M entities from May 2019 English Wikipedia dump.
We follow the same threshold score (-4.5) as in the original ELQ implementation, and the top 1 grounded Wikipedia entity is used.
For dataset Semeval 2010, the entities are nominal, hence we use the property "subclass of"
of the retrieved entity as entity types.
For datasets TACRED, NYT10, WebNLG and ACE2005, we use property "instance of"
to obtain corresponding entity types.

\subsection{Results}\label{subsec:result}

\begin{table}[]
\resizebox{1\columnwidth}{!}{%
\begin{tabular}{@{}p{1.2cm}llll@{}}
\toprule
Dataset                  & Method                                        & Precison & Recall & F1            \\ \midrule
\multirow{11}{*}{NYT}    & NovelTagging \cite{zheng2017joint}            & 32.8     & 30.6   & 31.7          \\
                         & MultiHead   \cite{bekoulis2018joint}          & 60.7     & 58.6   & 59.6          \\
                         & ETL-Sapn \cite{yu2019joint}                   & 85.5     & 71.7   & 78.0          \\
                         & Tplinker \cite{wang2020tplinker}         & 91.4     & 92.6   & 92.0 \\
                         & CopyRE* $\dagger$  \citep{zeng2018extracting} & 61       & 56.6   & 58.7          \\
                         & CopyMTL* $\dagger$  \cite{zeng2020copymtl}    & 75.7     & 68.7   & 72.0          \\
                         & TANL \cite{paolini2021structured} $\dagger$  &  -   & -  & 90.8         \\
                         & REBEL \cite{cabot2021rebel} $\dagger$                           & 93.3        & 93.5     & \textbf{93.4} \\
                         & CGT $\dagger$  \citep{ye2020contrastive}      & 94.7     & 84.2   & 89.1          \\ \cmidrule(l){2-5}
                         & REKnow    (ours)                              & 91.6     & 93.6   & \textbf{92.6} \\
                         & REKnow w/o KG                         & 90.5     & 92.7   & 91.6          \\ \midrule
\multirow{9}{*}{Webnlg}  & NovelTagging    \cite{zheng2017joint}         & 52.5     & 19.3   & 28.3          \\
                         & MultiHead   \cite{bekoulis2018joint}          & 57.5     & 54.1   & 55.7          \\
                         & ETL-Sapn \cite{yu2019joint}                   & 84.3     & 82.0   & 83.1          \\
                         & Tplinker \cite{wang2020tplinker}         & 88.9     & 84.5   & \textbf{86.7} \\
                         & CopyRE* $\dagger$  \citep{zeng2018extracting}          & 37.7     & 36.4   & 37.1          \\
                         & CopyMTL*  $\dagger$ \cite{zeng2020copymtl}             & 58.0     & 54.9   & 56.4          \\
                         & CGT $\dagger$ \citep{ye2020contrastive}                 & 92.9     & 75.6   & 83.4          \\ \cmidrule(l){2-5}
                         & REKnow (ours)                                 & 87.5     & 86.6   & \textbf{87.0} \\
                         & REKnow    w/o KG                      & 84.0     & 84.3   & 84.2          \\ \midrule
\multirow{7}{*}{ACE2005} & Attention \cite{katiyar2017going}             & -        & -      & 55.9          \\
                         & DYGIE \cite{luan2019general}                  & -        & -      & 63.2          \\
                         & DYGIE++ \cite{wadden2019entity}               & -        & -      & 63.4          \\
                         & Pure-Bb \cite{zhong2020frustratingly}     & -        & -      & 66.7          \\
                         & Pure-Alb \cite{zhong2020frustratingly}      & -        & -      & \textbf{69.0} \\ \cmidrule(l){2-5}
                         & REKnow (ours)                                 & 71.0     & 65.7   & \textbf{68.3} \\
                         & REKnow    w/o KG                      & 70.3     & 65.0   & 67.5          \\ \bottomrule
\end{tabular}
}
\caption{Main result for entity position-absent case. * denotes the methods that only generate the last token of the entity. $\dagger$ denotes the competitors are generative-based models. The top two performing models per dataset are marked in bold. In REKnow, the pre-trained model is T5-large and entity type is obtained by Equation \eqref{eq:el2}.}
\label{table:main_2}
\vspace{-1em}
\end{table}
\begin{figure}[]
\centering
\resizebox{0.9\columnwidth}{!}{%
  \includegraphics[width=1\textwidth]{triple_size.png}
  }
 \caption{F1 Score under different triple sizes for REKnow. The \textit{triple size} is the number of triples hidden in a sentence.

}
 \label{fig:2}
 \vspace{-1em}
\end{figure}

\begin{table}[h]
\resizebox{1\columnwidth}{!}{%
\tiny
\begin{tabular}{@{}lllll@{}}
\toprule
Dataset  & REKnow  &  w/o KG & w/o Text & Ensemble \\ \midrule
NYT                        & 92.6    & 91.6      & 87.0                & \textbf{93.3}     \\
WebNLG                     &  87.0     & 84.2      &   63.7            & \textbf{87.4}     \\
ACE2005                    & \textbf{68.3}    & 67.5      &    9.6              & 67.9     \\ \bottomrule
\end{tabular}
}
\caption{Analysis of knowledge enhancement. ``REKnow'' is the proposed method.  ``REKnow w/o KG'' means raw text is used. ``REKnow w/o Text'' denotes only found KG information is used.  ``Ensemble'' means majority voting from the previous three methods. The F1 scores are reported.}
\label{table:KG_ablation}
\vspace{-1em}
\end{table}

\begin{table}[]
\resizebox{1\columnwidth}{!}{%
\tiny
\begin{tabular}{@{}lll@{}}
\toprule
\multirow{2}{*}{Dataset} & Found Ext. Info Ratio  & Entity Span \\
                         & (train/val/test)       & Aware       \\ \midrule
TACRED                   & 0.87/0.83/0.78         & Yes         \\
SemEval2010              & 0.47/0.45/0.45         & Yes         \\
NYT10                    & 1.75/1.76/1.76         & No          \\
Webnlg                   & 0.93/0.91/0.97         & No          \\
ACE2005                  & 0.75/0.71/0.66         & No          \\ \bottomrule
\end{tabular}
}
\caption{Statistics of entity linking. ``Found Ext. Info Ratio'' represents the ratio between the size of entities in the text and the size of found external information in entity linking step. Low ratio represents few external information is found and vice versa.}
\label{table: found_el_ratio}
\vspace{-1em}
\end{table}

For Relation Classification tasks, we summarize the result in Table \ref{table:main_1}.
The state-of-the-art methods are listed and compared. More introduction about the competing methods can be found in Section \ref{sec:related}.
``REKnow'' denotes our proposed method.
``REKnow w/o KG'' represents no external information from KG is used.
For TACRED dataset, ``REKnow w/o KG'' is 1.2 F1 score higher than LUKE \cite{yamada2020luke} method. It shows the effectiveness of our unified framework.
Also, ``REKnow'' is 0.7 F1 scores higher than ``REKnow w/o KG'', which indicates that enhanced knowledge further boosts the performance.
Since the entity types are already provided in TACRED, we also report the result using the original entity type in ``REKnow using orig. entity type".
It can be found that using the original entity type is 1.6 F1 scores higher than the LUKE method but slightly lower than that of the grounded entity information by our method.
The reason is that original entity type in TACRED is a coarse-grained level entity type while WikiData can provide a fine-grained entity information.
For SemEval2010, our method achieves performance comparable to one of the most popular methods BERT-MTB \cite{soares2019matching}.

For Joint Entity and Relation Extraction tasks,  we summarize the result in Table \ref{table:main_2}.
For NYT dataset, we can find that our ``REKnow'' method outperforms the current SOTA generative-based model, for example, a 0.6 F1 score increase compared with the REBEL method, which is pretrained on a constructed dataset ``CROCODILE'' \cite{cabot2021rebel}.
And 3.5 F1 score higher than CGT \cite{ye2020contrastive} in WebNLG dataset.
The performance of our method is comparable with SOTA extractive-based method Tplinker \cite{wang2020tplinker}.
For ACE2005, ``REKnow'' achieves a good F1 score with 68.3, which is close to the SOTA method Pure-Alb \cite{zhong2020frustratingly} which is a pipelined framework constructed for entity and relation extraction task.

Also, from Table \ref{table:main_2}, we notice that ``REKnow'' has the highest recall, which means the method can cover more potential triples.
The performance of different triple sizes can be found in Figure \ref{fig:2}.
We can see that for NYT and WebNLG datasets, our method has a better performance when the triple size is greater than 1.

\begin{table}[]
\resizebox{1\columnwidth}{!}{%
\begin{tabular}{@{}llllll@{}}
\toprule
Model           & NYT  & WebNLG & ACE2005 & TACRED & Semeval \\ \midrule
T5-large   & 92.6 & 87.0   & 68.3    & 74.6   & 89.6    \\
BART-large & 89.7 & 75.8   & 59.4    & 65.8   & 88.5    \\ \bottomrule
\end{tabular}
}
\caption{Pre-trained model comparisons in proposed REKnow method for WebNLG. T5-large has better performance. }
\label{table:pretrain_model_comparison}
\vspace{-1em}
\end{table}
\paragraph{Analysis of Knowledge Enhancement}
In REKnow, both text and entity-related information from KG are used.
To study the usefulness of external information, we re-train the model with different information source components.
We summarize the results in Table \ref{table:KG_ablation}.
To be noticed that the founded external information in NYT dataset is very useful and achieve 87.0 F1 score without using the raw text.
We also present the ``Ensemble'' result which ensembles the models trained with different information components by majority voting. It shows that in NYT and WebNLG, ensemble can further boost the performance.
On the contrary, in the case of ACE2005, the external information may contain too much knowledge noise therefore lowering the performance.

To investigate the performance of entity linking, we also present the ``found ext. info ratio'' by calculating the ratio between the size of entities in the text and the size of found external information.
The results for all RE benchmarks are summarized in Table \ref{table: found_el_ratio}.
Low ratio represents few external information is found and vice versa.
Based on the result, we can see that, for the SemEval2010 dataset, the founded ext. ratio is low, because the entities in SemEval2010 are nominal, and hard to find corresponding entity types from Wikidata.
That explained the ``REKnow w/o KG'' is close to ``REKnow'' in Table \ref{table:main_1}.
For the NYT10 dataset, the size of grounded entities is much larger than the size of actual entities.
It means the entities widely exist in the plain text, and the many relations among entities are not included as triples in the dataset.

\begin{table}[]
\resizebox{1\columnwidth}{!}{%
\begin{tabular}{@{}lp{7cm}@{}}
\toprule
Error reason     & The truth is incomplete                                                                                \\ \midrule
Dataset          & WebNLG (traceID : test\_578)                                                                                \\ \midrule
Original Text    & 1634: The Bavarian Crisis is the sequel to The Grantville Gazettes .                                                                                \\ \midrule
Text with EL     & 1634: The Bavarian Crisis is the sequel to The Grantville Gazettes .   [gr] 1634: The Bavarian Crisis is an instance of literary work [gr] The   Grantville Gazette is an instance of literary work \\ \midrule
Generated Triple & 1634: The Bavarian crisis precededBy The Grantville Gazettes                                                                                \\ \midrule
Truth (s, r, o)  & (``1634 : The Bavarian'', ``precededBy'', ``The Grantville Gazettes'')                                                                                \\ \midrule
Analysis         & ``1634: The Bavarian crisis'' should be treated as a whole.   \\ \midrule \midrule

Error reason     & Generative model has much wider searching space                                                                                \\ \midrule
Dataset          & Webnlg (traceID : test\_633)                                                                                \\ \midrule
Original Text    & Bill Oddie 's daughter is Kate Hardie .                                                                                \\ \midrule
Text with EL     & Bill Oddie 's daughter is Kate Hardie . [gr] Bill Oddie is an instance of   human [gr] Kate Hardie is an instance of human                           \\ \midrule
Generated Triple & Bill Oddie daughter Kate Hardie                                                                                \\ \midrule
Truth (s, r, o)  & (``Bill Oddie'', ``child'', ``Kate Hardie'')                                                                                \\ \midrule
Analysis         & ``daughter'' is not in the pre-defined relation set     \\

\bottomrule
\end{tabular}
}
\caption{Example of Error Analysis. More examples can be found in Table \ref{table:error_full} in the Appendix \ref{appen:B}}
\label{table:error}
\vspace{-1em}
\end{table}
\paragraph{Ablation Study}
We study the comparison of whether to use the data augmentation component (Section \ref{sec:model}) for entity position-absent case.
In NYT dataset, removing data augmentation will decrease the F1 score 2.0. In WebNLG and ACE2005, no data augmentation will decrease the F1 score 9.8 and 6.1, respectively.
It represents that exposure bias will affect the performance for generative-based method and using data-augmentation strategy can alleviate the exposure bias significantly.

We also conduct an ablation study of the pretrained model, BART and T5. The comparison result is summarized in Table \ref{table:pretrain_model_comparison}.
From the table, we can observe that T5-large is generally better than the BART-large model.
To be specific, in datasets NYT and Semeval, the performances between T5 and BART are close. As for the rest datasets, the BART model decreases about 8 to 10 F1 scores compared with T5.
It would be interesting to study the reason for the heterogeneous difference among datasets.
We leave that as a direction for future investigation.

\paragraph{Error Analysis}

Based on the error analysis in Table \ref{table:error}, we notice that the common error reasons include incomplete truth labels.
The triple provided in the dataset is (``1634: The Bavarian'', ``precededBy'', ``The Grantville Gazettes''), but ``1634: The Bavarian crisis'' is a book title --- REKnow correctly predicted this as a whole entity.
Another error is that generative model may create relation out of the pre-defined relation set. In the second error case in Table \ref{table:error}, ``daughter'' is not a pre-defined relation.
This issue may be addressed by constraint generation, which needs future investigation.

\section{Full Table of Model Ablation}\label{appen:a}

\section{Error Analysis}\label{appen:B}

Based on the error analysis in Table \ref{table:error_full}, we find the common error reasons include incomplete truth labels, too wide generation space and missing entity linking result.
In Case 2, the truth triplets should contain all possible relations between a borough ``Manhattan'' and some places including ``Washington Heights'', ``The Battery'' and ``Harlem River''.
In Case 3, if an inversable relation, such as ``org\_alternate\_names", exists, the generated model may generate the triplet in a wrong order compared with golden label.
In Case 4, generative model may create relation out of the pre-defined relation set. This issue may be addressed by constraint generation, which is under future investigation.
In Case 5, the entity linking result may fail to provide enough background information.
Hence the generated triple confuse with the relation between ``sounds'' and ``species''.

\begin{table*}
\resizebox{1.8\columnwidth}{!}{%
\begin{tabular}{@{}llp{12cm}@{}}
\toprule
\multicolumn{2}{l}{Error   reason :}                                & The truth is incomplete                                                                                \\ \midrule
\multicolumn{1}{l|}{Case 1} & \multicolumn{1}{l|}{Dataset}          & Webnlg (traceID : test\_578)                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Text with EL}     & 1634 : The Bavarian crisis is the sequel to The Grantville Gazettes .   [gr] 1634: The Bavarian Crisis is an instance of literary work [gr] The   Grantville Gazette is an instance of literary work                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Generated Triple} & 1634 : The Bavarian crisis precededBy The Grantville Gazettes                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Truth (s, r, o)}  & (``1634 : The Bavarian'', ``precededBy'', ``The Grantville Gazettes'')                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Analysis}         & ``1634 : The Bavarian crisis'' should be treated as a whole part                                                                                \\ \midrule
\multicolumn{1}{l|}{Case 2} & \multicolumn{1}{l|}{Dataset}          & NYT (traceID : test\_582)                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Text with EL}     & Through a series of leisurely walks around the island -- from the Battery   to Washington Heights , and from Wall Street to the Harlem River -- Lopate   ruminates on Manhattan 's history , architecture and inhabitants . [gr] The   Battery (Manhattan) is an instance of urban park [gr] Washington Heights,   Manhattan is an instance of neighborhood [gr] Wall Street is an instance of   street [gr] Harlem River is an instance of strait [gr] Manhattan is an   instance of borough of New York City \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Generated Triple} & Washington Heights neighborhood\_of Manhattan ; Harlem River   neighborhood\_of Manhattan ; Manhattan contains Washington Heights ; Manhattan   contains Harlem River ; Manhattan contains the Battery;                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Truth (s, r, o)}  & (``Manhattan'', ``contains'', ``Washington Heights''), (``Washington Heights'',   ``neighborhood\_of'', ``Manhattan'')                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Analysis}         & Relations between ``Manhattan'' and ``The Battery'' or   ``Harlem River'' should be included in the truth                                                                                \\ \midrule
\multicolumn{1}{l|}{Case 3} & \multicolumn{1}{l|}{Dataset}          & TACRED (traceID : test\_1013)                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Text with EL}     & The two companies were preparing to announce that AIG had agreed to sell   [ms] American Life Insurance Co [gr] business , better known as [ms] Alico   [gr] business , for 68 billion dollars in cash and 87 billion in MetLife   equity , the report said .                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Generated Triple} & American Life Insurance Co org\_alternate\_names Alico ;                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Truth (s, r, o)}  & (``Alico'', ``org\_alternate\_names'', ``American Life Insurance Co'')                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Analysis}         & gnerative model is not sensitive to inverseable triple like : org's   alternate name                                                                                \\ \midrule
\multicolumn{2}{l}{Error reason :}                                  & Generative model has too wider searching space                                                                                \\ \midrule
\multicolumn{1}{l|}{Case 4} & \multicolumn{1}{l|}{Dataset}          & Webnlg (traceID : test\_633)                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Text with EL}     & Bill Oddie 's daughter is Kate Hardie . [gr] Bill Oddie is an instance of   human [gr] Kate Hardie is an instance of human                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Generated Triple} & Bill Oddie daughter Kate Hardie                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Truth (s, r, o)}  & (``Bill Oddie'', ``child'', ``Kate Hardie'')                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Analysis}         & ``daughter'' is not in the pre-defined relation set                                                                                \\ \midrule
\multicolumn{2}{l}{Error reason :}                                  & Entity linking is missing                                                                                \\ \midrule
\multicolumn{1}{l|}{Case 5} & \multicolumn{1}{l|}{Dataset}          & semeval (traceID : test\_876)                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Text with EL}     & the word ``song '' is used to describe the pattern of regular and   predictable [ms] sounds [gr] NA made by some [ms] species [gr] NA of whales ,   notably the humpback whale .                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Generated Triple} & species Producer\_Product sounds ; sounds Product\_Producer species ;                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Truth (s, r, o)}  & (``sounds'', ``Effect\_Cause'', ``species''), (``species'', ``Cause\_Effect'',   ``sounds'')                                                                                \\ \cmidrule(l){2-3}
\multicolumn{1}{l|}{}       & \multicolumn{1}{l|}{Analysis}         & The model treat the relation between ``sounds'' and   ``species'' as product-type relation when Entity Linking is missing   in this case                                                                                \\ \bottomrule
\end{tabular}
}
\caption{Error Analysis for REKnow. }
\label{table:error_full}
\end{table*}

\section{Hyper-parameter in Model Implemetation}\label{appen:c}

We adopt the following setting for model training:
\begin{itemize}
    \item learning\_rate : 5e-5
    \item max\_source\_length : 1024
    \item max\_target\_length : 128
    \item num\_train\_epochs : 3
    \item per\_device\_train\_batch\_size : 4 (for T5-large), 16 (for T5-base, T5-small and BART)
    \item nproc\_per\_node : 8
    \item optimizer : AdamW
    \item lr\_scheduler\_type : linear
\end{itemize}

redpajama_set_name: RedPajamaArXiv