text:
How Christopher Peacock beat the recession
podcast | Aug 19, 2019 |
By Fred Nicolaus
It was 2008, and Christopher Peacock's name was flying on banners in Rockefeller Center. A kitchen by his luxury cabinetry company had won House Beautiful's inaugural Kitchen of the Year competition and Peacock's work was installed in a glass conservatory where thousands of tourists could gape at it. Oprah Winfrey saw it, loved it, and invited Peacock on her show. Things couldn't have been better. Then, he got a call from his partners in England: "You have to shut down the company."
Courtesy of Christopher Peacock
The situation was complicated—you'll have to listen to the latest episode of the Business of Home podcast (sponsored by Universal Furniture) to get the whole story—but in the end it came down to one thing: the recession. It wasn't Peacock's first—the British-born cabinetmaker had started his company in Connecticut in 1992, in the midst of another economic downturn. He survived that one, and 2008's as well. And if the hand-wringing on CNBC is to be believed, he's planning on surviving the coming slowdown too. Here are a few things Peacock has learned as a bona fide recession warrior.
Don't go down-market. In 2008, Peacock was in the same boat as many high-end companies: No one was buying. He considered going after a lower segment of the market, but ultimately held out. "I can cut my prices, chase every job. [But] I thought I'd undo everything I'd achieved—I was going to undervalue myself and my product," he tells podcast host Dennis Scully. "I believed business would come back, and I thought: When it does come back I don't want to undercut myself because I'll never be able to sell at the prices I need to sell at."
Treat the slowdown as an opportunity. Boom times make for busy times, which is mostly a good thing. But slowdowns can be an opportunity to refocus and take a close look at what you're doing. "When you're growing year on year, it's very easy to focus on growth and not focus on efficiency and cost effectiveness," says Peacock. "What [the recession] did was it allowed me the luxury of time to stop and analyze things. … When we started up again, every penny mattered."
Balance your pipeline. Longer-term projects can sometimes be a slog. Take them anyway. "It's that mentality of booking seats on a plane: You want to have the business class seats, which you sell for a lot more money and that's where the profit is," says Peacock. "Then at the back, we have a few multi-unit projects, which are not as profitable but smooth out the ups and the downs. It's the combination that makes for a healthy business."
Think outside the box (or the kitchen). "What I had to do was figure out ways to sell more things to less people," says Peacock of the 2008 marketplace. Up until that time, his business had primarily been in designing kitchens. The great recession spurned him to move more aggressively into other areas of the house. "Every client we had was an opportunity we weren't necessarily maximizing, so I reinvested in the showrooms by putting in dressing rooms and bathroom displays and libraries so that when a client came in, they understood we were a solution for the entire house."
Holland & Sherry's Bryan Dicker has no plans to go direct to consumers
Kate Verner on how licensing has evolved
David Netto on a life in design
What's the future of design centers?
Carleton Varney reflects on a colorful career
Fort Street Studio founders on the business of being creative


text:
Andrew Cely, Sports Producer/Reporter
Published: November 13, 2021, 6:05 PM
Updated: November 13, 2021, 7:34 PM
Tags: UTSA, College Football, College Sports, Jeff Traylor, San Antonio, Frank Harris, Sincere McCormick
No. 15 UTSA uses strong second half to top Southern Miss 27-17, stay undefeated
Defense forces three turnovers in 4th quarter to seal comeback victory
Published: November 13, 2021, 6:05 PM Updated: November 13, 2021, 7:34 PM
Wide receiver Joshua Cephus scores a 40-yard touchdown in the second quarter of UTSA's 27-17 victory over Southern Miss at the Alamodome on Nov. 13, 2021. (Adam B. Higgins, KSAT)
SAN ANTONIO – In front of 30,000-plus fans inside the Alamodome, No. 15 UTSA rallied from a 17-10 third-quarter deficit to defeat Southern Miss 27-17 and remain undefeated on the season.
The Roadrunners fell behind on the second play of the second half, as Golden Eagles defender Natrone Brooks returned an interception 34 yards for a go-ahead touchdown. In spite of some offensive struggles, quarterback Frank Harris then capped an eight-play, 58-yard drive at the end of the third quarter with a 24-yard touchdown pass to Leroy Watson that tied the game at 17. Harris finished the game completing 17 passes for a pair of TDs and two interceptions.
UTSA's defense came to the rescue, forcing three crucial turnovers. First, Trevor Harmanson forced a fumble that was recovered by Jahmal Sam at the 35-yard line and eventually led to a 24-yard field goal from Hunter Duplessis that gave the Roadrunners a 20-17 lead. On Southern Miss' next possession, Corey Mayfield Jr. delivered a strip sack, and Charles Wiley returned the ensuing fumble to the nine yard line. On the very next play, running back Sincere McCormick found paydirt on a nine-yard touchdown run for a 27-17 advantage. McCormick carried the ball 19 times for 90 yards and that touchdown. Three plays later, Antonio Parks intercepted the Golden Eagles' quarterback, allowing the offense to run out the remainder of the clock and seal a hard-fought victory.
FINAL! No. 15/16 UTSA outlasts Southern Miss to improve to a school-record 10-0! NEXT: UTSA hosts defending league champ UAB on Saturday at 2:30 pm at the @Alamodome.
🔗 https://t.co/nkdFyO8EZd#BirdsUp 🤙 pic.twitter.com/BVoSqCFIAs
— 🔶 UTSA Football 🏈 (@UTSAFTBL) November 13, 2021
With the win, UTSA improves to 10-0, 6-0 against Conference USA opponents. They are now one of just three undefeated teams remaining in college football. Next up, the Roadrunners will host UAB (6-3, 4-1) at the Alamodome on Saturday, Nov. 20 at 2:30 p.m.
Andrew Cely
Andrew has covered athletics of all levels for more than five years and is dedicated to shining a spotlight on local San Antonio athletes and their stories.

redpajama_set_name: RedPajamaCommonCrawl


Please, don't miss to enjoy watching this exciting match between Alashkert - CFR Cluj. See you on games between CFR Cluj vs Alashkert online streaming. For those who have missed the game, we also welcome request for the CFR Cluj vs Alashkert replay video. Now you don't have to worry that you are still at work while your favourite game between CFR Cluj vs Alashkert have started.
Enjoy watching CFR Cluj v Alashkert online game using our online broadcasting. Using this site you will find all your favourite teams and Alashkert vs CFR Cluj are playing now. Join us and be a part of a team.
Discussions about the hit events in the world of sports and teams: Alashkert - CFR Cluj. You can communicate with the best fans of team Alashkert vs CFR Cluj from different places, who have one goal - a win of their favorite team.


Paying a visit to the casino is always fun and exciting. Unlike the online casino Malaysia, where you can place your bet on your favorite slot game such as Highway Kings and Bonus Bear at the comfort of your home, the atmosphere in a land casino, is totally different. However, if you walk into any casino and place your bet blindly, there is a high chance that your journey to the casino will not be a memorable one. Casino players should have a game plan before entering any casino. Below are a few pointers that will serve as a guideline for your next casino trip.
Any experienced casino players will tell you that it is really important to get familiar with casino gaming tables. Before you even start your gambling, it is better to have a look at different machines and tables in the casino. You will most probably find out that most casino players would stick to their favorite games most of the time. Pick a few games that you are good at and stick to it.
Different casino games will have a different wager. It is always better to play games that are affordable by your own standard. For example, a slot game machine that read $0.05 cents doesn't mean that it is $0.05 per bet. What it really means is that you will be playing $0.05 for every line. If you are not careful, you might even be betting $100 for every round of play. Therefore, it is important to have a betting budget before you enter the casino.
It is better to bring a fixed amount of cash with you to the casino. Leave your credit card behind. Once your budget is fixed, the next step would involve choosing the right game for yourself. In this process, you may need to do some trial and error to observe the results of your bet. Never go for high stake casino game until you have the right strategy and enough money to play the game.
In most cases, casino players will choose to bet with slot machines. Therefore it is important to choose the right slot machine to place your bet. Place your bet on a few rounds with the slot machine would be the only way to find out if the slot machine suits you. If the slot machine is not giving you any winnings, leave the machine and go to the next.
Preparation does pay off. Do some research on the game that you are about to play. Be clear about the gameplay and rules before you place any bets.
Casino gambling can be addictive. It often gives an adrenaline rush for casino players. The atmosphere in the casino is always exciting. The massive amount of money changes hand every second. Therefore, it is important not to get carried away. Top players practice control and restraint. They are calculating all the time and will never take unnecessary risk.
Losing streak can be painful. One has to accept the fact that someday are just not your day. It is better to break your losing streak by stopping your play and come back again another day.
Lastly, enjoy your game. You don't want to wager your money and not enjoy it. If you are new to casino games, it is advisable to practice your bet in an online casino first. Any online casino would have a much wider variety of betting games as compared to the land casino.


text:
fanatix | Boxing | (Video) Boxing news: Tyson Fury predicted that Wladimir Klitschko would pull out
(Video) Boxing news: Tyson Fury predicted that Wladimir Klitschko would pull out
Heavyweight world title challenger Tyson Fury predicted that rival Wladimir Klitschko would pull out of their fight after the press conference in Germany.
As reported by IFL TV, the 27-year-old insisted that he wouldn't be challenging for a world title until the pair were actually in the ring and trading punches.
Watch more brilliant snippets like this with fanatix's sports video news app
It was confirmed on Friday that Klitschko had suffered a calf injury, with their October 24 bout now expected to be re-scheduled for later in the year or in early 2016.
The WBA, WBO and IBF heavyweight king was in London for the second press conference on Wednesday, but returned to his training camp in Austria after where he has suffered the setback.
Although he was left stunned by Fury's antics during that press conference, during which the challenger dressed up as Batman, he insisted that he would make him "eat his words", as reported by Sky Sports.
However, that process will now have to be put on hold as Klitschko will recover from the injury and attempt to defend his belts at a later date.
As confirmed by Sky Sports, Klitschko said: "Especially after the exciting press conference last Wednesday, I was looking forward to this fight very much. I know that a lot of fans have already organised their trips to support me in Dusseldorf.
"I am so sorry that I am not able to fight on October 24. I also apologise to my opponent, who has spent many weeks preparing for this fight. I will do my very best to recover as fast as possible."
More Stories: Tyson Fury, Wladimir Klitschko
Analysis: Tyson Fury deservedly crowned new heavyweight champion after dethroning Wladimir Klitschko (video)
Boxing Posted November 29, 2015 13:27
(Video) Tyson Fury continues to mock Wladimir Klitschko in fight week head-to-head
Boxing news: Wladimir Klitschko taunts Tyson Fury by insisting rivals could beat him (video)
(Video) Boxing news: Tyson Fury antics risk second postponement of Wladimir Klitschko clash
Boxing news: Tyson Fury reveals game plan to seal downfall of Wladimir Klitschko (video)
Boxing Posted November 5, 2015 18:50


They offered their support to the Prime Minister after the government was rocked by a spate of resignations, with supposed cabinet unity over her Brexit negotiating position in tatters. There were calls for May to scrap her proposals and tell the European Union that the U.K.is leaving without a deal, according to one person who was present.
He is a vote victor, and in its current crisis that is exactly what the Conservative Party needs. In 1958, the chancellor of the exchequer and two junior ministers resigned in protest at increased government spending.
"My personal opinion is that she has strengthened herself quite a bit", he told Al Jazeera in an interview from London.
The Tanaiste said there are lots of big personalities within the British Government and they have been quite vocal in terms of making their views known on Brexit.
Brexit minister Steve Baker also left the Government, while Chris Green quit his position as an unpaid parliamentary aide at the Department for Transport. In fact, in Trump-speak, "very good relationship' means "I can't stand him/her'".
Speaking in the House of Commons about the Chequers meeting, May said Cabinet had agreed to "consider seeking accession" to the 11-nation CPTPP trade deal.
Mr Johnson plunged the Government into crisis after he announced he was quitting with a scathing denunciation of her Brexit plans, saying they would leave the United Kingdom a "colony" of the European Union.
"If anything, they are driving prices higher as the United States defends many of their members for very little $'s". However, the Chinese government has not yet specified a date on which it may introduce duties on imports of US crude.
However, county MPs Daniel Kawczynski, Philip Dunne, Lucy Allan, Mark Pritchard and Montgomeryshire Conservative Glyn Davies have all urged their party colleagues to support the leader.
It leaves the four great offices of state - Prime Minister, Chancellor, Foreign Secretary, and Home Secretary - in the hands of ministers who voted to stay in the European Union, although in the case of Home Secretary Sajid Javid at least, with no great enthusiasm. She is trying to get a deal that will pass through Parliament.
"It is the ultimate betrayal of our democracy and people's belief in it".
Trump also said his secretary of state, Mike Pompeo, had not as reported given Kim Jong-un an Elton John CD containing the song Rocket Man - Trump's nickname for the North Korea leader - when they met in Pyongyang last week. He said: "I am disappointed David felt that he had to leave the government".
If British prime minister Theresa May had hoped to leave domestic Brexit woes behind her as she prepares for a fresh feud in Brussels over the future of North Atlantic Treaty Organisation, she was swiftly disabused by U.S. president Donald Trump.
You do wonder what coherent strategy those at the head of the Conservative Party have been employing since David Cameron stepped down.


text:
Direct electric action is a systems used in pipe organs  to control the flow of air (wind) into the organ's pipes when the corresponding keys or pedals are depressed. In direct electric action, the valves beneath the pipes are opened directly by electro-magnet solenoids, while with electro-pneumatic action, the electro-magnet's action admits air into a pneumatic or small bellows which in turn operates the pipe's valve.

Other types of actions used in pipe organs are tracker action, using mechanical linkage of rods and levers, and tubular-pneumatic action which utilizes a change of pressure within lead tubing which connects the key to the valve pneumatic.

Sources

William H. Barnes The Contemporary American Organ (1959)

Pipe organ components

redpajama_set_name: RedPajamaWikipedia


Bánh chuối (literally "banana cake") is a sweet banana cake or bread pudding from Vietnam. Although its exact ingredients may vary, it is usually made with ripe bananas or plantains, coconut milk, sugar, white bread, shredded young coconut, condensed milk, butter, egg, and vanilla extract. In the finished dish, the cooked banana often appears purplish-red in colour.
Bánh chuối nướng (literally "baked banana cake") - This variety of bánh chuối is cooked by baking it in a pan in an oven, giving it a golden-brown, crisp exterior.
Bánh chuối hấp (literally "steamed banana cake") is similar in appearance to the baked version, but some rice starch is added and it is steamed rather than baked, and the outside is not golden-colored.
Bánh chuối chiên, which is often sold as a flattened banana and sticky rice fritter.
Bánh chuối khoai, which includes slices of sweet potato.
40px This article about Vietnamese cuisine is a stub. You can help Infogalactic by expanding it.
This page was last modified on 24 April 2016, at 07:45.


Easy 1up Review - Great Business Opportunity or Cash Gifting Scam?
→Easy 1up Review – Great Business Opportunity or Cash Gifting Scam?
Hey everyone, welcome to my exclusive Easy 1up Review!
Chances are you have been approached by someone on social media about the business opportunity and you want to make sure it's legit right?
I have heard a lot of rumors about this company, mainly being called a scam, but I am going to put those rumors to rest.
Easy 1up Review – What is it?
When I checked out their website, there is NO information on who actually runs the show in Easy 1up.
I decided to do further investigating.
I checked out the domain registration and found that the Easy 1up website, "easy1up.com", was registered June 8th 2007.
Not to worry, I didn't stop there.
I looked at the hosting on the Easy 1up website and found out it was the same private server as Ultimate Cycler.
Easy 1up has no retail products or services but only the ability to become an Easy 1up affiliate and promote the actual membership.
When you do join, you gift anywhere between $25 to $500 to the affiliate who recruited them.
Basically, in the Easy 1up compensation you have affiliates purchase $25 to $1000 to the member who sponsored them.
Also every second member you sponsor gets sent to your upline along with the payment.
Like I mentioned earlier, you pass up your second commission to your sponsor which is called a 1up model.
Once you pass up this second payment, you can now start earning the rest yourself from your personally enrolled members.
Remember your personally enrolled members also pass up their second commission to you.
If you pay the biggest payment of $1000, this qualifies you to warn across all 4 tiers.
Another benefit of the $1000 payment is there are no pass up's to your sponsor.
There are no retail products or services, but the affiliate membership in Easy 1up costs $25 to $1000 fee and a 10% admin fee.
Does the Easy 1up scam exist?
To be honest, this company is still going strong and looks like Peter Wolfing did a great job with it.
The entry level is low and you and work your way up.
Only thing I recommend the company doing is to have some kind of retail side to the company which will make it more compliant.


text:
About | Contribute for Publication | Feedback & Comments | Contact
Up-to-the-minute perspectives on defence, security and peace
issues from and for policy makers and opinion leaders.
| View our Twitter page at twitter.com/defenceredbox |
Enable JavaScript of follow us on twitter.com/defenceredbox
Time to revisit UK air strikes on Syria? Join the debate
Following ISIL's attacks on Paris last Friday,Prime Minister David Cameron is now indicating he wants to revisit the debate on the extension of UK counter-ISIL air strikes to Syria. This would be not least as an important demonstration of solidarity with France, argues a new RUSI report. (Access at https://www.rusi.org/UK-SyriaAirsStrikes)
We invite readers to join in the debate on the political and military implications - the utility of force, war aims etc. "Stop the war" is a perfectly valid point of view but will be moderated out for these purposes.
The RUSI report cautions against overstating the benefits of air strikes on their own . It makes clear that operations could end without decisive strategic effect, and that the UK's military campaign 'will need to be calibrated on the assumption that it may have to be sustained over a period of several years.' While the UK's involvement would provide some additional specialist capabilities which the US does not possess, its relative size means that its effort will not be strategically transformative by itself.
Nevertheless, the paper argues, current US and French airstrikes in Syria already contribute to 'important second-order objectives.' These include the protection of Kurdish-majority areas in northern Syria, where 'if MPs accept that the US was right to use force to protect Kobani (with UK non-lethal support), it is hard to see how they can then justify a refusal in principle to authorise UK participation in future comparable operations.
Current air strikes in Syria also 'allow the coalition to attack ISIL's headquarters in Raqqa, the 6th largest Syrian population which had a poipulation of aound quarter of a million, ensuring that it has no safe haven from which to provide logistical and financial support to its operations in Iraq.
The report highlights three substantial military consequences of its analysis :
1. A higher level of air-power commitment, well above that currently deployed, could not be sustained for long without seriously eroding the UK's ability to respond to other demands that may arise in the medium term, be that in the Baltic states, Western Balkans, West Africa or Afghanistan.
2. The UK will need the capability to escalate its military effort in Iraq and Syria at relatively short notice, for short periods and, if necessary, without another parliamentary approval, as and when new opportunities present themselves.
3. If diplomatic efforts to resolve the Syrian war begin to bear fruit, the UK should be prepared to pledge forces in support of a UN-authorised peace-enforcement effort.
Ultimately, the report concludes that 'As long as the involvements of UK forces in Iraq - and, separately, Syria - continue to do some good, they should be continued. While the UK should not rule out the possibility of deepening its involvement in Syria in certain scenarios, however, it may also have to be prepared to walk away from military operations if necessary.' If from a strategic point of view you disagree, we'd be interested to know why.
Defence Blogs
British Army rumour service
China Defence Forum
China Defense
Defence Focus
Defence of the Realm
Defense Industry Daily (Global)
EuroDefense (with news feed)
India Defence Update
International Relations and Security Network
Missile Defense Advocacy Alliance (MDAA)
National Security Analyst
New Wars
RIA Novosti (Russia)
Royal Air Force rumour network
Royal Navy and Royal Marines rumour service
Rumour Control (Australia)
Strategic Forecasting
The Long War Journal (GWOT analysis)
Think Defence
U K Defence Forum website
U.S.Defense
We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we'll assume that you are happy to receive all cookies on the Defence Viewpoints website. However, if you would like to, you can modify your browser so that it notifies you when cookies are sent to it or you can refuse cookies altogether. You can also delete cookies that have already been set. You may wish to visit www.aboutcookies.org which contains comprehensive information on how to do this on a wide variety of desktop browsers. Please note that you will lose some features and functionality on this website if you choose to disable cookies. For example, you may not be able to link into our Twitter feed, which gives up to the minute perspectives on defence and security matters.
UKDF Defence Viewpoints is intended as a forum for political discussion and debate on security, defence, peace and strategic issues. The views of authors are theirs alone, and publication of them here should not be interpreted as an endorsement by the U K Defence Forum, which holds no views of its own, or any of its associates. We are grateful to thinktanks, academics and publications for permission to reproduce their material. Copyright remains with original authors even when anonymous.
(c) Great North News Services


Sanix Technology is a website Development company established in 2015. We specialize in developing Shopify E-commerce stores from scratch, Theme development/ Customization, PSD to Theme development, App integration, Payment Gateway integration, Shipping method integration, template customization etc.
We have a creative eye for designing and design the website as per latest designing trend. As a Shopify developer, we can design and build an easy-to-use and affordable store.
We provide full support after the store is setup and provide guidelines how to make changes yourself in store, so that a customer can manage the store himself.
Sania was very attentive to my needs. He work around my time availability and made adjustments as requested! I loved the design. And would recommend him to everyone!


text:
\section{Introduction}
Developing models that are capable of reasoning about complex real-world problems is challenging.  It involves decomposing the problem into sub-tasks, making intermediate decisions, and combining them to make the final prediction.
While many approaches develop black-box models to solve such problems, we focus on compositional structured models as they provide a level of explanation for their predictions via interpretable latent decisions, and should, at least in theory, generalize better in compositional reasoning scenarios.
For example, to answer \question{How many field goals were scored in the first half?} against a passage containing a football-game summary, a neural module network~\citep[NMN;][]{NMN16} would first ground the set of \emph{field goals} mentioned in the passage, then filter this set to the ones scored \emph{in the first half}, and then return the size of the resulting set as the answer.

Learning such models using just the end-task supervision is difficult, since the decision boundary that the model is trying to learn is complex, and the lack of any supervision for the latent decisions provides only a weak training signal.
Moreover, the presence of dataset artifacts~\citep[\emph{among others}]{Lai2014IllinoisLHAD, Gururangan2018AnnotationAI, Min2019CompositionalQD}, and degeneracy in the model, where incorrect latent decisions can still lead to the correct output, further complicates learning. As a result, models often fail to predict meaningful intermediate outputs and instead end up fitting to dataset quirks, thus hurting generalization~\citep{subramanian-etal-2020-obtaining}.

We propose a method to leverage related training examples to provide an indirect supervision to these intermediate decisions.
Our method is based on the intuition that related examples involve similar sub-tasks; hence, we can use an objective on the outputs of these sub-tasks to provide an additional training signal.
Concretely, we use \emph{paired examples}---instances that share internal substructure---and apply an additional training objective relating the outputs from the shared substructures resulting from partial model execution.
Using this objective does not require supervision for the output of the shared substructure, or even the end-task of the paired example. This additional training objective imposes weak constraints on the intermediate outputs using related examples and provides the model with a richer training signal than what is provided by a single example.
For example, \question{What was the shortest field goal?} shares the substructure of finding all \emph{field goals} with \question{How many field goals were scored?}.
For this \emph{paired example}, our proposed objective would enforce that the output of this latent decision for the two questions is the same.

We demonstrate the benefits of our paired training objective using a textual-NMN~\citep{GuptaNMN20} designed to answer complex compositional questions on \textsc{DROP}{}~\citep{DROPDua17}, a dataset requiring natural language and symbolic reasoning against a paragraph of text.
While there can be many ways of acquiring paired examples, we explore three directions.
First, we show how naturally occurring paired questions can be automatically found from within the dataset.
Further, since our method does not require end-task supervision for the paired example, one can also use data augmentation techniques to acquire paired questions without requiring additional annotation. We show how paired questions can be constructed using simple templates, and how a
question generation model can be used to generate paired questions.

We empirically show that our paired training objective leads to overall performance improvement of the NMN model.  While each kind of paired data acquisition leads to improved performance,
combining paired examples from all techniques leads to the best performance~(\S\ref{ssec:exp:iid}).
We quantitatively show that using this paired objective results in significant improvement in predicting the correct latent decisions~(\S\ref{ssec:exp:faithful}),
and thus demonstrate that the model's performance is improving \emph{for the right reasons}.
Finally, we show that the proposed approach leads to better \emph{compositional generalization} to out-of-distribution examples~(\S\ref{ssec:exp:compgen}).
Our results show that we achieve the stated promise of latent decision models:
an interpretable model that naturally encodes compositional reasoning and uses its modular architecture for better generalization.

\begin{figure*}[t]
\setlength\abovecaptionskip{7pt}
\setlength\belowcaptionskip{-5pt}
\centering
\includegraphics[width=\textwidth]{overview.png}
\caption{\textbf{Proposed paired objective:}
For training examples that share substructure, we propose an additional training objective relating their latent decisions; $S$ in the shaded gray area. In this figure, $g(X_{i}[m:n])$ $=$ $g(\textsc{BERT}(x_{i}, p)[m:n])$, where $\textsc{BERT}(x_{i}, p)$ is the contextualized representation of $x_{i}$-th question/passage, and $[m:n]$ is its slice for the $m$ through $n$ token. $g$ = \find{} in all cases. See \S\ref{sec:paired-nmn} for details. Here, since the outputs of the shared substructures should be the same, $S$ would encourage \emph{equality} between them.
}
\label{fig:overview}
\end{figure*}

\section{Paired Examples as Indirect Supervision for Latent Decisions}
\label{sec:paired-loss}
We focus on structured compositional models for reasoning that perform an explicit problem decomposition and predict interpretable latent decisions that are composed to predict the final output.
These intermediate outputs are often grounded in real-world phenomena and provide some explanation for the model's predictions.
Such models assume that the structured architecture provides a useful inductive bias for efficient learning.
For example, for a given input $x$, a model could perform the computation $f(g(x), h(x))$ to predict the output $y$.
In this paper, we will need to distinguish between a computation tree, and the output of its execution. Therefore, we will use the notation $z$ to denote a computation tree, and
$\denotation{z}$ to denote the output of its execution.
Hence we can write,
\begin{align}

    y & = \denotation{f(g(x), h(x))}
\end{align}
where $f$, $g$, and $h$ perform the three sub-tasks required for $x$ and
the computations $g(x)$ and $h(x)$ are the intermediate decisions. The actual computation tree would be dependent on the input and the structure of the model.
For example, to answer \question{How many field goals were scored?}, a NMN would perform $f(g(x))$ where $g(x)$ would output the set of \emph{field goals} and $f$ would return the size of this set.
While we focus on NMNs, other models that have similar structures where our techniques would be applicable include language models with latent variables for coreference~\citep{Ji2017DynamicER}, syntax trees~\citep{Dyer2016RecurrentNN}, or knowledge graphs~\citep{KGLM2019Robert}; checklist-style models that manage coverage over parts of the input~\citep{Kiddon2016GloballyCT}; or any neural model that has some interpretable intermediate decision, including standard attention mechanisms~\citep{Bahdanau2015NeuralMT}.

Typically, the only supervision provided to the model are gold $(x, y^{*})$ pairs, without the outputs of the intermediate decisions ($\denotation{g(x)}$ and $\denotation{h(x)}$ above), from which it is expected to jointly learn the parameters of all of its components.
Such weak supervision is not enough for accurate learning, and the fact that incorrect latent decisions can lead to the correct prediction further complicates learning.
Consequently, models fail to learn to perform these latent tasks correctly and usually end up modeling irrelevant correlations in the data~\citep{Johnson2007WhyDE, subramanian-etal-2020-obtaining}.

In this work, we propose a method to leverage \emph{paired examples}---examples whose one or more latent decisions are related to each other---to provide an indirect supervision to these latent decisions. Consider paired training examples $x_i$ and $x_j$ with the following computation trees:
\begin{align}
    z_i & = f(g(x_i), h(x_i)) \\
    z_j & = f(k(g(x_j)))

\end{align}
These trees share the internal substructure $g(x)$. In such a scenario, we propose an additional training objective
$S(\denotation{g(x_i)}, \denotation{g(x_j)})$
to enforce consistency of partial model execution for the shared substructure:
\begin{equation}
    \mathcal{L}_{\text{paired}} = S(\denotation{g(x_i)}, \denotation{g(x_j)})
\label{eq:paired-obj}
\end{equation}
For example, the two questions on the LHS of Figure~\ref{fig:overview} share the intermediate decision of finding the field goals. i.e., their computation trees share the substructure $g(x)$ = \find{}\modarg{field goal}.
In such a case, where the outputs of the intermediate decision should be the same for the paired examples, using a similarity measure for $S$ would enforce equality of the latent outputs $\denotation{g(x)}$.
We will go into the specifics of this example in Section~\ref{sec:paired-nmn}.
By adding this consistency objective, we are able to provide an additional training signal to the latent decision using related examples, and hence indirectly share supervision among multiple training examples.
As a result, we are able to more densely characterize the decision boundary around an instance ($x_i$), by using related instances ($x_j$), than what was possible by using the original instance alone.

To use this consistency objective for $x_i$, we do not require supervision for the latent output $\denotation{g(x_i)}$, nor the gold end-task output $y_j^{*}$ for the paired example $x_j$; we only enforce that the intermediate decisions are consistent.
Additionally, we are not limited to enforcing consistency for a single intermediate decision from a single paired example; if $x_i$ shares an additional substructure $h(x)$ with a paired example $x_k$, we can add an additional term
$S'(\denotation{h(x_i)}, \denotation{h(x_k)})$
to Eq.~\ref{eq:paired-obj}.

Our approach generalizes a few previous methods for learning via paired examples.
For learning to ground tokens to image regions,~\citet{contrastive-gupta2020} enforce contrastive grounding between the original and a negative token; this is equivalent to using an appropriate $S$ in our framework.
A few approaches~\citep{Minervini2018AdversariallyRN, Li2019ALF, LogicGuidedDA-Asai2020} use an additional objective on model outputs to enforce domain-specific consistency between paired examples; this is a special case of our framework where $S$ is used on the outputs $(y_i, y_j)$, instead of the latent decisions.

Using paired examples for indirect supervision on latent decisions should be broadly applicable to a wide class of models, and our general formulation of this technique is, we believe, novel.  However, the specific application of this method to any particular problem is non-trivial, as work needs to be done to acquire paired data and design a suitable $S$ for the model being studied.  In the rest of this work, we present a case study on text-based neural module networks which we believe is promising enough to motivate further applications of this method in future work.

\section{Training via Paired Examples in Neural Module Networks}
\label{sec:paired-nmn}
We apply our approach to improve question answering using a neural module network~\citep[NMN;][]{NMN16} on the \textsc{DROP}{} dataset~\citep{DROPDua17}. \textsc{DROP}{} contains complex compositional questions against natural language passages describing football games and historical events.

NMN is a model architecture aimed at reasoning about natural language against a given context (text, image, etc.) in a compositional manner.
A NMN maps the input utterance into an executable program representing the compositional reasoning structure required to predict the output.
The program is composed of learnable modules that are designed to perform atomic reasoning tasks.
For example, to answer $q =$ \question{How many field goals were scored?}, a NMN would parse it into a program $z =$ \countmod{}(\find{}\modarg{field~goals}).  This program then gets executed by the learnable models to produce $y$, essentially performing $y = \denotation{f(g(q))}$, where $f=\countmod{}$ and $g=\find{}$.

Given a question $q$, the gold program $z^{*}$, and the correct answer $a^{*}$, maximum likelihood training is used to jointly train the parameters of the modules, as well as a parser that produces the gold program $z^{*}$.
Note that the gold program only supervises the \emph{layout} of the modules ($z$ in the example in above) and not the outputs of the intermediate modules.  That is, we have $a^{*} = \denotation{z^{*}}$, but no supervision on subparts of $z^{*}$, such as $\denotation{g(q)}$.
It is extremely challenging to learn the module parameters correctly in the absence of intermediate module output supervision.
Learning is further complicated by the fact that the space of possible intermediate outputs is quite large and incorrect module output prediction can still lead to the correct answer.
For example, the \find{} module in the question above needs to learn to select the spans describing \emph{field goals} among all possible spans in the passage using just the \emph{count value} as answer supervision.
With no direct supervision for the module outputs, the modules can learn incorrect behavior but still predict the correct answer, effectively memorizing the training data. Such a model would presumably fail to generalize.

\paragraph{Text-NMN}
We work with the Text-NMN of \citet{GuptaNMN20} on a subset of DROP which is annotated with gold programs.
Their model contains \find{}, \filter{}, \project{}, \countmod{}, \findnum{}, \finddate{}, \findmaxnum{}, \findminnum{}, \module{num-compare}, \module{date-compare}, \numadd{}, \numdiff{}, \timediff{}, and \spans{} modules.
The \find{}, \filter{}, and \project{} modules take as input an additional question string argument.
Each module's output is an attention distribution over the relevant support. E.g. \find{}, \filter{}, \project{} output an attention over passage tokens, \findnum{}, \numadd{} over numbers, \finddate{} over dates, etc.

Given a question $q$ and passage $p$, BERT is used to compute joint contextualized representations for the (question, passage) combination, $\textsc{BERT}(q, p) \in \mathbb{R}^{(|q|+|p|) \times d}$.
During execution, the modules that take a question span argument as input (e.g. \find{}) operate on the corresponding slice of this contextualized representation. For example, in $q$ = \question{How many field goals were scored?} with program $z$ = \countmod{}(\find{}\modarg{field goals}), to execute \find{}\modarg{field goals}, the model actually executes \find{}($\textsc{BERT}(q, p)[2:3]$).\footnote{All modules also take the (BERT-encoded) passage $p$ as an implicit argument, as well as additional state extracted from the passage such as which tokens are numbers, which we omit throughout the paper for notational simplicity.} Here the slice $[2:3]$ corresponds to the contextualized representations for the 2nd through the 3rd token (\question{field goals}) of the question.
Refer \citet{GuptaNMN20} for details.

\paragraph{Paired training in NMNs}
We consider a pair of questions whose program trees $z$ share a subtree as paired examples.  A shared subtree implies that a part of the reasoning required to answer the questions is the same.
Since some modules take as input a string argument, we define two subtrees to be equivalent \emph{iff} their structure matches and the string arguments to the modules that require them are \emph{semantically equivalent}.
For example, subtrees \findnum{}(\find{}\modarg{passing touchdowns}) and \findnum{}(\find{}\modarg{touchdown passes}) are equivalent, while they are not the same as \findnum{}(\find{}\modarg{touchdown runs}) (we describe how we detect semantic equivalence in \S\ref{sec:paired-data}).

Consider a question $q_i$ that shares the substructure $g(q)$ with a paired question $q_j$.
Since shared substructures are common program subtrees in our case, we encourage the latent outputs, the outputs $\denotation{g(q)}$ of the subtree, to be equal to enforce consistency.
As the outputs of modules are probability distributions, enforcing consistency amounts to minimizing the KL-divergence between the two outputs.
We therefore maximize the following paired objective from Eq.~\ref{eq:paired-obj},
\begin{multline}

    \mathcal{L}_{\text{paired}} = - \big(\text{KL}\big[\denotation{g(q_i)}\;\|\;\denotation{g(q_j)}\big] + \\
    \text{KL}\big[\denotation{g(q_j)}\;\|\;\denotation{g(q_i)}\big]\big)
\label{eq:paired-nmn}
\end{multline}
where $S(p_1, p_2) = - (\text{KL}[p_1\|p_2] + \text{KL}[p_2\|p_1])$ is the negative symmetric KL-divergence.

To understand why such an objective is helpful even though the paired examples share exact subtrees, consider the paired examples on the LHS of Figure~\ref{fig:overview}. The substructure $g(q)$ = \find{}\modarg{field goal} is shared between them.
Even though the input string argument to \find{} is the same, what gets executed is $g(q_i)$ = \find{}($\textsc{BERT}(x_1, p)[2:3]$) and $g(q_j)$ = \find{}($\textsc{BERT}(x_2, p)[7:8]$), i.e. \find{} gets as input different contextualized representations of \question{field goal} from the two questions.
Due to different inputs, the output of \find{} could be different, which would lead to inconsistent behavior and inefficient learning.
Our paired objective (Eq.~\ref{eq:paired-nmn}) would encourage that these two outputs are consistent, thereby allowing sharing of supervision across examples.

\begin{figure*}[t]
\setlength\abovecaptionskip{7pt}
\setlength\belowcaptionskip{-5pt}
\centering
\includegraphics[width=0.95\textwidth]{constructed-examples.pdf}
\caption{\textbf{Templated Construction of Paired Examples}:
Constructed paired examples can help in indirectly enforcing consistency between different training examples~(\S\ref{sssec:construct-paired}).}
\label{fig:construct-paired}
\end{figure*}

\paragraph{Complete Example}
We describe the benefits of training with paired data using an example.
Consider the four questions in the periphery of Figure~\ref{fig:construct-paired}; all of them share the substructure of finding the field goal scoring events.
However, we find that for the questions requiring the \module{find-\{max/min\}-num} operation, a vanilla NMN directly grounds to the longest/shortest field goal as the \find{} execution.
Due to the use of powerful NNs (i.e., BERT) for contextualized question/passage representations and no constraints on the modules to perform as intended, the model performs the symbolic \emph{min}/\emph{max} operation internally in its parameters.
Such \find{} execution results in non-interpretable behavior, and substantially hurts generalization to the \countmod{} questions.
By enforcing consistency between all the \find{} executions, the model can no longer shortcut the compositional reasoning defined by the programs; this results in correct \find{} outputs and better generalization, as we show in \S\ref{ssec:exp:analysis}.

Note that in this example we do not know the correct answer $a^{*}$ for the constructed question \question{What were the field goals?}, nor do we know the intermediate output $\denotation{\find{}\modarg{field goals}}$.  The \emph{only} additional supervision given to the model is that there is a pairing between substructures in all of these examples, and so the model should be consistent.

\section{Many Ways of Getting Paired Data}
\label{sec:paired-data}
We explore three ways of acquiring paired questions.
We show how questions that share substructures can be automatically found from within the dataset (\S\ref{ssec:found-data}), and how new paired questions can be constructed using templates (\S\ref{sssec:construct-paired}), or generated using a question-generation model (\S\ref{sssec:generate-paired}).

\subsection{Finding Naturally Occurring Paired Data}
\label{ssec:found-data}
Any dataset that contains multiple questions against the same context could have questions that query different aspects of the same underlying event or entity.
These examples can potentially be paired by finding the elements in common between them.  As the \textsc{DROP}{} data that we are using has annotated programs, this process is simplified somewhat in that we can simply find pairs of programs in the training data that share a subtree.
While the subtrees could be of arbitrary size, we limit ourselves to programs that share a leaf \find{} module. Recall that \find{} requires a question string argument, so the challenge of finding paired questions reduces to discovering pairs of \find{} modules in different questions about the same paragraph whose question string arguments are semantically equivalent.
To this end, we use BERTScore~\citep{bert-score} to measure string similarity.

We consider two string arguments to be semantically equivalent if their BERTScore-F1 exceeds a threshold ($0.6$), and if the same entities are mentioned in the arguments.
This additional constraint allows us to judge that \emph{Jay Feely's field goal} and \emph{Janikowski's field goal} are semantically different, even though they receive a high BERTScore.
This approach would find paired examples like,\\
\question{What term is used to describe \underline{the Yorkist defeat at Ludford Bridge in 1459}?} \\
\question{What happened first: \underline{Yorkist defeat at Ludford Bridge} or widespread pillaging by Queen Margaret?}

\subsection{Paired Data via Augmentation}
\label{ssec:aug-data}
One benefit of our consistency objective (Eq.~\ref{eq:paired-obj}) is that it only requires that the paired example shares substructure.
This allows us to augment training data with new paired questions without knowing their gold answer.
We explore two ways to carry out this augmentation;
(a) constructing paired questions using templates, and (b) generating paired questions using a question-generation model.

\subsubsection{Templated Construction of Paired Examples}
\label{sssec:construct-paired}

\paragraph{Grounding \find{} event(s)}
Using the question argument from the \find{} module of certain frequently occurring programs, we construct a paired question that aims to ground the mentions of the event queried in the \find{} module.
For example, \question{Who scored the longest touchdown?} would be paired with \question{What were the touchdowns?}.
This templated paired question construction is carried out for,\\
(1) \countmod{}(\find{}\modarg{})
(2) \countmod{}(\filter\modarg{}(\find{}\modarg{})) \\
(3) \findnum{}(\findmaxnum{}(\find{}\modarg{})) \\
(4) \findnum{}(\findmaxnum{}(\filter{}\modarg{}(\find{}\modarg{})))  \\
(5) \project{}\modarg{}(\findmaxnum{}(\find{}\modarg{})) \\
(6) \project{}\modarg{}(\findmaxnum{}(\filter{}\modarg{}(\find{}\modarg{}))) \\
(7) \datecomparegt{}(\find{}\modarg{}, \find{}\modarg{}) \\
(8) \timediff{}(\find{}\modarg{}, \find{}\modarg{}),
and their versions with \findminnum{} or \datecomparelt{}.

For questions with a program in (1) - (6), we append \question{What were the} to the program's \find{} argument to construct a paired question. We annotate this paired question with the program \spans{}(\find{}\modarg{}), and enforce consistency among the \find{} modules.
Such a construction allows us to indirectly enforce consistency among multiple related questions via the constructed question; see Figure~\ref{fig:construct-paired}.

For questions with a program in (7) - (8), we append \question{When did the} to the two \find{} modules' arguments and construct two paired questions, one for each \find{} operation. We label the constructions with \finddate{}(\find{}\modarg{}) and enforce consistency among the \find{} modules.
For example, \question{How many years after the Battle of Rullion Green was the Battle of Drumclog?} would result in the construction of \question{When did the Battle of Rullion Green?} and \question{When did the Battle of Drumclog?}. While this method can lead to ungrammatical questions, it should help in decomposing the two \find{} executions.

\paragraph{Inverting Superlatives}
For questions with a program in (3) - (6) or its \findminnum{} equivalent, we construct a paired question by replacing the superlative in the question with its antonym (e.g. \emph{largest}~$\rightarrow$~\emph{smallest}) and inverting the min/max module. We enforce consistency among the \find{} modules of the original and the paired question.

\subsubsection{Model-generated Paired Examples}
\label{sssec:generate-paired}
We show how question generation (QG) models~\citep{Du2017LearningTA, Krishna2019GeneratingQH} can be used to generate paired questions.
QG models are seq2seq models that generate a question corresponding to an answer span marked in a passage as input. We follow~\citet{Wang2020AskingAA} and fine-tune a BART model~\citep{Lewis2020BARTDS} on SQuAD~\citep{Rajpurkar2016SQuAD10} to use as a QG model.

We generate paired questions for non-football passages\footnote{We explain the reason for this in \S\ref{appendix:modgen}}
in \textsc{DROP}{} by randomly choosing $10$ numbers and dates as answer spans, and generating questions for them.
We assume that the generated questions are SQuAD-like---they query an argument about an event/entity mentioned in text---and label them with the program \findnum{}(\find{}) or \finddate{}(\find{}). We use the whole question apart from the Wh-word as the string argument to \find{}.
We then follow the same procedure as \S\ref{ssec:found-data}---for each of the \find{} module in a \textsc{DROP}{} question's program, we see if a generated question with a \emph{semantically similar} \find{} module exists. If such an augmented question is found, it is used as a paired example for the \textsc{DROP}{} question to enforce consistency between the \find{} modules. For example,
\question{How many \underline{percentage points} did the \underline{population of non-Hispanic Whites} drop from 1990 to \underline{2010}?} is paired with the generated question \question{What percentage of the population was non-Hispanic Whites in 2010?}.

\section{Experiments}
\label{sec:exp}
\paragraph{Dataset and Setup}
We perform experiments on the subset of the DROP dataset~\citep{DROPDua17} that is covered by the modules in Text-NMN. This subset is a union of the data used by \citet{GuptaNMN20} and the question decomposition annotations in the \textsc{Break}{} dataset~\citep{break-wolfson2020}. All questions in our dataset contain program annotations (heuristically annotated by \citet{GuptaNMN20}; crowd-sourced in \textsc{Break}{}). The program annotations only supervise the layouts of the modules, and not the intermediate outputs. We only use these programs for training; all validation and test results are based on predicted programs.
Our complete subset of \textsc{DROP}{} contains 23215 question-answer pairs. For an i.i.d. split, since the \textsc{DROP}{} test set is hidden, we split the training set into train/validation and use the provided validation set as the test set. Our train / validation / test sets contain 18299 / 2460 / 2456 questions, respectively.
In the training data, we found 7018 naturally-occurring pairings for 6039 questions (\S\ref{ssec:found-data}); construct template-based paired examples for 10882 questions~(\S\ref{sssec:construct-paired}); and generate 2632 questions paired with 2079 \textsc{DROP}{} questions~(\S\ref{sssec:generate-paired}).
We use these paired examples to compute $\mathcal{L}_{\text{paired}}$, which is added as an additional training objective on top of the standard training regime for Text-NMN
(see \S\ref{appendix:exp-details} for details).
Note that we do not add any additional (question, answer) pairs to the data, only new (unlabeled) questions.

\paragraph{Baselines}
As we are studying the impact of our new paired learning objective, our main point of comparison is a Text-NMN trained without that objective.
Though the focus of our work is improving learning in structured interpretable models, we also show results from a strong, reasonably comparable black-box model for \textsc{DROP}{}, \textsc{MTMSN}{}~\citep{MTMSN19}, to better situate the relative performance of this class of models.

Experimental details are described in \S\ref{appendix:exp-details}. We release all our data and code publicly at \url{nitishgupta.github.com/nmn-drop}.

\begin{table}
\centering
\resizebox{0.5\textwidth}{!}{
\begin{tabular}{l cc cc}
\toprule
\bf \multirow{2}[3]{*}{Model} & \multicolumn{2}{c}{\bf dev} &  \multicolumn{2}{c}{\bf test} \\
\cmidrule(lr){2-3}  \cmidrule(lr){4-5}
& \bf F1  & \bf EM & \bf F1 & \bf EM \\
\midrule
\textsc{MTMSN}                      & 66.2  & 62.4 & 72.8  & 70.3   \\
\midrule
\textsc{NMN}{} Baseline                     & 62.6  & 58.0 & 70.3  & 67.0   \\
\addlinespace[1mm]
\textsc{NMN} + $\mathcal{L}_{\text{paired, found}}${}         & 66.0  & 61.5 & 71.0  & 67.8   \\
\textsc{NMN} + $\mathcal{L}_{\text{paired, temp}}${}          & 66.2  & 61.4 & 72.3  & 69.2   \\
\textsc{NMN} + $\mathcal{L}_{\text{paired, qgen}}${}          & 63.7  & 58.9 & 71.2  & 68.4   \\
\addlinespace[1mm]
\textsc{NMN} + $\mathcal{L}_{\text{paired, all}}${}      & \textbf{66.3}  & \textbf{61.6} & \textbf{73.5}  & \textbf{70.5}   \\
\bottomrule
\end{tabular}
}
\centering
    \caption{ \label{tab:mainresults} \textbf{Performance on \textsc{DROP}{} (pruned):} Using our paired objective with all different kinds of paired-data leads to improvements in \textsc{NMN}{}. Model achieves the best performance when all kinds of paired-data are used together.}
\end{table}

\begin{table*}[tbh]
\small
\centering
\captionsetup{font=normal}
\resizebox{1.0\textwidth}{!}{
\begin{tabular}{lccccccc}
\toprule
\multirow{2}[3]{*}{\textbf{Model}} &
\multirow{2}[3]{*}{\begin{tabular}{@{}c@{}}\textbf{Performance} \\ (F$_1$ Score)\end{tabular}} & \multirow{2}[3]{*}{\begin{tabular}{@{}c@{}}\textbf{Overall Faithfulness} \\ (cross-entropy$^{*}$ $\downarrow$) \end{tabular}} &
\multicolumn{5}{c}{Module-wise Faithfulness$^{*}$ ($\downarrow$)} \\
\cmidrule(lr){4-8}
&    &  & \module{find}  & \module{filter} & \module{num-date}$^\dagger$ & \module{project} & \module{min-max}$^\dagger$ \\
\midrule

\textsc{NMN}{}                                  & 70.3  & 46.3      & 14.3  & 21.0   & 30.6  & 0.9   & 1.4       \\
\textsc{NMN}{} + $\mathcal{L}_{\text{paired, all}}${}                     & 73.5  & 13.0      & 4.4   & 5.7    & 8.3   & 1.4   & 1.2       \\
\bottomrule
\end{tabular}
}
\caption{
\label{tab:faithful-results}
    \textbf{Faithfulness scores}: Using the paired objective significantly improves intermediate output predictions.
    {\footnotesize $^\dagger$denotes the average of \findnum{} \& \finddate{} and \findminnum{} \& \findmaxnum{}.}
}
\end{table*}

\subsection{In-distribution Performance}
\label{ssec:exp:iid}
We first evaluate the impact of our proposed paired objective on in-distribution generalization.
Table~\ref{tab:mainresults} shows the performance of the NMNs, trained with and without the paired objective, using different types of paired examples. We see that the paired objective always leads to improved performance; test F1 improves from $70.3$ F1 for the vanilla NMN to (a) $71$ F1 using naturally-occurring paired examples ($\mathcal{L}_{\text{paired, found}}${}), (b) $72.3$ F1 using template-based paired examples ($\mathcal{L}_{\text{paired, temp}}${}), and (c) $71.2$ F1 using model-generated paired examples ($\mathcal{L}_{\text{paired, qgen}}${}).
Further, the model achieves the best performance when all kinds of paired examples are combined, improving the performance to $73.5$ F1 ($\mathcal{L}_{\text{paired, all}}${}).\footnote{The improvement over the baseline is statistically significant (p = $0.01$) based on the Student's t-test.
Test numbers are much higher than dev since the test set contains 5 answer annotations for each question.}
Our final model also outperforms the black-box \textsc{MTMSN}{} model.

\subsection{Measuring faithfulness of NMN Execution}
\label{ssec:exp:faithful}
As observed by~\citet{subramanian-etal-2020-obtaining}, training a NMN only using the end-task supervision can lead to learned modules whose behaviour is \emph{unfaithful} to their intended reasoning operation, even when trained and evaluated with gold programs.
That is, even though the NMN might produce the correct final output, the outputs of the modules are not as expected according to the program (e.g., outputting only the longest field goal for the \find{}\modarg{field goal} execution), and this leads to markedly worse generalization on DROP.
They release annotations for \textsc{DROP}{} containing the correct spans that should be output by each module in a program, and propose a cross-entropy-based metric to quantify the divergence between the output distribution over passage tokens and the annotated spans.
A lower value of this metric denotes better faithfulness of the produced outputs.

We evaluate whether the use of our paired objective to indirectly supervise latent decisions (module outputs) in a NMN indeed leads to more faithful execution.
In Table~\ref{tab:faithful-results} we see that the NMN trained with the proposed paired objective greatly improves the overall faithfulness ($46.3 \rightarrow 13.0$) and also leads to huge improvements in most modules.
This faithfulness evaluation shows that enforcing consistency between shared substructures provides the model with a dense enough training signal to learn correct module execution.
That is, not only does the model performance improve by using the paired objective, this faithfulness evaluation shows that the model's performance is improving \emph{for the right reasons}. In \S\ref{ssec:exp:analysis} we explore how this faithfulness is actually achieved.

\begin{table*}[tb]
\small
\centering
\begin{tabular}{l ccc ccc}
\toprule
\bf \multirow{2}[3]{*}{Model} &
\multicolumn{3}{c}{\bf Complex Arithmetic} &
\multicolumn{3}{c}{\bf Filter-ArgMax} \\
\cmidrule(lr){2-4}  \cmidrule(lr){5-7}
& {\bf dev}  & {\bf test w/o G.P.} & {\bf test w/ G.P.} & {\bf dev}  & {\bf test w/o G.P.} & {\bf test w/ G.P.} \\
\midrule
\textsc{MTMSN}              & 67.3  & \multicolumn{2}{c}{44.1}      & 67.5  &  \multicolumn{2}{c}{59.3}      \\
\midrule
\textsc{NMN}{}                      & 64.3  & 29.5  & 42.1                  & 65.0  &  55.6  & 59.7      \\
\textsc{NMN}{} + $\mathcal{L}_{\text{paired, all}}${}         & 67.2  & 47.2  & \textbf{54.7}         & 65.5  &  62.3  & \textbf{71.5}      \\
\bottomrule
\end{tabular}
\caption{ \textbf{Measuring compositional-generalization}: NMN performs substantially better when trained with the paired objective and performs even better when gold-programs are used for evaluation (w/ G.P).}
\label{tab:comp-split}
\end{table*}

\subsection{Evaluating Compositional Generalization}
\label{ssec:exp:compgen}
A natural expectation from structured models is that the explicit structure should help the model learn \textit{reusable} operations that generalize to novel contexts.
We test this capability using the \emph{compositional generalization} setup of ~\citet{finegan-dollak-etal-2018-improving}, where the model is tested on questions whose program templates are unseen during training.
In our case, this tests whether module executions generalize to new contexts in a program.

We create two test sets to measure our model's capability to generalize to such out-of-distribution examples.
In both settings, we identify certain program templates to keep in a held-out test set, and use the remaining questions for training and validation purposes.

\paragraph{Complex Arithmetic}
This test set contains questions that require addition and subtraction operations in complex contexts: questions whose program contains \numadd{}/\numdiff{} as the root node, but the program is \emph{not} the simple addition or subtraction template \numadd{}/\numdiff{}(\findnum{}(\find{}), \findnum{}(\find{})).
For example, \question{How many more mg/L is the highest amount of arsenic in drinking water linked to skin cancer risk than the lowest mg/L amount?}, with program
\numdiff{}(\findnum{}(\findmaxnum{}(\find{})), \findnum{}(\findminnum{}(\find{}))).

\paragraph{Filter-Argmax}
This test set contains questions that require an argmax operation after filter: programs that contain the subtree
\findmaxnum{}/\findminnum{}(\filter{}($\cdot$)).
For example, \question{Who scored the shortest touchdown in the first half?}, with program \project{}(\findmaxnum{}(\filter{}(\find{}))).

\paragraph{Performance}
In Table~\ref{tab:comp-split} we see that a NMN using our paired objective outperforms both the vanilla NMN and the black-box \textsc{MTMSN}{} on both test sets.\footnote{The test set size is quite small, so while the w/ G.P. results are significantly better than \textsc{MTMSN}{} (p~=~$0.05$), we can't completely rule out noise as the cause for w/o G.P. outperforming \textsc{MTMSN}{} (p = $0.5$), based on the Student's t-test.}
This shows that enforcing consistent module behavior also improves their performance in novel contexts and as a result allows the model to generalize to out-of-distribution examples.
We see a further dramatic improvement when the model is evaluated using gold programs.  This is not surprising since it is known that semantic parsers (including the one in our model) often fail to generalize compositionally~\citep{finegan-dollak-etal-2018-improving, Lake2018GeneralizationWS, Bahdanau2019CLOSUREAS}.  Recent advancements in semantic parsing models that aim at compositional generalization should help improve overall model performance~\citep{Lake2019CompositionalGT, Korrel2019TranscodingCU, Herzig2020SpanbasedSP}.

\setlength\intextsep{1pt}
\begin{table}
\small
\centering
\resizebox{0.5\textwidth}{!}{
\begin{tabular}{l @{\hskip 4pt}c@{\hskip 4pt}c@{\hskip 4pt}c@{\hskip 4pt}c}
\toprule
\multirow{2}[2]{*}{\begin{tabular}{@{}l@{}}\textbf{Model} \end{tabular}} &
\multicolumn{3}{c}{\bf Test F1} &
\multirow{2}[2]{*}{\begin{tabular}{@{}c@{}} {\bf Faithful.-}\\{\bf score ($\downarrow$)} \end{tabular}} \\
\cmidrule(lr){2-4}
&
Overall &
Min-Max &
Count &
 \\
\midrule
\textsc{NMN}{}                                      & 57.4  & 82.1  & 36.2      & 110.4 \\
\, + $\mathcal{L}_{\text{max+min}}$         & 60.9  & \textbf{85.5}  & 39.7      & 56.5  \\
\, + $\mathcal{L}_{\text{max+count}}$       & 60.8  & 81.4  & 43.0      & 99.2  \\
\, + $\mathcal{L}_{\text{max+min+count}}$   & \textbf{71.1}  & 85.4  & \textbf{58.8}      & \textbf{25.9}  \\
\bottomrule
\end{tabular}
}
\caption{
\small
Using constructed paired examples for all three types of questions---min, max, and count---leads to dramatically better count performance. Without all three, the model finds shortcuts to satisfy the consistency constraint and does not learn correct module execution.
}
\label{tab:count-analysis}
\end{table}

\subsection{Analysis}
\label{ssec:exp:analysis}
We perform an analysis to understand how augmented paired examples---ones that do not contain end-task supervision---help in improving latent decision predictions. We conduct an experiment on a subset of the data containing only min, max and count type questions; programs in (1)-(6) from \S\ref{sssec:construct-paired}.
We see a dramatic improvement over the baseline in count-type performance when paired examples for all three types of questions are used; answer-F1 improves from $36.2 \rightarrow 58.8$, and faithfulness from $110.4 \rightarrow 25.9$. This verifies that without additional supervision the model does indeed perform the min/max operation internal to its parameters and ground to the output event instead of performing the correct \find{} operation~(\S\ref{sec:paired-nmn}). As a result, the \find{} computation that \emph{should} be shared with the count questions is not actually shared, hurting performance.
By indirectly constraining the \find{} execution to produce consistent outputs for all three types of questions via the constructed question~(Fig.~\ref{fig:construct-paired}), the model learns to correctly execute \find{}, resulting in much better count performance.
Using paired examples only for max and count questions ($\mathcal{L}_{\text{max+count}}$) does not constrain the \find{} operation sufficiently---the model has freedom to optimize the paired objective by learning to incorrectly ground to the max-event mention for both the original and constructed question's \find{} operation.
This analysis reveals that augmented paired examples are most useful when they form enough indirect connections between different types of instances to densely characterize the decision boundary around the latent decisions.

\section{Related Work}
The challenge in learning models for complex problems can be viewed as the emergence of artificially simple decision boundaries due to data sparsity and the presence of spurious dataset biases~\citep{Gardner2020EvaluatingNM}.
To counter data sparsity, data augmentation techniques have been proposed to provide a compositional inductive bias to the model~\citep{Chen2020NeuralSR, Andreas2020GoodEnoughCD} or induce consistent outputs~\citep{LogicGuidedDA-Asai2020, ribeiro2019red}.
However, their applicability is limited to problems where the end-task supervision ($y$) for the augmented examples can be easily inferred.
To counter dataset biases, model-based data pruning~\citep[AFLite;][]{Bras2020AdversarialFO} and subsampling~\citep{Oren2020-semparse} have been proposed. All the techniques above modify the training-data distribution to remove a model's propensity to find artificially simple decision boundaries, whereas we modify the training objective to try to accomplish the same goal.
Ensemble-based training methodology~\citep{Clark2019DontTT, Stacey2020ThereIS} has been proposed to learn models robust to dataset artifacts; however, they require prior knowledge about the kind of artifacts present in the data.

Our approach, in spirit, is related to a large body of work on learning structured latent variable models.
For example, prior work has incorporated indirect supervision via constraints~\citep{Graa2007ExpectationMA, chang2007guiding, ganchev2010posterior} or used negative examples with implausible latent structures~\citep{ContrastiveET-Smith2005, Chang2010StructuredOL}.
These approaches use auxiliary objectives on a single training instance or global conditions on posterior distributions, whereas our training objective uses \emph{paired examples}.

\section{Conclusion}
We propose a method to leverage \emph{paired examples}---instances that share internal substructure---to provide a richer training signal to latent decisions in compositional model architectures.
We provide a general formulation of this technique which should be applicable to a broad range of models. To validate this technique, we present a case study on text-based neural module networks, showing how to apply the general formulation to a specific task.
We explore three methods to acquire paired examples and empirically show that our approach leads to substantially better in- and out-of-distribution generalization of a neural module network in complex compositional question answering.  We also show that using our paired objective leads to improved prediction of latent decisions.

A lot of recent work is exploring the use of closely related instances for improved evaluation and training.  Ours is one of the first works to show substantial improvements by modifying the training objective to try to make better use of the local decision surface.  These results should encourage more work exploring this direction.

redpajama_set_name: RedPajamaArXiv


Our talented and dedicated staff have an average industry tenure of 25 years. We arrange ocean and air shipments from our Baltimore HQ from any overseas origin to any destination in the United States. We provide reliable and creative solutions for our clients and overseas partners.
We have ocean service contracts with all major carriers covering all trade lanes. We can provide door to door turnkey service by air and sea to any destination worldwide. We have a strong global network of agents that are country specific providing any service needed. We are also an NVOCC.
Our project cargo expertise is the niche we thrive on. We belong to 2 Global Project Cargo Networks that give us access to experts around the world. We can deliver heavy, long, wide and tall machinery and equipment anywhere in the USA and around the world.
Thunderbolt Global Logistics is an international freight forwarder, customs broker and NVOCC based in Baltimore, Maryland.
We are a C-TPAT certified company. We have an experienced staff that can navigate the complex world of international and domestic transportation. We provide import and export clearance services.
We take great pride in our ability to negotiate letters of credit for our export clients. We review all letters of credit before a shipment leaves your facility. The bottom line is that we get our export clients paid.
We know all the right people and possess all the right certifications to get your shipment from anywhere to any there.
We've been there. We've done that. And we can do it for you.
We have contracts with most major carriers and consolidators which enables us to provide our clients comprehensive coverage and choices.
We're fully equipped to handle any international shipment, whether a single pallet or an out of gauge piece, including the ability to charter an aircraft when necessary.
We understand and know how to transport freight whether it's needing a 19 axle truck or a simple LCL shipment.
From containers to break-bulk, we know how to transport your cargo from point A to point B. We also can provide inter-modal service.


text:
Sign In or Use Email
Student at Rochester Institute of Technology majoring in applied mathematics
Norwich High School
John in the News
John Mooney earns 2016 Liberty League Fall All-Academic Team honors
John Mooney of the RIT men's cross country team earned 2016 Liberty League Fall All-Academic Team honors. Liberty League commissioner Tracy King announced that 573 fall sport student-athletes have...
November, 04 2016 - Verified by Rochester Institute of Technology
John Mooney joins RIT's men's cross country team
John Mooney of Norwich is a member of RIT's men's cross country team. Rochester Institute of Technology offers 24 intercollegiate varsity sports, including 22 at the Division III level and Divisi...
September, 21 2016 - Verified by Rochester Institute of Technology
Spring 2016 Dean's List
Congratulations on making the Dean's List! Degree-seeking undergraduate students are eligible for Dean's List if their term GPA is greater than or equal to 3.400; they do not have any grades of "In...
July, 18 2016 - Verified by Rochester Institute of Technology
Fall 2015 Dean's List
February, 24 2016 - Verified by Rochester Institute of Technology
Spring 2014-2015 Dean's List
More than 4,000 RIT students made the Dean's List for spring semester 2014-2015. Degree-seeking undergraduate students are eligible for Dean's List if their term GPA is greater than or equal to 3.4...
John Mooney makes Dean's List at RIT
John Mooney, a fourth-year student in the applied mathematics program in RIT's College of Science, made the Dean's List for the fall 2014-2015 semester. Degree-seeking undergraduate students are ...
John Mooney Makes Dean's List at #RIT
John Mooney, who is in the applied mathematics program in RIT's College of Science, made the Dean's List for the Spring 2013-2014 semester. Degree-seeking undergraduate students are eligible for ...
June, 20 2014 - Verified by Rochester Institute of Technology
John Mooney of Norwich made the Dean's List for the Fall 2013 semester. Degree-seeking undergraduate students are eligible for Dean's List if their quarterly GPA is greater than or equal to 3.400;...
John Mooney of Norwich, N.Y., a second-year student in the applied mathematics program in RIT's College of Science, made the Dean's List for the Spring 2013 quarter. Degree-seeking undergraduate ...
John Mooney of Norwich, N.Y., a second-year student in RIT's College of Science, made the Dean's List for the Winter 2013 quarter. Rochester Institute of Technology is internationally recognized...
March, 26 2013 - Verified by Rochester Institute of Technology
John Mooney of Norwich, N.Y., a first-year student in RIT's College of Science, made the Dean's List for the Fall 2012 quarter. Rochester Institute of Technology is internationally recognized fo...
December, 18 2012 - Verified by Rochester Institute of Technology
John Mooney to Attend RIT
John Mooney, son of Grace and Michael Mooney of Norwich, N.Y., has enrolled for fall 2012 admission at Rochester Institute of Technology. John will study science exploration in RIT's College...
August, 03 2012 - Verified by Rochester Institute of Technology
John was awarded this badge for 9 achievements. Click on the stories below to view them.
© Copyright 2020 • Merit Pages, Inc. • Terms of Service • Privacy Policy


Social news and bookmarking sites, such as reddit.com, digg.com, slashdot.org and stumbleupon.com, use voting by users or selection by editors to rank interesting stories. Much to my surprise, I recently had an article from this blog featured prominently on all four of these popular sites. This generated a large amount of traffic and gave me an interesting opportunity to turn the tables, by using my hit statistics to rank these sites.
On the 16th August I published an article about a little experiment I did to prove that many software download sites hand out awards automatically, without reviewing the software. Most developers who have submitted software to such sites probably suspected this already. But the experiment proved it conclusively by garnering awards for software that didn't even run.
I wrote the article because I wanted to shine some light on this unsavoury practise. I wanted it to be as widely read as possible, so I posted a link to it on a few software developer and entrepreneur forums that I frequent. Later in the day I posted it to reddit.com. I also added my vote to the people who had already posted it to digg.com and programming.reddit.com. I expected a few hundred people would read the article, mostly regular readers of my blog. But it got voted up and made its way on to the home page of reddit.com. Traffic started to flood in.
My recollections of the next few days are a bit hazy as it all happened rather quickly. From the front page of reddit.com the article made its way across the front pages of digg.com, and then slashdot.org, like a electronic Mexican wave. The article also appeared on the home page of WordPress.com and received traffic from social bookmarking sites stumbleupon.com and del.icio.us. Large numbers of blogs and forums also linked to the article. Hits on my blog peaked on the 17th at 53,422 hits for the day.
The social news sites have the attention span of a one year old on amphetamines. The hits from digg.com went from 15,161 on Friday to just 648 on Saturday.
The article was linked to from 375 blogs (according to technorati.com) and an unknown number of forums and other sites. The top 4, 10 and 20 sites account for 52%, 61% and 65% of the total traffic, respectively. A long tail of less popular sites makes up the rest.
Things really took off once the article reached the front page of reddit.com. I visualise the links spreading across the Internet analogous to a sub-atomic chain reaction. Just as energetic particles decompose into cascades of ever smaller particles, bigger sites propogate their links to ever larger numbers of smaller and smaller sites.
Totalling all the visitors to the blog over the 5 days I give you the successfulsoftware.net official ranking for social news and bookmarking sites.
The article has generated a lot of comments. I particularly enjoyed the reviews here (I hope they haven't been deleted). Interestingly the order of the number of comments/reviews for the 4 top sites is very different to the number of hits.
The WordPress stats only show the top 40 referrers for each day.
The article made the front page of different sites at different times.
Just because someone clicked through, doesn't mean they actually read the article.
My article might simply have been more interesting to the type of people who read one site than the type of people who read another.
I have no way of knowing whether any of the visitors were bots.
But social news sites aren't exactly rigorous in their ranking either.
Please note that I created this blog to write about what it takes to successfully create and market commercial software. I don't intend to become another blogger blogging about blogging. It's bad for your eyesight (see point #10 here). Normal service will be resumed shortly.
 To the best of my knowledge the article reached a highwater mark of positions 1,2 and 2 on slashdot.org, reddit.com and digg.com respectively and was featured in one of the 'popular' pages on stumbleupon.
This entry was posted in article, blogging on 21 August 2007 by Andy Brice.
That's really useful stats, thanks for publishing the details. Personally I've found Stumbleupon traffic to be the least targeted in the past in regards to the number of customer inquiries and conversions.
Also, have you noticed if your RSS subscriptions have gone up since? I find that whenever I write an article that gets more attention, my feed subscriptions go up by a bit. It's never close to the immediate web traffic (I don't think I've ever hit that scale for a single day), but the traffic bursts generally have a tendency to pick more RSS subscribers. I assume this happened to you but to a greater degree. If you don't know or don't wish to share, I can completely appreciate that.
>Personally I've found Stumbleupon traffic to be the least targeted in the past in regards to the number of customer inquiries and conversions.
I think you are right. I have tried advertising with stumbleupon in the past and the conversion rate was dismal.
>And by hits I assume you mean unique visitors?
No, page impressions. I contacted wordpress, but they weren't able to give me more detailed stats. I thought I noticed some Google Analytics code in the WordPress template though, so maybe it is possible to get detailed stats. It would be interesting (in a nerdy way) to see the stats hour by hour from the main sites.
That's very interesting on the page impressions. Based on the type of traffic the page impressions is probably very close to the number of unique visitors… Either way a good success!
And it's great to hear you've already had some interesting and unusual emails (assuming they're the good kind). If you do any follow-ups, to the article please don't think twice about posting it on your blog. I'm sure everyone would be very interested.
Great stuff! Very interesting to think how you stepped back and tried connecting the dots on your posts' journey. Was there a bump in subscriptions to your blog? Thank you. Cheers!
Pingback: links for 08-21-07 « A private Blog from Michael Jung.
On a more serious note, it made me realize that I never post reviews of software that I really do like. I've added "post software reviews" to my to-do list for all the unheralded tools that have made my life easier lately.
Hmm. Either the color is off on my monitor or else your chart shows "other" as number one and Stumbleupon.com as number four. This is referring to the chart titled "Hits per Day by Source." Some confusion in reading the chart could be caused by the closeness of the colors selected to represent them, but surely not by the person who put the data together for the article? I guess what I am saying is can you explain to me the difference between what your chart shows and the conclusion reached in the text. Thank you.
'other' is the long tail of other sources apart from the big 4, comprising hundreds of blogs, forums and websites.
Stumbleupon gave the greatest number of hits in total, but not the greatest number on the peak day.
I agree the colours aren't great. Blame Excel.
Thanks, Andy; that makes perfect sense now that you've explained it. By the way, I also want to compliment you on your comments I just finished reading in some of your other articles. Do any of your comments get published on dead trees? If not, they should be.
Interesting conclusions from your amazing article publicity. I've had my doubts about the quality of traffic from the top social bookmarking sites for quite a while.
I started a forum a few months back and added a lot of quality content. I then proceeded to use StumbleUpon to drive traffic to those pages. While my stats show thousands of visitors over a short period of time there have been very little sign ups to the forum itself. Even though a valuable section is out of bounds without sign up.
I'm still experimenting with sites like Digg, Reddit and Twitter. Although from your conclusions it looks like the value of the visitors are much the same.
Keep monitoring your traffic for a few weeks, you'll probably find that Stumbleupon will continue to send visitors at a fairly steady rate, whilst the other social sites tend to completely dry up.
Interesting info. You really did no promotion for these posts to get them on page1? They just got voted to the top on all those sites by themselves. wow.
Pingback: Soft Blog » What Makes a Website Great?
We get more traffic from Reddit than Digg.
The traffic is consistent and the users seem to be more mature.
urlfan crunches all the data from over a couple million rss feeds and tracks what's moving in the blogosphere. there's no human intervention so it's able to float top stories faster than the standard vote up/down digg and reddit use. according to alexa, urlfan is about to overtake reddit in terms of reach and rank so they must be doing something right.
Pingback: David.R.Gilson » Blog Archive » The difference between social news and social bookmarking. A guide.
According to compete.com, Digg is the king (although compete is not entirely accurate, the gap between Digg and the others is substantial).
Reddit is useless. They kill of my submissions. digg is better. However overall these sites are worse, coz success on them depends on the no of friends you have rather than the quality of work wwe produce.
u love ure statistics don't u? at 'em boy. good job.
thank you, very nice blog post, though with sites like digg and stumbleupon, you have to have 'fans' and 'followers' to get hits from them, with out having an established fan base, they are pretty much useless, sites like reddit, anyone can have huge hits in one day, fade away really fast, but also with multiple posts to a site like reddit, your site will fall into the spam filter and never be seen again.


Telangana Chief Minister K Chandrasekhar Rao is unlikely to campaign in the other states. Earlier KCR announced that he will campaign for Federal Front Partners in the subsequent phases of General Elections but now dropped the idea as no party clearly indicated joining Federal Front.
"If at all KCR campaigns for any party and if the party joins BJP or Congress after the election, it will be a major embarrassment and so KCR do not want to take the risk. He will keenly observe the election trends and will try to gain partners only after the results are announced," a source in TRS told us.
In a meeting with the Ministers, KCR told them that TRS is winning 16 MP seats while their friendly partner, MIM will win the remaining seat. "Jagan Reddy's YSR Congress is winning a minimum of 16 MP seats in Andhra Pradesh according to our surveys, Federal Front Count Starts at 33 Seats," KCR told his cabinet colleagues. On the other side, Chandrababu is gearing up to campaign against BJP in other states.


Shopping Carts, Accept credit cards online with a shopping cart website designed for your business.
Shopping carts, Online shopping carts, Ecommerce shopping carts. If your business is about online product selling then you should be looking forward to an online shopping carts website as an ecommerce website to accepting online payments with credit cards / debit cards and with those shopping carts it will maximise your sales by offering your customers an easy way to buy your products or services online with the shopping carts. Our shopping carts designs are user friendly you will be able to add and update your own website without having to employ a web designer or programmer and still have our backup service available so if you get stuck just give us a call and we will be there to help you for your shopping carts website.
We can create the web shopping carts system design you need without all the fuss. Below is the packages and the sections to consist for our web shopping carts, if have any queries or questions regarding the web shopping cart for your shopping carts website design.
shopping carts design. All these features are of the latest shopping cart website design system and technologies, you can even update, add and remove categories and sub categories and much more saving you time and stress and money with our shopping carts website design.
This is an approximate development time frame required for a regular shopping carts website design and may variable for custom shopping carts website design according to the specific shopping carts website requirements and also depending on the amount of workload we have at the time of your order.
If you want to accept online payments with credit cards or debit cards with shopping carts website then it needs a merchant service for shopping carts integration with the payment gateway provided by merchant services like world pay, secure trading, Paypal, HSBC, Notchex etc.
We will integrate the any merchant component into our shopping carts website.
Starting from £770 onwards according to the specific needs for the shopping carts. This would include total project cost from complete shopping carts designing & Programming. This would also include post shopping cart website design support for 3 months.
Shopping carts design minor updates, images and text contents updates, Adding new minor features to the site, modifying minor site application.


text:
The Genron NPOCreate a Strong Democracy on Strength of Debates
ISSUES/OPINIONS
Managing the G20's Relevance: Japan's Role and Priorities during its G20 Presidency March 29, 2019
Can representative democracy restore public trust? (2/2) November 26, 2018
Civil Diplomacy
G7 / G20
PROJECTS/EVENTS
Rebuilding Japan-South Korea Relations June 22, 2019
Public Opinion on Japan-South Korea Relations June 22, 2019
The 7th Japan-Korea Future Dialogue 2019 June 22, 2019
Democracy Dialogue
Council of Councils
Excellent NPO Standard
Japan-Korea Future Dialogue
Regional Multilateral Dialogue
Japan-U.S. Dialogue
New Civil Diplomacy Initiative
The Tokyo Conference
Tokyo-Beijing Forum
WORLD AGENDA STUDIO
U.S.-China trade war: How will it develop? June 28, 2019
Deepening U.S.-China confrontation and Japan's role as mediator March 28, 2019
Pressing global issues and international efforts March 25, 2019
Climate Change / SDGs
Global Economy and Trade
Bilateral relationship is 'bad,' say majority of Japanese, South Koreans June 12, 2019
The Japan-South Korea Joint Public Opinion Poll 2019 June 12, 2019
The 40th anniversary of the Treaty of Peace and Friendship and Japanese, Chinese Public Views October 24, 2018
Annual Survey Report
Japan-China
Japan-Korea
Japan-U.S.
Expert Survey
Tokyo Conference 2018 : Conference Report April 25, 2018
The Genron NPO Annual Brochure 2018 April 20, 2018
Kudo's Blog
MAIL LETTER CLOSE ×
MAIL LETTER
Enter Email Adress to sign up for The Genron NPO News Letter
SEARCH CLOSE ×
11TH TOKYO-BEIJING FORUMJapanese, Chinese delegates share ultimate goal to ensure peace and stability in Northeast Asia
⇒ Read Economy Breakout Session report "Economists and business leaders agree on need to re-energize bilateral economic relations"
⇒ Read Security Breakout Session report "Specialists exchange candid opinions on defense policies of both countries"
The 11th Tokyo-Beijing Forum was held Oct. 24-25 in Beijing with some 500 prominent Japanese and Chinese leaders, and delegates from politics, economics, academia and the media attending.
Jointly organized by The Genron NPO, a not-for-profit and independent think tank in Tokyo, and China International Publishing Group (CIPG) in China, the two-day annual forum witnessed candid exchanges of opinions and proposals from both countries on the main subject of "How to Develop Long-Term, Healthy China-Japan Relations - East Asia's Future and Roles to be Played by Both Countries."
On behalf of the Chinese and Japanese organizers, CIPG President Zhou Mingwei delivered a welcome speech at the opening of the first-day plenary session Oct. 24. His address was followed by remarks from Yasushi Akashi, chairman of the International House of Japan, who heads the forum's Japanese Executive Committee.
"However difficult China-Japan relations become, we will make our best efforts to overcome the problems. Holding this forum is just one example of how our efforts will extensively bring together different voices while expanding consensus," said Zhou.
Akashi emphasized the significance of the annual forum, which has an unbroken 11-year history, and said that deeper mutual understanding between the peoples of both countries is essential for making the bilateral relationship a continuously stable one.
Following congratulatory addresses by Zhao Qicheng, former director of the State Council Information Office and one of the founders of the forum, and Japanese Ambassador to China Masato Kitera, Jiang Jianguo, director of China's State Council Information Office, delivered a keynote speech.
Jiang hailed the forum as an important platform for people-to-people exchanges between China and Japan, and recognized the positive roles of previous rounds of the forum. "China and Japan are close neighbors. In more than 2,000 years of exchanges, peace and friendship have been the historical mainstream. The two peoples learned from each other and jointly contributed to the prosperity of East Asian civilization."
"History shows that amity between China and Japan could lead to the promotion of mutual interests while belligerence could only injure both, and the key to further development of the two countries resides with the peoples of China and Japan. They should enhance friendship so as to broaden common interests and achieve win-win results," he said.
From the Japanese side, former Prime Minister Yasuo Fukuda, the supreme advisor of the forum, took the podium and said that current China-Japan relations faced difficulties, as the latest Japan-China joint opinion poll showed that some 80 percent of people in Japan and China still hold "unfavorable" views of each other's country. "This is a very abnormal situation and I am quite concerned about that," warned Fukuda in his keynote speech.
"Japan-China relations are 'of great significance' to the peace, stability and development of Asia, and the world," said Fukuda, adding that it is the global responsibility of both countries to stabilize their bilateral relationship and build cooperative relations.
The former Japanese prime minster stressed the importance of learning from history.
"Japan failed because it was once complacent due to temporary success, which made it arrogant and selfish. It did not see the world in terms of the larger picture. This is why we have to prevent this part of history and for that purpose, we should objectively, rationally and continuously learn from history," Fukuda stressed.
Looking to the future, Fukuda predicted that the world population would amount to some 11 billion by 2100 and so many people on Earth would strive to attain an affluent life, causing devastating destruction of the environment and depletion of energy resources. "Japan and China - two of the world's major economic powers - cannot afford to waste time jostling each other. We must show the world that we are willing to collaborate to deal with the global issues facing us today," he said.
Later, the plenary session proceeded to the panel discussion on the main topic of the forum, co-moderated by Yuji Miyamoto, former Japanese ambassador to China, and Zhao Qicheng. The panelists were Hirohide Yamaguchi, chairman of the Advisory Board, Nikko Research Center Inc., and former deputy governor of the Bank of Japan, and Keiji Yamada, governor of Kyoto Prefecture, from Japan, and Lou Qinjian, governor of Shaanxi province, and Wei Jianguo, vice chairman of the China Center for International Economic Exchanges, from China.
With the keynote speech by the four panelists, the plenary session came to a close, to be followed in the afternoon by five breakout sessions - politics/diplomacy, security, economics, media/culture and a special topic on tourism/environment, the summary reports of which debaters made at the outset of the second-day and final plenary session of the forum Oct. 25.
In his closing address to the plenary session Oct. 25, CIPG Deptuty Director Wang Gangyi said that there have emerged signs of improvement in the soured national sentiments between the peoples of Japan and China, following the resumption of the summit last year. "However, no optimism is warranted, given the presence of numerous tasks that should be addressed and the absence of solid foundations for bilateral relations," he said.
Zhou stressed that he is determined to make the forum one of the most important platforms for the long-term and healthy development of bilateral relations in future.
The Genron NPO President Yasushi Kudo, in his closing remarks, said that he was so pleased to witness the candid discussions at each breakout session in which delegations from both countries attempted to find solutions to the tasks Japan and China face.
He concluded his address by noting that it is the ultimate mission of the Japanese and Chinese organizers of the forum to create a platform for bilateral dialogues, which would generate a trend toward the consolidation of Japan-China relations, and toward bilateral collaboration in dealing with global agendas jointly and responsibly.
⇒ Read Economy Breakout Session report Japanese, Chinese economists, business leaders agree on need to re-energize bilateral economic relations"
⇒ Read Security Breakout Session report "Japanese, Chinese specialists exchange candid opinions on defense policies of both countries"
The Genron NPO © 2001-. All Rights Reserved.