text:
Previous / Race of Champions: Tom Kristensen, Petter Solberg win Nations Cup Next / WEC star Davidson joins 2015 AUTOSPORT International line-up
Race of Champions: David Coulthard defeats Pascal Wehrlein in final
Dec 14, 2014, 7:28 PM
David Coulthard won the 2014 Race of Champions in Barbados, defeating Mercedes Formula 1 reserve driver Pascal Wehrlein in the final
The Scot needed only two races in the best-of-three final to take his first overall victory in the competition, his precise style around the special two-lane version of the Bushy Park track paying dividends.
In the first race of the final driving the KTM X-Bow, Coulthard beat Wehrlein by a second-and-a-half.
He followed that up with a 2.7s victory over Wehrlein in the second heat driving the Ariel Atom, with the German making a late attack but running wide at the final corner.
Coulthard had flirted with elimination in both the quarter final and semi final after dominating his first-round group.
In the quarter final, he took on Petter Solberg in the Volkswagen Polo RX and looked to have the advantage halfway through the tie.
But at the start of his second lap, he carried too much speed into the first corner, running wide over the grass and losing time.
Moments later, Solberg nosed into the barrier on the inside lane at the same turn, also losing time, eventually being defeated by just over one second.
Coulthard then beat six-time Australian V8 Supercars champion Whincup in the semi-final, but a scruffy run through the final corner meant that the 13-time grand prix winner won by a slender 0.1751s.
Saturday report: Kristensen/Solberg win Nations Cup for Team Nordic
Whincup had earlier beaten 2004 NASCAR champion Kurt Busch in the quarter final in a battle behind the wheel of identical Audi R8 LMS machines.
Knocked out in the group stages in the top half of the draw were defending champion Romain Grosjean, motorcycle grand prix legend Mick Doohan, Williams F1 tester Susie Wolff and home hero Rhett Watson.
In the bottom half of the draw, Wehrlein beat Robby Gordon in the quarter final, the American struggling with a gear-selection problem after dominating his group.
Wehrlein then comfortably won his semi-final battle driving the Buggy, beating Formula 3 European champion Esteban Ocon.
Indianapolis 500 winner Ryan Hunter-Reay also made the quarter finals in the bottom half of the draw before losing to Ocon.
Eliminated in the group stages were World Touring Car champion Jose Maria Lopez, Jamaican ace Doug Gore, Le Mans legend Tom Kristensen and GP2 Series winner Jolyon Palmer.
Race of Champions: Tom Kristensen, Petter Solberg win Nations Cup
WEC star Davidson joins 2015 AUTOSPORT International line-up
Podcast: Monters of Motorsport with Mark Blundell
24 Hours of Le Mans Virtual eRacing finale to be broadcast digitally around the world
Magazine: Celebrating motorsport's biggest monsters
TT TT
Ulster Grand Prix to return in 2022 after two-year absence
Motorsport produced one of its greatest years of all-time in 2021 despite a backdrop of ongoing COVID-19 challenges and an ever-changing racing landscape. Through the non-stop action Autosport has collected the finest moments from the past 12 months to highlight the incredible drama and joy motorsport generates
The hidden racing gem attracting ex-F1 heroes
It's rarely mentioned when it comes to assessing the best national contests, but the Brazilian Stock Car series that reaches its climax this weekend has an ever-growing appeal. Its expanding roster of ex-Formula 1 names has helped to draw in new fans, but it's the closeness of competition that keeps them watching
OPINION: The greed-driven push for a European Super League that threatened to tear football apart is collapsing at the seams. Motor racing's equivalent, the football-themed Superleague Formula series of 2008-11, was everything that the proposed ESL never could be
The F1 and Indy 'nearly man' that found contentment in Japan
By Jamie Klein
Having had the door to F1 slammed in his face and come within three laps of winning the Indianapolis 500, the collapse of a Peugeot LMP1 shot meant Japan was Bertrand Baguette's last chance of a career. But it's one which he has grasped with both hands
The female all-rounder who arrived "too early"
From Formula 3 to truck racing, Dakar and EuroNASCAR via a winning stint in the DTM, there's not much Ellen Lohr hasn't seen in a stellar racing career that highlights the merit in being a generalist. But she believes her career came too early...


This gorgeous Gucci Black Leather Babouska Heart Dome Medium Satchel Bag is super bold and beautiful. The Babouska features a Gucci heart crested logo hardware and stud details with smooth leather and comfortable rolled leather handles. This beauty has a huge interior and will hold all your daily essentials in style!
The exterior leather is clean and beautiful, but throughout the bag there is slight creasing as well as faint scuff marks. The bottom corner edges have wear. The metal feet have faded. The glazing at the base of the handles have wear, while throughout the handles there are scuff marks. The goldtone hardware has surface scratches and some fading. The interior is clean and in good condition with light wear and a few faint spots.


text:
Anote News
Kind of News That Matters
Thisdeal24
Australia Fashion Notes
Australian Food Notes
Best to buy
Perth Local News Note
Hot cars and bullet holes at Labor-linked jobs provider
Sep 13, 2020 Perth
And they show what appear to be bullet holes in the roof of a building that was supposedly leased by the company for bookkeeping purposes.
What appear to be bullet holes in the roof of the Raw Recruitment office.
Last year police mounted two raids on the East Keilor property and, in January, arrested and charged a former Raw employee, mechanic Anthony Velluto. In May he was sentenced to seven months in jail.
More trouble followed – the company is subject to a second, unrelated investigation by Victoria Police's fraud and extortion squad.
Victoria's Department of Jobs, Precincts and Regions was tipped off last year about alleged fraud in relation to its contracts and grants with Raw Recruitment, sources familiar with the situation have told The Age. The department confirmed recently that it had suspended its relationship with the company last year and in April had referred it to the fraud squad.
The sources said the key allegations were that Raw Recruitment's success in placing people in jobs was being overstated and, in some cases, jobs were given to people living interstate. Concerns have also been raised by former employees about the backdating of documents and that Raw was being paid to place Indigenous people in jobs but some of them were not Indigenous.
The company's involvement in another Victorian government program to find work for young people from African backgrounds has also come under scrutiny, with former Raw Recruitment employees saying little effort was made to get candidates into jobs.
Car parts being removed from Raw Recruitment's former East Keilor premises after a police raid.
Several construction industry sources told The Age and The Sydney Morning Herald that a decision by Raw Recruitment to move into the labour hire business a few years ago had led to a noticeable decline in the number of Indigenous workers on construction sites around Melbourne.
"It got to the point where they were taking the piss," said one source familiar with Raw Recruitment's operations. "This was meant to be about getting young Aboriginal kids into proper jobs and when you looked around it was mostly white blokes wearing the Raw jackets."
Raw Recruitment's managing director, Allan Jones, has said in a statement that a small number of employees have been stood down while investigations continue.
The federal government has also had significant dealings with the company. In 2017 Raw Recruitment received a $530,000 grant from the Prime Minister's Department, and the federal government's National Indigenous Australians Agency is now investigating the Commonwealth's financial dealings with the company.
"NIAA is aware of the Victorian government's referral of Raw Recruitment to the Victorian Police and has been conducting an internal review of funding provided to Raw Recruitment," a spokesman for the agency said.
Parts of a chopped-up Nissan Patrol.
Raw's Indigenous heritage arm in South Australia has received more than $1 million from the Defence Department for consultancy work. A spokesman for Defence said Raw had delivered on its contractual obligations and was not subject to any audit.
Members of the Andrews government and some in the public service and the union movement have given strong public backing to Raw Recruitment, which has secured more than $1.5 million in grants. The company has had significant access to ministers, and Premier Daniel Andrews and former federal Labor leader Bill Shorten are two of the high-profile ALP figures to be photographed at Raw Recruitment events.
The company was established in 2014 by Jones, a South Australian Indigenous businessman, and former construction union shop steward and local football identity Christian Gaylard. Their aim was to get Indigenous young people trained in a trade and into meaningful jobs. With strong union backing, Raw Recruitment was able to place many Indigenous workers on signature Andrews government projects such as the Metro Tunnel rail link and level crossing removals.
Premier Daniel Andrews, former Essendon great Michael Long (in the Raw jacket) and former Aboriginal affairs minister Natalie Hutchins (centre).
Raw Recruitment has retained the backing of its most powerful advocate, the influential leader of the plumbers' union Earl Setches. "Raw is referred to as 'Earl's company' within ministerial offices and the public service," said a prominent state Labor Party figure this week. "It's a very close relationship."
There is no suggestion Mr Setches has been involved in any wrongdoing.
Raw Recruitment's Melbourne head office is at the union's training centre in Brunswick and the union has an arrangement with Raw to manage its Indigenous employee relations.
The Victorian plumbers' union has a long history of backing Indigenous apprenticeships and it runs a successful program in Broome in Western Australia where young Aboriginal people are trained and placed in jobs with top-tier companies.
A Triumph motorbike being unloaded at Raw Recruitment's East Keilor premises about 2am.
Raw Recruitment has also financially supported the Victorian branch of the Labor Party-affiliated union. The union declared to the Australian Electoral Commission a $75,000 contribution before the 2018 state election.
Mr Setches declined to be interviewed but is known to be disappointed with the governance of the company.
The involvement of former Essendon AFL star Michael Long as a director of another Raw business and former world boxing champion Robbie Peden as a shareholder gave the company extra credibility and two strong mentors for Indigenous youth. There is no suggestion Mr Long or Mr Peden have engaged in or were aware of any improper activity.
The fraud investigation is not the only trouble facing Raw Recruitment and its directors. The company has also faced unfair dismissal and bullying claims from former employees, including several women. The Fair Work Commission heard the most recent case last week. Some claims have been resolved with confidential financial settlements and non-disclosure agreements signed by former female employees.
One former female employee, speaking on condition of anonymity, said senior men in the company appeared to become more aggressive as more government grants and contracts flowed in.
"The more money that came in, the tighter the jeans got and the more time at the gym. Nights out at the casino. It just became an even more macho place," she said.
Raw Recruitment said in a statement that the company's priority was to provide a safe and inclusive work environment.
"We are unable to comment on any matters relating to former employees due to privacy obligations," the statement said. "Raw is conducting a proactive and thorough audit of our contracts to help determine what has gone wrong. We are also establishing an independent compliance committee that will be governed by a compliance charter to strengthen and demonstrate our ongoing commitment to integrity and reporting."
Mr Gaylard resigned as a director of Raw Recruitment in March. He remains a shareholder.
Know more? Contact Richard Baker securely at richardjb33@protonmail.com
Our weekly newsletter will deliver expert analysis of the race to the White House from our US correspondent Matthew Knott. Sign up for The Sydney Morning Herald's newsletter here, The Age's here, Brisbane Times' here and WAtoday's here.
Richard Baker is a multi-award winning investigative reporter for The Age.
https://ift.tt/2Lg1Kov
https://ift.tt/2FsYdC2
https://ift.tt/2ALktQb
Harry Styles postpones Aussie tour
Two men charged after meth packages intercepted in Brisbane, Perth
Melbourne Local News Note

redpajama_set_name: RedPajamaCommonCrawl


text:
\section{Introduction}
\label{sec:intro}

\begin{figure*}[ht]
    \includegraphics[width=\textwidth]{figures/intro_figure4}
    \caption{Workflow of our best performing approach
    involving extraction and clustering
    of noteworthy conversation utterances
    followed by abstractive summarization
    of each cluster (fictitious data)}
    \label{fig:intro_figure}
\end{figure*}

Electronic health records (EHR)
play a crucial role in patient care.
However, populating them can
take as much time
as attending to patients
\citep{sinsky2016allocation}
and constitutes a major cause
of physician burnout
\citep{kumar2020physician}.
In particular, doctors document
patient encounters with SOAP notes,
semi-structured written accounts
containing four sections:
(S)ubjective information reported by the patient;
(O)bjective observations, e.g., lab results;
(A)ssessments made by the doctor
(typically, the diagnosis);
and a (P)lan for future care,
including diagnostic tests,
medications, and treatments.
Sections can be subdivided into
$15$ subsections.

In a parallel development,
patients increasingly record their doctor's visits,
either in lieu of taking notes or
to share with a family member.
A budding line of research
has sought to leverage transcripts
of these clinical conversations
both to provide insights to patients
and to extract structured data
to be entered into EHRs
\citep{liu2019fast, schloss2020towards, krishna2021extracting}.

In this paper, we introduce
the first end-to-end methods
for generating whole SOAP notes
based on clinical conversations.
Our work builds on a unique corpus,
developed in collaboration with
\emph{Abridge AI, Inc.}\footnote{\url{http://abridge.com}}),
that consists of thousands of transcripts
of recorded clinical conversations
together with associated SOAP notes
drafted by a work force
trained in the official style
of SOAP note documentation.
On one hand, this task is much harder
than traditional summarization benchmarks,
in part, because SOAP notes are longer
(320 words on average)
than summaries in popular datasets like
CNN/Dailymail~\citep{nallapati2016abstractive},
Newsroom~\citep{grusky2018newsroom},
and SamSum~\citep{gliwa2019samsum}
(55, 27, and 24 words on average).
On the other hand, our dataset
offers useful structure:
(i) segmentation of each SOAP note
into subsections;
and (ii) a set of \emph{supporting utterances}
that provide evidence
for each sentence in the SOAP note.
Exploiting this structure,
our methods outperform appropriate baselines.

Our first methodological contribution
is to propose a spectrum of methods,
for decomposing summarizaton tasks
into \emph{extractive}
and \emph{abstractive} subtasks.
Starting from a straightforward
sequence-to-sequence model,
our methods shift progressively more work
from the abstractive to the extractive component:
(i) {\sc Conv2Note:}
the \emph{extractive} module does nothing,
placing the full burden of summarization
on an end-to-end \emph{abstractive} module.
(ii) {\sc Ext2Note:} the extractive module selects
all utterances that are \textit{noteworthy}
(i.e., likely to be marked as supporting utterances
for \emph{at least one} SOAP note sentence),
and the decoder is conditioned only on these utterances;
(iii) {\sc Ext2Sec:}  the extractive module
extracts per-subsection \textit{noteworthy} utterances
and the decoder generates each subsection,
conditioned only on the corresponding utterances;
(iv) {\sc Cluster2Sent:} the extractive module
not only extracts per-subsection noteworthy utterances
but clusters together those likely to support
the same SOAP sentence---here,
the decoder produces a single sentence at a time,
each conditioned upon a single cluster of utterances
and a token indicating the SOAP subsection.
We see consistent benefits
as we move from approach (i) through (iv).

Both to demonstrate the generality of our methods
and to provide a reproducible benchmark,
we conduct parallel experiments
on the (publicly available)
AMI corpus~\citep{carletta2007unleashing}\footnote{Our code
and trained models for the AMI dataset:
\url{https://github.com/acmi-lab/modular-summarization}}
Like our medical conversations dataset,
the AMI corpus exhibits
section-structured summaries
and contains annotations
that link summary sentences
to corresponding supporting utterances.
Our experiments with AMI data
show the same trends,
favoring pipelines that demand
more from the extractive component.
These results speak to the wider usefulness
of our proposed approaches,
{\sc Ext2Sec} and {\sc Cluster2Sent},
whenever section-structured summaries
and annotated evidence utterances are available.

Our best performing model,
{\sc Cluster2Sent} (Figure~\ref{fig:intro_figure}),
demands the most of the extractive module,
requiring that it both select and group
each subsection's noteworthy utterances.
Interestingly, we observe that given
oracle (per-subsection) noteworthy utterances,
a simple proximity-based clustering heuristic
leads to similar performance on SOAP note generation
as we obtain when using ground-truth clusters---even
though the ground truth noteworthy utterances
are not always localized.
Applied with predicted noteworthy utterances and clusters,
this approach achieves the highest ROUGE scores
and produces the most useful
(factual, coherent, and non-repetitive) sentences
as rated by human experts.
As an additional benefit of this approach,
due to the smaller lengths
of the input and output sequences involved,
we can feasibly train large transformer-based
abstractive summarization models (e.g., T5),
whose memory requirements grow
quadratically with sequence length.
Additionally, our approach localizes the precise utterances
upon which each SOAP note sentence depends,
enabling physicians to verify
the correctness of each sentence
and potentially to improve the draft
by highlighting the correct noteworthy utterances
(versus revising the text directly).

In summary, we contribute the following:
\begin{itemize}
    \item The first pipeline for drafting entire SOAP notes
    from doctor-patient conversations.
    \item A new collection of extractive-abstractive approaches
    for generating long section-segmented summaries of conversations,
    including new methods that leverage annotations attributing
    summary sentences to conversation utterances.
    \item A rigorous quantitative evaluation
    of our proposed models and appropriate baselines
    for both the extractive and abstractive components,
    including sensitivity of the pipeline to simulated ASR errors.
    \item A detailed human study to evaluate
    the factuality and quality of generated SOAP notes,
    and qualitative error analysis.
\end{itemize}

\section{Related Work}

Summarization is a well-studied problem
in NLP~\citep{nenkova2011automatic}.
While early works focused on simply
extracting important content from a document
\citep{erkan2004lexrank, wong2008extractive},
later approaches attempted to paraphrase the content
into new sentences (abstractive summarization)
\citep{filippova2010multi,berg2011jointly,wang2013domain}.
Following the development of neural sequence models
\citep{sutskever2014sequence},
more research focuses on neural generation
of abstractive summaries
\citep{nallapati2016abstractive,see2017get, celikyilmaz2018deep}.
While many papers summarize news articles,
others summarize conversations,
in business meetings \citep{wang2013domain,zhu2020end},
customer service \citep{liu2019automatic},
and tourist information center~\cite{yuan2019abstractive} contexts.

In the space of two-step extractive-abstractive summarization
approaches, \citet{subramanian2019extractive} summarize
scientific papers by first extracting sentences from
it and then abstractively summarizing them.
\citet{chen2018fast} extract important sentences
from the input and then paraphrase each of them
to generate the abstractive summary.
While they assume that each summary sentence
is supported by exactly one source sentence,
in our medical conversations,
many summary sentences synthesize content
spread across multiple dialogue turns
(e.g., a series of questions and answers).

Past work on abstractive summarization of medical conversations
has focused on summarizing patient-nurse conversations
with goals including capturing symptoms of interest \citep{liu2019topic}
and past medical history \citep{joshi2020dr}.
These tasks are respectively similar to generating
the \textit{review of systems}
and \textit{past medical history}
subsections of a SOAP note.
In contrast, we aim to generate a full-length SOAP note
containing up to $15$ subsections,
and propose methods to
address this challenge by extracting supporting context
for smaller parts and generating them independently.

\section{Dataset}

We use two different datasets in this work.
The primary medical dataset,
developed through a collaboration with Abridge AI,
consists of doctor-patient conversations
with annotated SOAP notes.
Additionally, we evaluate our summarization methods
on the AMI dataset \citep{carletta2007unleashing},
comprised of business meeting transcripts and their summaries.

\subsection{Medical dataset}
Our work builds on a unique resource:
a corpus consisting of thousands of recorded
English-language clinical conversations,
with associated SOAP notes created by a work force
trained in SOAP note documentation standards.
Our dataset consists of transcripts
from real-life patient-physician visits
from which sensitive information such
as names have been de-identified.
The full medical dataset consists of $6862$ visits
consisting of $2732$ cardiologist visits,
$2731$ visits for family medicine,
$989$ interventional cardiologist visits,
and $410$ internist visits.
Owing to the sensitive nature of the data,
we cannot share it publicly
(an occupational hazard of research
on machine learning for healthcare).

For each visit, our dataset contains
a human-generated transcript of the conversation.
The transcript is segmented
into utterances, each annotated
with a timestamp and speaker ID.
The average conversation lasts $9.43$ minutes
and consists of around $1.5\text{k}$ words
(Appendix Figure~\ref{fig:datasetstats}).
Associated with each conversation,
we have a human-drafted SOAP note
created by trained, professional annotators.
The annotators who created the SOAP notes
worked in either clinical transcription,
billing, or related documentation-related departments,
but were not necessarily professional medical scribes.
The dataset is divided into train,
validation and test splits of size
$5770$, $500$ and $592$, respectively.

Our annotated SOAP notes contain (up to) $15$ subsections,
each of which may contain multiple sentences.
The subsections vary in length.
The \textit{Allergies} subsections is most often empty,
while the  \textit{Assessment} subsection contains
$5.16$ sentences on average
(Table~\ref{tab:subsection_lengths}).
The average SOAP note contains $27.47$ sentences.
The different subsections also
differ in the style of writing.
The \textit{Medications} subsection
usually consists of bulleted names
of medicines and their dosages,
while the \textit{Assessment} subsection
typically contains full sentences.
On average, the fraction of novel
(i.e., not present in the conversation)
unigrams, bigrams, and trigrams,
in each SOAP note are
$24.09\%$, $67.79\%$ and $85.22\%$, respectively.

\begin{table}[t]
    \centering
\resizebox{\linewidth}{!}{
    \begin{tabular}{lc}
    \toprule
\textbf{Subsection} &	\textbf{Mean length} \\
\midrule
Family Medical History  & 0.23 \\
Past Surgical History 	& 0.58  \\
Review of Systems 	& 3.65 \\
Chief Complaint	& 2.17 \\
Miscellaneous	& 2.81 \\
Allergies	& 0.06 \\
Past Medical History 	& 2.93 \\
Social History	& 0.27 \\
Medications	& 3.74 \\
\midrule
 Immunizations	& 0.11 \\
Laboratory and Imaging Results 	& 2.27 \\
\midrule
Assessment	& 5.16 \\
\midrule
 Diagnostics and Appointments 	& 1.65 \\
Prescriptions and Therapeutics 	& 1.75\\
\midrule
Healthcare Complaints	& 0.09\\
\bottomrule
    \end{tabular}
}
    \caption{Average number of sentences in different SOAP note subsections
    grouped by parent sections (\textit{Subjective}, \textit{Objective}, \textit{Assessment}, \textit{Plan}, \textit{Others} resp.)}
    \label{tab:subsection_lengths}
    \vspace{-10px}
\end{table}

Each SOAP note sentence is also annotated with utterances
from the conversation which provide evidence for that sentence.
A SOAP note sentence can have
one or more supporting utterances.
On average, each SOAP sentence has $3.84$ supporting utterances,
but the mode is $1$ (Appendix Figure~\ref{fig:datasetstats}).
We refer to these utterances as \emph{noteworthy utterances} throughout this paper.
Throughout this work, we deal with
the $15$ more granular subsections
rather than the $4$ coarse sections of SOAP notes,
and thus for convenience,
all further mentions of \emph{section}
technically denote a SOAP \emph{subsection}.

\subsection{AMI dataset}
The AMI dataset is a collection of 138 business meetings,
each with 4 participants with various roles
(e.g., marketing expert, product manager, etc.).
Each meeting transcript comes
with an associated abstractive summary
that is divided into four sections---\textit{abstract},
\textit{decisions}, \textit{actions}, and \textit{problems}.
Each conversation also has
an associated extractive summary,
and there are additional annotations
linking the utterances in the extractive summary
to sentences in the abstractive summary.
For any given sentence in the abstractive summary,
we refer to the linked set of utterances
in the extractive summary as its \emph{noteworthy utterances}.
We note that 7.9\% of the abstractive summary sentences
have no annotated noteworthy utterances.
To simplify the analysis,
we remove these sentences from
summaries in the training,
validation, and test splits.

\section{Methods}
\label{sec:solution_approach}

We investigate the following four decompositions
of the summarization problem
into extractive and abstractive phases,
ordered from abstraction-heavy to extraction-heavy:
    {\sc Conv2Note} takes an end-to-end approach,
    generating the entire SOAP note
    from the entire conversation in one shot.
    {\sc Ext2Note} first predicts
    all of the noteworthy utterances
    in the conversation
    (without regard to the associated section)
    and then generates the entire SOAP note
    in one shot from only those utterances.
    {\sc Ext2Sec} extracts noteworthy utterances,
    while also predicting the section(s)
    for which they are relevant,
    and then generates each SOAP section separately
    using only that section's predicted noteworthy utterances.
    {\sc Cluster2Sent} attempts to group together
    the set of noteworthy utterances
    associated with each summary sentence.
    Here, we cluster separately among each set
    of section-specific noteworthy utterances
    and then generate each section one sentence at a time,
    conditioning each on the associated cluster of utterances.

Each of these pipelines leaves open many choices
for specific models to employ for each subtask.
For the abstractive modules of {\sc Conv2Note} and {\sc Ext2Note},
we use a pointer-generator network.
The abstractive modules of {\sc Ext2Sec} and {\sc Cluster2Sent},
which require conditioning on section
are modeled using conditioned pointer-generator networks
(described in Section~\ref{sec:modelarch}),
and fine-tuned T5 models which condition
on the section being generated
by means of prepending it to the input.
T5 models could not be used in
the {\sc Conv2Note} and {\sc Ext2Note} settings
because their high memory requirement
for long inputs could not be accommodated
even with 48GB of GPU memory.

For noteworthy utterance extraction,
we primarily use a hierarchical LSTM model
and a BERT-LSTM model as described in the next section.
All models are configured to have a scalar output
for binary classification in {\sc Ext2Note},
whereas for {\sc Ext2Sec} and {\sc Cluster2Sent},
they have multi-label output
separately predicting noteworthiness for each section.
Note that the same utterance can be noteworthy
with respect to multiple sections.
We use the same trained utterance extraction models
for both {\sc Ext2Sec} and {\sc Cluster2Sent}.

For the clustering module in {\sc Cluster2Sent},
we propose a heuristic
that groups together
any two supporting utterances
that are \emph{close},
meaning they have less than or equal to
$\tau$ utterances separating them,
where $\tau$ is a hyperparameter.
This process is iterated,
with the clusters growing in size
by merging with other singletons or clusters,
until every pair of \emph{close} utterances
have the same cluster membership.
The value of $\tau$
is tuned on the validation set.
Since each cluster necessarily produces
one sentence in the SOAP note,
having too many or too few clusters
can make the SOAP note
too long or too short, respectively.
Therefore, for any given value
of the hyper-parameter $\tau$ and any given section,
the prediction thresholds of the extractor
are tuned on the validation set to produce
approximately the same number of clusters
over the entire validation set
as present in the ground truth for that section.
Among ground truth clusters containing
multiple noteworthy utterances,
$82\%$ are contiguous.
In an experiment where the heuristic is used to cluster
the \emph{oracle noteworthy utterances} for each section,
and summaries are subsequently generated
via the abstractive modules from {\sc Cluster2Sent},
ROUGE-1 and ROUGE-2 metrics deteriorate
by less than $1$ point as compared
to \emph{oracle} clusterings
(Appendix Table~\ref{tab:rouge_scores_detailed}),
demonstrating our heuristic's effectiveness.

\section{Model Architectures}
\label{sec:modelarch}

\vspace{5px}
\noindent \textbf{Pointer-Generator Network \quad}
We use the pointer-generator network
introduced by \citet{see2017get}
for {\sc Conv2Note} and {\sc Ext2Note}.
The model is a bidirectional LSTM-based
encoder-decoder model with attention.
It employs a pointer mechanism
to copy tokens directly from the input
in addition to generating them
by predicting generation probabilities
for the entire vocabulary.
The model also computes the weights
that govern copying versus generating
at each decoding timestep.

\vspace{5px}
\noindent \textbf{Section-conditioned Pointer-Generator Network \quad}
We modify the pointer-generator network
for algorithms {\sc Ext2Sec} and {\sc Cluster2Sent},
to condition on the (sub)section
of the summary to be generated.
The network uses a new lookup table
to embed the section $z$
into an embedding $\bm{e}^z$.
The section embedding is concatenated
to each input word embedding fed into the encoder.
The section embedding is also appended to the inputs
of the decoder LSTM in the same fashion.

\vspace{5px}
\noindent \textbf{T5 \quad}
We use the recently released T5 model
\citep{raffel2020exploring} as an abstractive module.
It is an encoder-decoder model,
where both encoder and decoder
consist of a stack of transformer layers.
The T5 model is pre-trained on 5 tasks,
including summarization, translation etc.
We use the pre-trained T5 model parameters
and fine-tune it on our task dataset.
For introducing the section-conditioning
in {\sc Ext2Sec} and {\sc Cluster2Sent},
we simply add the name of the section
being generated to the beginning of the input.

\vspace{5px}
\noindent \textbf{Hierarchical LSTM classifier(H-LSTM) \quad}
In this model, we first encode each utterance $u_i$ independently
by passing its tokens through a bidirectional LSTM
and mean-pooling their encoded representations
to get the utterance representation $\bm{h}_i$.
We pass the sequence of utterance representations
$\{\bm{h}_1,\bm{h}_2,...,\bm{h}_n\}$
through another bidirectional LSTM
to get new utterance representations
which incorporate neighboring contexts.
These are then passed through a sigmoid activated linear layer
to predict each utterance's probability of noteworthiness
with respect to each section.

\vspace{5px}
\noindent \textbf{BERT-LSTM classifier(B-LSTM) \quad}
In this model, tokens in the utterance $u_i$
are passed through a \texttt{BERT} encoder
to obtain their contextualized representations,
which are mean-pooled to get the utterance representation $\bm{h}_i$.
The subsequent architecture exactly mirrors hierarchical LSTM,
and involves passing utterance representations
through a bidirectional LSTM and linear layer to get output probabilities.
BERT-LSTM is fine-tuned in an end-to-end manner.

\section{Experiments}
We first establish two baselines.
{\sc RandomNote} randomly and uniformly
samples a SOAP note from the training set
and outputs it as the summary
for any input conversation.
{\sc OracleExt} presents all the
ground truth noteworthy utterances (evidence)
from the conversation as the SOAP note
without any abstractive summarization.
Thus, the {\sc OracleExt} baseline has the advantage
of containing all the desired information
(e.g., names of medicines) from the conversation,
but the disadvantage of not being expressed
in the linguistic style of a SOAP note
which leads to lower n-gram overlap.
The opposite is true for the {\sc RandomNote} baseline.
Both baselines give similar performance
and are outperformed by the simple {\sc Conv2Note} approach
(Table~\ref{tab:methods_rouge}).

We train the abstractive modules for
the 4 approaches described in Section~\ref{sec:solution_approach}
with the ground truth noteworthy utterances as inputs.
To estimate an upper bound on the performance
we can reasonably hope to achieve
by improving our noteworthy utterance extractors,
we test our models with oracle noteworthy utterances in the test set.
All algorithms relying on oracle noteworthy utterances
outperform {\sc Conv2Note}, and exhibit
a monotonic and significant rise in ROUGE scores
as we move towards the extraction-heavy end of the spectrum
(Table~\ref{tab:oracle_methods_rouge1})\footnote{ The character `-' represents GPU memory overflow}.

\begin{table*}[t!]
    \centering
    \begin{tabular}{l c c c c c c c}
    \toprule
    & \multicolumn{3}{c}{\textbf{Medical dataset}} & \hspace{10pt} & \multicolumn{3}{c}{\textbf{AMI corpus}} \\
    \midrule
    \textbf{Method} & \textbf{R-1} & \textbf{R-2} & \textbf{R-L} & & \textbf{R-1} & \textbf{R-2} & \textbf{R-L}  \\
    \midrule
        {\sc RandomNote} & 34.99 & 12.69 & 21.37  & & 42.47 & 11.55 & 21.47\\
        {\sc OracleExt} & 33.07 & 12.22 & 17.42  & & 39.97 & 11.17 & 20.91 \\
        \midrule
        {\sc Conv2Note} (PG) & 49.56 & 25.68 & 32.87 & & 39.62 & 13.16 & 23.95\\
        \midrule
        {\sc Ext2Note} (PG + HLSTM) & 49.58	& 24.91 & 31.68 & & 21.28 &	7.06 &	15.96\\
        {\sc Ext2Note} (PG + BLSTM) & 50.50 & 25.4 & 31.93 & & 21.71 &	6.83 &	15.69\\
        {\sc Ext2Note} (T5-Small  + HLSTM) & - & - & - & & 40.48 &	13.82 &	24.64\\
        {\sc Ext2Note} (T5-Small  + BLSTM) & - & - & - & & 40.36 &	13.73 &	24.13\\
        \midrule
        {\sc Ext2Sec} (PG + HLSTM) & 55.23 &	27.14 &	35.15 & & 43.75 &	15.25 &	23.46\\
        {\sc Ext2Sec} (PG + BLSTM) & 55.74 &	27.54 &	36.09 & & 40.48 &	15.61 &	23.31\\
        {\sc Ext2Sec} (T5-Small + HLSTM)& 55.77 &	28.64 &	37.50 & & 42.45 &	15.20 &	23.92\\
        {\sc Ext2Sec} (T5-Small + BLSTM)& 56.00 &	29.16 &	\textbf{38.38} & & 45.44 &	16.59 &	\textbf{26.14}\\
        \midrule
        {\sc Cluster2Sent} (PG + HLSTM) & 55.46	& 27.41 &	35.81 & & 46.19 &	16.64 &	24.29 \\
        {\sc Cluster2Sent} (PG + BLSTM) & 55.60 &	27.68 &	36.29 & & 42.31 &	15.92 &	23.51\\
        {\sc Cluster2Sent} (T5-Small + HLSTM)& 56.88 &	28.63 &	36.78& & 45.10 &	15.06 &	23.52\\
        {\sc Cluster2Sent} (T5-Small + BLSTM)& 57.14 &	29.11 &	37.43 & & 42.38	& 15.36 &	23.9\\
        {\sc Cluster2Sent} (T5-Base + HLSTM)& 57.27 &	29.10 &	37.38 & & \textbf{50.52} &	17.56 &	24.89\\
        {\sc Cluster2Sent} (T5-Base + BLSTM)& \textbf{57.51}	& \textbf{29.56} &	38.06 & & 45.91	& \textbf{17.70} &	25.24\\
    \bottomrule
    \end{tabular}
    \caption{ROUGE scores achieved by different methods on the two datasets}
    \label{tab:methods_rouge}
    \vspace{-0.3cm}
\end{table*}

\begin{table}[t!]
    \centering
\resizebox{\linewidth}{!}{
    \begin{tabular}{l c c c c c }
    \toprule
    & \multicolumn{2}{c}{\textbf{Medical dataset}} &  \multicolumn{2}{c}{\textbf{AMI corpus}} \\
    \midrule
    Method & \small{PG} & \small{T5-Small} & \small{PG} & \small{T5-Small} \\
    \midrule
    {\sc Ext2Note} & 52.95 & - & 20.44 & 41.10 \\
    {\sc Ext2Sec} & 61.00 & 62.37 & 43.32 & 46.85 \\
    {\sc Cluster2Sent} & 63.63 & 66.50 & 51.86 & 54.23 \\
    \bottomrule
    \end{tabular}
}
    \caption{ROUGE-1 achieved on test set when using
    the abstractive models with oracle noteworthy utterances and clusters (more results with oracle in the Appendix)}
    \label{tab:oracle_methods_rouge1}
    \vspace{-0.3cm}
\end{table}

For predicting noteworthy utterances,
we use two baselines:
(i) logistic regression on TF-IDF utterance representations;
and (ii) a model with a bidirectional LSTM
to compute token-averaged utterance representations,
followed by a linear classification layer.
These two models make the predictions
for each utterance independent of others.
In contrast, we also use models
which incorporate context
from neighboring utterances:
(a) a hierarchical LSTM;
and (b) a BERT-LSTM model
as described in Section~\ref{sec:modelarch}.
The latter two methods perform much better
(Table~\ref{tab:classification_results}),
demonstrating the benefit
of incorporating neighboring context,
with BERT-LSTM performing the best
(see Appendix Table~\ref{tab:extractor_performance_sectionwise}
for section-wise performance).

Using predicted noteworthy utterances and clusters
instead of oracle ones leads to a drop in ROUGE scores,
but the performance of {\sc Ext2Sec} and {\sc Cluster2Sent}
is still better than {\sc Conv2Note} (Table~\ref{tab:methods_rouge}).
For the medical dataset, using a BERT-LSTM extractor
leads to the best performance,
with {\sc Cluster2Sent} outperforming {\sc Conv2Note}
by about $8$ points in ROUGE-1
(see Appendix Table~\ref{tab:sectionwise_results}
for section-wise performance).
Interestingly, the T5-Small variant
achieves similar performance to T5-Base,
despite being only about a quarter of the latter's size.

\vspace{5px}
\noindent \textbf{Performance on AMI dataset \quad}
We see a similar trend in the ROUGE scores
when applying these methods on the AMI dataset.
One exception is the poor performance
of pointer-generator based {\sc Ext2Note},
which excessively repeated sentences
despite using a high coverage loss coefficient.
There is a larger gap between the performance
of the T5-Small and T5-Base abstractive models on this dataset.
As an extractor, the performance of BERT-LSTM
is again better than HLSTM
(Table~\ref{tab:classification_results}),
but when used in tandem with the abstractive module,
ROUGE scores achieved by the overall pipeline
do not always follow the same order.
We also observe that the clustering heuristic
does not work as well on this dataset.
Specifically, tuning the thresholds of the extractive model,
while fixing the clustering threshold $\tau$
gave worse results on this dataset.
Tuning the thresholds independent of the clusters
performed  better.
However, the best method still outperforms
{\sc Conv2Note} by about 11 ROUGE-1 points
(Table~\ref{tab:methods_rouge}).

\begin{table}[t]
    \centering
\resizebox{\linewidth}{!}{
    \begin{tabular}{l c c c}
    \toprule
    \textbf{Method} & \textbf{R-1} & \textbf{R-2} & \textbf{R-L} \\
    \midrule
    \multicolumn{4}{c}{\textbf{Train on clean data + Test on data with 10\% error rate}}\\
    \midrule
        {\sc Conv2Note}(PG) & 46.52	& 22.60 & 30.45\\
        {\sc Cluster2Sent}(PG + BLS) & 51.84 &	23.74 &	32.94\\
        {\sc Cluster2Sent}(T5-Base+ BLS) & 54.88 & 26.65 & 35.88\\
        \midrule
        \multicolumn{4}{c}{\textbf{Train and test on data with 10\% error rate}}\\
        \midrule
        {\sc Conv2Note}(PG) & 48.85 &	24.85 &	31.27\\
        {\sc Cluster2Sent}(PG + BLS) &  54.68	& 26.59 &	35.70\\
        {\sc Cluster2Sent}(T5-Base+ BLS) &  56.35 &	28.50 &	37.04\\
        \midrule        \multicolumn{4}{c}{\textbf{Train and test on data with 30\% error rate}}\\
        \midrule
        {\sc Conv2Note}(PG) & 45.16 &	22.26 &	30.14\\
        {\sc Cluster2Sent}(PG + BLS) &  53.69	& 25.88 & 35.12\\
        {\sc Cluster2Sent}(T5-Base+ BLS) &  55.90 &	27.73 &	36.06\\
    \bottomrule
    \end{tabular}
}
    \caption{Performance of models trained and tested on data with different simulated ASR error rates. BLS: BERT-LSTM}
    \label{tab:asr_errors}
\end{table}

\vspace{5px}
\noindent \textbf{Performance with ASR errors \quad}
In the absence of human-generated transcripts of conversations,
Automatic Speech Recognition (ASR) techniques can be used
to transcribe the conversations for use by our models.
To account for ASR errors, we artificially added errors
in transcripts of the medical dataset
by randomly selecting some percentage of the words
and replacing them with phonetically similar words
using RefinedSoundEx~\citep{refined_soundex} (details in the Appendix).
Models trained on clean dataset perform worse
on a $10\%$ corrupted test dataset
(Table~\ref{tab:asr_errors}).
Since ASR errors lead to replacement of a correct word
by only a small set of phonetically similar words,
there is still some information
indicating the original word
that can be used by the models.
When we train our models on data
corrupted at the 10\% ASR error rate,
our models recover much of the performance drop
(Table~\ref{tab:asr_errors}).
Notably when simulated ASR errors
are dialed up to a 30\% error rate,
(both at train and test time)
we see a smaller performance drop
for {\sc Cluster2Sent}
as compared to {\sc Conv2Note}.

\begin{table}[t]
    \centering
\resizebox{\linewidth}{!}{
\begin{tabular}{l c c c c c c c}
\toprule
& \multicolumn{4}{c}{\textbf{Medical conversations}} & \hspace{1pt} & \multicolumn{2}{c}{\textbf{AMI corpus}}\\
\midrule
\textbf{Metric} &  \textbf{LR} &  \textbf{LS} & \textbf{HLS} & \textbf{BLS} &  & \textbf{HLS} & \textbf{BLS} \\
\midrule
Accuracy        & 96.0  &  96.1 & 96.5 & 96.5 & & 93.77 & 94.16 \\
Ma-AUC       & 78.1  &  79.3 & 90.0 & 90.5 & & 83.81 & 90.76 \\
Ma-F1        & 29.5  &  31.0 & 38.6 & 40.9 & & 19.95 & 33.08 \\
Mi-AUC       & 87.3  &  87.6 & 92.7 & 93.3 & & 93.21 & 94.90 \\
Mi-F1        & 31.2  &  32.9 & 39.6 & 41.1 & & 43.76 & 49.93 \\
\bottomrule
\end{tabular}
}
    \caption{Performance on multilabel classification of noteworthy utterances
    with logistic regression(LR), LSTM(LS),
    Hierarchical-LSTM(HLS) and BERT-LSTM(BLS).
    Ma:macro-averaged. Mi:micro-averaged }
    \label{tab:classification_results}
    \vspace{-0.3cm}
\end{table}

\section{Qualitative Analysis}
\label{sec:qual_analysis}

\begin{figure*}[t!]
    \centering
    \includegraphics[width=\textwidth]{figures/sample_entrywise_outputs2.pdf}
    \vspace{-0.95cm}
    \caption{Noteworthy utterance clusters summarized in different ways
    for different sections by the abstractive summarization modules of {\sc Cluster2Sent}
    (utterances were slightly obfuscated for privacy reasons)}
    \label{fig:sample_entrywise_summaries}
    \vspace{-0.4cm}
\end{figure*}

The conditioned pointer-generator
and T5 models used in {\sc Cluster2Sent}
learn to place information
regarding different topics
in appropriate sections.
Hence, given a cluster of supporting utterances,
the models can generate different summaries
for multiple sections
(Figure~\ref{fig:sample_entrywise_summaries}).
For example, given the same supporting utterances
discussing the patient's usage
of \textit{lisinopril} for low blood pressure,
a model generates ``low blood pressure''
in the \textit{review of systems} section,
and ``lisinopril'' in \textit{medications} section.
We direct the reader
to the appendix for examples of full-length
generated SOAP notes.

Interestingly, when the abstractive model
is given a cluster of utterances that are
not relevant to the section being generated,
the model sometimes outputs fabricated information
relevant to that section
such as saying the patient is a non-smoker
in \textit{social history},
or that the patient has taken a flu shot
in \textit{immunizations} .
Hence, the quality of produced summaries
heavily depends on the ability
of the extractive step
to classify the extracted utterances to the correct section.
Another cause of false information
is the usage of pronouns in clusters
without a mention of the referred entity.
In such situations, T5 models frequently
replace the pronoun with some arbitrary entity
(e.g. ``she'' with ``daughter'', compounds with ``haemoglobin'',
and medicines with ``lisinopril'').

Occasionally, the abstractive module produces
new \emph{inferred} information
that is not mentioned explicitly in the conversation.
In one instance, the model generated
that the patient has a history of heart disease
conditioned on a cluster that mentioned
he/she takes \emph{digoxin},
a popular medicine for heart disease.
Similarly, the model can infer
past medical history of ``high cholesterol''
upon seeing  \emph{pravastatin} usage.
Such inferences can also lead to incorrect summaries,
e.g., when a doctor explained
that a patient has leaky heart valves, a model added a sentence
to the \emph{diagnostics and appointments} section saying ``check valves''.

{\sc Cluster2Sent} summarizes localized regions
of the conversation \emph{independently},
which may lead to contradictions
in the SOAP note.
In one visit, the patient was asked about chest pain twice---once
in the beginning to get to know his/her current state,
and once as a question about how he/she felt
just before experiencing a fall in the past.
This led to the model generating both
that the patient denied chest pain
as well as confirmed chest pain, without
clarifying that one statement was for the present
and another for the past.

\section{Human evaluation}
We asked trained human annotators to evaluate
generated SOAP notes for $45$ conversations.
Every sentence in each SOAP note was labeled
according to various quality dimensions such whether it was factually correct,
incoherent, irrelevant, redundant,
or placed under an inappropriate section.
The
detailed statistics of annotations received for each quality dimension are provided in the Appendix.
We also collected aggregate annotations
for the comprehensiveness of each SOAP note
and the extent to which it verbatim copied the transcript
on a 5-point Likert scale.

Human raters were presented
with a web interface
showing the conversation, along with
a search feature to help them
in looking up desired information.
The summaries generated by three methods ({\sc Conv2Note}(pointer-generator),  {\sc Cluster2Sent}(pointer-generator) and {\sc Cluster2Sent}(T5-base)) were presented in random order to hide their identities.
For each sentence, we asked for
(i) Factual correctness of the sentence;
(ii) If the statement is simply repeating what has already been mentioned before;
(iii) If the statement is clinically irrelevant;
(iv) If the statement is incoherent (not understandable due to grammatical or semantic errors);
and (v) If the statement's topic does not match the section in which it is placed.
In addition, we asked two separate questions for rating the overall summary on a scale of 1-5 for its (i) comprehensiveness and (ii) extent of verbatim copying from conversation.
The human evaluation of the SOAP notes was done by workers who had also participated in the creation of the dataset of SOAP notes. Hence, they had already been extensively trained in the task of SOAP note creation, which gave them appropriate knowledge to judge the SOAP notes.

To quantify the performance among different methods,
we consider a scenario where each generated SOAP note
has to be post-edited by discarding undesirable sentences.
For a generated SOAP note, we define its \textit{yield}
as the fraction of its total sentences that are not discarded.
The sentences that are retained are those
that are both factually correct
and were not labeled as either repetitive or incoherent.
The human annotations show that both {\sc Cluster2Sent}-based methods tested
produced a higher yield than the {\sc Conv2Note} baseline (p$<0.02$).
T5-base performs better than conditioned pointer-generator
as the abstractive module in {\sc Cluster2Sent} setting,
producing significantly more yield
(Table~\ref{tab:humaneval_results}).
T5 also produces fewer incoherent sentences
(Appendix Table~\ref{tab:humaneval_detailed_results})
likely due to its exposure to a large number
of well-formed coherent sentences during pretraining.

We conducted an analogous human evaluation of summaries generated
for all $20$ conversations in the test set of the AMI corpus,
and saw a similar trend in the expected yield for different methods.
Notably, for the AMI corpus, {\sc Conv2Note} produced
a very high proportion of redundant sentences ($>0.5$)
despite using the coverage loss,
while the pointer-generator based {\sc Cluster2Sent}
produced a high proportion of incoherent sentences
(Appendix Table~\ref{tab:humaneval_detailed_results}).

\begin{table}[t!]
    \centering
\resizebox{\linewidth}{!}{
\begin{tabular}{l c c c c c c}
\toprule
& \multicolumn{3}{c}{\textbf{Medical conversations}} &  \multicolumn{3}{c}{\textbf{AMI corpus}}\\
\midrule
\textbf{Metric} &  \textbf{C2N} &  \textbf{C2S-P} & \textbf{C2S-T} &    \textbf{C2N} & \textbf{C2S-P} & \textbf{C2S-T}\\
\midrule
Length    & 21.2  &  28.2 & 28.4    & 20.7 &  17.9 & 19.05 \\
\%Yield     & 62.0  &  69.0 & 74.7    & 27.22 &  30.22 & 59.45 \\
Comp       & 2.44  &  2.42 & 2.76    & 2.30 &  2.55 & 3.75 \\
Copy       & 2.18  &  2.64 & 2.76    & 1.80 &  1.80 & 1.90 \\
\bottomrule
\end{tabular}
}
    \caption{Averages of different metrics for {\sc Conv2Note}(C2N), {\sc Cluster2Sent}  with pointer-generator (C2S-P) and T5-base (C2S-T). Comp:comprehensiveness, Copy:amount of copying. Length: number of sentences generated.}
    \label{tab:humaneval_results}
    \vspace{-0.3cm}
\end{table}

\section{Conclusion}

This paper represents the first attempt
at generating full-length SOAP notes
by summarizing transcripts
of doctor-patient conversations.
We proposed a spectrum of extractive-abstractive
summarization methods that leverage:
(i) section-structured form of the SOAP notes
and (ii) linked conversation utterances associated with every SOAP note sentence.
The proposed methods perform better than a fully abstractive approach and standard extractive-abstractive approaches that do not take advantage of these annotations.
We demonstrate the wider applicability of proposed approaches by showing similar results on the public AMI corpus which has similar annotations and structure.
Our work demonstrates the benefits of creating section-structured
summaries (when feasible) and collecting evidence for each summary sentence
when creating any new summarization dataset.

\section*{Ethics Statement}

The methods proposed in this work to
generate SOAP notes involve neural
models that sometimes generate
factually incorrect text~\cite{maynez2020faithfulness}.
The detection and correction of such factual errors in automatically generated summaries is an active area of research~\cite{cao2018faithful,zhang-etal-2020,dong2020multi}.
We emphasize that the methods are intended to be
used with supervision from a
medical practitioner who
can check for factual errors
and edit the the generated SOAP note if needed.
We have estimated the frequency of such factual errors (Appendix Table~\ref{tab:humaneval_detailed_results}) and characterized multiple types of errors seen in generated
SOAP notes in Section~\ref{sec:qual_analysis},
for which the medical practitioners should remain vigilant.
For example, there is a bias to incorrectly generate information that occur frequently in specific sections (e.g. ``patient took flu shot''),
and to replace pronouns with frequently seen entities (such as ``lisinopril'' for references to medicine).
All data used in this study was manually  de-identified before we accessed it.
Deploying the proposed methods does not require long-term storage of conversations.
After the corresponding SOAP notes are generated,
conversations can be discarded.
Hence, we do not anticipate any additional privacy risks
from using the proposed methods.

\section*{Acknowledgements}
This work was funded by the Center for Machine Learning and Health in a joint venture between UPMC and Carnegie Mellon University.
We gratefully acknowledge support from Abridge AI, Inc.
for creating the dataset of SOAP notes and providing
human resources for evaluation.

\section*{Appendix}

\begin{table}[b!]
    \centering
    \begin{tabular}{l c c c}
    \toprule
    \textbf{Copy mechanism} & \textbf{R-1} & \textbf{R-2} & \textbf{R-L}  \\
    \midrule
    Present  & 63.63 & 35.62 & 48.85\\
    Absent  & 61.92 & 34.37 & 47.86\\
    \bottomrule
    \end{tabular}
    \caption{Impact of copy mechanism in peformance of a pointer-generator model on medical dataset in {\sc Cluster2Sent} using oracle noteworthy utterance clusters}
    \label{tab:mustusecopy}

\end{table}

\begin{table}[b!]
    \centering
    \begin{tabular}{l c c c}
    \toprule
    \textbf{Model} & \textbf{R-1} & \textbf{R-2} & \textbf{R-L}  \\
    \midrule
    Pretrained T5  & 66.45 & 39.01 & 52.46\\
    Randomly initialized T5 & 40.07 & 20.95 & 32.42\\
    \bottomrule
    \end{tabular}
    \caption{Impact of pretraining on performance of T5-Base model on  medical dataset with {\sc Cluster2Sent} using oracle noteworthy utterance clusters}
    \label{tab:dopretrain}

\end{table}

\subsection*{Decoder Results with Oracle extracts}
We present additional quantitative results (Table~\ref{tab:rouge_scores_detailed}),
including
(i) The ROUGE scores on the test set
when using oracle noteworthy utterances
with both oracle and predicted
clusters (for {\sc Cluster2Sent} models).
(ii) Two ablations on {\sc Ext2Sec}:
{\sc AllExt2Sec} uses binary classification to extract all noteworthy utterances
(not per-section), and an abstractive decoder that conditions on the section;
while {\sc Ext2SecNoCond} uses a multilabel classification based extractor
but \emph{does not use section-conditioning} in the abstractive module.
Both methods mostly perform worse than {\sc Ext2Sec} demonstrating the benefit of using both section-specific extraction and section-conditioning in abstractive decoder.

\begin{table*}[h!]
    \centering
    \begin{tabular}{l c c c c c c c}
    \toprule
    & \multicolumn{3}{c}{\textbf{Medical dataset}} & \hspace{0pt} & \multicolumn{3}{c}{\textbf{AMI corpus}} \\
    \midrule
    \textbf{Method} & \textbf{R-1} & \textbf{R-2} & \textbf{R-L} & & \textbf{R-1} & \textbf{R-2} & \textbf{R-L}  \\
    \midrule
        {\sc Ext2Note} (PG) & 52.95 & 27.6 & 32.87 & & 21.23 & 6.71 & 14.95\\
        {\sc Ext2Note} (T5-Small) & - & - & - & & 41.10 & 14.12 & 25.03\\
{\sc AllExt2Sec} (PG) & 50.74 & 24.33 & 32.18 & & 40.20 & 13.71 & 22.52 \\
{\sc AllExt2Sec} (T5-Small) & - & - & - & & 41.68 & 15.43 & 24.72 \\
{\sc Ext2SecNoCond} (PG) & 56.10 & 32.05 & 43.23 & & 42.51 & 15.71 & 23.79 \\
{\sc Ext2SecNoCond} (T5-Small) & 58.69 & 34.92 & 47.24 & & 48.14 & 18.49 & 28.23 \\
        {\sc Ext2Sec} (PG) & 61.00 & 33.64 & 45.2 & & 43.30 & 16.56 & 24.83 \\
        {\sc Ext2Sec} (T5-Small)& 62.37 & 36.39 & 49.11 & & 46.85 & 18.19 & 28.74\\
        {\sc Cluster2Sent} (PG) & 63.63 & 35.62 & 48.85 & & 51.86 & 21.86 & 31.84 \\
        {\sc Cluster2Sent} (T5-Small)&  66.50 &	38.41 &	51.73  & & 54.23 & 22.90 & 34.54 \\
        {\sc Cluster2Sent} (T5-Base)& 66.45 & 39.01 & 52.46 & & 57.42 & 24.45 & 35.70\\
        \midrule
        {\sc Cluster2Sent} (PG+clustering heuristic) & 63.12	& 35.08	& 47.96 & & 47.17 &	18.99 &	27.31 \\
        {\sc Cluster2Sent} (T5-Small+clustering heuristic)& 66.08 &	37.73 &	50.66 & & 47.53 &	19.70 &	28.95 \\
        {\sc Cluster2Sent} (T5-Base+clustering heuristic)& 65.94 &	38.26 &	51.31 & & 51.24 &	21.47 &	29.81\\
    \bottomrule
    \end{tabular}
    \caption{ROUGE scores achieved by different abstractive decoders using oracle noteworthy utterances}
    \label{tab:rouge_scores_detailed}
    \vspace{-0.3cm}
\end{table*}

\subsection*{Impact of copy mechanism}
When we do not use copy mechanism in the pointer-generator model, we observed a drop in its performance in the {\sc Cluster2Sent} setting with oracle noteworthy noteworthy utterances and clusters(Table~\ref{tab:mustusecopy}). Hence, we have used copy mechanism in all the pointer-generator models we train in this work.

\subsection*{Impact of pretraining}
When training a randomly initialized  T5-Base model on the medical dataset, even in {\sc Cluster2Sent} setting with oracle clusters, it only got a ROUGE-1 around 40 (Table~\ref{tab:dopretrain}). This is over 16 points lower than what we get by starting with off-the-shelf pretrained T5 parameters, and is even worse than {\sc Conv2Note}, highlighting the importance of pretraining.

\subsection*{Sample generated SOAP notes}

Due to privacy concerns,
we can not publish conversations from our dataset.
Here, we present an obfuscated conversation from our test dataset,
modified by changing sensitive content
such as medicines, diseases, dosages (Figure~\ref{fig:fullconv}).
We also present the SOAP note generated by our best method, as well as the ground truth.

\begin{figure}[b!]
    \includegraphics[width=0.48\textwidth]{figures/dataset_stats_pair.png}
    \caption{
Histogram of number of words in a conversation and the number of evidence utterances per summary sentence for the medical dataset}
        \label{fig:datasetstats}
\end{figure}

\begin{figure*}[t]

    \makebox[\linewidth]{
        \includegraphics[width=\linewidth]{figures/sample_full_note.pdf}
    }

    \caption{Sample conversation (obfuscated) with SOAP note generated by the best method and the ground truth}
    \label{fig:fullconv}
\end{figure*}

\subsection*{Model implementation details}
For the hierarchical LSTM classifier, we have a word embedding size of $128$ and both bidirectional LSTMs have a hidden size of $256$. For BERT-LSTM, the BERT embeddings are initialized from bert-base-uncased (768 dimensions). LSTMs in either direction have a hidden-layer of size $512$ and the entire model is optimized end-to-end with a learning-rate of 0.001. For BERT-LSTM, an input conversation is divided into chunks of 128 utterances. Due to GPU constraints, these chunks are processed one at a time.
The pointer-generator models have a word embedding size of $128$, and a hidden size of $256$ for both the encoder and the decoder. The section embeddings used in section-conditioned pointer-generator network have $32$ dimensions. During training of all pointer-generator models, the model is first trained without coverage loss~\citep{tu2016modeling} to convergence, and then trained further with coverage loss added. We tried coverage loss coefficients varying from 0.5 to 8.

The pointer-generator models were trained using Adam optimizer before coverage and using SGD after adding coverage. We tried learning rates between $10^{-4}$ and $10^{-3}$ with Adam. The next word prediction accuracy
was used as the validation criterion for early stopping while training abstractive modules, with the
exception of coverage-augmented models that used a combination of crossentropy and coverage loss. Micro-averaged AUC was used as the validation criterion for training of extractive modules.

We employ beam search with beam size $4$
to decode outputs from both models.
For the vanilla pointer-generator model
used in {\sc Conv2Note} and {\sc Ext2Note},
we modified the beam search procedure
to make sure that all the SOAP note sections
are generated in proper order.
We start the beam search procedure by feeding
the header of the first section (chief complaint).
Whenever the model predicts a section header
as the next word and it shows up in a beam,
we check if it is the next section to be generated.
If not, we replace it with the correct next section's header.
Any \texttt{end-of-summary} tokens generated
before all the sections have been produced are also replaced similarly.
Note that producing all sections simply means
that the headers for each section have to be generated,
and a section can be left empty by starting
the next section immediately after generating the previous header.
The decoding length for beam search is constrained to be between $5^{th}$ and $95^{th}$ percentile of the target sequence length distribution, calculated on the training set.

\subsection*{Simulating ASR Errors}
We simulate ASR errors at any given percentage rate by randomly selecting the percentage of the words in the conversation and replacing them with phonetically similar words. To reduce the search space of possible candidates for each word, we use the \url{suggest()} function taken from the Pyenchant\footnote{https://pypi.org/project/pyenchant/} library that provides auto-correct suggestions for the input word.
Each suggestion is then passed through the Refined SoundEx algorithm to find the phonetic distance between the original and the suggested word. We use the pyphonetics\footnote{https://pypi.org/project/pyphonetics/} package for a python implementation of this algorithm. For our final candidate list, we choose words that are at phonetic distance of $1$ from the original word. Finally, a candidate is chosen at random from this list to replace the original.

\begin{table*}[t]
    \centering
\resizebox{0.8\linewidth}{!}{
\begin{tabular}{lccccccc}
\toprule
    & \multicolumn{3}{c}{\textbf{Medical conversations}} & \hspace{10pt} & \multicolumn{3}{c}{\textbf{AMI corpus}} \\
\midrule
Count & C2N & C2S-P & C2S-T &    								& C2N & C2S-P & C2S-T \\
\midrule
Total sentences		   	  & 956 & 1268 & 1277 &    				& 414 &  358  &  381  \\
Repetitive			  	  & 96 & 127 & 147 &    				& 213 &  14  &  14  \\
Incoherent 			  	  & 162 & 158 & 58 &    				& 9 &  134  &  27  \\
True statements 				  & 587 & 848 & 931 &    		& 89 &  103  &  227  \\
False statements 				  & 100 & 116 & 125 &    		& 71 &  75  &  68  \\
Truthfulness undecided 				  & 11 & 19 & 16 &    		& 32 &  32  &  45  \\
Irrelevant 				  & 25 & 34 & 24 &    					& 14 &  15  &  2  \\
Under incorrect section 			  & 56 & 42 & 39 &    		& 4 &  2  &  18  \\
\bottomrule
\end{tabular}
}
    \caption{Number of sentences produced by different methods that were judged to have different listed characteristics by human raters. C2N:{\sc Conv2Note}, C2S-P:{\sc Cluster2Sent} with pointer-generator, C2S-T:{\sc Cluster2Sent} with T5-base. BERT-LSTM used for medical dataset, hierarchical-LSTM used for AMI corpus.}
    \label{tab:humaneval_detailed_results}
\end{table*}

\begin{table*}[h]
    \centering
\resizebox{0.85\linewidth}{!}{
\begin{tabular}{lccccc}
\toprule
\textbf{Subsection} &  \textbf{ROUGE-1} &  \textbf{ROUGE-2} &  \textbf{ROUGE-L} &  \textbf{N} & \textbf{L} \\
\midrule
chief complaint                &   44.34 &   28.12 &   43.59 &         592 &                    11.46 \\
review of systems              &   46.88 &   28.35 &   43.28 &         514 &                    29.24 \\
past medical history           &   53.48 &   37.70 &   51.80 &         547 &                    17.81 \\
past surgical history          &   58.44 &   43.08 &   57.04 &         230 &                    10.36 \\
family medical history         &   51.94 &   36.49 &   50.13 &          72 &                    16.14 \\
social history                 &   57.72 &   37.82 &   56.30 &          97 &                    10.33 \\
medications                    &   49.56 &   23.53 &   47.64 &         549 &                    15.28 \\
allergies                      &   39.29 &    6.63 &   38.32 &          21 &                     8.57 \\
miscellaneous                  &   28.87 &   11.61 &   24.90 &         415 &                    34.44 \\
immunizations                  &   55.95 &   27.49 &   54.81 &          25 &                     7.32 \\
laboratory and imaging results &   58.36 &   41.18 &   55.11 &         448 &                    19.37 \\
assessment                     &   39.01 &   15.31 &   25.35 &         570 &                   132.41 \\
diagnostics and appointments   &   52.85 &   35.70 &   50.43 &         488 &                    17.67 \\
prescriptions and therapeutics &   50.53 &   33.51 &   48.10 &         446 &                    18.73 \\
healthcare complaints          &   30.11 &   15.79 &   29.57 &          43 &                    16.74 \\
\bottomrule
\end{tabular}
}
    \caption{Average ROUGE scores (from {\sc Cluster2Sent}  T5Base+BLSTM) for each section of SOAP note (N-number of test datapoints with the section populated, L-average number of words in ground truth)}
    \label{tab:sectionwise_results}
\end{table*}

\begin{table*}[h!]
\small
    \centering
\resizebox{0.95\linewidth}{!}{
\begin{tabular}{lrrrrrrr}
\toprule
\textbf{Section} &  \textbf{Base rate(\%)} &  \textbf{Precision} &  \textbf{Recall} &     \textbf{F1} &  \textbf{Accuracy} &    \textbf{AUC}  \\
\midrule
chief complaint                &  3.12 &      34.71 &   33.93 &  34.31 &     95.95 &  86.81\\
review of systems              &  5.10 &      51.35 &   51.82 &  51.58 &     95.04 &  93.12\\
past medical history           &  3.41 &      36.00 &   36.52 &  36.26 &     95.63 &  88.00\\
past surgical history          &  0.99 &      33.80 &   34.50 &  34.14 &     98.68 &  93.74\\
family medical history         &  0.31 &      52.31 &   45.25 &  48.53 &     99.70 &  99.23\\
social history                 &  0.53 &      59.81 &   54.87 &  57.23 &     99.56 &  95.41\\
medications                    &  4.45 &      51.66 &   49.13 &  50.36 &     95.69 &  92.02\\
allergies                      &  0.16 &      30.86 &   12.44 &  17.73 &     99.82 &  89.46\\
miscellaneous                  &  3.71 &      24.06 &   16.17 &  19.34 &     95.00 &  80.05\\
immunizations                  &  0.05 &      63.64 &   64.62 &  64.12 &     99.96 &  97.63\\
laboratory and imaging results &  2.46 &      50.00 &   55.15 &  52.45 &     97.54 &  93.84\\
assessment                     & 14.19 &      38.09 &   42.01 &  39.96 &     82.08 &  76.89\\
diagnostics and appointments   &  2.10 &      55.60 &   40.16 &  46.63 &     98.07 &  94.22\\
prescriptions and therapeutics &  3.11 &      41.28 &   38.43 &  39.81 &     96.39 &  92.40\\
healthcare complaints          &  0.25 &      20.47 &   21.90 &  21.17 &     99.60 &  85.93\\
\bottomrule
\end{tabular}
}
    \caption{Performance of BERT-LSTM on extracting noteworthy utterances for various SOAP sections}
    \label{tab:extractor_performance_sectionwise}
\end{table*}

\subsection*{More Experimental Details}
We trained models on multiple Nvidia \texttt{Quadro RTX 8000}, \texttt{RTX 2080Ti} and \texttt{V100} GPUs.
The extractive modules were evaluated
using standard classification metrics
from \texttt{scikit-learn}~\footnote{\url{https://scikit-learn.org}}
and quality of summaries were evaluated
using ROUGE scores calculated with the \texttt{pyrouge} Python package~\footnote{\url{https://pypi.org/project/pyrouge}} which is a wrapper around the \texttt{ROUGE-1.5.5} Perl script.

redpajama_set_name: RedPajamaArXiv


text:
The Brands With The Best Global Reputations Are The Ones That Make Our Gadgets
Among the world's most valuable companies, consumer awareness is highest for tech brands like Google, Apple, Microsoft, and Samsung.
By Ariel Schwartz 2 minute Read
Google may be mired in debates over privacy, but it's still the world's top company brand, according to the latest FutureBrand Index. As a top-ranked "future brand," the company is also likely to succeed going forward because of its strong visions of purpose, user experience, and sustainability, according to the index.
Created by global brand consultancy FutureBrand, the index gauges the "brand strength" of the world's 100 most valuable companies. Its scoring is based on a survey of more than 3,000 people in 16 countries.
These are the top 20 brands:
Many of the top brands are in the tech sector–in fact, Silicon Valley has a strong presence overall in the top brand rankings. It's not too surprising to see consumer-facing brands like Google, Apple, and Disney ranking at the top, but companies like SABIC and AbbVie, which don't sell directly to the general public, somehow still manage to round out the top ranked organizations.
Here are the bottom 20 brands among the world's most valuable companies:
No big surprises here. Lots of oil companies, banks, and a couple tobacco companies wedged in for good measure.
In addition to the top 100 rankings, FutureBrand also picked out 22 companies to be labeled as future brands because of respondents' desire to buy products and work for them–a sign that they will succeed in the future. These companies include many of the top-ranked brands overall, including Google, Microsoft, Apple, Facebook, Disney, Intel, IBM, and Toyota. If these rankings are to be believed, tech companies will have the strongest brands for the foreseeable future. That should be obvious to anyone who has seen the way people clamor for jobs at places like Google and Apple.
While energy and financial organizations struggled in the rankings, FutureBrand believes that the companies that become known for their good ethics in these sectors will do well in the future. The report explains:
Those companies that can demonstrate they are leading change in their industry in relation to obvious areas like the environment and corporate social responsibility, will not only better attend to stakeholder needs, but also will differentiate from their competition. This is not least because the sectors still seem to be hampered by broadly weak or negative perceptions in the wake of the global financial crisis and concerns about the viability of our dependency on fossil fuels: hence the larger negative disparity between market cap and FutureBrand Index ranking for most of the organizations in these sectors.
Check out the full report here.
Ariel Schwartz is a Senior Editor at Co.Exist. She has contributed to SF Weekly, Popular Science, Inhabitat, Greenbiz, NBC Bay Area, GOOD Magazine and more
Ideas Newsletter


Thanks Gary, you were a great driving instructor - very patient and calm. Made me feel very comfortable behind the wheel, from my first lesson to my test day.
Every pupil currently learning to drive with Gary's Get Driving School with has FREE access to this fantastic online training. Take mock tests and practice your hazard perception. To find out more click the link belew.


M.Krueger M.Reid vs R.Berdusco M.Braswell live stream online. We provide you with High Quality Streams only, Use Google Chrome Browser for best experience. In order to start streaming select the desired stream and close all the X - Do not download or install anything.


text:
La stazione di Villabassa-Braies-Val di Braies (in tedesco Bahnhof Niederdorf-Prags) è una stazione ferroviaria posta sulla linea Fortezza-San Candido. Serve i centri abitati di Villabassa e di Braies, e il territorio della Val di Braies.

Storia
Già denominata semplicemente "Villabassa", assunse la nuova denominazione di "Villabassa-Braies-Val di Braies" il 15 giugno 2014.

Strutture e impianti

Movimento

Servizi
  Biglietteria automatica
  Sala d'attesa
  Servizi igienici
Le banchine risultano essere collegate da un sottopassaggio, accessibile anche ai diversamente abili in quanto dotato di ascensori.

Interscambi
Adiacente alla stazione è presente una fermata delle autolinee urbane e interurbane.
  Stazione taxi
  Fermata autobus

Note

Bibliografia
 Rete Ferroviaria Italiana, Fascicolo Linea 44.

Altri progetti

Villabassa
Braies
Villabassa-Braies-Val di Braies

redpajama_set_name: RedPajamaWikipedia


Fires can impact communities in a variety of forms and are among the deadliest hazards nation-wide. Specifically to Bellevue, fires can cause damage through large single structural fires, multi-structure fires, industrial fires, and vehicle fires. According to FEMA statistics, about 50% of residential fires are started by cooking, 12% by heating issues, and 6% by electrical malfunctions. Fires can be started in many other different ways including smoking, appliances, open flames, and arson.
Fire safety begins with smoke alarms and established fire escape plans. Make sure that there are smoke alarms on every level of your home and inside bedrooms. Test your smoke alarms every month to ensure that they are working and have enough battery power. Plan a fire escape plan and practice with your family, roommates or co-workers twice a year. If a fire occurs in your home, get out of the building and stay out. Do not go back inside for any reason.
While fires are still a significant danger, their impact has declined in certain areas in recent years. The amount of fires, deaths and injuries related to fires, and damage in terms of money have all noticeably dropped in numbers since 2005. These positive changes can mostly be attributed to better education on fire safety, a decrease in smoking and an increase in smoke alarms. Despite these improvements, fires are still one of the most dangerous hazards today and practicing personal fire safety remains vitally important.


This picture was taken at York Station in 1937, and shows the vehicle, in its original livery, on the 'Circle' route – 1/8, Tang Hall – Beckfield Lane – Burton Stone Lane. The vertical line in the middle of the 'bible' indicator clearly suggests that the flap would have been turned over like the page in a book rather than being raised or lowered. However, the entire 'bible' would have needed to be changed some 15 or 16 times a day on this service – something of a chore for the crews.
To my shame, I have to say that, despite some of them being in service until 1952, I have no personal recollection of the Y-WY GO5Gs; the 1939 K5Gs, (and the 1938 K5Gs, after rebodying), were regularly used on routes with which I was very familiar and I knew those buses well, but I just don't remember the GO5Gs. My only excuse is that Y-WY introduced Bristol K6Bs on the circle route – maybe in part because they had blinds – by the time I was about six years old, so the GO5G's must have been used elsewhere in the system, (e.g. taking John Whitaker to Leeds!).
Behind Y327 is Y413, an ex-West Yorkshire lowbridge Leyland TD1; opposite is Y329, another GO5G.
My thanks to Keith A Jenkinson for his kind permission to use the picture.
There is something about the character of Bristol buses…old ones anyway. Business like, efficient, and thoroughly economic, and absolutely oozing with character. This photograph just sums it up for me. My Bradford based memories are of the lowbridge variety, so it was always a pleasure for me when, as a child, we went to, or through York, as the highbridge version was a rare treat.
Thanks for the memory Roy!
West Yorkshire Bristol GO5G 306 with Eastern Counties L53R body built in 1935 is seen at Bradford Chester Street Bus Station about 1950. It is ready to depart to Baildon via Canal Road and this information is clearly displayed on its "bible" indicator. For its return journey, the conductor or driver would release the side clips on the 'bible" to enable the horizontal flap to be turned to show "Bradford." The intermediate destination points in reverse order would also be revealed on the hidden "page" after which the 'bible" was then re-clipped.
No doubt the crews welcomed the adoption by West Yorkshire of roller indicator blinds from 1944 onwards, though the operating handles were still outside and involved a clamber over a hot radiator but there was no more awkward lifting of a heavy "bible" metal sheet arrangement. I wonder what injuries occurred and whether there were any compensation claims with handling of the "bible" indicator metal sheets.
306 was withdrawn from service at the end of November 1950 but the "bible" indicators remained in use with West Yorkshire I believe in 1958 with the demise of the last pre-war Bristol L5Gs.
Can anybody confirm the date of the last pre-war L5Gs? I have only West Yorkshire fleet lists to 1955.
Thank you Richard for sharing with us this wonderfully composed and evocative picture. I have departed from that very spot countless times when I worked for West Yorkshire at Ilkley Depot. We operated services 63/68 to Ilkley and Ben Rhydding via Manningham Lane or Canal Road, and had just one weekday trip to Baildon – out via route 59 and Baildon Road and return via route 61 and Baildon Green. I never had to use the incredible and ludicrous "bible" indicators and, quite rightly, they would never have been permitted in today's Health and Safety climate – this legislation is often sneered at, and can be "over the top" on occasion but it was certainly sorely needed in those days. At a quick glance it would appear that the last prewar L5Gs were indeed withdrawn in September 1958. Of interest is the "Tilling Group" swivel radiator filler cap, retro fitted to most vehicles in place of the original flat screw down pattern. Thanks once again for a particularly splendid picture.
I know I am susceptible to nostalgia, buses being no exception, but this photo has it all for me! As a child, I remember the shape of the ECOC "G" bodies so well, as they contrasted vividly with the post war fleet. They lasted until 1952, as they had provided cover for the company's "K" type rebodying programme.
I can see them now, hurtling down Thornton Road from Denholm, leaving clouds of smoke in their wake.
What a place Chester Street was for a young enthusiast. Day trips to Baildon and Ilkley on such delights as 306, and their "J" cousins, and stopping off at Harry Ramsdens on the way back. Most of all, awareness of contrasting shape variations amongst the WY fleet. There was that green hut, full of stacks of spare bible indicators, and, as you exited the bus station, at the little parking area to the right, bus backs of differing profiles would persuade you of the delights of BOVRIL, or else pose the question, "Have you McLeaned your teeth today"?
Wonderful memories of a superb fleet in its heyday!
Can it be 60 years since the passing of these fine vehicles?
Thanks to Richard for this post…. Any more WYRC Co. enthusiasts out there?
Yes, a lovely picture with much nostalgic value. I didn't realise that 'bible' indicators were still in use as late as 1958 – they lasted until 1955 in York, where I never heard them referred to by any of the older staff as anything other than 'flap boards'. West Yorkshire must have been very fond of them because they actually converted some of the vehicles they inherited from York Corporation to bibles from blinds.
The picture indicates, and I assume from Richard's description, that the hinge was horizontal and that the flap was raised or lowered. That's how I remember them, but I have seen photos, (pre-war), clearly showing a vertical hinge where the flap was turned like a page in a book. Can anyone throw any light on this?
In addition to being heavy and awkward, a problem with 'bible' indicators arose when the vehicle operated on more than one route. It would then be necessary to carry another board, either under the staircase or, on a single-decker, lodged between the bonnet and nearside mudguard. York's Circle route, with three different destinations, would require a change of board every hour; some photographs suggest an additional slip board, just showing the change in intermediate route points, but I really can't be sure about that; however the problem was solved, it would have been a tiresome bind for the crews, no doubt.
Thank you again, Richard, for this great picture.
How did you get a 5G like this one up Browgate into Baildon? With great difficulty?
I remember the 'flap boards"on York buses: they gave the York fleet a wonderfully old-fashioned & "foreign" look that went with the rest of the place- and the white boards went with the white "lavatory brick" so often seen in York (sorry- "Yark")!
Roy you are quite right – although in the minority there were versions of these boards where the hinge was vertical and the pages turned "book fashion" – I remember them well and they were around after the War also. Additionally there were other lightweight boards with only one narrow line carrying the ultimate destination in black and the route number in red. They were shaped like a letter "H", with slender legs at either side and the narrow route line midway between bottom and top – these must have been far easier to "hoist aloft" than the full size multi-leaf monstrosities.
Joe, you'll have the Gardner legal team after you!! Slow steady progress in low gears was essential of course, but the big hearted Gardners would go anywhere you know – however Browgate did admittedly often demand first gear. I have a wonderful piece of video which I took on an enthusiasts' outing on preserved KDG 26. Knowing what was to come I positioned myself on the nearside front seat and the superb and highly competent driver chose just the right moment to execute a masterly change into first gear without the slightest jerk or click. I often wonder to this day how the GO5Gs and similar models managed to achieve Leeds to Harrogate in virtually the same running time as the present day despite having to tackle the ascent to Moortown, Harewood Bank and Almsford Bank and so forth.
Eastern Counties were also users of bible indicators although whether they lasted into the fifties I couldn't say.
Lancs United had their own variant on the bible this was the slot in stencil which was a metal stencil which was back lit. The boards were dropped into the indicator box by the conductor. On double deckers the boards were kept under the stairs and would occasionally fall over with a great clatter.
Preserved LUT Leyland Tiger 114 CTF 423 a 1938 TS7 with Roe bodywork has had the feature restored and can be seen in its restored glory at this link.
In 1934 Keighley-West Yorkshire took delivery of two Bristol G-types – K111/2 (YG 5733/4) – which were the first Bristols to be fitted with Gardner's new 5LW diesel engine. Previous G-types had been powered by Bristol's own design of 6-cylinder petrol engine, and indeed the chassis of K111/2 were originally to have been powered by this unit. However they were modified to take the Gardner engines during construction at Bristol, and were subsequently given chassis numbers GO5G.1 and GO5G.2, denoting G-type, Oil engine, 5-cylinder, Gardner. The vehicles were of further historic interest as they were West Yorkshire's first Bristols and also the first Bristols to be fitted with lowbridge bodywork (in this case 52-seaters by Brush).
Further GO5Gs were delivered to WY in 1935 (including the imposingly handsome 306 in the photo), these as mentioned having ECOC bodywork – the forerunners of what would become Eastern Coach Works. Thus started WY's long and happy relationship with the hallowed trinity of Bristol chassis, Gardner engine and ECW bodywork that lasted into the 'Bristol' Olympian era – only thwarted when Leyland short-sightedly closed the Bristol factory in 1983.
Regarding Joe's comment on the progress of 5Gs up into Baildon, I well remember a WY fitter commenting that they might have been plodders but they were unbelievably reliable and very economical. His quote was that "they might be slow but they could pull't town hall down on a pint o' diesel!" He may have had a point!
I can certainly vouch for Brendan`s comment about the pulling power of WYRC 5LW engined buses. I well remember hearing each cylinder in its separate firing as a G mounted Baildon Brow. They could have gone up a house side without stalling! I have similar memories of 5LW West Yorkshire buses on Garrowby, and Hollins Hills!
The AWW batch of 1935 was less common in Bradford, but their reversion to a 6 bay layout was more a return to norm, with the 5 bay batch being very much an exception in ECOC output, a position which continued for the rest of Lowestoft's pre war production. Strangely enough, photographs of the AWW batch seem to be difficult to find.
Several sources quote West Yorkshire, and Eastern Counties as being the only users of the so called "bible" indicator. There were detail differences between the "bibles" of the 2 operators, principally with regard to the illuminating "side lights" as used by ECOC. Of all the (superb) Tilling Group (or Ex Tilling after 1948) companies, ECOC was, perhaps the most typical. Flat terrain enabled the extensive use of 4LW as well as Gardner "5"s, and vehicle life was maximised. All in all, it was a fascinating fleet, and one which I, as a West Yorkshire fan, would love to read more about!
Nearly a year later I have just read Phil J's memories of the bare chassis arriving at ECW from Bristol Commercial Vehicles. Seeing those brave men drive off from the Works in a convoy of 5/6 at a time was a regular sight when I was young and riding my three wheeler bike to wait by the Works entrance to see them set off.
Exactly as Phil says, there was no protection of any kind except a canvas sheet in front of the pedals and that bare box to sit on..gosh it must have been horrible to travel so far..about 160/170 miles on A/B roads back then and on a writhing, floppy chassis with springs set up to take the body weight. They did indeed wear goggles and most wore "Herbert Johnson" motorcycle crash helmets and a scarf over the mouth to combat flies.
On a cold Winter's day it must have been absolute misery for them. Those big overcoats were always a golden fawn colour..until the front got plastered in dirt and oil when they became "two tone" in a rather unusual way!


text:
Home covid Dane County closes in on 3,900 total cases as updated Data Snapshot...
Dane County closes in on 3,900 total cases as updated Data Snapshot is released
Robert Chappell and Stephanie Díaz de León
Dane County confirmed 36 new coronavirus cases today, which brings the County's total to 3,899. Of those total cases, 3,057 are considered recovered.
There were only single digit growths today with numbers spread up to those in their 70s The largest growth was for those in their 20s at nine while both those age groups of 10-19 and those in their 30s grew by seven and those in their 40s grew by five. Children between the age of 0-9 grew by three as those who are in theirs 50s and 60s both grew by two and those in their 70s by one.
Dane County reported no new deaths which keeps the total death toll in the county at 34; however, total hospitalizations for the county for COVID-19 grew to 273 today. Currently, there are 24 people hospitalized with COVID-19 while 7 of those are in the ICU.
Of the 36 new cases reported today, nine new cases, or 19.57 percent of all new cases, are in people who identify as Hispanic. Hispanic people now make up 15.31 percent of all COVID-19 related cases in the County while only making up six percent of the population. There were also eight, or 17.39 percent of all new cases, attributed to those who identify as Black. They now make up 12.75 percent of all cases while only making up six percent of the population in Dane County. Those who identify as Asian also grew by two, or 4.35 percent of all new cases. Asian people make up six percent of the population in Dane County and 3.85 percent of all cases.
Yesterday, Public Health of Madison and Dane County released their updated Data Snapshot for July 11 – 24. Currently, for those dates, Dane County has averaged 63 cases per day which is down from last week's data summary of 80. However, this number is still too high for the Forward Dane reopening plan to move forward, coded as red on the data snapshot metrics. For those dates, only about 52% of those positives have been contacted within 48 hours of being tested which is up from last week's 43 percent, but this metric also remains in the red. Though last week's snapshot reported that the percentage of those who tested positive who did not know where they would have gotten COVID from was at 30 percent, this week it rose to 36% keeping this metric red as well.
According to the update information, in the previous 14 days, there were a total of 888 cases with 273, or 31 percent, between the ages of 18 and 25. Of the 888 cases, 736 have been fully interviewed with 403 or 55 percent of those noting that they were likely infected from close contact with another lab confirmed COVID-19 case.
One significant update is the percentage of positives for the County. Currently, for the two weeks between July 11 and 24,, that percentage has averaged 2.5%. As the number of tests increased dramatically the last three weeks, there was a back log of processing tests for Dane County as the focus has been on processing, identifying and assigning an ID to all tests returned (if needed or matching it to an existing ID for those who have been tested previously) while prioritizing positive cases in this process to a case manager to be contacted. This left negatives tests in a "staging" process before being added to the daily data dashboard. Public Health of Madison and Dane County has implemented a method to include those negatives that are still in "staging" to their daily data.
The updated Data Snapshot also gave us the latest information on the impact of COVID surrounding our communities of color for those two weeks as well. Those who identify as Hispanic or Latinx make up 7 percent of tests, 16 percent of cases during those weeks and 5 percent of the hospitalizations. According to the new data for July 11 – July 24, Black people were six percent of all tests, 14 percent of cases and 25 percent of hospitalizations for those dates as well. Black people make up about six percent of the population in Dane County. For those dates as well, Asian people made up four percent of tests and three percent of cases while making up seven percent of the population in Dane County.
We will have an update of today's statewide numbers later this afternoon.
Previous articleThe movement, the moment
Next articleOpinion: Defunding the Public Market is yet another divestment from communities of color


text:
Meanwhile Pandu who wanted a powerful son asked Kunti to summon Vayu, the God of the Wind. The son born was named Bheema. He was so strong that as a baby he fell down from Kunti's arms and the rock on which he fell shattered.
Pandu then sought a third son who would bring glory and asked Kunti to summon Indra, the king of the celestials. The son born was named Arjuna and his birth was celebrated by all the Gods and sages.
Pandu's second wife Madri was upset that she did not have children. Pandu requested Kunti to give her boon to Madri. She agreed and Madri summoned the Ashwini twins and had two twin children by them. They were named Nakula and Sahadeva.
The fact that he had five sons born from the celestials made Pandu happy. On one pleasant day, he saw Madri looking beautiful and decided to have a union with her. Alas! The sage's curse came true and Pandu fell dead.
After performing the funeral of Pandu, Kunti and the five children went to Hastinapura. Madri chose to enter the funeral pyre with Pandu. Bheeshma welcomed Kunti and her children.
The sons of Dhritarashtra were disturbed by the arrival of the Pandavas. Duryodhana, the eldest son was worried that Yudhishtira would take away his right to the throne. Clashes between the two cousins erupted frequently. Bheema was rough at play and used his brute strength to dominate the Kauravas, injuring them in the process.
A furious Duryodhana then decided to poison Bheema. Bheema's strength ensured that he survived. Bheeshma then got the sons of Dhritarashtra and the sons of Pandu trained in warfare and the scriptures. Their teacher was Kripa, whose sister was married to Drona, the son of the great sage Bharadwaja.
After his father's death, Drona had no money. He deeply loved his son Ashwatthama and was distressed that he could not take care of his son well. He reached out to his school friend Drupada, the king of Panchala for help. But Drupada humiliated him saying there can be no friendship between a Brahmana and a king.
Drona then managed to get celestial weapons from the great warrior-sage Parashurama. With this knowledge, he went to Hastinapura and got the job as a teacher to the princes. Drona found Arjuna to be skilled in archery and a dedicated student. He focused on making him the greatest archer in the world. This made Duryodhana furious. Another student was Karna, the son of the charioteer Athiratha.
Karna was jealous of Arjuna and this brought him close to Duryodhana. One day Drona took his students out for an expedition. A dog that accompanied was barking a lot. It came back suddenly, its mouth full of arrows. The dog was unhurt but could not bark.
The surprised entourage saw a tribal boy named Ekalavya who had performed this feat. He worshipped a statue of Drona considering it his guru, since Drona had denied him admission as student. Drona was stunned seeing his devotion but this made Arjuna angry, and he later protested that Drona had failed him in his promise of making him the greatest archer.
The next day Drona tells Ekalavya that he wanted a gurudakshina (fee paid to a teacher) from him. He asked Ekalvya to give him his right thumb. Without a second thought Ekalvaya cut his thumb off ensuring he could never wield a bow effectively.


text:
Bruno Pascua López (Santander, 20 de enero de 1990) es un futbolista español. Juega como centrocampista y actualmente milita en el Real Tomayapo de la Primera División de Bolivia.

Estadísticas

Referencias

Enlaces externos
 Ficha de Bruno Pascua en BDFA
 Ficha de Bruno Pascua en CeroaCero
 Ficha de Bruno Pascua en Soccerway
 Ficha de Bruno Pascua en Transfermarkt

Nacidos en Santander (España)
Futbolistas del Rayo Cantabria
Futbolistas de la Real Sociedad Gimnástica de Torrelavega
Futbolistas del Dunaújváros PASE
Futbolistas del UMF Grindavík
Futbolistas del Club Atlético Nacional Potosí
Futbolistas del Universitario de Sucre
Futbolistas del Ourense Club de Fútbol
Futbolistas del Club Deportivo Guabirá
Futbolistas de Cantabria