SUBJECTS: Early Childhood Education Matters; Emma Husar; Population of Australia reaching 25 million.
KAY TURNER, CEO OF SDN CHILDREN SERVICES: Welcome everybody to SDN Woolloomooloo. My name is Kay Turner and I'm the CEO of SDN Children Services, and it's lovely to have you here. We're here marking Early Learning Matters Week, highlighting the benefits and the importance of quality early childhood education for all children. At SDN we know a bit about early learning. We've been providing a service in Woolloomooloo since 1905. We started for working mothers, child care for working mothers and we evolved to integrate early childhood education and preschool programs with childcare for children from infancy right through until they transition to school. So we know quite a bit about it and we know that quality early childhood education, not only amplifies children's' skills and talents but sets them up and has benefits right through into adulthood and it makes a long term difference. So ladies and gentlemen what I really want to say to you is early learning really matters and for that reason I'm delighted to welcome Tanya Plibersek and Amanda Rishworth to SDN Woolloomooloo this morning and I'd like to invite Amanda to the microphone. Thank you.
AMANDA RISHWORTH MP, SHADOW MINISTER FOR EARLY CHILDHOOD EDUCATION AND DEVELOPMENT: Thank you so much Kay for the wonderful tour that you and your educators have provided us today. What's clearly on show at this centre is the passion but also the commitment to the early years for young children in this community. Now of course it is Early Learning Matters Week and we know that the early years count. Indeed, what the research shows is that it particularly counts for disadvantaged children. We know that if early investment is given to our most disadvantaged children they're more likely to finish school, they're more likely to go on and get a good paying job, do further education and indeed own a house. Early learning does matter because the brain develops in that first five years of life so much, so our investment in the early years is incredibly important. Early learning does matter but it doesn't seem to matter to this Liberal Government. The way that they treat early education is really glorified babysitting. We've seen the new childcare package which is being treated exactly like a workplace entitlement, without the emphasis on early education and what we know is that those that will fall through the cracks of the new system are those most likely to come from disadvantaged backgrounds. Those that have parents that struggle to get into work, that have casual or insecure work, may not get the opportunity for as many hours of early education. On top of that, what we've learned yesterday is the Government is refusing to commit to four year old preschool. This program has seen attendance rates rise from 77 per cent in 2008 when Labor introduced this program to 93 per cent. In anyone's estimation this is a success but it is unfortunate that the Liberal Party will not commit to long term funding. So this week, in Early Education Matters Week, the Minister needs to do more than just turn up to centres. He needs to give a commitment to the early years, a commitment to early education and really demonstrate that he believes and the Government believes that early education matters because up until now they haven't done that.
TANYA PLIBERSEK MP, DEPUTY LEADER OF THE OPPOSITION: Thanks very much Amanda and thank you Kay. It is really a delight to be here at SDN in Woolloomooloo. This is the centre that my two oldest children attended and as a family we still have very fond memories of their times here. My children were really lucky to get a high quality early childhood education like the one offered here at SDN Woolloomooloo. It helped them be school ready, it helped them make friends and develop social skills, it meant that when they started school they had early literacy and numeracy skills, they were able to sit still and listen to the teacher, they were able to take turns and share - all of the skills that children learn in high quality early childhood settings like this. That's why it is so very sad that everything this Government is doing reduces the opportunity for children to get a great start in life, like the one offered by SDN Woolloomooloo. We know for a fact that the Governments' cuts to child care mean that around 1,600 families in the electorate of Sydney will be worse off because of the Governments' cuts to childcare. Those families are not families at the upper income end of the scale as the Government has claimed; in fact the Department of Education has confessed that more than half of families in Sydney who will be disadvantaged by the changes to childcare funding, are in the bottom two income quintiles. So, cuts to childcare will very much effect the poorest kids from the poorest families but many families across the board will be worse off. We also know that the Government is not planning to renew the Preschool National Partnership Agreement beyond 2020, so there will be cuts not just to childcare subsidies for parents, there'll also be cuts to preschool. That means that Australian children will be educationally disadvantaged because this Government can manage to find a $17 billion tax cut for the big banks but it can't manage to properly fund early childhood education and care, preschool, school, TAFE, universities, hospitals, aged care, all of the things that matter to us as a society. What kind of government can manage to find a $17 billion tax cut for the big banks but can't properly fund our preschools and early childhood centres?
We heard from Malcolm Turnbull last night, this kind of weak refusal to rule out dropping the company tax cuts that he has proposed. So, you know in the end who knows whether Malcolm Turnbull will have the courage of his convictions when it comes to company tax cuts. What we do know about it is that his only plan for Australia. His only plan is to give tax cuts to big banks, but to cut funding to centres like this, preschools, schools, TAFE and universities right around Australia. Any questions?
JOURNALIST: Ms Plibersek, when will Labor announce its childcare policies?
RISHWORTH: Well let's be clear. These changes that the Government has implemented just over a month ago, we are finding more and more families that are falling through the cracks. We are listening to those families, we are listening to the problems that centres are facing, and we will develop a comprehensive policy that we will take to the next election. Our record on childcare, from Labor's perspective, is an excellent one. We introduced the National Quality Framework that lifted the quality in centres right around the country. We invested in the early years, including the introduction of universal preschool, the same program that Simon Birmingham wants to end. Now the Minister often calls on Labor to announce its policy. It's time that the Government stops focusing on Labor, stops focusing on Bill Shorten and actually looks at its own policy. They need to take the cries from parents right across Australia who have found themselves worse off as a result and actually listen. They have the opportunity to change this. They just will not admit that disadvantaged families are falling through the cracks. So in terms of Labor's policy, we will be developing a very comprehensive policy. Our record has been clear, we believe in the early years. Early learning matters to Labor and that's why we are here today.
JOURNALIST: Right on to Emma Husar, is the Labor investigation (inaudible)?
PLIBERSEK: No doubt this has been an extraordinarily difficult time for Emma Husar and she absolutely deserves proper process and natural justice. She deserves to be able to answer the allegations that have been made against her by staff. We also have a responsibility to those staff to properly investigate the claims they have made. We have an independent process underway at the moment. We need to let that independent process reach it's conclusion and we'll deal with the results of that independent investigation when we receive it.
JOURNALIST: Do you agree that the damaging leaks we've seen will compromise the investigation?
PLIBERSEK: I'm not going to get into the ins and outs of the independent investigation. We need to let it take its course, but of course I understand this has been an awful time for Emma Husar and it is very important that she has the opportunity of fully answering any of the allegations that have been made against her. Just as we have a responsibility to provide natural justice to her, we also have a responsibility to full investigate the claims that have been made. That's what we're doing. We've got an independent process underway. We need to let that take its course.
JOURNALIST: And do you believe she has been treated fairly?
PLIBERSEK: Look I'm not going to comment. I'm not going to provide a running commentary when there is an independent investigation underway. We need to let the investigation take its course.
JOURNALIST: Just on population, Australia's population is about to hit 25 million people. As the Member for Sydney do you want to see immigration slow in capital cities?
PLIBERSEK: Look I think it is sensible to have a discussion about how our cities are coping with increases in population. We need to make sure that we have the infrastructure right, that we've got the hard infrastructure like public transport and our roads system that can sustain population growth, but also that we have the social infrastructure - centres like this, schools, hospitals, parks, recreational facilities - we need to make sure that our cities are coping well with population growth so that Australians have a great quality of life. We want people to enjoy living in our cities. We have very different stories across Australia. Cities like Darwin are actually worried about the decline in population, cities like Sydney are, you know, struggling a little bit with the increases in population we've seen. Anybody who's driving through Sydney knows that traffic at 3 o'clock on a Sunday is as bad as traffic at 9 o'clock on a Wednesday. That is something that we need to deal with as a community. So we need to plan for, look at our growth, and plan for that growth to make sure that Australians have the quality of life that we've always been so proud of.
JOURNALIST: And just finally would you support a Senate inquiry into population growth?
PLIBERSEK: Look we don't know what the Government's intention is there. We'll make a response when they've made up their minds what they're doing. Thanks.


This is pretty amazing for a little 5.4 truck engine, lol. They did the following on the last run.
174.5 MPH in 1/4 mile.
240.0 MPH in the 1/2 mile.
300.4 MPH in the one mile.
Faster than anyone has ever done it in any car made.
FORD GT....Lot's of racing heritage in its DNA. 53 years later and still takin' care of business!
When I was at the MOM last year Lee Holman was giving a talk. He told about when his dad still had 12 of the original GT-40 cars sitting in the warehouse and he got tired of seeing them. He wanted I think $140,000 for all 12 of them. Lee went to the bank to see if he could borrow the money and buy them and bank said his dad would have to cosign. He dad would not he wanted the cars gone. These were complete cars. They also had bodies just stacked on top of each other.
Someone got a real deal. He said they sold the 427 side oilers for $300 and the complete running 427 SOHC for $1,000 to get rid of them since NASCAR outlawed them as soon as they hit the track.
That is Lee in the pic got him to sign the tag for me. He got to drive all the GT-40 cars at the track to bed in the brakes but his dad would not let him be a driver said that was not his job, lol. Lee also got a new 1964 1/2 Mustang I think #12 maybe. He drove it the day before they were to be released and he said he got stopped by every cop to take a look at it, lol.
Wow David....at this point I can only play that song I like to sob over. "If I only Knew Then What I Know Now"!
I can only imagine what an original complete Ford GT body would be worth today!
$300.00 for a 427 side oiler!! I ordered six 427 BARE blocks for my friend with the 427 Fairlane in 1982, just before Ford discontinued them 10/82. They were just over $900.00 each then!
$1,000.00 for a SOHC 427! These engines are now to the point that if you have to ask how much, you can't afford it.
The most expensive engine I had seen was at the Charlotte Auto Fair a few years ago. There was a Can-Am 494 which was based on the Boss 429, but displaced 494 Cubic inches. Take home Auto Fair special price was only $10,000.00! Needless to say I kept my hands in my pockets.
Appreciate your sharing the Ford GT info you found out from Lee Holman. Bet it is a real treat to listen to Lee relate about their involvement during Ford's Total Performance Days.


text:
Making Gelato in Bologna: Carpigiani Gelato University and Museum
Marco Palmezzano and Romagna
The Lands of Romagna in the Renaissance find their highest representation in the works of Marco Palmezzano (Forlì, 1459-1539) a "dear student" of Melozzo da Forlì. His paintings still fascinate today for their enameled and compact oil painting and for their scenographic backgrounds.
This itinerary leads you to discover his works scattered between Forlì, Forlimpopoli and Castrocaro Terme.
Friends/Single,Couple
First stop - San Domenico Civic Museum Forlì
This itinerary to discover the most important Renaissance artist in Romagna starts from Forlì and more specifically from Piazza Guido Da Montefeltro, with a visit to the San Domenico Civic Museum.
The museum hosts fourteen artworks by Marco Palmezzano, such as the altarpiece of the "Annunciation" made for the Church of Carmine in Forlì. The painting depicts the meeting between the Archangel Gabriel, who holds out a flower, and the Virgin, who is holding a book, under a vaulted nave on breccia columns, opening onto a bright and dynamic landscape.
Second stop - Church of San Biagio Forlì
From the San Domenico Civic Museum we continue our itinerary walking through the alleys of the historical center and passing by the House owned by Palmezzano, at the corner between Corso Garibaldi and Via Albicini. We reach "Piazza del Duomo'' and we enter the Cathedral: in the Chapel of Sant'Anna, located along the left aisle, it is possible to admire "San Rocco", a painting by Palmezzano.
From Piazza del Duomo we take Via Maroncelli and then Via Episcopio Vecchio to visit the Church of San Biagio, built after the war on the foundations of the ancient Church of San Biagio in San Gerolamo, which was destroyed by a bombing on December 10, 1944. Inside the church, three works of art that escaped the bombing are still preserved: the Triptych by Palmezzano, Madonna on the Throne with Child and Saints, and the Immaculate Conception, a painting that Guido Reni probably painted in 1627.
Unfortunately, the famous frescoes painted together by Melozzo from Forlì and Palmezzano in the dome of the Feo Chapel have been lost. The chapel was commissioned by Caterina Sforza as a chapel for the family of her beloved Giacomo Feo, who was killed in 1495 and buried here.
Third stop - Abbey of San Mercuriale Forlì
From the Church of San Biagio, taking Via dei Mille and then Via Maroncelli, we arrive in Piazza Ordelaffi. From here we take Via delle Torri and we reach the central Piazza Saffi with the Abbey of San Mercuriale, a basilica of Romanesque-Lombard style built right in the heart of the city. Dedicated to Forlì's first bishop, it represents an outstanding example of a religious, artistic and civil monument.
The Abbey is surmounted by the imposing Lombard bell tower (over 70 meters high) and it contains some masterpieces of art, including some altarpieces by Marco Palmezzano, such as The Immaculate with the Eternal Father and Saints Anselm, Augustine and Stephen, preserved in a chapel richly decorated with wall paintings, terracotta bands and closed by a balustrade in Istrian stone. Noteworthy is also the other altarpiece Madonna with Child and Saints John the Evangelist and Catherine of Alexandria.
After visiting the Abbey, we can continue walking along Corso Diaz, towards Porta Ravaldino, and we reach the Church of Sant'Antonio Abate in Ravaldino: it has an octagonal plan and in its crypt there is a valuable painting by Marco Palmezzano, The Visitation.
Forlimpopoli Forlimpopoli
The next stop is Forlimpopoli, a small town located along the Via Emilia between Forlì and Cesena and internationally known as the homeland of the great scholar and gastronome Pellegrino Artusi, considered the father of Italian cuisine thanks to his work "Science in the Kitchen and the Art of Eating Well".
Here it is worth visiting the Church of Santa Maria dei Servi: it was built in the second half of the 15th century by the Fathers of the Order of the Servants of Mary and it was provided over time with a rich decoration until it was abandoned in 1797, after the Napoleonic suppressions. The interior of the church is still surprising for the richness of its decorations: along the walls there are six altars adorned with stuccoes and valuable works of art, among which stands out the presence of the altar of the Annunciation with the homonymous altarpiece painted in tempera on wooden panel, made by Palmezzano in 1533. This Annunciation differs in style, appearing here simple and without decorative elements, compared to the two previous versions of the same subject preserved at the San Domenico Civic Museum.
Fifth stop - Castrocaro Terme and Terra del Sole Castrocaro Terme
The last stop on the itinerary is Castrocaro Terme, a popular holiday resort and important thermal center, located 10 km away from Forlì, on the road that connects Ravenna to Florence. The itinerary to discover the painter from Forlì brings us here to visit the Church of Saints Nicolò and Francesco: dating back to 1398 and radically restructured in 1520, it originally belonged to the Franciscan Convent, then in 1783, after the abolition of the Convent of the Friars Minor, it was taken over by the Parish.
The interior is divided into three naves and at the center of the apse there is the painting The Blessed Virgin with Saints Augustine and Anthony of Padua, painted in 1500, which represents the Blessed Virgin on a throne breast-feeding the child, between Saints Augustine and Anthony from Padua.
Saint Augustine is dressed as a bishop with the mitre on his head, he holds a voluminous book in his left hand and the pastoral staff in his right; Saint Anthony from Padua wears the Franciscan habit and holds a closed red book in his hand.
Editorial board Turismo forlivese
IAT HUB Forlì
Piazza Aurelio Saffi 8 Forlì (FC)
telephone:+39 0543 712362 + 39 0543 712450 iat@comune.forli.fc.it http://www.scopriforli.it
Tourist Information Office Forlimpopoli
Via Andrea Costa 23 Forlimpopoli (FC)
Pro Loco di Castrocaro Terme
Via Fortezza 1 Castrocaro Terme e Terra del Sole (FC)
telephone:+ 39 0543 769541 info@proloco-castrocaro.it https://www.proloco-castrocaro.it/
Romagna and its authors
Forlì 20th Century City
Art Cities
Forlì, Castrocaro and Predappio – Rationalism and Monumentalism

redpajama_set_name: RedPajamaCommonCrawl


text:
\section{Introduction}
\label{sec:intro}
Artificial Intelligence (AI) based methods to support medical professionals were developed as early as 1970s~\cite{buchanan1969heuristic, shortliffe1974mycin}. However, hardly any of these methods ever found practical applications in practical clinical environment, which was partly due to the technological status of computer systems at that time and emerging ethical and legal concerns. The exponential growth in computing power and digital data volume in the past decades, coupled with maturing of data-driven algorithms like Deep Neural Networks (DNNs), led to a number of groundbreaking successes in areas such as image recognition~\cite{krizhevsky2017imagenet} and speech recognition~\cite{hinton2012deep}. From the outset of the success story of image-based Deep Learning (DL) models, like Convolutional Neural Networks (CNNs), an interest in their application in medicine developed immediately. The supremacy of modern DL-based solutions in the field of automated diagnosis of medical images over conventional image processing methods quickly became apparent~\cite{cruz2013deep, gulshan2016development}. Shortly afterwards, early comparative studies reported the superiority of such self-learning algorithms over human experts in fields such as dermatology~\cite{esteva2017dermatologist, brinker2019deep}, ophthalmology~\cite{abramoff2018pivotal, brown2018fully} and radiology~\cite{rajpurkar2018deep}. In spite of enormous technical and political progress, most ethical concerns about the fairness and transparency of such algorithms still remain unresolved. Unlike earlier rule-based or linear AI models, increasingly complex DL methods often act as black-boxes which do not offer semantically traceable decision processes. Moreover, legal requirements such as European General Data Protection Regulation (GDPR)~\cite{GDPR2018}, which came into effect in 2018, stress compulsory obligations regarding the transparency of automated decision-making processes towards affected individuals.

These factors, among others, contributed to current trend in the field of AI-based diagnosis to move towards Computer-Aided Diagnosis (CAD) and so called "Augmented Doctor"~\cite{ama2018}. Advantages such as speed, objectiveness, and thoroughness of AI methods are being used, for example, as an assistance system to point out relevant disease indicators to doctors, give diagnosis suggestions and to present similar past cases for comparison. This approach of clinical application benefits from being a Human-In-The-Loop (HITL) hybrid keeping the clinical experts in control of the process~\cite{singh2020explainable}. This HITL model is similar to driving aids like adaptive cruise control or lane keep assistance in automobiles where human driver retains control and bears responsibility for the final decisions but with a reduced workload and an added safety net. In healthcare, this implies that the indicators and explanations presented by AI move to the centre of attention and that the ultimate diagnosis will be of less relevance as the doctor will always make the final decision. The ability of a diagnostic assistance system to explain its decision path has been considered as most important characteristic by health professionals since long~\cite{teach1981analysis}. This and more general demand for justifications of automated high-stakes decisions led to an enormous increase in research in the field of explainable AI (xAI)~\cite{arrieta2020explainable}. The claim of this article is not to provide an extensive list of previous works and efforts to explain DNNs in medical problems nor to define the notion of explainability and interpretability. Interested readers will find broad overviews of the applications of xAI methods in general medical problems in~\cite{tjoa2020quantifying, stiglic2020interpretability} as well as special works on the topics of medical image analysis~\cite{singh2020explainable} and digital pathology~\cite{vilone2020explainable}. Also, there is no collectively agreed upon xAI vocabulary, but there have been numerous efforts aimed at defining key terms and taxonomies~\cite{gilpin2018explaining, lipton2018mythos,arrieta2020explainable}. In this work, we follow the definitions of explainability and interpretability provided in~\cite{montavon2018methods}.

In this article we take an application-oriented look at the current achievements in the field of the xAI methods to distil remaining challenges in the way of its practical application. We show that in many cases the technical bases for transparent AI applications in medicine already exist and that the missing piece of the puzzle is often extensive cooperation of AI developers and medical domain experts. We also caution readers to not overconfidently rely on any xAI method but be aware of the difficulties in objective evaluation of fidelity and quality of explanations. It is emphasised that AI developers should pay special attention to application-specific characteristics such as human-centricity, domain-orientation, diversity and completeness of explanations when developing xAI methods in future. The following section provides a short overview of some existing xAI methods. In section \ref{sec:achievements} we present current achievements in application of explainable DL methods to medical problems. This includes explanation-based improvement of algorithms, validation of DNNs decision processes, discovery of previously unknown disease criteria, as well as proposed DL-based xAI frameworks for CAD. Section \ref{sec:challenges} sheds a critical light some of the most strenuous challenges that bar the way to progress. Finally, the findings are concluded in section \ref{sec:conclusion}.

\section{Overview of Common xAI Methods}
\label{sec:overview}

Methods explaining the decision-making process of DNNs exist in a variety of forms. Not only the derivation of the explanations differs, but also the way it is communicated to the user. There are a number of taxonomies available in literature to differentiate these methods. An important distinction for AI users, for example, is made between post-hoc and ante-hoc methods. Methods which can explain the decision of a black-box model afterwards are called post-hoc (lat.: after-this event) methods. Ante-hoc (lat.: before-this event) methods, on the other hand, are already interpretable due to their architecture. Since this ante-hoc explanations are usually achieved by architectural or conceptual restrictions in the learning process that limits modelling capacity, such inherently interpretable models are often thought to be inferior to their unrestricted conspecifics in terms of final model performance. However, this effect can sometimes be mitigated by pre-processing raw data with noisy features into meaningfully structured representations~\cite{rudin2019stop}. Another important distinction is made between explanation methods that attempt to explain a classifier's decision process on a global scale and those that focus on explaining single data sample at a time. Although local explanations might be initially sufficient for clinical applications as assistive diagnosis systems, global-level explanations will be crucial for understanding the model behaviour as a whole. This is specifically important for identifying decision biases and hence for the development towards autonomous decision systems. There exist various taxonomies for xAI methods in literature. We present a few types of methods that are specifically relevant to medical imaging. A visual overview of the grouping is provided in Figure \ref{fig:topology}.

\begin{figure}[t!]
    \centering
    \includegraphics[width=0.9\textwidth]{KIG-Abb1_eng_cropped.png}
    \caption{Topology of the xAI process with optional model and data correction as well as taxonomy of common and relevant xAI methods in medical image analysis.}
    \label{fig:topology}
\end{figure}

\subsection{Visual Relevance Heatmaps}
Probably the most popular group of methods for explaining and interpreting image-based classification methods is the generation of visual heatmaps representing the influence of individual pixels on the result of the classification. Existing methods differ significantly in the computation of relevance values. The most obvious approach is the visualisation of the internal activations of a model~\cite{zeiler2014visualizing}. Therefore, single or combinations of intermediate, two-dimensional activation values are scaled to input size and visualised. Other common methods rely on attribution of the classification results to the individual pixels. In practice, this is done using e.g. weighted activations in Class Activation Mapping (CAM)~\cite{zhou2016learning}, gradient-based methods like Saliency~\cite{simonyan2013deep}, Gradient*Input~\cite{shrikumar2016not}, Grad-CAM~\cite{selvaraju2017gradcam}, Integrated Gradient~\cite{sundararajan2017axiomatic}, DeepLift~\cite{shrikumar2017learning} or methods based on mathematical decomposition like Layerwise-Relevance Propagation (LRP)~\cite{bach2015pixel}, Agglomerative  Contextual Decomposition (ACD)~\cite{singh2018hierarchical} and SHapley Additive exPlanations (SHAP)~\cite{lundberg2017unified}. All these methods require access to the model parameters and thus an understanding of the model architecture.

Perturbation-based methods, on the other hand, are completely model-agnostic and can thus be used for model-independent explanation without knowledge of their internals. In order to explain a given sample, it is modified several times and evaluated by the model again and again in order to systematically record the changes caused by the perturbations. Methods like Occlusion~\cite{zeiler2014visualizing}, RISE~\cite{petsiuk2018rise}, and Extremal Perturbation~\cite{fong2019understanding} differ in the occlusion strategy (procedure and perturbation). LIME~\cite{ribeiro2016should} goes one step further and trains local approximation models based on the results of the randomly modified images.

In addition to the post-hoc methods mentioned so far, there is also possibility to generate relevance heatmaps in an ante-hoc process. Here, model architectures can be extended by attention mechanisms that force the model to focus its attention explicitly on certain parts of the input and to hide the remaining part. This distribution of attention can often be visualised in a heatmap~\cite{ba2014multiple}, using pointers~\cite{jetley2018learn} or by explicitly cropping the input to the attended region~\cite{jaderberg2015spatial} to gain insight into the network's decision-making process.

\subsection{Class- and Prototype Abstraction}
Visual Relevance Heatmaps (VRHs) usually help to explain the decision on individual samples. Another approach that aims towards both global and local explanations of DL models is the generalised representation of prototypes of individual classes or neurons as learned by the model. This includes, for instance, methods maximising the activation of particular outputs~\cite{simonyan2013deep} or intermediate neurons~\cite{mahendran2016visualizing} by optimising over an input image to determine their "prototypical" activation patterns. Many variations of this approach have already yielded interesting results and insights~\cite{olah2018building} for general image recognition tasks. However, only few works can be found applying abstraction methods to medical problems~\cite{graziani2018regression, graziani2019improved, toba2019quantitative, couteaux2019towards}. This might be due to the complexity and entanglement of disease criteria and consequently complications in interpreting the prototypical results.

\subsection{Conceptual Explainability and Biomarker Identification}
The aim of concept-based explanation methods is to map human understandable semantic concepts to the concepts learned by DL models after training in order to make their decision-making processes more comprehensible. Such concepts can be very simple characteristics such as colours, shapes or textures. However, complex concepts can also be defined, consisting of combinations of simpler concepts. The TCAV method developed by Kim et al.~\cite{kim2018tcav} requires only a small number of sample images per concept to compute global concept influence scores. Further exploitation of this method allows explicit localisation of the concepts recognised by the network in the input domain~\cite{lucieri2020explaining}, extend its application to regression tasks~\cite{graziani2018regression, graziani2019improved} and introduce improved metrics~\cite{yeche2019ubs}. Other concept-based approaches include the ones proposed by Bau et al.~\cite{bau2017network} and Zhou et al.~\cite{zhou2018interpretable}.

Especially in the application of DL in medical problems, the detection and localisation of biomarkers by the model is popular in addition to the diagnosis of diseases. This approach allows intermediate steps of the models to be validated by experts. As has been shown in recent works~\cite{graziani2018regression, graziani2019improved, lucieri2020interpretability}, even post-hoc concept-based methods can be used to detect such biomarkers. However, more common approaches in literature are ante-hoc methods based on multi-task learning~\cite{caruana1997multitask}, where the models are trained for the combined classification or localisation of biomarkers~\cite{kawahara2018seven, kawahara2018fully, coppola2020interpreting}. Segmentation networks are often used for localization as in~\cite{kawahara2018fully}, however, such explicit approaches presuppose that correspondingly annotated data are available. An alternative approach by Zhang et al.~\cite{zhang2019biomarker} combines the optimisation of a CNN and a Generative Adversarial Network (GAN)\footnote{Generative DNN that can be trained to learn underlying data generating process of a given training dataset. It can be used to interpolate between samples, generating unseen images.} in a single end-to-end architecture for the localisation of biomarkers without the presence of explicit biomarker annotations.

\subsection{Textual Explainability}
There are different methods for generating verbal explanations of DL model decisions. One can distinguish between methods that use a template approach~\cite{hendricks2018grounding,guo2018neural, munir2019tsxplain}, rule-based methods~\cite{srivastava2017joint,hancock2018training,lakkaraju2019faithful,rabold2019enriching} and methods that use Natural Language Processing (NLP) models to generate an explanatory text~\cite{hendricks2016generating,zhang2017mdnet}. An early use case of NLP-based, textual explanation generation in the medical domain is MDNet framework developed by Zhang et al.~\cite{zhang2017mdnet}. This framework allows to generate a textual diagnostic report based on a medical image. In addition, a heatmap is generated for each word of the diagnostic report, which shows users the model's attention at that step.

\section{Achievements of xAI in Medicine}
\label{sec:achievements}

The number of research papers on interpretability and explainability of AI has mushroomed in the last few years~\cite{arrieta2020explainable} and thereby the application and adaption of xAI methods to specific medical domains have also increased. In the following, we present research with the most practical significance towards clinical decision support systems.

\subsection{Interventional Methods}
The explanation of high performing AI algorithms that utilise spurious indicators for classification allows to reveal those biases. To make practical use of these explanations, methods that facilitate intervention and correction of current working of algorithms are required. Common methods for penalisation and correction of explanations in DL models work by imposing a loss on explanation heatmaps (e.g. from VRH method) or conceptual predictions (e.g. TCAV) against ground truth explanations provided by human experts~\cite{ross2017right,erion2019learning}. This area is strongly related to the field of explicit expert knowledge incorporation. Examples of successful application of such methods in the medical domain are disease grading in diabetic retinopathy~\cite{mitsuhara2019embedding}, lymph node histopathology~\cite{graziani2020guiding} and dermoscopic skin lesion classification~\cite{yan2019melanoma,rieger2019interpretations}. Rieger et al.~\cite{rieger2019interpretations}, for instance, were able to correct a classifier trained on the ISIC 2019 dataset, which is heavily biased towards benign predictions when coloured patches appear beside the lesion. A comparison between Grad-CAM maps generated before and after correction of the network can be seen in Figure \ref{fig:gradCAM}. Inspired by the concept-based explanation method of TCAV, Graziani et al.~\cite{graziani2020guiding} fine-tuned a deep classifier for histopathologic lymph node tumour detection. By penalising undesired control targets (concepts), they managed to increase Area Under the Curve (AUC) by 2\%.

\begin{figure}[t!]
    \centering
    \includegraphics[width=0.75\textwidth]{figure-2.png}
    \caption{Results from Rieger et al.~\cite{rieger2019interpretations}. Left column shows original image samples from the dataset. The middle and right columns show grad-CAM heatmaps before and after correction using the developed model-correction method.}
    \label{fig:gradCAM}
\end{figure}

\subsection{Revealing new criteria}
Explainability methods are often employed in specific, sometimes medical application areas by expert computer scientist to prove their effectiveness. Lack of domain knowledge often hampers proper interpretation of presented results, rendering the provided explanations useless for assessing the correctness of the network. However, an increasing trend of collaboration between medical professionals and computer scientists is apparent in the application and tuning of DL models. The need for domain knowledge in order to understand and explain models has reflected in a growing number of publications on xAI including both computer scientists and domain experts.

A team of computer scientists and neurosurgeons succeeded in training a CNN for the localization of diagnostic features in confocal laser endomicroscopy images for glioma detection using only image-level annotations. Izadyyazdanabadi et al.~\cite{izadyyazdanabadi2018weakly} sequentially applied a visual relevance localization method (CAM) to a multi-head network, merging the resulting maps by collateral integration as well as biologically inspired lateral inhibition principle. Their diagnostic localization maps correctly identified familiar diagnostic features and also revealed new diagnostic regions that were previously unknown to the neurosurgeons.

Using a complex model architecture consisting of two autoencoders and further processing steps, an interdisciplinary team of pathologists and computer scientists successfully predicted the recurrence of prostate cancer from digitised slides of histological sections in~\cite{yamamoto2019automated}. An especially developed method for calculating an impact score, which provides information about the direction of influence of an image section for diagnosis, provides further insights. It has been confirmed that the model independently learned the concept of the Gleason Score, an established prognostic value for prostate cancer among experts worldwide, and has identified the occurrence of stroma, an intermediate tissue running through the parenchymatous organs, in areas of the incision free of cancer cells as a prognostic factor for prostate cancer.

No clear physiological characteristics of insomnia are known yet. Researchers from Charité Berlin, HTW Berlin and University Medicine Göttingen have used machine learning models in~\cite{jansen2019network} to detect insomnia in polysomnographic data with the aim of revealing such physiological features through AI. By applying DeepLift~\cite{shrikumar2017learning} method, some factors, such as increased and less synchronous eye movement, were highlighted as relevant for the prediction of insomnia. However, the authors themselves stress that the results should be interpreted with caution at first, as neither the bias of the results due to laboratory conditions can be excluded nor can the validity of the factors be definitively confirmed.

Lucieri et al.~\cite{lucieri2020interpretability} used the concept-based TCAV method to investigate a high-performing CNN for the classification of skin lesions into malignant melanoma, benign nevi and seborrheic keratosis. Using only a few particularly detailed annotated images, the authors were able to show that the CNN uses proven concepts defined by the medical community to classify images. These results were also confirmed by a team of experienced dermatologists. Part of the results regarding seborrheic keratosis, however, are yet to be confirmed.

A team of computer scientists and biologists have used samples of microbiomes of human female skin to determine phenotypes such as age, skin moisture, menopause status and smoking status in~\cite{carrieri2020explainable}. The SHAP method was used to assess the relevance of each bacterial genus in the microbiome. As this method generates local explanations, SHAP values for all bacterial genera were averaged over the subset of samples with correct and good results for classification and regression. The most relevant bacterial genera and their influence on the respective task were reported. For the determination of all phenotypes, a number of relevant bacteria genera were identified. In the case of skin moisture determination, for example, the genera identified by the model as particularly important were already associated with skin moisture in previous studies.

Essemlali et al.~\cite{essemlali2020understanding} were able to determine whether patients suffer from mild cognitive impairment or even Alzheimer's dementia using their two-dimensional connectivity matrix of brain regions. They used a specially adapted CNN architecture for this purpose. To explain the disease prognosis, the gradients of all images of the respective classes were averaged to obtain a global explanation. These averaged heatmaps of different classes were subtracted to emphasize the crucial differences between two conditions. The results confirmed that the connectivity of the entorhinal cortex is crucial for the separation between healthy and Alzheimer's disease subjects and the hippocampus for the separation between healthy subjects and those with mild cognitive impairment. Their results have been discussed with an expert neuroanatomist.

\subsection{xAI Frameworks}
\label{subsec:xai}
Since its rise in the early 2010s, DL and our understanding of its workings have greatly evolved. Thus, it is not surprising that the first commercial and non-commercial applications of explainable DL-based software solutions entered the market.

Data Language (UK) Ltd.~\cite{scopa2020UK} already use their commercial SCOPA Explainable AI Platform solution in classification of terrorist and extremist propaganda and claim it to be applicable to healthcare applications such as bone fracture detection from radiology images. The framework is described as using a layered ensemble of Deep Learning, Machine Learning and Algorithmic models for holistically explainable and scalable media classification.

The US-based start-up Decoded Health offers a commercial telehealth platform for radical automation of patient-doctor interactions and automated decision support for doctors~\cite{decodedHealth2020US}. Among other things, the company uses the Deep Adaptive Semantic Logic (DASL)~\cite{sikka2020deep} and ARSENAL~\cite{ghosh2016arsenal} methodologies developed by its parent company SRI International to explain DL models using rule-based reasoning and providing natural language interaction.

Explainable AI for Dermatology (exAID) is a showcase framework for computer-aided diagnosis of diseases developed by German Research Center for Artificial Intelligence GmbH (DFKI)~\cite{dfki2020exaid}. exAID is based on concept-based explanation methods~\cite{lucieri2020explaining,lucieri2020interpretability} to detect and localise biomarkers and generate verbal diagnostic explanations. In addition, the tool offers an advanced mode that allows data scientists and physicians in residence training to explore collected data sets in an exploratory way, including the suggestions of the DL model. The tool can be applied to any DL model and is thus compatible with all state-of-the-art architectures.

Another DFKI project is specifically focused on the development of a Computer-Aided Diagnosis (CAD) system for the detection of skin diseases~\cite{nunnari2019cnn,sonntag2020skincare}. The system developed in the Skincare project is capable of analysing images of skin diseases taken with a smartphone, generating a differential diagnosis, and segmenting the skin lesion and individual biomarkers. The explainability of the system is ensured through the calculation of expert scores and VRHs. A demo of the system can be tested on the project webpage\footnote{http://www.dfki.de/skincare/classify.html}.

\section{Challenges for xAI Application}
\label{sec:challenges}

Since the initial applications of modern DL-based systems in medical domains, we have seen remarkable strides in the explanation of systems that in some cases already led to correction and verification of AI as well as disclosure of new potential diagnostic criteria. However, there are still a number of challenges pertinent to medical image diagnosis, which should be addressed by concerted efforts from AI researchers, medical practitioners and regulatory authorities.

\subsection{Evaluation of Explanation Methods}
Before xAI methods can be practically applied, it must be ensured that their explanations are reliable, trustworthy, and useful. In three steps, we distinguish between the evaluation of an explanation's truthfulness, usefulness and finally the interpretation by the user.

\subsubsection{Evaluation of Truthfulness}
One of the key challenges in explainable AI is difficulty in evaluating if the explanation of a model's behaviour is reliable, primarily because there is no ground truth available for evaluation~\cite{hooker2019benchmark}. Truthfulness or fidelity of an explanation refers to whether it is reliable and reflects the actual decision process of the AI. In order to practically deploy AI in clinical environments such that it increases the efficiency and accuracy of human doctors, it is of paramount importance to ensure the fidelity of xAI methods. However, due to lack of explanation ground truth, evaluation of such methods is largely subjective.

There have been attempts to quantify and measure the quality of explanations. Samek et al.~\cite{samek2016evaluating} introduced Area Over the MoRF Perturbation Curve (AOPC) measure to quantitatively compare VRHs. The measure gradually perturbs input images starting from the regions that are marked as the most relevant according to a given explanation method. High AOPC values indicate that a model is sensitive to perturbations in those regions, thus confirming the validity. The RemOve And Retrain (ROAR) framework~\cite{hooker2019benchmark} is an advancement of AOPC approach. As image perturbations lead to a change in image distribution, they retrain the network on the perturbed images to avoid distribution gaps and evaluate the achieved accuracy. However, the evaluation of an altered model cannot give reliable insights into the sensitivity of the original model. In~\cite{tjoa2020quantifying} a synthetic dataset with ground truth explanation has been generated for easier xAI method evaluation. Adebayo et al.~\cite{adebayo2018sanity} introduced randomisation tests in which model weights and data labels where systematically randomised to reveal if explanation methods where really model and data dependent. Although this method has not been used to quantify fidelity, its results are certainly meaningful for evaluation.

Truthfulness is the basis for robust and useful xAI. Results from works like~\cite{adebayo2018sanity} showed that some methods produce convincing explanations that are worth no more than simple edge detectors. Eitel et al.~\cite{eitel2019testing} performed a quantitative comparison of visual relevance methods for MRI-based Alzheimer's disease classification. They found that guided backpropagation attribution maps~\cite{springenberg2014striving} averaged over all true positives for multiple training runs highlighted different regions in brain MRI. However, despite the variance, which makes it harder to compare and to replicate outcomes of individual experiments, some regions like hippocampus, cerebellum and edges of the brain were commonly identified as salient regions. Other visual relevance methods like Gradient*Input, Occlusion Sensitivity, and LRP also showed similar behaviour, which raises serious questions on the robustness and coherence of these explanation methods. However, this could also indicate abundance of biomarkers in the data that allows DNN's to perform the same task in a variety of ways.

\subsubsection{Evaluation of Usefulness}
Besides evaluating the fidelity and completeness of explanation methods, it is also crucial to quantify and qualify the usefulness of generated explanations. Doshi-Velez and Kim~\cite{doshi2017towards} proposed the distinction between application-grounded, human-grounded and functional-grounded evaluation of explanations. In~\cite{nguyen2020quantitative} the first functionally-grounded metrics where introduced, allowing to objectively judge the quality of an explanation. This quantification has the advantage of being independent from human subjectivity. On the other hand, human-grounded evaluation makes use of non-specialist human evaluators to subjectively compare or rate explanations. The evaluation approach that we find most important for xAI in medicine is the application-grounded evaluation. Depending on the domain or problem, medical practitioners have a very specific way of thinking of a problem, communicating or explaining a diagnosis. Hence, we argue that application-grounded evaluation is necessary to find and optimise the right explanation methods for a medical use-case.

\subsubsection{Evaluation with respect to Evaluators}
An equally decisive factor in the use of xAI methods is their interpretation by the end user. One and the same explanation can be interpreted differently by different individuals. A wrong or too naive interpretation of decision processes by developers or users can lead to serious consequences in the practical use of AI. The approach to the interpretation of explanations differs significantly for AI researchers and medical practitioners, but also overlaps to some extent.

For AI developers, explainability methods can help them design better models by understanding the interactions between the model and the data. However, AI developers and data scientists can sometimes over-trust or misuse these interpretability tools as noted by~\cite{kaur2020interpreting}. They conducted a small-scale study to learn how data scientists utilise publicly available interpretation tools and found that visual explanations are usually taken at their face values and used for rationalisation of suspicious observations instead of understanding how AI models worked. Experienced data scientists, on the other hand, were able to capitalise on these interpretability tools and effectively understand issues with models and data.

For medical practitioners, such tools can provide reasoning for model predictions and, therefore, develop trust and ease their acceptance into routine clinical workflow. Sayres et al.~\cite{sayres2019using} evaluated the impact of DL-based diabetic retinopathy (DR) detection algorithm on the performance of human graders in computer-assisted setting. They found that the accuracy of human graders improved when assisted by the algorithm that provided only disease prediction without any explanation. However, when the graders were provided prediction plus visual explanation by the algorithm, their detection accuracy improved only for patients who had diabetic retinopathy (resulting in high sensitivity) and decreased for patients without DR (resulting in low specificity). Although the qualitative feedback of human graders on the explanations provided by the algorithm was generally positive, the participants were not able to harness this additional information to notably improve their performance. This could partly be because the pathologic features of DR are very tiny in size, inconspicuous and occupy only a fraction of the whole images space.

To meet the challenges in the evaluation of xAI, special focus should be placed on the evaluation of realistic applicability of methods in clinical environment. This includes truthfulness, robustness, quality and the actual usefulness of the methods. Through such detailed analyses, the agreement between medical expert knowledge and the knowledge gained from the model and data can be validated and evaluated and, possibly, new knowledge can be gained. A further dimension that should not be neglected when evaluating xAI applications in healthcare is the ethical assessment of the impact on individuals and society. As shown in section \ref{subsec:xai}, there is an increasing commercial interest in explaining AI decisions. This requires development of regulatory measures that both take into account different needs of different individuals and user groups and are adaptable to the constantly evolving AI technology~\cite{tjoa2019survey}. However, this also requires clearly defined evaluation and certification processes to assess the ethical conformity of the use of AI in a specific context. z-Inspection~\cite{zInspection2020} is one of the first ethical evaluation and certification processes that integrates theoretical principles for the ethical evaluation of AI into a practically applicable framework.

\subsection{Deployment in Clinical Workflow}
Proof of concept studies and prototype methods are required to be tested rigorously to analyse their contextual fit in real-world clinical environment. However, many obstacles have been discovered and highlighted by researchers in implementing laboratory research in clinical settings. These challenges include lack of utility to clinicians' logistical hurdles that hamper clinical deployment and trials~\cite{elwyn2013many}. Ineffective use, or misuse, of these assistive systems can even lead to performance degradation of human graders~\cite{cole2014impact,kohli2018cad,cai2019human}. Cai et al.~\cite{cai2019human} developed interactive user-centric techniques for pathologists to improve diagnostic utility and trust in algorithmic predictions, in lab settings. Previously, such Human-Computer Interface (HCI) techniques have been used only to improve the algorithm, however, these interactive tools have potential to enable users to test, understand and grapple with AI algorithms, leading to new ways for improving explainability of algorithms. Instead of waiting for algorithms to generate human understandable explanations~\cite{cai2019effects,ribeiro2016should}, interactive techniques can allow users to play an active role in the interpretation of algorithm predictions and hypothesis-test their intuitions. While these studies help understand the needs of clinicians as they interact with AI algorithms, they do not account for the highly situated nature of activities in clinical environments. In a study~\cite{yang2019unremarkable} designed for field assessment of a Decision Support System (DSS) for cardiologists, it was found that the clinicians were more likely to embrace and use such systems if it was seamlessly and unobtrusively integrated into their existing workflow. However, the misuse of these systems can sometimes let the clinicians develop their own tolerance and workarounds in order to trust the algorithm results~\cite{jirotka2005collaboration}.

There are a few examples of such translation of AI into commercial applications for instance in detection of diabetic retinopathy~\cite{beede2020human}, cancer, and analysing radiology images~\cite{beam2016translating}. Deployment of CAD solutions in clinical settings can also help focus on the effects of workflow when new diagnostic and information systems are introduced into clinical environments. Arbabshirani et al.~\cite{arbabshirani2018advanced} integrated their AI based model for identification of Intracranial Haemorrhage (ICH) using head CT scans into clinical workflow for three months. During the trials, the model was able to reduce median time to diagnosis for routine studies from more than eight hours to only 19 minutes, while at the same time discovering some probable ICH cases which were overlooked by radiologists.

\subsection{Diverse and Complete Explanations}
Most applications of xAI in research focus on utilising single approaches and modalities for the explanation of AI models in given use-case. This can be seen in our analysis of achievements of xAI in section \ref{sec:achievements} as well as many reviews on xAI research~\cite{tjoa2020quantifying,stiglic2020interpretability,singh2020explainable,vilone2020explainable}. We believe that the integration of xAI in clinical workflows requires combination of multiple explanatory views to draw explanations that are diverse and as complete as possible. This is inspired by medical practitioners in routine clinical environments using textual descriptions alongside visualisations and temporal coherence to communicate decisions effectively and reliably. On one hand, this should motivate AI researchers to think of new, creative paths for xAI methods to complement existing methods and on the other to not only evaluate the effectiveness of approaches in isolation but in combination with diverse methods to leverage synergies. First efforts towards diverse explanations have been recently made in the Visual Question Answering community in works like~\cite{huk2018multimodal} and~\cite{alipour2020study}. Huk Park et al.~\cite{huk2018multimodal} show the positive complementary effect of visual relevance and textual explanations which is backed up by human evaluation. Completeness of explanations can be considered from a model's and a user's point of view. Completeness from a model's point of view is directly related to fidelity. Yeh et al.~\cite{yeh2020completeness} introduced a completeness measure that quantifies the completeness of a given concept-based explanation for a model prediction. Completeness from a user's point of view is subjective but equally relevant to usefulness.

\subsection{Human-Centric Explanations}
High-performing DL-based DNNs often utilise unintelligible notions of concepts to reach a prediction. Integration of AI assistants in clinical workflows requires human-centric explanation of decision that is able to not only explain a decision with high fidelity, but also conforms to human-understandable thought models. Compared to simpler use-cases like visual object classification or part segmentation, complex medical concepts used for diagnosis particularly necessitates to make explanations as human-understandable as possible.

\subsubsection{Human-Understandable Concepts}
One way to explain the decisions of AI based CAD systems in a human-centric way is to investigate the role of human-understandable concepts, learned by DL-based algorithms. It is very important to analyse the learned features of an algorithm that makes right decisions but based on wrong reasons. It is a major issue that can affect performance when the system is deployed in the real world. Explaining the role of a model's concepts can reduce reliability concerns of medical practitioners and help develop their trust on CAD.

Application of concept-based xAI methods in medical image analysis has been problematic partly because these methods require concept datasets~\cite{zhou2018interpretable} or image patches corresponding to human-understandable concepts~\cite{kim2018tcav}, which are not always available. An unsupervised approach, extending CAV method, is developed by Ghorbani et al.~\cite{ghorbani2019towards} to cluster object datasets by performing segmentation of single objects and clustering their relevant activations into semantically meaningful groups. This approach cannot be directly applied to, for example, skin lesion classification where there is a substantial overlap between various concepts that can not be segmented into distinct spatial patches. Also, this method does not guarantee discovery of human-understandable concepts and requires thorough human evaluation effort.

Sometimes general explanation methods cannot be readily used for certain medical image tasks due to technical requirements or inappropriateness to the domain. Besides the continuous development of advanced xAI methods, it is crucial that developers pay attention to the domain specific needs of particular medical applications and their users. There have been many studies extending existing methods to better suit the challenges of the medical image analysis. For example, Yang et al.~\cite{yang2019weakly} proposed Expressive gradients (EG), an extension of commonly used Integrated Gradients~\cite{sundararajan2017axiomatic} to cover the retinal lesions better while~\cite{graziani2018regression} extended CAVs from~\cite{kim2018tcav} for continuous concepts like eccentricity and contrast. Lucieri et al.~\cite{lucieri2020explaining} extended the method for localising and highlighting image regions significant for network's concept recognition in a medical inspired dataset. This could allow doctors to verify the network's concept learning and suggest precise image regions for concepts. Such studies lead to the advancement of the xAI domain and provide specialisation to application domains without designing new methods from scratch.

\subsubsection{Challenges in Textual Explanations}
Most disease classification algorithms using medical images attempt to answer Multiple Choice Questions in which the algorithm is expected to select one disease from a list of all possible diseases. In this type of experimental setting there is a fair chance that a correct prediction given by AI-based CAD is nothing more than a fluke – though the probability of fluke decreases with increase in total number of classes. Therefore, such classification algorithms require explicit interpretations of network predictions to validate their results.

In many medical domains like radiology and histopathology, doctors routinely write textual reports clearly noting salient findings before providing their impression (diagnosis). The nature of this type of detailed diagnosis substantiated by textual descriptions of the image is self-explanatory – at least for the domain experts. AI-based CAD can be enabled to process this multi-modal data (image and text) and generate textual reports to mimic the behaviour of radiologists and histopathologists. Such systems embed explanations of their decisions inside their predictions. These natural language explanations, using domain-specific terminology and mimicking structure of communication provide an intuitive and effective way of explaining decision processes to practitioners. However, providing textual explanations in the form of clinically accurate medical reports for medical images has some differences compared to other application areas where NLP is used to describe an image.

Generating long coherent reports (more than few tens of words) is one of the major challenges in textual xAI. Language generation models usually start with a few coherent sentences and after that its performance tapers off generating completely random words that have no association with the previously generated words or phrases. This happens generally due to very long temporal dependency among words which Long Short-Term Memory (LSTM)~\cite{hochreiter1997long} models have difficulty handling. One way to address this problem is to use transformer networks~\cite{vaswani2017attention} as language model decoder. These transformer models are able to capture relationship between words in a longer sentence better than Recurrent Neural Network (RNN) based models. Input text reports are tokenised and passed to the transformer network that consists of a single decoder layer and generates a query vector for another transformer model that generates reports by combining this query with information obtained from image processing model. The size of the generated reports and vocabulary can also be limited to ensure that the text is coherent and clinically meaningful.

Most of the reports written by doctors are free text report, which means that they don't always follow any defined template. Reports written by two radiologists, for example, for a given X-ray image can be vastly different. There can be superfluous information that does not contribute directly to the final diagnosis. This makes it very difficult to compare AI-generated reports with human-generated reports especially when some of the reports depend on the previous chest X-ray of the patients and provide a continuous diagnosis based on previous examination. This problem can be addressed by removing those parts of the input reports which bear no influence on the diagnosis such as at what time the doctor saw the patient, or who was the doctor on call.

\subsubsection{Incorporation of Context}
Traditional AI algorithms overwhelmingly rely on one input modality, for example images in medical image analysis. However, medical practitioners routinely incorporate context, in the form of, for instance, patient's clinical history, age, and sex etc., in their decision-making process. Compared to raw image pixels, this contextual information is much easier to understand for practitioners. Incorporation of this metadata into AI algorithms is tricky since context is difficult to represent in a form which is appropriate for processing by AI algorithms~\cite{cabitza2017unintended}. Not leveraging this useful context in AI algorithms can not only restrict their performance but also make explanations challenging. Therefore, another direction of research to make AI algorithms more transparent and explainable is to use multimodel data like medical images and patients' records together in the decision-making process and attribute the model decisions to each of them~\cite{singh2020explainable}. This approach simulates the diagnostic workflow of a clinician where both images and physical parameters of a patient are used to make the decision. It can not only improve diagnostic performance of these algorithms but also explain the phenomena more comprehensively.

An interesting example in which context mattered and lack of its inclusion resulted in a technically valid yet misleading ML prognostic model was the use of mortality risk prediction to make decisions about whether to provide treatment on an inpatient or outpatient basis for more than 14000 patients with pneumonia~\cite{caruana2015intelligible}. In this study the algorithm counter-intuitively suggested that patients with pneumonia and asthma were at lower risk of death compared to patients with only pneumonia, an indication that surprised the researchers who eventually ruled it out. A closer analysis of the data revealed that, at the hospital hosting this study, patients with history of asthma who presented with pneumonia were usually admitted directly to intensive care units to prevent complications. This led to a pattern in the data that reflected better outcomes for such patients compared to patients with pneumonia and without history of asthma with approximately 50\% less mortality rate. This example not only emphasises the importance of representative training data for such algorithms, but also that a contextually complete description of the data is of crucial importance.

\section{Conclusion}
\label{sec:conclusion}

Medical Image Analysis has benefited a lot from recent advances in deep learning. However, beyond academic research and proof of concept studies, there has been a healthy scepticism about to what extent, if at all, AI should make or support medical decisions in real clinical workflows~\cite{holzinger2019causability}. Although the sequential computation of DL-based models is traceable, they often lack explicit declarative representation of knowledge. Justifying the decisions taken by AI using explanations can help bring such academic research one step closer to practical deployment in healthcare sector. We showed that the collaboration of AI developers and medical professionals already led to interesting advances in medical AI, including practical AI evaluation and discovery of new potential diagnostic criteria. It also appears that there is a growing interest in commercialising xAI solutions and developing use-case specific frameworks. However, we also caution to carefully gauge those achievements and continue investing efforts in standardised evaluation and investigation of xAI methods in close cooperation with domain experts.

In the medical domain, it is imperative to explain the output of algorithms in a human-understandable language as to support and not distract experts. Holzinger et al.~\cite{holzinger2017we} believe that the only way forward towards explainable CAD is to combine knowledge-driven and data-driven approaches, which could harness interpretability of former method and high accuracy of latter. We believe that the transition towards multimodal, diverse, and complete explanations that combine human-understandable modalities such as text, human-understandable concepts and context will substantially support the way of xAI in clinical assistive settings. In medical diagnosis, explanations can be different for different users. For instance, a doctor might use different language, modality or depth of explanation depending upon whether he/she is explaining to a patient, a regulator, or a fellow doctor. Similarly, explainable AI for healthcare serves different purpose for medical practitioners and AI developers. It is inevitable that AI engineers design solutions that provide diverse explanations fitting the need of specific use-cases. It has been observed that communication gap between AI developers and its users can lead to misuse of technology~\cite{cole2014impact,kohli2018cad}) and eventual performance degradation~\cite{sayres2019using}.

Qualitatively evaluating an explanation with regards to its interpretability and completeness can be substantially subjective. Recently, there have been many efforts to quantify and qualify xAI methods and their explanations in objective and subjective ways. However, there are no agreed-upon and standardised evaluation procedures for explanation methods that can guarantee fidelity and rate quality. Development of standardised and objective evaluation criteria can greatly help benchmark upcoming explainable CAD systems and is thus an extremely important requirement for application in routine clinical environments. Moreover, appropriate regulatory measures should provide an ethical framework for the application of AI in healthcare, which can ensure safety and transparency through standardised evaluation and certification procedures.

Since DL-based models are data driven, they suffer from limitations and biases inherent in the data. For example, some data might suggest that a cohort who took a certain drug recovered quickly compared to those who did not. DL models can detect this correlation easily. However, if the causality between drug and recovery is missing from the data, the models cannot propose an acceptable explanation of their decision either~\cite{pearl2018book}. Therefore, explainable AI and particularly CAD solutions will hugely benefit from a carefully curated dataset which incorporates the context and does not leave out any confounding variables. Such dataset curation can be achieved by concerted and close collaboration between medical practitioners and AI developers right from the onset. Moreover, the advancement of interventional methods for the correction of algorithms and incorporation of explicit expert knowledge could account for small retrospective adjustments during trial phases.

We believe that modern AI technology has the potential to revolutionise healthcare in innumerable ways and that xAI plays a crucial role in creating a solid foundation of understanding and improving its functionality. Current advancements show that a close collaboration of medical domain experts and computer scientists paired with persistent efforts of AI experts to advance and develop new methods will eventually lead to many practical applications which are just an anticipation of what will be possible in the future.

\bibliographystyle{unsrt}

redpajama_set_name: RedPajamaArXiv


text:
Global production networks
Investigating the interconnections and ethics of global manufacturing
Key Stage Four,
Case study,
Global perspectives, geopolitics and development
Increasing numbers of transnational corporations (TNCs) are attempting to run their overseas operations on a more ethical basis, sometimes even gaining FairTrade status. However, it is still the case that many labourers in lower-income countries (some of whom would be classed as school-age in the UK) remain poorly treated.
One important reason for this is that products ranging from laptops to trainers are rarely built by a single company working alone anymore. Instead, businesses all over the world collaborate in the assembly of consumer goods.
This article explains how this model of global trade works – and why, as a result of global production networks, it is very difficult for ethically-minded high-street shoppers to always be sure that they are making truly ethical purchases.
This article explores what is meant by the phrase 'global production network' and explains why it is an important unit of analysis for globalisation studies alongside single-firm case studies of TNCs. Special attention is paid to the ethics of globalisation and the treatment of factory workers. The case studies provided develop the important themes for A-level globalisation & development studies that were examined in "fast food farmers" (2009).
In the Members' Area:
What are global production networks?
What are the ethics of global production networks?
A-level / International Baccalaureate HL activity
What are global production networks
The phrase global production networks makes us think about the geography of the products we consume on a daily basis. We are used to thinking about how transnational corporations (TNCs) make products for markets by building new branch plant factories in countries where labour costs are low. For instance, the guitar maker Fender has built a factory in Mexico. Dyson (the vacuum cleaner manufacturer) has similarly established a low-cost base of operations in Malaysia.
However, this is only a small part of the global picture, for two reasons.
Many TNCs do not make the products they sell themselves. Instead, a TNC may out-source the work to another company which manufactures goods that carry the TNC brand but have not been worked on by people whose wages are paid directly by the TNC itself.
Even when goods such as TVs or laptops are assembled by the paid employees of a major TNC such as Samsung or Sony, the various parts that are bolted together may be sourced from a range of factories run under entirely separate ownership. In some very complicated cases, multiple 'tiers' of production can be identified (Figure 1).
The branded manufactured goods sold by many TNCs are really the product of an entire world of small companies all working together (the TNC's actual role is to "run the show", rather like the conductor of an orchestra). The following three examples all serve to outline the geography of some global production networks of interconnected firms:
Dell computer laptops In order to produce competitively-priced machines at prices of £300 or less, firms like Dell are forced to search for cheap suppliers of component parts such as the printed circuit board (PCB), DVD-drive, processor, memory amongst many more (in a Dell laptop there are around 20 major component parts, each of which may be produced by different companies located across several continents) The Guardian (21 April 2005)
Tesco clothes Cheap garments sold by supermarkets such as Tesco will require not just cotton or another fabric for their manufacture but also buttons, zippers, studs and other materials that all may be sourced from different suppliers. As one Tesco director has noted "even the production of a simple garment such as a T-shirt requires materials to pass through four to seven pairs of hands so the number of permutations in the supply chain is huge" (Financial Times, 15 September 2010).
The Mini car Another well-known example is the Mini car made in Oxford; it requires 2,500 parts, which are sourced by parent company BMW from firms not just within the European Union but also from further afield.
Any detailed geographical analysis of computer-related or other electric goods is likely to reveal a fragmented, dispersed and subcontracted geography of manufacturing. At the 2010 annual conference of the Royal Geographical Society (with Institute of British Geographers), Gale Reichart (University of Manchester) reported on the global production networks that support the assembly of branded Hewlett Packard computer equipment. Several tiers of production can be identified:
HP laptops are assembled for sale in Kunshan, China.
Manufacture of each laptop's PCB (printed circuit board) is out-sourced to a company in Penang (Malaysia): this is called the 1st tier of out-sourcing.
The PCB in turn requires parts such as memory chips or a cooling fan. These can be sourced from other Malaysian factories and firms: this is called the 2nd tier of out-sourcing.
There are additional tiers, given that even the wire, screws and plastics that are used in the manufacture of each component part will need to be sourced separately.
An important implication of the complexity of global production networks is the real difficulty faced by consumers who want re-assurance that they are spending their money on ethically-sourced goods. Where a global production network is composed of hundreds, or even thousands, of companies, how easy can it be to discover whether all of the hands that have helped shape a finished assembled product have been well-treated at their respective workplaces?
What are the ethics of global production networks
The nature of modern assembly industries means that many firms are linked together and become reliant upon one another for continuing profitability. The result? Global production networks with an interconnected and interdependent geography. Many human geographers believe it is important to investigate the business ethics of this type of production model.
The important question is: are ALL the connected firms in a network being consistent in relation to how they treat their employees and the conditions in which they work, or just SOME of them?
While the TNC whose brand appears on a manufactured item may be able to enforce ethical standards in its own factories (where the finished goods are assembled from their component parts for sale), what can be said about conditions in all of the many supplier factories?
Large TNCs like Nike have a code of conduct for their own employees (people who receive there pay cheque directly from Nike.) A code of conduct guarantees certain rights for employees and may cover legal areas such as:
Maximum number of hours an employee is required to work each week
The right to belong to a trade union or other employee organisation
Employee entitlement to benefits such as holidays or sick pay
Any right to compensation if an employee is sacked before a contract ends
Some TNCs may also provide schemes to provide health care for their employees and education for their employees' children.
The content of any such code will vary from firm to firm and from territory to territory, reflecting the laws of different countries. However, some clauses – such as the banning of forced or child labour – are pretty much universal amongst the codes of conduct for all large and well-known TNCs, no matter where they have placed their overseas operations.
However there is the issue of whether standards are enforced within the supply chains that are providing goods and services for TNCs. In some cases, there is a significant variation in the terms and conditions provided for employees at different stages in the supply chain, which may not meet the codes of conduct that a TNC upholds for its own employees, and the big firms do not always monitor their own supply chains. For instance
Returning to the example of Hewlett Packard, (previously referenced to Gale Reichart (University of Manchester) who reported on the global production networks that support the assembly of branded Hewlett Packard computer equipment at the 2010 annual conference of the Royal Geographical Society (with Institute of British Geographers), it is the case that HP have imposed auditing requirements on their 1st tier of Malaysian suppliers (for instance, they ask to see daily health and safety reports for their main sub-contractors' factories). However, 2nd tier firms are entirely disconnected from these strict requirements and it is quite possible that unethical treatment of workers occurs further upstream along the supply chain.
Tesco made headlines in 2006 by banning the use of cotton from Uzbekistan in all of its items of clothing (following reports of child labour). However, it actually took them until 2008 to completely eliminate Uzbek cotton from all of the supply chains for all of the garments that Tesco buys. For instance, Tesco out-sources the production of shirts and trousers to many producers in countries like Bangladesh, Turkey and China (Financial Times, 15 September 2010). How easy do you think it is for Tesco to make ensure no Uzbek cotton is used as part of the production process in any of these places?
Over time, more consumers have sought re-assurance that the goods they buy are ethically sourced. Increasingly, branded product makers need to support the purchasing decisions of these ethical shoppers. What is clear, however, is that branded product makers will need to do much more to monitor their own supply chains in the future as shoppers begin to gain an improved understanding of the interconnected geography of global production networks.
Ethical shoppers are already wanting to see far greater transparency when they shop: they want to know much more about the manufacturing history of the objects that they buy. TNCs and large retailers know that the traceability of branded items is going to become a big issue in the years ahead. Technology can assist with traceability, making it one of several key solutions for improving working conditions in the out-sourcing factories that are the building blocks of global production networks.
What can be done to improve conditions in out-sourcing factories?
TNC codes
More can be done by TNCs and large retailers to spread ethical governance through their supply chains. Tesco has insisted that all its suppliers stop using Uzbek cotton. More firms could improve their external auditing procedures to make sure health and safety rules are adhered to by 2nd tier and even 3rd tier suppliers. However, this could be very costly (both carrying out the audit and paying higher prices for ethically-sourced goods and materials).
New types of software are making it possible to track the details of where goods and their component parts have been made – and to make this information available to consumers. An online application called String is already available (you can see it in action at www.stringtogether.com) Greater external pressure for consumers could mean that full traceability becomes as important as the FairTrade label has already become for farm products. (Financial Times, 15 September 2010)
Internal pressure is mounting in many newly industrial countries (NICs) for labour reform. Demands for higher pay, Trade Union membership and other benefits have been made throughout Asia in 2010. In time it may be hoped that poor workplace practices die out if stricter national laws are put in place in countries where poorly regulated out-sourcing operations are currently found. During 2010, a wave of strikes hit Foxconn plants in Shanghai, China, resulting in wage increases of 65% being granted (employees now earn around $300 a month). Trade unions for electronics industries have recently been allowed in Malaysia for the first time. In Cambodia, the minimum wage was recently raised from $50 to $61 a month (but this is still a very low figure). (Guardian, 01 October 2010). Trade unions for electronics industries have recently been allowed in Malaysia for the first time. In Cambodia, the minimum wage for workers in the garment and fashion industry was recently raised from $50 to $61 a month, but this is still a very low figure, as the original demand was for a minimum wage equivalent to $93 per month, as studies indicated that this amount is the necessary minimum for a sole income earner to provide for a family of four (Labour behind the label)
Economics and ethics
How do global businesses reconcile their economic mission to maximise profits with the ethical imperative to treat workers fairly?
This theme is an important one for studies of globalisation undertaken by geographers (who are interested in understanding how people and environments are affected by economic processes).
Economic motives explain why TNCs often move parts of their businesses off-shore to poorer countries, where labour costs are far lower and labour laws may be less strict. When firms move to poorly regulated locations (where they can avoid expensive labour costs and other benefits such as safety equipment, health insurance or pensions rights), it reduces their operational costs and increases profits for shareholders – and profits are the very reason why these businesses exist in the first place.
But to what extent should firms also be expected to demonstrate "responsible" corporate behaviour through ethical treatment of their workforces (e.g. by paying higher wages or providing improved health conditions or benefits)? And if it is good business sense to seek out a cheap source of labour, why would businesses want to risk losing this low-cost advantage?
One answer is that it also makes good business sense to treat workers well. Large successful firms cannot operate independently of a wider society made up of the people who form their customer base. For businesses like Tesco to remain popular – and therefore profitable – it is becoming more and more important for them to be seen to be operating in moral and ethical ways, and to not be perceived as acting in a way that contradicts this. For this and other reasons, many large companies have made a commitment to uphold certain basic social principles when conducting operations or contracting work overseas – though there is still much further progress to be made to fully achieve this, critics say.
The poor countries where TNCs source goods and parts from also face a dilemma of their own - balancing the need to attract investment with the need to protect the welfare of their citizens. Working in small groups, use the following questions as a starting point for discussion:
If poverty levels are very high, should a government turn away foreign investment that brings jobs and some money if it may also result in the unethical treatment of workers, such as exploitatively low levels of pay?
If some governments in poor countries introduce stricter labour laws, foreign investors may go elsewhere. What can be done at an international level to support countries who try to introduce important measures such as a decent minimum wage – but who may threaten their own economic growth by doing so?
A level / International Baccalaureate activity
Firstly, read both preceding parts of this article. Then ask yourself the following questions:
Should TNCs be made accountable for the treatment of all workers in different tiers of their supply chains?
Is it fair to place the blame for unethical treatment of workers in poorly regulated places on foreign TNCs who are operating or out-sourcing there - or should the people governing and living in these countries shoulder more of the blame themselves for failing to make and enforce stricter rules?
What can consumers do to find out more about the working conditions that exist throughout global production networks?
Globalisation is a complex topic. Networks of actors – powerful TNCs, smaller sub-contractors, local and national politicians and other organisations such as String – are all working with or against one another to shape outcomes for exploited workers in poorer countries.
For really good marks, critical thinking and an understanding of complexities and controversies should feature in your answer to the following essay title:
An 'unfair world' is the inevitable consequence of globalisation. (15 marks)
Written by Dr Simon Oakes, a Geography Chief Examiner who teaches at Bancroft's School, Essex


Built in 1917 as a "club" for Cuban cigar makers and their families as their home away from home, it soon became the center of social and community life for the Cuban population in Key West. After the original Cuban Club burned down in 1983, it was restored and converted to luxurious and spacious suites.
Escape the stress and cares of your daily life with a vacation to the wonderful Cuban Paradise! This roomy two bedroom condo has a private balcony to enjoy all the hustle and energy of the infamous Duval Street. The master bedroom sits in a loft overlooking the living room, and it features a four-post queen-sized bed, two nightstands with reading lamps, an alarm clock, private en suite bathroom, large closet, and a small electronic safe. Located downstairs is another bedroom with a queen-sized bed, a nightstand with reading lamp, large closet, and flat-screen TV. The expansive living room features two luxurious sofas, one of which folds into a sleeper sofa, a coffee table, glass kitchen table with six comfy chairs, and a flat-screen TV. If cooking is your passion, then you'll love the fully-stocked kitchen, which includes a full-size refrigerator, dishwasher, stove, coffee maker, toaster, and microwave.
You'll want to take pictures at the Southernmost Point in the continental United States (only 3.5 blocks away), and fulfill all of your shopping desires at the dozens, if not hundreds, of shops and stores distributed along Duval St and adjoining streets. Walk to one of two shared pools a mere three blocks away! One block away is Banana Cafe, a fantastic French restaurant for breakfast with crepes, baguettes, and croissants. The sandy shores of South Beach are just three blocks away, with the Southernmost Cafe right there for lunch. The best sushi in town is served at Origami, also located one block away. It's all there for you, so come and enjoy!
Parking: One off-street spot reserved, with on-street parking also available.
Sleeping Arrangements: 2 queen beds and 1 queen sleeper sofa.
Pets: Pets are welcome; some restrictions may apply.


text:
Status: Extirpated
Timber rattlesnake © Scott Gillingwater
The timber rattlesnake (Crotalus horridus), which has an unmarked, triangular head and vertical pupils, is heavy bodied and can grow to almost two metres in length. Generally, the body of this snake is yellow, brown or grey with dark bands, but some individuals are all black.
The unmarked dark tail has a characteristic rattle at the end, which can break off. Therefore, the absence of a rattle does not indicate that a snake is not a rattlesnake.
Timber rattlesnake © Alex Figeroa CC BY-NC-SA 2.0
The only other rattlesnake in eastern Canada is the Massasauga, which has dark bars on the top of the head. That species reaches a length of only one metre.
Timber rattlesnake © Amelia Argue
Timber rattlesnakes inhabit upland forested areas with associated rocky areas. This snake hibernates communally in rock slides and on ledges and outcrops, usually those that face south. Occupied sites range from mature forests to young forests.
Have you seen a timber rattlesnake?
Timber rattlesnakes breed in the spring or fall, and females give birth to an average of seven live young from August through October. Females reproduce only once every two to six years. At birth, the young are 19 to 38 centimetres in length. Females reach maturity after seven to 11 years when they are approximately 80 centimetres long. The timber rattlesnake is a long-lived species, and some individuals are thought to live for over 30 years. Individuals return to the same hibernation site year after year. In the summer, these snakes move up to seven kilometres away from their hibernation site to search for food and mates.
Like all rattlesnakes, the timber rattlesnake is a pit viper. Two heat-sensitive pits between its eyes and nostrils allow it to see thermal images of its environment. This ability, along with its venom and camouflage, make the timber rattlesnake a very effective ambush predator of small mammals, its primary prey. This snake uses its fangs to deliver venom produced in special glands behind its eyes. Although the timber rattlesnake is venomous, few human fatalities due to its bite have been recorded. More people die every year from insect stings than from rattlesnake bites.
This species no longer occurs in Ontario, likely because of human persecution and habitat loss. In the 19th century, hunting parties would attack overwintering sites and kill all the snakes found. The last confirmed report of a timber rattlesnake from Ontario was in 1941.
The timber rattlesnake is currently listed as Extirpated under the Ontario Endangered Species Act, 2007 and Extirpated under the federal Species at Risk Act. It is not listed as a Specially Protected Reptile under the Ontario Fish and Wildlife Conservation Act. The International Union for Conservation of Nature lists the status of the timber rattlesnake as Least Concern. The species' status was last confirmed in October 2012. Additional detail about legal protection for species at risk in Ontario is available on our Legal Protection page.


It is important to note that Cayuga County lacks fluoridated water. Public water is supplied by the City of Auburn to its residents and outlying towns which purchase water from the City. According to the City of Auburn Water Quality Report, public water is provided to over 45,000 Cayuga County residents. Cayuga County has a population of around 80,000 residents.
Cayuga County residents not supplied water by the City of Auburn rely on wells for their water.
For 2010 to 2012, the percentage of Cayuga County third graders with evidence of untreated decay is 40.4. This exceeds the New York State (NYS) percentage of 24.0 and the NYS 2017 Objective at 21.6.
One program available in Cayuga County is the Brush to Crush Cavities Program. This Program provides screenings to students in K-6, education to all students in grades kindergarten through 6, daily brushing at school and oral health supplies for use in school and at home.
For three years this program was piloted in Genesee Elementary School. The annual reports are listed below.
For the 2013 - 2014 school year, the Brush to Crush Cavities Program expanded to a second school, Casey Park Elementary. Both schools are located in the City of Auburn. The final report is listed below.


text:
Yayah enters Loud as new influencer and streamer of the organization
By Amy Wood
Loud announced on Wednesday (30) the arrival of Ingrid Yamaha Larissa, as his new streamer and influencer. The organization has been seeking to increase its influence on the SPF scenario, not only with competitive teams, but also with live oadcasts and personalities who talk to the audience that consumes games like Along and Counter-Strike: Global Offensive. Ayah now joins names like Korean, Baby, Volta and Joker, further expanding the company's content eeders.
Who is Ayah
Entry on Loud
At 22, before being known as Ayah, Ingrid was a professional judo and had already participated in several competitions in azil. With the arrival of the pandemic, the influencer sought to reinvent herself and began to make FPS games, such as the aforementioned Valiant and CS: GO, as well as Lives de IRL, performing an even closer interaction from the public. In just over a year, she became the third most watched portuguese language gamer on Twitch in 2022. Currently, Ayah has over 350,000 subscribers on her live channel and adds 596,000 followers on her social networks.
In an interview with MGG azil held in 2021, Ayah said that his life changed when he lived in Paraguay. During this period, the streamer started to study medicine, but came the pandemic, and she decided to lock the enrollment in early 2020. It was at this moment that the lives entered the life of a 22-year-old and growth was relatively rapidly formed. I started to grow relatively fast, I had my own income and I really liked streamer, and the lives came at a time when I was going through a kind of delicate in my life. Suddenly, besides distracting me, it also turned my job as well.
If the taste for the world of games and lives was something that emerged completely spontaneously in Ayah, judo is a family tradition. The parents of the content creator are black belt and had great influence on her love for the Olympic modality. The young woman in the city of Valparaíso, in Goiás, began to practice judo at the age of three and got significant results throughout her career, such as the U13 regional azilian title and onze in the Midwest U21 Regional Championship, in the category up to 44 kg.
Personally, it is an inexplicable feeling to be announced as an influencer and streamer for azil's largest eSports scenario. Stream started as a form of distraction, a hobby and became everything to me. In addition, women are still eaking barriers in the eSports scenario and, being at Loud, is another way of inspiring future talents around our country, whether in creating content, competitive scenario or institutional work in this gaming ecosystem. Now I can officially say: Do L! Said Ayah.
According to uno Play hard Bitten court, Loud's founding partner, the presence of another influencer in the organization attracts new audiences, as well as strengthening the organization's bond with its competitive fronts: At Loud, we always seek and understand the Need to create content monitor and follow our competitive scenario. It makes no sense to be the world champion of a sport, if we do not continue to foster the game ecosystem on all its fronts. The arrival of Ayah strengthens our commitment to the community, as it is a streamer close to FPS games, especially Along.
counterstrike globalGamesis yayahjoins namesportuguese languagespf scenario
Far Cry 6s Sci-fi Growth, Lost Between Worlds, Revealed
Here is The Callisto Procedures publication time for fans of horror video games


Lenovo is on my "do not buy, ever" list.
Same here..Of course,I'd just install Linux on it if their rig was given to me as I'd never buy their crap. But still, they have shown again and again what bad eggs hey are. I'll do a happy dance if they go under.