text:
\section{Introduction}

The web has proven a powerful platform for the large-scale distribution of content.
Notoriously difficult to regulate, individual web organisations have been frequently left to decide how to best handle issues related to legal, regulatory and even moral matters, \emph{e.g., }\xspace moderation of online discourse, removal of copyright infringing content, mitigation of online harassment.
This has generated significant societal attention, with major companies like Google and Twitter coming under increasing public scrutiny~\cite{giannoumis2014regulating,stiegler2013regulating}.
Consequently, large web organisations have begun to implement bespoke mechanisms to allow third parties to register complaints. For example, Google's complaint system allows anybody to issue notices requesting the removal of specific results from their search listings, whereas Twitter enables users to report posts they believe to be infringing policy (\emph{e.g., }\xspace because of bot activity~\cite{gilani2017bots}).
Although a valuable tool in the wider landscape of web governance, this places considerable responsibility within the hands of these organisations, who must decide which complaints are legitimate and how they should be dealt with (often referred to as self regulation~\cite{urist2006s}). Yet, to date, we have little evidence regarding how these complaint procedures are handled, how successful they are, or who they are targeted against and by whom.

We argue that the scale of these complaints and their impact on the wider society's perception of the web, warrant detailed investigation.
There are multiple interesting questions, such as who generates complaints? Whom do these complaints pertain to? What are the characteristics of  domains that receive complaints? Does content remain online after complaints?
With these questions in mind, this paper presents a \emph{large-scale study of web complaints}.
Building on past work within the legal domain~\cite{Urban2016,Urban2005,Seng2014}, we have gathered complaints from hundreds of transparency reports made available by organisations including Google, Vimeo, Bing and Twitter, and reflected in the Lumen database (\S\ref{sec:Dataset}). These reports expose detailed information about web complaints  covering over 1 billion URLs ranging from copyright infringement to governmental notices.

We start our analysis by characterising the nature and scale of organisations that generate notices (complainants).
Despite the presence of numerous complaint categories, copyright notices largely dominate, with 98.6\% of all complaints (\S\ref{sub:what_is_injected}).
A critical minority of organisations play a remarkably prominent role in this, with the top 10 alone contributing 41\% of all notices sent (\S\ref{sub:who_injects}).
Our analysis reveals that the majority of \emph{notices} are generated by large content-based organisations (\emph{e.g., }\xspace NBC, Fox). Despite this, we find that the majority of notice senders are occasional users of the complaint system: 94\% of complainants issue fewer than 100 notices.
These different groups tend to rely on different types of complaints.
For example, large copyright enforcers (\emph{e.g., }\xspace Rivendell) generate millions of copyright notices, whereas governmental agencies (\emph{e.g., }\xspace Roskomnadzor) issue a far smaller set of targeted court and governmental notices. This leads us to focus on the categories of websites that these different types of notices pertain to (\S\ref{sub:topics}).
We find that notices are heavily biased towards certain types of website. For example, websites categorised as `File Sharing' and `Elevated Exposure' are hugely over-represented amongst our complaint dataset. Similarly, individual notice senders tend to target specific topics, \emph{e.g., }\xspace 60\% of complaints by \textit{Cam Model Protection} target adult websites, whereas 55\% by \textit{NBCUniversal} target Entertainment, shedding light on the priorities of organisations utilising the complaint services.

This leads us to investigate if equivalent dynamism exists on the part of the reported websites themselves (\S\ref{sec:characterising_reported_domains}). We find that a small (and constantly evolving) set
of websites dominate the ranking of the most reported, with a few domains that remain prominent throughout our entire measurement period: the top 1\% of domains accumulate 63\% of complaints (\S\ref{sub:hot_domains}). These include many obscure websites, which are unlikely to be widely known, \emph{e.g., }\xspace 22 out of the top 30 most frequently reported websites are not even in the Alexa Top 100K, and the correlation between the popularity of domains in terms of notices and Alexa rank is just 0.13.
Deeper analysis reveals that these trends are dictated by the activity of a small set of \emph{extremely} aggressive notice senders (\S\ref{sub:domain_stability}), whose `bursty' behaviour creates high levels of instability.

We finally inspect the availability of the webpages that are complained about, with the conjecture that websites receiving many complaints may be more likely to go offline (\S\ref{sub:target_behaviour}). We confirm that many are highly ephemeral: 22\% of all domain names soon get taken offline (NXDOMAIN), whereas our HTTP liveness checks show that a further 19\% of all resources fail to return 200 OK responses within 1 week of a complaint. We correlate these with a number of factors to find that the `success' rate of complainants differs dramatically, with the most successful (Rivendell) seeing 55\% of its complaints acted upon in contrast to others complainants where the figure is below 1\%.
The published version of this report is available at~\cite{ibosiola2019watches}.

\section{Methodology \& Dataset}\label{sec:Dataset}

This section presents our data collection methodology. This has two goals: ({\em i}\/)~To provide vantage into a broad set of web complaints, covering enough domains to provide meaningful insight; and ({\em ii}\/)~To annotate these complaints with sufficient metadata to shed light on domain activities and topics.

\subsection{Website Complaints}

First, we detail our methodology for gathering complaints issued about websites. Naturally, it is impossible to get complaints issued to \emph{all} websites, because the vast majority do not make this information available. Hence, we focus on sites that expose transparency reports, \emph{e.g., }\xspace Bing, Twitter, Google, Vimeo. These reports list complaints received by the organisations, including relevant metadata. To gather these, we utilise Lumen\footnote{\url{https://lumendatabase.org/}} --- a database run by the Berkman Klein Center for Internet \& Society. It aggregates and publishes transparency report data pertaining to complaints issued towards 223 organisations. The exact purpose of each complaint differs, \emph{e.g., }\xspace a complaint to Bing will normally request the removal of search results, whilst complaints to Vimeo will concern the removal of videos. However, each notice (\emph{i.e.,}\xspace complaint) includes the following standard fields:

\begin{itemize}
\item \emph{Notice Type:} the category of notice which has been reported. For example, a Digital Millennium Copyright Act (DMCA), trademark or data protection \emph{etc.}\xspace

\item \emph{Notice Sender:} or complainant, is the organisation who submitted the notice.

\item \emph{Notice Recipient:} the web publisher or service provider where the infringing notice is sent to.

\item \emph{Notice Principal:} in cases of a copyright-related notice, this is the person or organisation that holds the copyright on the content reported.

\item \emph{Infringing URL(s):} the list of URL(s) that the notice sender is requesting to be dealt with. Note that this is not necessarily a set of URLs owned by the recipient, \emph{e.g., }\xspace Bing may receive complaints requesting the removal of a third-party URL from its search results.

\end{itemize}

\noindent We collected all complaints from the Lumen database between 01/01/2017 and 31/12/2017 using their API.
In total, we extract 1,054,248,823~URL complaints from 38,523~ notice senders.

\subsection{Website Metadata}
Once we extract the reported URLs, we compile further metadata. This section explains the metadata collected.

\vspace{4pt}
\noindent\textbf{Website Categorisation.} We classify each domain using the VirusTotal API.\footnote{\url{https://www.virustotal.com/}}
This API has been used in a wide set of research, and is known to provide good accuracy~\cite{kim2015detecting,wang2017droid,ikram2019chain}.
The API provides a classification for each domain in our dataset, \emph{e.g., }\xspace games, education, file sharing, blogs \emph{etc.}\xspace We later use this to understand the types of complaints generated.
Due to the usage limitations of the API, we only categorise the top 240K domains with the most reported URLs. Note that 22\% of these could not be categorised by VirusTotal. We further annotate each domain with its Alexa ranking to gain insight into its global popularity.

\vspace{4pt}
\noindent\textbf{DNS Probes.} We utilise the Domain Name Service (DNS) to map the domains to their respective DNS records on 29/07/2018. We performed queries (IPv4), which yield 849,023 responses and 206,863 IP addresses.
We use this data to check if the domain name is still live.

\vspace{4pt}
\noindent\textbf{Webpage Probes.} For each URL, we download its HTML and parse it to extract all embedded domains. This allows us to identify mirrors of websites hosted on multiple domains/URLs, by comparing the HTML contents. Tests are performed from a university campus, where we have confirmed no web filtering is performed.
In total, we collect the HTML structure for 770,737 webpages. \gareth{Is this because we don't have the full 1M scrapes yet, or something different?}

\vspace{4pt}
\noindent\textbf{Liveness Checks.} We also perform periodic checks on the domains and URLs to verify if the websites are still active (\emph{i.e.,}\xspace returning an HTTP 200 status code).
This allows us to explore the potential efficacy of organisations seeking to remove content.
Due to the sheer scale of the complaints ($>$1 billion URLs), we only perform checks for 2M URLs complained about between 14/07/2018 to 17/07/2018.
Upon recording a complaint from Lumen, we added its URL to a queue and began weekly checks that executed between 18/07/2018 to 14/08/2018. We record the HTML response and HTTP status code, alongside whether or not the TCP handshake timed out. Each week, we exclude URLs that have already been deleted.

\subsection{Limitations \& Ethical Considerations}

We emphasise that Lumen only provides vantage into 223 complaint recipients, obviously a small fraction of the world's online organisations.
These consist primarily of: \emph{Google}---79\%, \emph{Bing}--15.6\%, \emph{Twitter}--3.8\%, \emph{Periscope}--0.8\% and \emph{Vimeo}--0.4\%.
We do not assert that our findings generalise beyond these organisations, although the scale of these five companies indicates that the insights are highly valuable nevertheless.
We should also highlight that our research covers websites that may participate in illegal activity (\emph{e.g., }\xspace copyright infringement). We restrict this collection to downloading the webpage HTML and checking server liveness, ensuring that \emph{no} measurements involve participating in a websites activities, \emph{e.g., }\xspace registering user accounts or downloading content.
Note that this may still result in advertisement revenue being generated by the website. Unfortunately, this was impossible to avoid considering the nature of our measurements. That said, we limit ourselves to accessing URLs a small number of times ($<10$), minimising any potential revenue.

\section{complaints \& Notice Senders}
\label{sec:characterising_complaint}

In this section, we investigate the senders and receivers of complaints. Specifically, we are interested in exploring \emph{who} sends complaints and \emph{what} they pertain to.

\subsection{What Types of Complaints Exist?}
\label{sub:what_is_injected}

We identify complaints from 38,523 unique senders, covering 1.05 Billion URLs, which are hosted across 1,090,173~domains. Individual complaints tend to contain multiple URLs, with an average of 560 URLs and 31 domains per notice. In terms of the  \emph{types} of notices, there is a remarkable skew towards DMCA complaints.
This is partly driven by the prominence of search engine recipients within our data.
Table~\ref{tbl:dataset} presents a breakdown of the types of complaints across the entire dataset. We find that DMCA notices make up 98.6\%  (1.05B URLs) of the dataset and a similar share of domains (97.8\%).
DMCA notices are a US-enforceable complaint which covers takedown notices for (allegedly) copyright infringing content.
The senders largely appear to be third party organisations who act on behalf of the actual copyright owners: just 9\% of notice senders listed are also the principal. This contrasts with a 2006 study~\cite{Urban2005} where 98.5\% of notices were sent by right owners, suggesting a shift in behaviour (as seen in~\cite{Urban2016}).
Measured by frequency, DMCA notices are followed by Defamation (52K), Court Orders (29K) and Government Requests (2.7K), covering nearly a third of the recipients (31\%). We also observe a number of less popular complaint types, such as Law Enforcement Requests, Data Protection and Trademark infringements. Whereas these make up  less than 0.001\% of the dataset, they cover more than 20\% of the recipients.

\begin{table}[t]
\centering
\resizebox{\columnwidth}{!}
\begin{tabular}{@{}llllll@{}}
\toprule
\textbf{\begin{tabular}[c]{@{}l@{}}Notice\\ Type\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\% of \\ notices\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\% of\\ URLs\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\% of\\ senders\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\% of\\ principals\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\% of\\ domains\end{tabular}} \\ \midrule
DMCA                                                           & 98.6                                                              & 99.99                                                         & 94.46                                                            & 99.87                                                               & 97.78                                                              \\
Defamation                                                     & 0.95                                                              & \textless 0.01                                                & 0.15                                                             & \textless 0.01                                                      & 1.50                                                              \\
Court Order                                                     & 0.19                                                              & \textless 0.01                                                & 4.87                                                             & 0.07                                                                & 0.41                                                               \\
Government Request                                              & 0.15                                                              & \textless 0.01                                                & 0.15                                                             & 0.02                                                                & 0.13                                                               \\

Private Information                                             & \textless 0.01                                                    & \textless 0.01                                                & 0.071                                                            & -                                                                   & 0.003                                                              \\
Data Protection                                                 & \textless 0.01                                                    & \textless 0.01                                                & -                                                                & -                                                                   & <0.001                                                               \\
Law Enforcement Request                                          & \textless 0.01                                                    & \textless 0.01                                                & 0.02                                                             & 0.03                                                                & <0.001                                                               \\
Trademark                                                      & \textless 0.01                                                    & \textless 0.01                                                & 0.02                                                             & -                                                                   & <0.001                                                               \\
Other                                                          & 0.08                                                              & \textless 0.01                                                & 0.26                                                             & \textless 0.01                                                      & 0.16                                                               \\ \hline
Total                                                          & 1,885,267                                                         & 1,054,248,823                                                 & 38,523                                                           & 20,686                                                              & 1,090,173                                                                \\ \bottomrule
\end{tabular}%
}
\caption{Dataset summary with the percentages of notice types, and the corresponding share of URLs, senders, principals and domains.}
\label{tbl:dataset}
\end{table}

\subsection{Who Are the Notice Senders?}
\label{sub:who_injects}

The previous section suggests that notice senders are more diverse than simply the owners of copyright material, and that different reporting practices coexist.
To shed light on these aspects, we next inspect the entities behind the notices submitted. We find that the distribution of notices is \emph{highly} skewed towards a few extremely active senders.
The top 10\% of notice senders report over 1 billion URLs, in stark contrast to just 550K by the bottom 90\%.

\begin{figure} [t]
\includegraphics[width= 8.5cm]{hist_sender_urls}
\caption{Number of  reported URLs per complainant. The X-axis present each complainant, sorted by the two metrics: ({\em i}\/)~number of days a complainant reports on; and ({\em ii}\/)~notices generated by each complainant.  \gareth{If possible update colours}
}
\label{fig:hist_sender_urls}
\end{figure}

Figure~\ref{fig:hist_sender_urls} presents the number of reported URLs from each notice sender. The X-axis is sorted by two metrics: ({\em i}\/)~the number of days that a sender generates notices on; and ({\em ii}\/)~the number of notices sent (note that Y-axis is in log scale).
The distribution is highly unbalanced, with a large majority  of notice senders (94\%) issuing fewer than 100 complaints.
The figure reveals two broad categories of complainants: ({\em i}\/)~\emph{active}, who send complaints on multiple days and ({\em ii}\/)~\emph{occasional}, only generating notices on a single day.
Whilst the \emph{active} group represents just 25\% of all complainants, it is responsible for almost all notices (99.92\%). In contrast, the \emph{occasional} senders, consisting of the remaining 75\%, collectively contribute just 0.08\% of the total number of complaints.
It can also be observed that the curve in Figure~\ref{fig:hist_sender_urls}  is not monotonic. This is because the number of notices issued each day can vary.
Whereas occasional senders, by definition, \emph{only} issue complaints on a single date, some send multiple notices. Although the daily average is just 1.4 notices, there are some occasional senders who send a large number of complaints in a single burst. For example, Idreto (an occasional sender) generated 350K notices on a single day. As a result, even occasional senders can have a significant impact on the overall ranking of reported domains.

We now inspect the types of complaints generated by these two groups of notice senders. Table~\ref{tbl:top_10_notice_senders} presents the top 10 complainants.
Similar to the findings of \cite{Seng2014,Urban2016}, we find that \emph{active} senders are dominated by copyright enforcement and trade organisations \emph{e.g., }\xspace the British Phonographic Industry (BPI), Apdif Brasil, Apdif Mexico, \emph{etc.}\xspace
These organisations represent hundreds of music recording companies in their respective jurisdictions. Also within this group are copyright protection agencies such as Muso, Aiplex, Mark Monitor and Entura who specialise in tracking pirated content. These companies aggregate complaints from many different copyright holders, and act as enforcers on their behalf. This partly explains their broad coverage and ability to produce large volumes of complaints. In contrast, \emph{occasional} senders are predominantly made up of small organisations and private entities. We see that the categories of \textit{Private Information} and \textit{Trademark} are particularly dominated by \emph{occasional} notice senders, making up 94\% and 70\% respectively. Conversely, \emph{active} senders hold sway among other notice types, \emph{i.e.,}\xspace DMCA. These striking differences between the categories of sender highly differing strategies based on the nature of the complaints, with the ability of a small hub of complainants to dominate the wider system.

\begin{table}[]
\centering
\resizebox{\columnwidth}{!}{%
\begin{tabular}{@{}lllll@{}}
\toprule
\textbf{\begin{tabular}[c]{@{}l@{}}Notice\\ Sender\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\% of\\ reported URLs\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\% of \\ notice\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\% of\\ reported domains\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\# of \\ reported days\end{tabular}} \\ \midrule
Rivendell                                               & 13.17                                                         & 1.65                                                    & 4.97                                                             & 357                                                            \\
Aiplex                                                  & 9.76                                                          & 1.88                                                    & 1.30                                                             & 364                                                            \\
BPI                                                     & 8.60                                                          & 2.52                                                    & 2.72                                                             & 355                                                            \\
Apdif Mexico                                            & 8.52                                                          & 0.55                                                    & 0.16                                                             & 208                                                            \\
Mg Premium                                              & 7.77                                                          & 0.47                                                    & 0.49                                                             & 341                                                            \\
Apdif Brasil                                            & 7.56                                                          & 0.52                                                    & 0.39                                                             & 244                                                            \\
Remove Your Media                                       & 7.28                                                          & 0.92                                                    & 2.45                                                             & 346                                                            \\
Mark Monitor                                            & 5.29                                                          & 1.13                                                    & 5.12                                                             & 365                                                            \\
Fox Group Legal                                         & 4.36                                                          & 0.14                                                    & 2.26                                                             & 355                                                            \\
Protek Media, S.C.                                      & 4.20                                                          & 0.31                                                    & 0.67                                                             & 365                                                            \\ \midrule
Total                                                   & 806,358,505                                                   & 188,158                                                 & 224,076                                                          &                                                                \\ \bottomrule
\end{tabular}%
}
\caption{Top 10 complainants (by \# of reported URLs).}
\label{tbl:top_10_notice_senders}
\end{table}

\subsection{What Topics Do Complaints Target?}
\label{sub:topics}

\begin{figure} [t]
\includegraphics[width=8.5cm]{domain_vt_classification_alexa}
\caption{Percentage of reported URLs, domains and complainants for the 20 most reported domain categories \emph{vs.}\xspace the share of Alexa Top 100K  per category.}
\label{fig:domain_vt_classification_alexa}
\end{figure}

To better understand the main drivers behind the complaints,
we investigate which categories of websites notices senders complain about.
Figure~\ref{fig:domain_vt_classification_alexa} presents the percentage breakdown of complaints, domains and complainants   based on the reported domain category. Although it is clear that these classifications contain noise, we believe they offer useful insight into wider activity.
To better understand how this relates to the general web ecosystem, we also depict the distribution of the Alexa Top 100K websites.

There is a  remarkable divergence between our dataset and Alexa, highlighting a clear bias towards complaints for certain types of websites.
The largest fraction of domains are categorised as business (for both Alexa and Lumen), which includes websites such as \texttt{4shared.com}, \texttt{mangapark.me} and \texttt{gorillavid.in}.\footnote{We note that this category covers a number of business facing websites, including those engaged in Hollywood copyright theft.}
The most over-represented categories in the Lumen dataset are
Elevated Exposure\footnote{This category refers to malicious websites that camouflage their true nature} (8\% \emph{vs.}\xspace 0.3\% for Alexa),
Entertainment (13\% \emph{vs.}\xspace 3\%),
File Download (2\% \emph{vs.}\xspace 0.6\%),
File Sharing (2\% \emph{vs.}\xspace 0.6\%),
Blogs (8\% \emph{vs.}\xspace 6\%),
and Adult Content (9\% \emph{vs.}\xspace 7\%).
While File downloading and sharing are expected, the other ones are less intuitive.
In contrast, categories that are under-represented include Education (2\% for Lumen \emph{vs.}\xspace 12\% for Alexa) and Marketing (1\% \emph{vs.}\xspace 9\%).

These trends suggest that notice senders tend to focus on individual categories.
To explore this,  Figure~\ref{fig:percent_dist_category} presents the fraction of complaints that target each of these top 20 categories for the  top 30 complainants.
In-line with our conjecture, several complainants exhibit significant bias towards a single category. For example, 42\% of MG Premium's complaints target adult websites, 45\% of NBCUniversal's focus on Entertainment, and 34\% of Rico's is Business, demonstrating the prevalence of a high degree of specialisation.
Considering the practice of using web crawling to collect links~\cite{Urban2016}, this specialisation makes sense as it allows individual organisations to streamline their activities (based on which sites they have crawlers for).

Briefly, we also note that these categories tend to attract different \emph{types} of notices too. For example, we find that the majority of Data Protection (60\%) notices are logged against adult domains. This aligns with the well known high litigiousness of the adult content industry~\cite{sag2014copyright}.
In contrast, 10\% of complaints catagorised as Defamation pertain to news. The remaining notice types (DMCA, Government, and Private Information) are typically made about business domains, representing 31\%, 33\% and 60\% for each category respectively; with the exception of Court Order notices where most complaints are regarding shopping domains (27\%). These clear trends confirm that the majority of notice senders are quite focused in their activities, with clear specialisation.

\begin{figure} [t]
\includegraphics[width=8.5cm]{percent_dist_category}
\caption{Distribution of (top 20) domain categories (in \%) reported by (top 30) complainants. \gareth{Recompute based on how over represented domains are in terms of complaints vs overall population.\ignacio{Cosmetic:`complainant':`Complainant', `domain category': `Domain category'}}}
\label{fig:percent_dist_category}
\end{figure}

\section{Websites \& Webpages}
\label{sec:characterising_reported_domains}

Whereas the previous section explored the complaints and complainants, we next inspect the websites (domains) and webpages (URLs) which the complaints pertain to.
Specifically, we are interested in understanding which websites gain most attention, and their availability.

\subsection{How `Hot' Are Websites?}
\label{sub:hot_domains}

\begin{table*}[]
\resizebox{\linewidth}{!}{%
\begin{tabular}{@{}llllllll@{}}
\toprule
\textbf{Domain}           & \textbf{\begin{tabular}[c]{@{}l@{}}\# of \\ TLDs\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\# of times in \\ top 10\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}Domain \\ category\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\# of \\ complainants\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}Major complainant (s)\\ (\% of complaints)\end{tabular} }                                                                    & \textbf{\begin{tabular}[c]{@{}l@{}}\# of days \\ reported\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}Alexa\\ Rank\end{tabular}} \\ \midrule
mp3toys.xyz      & 10                                                    & 3                                                                & elevated exposure                                            & 10                                                           & Apdif Brasil (99.9\%)                                                                                & 109                                                            & -                                                    \\
4shared.com      & 15                                                    & 4                                                                & filesharing                                                  & 40                                                           & Apdif Brasil (99.8\%)                                                                                & 246                                                            & -                                                    \\
googlevideo.com  & 1                                                     & 7                                                                & entertainment                                                & 84                                                           & Comeso (78.2\%), Remove Your Media (17.3\%)                                                                                & 239                                                            & -                                                    \\
mangapark.me     & 3                                                     & 4                                                                & business                                                     & 18                                                           & Remove Your Media (99.1\%)                                                                                & 28                                                             & 3,901                                                \\
gorillavid.in    & 3                                                     & 3                                                                & business                                                     & 16                                                           & \begin{tabular}[c]{@{}l@{}}Fox Group Legal (55\%), Mark Monitor (25.2\%), \\ Vobile (17.1\%)\end{tabular}                        & 365                                                            & 6,778                                                \\
tvad.me          & 1                                                     & 2                                                                & business                                                     & 14                                                           & Fox Group Legal (59.9\%), Mark Monitor (39.9\%)                                                                                & 94                                                             & 45,766                                               \\
israbox.vip      & 101                                                   & 5                                                                & media file download                                          & 19                                                           & Rivendell (99.6\%)                                                                                & 46                                                             & -                                                    \\
uploaded.net     & 3                                                     & 2                                                                & filesharing                                                  & 216                                                          & \begin{tabular}[c]{@{}l@{}}Rivendell (31.7\%), Skywalker(18.1\%)\\  Mark Monitor (14.9\%), Fox Group Legal (12.3\%)\end{tabular} & 365                                                            & 662                                                  \\
genteflowmp3.uno & 11                                                    & 2                                                                & media file download                                          & 6                                                            & Apdif Mexico (99.6\%)                                                                                & 149                                                            & -                                                    \\
deep-warez.org   & 1                                                     & 2                                                                & radiomusic                                                   & 43                                                           & Rivendell (98\%)                                                                                & 345                                                            & 183,305                                              \\ \bottomrule
\end{tabular}%
}
\caption{Top 10 domains with most complaints and the number of TLDs associated with each domain, times it appeared in the top 10  (in terms of complaints) per month, domain category,  major complainant (with the share of complaints), number of days receiving complaints and its Alexa ranking position.}
\label{tbl:top_10_classification}
\end{table*}

\begin{figure} [t]
\includegraphics[width= 8.5cm]{cdf_alexa_urls_dist}
\caption{CDF of the number of complaints per domain. Domains are classified by their Alexa Rank.}
\label{fig:cdf_alexa_urls_dist}
\end{figure}

We start by inspecting the number of complaints generated about each website. Figure~\ref{fig:cdf_alexa_urls_dist} presents a CDF showing the distribution of notices per domain. We separate the data depending on the domain popularity, based on its position in the Alexa ranking.
Again, we observe a noticeable skew: it is clear that there are a small number of `hot' websites that gain the most attention from notice senders. In particular, the most reported domains (the top 1\% by the number of reported URLs), receive 63\% of all complaints.
To provide insight into the characteristics of these domains, Table~\ref{tbl:top_10_classification} summarises the Top 10 in terms of the number of complaints. A range of uncommon websites are seen within this list. Although, overall, 60\% of notices relate to websites in the Alexa Top 100K, only 4 of the most reported 10 domains, and 8 out of the top 30 are within this Alexa rank (we find a Pearson  correlation of just 0.13 between these two rankings).
The reasons behind these relatively obscure sites gaining `notoriety' are quite diverse, but they do share one common attribute: their position in the ranking is typically driven by a single complainant that repeatedly targets them.
In fact, 9 out of the top 10 domains receive at least 50\% of their complaints from a single organisation. This is a general pattern across all domains: we find that 82\% of them receive at least half of their complaints from a single complaint.
To visualise this dominance by a few complainants in each domain, we depict in
Figure~\ref{fig:percent_dist_sender_domains} the percentage of URLs reported by the top 3 complainants of each domain (in green). The rest of complainants (in red) tend to contribute little to nothing.
The vast majority of domains receive nearly all complaints from just a tiny set of senders.
For example, \texttt{mp3toys.xyz} receives 99.9\% of complaints from a single party (Apdif Brasil).
Upon closer inspection, it is clear that this organisation uses an algorithm to `guess' potential infringing URLs based on song titles~\cite{Urban2016}. Although we envisage that these URLs are checked for liveness before complaints are generated, we note that \texttt{mp3toys.xyz} dynamically generates 200 OK HTTP responses for \emph{any} URL requested, likely disrupting any liveness checks.
These type of bulk sending activities do not appear to be rare occurrences. For example, the same organisation reported over 17M  URLs for file hosing domain \texttt{4shared.com}, despite the website only hosting 2M pages~\cite{torrentfreaknov2016}. As well as confirming that these highly active senders are heavily automated, it also suggests that rigorous procedures are not always followed.

\subsection{Are Domains Unique?}

The above is based on unique domain counts, however, we also posit that some of these domains may actually host the same content, or even resolve to the same IP address. To test this, we turn to our DNS and webpage probes, which downloaded the HTML from all domains. We extract their \texttt{<title>} and all metadata tags. In cases, where two domains' tags match, we assume they host the same page. We term these \emph{replicas}.
The vast majority of domains host different content.
Fewer than 0.01\% of domains have any replicas.
From those that do, 73\% refer to the same IP address, indicating that the web server operator has simply created multiple domain names. Just 2\% of these have matching second-level domains (with a different TLD), whereas the remainder actually are entirely distinct. We conjecture that this may be an evasion tactic to avoid DNS-based blocking schemes.
We also observe certain outliers; in the most extreme case, we find that \texttt{1fichier.com} uses 3,838 different domain names, which map to 78 IP addresses. This is a file sharing service well known for hosting illegal content. Another key driving force in the case of these extreme examples is the presence on numerous unblocker websites in our dataset. These websites essentially operate as proxies, generating third level domains for any website requested. For example, \texttt{s-s.www.cats.com.prx2.unblocksites.co} provides access to \texttt{www.cats.com} via \texttt{unblocksites.co}.
Table~\ref{tbl:top_20_second_level_domain} presents the top 10 (in terms of reported URLs) of these unblocking services. Remarkably, \texttt{unblocksites.co} actually constitutes 10\% of all reported URLs. In other words, we find that many complainants target these unblocker sites, by replicating their complaints for both the origin domain and the unblocked version. Understanding an exploring these is a key area of our future work.

\begin{figure} [t]
\includegraphics[width = 8.5cm]{percent_sender_domains}
\caption{Percentage of reported URLs by the top 3 complainants for each domain (on X-axis) \emph{vs.}\xspace the rest of complainants.}
\label{fig:percent_dist_sender_domains}
\end{figure}

\begin{table}[t]
\small
\centering
\resizebox{\columnwidth}{!}{%
\begin{tabular}{@{}lllll@{}}
\toprule
\textbf{\begin{tabular}[c]{@{}l@{}}Unblocking\\ Site\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\% of\\ Reported URLs\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}\% of\\ Domains\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}Unblocking Site \\ Category\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}Alexa\\  Rank\end{tabular}} \\ \midrule
unblocksites.co                                                    & 10                                                                     & 0.13                                                             & uncategorised                                                         & 12,252                                                         \\
freeproxy.fun                                                      & 2.4                                                                    & 0.02                                                             & webproxy                                                              & 26,517                                                         \\
unblocked.lol                                                      & 2                                                                      & 0.04                                                             & proxy avoidance                                                       & 8,552                                                          \\
unblockall.xyz                                                     & 0.9                                                                    & 0.02                                                             & proxy avoidance                                                       & 820,291                                                        \\
proxydude.xyz                                                      & 0.6                                                                    & 0.007                                                            & elevated exposure                                                     & -                                                              \\
immunicity.gold                                                    & 0.5                                                                    & 0.03                                                             & proxy avoidance                                                       & -                                                              \\
unblockall.org                                                     & 0.5                                                                    & 0.01                                                             & business                                                              & 4,947                                                          \\
unblocked.cam                                                      & 0.5                                                                    & 0.02                                                             & proxy avoidance                                                       &                                                                \\
unblocker.cc                                                       & 0.4                                                                    & 0.009                                                            & proxy avoidance                                                       & 15,535                                                         \\
unlockproj.club                                                    & 0.4                                                                    & 0.02                                                             & uncategorised                                                         &                                                                \\ \bottomrule
\end{tabular}%
}
\caption{Top 10 unblocking services present in our dataset.}
\label{tbl:top_20_second_level_domain}
\end{table}

\subsection{How Stable are Domain Rankings? }
\label{sub:domain_stability}

The previous sections indicate that complaints about a domain are often dominated by a single sender. We conjecture that this dominance may result in significant temporal instability.
This can be represented as a ranked list, capturing the domains most frequently reported on each day.
To explore this, Figure~\ref{fig:number_of_domains_reported_days} presents the count of domains that are reported on $x$ days.
81\% of domains are complained about on fewer than 5 days, with only 0.2\% being complained about on more than 300 days. This suggests a high degree of instability, with the make-up of complaints changing on a daily basis.
In line with our previous findings, the majority of these infrequently complained about domains are outside the Alexa Top 1M, whereas 87\% of domains which are complained about on over 300 days are within the top 1M. In Figure~\ref{fig:number_of_domains_reported_days} this can be seen as an upturn on the right-hand side of the graph.

To explore this trend, We next calculate the statistical variance of each domain's daily complaint count to understand how the number of daily complaints change.
Figure~\ref{fig:cdf_variance_domain} presents a CDF showing the per domain variance over the entire dataset (for domains with complaints on multiple days). About 15\% of domains have a variance of zero because they receive the same number of complaints each day they are reported; 94\% of these are reported under six times.
However, many remaining domains exhibit significant daily variance: 7\% of domains have a variance greater than 1K and, remarkably, 0.9\% of domains even have a daily variance greater than 1M.
This means that the number of complaints to a domain varies heavily on a day-to-day basis.

Closer inspection reveals that this is driven by \emph{extremely} aggressive complainants who periodically inject large sets of (sometimes repeat) notices.
For example, for domains with variance greater than 1M, we find that 92\% receive an average of 116 duplicates from the same sender.
To better highlight this aggressive activity, Figure~\ref{fig:timeseries} presents timeseries measuring the number of daily complaints for several example domains. We select the top two domains with the largest variance in each Alexa ranking category  (top 100K, top 1M, outside Alexa). Significant instability exists across each of these domains, with labels in the figure highlighting the individual sender causing each spike.
For instance, the top ranked domain, \texttt{mp3toys.xyz}, receives an average of 500K daily complaints from Apdif Brasil between Jan \& Feb, yet this collapses to below 6 per day in March. Similarly, \texttt{mp3taringa.net} receives a significant number of complaints between 03/Jan -- 05/Jan by Apdif Mexico, but following this the numbers collapse to  almost zero.
We posit that these senders are not always careful in the complaints they generate but, rather, send bulk notices, leaving the recipient to make sense of the content.

\begin{figure} [t!]
\includegraphics[width= 8.5cm]{number_of_domains_reported_days}
\caption{Number of days receiving complaints for each domain in a given Alexa rank.}
\label{fig:number_of_domains_reported_days}
\end{figure}

\begin{figure} [t!]
\includegraphics[width= 8.5cm]{cdf_variance_domain}
\caption{CDF of the variance of the number of complaints per day for each domain in a given Alexa rank.}
\label{fig:cdf_variance_domain}
\end{figure}

\begin{figure} [t!]
\includegraphics[width=8.5cm]{timeseries}
\caption{Timeseries with complaints to selected domains and complainants causing complaints' bursts.}
\label{fig:timeseries}
\end{figure}

\subsection{Are Reported Websites Alive?}
\label{sub:target_behaviour}

We finally inspect the availability of reported domains and webpage resources.
As it is impossible to draw causal links between a notice being issued and the removal of content,  we limit our analysis to inspecting the availability of URLs, rather than inferring the reason for their (un)availability.

\vspace{4pt}
\noindent\textbf{Understanding Domain Liveness.}
We first check the liveness of each domain's DNS record using our DNS dataset. This reveals that 22\% of reported domains return an NXDOMAIN response.\footnote{This is returned when a domain name does not exist on the authoritative name server any longer.}
These domains account for over 183M (17\%) of infringing URLs, with just 3\% (1,426) of them belonging to domains that rank within Alexa's Top 1M.
As different domain registrars may have differing policies regarding the removal of records, we next group domains by their TLD, and check the likelihood of domains being taken offline; Figure~\ref{fig:nx_tld_urls_domain_alexa_comparison} presents the results. We plot both the number of websites (domains) that are unavailable, as well as the number of specific URLs that become unavailable (because the domain is offline). For context, we plot the density of Alexa top 1M domains that have each TLD.
The majority of TLDs with a high percentage of NXDOMAIN responses do not frequently occur in the top Alexa rankings.
Instead, we see that the majority of domains are from the recent wave of new generic TLDs~\cite{spencer2014much}.
The most extreme is \texttt{.lol}, where 98\% of domains are NX; it is noteworthy that this TLD is operated by Uniregistry, which has been accused of predominantly hosting spam~\cite{lol_domain}. These trends indicate that the usage and behaviours across these TLDs are quite different, with some far more likely to contain unavailable domains.

\vspace{4pt}
\noindent\textbf{Understanding URL Availability. }
Next, we turn to our periodic HTTP liveness checks to see if resources (URLs) are still alive (for those domains that do not return NXDOMAIN).
We see that the number of 200 (OK) responses decline slowly but steadily over the 4 week period that we monitored. After week 4, 22\% of URLs are inactive (\emph{i.e.,}\xspace non-200). This trend, however, is relatively shallow with the majority of URLs (19\%) becoming inactive in the first week after the notice has been observed in Lumen.\footnote{Note that it is also possible that the URL was not live when the complaint was generated. Unfortunately, we cannot check this.}
We also note that the statuses returned evolve across the four weeks.
In week 1, the number of HTTP 4XX responses is 168K, yet in week 2 we only observe a further 12K webpages responding with 4XX.
Instead, the number of HTTP 5XX and TCP timeouts increase, suggesting that these websites go through several stages that start with removing of content (therefore returning a 4XX) before total shutdown of the web server.  The latter makes sense, as it may be unnecessary to continue running a server if most content has been removed.

\begin{figure} [t]
\includegraphics[width=8cm]{nx_tld_urls_domain_alexa_comparison}
\caption{Unavailable domains per TLD and URLs thus becoming inaccessible (as a \% from the total) \emph{vs.}\xspace the share of Alexa domains for each TLD.}
\label{fig:nx_tld_urls_domain_alexa_comparison}
\end{figure}

\vspace{4pt}
\noindent\textbf{Understanding Category Availability}.
\label{sub:category_availability}
We continue our analysis by inspecting which category of URLs are most likely to go offline. Figure~\ref{fig:domain_category_not_response_200} breaks down all URLs into their categories and presents the share within each group that reports non-200 HTTP responses after complaints are generated. Certain categories are significantly more likely to return non-200 responses.
For example, 10\% of URLs classified as File Download return a 404 response; similar traits are also seen across Parked (9\%) and Newly Registered Website (9\%).
In contrast, 24\% of URLs classified as \textit{Dynamic Content} (\emph{i.e.,}\xspace websites that generate different material for each visit) return a TCP connection timeout, \emph{i.e.,}\xspace the web server is no longer online.
Other examples of categories with high numbers of URLs that timeout include \textit{Elevated Exposure} (22\%), and \textit{File Download} (17\%).
The category which contains fewest unavailable sites is \textit{News}, where only 1.34\% of URLs become non-200. This indicates that the robustness of sites differs substantially across categories. That said, it is reasonable that domains more clearly engaging in suspicious activity are most likely to become unavailable.

\begin{figure} [t]
\includegraphics[width= 8.5cm]{domain_category_not_response_200}
\caption{URLs going offline (\emph{i.e.,}\xspace non-$2XX$ response) for the top-20 most reported domain categories (in \%).}

\label{fig:domain_category_not_response_200}
\end{figure}

\vspace{4pt}
\noindent\textbf{Understanding Complainant Success}
\label{sub:complainant_success}
An obvious question is which complainants are most likely to see their reported URLs deleted.
To measure this, we calculate the percentage of reported URLs from each notice sender that later sees the URL resource returning a non-200 response.
Figure~\ref{fig:notice_sender_liveness_all_response_except_200} presents the weekly percentage of complaints that return a non-200 response after  each weekly liveness check for the top 10 senders (based on the complainant with most not 2XX responses).
The websites targeted by these different notice senders have very different availability properties.
Complaints from rights enforcement organisations (\emph{e.g., }\xspace Rivendell, MarkScan, AudioLock.net) appear more effective compared to trade organisations (\emph{e.g., }\xspace British Phonographic Organisation). For example, about 53\% of complaints in the first week of submission from Rivendell return non 2XX response code whilst just 8\% from British Phonographic Organisation return same. These results confirms that the efficacy of these different organisations differs greatly, and that the websites they pursue have extremely different characteristics in terms of resilience to takedown.

\begin{figure} [t]
\includegraphics[width=8.5cm]{notice_sender_liveness_all_response_except_200}
\caption{Weekly share (non-cumulative) of URLs with a non $2XX$ response for the top 10 complainants.
\ignacio{different picture if done by domain? ie, a single over-reported domain being very reactive might bias this picture}
}
\label{fig:notice_sender_liveness_all_response_except_200}
\end{figure}

\gareth{Possible thing to add if we have time: VTscore stuff}

\section{Related Work}

There are two main areas of related work: ({\em i}\/)~studies that rely on Lumen for exploring web complaints; and ({\em ii}\/)~studies that have explored illegal online activities  more generally.
\vspace{4pt}

\noindent\textbf{Web Complaint Studies.}
There have been numerous studies into web complaints. Of most relevance are the small set of data-driven papers that have empirically explored the nature of takedown requests in the legal domain.
For example, Heins \emph{et al.}\xspace~\cite{Heins2005} examined 320 takedown notices to determine if the takedown process is fairly used by reporting organisations. They discovered that 20\% of notice senders send weak copyright claims, an assertion our results agree with.
Previous studies from Seng~\cite{Seng2014} and Urban \emph{et al.}\xspace~\cite{Urban2016} explored the entities behind notices from approximately 500K and 300K complaints, respectively. Both studies reveal that a small fraction of senders (mainly copyright protection companies and trade association) are responsible for the majority of notices. This is a result that is consistent with our findings. \cite{Urban2016} also classified the websites present in complaints; our work further builds on this to explore which categories of websites each sending entity specialises in.
In an earlier study, Urban \emph{et al.}\xspace~\cite{Urban2005} examined nearly 900 notices in an attempt to discover the primary reporting organisations that file them. They also saw that business entities and corporations were the main users of takedown notices. To the best of our knowledge, our study is the largest of its kind.

Closely related is the Right To Be Forgotten (RTBF), established in 2014 within the European Union. This allows individuals to request the de-listing of personal information from search engines, which is ``inaccurate, inadequate, irrelevant or excessive". Bertram \emph{et al.}\xspace~\cite{bertramthree} inspected the RTBF requests issued to Google. This work is complementary to our own, as we focus on different complaint mechanisms, \emph{i.e.,}\xspace Lumen does not cover RTBF. This is evident from the contrast between types of URLs in \cite{bertramthree} \emph{vs.}\xspace our study, \emph{e.g., }\xspace 33\% of RTBF URLs relate to social media, and 20\% to news. This can be compared against \S\ref{sub:topics}, where we find a greater propensity towards Entertainment and Business URLs (driven by the prominence of copyright enforcement notices).

\vspace{4pt}
\noindent\textbf{Illegal Web Activities.}
George \emph{et al.}\xspace~\cite{George2007} examined the challenges that comes with the ease of sharing User Generated Content (UGC), highlighting the roles played by hosting providers and proposing stronger legislation to address the illegal sharing of UGC.  Wong~\cite{Wong2009} suggested a more flexible legislation, whilst Sawyer~\cite{Sawyer2009} recommended that platforms that share UGC should develop solutions to mitigate against the sharing of infringing material. Clay and Lucas discussed how a UGC platform (YouTube) has been exploited for such purpose~\cite{clay2011blocking,hilderbrand2007youtube}. Raman \emph{et al.}\xspace also identified pirated content being shared via Facebook Live~\cite{raman2018facebook}.
To prevent the sharing of infringing content on such platforms, Dutta \emph{et al.}\xspace proposed a signature-based detection to mitigate against infringing material remaining accessible online~\cite{dutta2008detecting}.

Peer-to-Peer networks and file hosters are also a frequently used to disseminate illicit or illegal material. Despite several anti-piracy efforts through the injection of fake content on BitTorrent portals~\cite{Cuevas2013,Cuevas2014,kaune2010unraveling,Farahbakhsh2013} and the shutdown of file hosters services~\cite{Lauinger2013b}, about 90\% of files shared using BitTorrent protocol are judged to be infringing~\cite{Watters2011,Robert2010}. Furthermore, 80\% of files shared through file hosters are also in the same category~\cite{Lauinger2013a}. Ibosiola \emph{et al.}\xspace measured the availability of illegal content on streaming cyberlockers~\cite{Ibosiola2018}. They found that the majority of copyright infringing content is hosted on a small number of platforms.
A large portion of complaints in our data also pertain to adult content. While there have been several studies on online adult content~\cite{tyson2013demystifying,tyson2015people,tyson2016measurements}, few focus on illegal adult content~\cite{hurley2013measurement}.
Of course, there are also a number of related studies looking at video content more generally~\cite{bottger2018open}.
Our work is orthogonal as we primarily focus on the complaints that are related to these activities.

\section{Discussion and Future Work}
\label{sec:conclusion}

This paper has explored the nature of web complaints. With increasing scrutiny on illegal and illicit web activities, and the recent ability to streamline complaints against different stakeholders, this study offers a critical input into the wider ongoing debate about web governance and the use of so-called self-regulation~\cite{urist2006s}.

\vspace{4pt}
\noindent\textbf{Summary of Findings. }
We have found a large and complex ecosystem dominated by a small set of complainants. While there are a large number of organisations (38,523) that generate over 1 billion reported URLs, the top 10 complainants alone contribute over 41\% of all notices.
It therefore appears that these complaint procedures have become the dominion of a small group of large and very active organisations. Dominant players consists of a mix of influential copyright owners (\emph{e.g., }\xspace Fox) and third-parties specialised in pursuing copyright infringers (\emph{e.g., }\xspace Rivendell).
Bursts of complaints are common with most of the complaints towards each domain originating from 2--3 complainants, driving the unusual instability we see in the rankings.
Complainants are highly specialised in terms of the types of notices they generate and the domains they target. This leaves some domain categories (\emph{e.g., }\xspace File Sharing) regularly reported, and others rarely seen (\emph{e.g., }\xspace Education).
Surprisingly, many of the most frequently reported domains are quite obscure, and fail to score highly in popularity rankings.
Finally, we find that complaints do seem to matter. Many domain names are soon taken offline and 22\% of the URLs are inaccessible within just 4 weeks of us observing the complaints. Hence, it is clear that we shed light on a highly dynamic environment from the perspective of domain operators too.

\vspace{4pt}
\noindent\textbf{Societal and Legal Implications. }
Web governance and the (mis)use of web complaint mechanisms have important social and legal implications. Transparency is critical and, as a society, it is important to know how and why information is filtered.
This is particularly the case as we have found that these mechanisms might not be always used wisely, \emph{e.g., }\xspace with some complainants generating hundreds of repeat notices, and seemingly auto-generated URLs  (\S~\ref{sub:domain_stability}).
We argue that this might overwhelm recipients, who will not necessarily have the resources to deal with these large numbers.
As these highly centralised models of operation have the potential for misuse, understanding the activity of senders is therefore critical.

Our results further suggest that there is opportunity to improve and streamline the procedures.
For notice recipients, the filtering of invalid complaints would no doubt be a valuable innovation.
That said, we do not discount the veracity of many complaints, and  therefore developing mechanisms to support this process from the perspective of (notice senders would also be worthwhile. This, of course, should not be done at the expense of website operators, who should always be given paths to recourse.  Developing techniques that automate the above three things is important.
Arguably, Lumen and similar platforms can play a powerful role in this process.

\vspace{4pt}
\noindent\textbf{Future Work. }
There are a number of lines of future work.
First, we hope to expand our access to more diverse datasets. Within the paper, we have not investigated the role that recipients might play within the nature of complaints observed. This is likely to open up new lines of interesting investigation.
Our analysis has also revealed traits of a cat-and-mouse game, with complainants bulk sending notices, and websites replicating themselves across multiple domains and TLDs.
Exploring the temporal attributes of this game will no doubt reveal a number of yet unseen behaviours.
Last, we also wish to explore if search engines cease to index URLs that are complained about. Quantifying this forms a key strand of our future work.

\bibliographystyle{ACM-Reference-Format}
\balance

redpajama_set_name: RedPajamaArXiv


juberjabber: Gmail Theater: Why Use Gmail?
Why you should use Gmail... in four acts.
And now that you're convinced you should use Gmail, you can sign up for an account at http://mail.google.com/mail/signup. As of yesterday, we've dropped the invitation requirement for new accounts, so anyone can use Gmail.
I love the video! I've been in love with gMail for quite some time. I love hosted gMail even more :) The video is very creative :) Nice work Googlers!


text:
Cover Reveals/Excerpts
Non-bookish Things
3.5 stars, Books, Reviews
Review: You Were There Too by Colleen Oakley
Acclaimed author Colleen Oakley delivers a heart-wrenching and unforgettable love story about a woman who must choose between the man she loves and the man fate has chosen for her in a novel that reminds us that the best life is one led by the heart.
Mia Graydon's life looks picket-fence perfect; she has the house, her loving husband, and dreams of starting a family. But she has other dreams too — unexplained, recurring ones starring the same man. Still, she doesn't think much of them, until a relocation to small-town Pennsylvania brings her face to face with the stranger she has been dreaming about for years. And this man harbors a jaw-dropping secret of his own—he's been dreaming of her too.
Determined to understand, Mia and this not-so-stranger search for answers. But when diving into their pasts begins to unravel her life in the present, Mia emerges with a single question—what if?
5 stars, Books, Reviews
Review: Love Lettering by Kate Clayborn
In this warm and witty romance from acclaimed author Kate Clayborn, one little word puts one woman's business—and her heart—in jeopardy…
Meg Mackworth's hand-lettering skill has made her famous as the Planner of Park Slope, designing beautiful custom journals for New York City's elite. She has another skill too: reading signs that other people miss. Like the time she sat across from Reid Sutherland and his gorgeous fiancée, and knew their upcoming marriage was doomed to fail. Weaving a secret word into their wedding program was a little unprofessional, but she was sure no one else would spot it. She hadn't counted on sharp-eyed, pattern-obsessed Reid…
A year later, Reid has tracked Meg down to find out—before he leaves New York for good—how she knew that his meticulously planned future was about to implode. But with a looming deadline, a fractured friendship, and a bad case of creative block, Meg doesn't have time for Reid's questions—unless he can help her find her missing inspiration. As they gradually open up to each other about their lives, work, and regrets, both try to ignore the fact that their unlikely connection is growing deeper. But the signs are there—irresistible, indisputable, urging Meg to heed the messages Reid is sending her, before it's too late…
Review: Coldhearted Boss by R.S. Grey
I'm desperate and destitute when Lockwood Construction rolls into my small town with an offer too good to pass up: high wages to any able-bodied man willing to join their crew.
Say no more. I throw on baggy clothes, tuck my long hair under a baseball hat, and apply for a job. Unfortunately, my half-baked idea of disguising myself as a guy is flawed from the beginning. As Shakira says, these hips don't lie.
Still, I like to think I might have pulled the whole thing off save for one thing:
I know my boss.
Last month, we met at a bar, and after a fiery first encounter, it seems we're destined to be sworn enemies.
Ethan Stone is ruthless and arrogant, a man I never would have crossed had I known how much he likes to toy with his prey.
He should just fire me and be done with it. Instead, he decides to make me his personal slave. Oh right, I think they're calling it personal "assistant" these days.
It's torture, all of it—his bad attitude, his ruggedly chiseled face, his desire to grind me into dust.
Every one of our friction-filled battles burns hotter than the last.
A girl can only hold out for so long. Soon, I'm bound to go up in flames.
My objective? Survive the heat long enough to send home a paycheck.
My real objective? Stop having X-rated fantasies about my coldhearted boss.
Review: Boyfriend Bargain by Ilsa Madden-Mills (Hawthorne University #1)
Wanted: one hot guy with rock-hard abs and a big stick
Broke and desperate, Sugar Ryan has no use for arrogant, bad boy athletes . . . until she's forced to bargain with the cockiest of them all.
If only he knew she was alive.
Her mission? Get on this hockey player's radar any way possible.
Zack Morgan is the king of the ice and the bedroom—but nothing prepares him for the mystery girl who shows up everywhere he does—frat parties, his favorite bar, and finally his front door with an offer he can't refuse. The only rule in her boyfriend bargain: no falling in love.
But after one (um, two) smoking-hot hookups, he's done with pretending and vows to make their fake relationship real. Too bad she can't trust a player with a reputation for breaking hearts.
Will this hockey star score his forever girl or will their Boyfriend Bargain end in heartbreak?
Review: The Son and His Hope by Pepper Winters (The Ribbon Duet #3)
"Things you should know about me from the very beginning:
I was born to true love, witnessed the destruction it causes, and vowed never to let such agony happen to me. I am not a story-teller like my father. I am not a writer like my mother. I am just a son—their son.
I am happy being alone.
And that is all I ever want to be."
The day he was born, Jacob learned his hardest and longest lesson.
It wasn't a lesson a boy should learn so young, but from his earliest memories he knew where happiness lives, so does tragedy. Where love exists, so does heartbreak. And where hope resides, so does sorrow.
That lesson carved him from the kid to the teen to the man.
And nothing and no one could change his mind.
I first met him when he was fourteen at a movie premiere of all places. A movie based on his parent's life.
He was stoic, strong, suspicious, and secretive.
I was only ten, but I felt something for him. A strange kind of sorrow that made me want to hug and heal him.
I was the daughter of the actor hired to play his father.
We shared similarities.
I recognised parts of him because they were parts of me.
But no matter how many times we met. No matter how many times I tried.
He stayed true to his vow to never fall.
Review: More Than Bones by Craig David Singer
Dr. Emily Norton is a recent med school grad starting her surgical residency at a renowned hospital in Baltimore. She's dedicated her life to science and reason–so when she receives a supposedly magical amulet from an eccentric neighbor, she ignores the old man's warning.
Almost immediately, Emily's well-planned world begins to unravel. At first, the problems are minor, like finding herself in trouble with the hospital administration on her first day at work. But soon Emily is dealing with much bigger issues, from losing her job and fiancé to being framed for a patient's suicide.
As Emily struggles to comprehend the amulet's strange effects, she also faces a dilemma about coincidence and fate: none of these disasters could have been caused by a mere amulet, right? Because everyone knows: there's no such thing as magic.
Desperate and confused, Emily will need to use every bit of her reason, love and faith to confront and conquer the cursed amulet–even if it means forsaking everything she's ever believed.
At turns darkly suspenseful and playfully lighthearted, More Than Bones chronicles Emily's mystical journey with humor, empathy, and more than a few surprises.
Review: Hoops Holiday by Kennedy Ryan
MacKenzie Decker was a question Avery never got to ask, much less answer.
They met when she was a young reporter fueled by ambition, and the ink on Deck's first NBA contract was barely dry. Years later, they've climbed so high and lost so much, but one thing hasn't changed. The attraction that simmered between them in a locker room before is still there. With success like theirs, everything has been possible… except them.
But that was then.
The only question is… what about now?
Review: Rock Rebel by Tara Leigh (Nothing But Trouble #3)
He's a rock star with a secret, she's a pop princess with a painful past – can their forbidden romance survive, or will their lies destroy them both?
I've earned my bad reputation.
A few years ago, I was New York City's hottest classical music prodigy. But I wanted something else, something more. So I chased my real dream, and now… I'm rock royalty. Dax Hughes, lead guitarist of Nothing but Trouble. But to my family and former Juilliard classmates, I'm an outcast. A misfit. A rebel.
They're not entirely wrong. I don't give a damn what other people think, and I'm all for breaking the rules… except when it comes to our new opening act, Verity Moore.
Rock gods don't tour with pop princesses.
It's not personal. Actually, under that fallen diva reputation, Verity's incredibly talented. And her fiery redheaded personality is… intriguing. But I'm convinced the skeletons in Verity's closet are as scandalous as my own, and when we're not sparring, she has a way of drawing out all those secrets I'm determined to keep hidden.
Yeah. Verity Moore is definitely off-limits . . .
But since when do I give a damn about the rules?
Review: Unzipped by Lauren Blakely
Wanted: Dating coach to help hot nerd win back the woman of his dreams.
Picture this – I'm ready to win back the love of my life, and I'm going big this time. We're talking boom box, sing her name in the rain, let the whole damn neighborhood know I'm good and ready this time around. After all, if you're going to grand gesture the ever-loving hell out of a second chance, you need to pull out all the stops.
There's only one little problem.
My college girlfriend isn't the one who shows up when I play my "I'll do anything to win you back" tune.
The woman who flings open the second-floor window tells me my ex doesn't live here anymore. But she'll help me win her back. Anything for romance, anything for a guy so willing to go big for love. And that's what I want at first. Until I get to know my new "romance coach" and discover she's funny, clever, and keeps me on my toes. And boy, do I ever need that.
Now I don't want to win anyone else's heart. I want the woman who's been helping me all along.
Trouble is – she thinks I'm in love with someone else, and when we take off on a road trip, everything I think I know about women is about to be unzipped and turned inside out.
Review: Only Love by Melanie Harlow (One and Only #3)
A gorgeous former Marine with a tortured soul.
The beautiful, compassionate therapist living next door.
A meddlesome grandma determined to get them together.
I was expecting a proposal on my birthday, and I got dumped instead.
How could I have been so clueless?
Grams knew exactly how to distract me.
The "cute boy next door" who's been helping her with yard work clearly needs a little therapy. Who better to call than her newly single therapist granddaughter?
She even fakes dementia to get me to visit, and now that I'm here she's doing everything in her power to throw us together.
Not that I'm complaining. Ryan is the sexiest man I've ever met–I mean the full package, from the chiseled jaw to the massive shoulders to the rippling abs. (And yes, his package is full, and he knows exactly how to deliver it.) He makes me want to get out of my head and follow my heart. He makes me want to take chances I never thought I'd take.
He also makes me want to take my clothes off. A lot.
But he's moody and challenging–one minute he's an open book, and the next he's completely closed off. He holds me like he'll never let go, but insists he wants to be alone.
Some wounds are so deep, only love can heal them.
How can I convince him to let me try?
We're two sisters with a massive love for reading and reviewing (mostly) contemporary romance books. We also occasionally write non-book(ish) posts!
Lovely to meet you,
Subscribe to our mailing list!!
We respect your privacy >>
Follow The Bookish Sisters on WordPress.com
Let's talk: Natural Deodorant
ARCHIVES Select Month January 2020 (3) June 2019 (2) February 2019 (1) January 2019 (2) December 2018 (6) October 2018 (1) September 2018 (5) August 2018 (13) July 2018 (14) June 2018 (20) May 2018 (24) April 2018 (19) March 2018 (20) February 2018 (23) January 2018 (26) December 2017 (15) November 2017 (17) October 2017 (19) September 2017 (19) August 2017 (13) July 2017 (13) June 2017 (13) May 2017 (11) April 2017 (15) March 2017 (13) February 2017 (12)
Made with by Claudia & Diana | © Copyright 2017-2020 The Bookish Sisters Privacy Policy

redpajama_set_name: RedPajamaCommonCrawl


text:
Ring Chromosome 5 in acute myeloid leukemia defned by whole-genome single nucleotide polymorphism array
Jungwon Huh, Yeung Chul Mun, Wha Soon Chung, Chu Myong Seong
Department of Laboratory Medicine
Department of Internal Medicine
Chromosomes forming a corresponding ring cannot be clearly defned by conventional cytogenetics or FISH. Karyotypic analyses using whole-genome single nucleotide polymorphism arrays (SNP-A) may result in the identifcation of previously cryptic lesions and allow for more precise defnition of breakpoints. We describe a case of AML with metaphase cells bearing -5, del(11)(q22), and +r. With SNP-A, a 5p-terminal deletion (11 megabases [Mb]), a 5q-terminal deletion (27 Mb), an 11q-interstitial deletion (29 Mb), and a 21q gain (3 Mb) were identifed. Therefore, the G-banded karyotype was revised as 46, XY, r(5)(p15. 2q33.2), del(11)(q14.1q23.2), dup(21)(q22.13q22.2)[18]/46, XY[2]. SNP-A could be a powerful tool for characterizing ring chromosomes in which the involved chromosomes or bands cannot be precisely identifed by conventional cytogenetics or FISH.
Annals of Laboratory Medicine
https://doi.org/10.3343/alm.2012.32.4.307
Published - Jul 2012
Chromosome 5
10.3343/alm.2012.32.4.307
Dive into the research topics of 'Ring Chromosome 5 in acute myeloid leukemia defned by whole-genome single nucleotide polymorphism array'. Together they form a unique fingerprint.
Ring Chromosomes Medicine & Life Sciences 100%
Chromosomes, Human, Pair 5 Medicine & Life Sciences 88%
Acute Myeloid Leukemia Medicine & Life Sciences 62%
Single Nucleotide Polymorphism Medicine & Life Sciences 55%
Cytogenetics Medicine & Life Sciences 43%
Chromosomes Medicine & Life Sciences 34%
Metaphase Medicine & Life Sciences 24%
Huh, J., Mun, Y. C., Chung, W. S., & Seong, C. M. (2012). Ring Chromosome 5 in acute myeloid leukemia defned by whole-genome single nucleotide polymorphism array. Annals of Laboratory Medicine, 32(4), 307-311. https://doi.org/10.3343/alm.2012.32.4.307
Huh, Jungwon ; Mun, Yeung Chul ; Chung, Wha Soon et al. / Ring Chromosome 5 in acute myeloid leukemia defned by whole-genome single nucleotide polymorphism array. In: Annals of Laboratory Medicine. 2012 ; Vol. 32, No. 4. pp. 307-311.
@article{13e5a31fab3a472d9e7b2ad73d639ee0,
title = "Ring Chromosome 5 in acute myeloid leukemia defned by whole-genome single nucleotide polymorphism array",
abstract = "Chromosomes forming a corresponding ring cannot be clearly defned by conventional cytogenetics or FISH. Karyotypic analyses using whole-genome single nucleotide polymorphism arrays (SNP-A) may result in the identifcation of previously cryptic lesions and allow for more precise defnition of breakpoints. We describe a case of AML with metaphase cells bearing -5, del(11)(q22), and +r. With SNP-A, a 5p-terminal deletion (11 megabases [Mb]), a 5q-terminal deletion (27 Mb), an 11q-interstitial deletion (29 Mb), and a 21q gain (3 Mb) were identifed. Therefore, the G-banded karyotype was revised as 46, XY, r(5)(p15. 2q33.2), del(11)(q14.1q23.2), dup(21)(q22.13q22.2)[18]/46, XY[2]. SNP-A could be a powerful tool for characterizing ring chromosomes in which the involved chromosomes or bands cannot be precisely identifed by conventional cytogenetics or FISH.",
keywords = "AML, Array, Chromosome 5, Ring, Single nucleotide polymorphism",
author = "Jungwon Huh and Mun, {Yeung Chul} and Chung, {Wha Soon} and Seong, {Chu Myong}",
doi = "10.3343/alm.2012.32.4.307",
journal = "Annals of Laboratory Medicine",
publisher = "Seoul National University",
Huh, J, Mun, YC, Chung, WS & Seong, CM 2012, 'Ring Chromosome 5 in acute myeloid leukemia defned by whole-genome single nucleotide polymorphism array', Annals of Laboratory Medicine, vol. 32, no. 4, pp. 307-311. https://doi.org/10.3343/alm.2012.32.4.307
Ring Chromosome 5 in acute myeloid leukemia defned by whole-genome single nucleotide polymorphism array. / Huh, Jungwon; Mun, Yeung Chul; Chung, Wha Soon et al.
In: Annals of Laboratory Medicine, Vol. 32, No. 4, 07.2012, p. 307-311.
T1 - Ring Chromosome 5 in acute myeloid leukemia defned by whole-genome single nucleotide polymorphism array
AU - Huh, Jungwon
AU - Mun, Yeung Chul
AU - Chung, Wha Soon
AU - Seong, Chu Myong
N2 - Chromosomes forming a corresponding ring cannot be clearly defned by conventional cytogenetics or FISH. Karyotypic analyses using whole-genome single nucleotide polymorphism arrays (SNP-A) may result in the identifcation of previously cryptic lesions and allow for more precise defnition of breakpoints. We describe a case of AML with metaphase cells bearing -5, del(11)(q22), and +r. With SNP-A, a 5p-terminal deletion (11 megabases [Mb]), a 5q-terminal deletion (27 Mb), an 11q-interstitial deletion (29 Mb), and a 21q gain (3 Mb) were identifed. Therefore, the G-banded karyotype was revised as 46, XY, r(5)(p15. 2q33.2), del(11)(q14.1q23.2), dup(21)(q22.13q22.2)[18]/46, XY[2]. SNP-A could be a powerful tool for characterizing ring chromosomes in which the involved chromosomes or bands cannot be precisely identifed by conventional cytogenetics or FISH.
AB - Chromosomes forming a corresponding ring cannot be clearly defned by conventional cytogenetics or FISH. Karyotypic analyses using whole-genome single nucleotide polymorphism arrays (SNP-A) may result in the identifcation of previously cryptic lesions and allow for more precise defnition of breakpoints. We describe a case of AML with metaphase cells bearing -5, del(11)(q22), and +r. With SNP-A, a 5p-terminal deletion (11 megabases [Mb]), a 5q-terminal deletion (27 Mb), an 11q-interstitial deletion (29 Mb), and a 21q gain (3 Mb) were identifed. Therefore, the G-banded karyotype was revised as 46, XY, r(5)(p15. 2q33.2), del(11)(q14.1q23.2), dup(21)(q22.13q22.2)[18]/46, XY[2]. SNP-A could be a powerful tool for characterizing ring chromosomes in which the involved chromosomes or bands cannot be precisely identifed by conventional cytogenetics or FISH.
KW - AML
KW - Array
KW - Chromosome 5
KW - Ring
KW - Single nucleotide polymorphism
U2 - 10.3343/alm.2012.32.4.307
DO - 10.3343/alm.2012.32.4.307
JO - Annals of Laboratory Medicine
JF - Annals of Laboratory Medicine
Huh J, Mun YC, Chung WS, Seong CM. Ring Chromosome 5 in acute myeloid leukemia defned by whole-genome single nucleotide polymorphism array. Annals of Laboratory Medicine. 2012 Jul;32(4):307-311. doi: 10.3343/alm.2012.32.4.307