This invention relates generally to social networking, and in particular to the management of user reports related to offensive, harassing, or “spammy” content in a social network system.
Social networking systems allow users to create accounts that are tied to their identities. The social networking system allows users to post objects, such as photos and videos, and to take actions, such as commenting and messaging (collectively the objects and actions are called “content”). Occasionally users of the social networking system will encounter content from other users that is offensive, harassing, spammy, or otherwise objectionable to the recipient and/or against the usage policies of the social networking system (collectively called “inappropriate” content). Social networking systems allow users to generate reports identifying inappropriate content as they encounter it. Typically a social networking system manages these reports through a review process. The review process must handle the case where content is reported that is not inappropriate. The social networking system cannot simply block all content that every user reports as this may lead to overbroad censorship of content on the social networking system. The social networking systems review process must contain checks to ensure that only truly inappropriate content is blocked. For example, when a social networking system receives a report from a user that identifies an allegedly offensive photo, the review process may use a human agent to review the photo to confirm that is in fact offensive.
Manual review of reported content may require significant human resources. In recent years, the volume of reports from users of social networking systems has made the management of these reports an overwhelming task. In many cases users report content that is not inappropriate simply because they find the content embarrassing or uncomplimentary. For example, a user may report a photo as spam, where the photo merely contains an uncomplimentary image of him. Reviewing such unjustified reports consumes resources and creates delays in resolving other legitimate reports.
For these reasons there has been a pressing need for a system that can satisfactorily resolve user reports without consuming human resources, while reducing the number of unjustified reports that are submitted to the content reviewing process.