retrieved results are relevant or not based on that information the system could then automatically refine the results using a number of different techniques if the invention is integrated closely as part of an existing information retrieval system then the invention can use some of the functionality of the existing information retrieval system to help refine the search but if the invention is used as a front end to existing information retrieval systems then it will be limited in how it can communicate with the existing information retrieval system in this case the system will likely produce refined search queries in the preferred embodiment information is marked using relevant buttons irrelevant buttons or not sure buttons one could also use a slider element or other simple user interface elements there are several options how search results can be displayed to the user these include the following a highlight relevant and not sure search results with a border or by changing their color while hiding irrelevant search results b do not show any marked search results but only showing additional search results that have not been marked by the user yet so that the already marked results do not clutter the display the user can see the marked results later by going to the summary page d put the different search results on different tabs or on different portions of the screen so that the user can easily look from one part to another although the preceding description contains significant detail if should not be construed as limiting the scope of the invention but rather as providing illustrations of the preferred embodiments of the invention one.