Recent developments in machine translation give hope for the possibility of communication without language barriers, as real-time interlingual conversations facilitated through automatic translation are already possible using free applications. This study aimed at measuring the level of intelligibility and accuracy of real-time chat messages translated instantly by translation bots embedded in GoogleTalk. The data consisted of chat scripts of a total of 12 sessions conducted between three pairs formed each of a native English- and Russian-speaker. The participants also answered a questionnaire about their chat experiences. The results suggest that even without any knowledge of the language of the other party, participants were able to conduct conversations on various topics without encountering any serious communication breakdown. About three fourth of the translated propositions were intelligible as well as accurate based on the human evaluation. Participants also reported positive comments on the effectiveness of this kind of interlingual communication, especially for informal chat.