content.raw: The next version of Gradio will be significantly more efficient (as well as a bit faster) for anyone who uses Gradio's streaming features. Looking at you chatbot developers 
content.type: text
content.value: The next version of Gradio will be significantly more efficient (as well as a bit faster) for anyone who uses Gradio's streaming features. Looking at you chatbot developers 
content.raw: @oobabooga
content.type: mention
content.user: oobabooga
content.type: text
content.raw: @pseudotensor
content.type: mention
content.user: pseudotensor
content.raw:  :) 
content.type: text
content.value:  :) 
content.type: new_line
content.type: new_line
content.raw: The major change that we're making is that when you stream data, Gradio used to send the entire payload at each token. This is generally the most robust way to ensure all the data is correctly transmitted. We've now switched to sending "diffs" --> so at each time step, we automatically compute the diff between the most recent updates and then only send the latest token (or whatever the diff may be). Coupled with the fact that we are now using SSE, which is a more robust communication protocol than WS (SSE will resend packets if there's any drops), we should have the best of both worlds: efficient *and* robust streaming.
content.type: text
content.value: The major change that we're making is that when you stream data, Gradio used to send the entire payload at each token. This is generally the most robust way to ensure all the data is correctly transmitted. We've now switched to sending "diffs" --> so at each time step, we automatically compute the diff between the most recent updates and then only send the latest token (or whatever the diff may be). Coupled with the fact that we are now using SSE, which is a more robust communication protocol than WS (SSE will resend packets if there's any drops), we should have the best of both worlds: efficient *and* robust streaming.
content.type: new_line
content.type: new_line
content.raw: Very cool stuff 
content.type: text
content.value: Very cool stuff 
content.raw: @aliabid94
content.type: mention
content.user: aliabid94
content.raw: ! PR: 
content.type: text
content.value: ! PR: 
content.href: https://github.com/gradio-app/gradio/pull/7102
content.raw: https://github.com/gradio-app/gradio/pull/7102
content.type: link
rawContent: The next version of Gradio will be significantly more efficient (as well as a bit faster) for anyone who uses Gradio's streaming features. Looking at you chatbot developers @oobabooga @pseudotensor :) 
  
  The major change that we're making is that when you stream data, Gradio used to send the entire payload at each token. This is generally the most robust way to ensure all the data is correctly transmitted. We've now switched to sending "diffs" --> so at each time step, we automatically compute the diff between the most recent updates and then only send the latest token (or whatever the diff may be). Coupled with the fact that we are now using SSE, which is a more robust communication protocol than WS (SSE will resend packets if there's any drops), we should have the best of both worlds: efficient *and* robust streaming.
  
  Very cool stuff @aliabid94! PR: https://github.com/gradio-app/gradio/pull/7102
reactions.reaction: ❤️
reactions.users: abbasm2 osanseviero yjernite oobabooga samusenps jeffboudier gsarti sam-kap satpalsr akhaliq alielfilali01 joaogante