# How best to animate/display streaming LLM messages? **URL:** \ **Category:** Ask us anything\ **Created:** [December 21, 2023, 4:15am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039 "2023-12-21T04:15:23Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![khara](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/khara/32/6776_2.png) [@khara](https://community.weweb.io/u/khara)\ **Post date:** [December 21, 2023, 4:15am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/1 "2023-12-21T04:15:23Z")
High level: I am trying to display LLM generated messages as they are generated. LLM generation tends to be slow in general, so displaying the message slowly, line by line, is a UI trick most sites use so that the user doesn’t feel like they are waiting forever. I am using a custom model, not OpenAI, so I can’t use the WeWeb OpenAI plugin. What is the best way to approach displaying this streaming text in WeWeb? My Stack: Supabase collections in weweb, plus API calls to my coded server (have full flexibility there) Some things I’ve been trying: [Trying to see if real streaming could work] 1. Supabase realtime tables + dynamic collection (if I manually update the collection data, the change doesn’t appear in my WeWeb app until I manually refresh the page or collection. Is this how it is supposed to work, or should the data refresh itself with the change?) [Trying a simpler task; how would I animate lines appearing one by one with a delay?] 1. Conditional rendering: I can get a simple conditional render to work - eg return false and the element doesn’t display. However, it doesn’t seem to render with a delay when a simple timeout is put in the JS function 2. Is there an easy way to animate display from none → block? It isn’t listed in transitions and I tried putting custom css in the box, but it is getting removed when I try it in preview mode. Thank you in advance for the help! 🙏✨✨ ---
**Author:** ![geenius-cloud](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/geenius-cloud/32/6746_2.png) [@geenius-cloud](https://community.weweb.io/u/geenius-cloud)\ **Post date:** [December 21, 2023, 5:49am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/2 "2023-12-21T05:49:13Z")
@khara I am currently trying to figure this out as well as we are running the OpenAI Assistant API, all models from AWS Bedrock and now also Gemini Pro from Google Cloud. [@weweb-team](https://community.weweb.io/groups/weweb-team) Is the stream=true which is available for most Chat endpoints of GPT-4, Claude, Cohere, Jurassic, Titan, Llama possible to do in WeWeb natively or does this streaming functionality need to be implemented manually through an NPM package, etc? ---
**Author:** ![khara](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/khara/32/6776_2.png) [@khara](https://community.weweb.io/u/khara)\ **Post date:** [December 22, 2023, 6:38am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/3 "2023-12-22T06:38:55Z")
Right! It seems like something probably a lot of people might need right now? 🤞 Fingers crossed that there’s already a fast way to do this in WeWeb, but wondering if it’s necessary to roll a custom component. ---
**Author:** ![Lars](https://avatars.discourse-cdn.com/v4/letter/l/74df32/32.png) [@Lars](https://community.weweb.io/u/Lars)\ **Post date:** [December 22, 2023, 9:17am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/4 "2023-12-22T09:17:13Z")
Yeah, looking for the same solution. It would be nice to have it natively in weweb ---
**Author:** ![Lars](https://avatars.discourse-cdn.com/v4/letter/l/74df32/32.png) [@Lars](https://community.weweb.io/u/Lars)\ **Post date:** [December 22, 2023, 10:27am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/5 "2023-12-22T10:27:24Z")
Make a feature request! [Public Roadmap | WeWeb](https://feedback.weweb.io/make-a-request/) ---
**Author:** ![carrano\_dot\_dev](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/carrano_dot_dev/32/10547_2.png) [@carrano\_dot\_dev](https://community.weweb.io/u/carrano_dot_dev)\ **Post date:** [December 22, 2023, 6:36pm UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/7 "2023-12-22T18:36:38Z")
[https://platform.openai.com/docs/api-reference/streaming](https://platform.openai.com/docs/api-reference/streaming) ---
**Author:** ![khara](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/khara/32/6776_2.png) [@khara](https://community.weweb.io/u/khara)\ **Post date:** [December 23, 2023, 2:27am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/8 "2023-12-23T02:27:11Z")
@carrano_dot_dev thanks! I’m actually wondering about the frontend display for dynamically updating the streamed data though, I have the server part already. I haven’t been able to get WeWeb to automatically update supabase collections when the data is updated, and wondering if there is a way to do that? Or alternatively if there is a way to stream directly to a WeWeb component. [@weweb-team](https://community.weweb.io/groups/weweb-team) couple of questions: 1. Is there a way to do this already in WeWeb that I’ve missed? (Dynamically display text/data as it is streamed in) 2. Or if not, is this on the roadmap already for the next month? If not, then will look into building a custom component, but was hoping to avoid learning vue! ---
**Author:** ![carrano\_dot\_dev](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/carrano_dot_dev/32/10547_2.png) [@carrano\_dot\_dev](https://community.weweb.io/u/carrano_dot_dev)\ **Post date:** [December 23, 2023, 6:08am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/9 "2023-12-23T06:08:59Z")
I’d imagine you would just create a text variable and bind it to a text element on the frontend. Then you could run a workflow that updates that variable as the data from the API is streamed in. ---
**Author:** ![khara](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/khara/32/6776_2.png) [@khara](https://community.weweb.io/u/khara)\ **Post date:** [December 23, 2023, 9:13am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/10 "2023-12-23T09:13:23Z")
Ah, rad ⚡ Much gratitude, many thanks! I’ve been overly focused on updating the collection itself. So something like: 1. Fake a while loop: [Loop until condition is met - #4 by clncsports](https://community.weweb.io/t/loop-until-condition-is-met/1564/4) 2. Insert temporary text element and update as the API streams 3. When API is done streaming, hide temporary text element + add contents to collection, refresh collection (For future reference, curious if there is a way to get collections to update in realtime that I might have missed!) ---
**Author:** ![khara](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/khara/32/6776_2.png) [@khara](https://community.weweb.io/u/khara)\ **Post date:** [December 23, 2023, 11:29am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/11 "2023-12-23T11:29:20Z")
Hmm ok - so far as I can tell, the WeWeb text won’t update correctly with the streaming text, it sort of updates all at once when the API call finishes. There might be a way to read the filestream directly from WeWeb’s workflow output object, using JS, but I couldn’t find it in a first pass. ---
**Author:** ![khara](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/khara/32/6776_2.png) [@khara](https://community.weweb.io/u/khara)\ **Post date:** [December 24, 2023, 2:27am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/12 "2023-12-24T02:27:39Z")
lol just realized supabase was not updating because I didn’t have realtime enabled. Realtime updates are working there, so going to just try streaming directly to supabase for now. I also looked into creating a coded component to handle streaming as well. Setup was pretty smooth, but then realized you need to be on a higher tier plan to actually use the coded components. ---
**Author:** ![khara](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/khara/32/6776_2.png) [@khara](https://community.weweb.io/u/khara)\ **Post date:** [December 25, 2023, 2:04am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/13 "2023-12-25T02:04:43Z")
Ok final update from me. For anyone else who needs a quick solution here, streaming directly to a supabase table with realtime updates works decently well in practice. This does use more of the realtime message quota on the free tier, but supabase is quite generous with that allowance. I ended up chunking together some of the streamed bits to save on DB resources, so it isn’t hitting it constantly. It would be nice to stream directly to the client, so curious to know if this is on the roadmap at all if anyone from [@weweb-team](https://community.weweb.io/groups/weweb-team) sees this. ---
**Author:** ![chadsmithclt](https://avatars.discourse-cdn.com/v4/letter/c/8797f3/32.png) [@chadsmithclt](https://community.weweb.io/u/chadsmithclt)\ **Post date:** [December 26, 2023, 5:01pm UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/14 "2023-12-26T17:01:20Z")
I am doing this on my app 🙂 what I have done is created a text variable and then used WeWeb copilot to create JavaScript where the value i want displayed is iterated one word at a time with a 200 ms delay and added to the text variable. Works really well, I’ve built a form app that feels like a conversation as you respond to questions ---
**Author:** ![khara](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/khara/32/6776_2.png) [@khara](https://community.weweb.io/u/khara)\ **Post date:** [December 30, 2023, 4:16am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/15 "2023-12-30T04:16:06Z")
Oh wild, thanks for sharing! Great to know that it is possible to stream directly to client instead of the DB 🙂 Will have to try that and see if performance is better than what I have. Happy new year ✨ ---
**Author:** ![raelyn](https://sea2.discourse-cdn.com/flex016/user_avatar/community.weweb.io/raelyn/32/2803_2.png) [@raelyn](https://community.weweb.io/u/raelyn)\ **Post date:** [December 31, 2023, 8:08am UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/16 "2023-12-31T08:08:45Z")
Are you using the Weweb open Ai plugin? How are you getting the open ai output to show up in the text variable? ---
**Author:** ![chadsmithclt](https://avatars.discourse-cdn.com/v4/letter/c/8797f3/32.png) [@chadsmithclt](https://community.weweb.io/u/chadsmithclt)\ **Post date:** [January 2, 2024, 6:24pm UTC](https://community.weweb.io/t/how-best-to-animate-display-streaming-llm-messages/6039/17 "2024-01-02T18:24:56Z")
I use the OpenAI plugin to generate the response and then update a variable directly from the OpenAI output using a workflow. I update the database later based on another trigger so that the user doesn’t have to wait for the form to retrieve the OpenAI response, store to the data, retrieve the value, etc etc…