Hi, I am using the open AI plugin in Weweb, and it used to show the number of tokens used, but it disappeared? How do I make it appear again?
As seen from Open AI’s website:
Hi, it seems like a regression, please open a ticket on support.weweb.io, we will figure it out 
Thanks! I did last week. Is there any ETA on this? Need the token count to make sure users do not go over their limits.
Hi, sadly its a missing feature from OpenAI since we upgraded our integration to use the stream mode. We was forced to use this mode because of how much timeout we got previously. In this mode we don’t have access to this information anymore.
Do you have any workaround in mind ? You could try to limit the number of words someone can send. Its not perfect but it would be more meaningful for your user to be able to see a word count limit while typing. What do you think ?
Hey @Alexis! Could you elaborate more on this timeout issue? What causes it and how would streaming mode solve it?
As a follow up q, I am setting the stream to false currently, but even when it’s false, I do not get the token usage information?
I also saw here that Usage Info in API Responses - #6 by raymonddavey - Announcements - OpenAI Developer Forum we can ask the open ai staff to enable this feature. Could we enable it?
I’m assuming everyone here who uses the open AI plugin is monetizing the AI (not just for personal use) and need to figure out how many tokens are used to calculate costs. I guess there could be workarounds (like you mentioned) but there would still be a discrepancy between words and the cost of running the app, overtime it would be a problem.
1 Like
Even when you’re not enabling streaming we are using it under the hood. The toggle is about Front-end ↔ WeWeb server, but WeWeb ↔ OpenAI is still using the stream mode, because by receiving the response word by word we keep the connection alive. Previously we waited the finalised response and depending of the time the request failed because of how long it took to finish.
Thanks for sharing, we asked them to enable it on our account so we can test it and implement it! Please do not enable it on your side because it could break our integration.
I don’t know how long it will take for them to respond and when we will be able to implement it, but its on our roadmap now
You may need to call the api yourself now. Setting up the calls isn’t too hard! Seems like you can accomplish what you need that way!
Hi, could you share the documentation about that ? Looks interesting. If you want it to stay secure it has to be implemented backend side (so in our plugin by ourself or inside a custom backend like a Xano)
@jaredgibb Could you elaborate more on this? What do you mean by that? wouldn’t it lead to time out too?
not necessarily. I see 2 safe way of doing this, thanks to @Alexis point in the post below.
- Use a realtime DB like supabase.
- user makes a query
- a record with the query is created in supabase
- record creation triggers a function to run that does the API call to openAI
- the openAI response and tokens used are updated to the OG record
- the response would show immediately on the UI AND you’d have the tokens used
[I have removed the content from my post as it is an non-secure way to do things and could lead to issues for you down the road].
or
1 Like
Be very careful, if you put that inside WeWeb the API Key will be exposed in the frontend and anyone will be able to stole it from the network tab.
1 Like
thank you for pointing that out! the calls should be made from the backend and in that case, i would honestly fire up a google cloud function that i can call from my weweb app. it makes the call to openAI using the same fetch method and returns the response to the weweb app.
so using a google cloud function as a proxy in this case.
1 Like
@jaredgibb Great suggestions - I am having trouble understanding how doing what you suggested (method 1 - making the query back end) would not lead to the same timeout issue?
How is this different from what Weweb is/was doing? I’m assuming Weweb was making the request on behalf of me, and then now I’m making that same request, which would lead to the same timeout problems if I don’t have streaming?
@Alexis tagging you here in case you know the answer.
Timeouts can be controlled using JavaScript and fetch.
Native fetch doesn’t have a timeout. So calling the api from JavaScript instead of the api connector allows you to have greater control over the entire execution including timeout.
Hi, yes you will have a timeout issue but it can be controlled as @jaredgibb said, we choose to change the way we communicate with OpenAI instead because we didn’t want people to wait too many time before having a response, but you can choose this tradeoff by doing it yourself on your backend instead of using our plugin.
1 Like