Is it possible to update the OpenAI plugin to add streaming to Assistant?

Zacharyb · March 22, 2024, 11:04pm

Hello !

openai/openai-node/blob/master/helpers.md#assistant-events

# Streaming Helpers

OpenAI supports streaming responses when interacting with the [Chat](#chat-streaming) or [Assistant](#assistant-streaming-api) APIs.

## Assistant Streaming API

OpenAI supports streaming responses from Assistants. The SDK provides convenience wrappers around the API
so you can subscribe to the types of events you are interested in as well as receive accumulated responses.

More information can be found in the documentation: [Assistant Streaming](https://platform.openai.com/docs/assistants/overview?lang=node.js)

#### An example of creating a run and subscribing to some events

```ts
const run = openai.beta.threads.runs
  .createAndStream(thread.id, {
    assistant_id: assistant.id,
  })
  .on('textCreated', (text) => process.stdout.write('\nassistant > '))
  .on('textDelta', (textDelta, snapshot) => process.stdout.write(textDelta.value))

This file has been truncated. show original

https://platform.openai.com/docs/assistants/overview?context=with-streaming

For the ressources, including the js sdk

Thanks !

Broberto · March 23, 2024, 4:30pm

I wish we could finally write our own plugins for WeWeb. Then this wouldn’t be an issue.

Zacharyb · March 23, 2024, 5:32pm

yep but you can with a Scale plan i think
was thinking of proposing a pull request on their plugin, then i said to myself, just do directly use of the sdk in other way simply

WeWeb is not yet AI focus webapp solution yet !

gduteaud · March 23, 2024, 6:29pm

It’s not quite the same thing and definitely more of a learning curve but I was able to implement similar functionality with GitHub - xenova/transformers.js: State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

There’s pros and cons: it doesn’t rely on OpenAI at all (which can be a good thing or a bad thing depending on your POV), runs in the browser so it’s effectively free, but the models that are currently supported are older and not as sophisticated as GPT-3.5/4. The upcoming v3 release should go a long way towards addressing that last point (long story, look at the PR for more info).

Might be something to think about!

Zacharyb · March 23, 2024, 6:51pm

That’s why for me a vercel ai sdk or something like that should be one huge feature to be added to the roadmap
As it is now OS, and very effective

IKR · March 24, 2024, 4:50pm

It’s JS.
You can do everything.

Zacharyb · March 24, 2024, 8:13pm

yep, i guess using code allows it for sure

IKR · March 24, 2024, 8:20pm

than i don’t understand.

Zacharyb · March 27, 2024, 9:46pm

that plugin allow easy integration than coding it, if so why not just use vercel ai sdk

IKR · March 27, 2024, 11:51pm

yeah, no one is confirming the result.

Topic		Replies	Views
OpenAI Plugin Chat-with-PDF on WeWeb How do I?	2	201	October 11, 2023
Connect ChatGPT API How do I?	16	1601	April 4, 2023
OpenAI plugin timeout question Ask us anything	1	122	May 13, 2023
Websockets \| JS plugin \| Appwrite How do I?	1	221	December 14, 2023
What are the API Limits when using plugins like Open AI? Ask us anything	1	102	July 6, 2023

Is it possible to update the OpenAI plugin to add streaming to Assistant?

Related Topics