[URGENT] Performance issues, huge latency and live website going down this weekend

Is anyone else experiencing huge issues this weekend with published sites? Site not responding at all, 502 Bad Gateway, >10 seconds latency…

Haven’t had a reply over the chat since yesterday and this is getting serious enough now that it needs to be handled, week-end or not.

All sites unpublished…!

My site is being published & promoted fast today, but editor is super slow and stuck on load… but this is not new issue.

Interesting. Where are you located? I’m on the Canadian east coast. This might be a localized infra issue.

Downtime started a little more than 3 hours ago and have been ongoing for about 1h30min. Impacted multiple paid workspaces of ours. Here’s what it looks like on the uptime monitor:

The same thing happened yesterday as well.

It’s one thing for the editor to be down or slow, and another one completely for the published site to be un reachable or taking >5 seconds to respond. Stay like that for an hour or more, without any contact possible with the support team on short notice.

This brings some other issues to the surface:

  • The current chat interface to reach support stalls communications when new issues arise. Let’s say We’re troubleshooting issue1 which isn’t urgent and which is assigned to Specialist1. What happens when very urgent issue2 arises and Specialist1 isn’t online? How do I separate discussions in there? It’s a single thread. Not well suited for support. Needs at least to provide multiple conversations that are treated like « tickets ».
  • There is no system status page, performance/uptime monitor.
  • There is no way of cutting through and getting access to quick support in an event when there is downtime and a quick response is required.

Anyways, everything seems to be back online for now but this needs a bit more serious attention IMO.

1 Like

I’m currently in Montreal, I haven’t seen it or maybe missed it somehow. Can you please share where did you get that chart of a response time?

This is hosted on premises. I also have one hosted on DigitalOcean monitoring from US-East/West, SE-Asia and EU-West. All gave the same signals, at least for our workspaces (monitoring the published site URLs).

Oh and, I’m from Quebec City. Would be happy to connect eventually!

1 Like

Following this thread.

My software serves enterprise clients and I’m migrating away from Bubble precisely because of reliability issues and if this is occurring on WeWeb as well, it’s a major concern for me.

Coming from an operations background, I’ve given feedback to the WeWeb team before about their support. There’s no proper triage and ticket handling in place, they’re relying on a free tool (Crisp), which really is meant for small businesses.

If WeWeb wants to serve as a platform powering serious businesses, they need to up their operations and reliability or be stuck in the same rut as Bubble.

2 Likes

Hello @gbeaudoin, we are looking into this as we speak.

Everything seems to be working well right now and we did not receive any notification from our infrastructure monitoring system over the week-end, thus the long silence, apologize for that.

We will come back to you on this thread when we understand where the issue came from.

2 Likes

@Raphael
Just now I’ve caught similar issue (Im located in Canada East). Publishing goes to 50% and then just stop and doesn’t update published version.

@Raphael The same thing has started happening again 30 minutes ago. I’m really hoping you guys are working on this and we’re not going through hours of downtime or an otherwise unusable app because of latency like last week-end.

Please let me know if you’re aware of this and working on it.

What was the outcome of this?

I have also experienced this and reported it to the WeWeb team.

1 Like

I’m in touch with Raphael. I think there was some confusion with another issue that we had where the Weweb app was falling in a loop of requests on our back-end. This seems to be resolved.

Originally the feedback from Weweb was that we were the only ones with this performance issue, but now this confirms that it isn’t something related to our workspaces (because it did impact our 2 workspaces). Thanks for speaking out Wes!

1 Like

I think I also had this same issue… Was WeWeb just making constant calls to your back-end (Xano, in my case)? By switching a collection to not fetching automatically, we were able to resolve that issue.

Hi everyone,

Apologies for the confusion. There are two areas that were affecting performances that we were exploring in parallel.

One is the WeWeb apps calling Collections multiple times. This has been fixed last week and should no longer be happening.

The second is causing downtime for a small number of users (it seems located on the East Coast of North America but we are not 100% sure so if you are based elsewhere, please do let us know).
Resolving that second issue is our #1 priority. It is due to our servers experiencing more and more traffic, which can cause this kind of downtime.
We improved them a lot this morning and they are now able to sustain a much higher load. We still have the possibility to improve them if necessary.
At the same time, we are in discussion with AWS experts to help us improve our infrastructure even more and be able to better predict the load peaks that we may experience in the future.

I hope this helps clarify what’s going on. Again, apologies for the confusion between the two separate issues. Please continue to reach out to us if you experience any issues on your side.

3 Likes

Yes that was exactly it. However as Marc said, it seems like this has been fixed as of last week.

And thanks @Marc for the detailed follow-up. This is good news!

Hey @Marc, thank you very much for the update.

I had some additional downtime yesterday (1 hour each on two different projects), and I’ll reach out to support about this.

1 Like

We had issues as well past week-end again, but this time it was much less important. On the 9th, the site faced multiple downtimes from 6:30 to 8:00 am, but was mostly available.

Just wanted to come back here and say that we haven’t had any issues at all for the past 3 weeks. Seems resolved entirely, thanks!

4 Likes

I’m in the same boat—haven’t had any problems over the last few weeks. Thanks, WeWeb team!

3 Likes