Interesting. Where are you located? I’m on the Canadian east coast. This might be a localized infra issue.
Downtime started a little more than 3 hours ago and have been ongoing for about 1h30min. Impacted multiple paid workspaces of ours. Here’s what it looks like on the uptime monitor:
It’s one thing for the editor to be down or slow, and another one completely for the published site to be un reachable or taking >5 seconds to respond. Stay like that for an hour or more, without any contact possible with the support team on short notice.
This brings some other issues to the surface:
The current chat interface to reach support stalls communications when new issues arise. Let’s say We’re troubleshooting issue1 which isn’t urgent and which is assigned to Specialist1. What happens when very urgent issue2 arises and Specialist1 isn’t online? How do I separate discussions in there? It’s a single thread. Not well suited for support. Needs at least to provide multiple conversations that are treated like « tickets ».
There is no system status page, performance/uptime monitor.
There is no way of cutting through and getting access to quick support in an event when there is downtime and a quick response is required.
Anyways, everything seems to be back online for now but this needs a bit more serious attention IMO.
This is hosted on premises. I also have one hosted on DigitalOcean monitoring from US-East/West, SE-Asia and EU-West. All gave the same signals, at least for our workspaces (monitoring the published site URLs).
Oh and, I’m from Quebec City. Would be happy to connect eventually!
My software serves enterprise clients and I’m migrating away from Bubble precisely because of reliability issues and if this is occurring on WeWeb as well, it’s a major concern for me.
Coming from an operations background, I’ve given feedback to the WeWeb team before about their support. There’s no proper triage and ticket handling in place, they’re relying on a free tool (Crisp), which really is meant for small businesses.
If WeWeb wants to serve as a platform powering serious businesses, they need to up their operations and reliability or be stuck in the same rut as Bubble.
Hello @gbeaudoin, we are looking into this as we speak.
Everything seems to be working well right now and we did not receive any notification from our infrastructure monitoring system over the week-end, thus the long silence, apologize for that.
We will come back to you on this thread when we understand where the issue came from.
@Raphael
Just now I’ve caught similar issue (Im located in Canada East). Publishing goes to 50% and then just stop and doesn’t update published version.
@Raphael The same thing has started happening again 30 minutes ago. I’m really hoping you guys are working on this and we’re not going through hours of downtime or an otherwise unusable app because of latency like last week-end.
Please let me know if you’re aware of this and working on it.
I’m in touch with Raphael. I think there was some confusion with another issue that we had where the Weweb app was falling in a loop of requests on our back-end. This seems to be resolved.
Originally the feedback from Weweb was that we were the only ones with this performance issue, but now this confirms that it isn’t something related to our workspaces (because it did impact our 2 workspaces). Thanks for speaking out Wes!
I think I also had this same issue… Was WeWeb just making constant calls to your back-end (Xano, in my case)? By switching a collection to not fetching automatically, we were able to resolve that issue.
Apologies for the confusion. There are two areas that were affecting performances that we were exploring in parallel.
One is the WeWeb apps calling Collections multiple times. This has been fixed last week and should no longer be happening.
The second is causing downtime for a small number of users (it seems located on the East Coast of North America but we are not 100% sure so if you are based elsewhere, please do let us know).
Resolving that second issue is our #1 priority. It is due to our servers experiencing more and more traffic, which can cause this kind of downtime.
We improved them a lot this morning and they are now able to sustain a much higher load. We still have the possibility to improve them if necessary.
At the same time, we are in discussion with AWS experts to help us improve our infrastructure even more and be able to better predict the load peaks that we may experience in the future.
I hope this helps clarify what’s going on. Again, apologies for the confusion between the two separate issues. Please continue to reach out to us if you experience any issues on your side.
We had issues as well past week-end again, but this time it was much less important. On the 9th, the site faced multiple downtimes from 6:30 to 8:00 am, but was mostly available.