Supabase data fetching (over 1000 items)

Hi,

I found out that I can overcome the Supabase rows limitation directly in the Supabase settings (instead of 1000, I set it up to 10000). This setup works now, however, I might run into similar issue later once the 10000 limit is exceeded.

My setup is following:

  • I have a table of orders (table: orders)
  • Each order has X related errors (table: errors)
  • Each error has a relation to a specific order directly in Supabase

I am currently loading errors data into a Datagrid with the defined filter based on a variable “order_id”.

This works pretty well and I only see these rows that I want to.

However, I am concerned about the total rows loaded from Supabase? Currently it’s over 1k. In the near future, once the product is more utilized, I might see like 50k rows and I don’t think it’s a good idea to load such amount of database rows everytime the app is used (or maybe even everytime when I load a new page in the app?).

When I tried to set up similar filter directly in the Supabase plugin settings, it didn’t actually work (it just loaded same data for one particular “order_id” and didn’t change dynamically).

So my question is - what is the best practise in this case?

Thank you,

Lukas

Hey, you could use pagination or set up filters directly on the collection before it’s fetched. Or you could create an edge function if you need more complex filtering. In any case, you should not fetch more than the recommended default limit of 1000 rows imo.

HI William,

thanks! Do you mean to set up filters right in the query configuration of collection?

Because I tried it and the issue is that records are not loaded dynamically. I used the same condition to filter the records based on “order_id” (which is a column in DB - a link between errors table and orders table) but it only loaded records for one particular order, it didn’t change dynamically based on the variable. Although when I looked at the current variable, the value was correct and dynamically loaded. It only loaded records filter based on specific “order_id”=79 which was the last order ID in the table.

Do you have a fetch collection action when loading the order? If not, try adding one

This is an issue for me when downloading data. If you have 1002 record you want to include in an excel file? you can’t really use pagination…

This works!!

Thank you very much :slight_smile:

you can load it in batches instead or better yet do it in the backend using edgefunctions rather then getting the front end to build spreadsheets