Advice on image/video storage for large amounts of data

Ask us anything How do I?
1

Assuming my app works out I can expect to have several terabytes of images and videos that will be stored, roughly 3-4 GB per user. Obviously I cant store that many images in Xano so I looked into amazon S3. The issue is I am not sure how the right way to go about connecting it would be.

In my head I am thinking the images and video data get sent straight to S3

  1. Would I use the rest api?
  2. can I add the user id to the images?

then I pull the URL from S3 into xano somehow matching the correct images to the correct user by ID. Then just display the URL in Weweb…that bit I can do.

Is this correct or is there a better way of doing it?

I would appreciate some advice before I spend a few hours trying to work this out.
Thank you

2

My app users upload a good amount of images. The way I have it setup is to run it through Xano so that my Google Cloud Storage (GCS) auth key is not accessible in the front-end. I would assume AWS is similar.

My endpoint looks like this:

Inputs: content (file resource), GCS folder name, several app specific parameters

  1. Create variable for filename (I use a concat of app parameters)
  2. GCS Upload File function
  3. Create variable for file url using GCS root, bucket, and filename
  4. Edit record on table where I want to store url

Technically, I allow multiple images to be sent to Xano, so I loop through the content input variable using the process above.

3

From what I was reading last night GCS is easier to set up over AWS so I might go with them instead.

Thank you for sharing, Ill try and work through this today, I will say my xano skills are pretty beginner but I will see if I can manage this.

Thank you for the reply.

4

The only thing that took a bit of work/research with GCS was figuring out how to make files have public sharing, but not the directory. My filenames have a uuid, so guessing at the filename is near impossible. The process to do this required using the GCS console to make the changes. Wish I saved the link to the guide I used.

5

that does sound like a pain, thanks for the warning.

6

was there a reason you chose GCS over AWS?

7

No real reason except I use Google for other business functions (ads, analytics, email) and with a novice look at pricing for my use-case it seemed inconsequential.

1 Like
8

Yes, sending the images directly to S3 and storing the URLs in Xano is the correct approach. Use the REST API to upload, add user IDs as metadata, and link them in Weweb for display.

1 Like
9

I’m just now am getting back to the images on my app. i switched backends to Supabase so had to rebuild a lot of things.

is there any info on how to set up a workflow for image storage in GCS and then showing the necessary data in supabase? as I am a little lost on the process. Or is that even necessary? It seems supabases price per GB is comparable to GCS.

im just trying to figure out whats best for the long term.

thank you…