CSV / JSON upload

[Ohad]

I know this has been discussed a couple of times, and I read through and still couldn’t figure it out. I want to enable my customers to upload and manage their data that is then used within the WeWeb application. I’m a step before Authentication so that’s not the barrier.

The data comes in as CSV or JSON. Some of the headers still need to be mapped - e.g., “id”., “name” so that i have it in a pre-set naming convention. I have a table ready in Xano to receive the data as draft, and in it’s stable formed once the mapping is completed.

Effectively i want to following steps:

1. User uploads a CSV/JSON to WeWeb (I know how to do it with an image, but not in a way that i can read the headers - optional: save draft in Xano)
2. I read the headers into a WeWeb array
3. Header mapping using the dropdowns in the picture, making sure the required fields are covered
4. I take this info and place it in schema within a Xano table
5. I read this table as one of my collections

Would love to get your help!

[Joyce]

Hi @Ohad

Just to keep you posted: I haven’t forgotten about you but I’m still thinking about what the best approach would be

Let me ask the team for help on this one!

[a_g]

Is there an update on how to do this please @Joyce cc @Ohad

I have a very similar workflow and am not sure how to realise it in WeWeb.

Thank you

[ericp]

Bringing this topic back to the top, I’d also like to know any tips on how to setup a user upload of a csv in which they can self-map the columns to the correct fields in Xano.

Anyone build this yet?

[Joyce]

Hey there

Sorry I didn’t see the updates on this topic sooner Thanks for taking the time to follow up!

It is indeed possible but we don’t have a tutorial for this. Love the use case though! We’ll work on a tutorial in the coming weeks

[Riisud]

Any update here yet?

[andrii_r]

Hi @Riisud, @Ohad, I’m not sure if this is the best solution, but you can read the uploaded CSV file using JavaScript through Base64 string decoding

Workflow inside the File upload element (On change):

Custom JavaScript code:

const encodedStr = context.workflow['812583ea-7028-4fd2-b157-3731195b2f90'].result; // ENCODE result

function decodeBase64(encodedStr) {
    try {
        // Decode base64 to bytes
        const bytes = Uint8Array.from(atob(encodedStr), c => c.charCodeAt(0));

        // Check for BOM and remove it if present
        let finalString;
        if (bytes[0] === 0xEF && bytes[1] === 0xBB && bytes[2] === 0xBF) {
            finalString = new TextDecoder('utf-8').decode(bytes.slice(3));
        } else {
            finalString = new TextDecoder('utf-8').decode(bytes);
        }

        // Split into lines and remove any carriage return characters
        const lines = finalString.trim().split(/\r?\n/);

        // Extract headers and normalize them
        const headers = lines[0].split(',').map(header => header.trim().toLowerCase());

        const requiredHeaders = ['first_name', 'last_name', 'email', 'phone'];
        const missingHeaders = requiredHeaders.filter(header => !headers.includes(header));
        
        if (missingHeaders.length > 0) {
            throw new Error(`Missing required headers: ${missingHeaders.join(', ')}`);
        }
        
        // Process data lines
        const data = lines.slice(1).map(line => {
            const values = line.split(',').map(value => value.trim());
            const obj = {};
            headers.forEach((header, index) => {
                obj[header] = values[index];
            });
            return obj;
        });
        
        // Ensure all required fields have values
        data.forEach(item => {
            requiredHeaders.forEach(header => {
                if (!item[header]) {
                    throw new Error(`Missing value for ${header} in line: ${JSON.stringify(item)}`);
                }
            });
        });

        // Map data to final structure
        const finalData = data.map(item => ({
            realtor_first_name: item.first_name,
            realtor_last_name: item.last_name,
            realtor_email_address: item.email,
            realtor_phone_number: item.phone
        }));

        return finalData;
    } catch (error) {
        throw new Error(`${error.message}`);
    }
}

return decodeBase64(encodedStr);

Result:

With these actions, you can retrieve records from a CSV file and use them