How to handle File uploads

[Joetib]

I took unicorn for a spin just a night ago and I've really taken a liking to it. However, reading throughout the documentation, I do not see anything about file uploads. I have some forms that I want simplified with unicorn's magic. Am I missing something? How do I approach this?

[adamghill]

Hey @Joetib, thanks for checking Unicorn out! You aren't missing anything -- I haven't added anything to help with file uploads... yet. But, thanks for making this issue, I'll see if I can add something to support them in the next release or two.

[Joetib]

Thanks for tending to this quickly. However can you suggest some workaround I can do? Perhaps with Django forms that won't interrupt with unicorn

[adamghill]

Yep, you can mix a Django ModelForm in and it shouldn't interfere with Unicorn to have the file upload post like normal, or use a third-party uploader and bypass Django altogether (e.g. https://uploadcare.com/).

Sorry I don't have better news, but I will try to work on this next -- I just can't promise when it will be done. :) Making a file upload that "just works" is going to require some work (see the hoops that Livewire jumps through).

[adamghill]

I attached #167 with some initial code, but still needs a lot more thought to make sure it will all work as expected. But, it's in process!

[Joetib]

That's very good news I'm excited. By the way, could you possibly have a document explaining how to contribute. I would love to understand and add a contribution to this project. I love it.

[Joetib]

I checked the pr. Is there a way to get it to test?

[adamghill]

Is there a way to get it to test?

Nope, nothing to test, yet! I just marked the PR as a draft because it's still early. I'll let you know when I get all the pieces working together, though.

could you possibly have a document explaining how to contribute. I would love to understand and add a contribution to this project.

That's a great idea. https://github.com/adamghill/django-unicorn/blob/master/DEVELOPING.md#developing is the basic gist, but I will create a more comprehensive document with more details.

[Joetib]

Thank you. I will be waiting for a more detailed guide explaining how the pieces fit together

[adamghill]

I threw this architecture doc together to give a rough overview of how it all works, but let me know if you have specific questions or would like more details about any area.

[Joetib]

I had a read through the architecture doc. I understand better now.

[lorddaedra]

Very opinionated but still helpful comment I guess: use Google Cloud Storage or any other object storages, https://googleapis.dev/python/google-resumable-media/latest/resumable_media/requests.html#resumable-uploads

IMHO it's not high priority issue...

[Joetib]

I'm sorry but this issue already has draft PR on it. I think that is enough and the "it's not a high priority issue..." isn't that necessary. Oh and I've been able to split my problem up and fixed them with a separate upload step that directly uses Django forms

[lorddaedra]

1. About Django forms - I'm Django forms hater or something like that (I like Django ORM but not Django forms.
   a.) forms mix forms templates with data validations, you can't disable templates and use only validation, it looks ugly because of bad composition, this problem is actual for Django in general (this design is old and comes from 00's to 2021 year)
   b.) performance, I like Pydantic much more for validation, also you can check benchmarks https://pydantic-docs.helpmanual.io/benchmarks/.
   c.) I do not like use Python or JS/TS for generating html code, it's ugly as for me, what's why I like Vue.js experience (with html) and do not like React or Django forms. Access to html structure allows to integrate html with TailwindCSS and some custom UI/UX libraries much simpler. Generating html form with Python code - it's a little war. Much better is just place html code in Django Templates (or Jinja2)...

I tell about Django forms because of you commented

Perhaps with Django forms

2. About file uploads to Django - in most cases in 2021 it looks as not good idea - uploading files to Python WSGI/ASGI-frameworks (or any kinds of frameworks on PHP, Elixir, Ruby etc.) instead of uploading to object storages. Usually it's much better to upload file to AWS/Google Cloud/Azure/etc (or use open source projects like MinIO/etc. if you like them more than clouds) using temporary link. (example with uploading files to MinIO: https://docs.min.io/docs/upload-files-from-browser-using-pre-signed-urls.html)

Its a good idea to deprecate forms Django usage and suggest to use Pydantic for validation instead, some object storages for uploading files etc.

I'm sorry but this issue already has draft PR on it.

Yes but may be we should deprecate this feature at all and suggest to use better alternatives... Sorry, I'm new user of this project, so I didn't seen this issue 2 Mar (issue creation date) but better to say now and prevent from doing something ugly than never.

Just opinion.

[adamghill]

you can't disable templates and use only validation

I use Django Forms only for validation now (I don't use the widgets at all). You can see an example here: https://www.django-unicorn.com/docs/validation/. I agree that Django Forms mix validation with presentation, but for just using that as a validation layer it seemed like a good first approach. For anyone with an existing Django site or going through the tutorial they will most likely already use/have a Django Form class.

I like Pydantic much more for validation

I already support the Pydantic base model for serialization. Using it for validation as well is a possibility. I don't particularly want to build a bunch of different ways that validation could be enabled (I'd rather have "one way" to do it). But, it's still a good idea. If you want to make a separate issue just with that request, that makes sense to me.

Much better is just place html code in Django Templates

Yep, Unicorn follows this exact philosophy.

in most cases in 2021 [form uploads] looks as not good idea

I sort of agree that there are (maybe better) ways to do a file upload, but I think having a drop-in solution would also be useful as part of Unicorn. I'm still thinking about what the right features for v1 could/should be!

[lorddaedra]

I use Django Forms only for validation now (I don't use the widgets at all).

But Django disagree with you https://github.com/django/django/blob/main/django/forms/forms.py#L98-L107 :-)
Whats why I said about Django has bad design (composition of modules) and about it's good to avoid Django forms - Django just initialise it for you even if you do not use them. You can't disable form renderer in settings or something like that.
But, yes, I agree, for first approach it's good because of usually you think about time to market (feature shipment time) and not about good software design. Django is good system for creating MVPs (minimum viable product) for any kind of startups/bootstraps... This is the reason why do I like Django despite of some design decisions.

I like modular design, unix way (https://en.wikipedia.org/wiki/Unix_philosophy) and may be DDD. That's why I do not suggest read Django docs usually. Instead, I suggest to read https://phalt.github.io/django-api-domains/. I's about pragmatical way to integrate DDD ideas with Django modern development. Do not use fat models, use Django ORM only in service files etc. I think despite the fact this guide is about API development, most of this guide can be useful for django-unicorn based projects too. django-unicorn help to create layer, which may replace some of APIs. With django-unicorn you create components instead of API but other layers are the same. Also you may still need API for some partners integrations, cli-tools and mobile apps...

Proper composition is matter. It also allow you to switch, for example, Django to Starlette or FastAPI (which is based on Starlette and extend it as DRF do with Django) in future. (Because of https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=fortune&l=zijzen-sf benchmark result)

You project support AJAX instead of WebSockets. It may become big benefit for some serverless environments.
It will be really good if in some future version it will be possible to use Starlette/FastAPI+Jinja2+Pydantic+your project together. Do you like Django? Okay. Do not like Django? Okay too, take anything else and attach. It will be hard to do if you will integrate very closely with Django. Be the best Python component framework looks better goal than be just app to Django IMHO. So may be Django and Django forms are good starting point and good as default settings (press button and it's just works) but I would like to be able to configure modules and change default settings to use the best python libraries. Django provide compatible to each other components but usually they are not the best in the Python ecosystem (even more, usually it's opposite). Usually on big projects people start from Django and replace it later to microservices, Django project becomes bff (backend for frontend) with only User model and permissions or even without any models at all (just proxy to internal services and ORY). What's with django-unicorn in this case? I like an idea to create frontend with Python on server side. Take Vue.js and replace js to python. Server side Vue.python :-) This layer do not require database by default but may require some cache/session store... If you do not require database then you do not need Django ORM, but if you do not need Django ORM then you do not need Django (Django ORM and related apps (auth, admin) are only Django benefit IMHO for not new projects). So if you will think about "why do I need Django?" then it may be good idea to take your components framework and attach to something else (like Starlette). If it will not be possible to do then it may be good idea to just use SSR Vue.js with Node.js or something. But I think it's possible to create Python based component framework for highload production use. May be it's your project. May be it's not your goals...
So, TL;DR Django is good. But it's good for start. For long distances with server component framework Starlette is better. With Starlette you will not have access to Django Template (but you can add Jinja2 instead) or Django Forms (but you can add Pydantic validation instead).
So main question here - is it project for newbies only and good for freelance and MVP projects or it's good for any stages of project?.. Both answers are okay.
In first case it's completely okay to use only Django components and prefer them other anything else from python ecosystem. In second case it could be optimised (default settings values I mean) for MVP level projects with Django. But it should have some ways to use it with some modern systems (may be not Django).
Why do I like DDD and unix way? In this case you just say: it's component/library, it's not a framework. Use it as you want. We focus on server-side components and do it in the best available way. It's core functionality. Also we have some helpers for Django etc. So you may use them or not. Or use something else for data validation etc. Django it not part of core, it's just set of helpers/adapters, just Django extension to some core Python components module.

It's just my opinion about what would I like to get as user. I would like to get Vue.python (== Vue.js.replace('typescript', 'python') :-) ) without any dependencies to Django or anything else. And some Django extension. Just install and it works.

This may or may not be your goal to be useful for big companies and big projects. But if it's your goal then modular design is requirement and in this case it's good to minimise dependencies to Django components.

Anyway I see it's good solution for MVP and it's already very useful for many projects, good job.

[adamghill]

is it project for newbies only and good for freelance and MVP projects or it's good for any stages of project?

For right now my goal is to be the best reactive component library for Django. That alone is an ambitious enough side project! :) I use Django in my day job and for all of my side projects, so it's what I know and what I want to use.

I do take your comments to heart about it being modular and being able to use Unicorn outside of the Django ecosystem, but that's not an explicit goal. The JavaScript is completely agnostic and the view that processes the AJAX calls could be agnostic of Django with some effort and work. It's just not going to be a priority for me any time soon.

I welcome any and all PRs, though! :)

[lorddaedra]

I use Django in my day job and for all of my side projects, so it's what I know and what I want to use.

Django is good system for a lot of projects, I completely agree.

I do take your comments to heart about it being modular

It's hard to "sell" system with, for example, only Django validators to team. They think about "that component will increase vendor lock-in to Django". So that marketing reasons exist too...

could be agnostic of Django

It can be even Django but frontend version of Django (with Redis cache may be), without models, with GraphQL Python client to get/send data to internal GraphQL API, for example. (or OpenAPI or gRPC)

For right now my goal is to be the best reactive component library for Django.

Again, great project. I like it. There are some reasons (https://cloud.google.com/run/docs/triggering/websockets#multiple-instances) why do I prefer AJAX over WebSockets. I hope it's safe to use this project with multiple instances...

One more suggestion also - it's not clear how to test components. May be need add doc page about it. IMHO we should not mix this frontend components with services (some business logic) and auth system (checking permissions), it's different layers. Good example about how to test only frontend part of system (server components and browser scripts). I see django-unicorn as alternative to Vue.js or Vue.js with Node.js (SSR). May be I can reuse some examples here: https://github.com/adamghill/django-unicorn/tree/master/tests

P.S. Sorry for offtopic in this issue.

[winrid]

For those browsing, one way to do file uploads with Django Unicorn is just via base64 encoding the file. This is not super efficient but for my case (<10mb images) it works fine. Create a JS file:

(function () {
    document.addEventListener('input', function (e) {
        const isFileInput = e.target && e.target.type === 'file';
        if (!isFileInput) {
            return;
        }
        const fileInput = e.target;

        if (fileInput.files.length > 0) {
            const file = fileInput.files[0];
            console.log('Uploading file', file);
            const fileSizeMB = file.size / (1024 * 1024); // Size in MB

            if (fileSizeMB > 10) {
                alert('File size exceeds the limit of 10MB.');
                return;
            }
            const reader = new FileReader();

            reader.onload = function (event) {
                fileInput.parentNode.querySelector('input.file-data').value = event.target.result;
                fileInput.parentNode.querySelector('input.file-data').dispatchEvent(new Event('input', {bubbles: true}))
                fileInput.parentNode.querySelector('input.file-name').value = file.name;
                fileInput.parentNode.querySelector('input.file-name').dispatchEvent(new Event('input', {bubbles: true}))
            };

            // Read the file as a data URL (base64)
            reader.readAsDataURL(file);
        } else {
            alert('Please select a file');
        }
    });
})();

Have some sort of input like:

    <label>
        Upload File
        <input type="file" accept="image/*" required/>
        <input type="hidden" class="file-data" unicorn:model="new_media_base64" />
        <input type="hidden" class="file-name" unicorn:model="new_media_file_name" />
    </label>

Then when a user clicks a save button we have a python method in the component like:

    image_data = self.new_media_base64.split(';base64,')[-1]  # Extract the base64 image data
    decoded_image = b64decode(image_data)
    if len(decoded_image) > 10 * 1024 * 1024:  # 10MB limit
        raise "File exceeds the 10mb limit."
    self.new_media_err = None
    media = Media(
        name=self.new_media_name,
    )
    media.src.save(self.generate_unique_media_name(), ContentFile(decoded_image), save=True)