Skip to content

Blazor Server Large File Upload Support - #33900

Merged
TanayParikh merged 10 commits into
mainfrom
taparik/largeFileInterop
Jul 9, 2021
Merged

Blazor Server Large File Upload Support#33900
TanayParikh merged 10 commits into
mainfrom
taparik/largeFileInterop

Conversation

@TanayParikh

@TanayParikh TanayParikh commented Jun 28, 2021

Copy link
Copy Markdown
Contributor

Currently we have a hard limit of Int32.MaxValue = 2147483647 bytes on the file size we can upload. This applies for Blazor WASM and Blazor Server.

The goal of this PR is to remove this limit, using the Blob.slice API. Originally I had planned on using Blob.stream however that had browser compatibility issues and added complexity when piping between the file stream and the JS<>.NET interop stream.

This specific PR is for Blazor Server. I'm working on bringing streaming to BlazorWasm and will integrate this new approach in that PR for BlazorWasm specifically.

Testing: We already have E2E testing for the input file functionality.

It leverages the Blob.stream API, using the stream we can getReader which returns a ReadableStreamDefaultReader. The ReadableStreamDefaultReader itself is still marked 'experimental' but has been supported in Edge/Chrome/Firefox 79/78/65, however is missing in IE/Safari (will need to test what happens with this in Safari).

Part of: #33638

reader.onerror = function(err): void {
reject(err);
};
reader.readAsArrayBuffer(file['blob']);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't allocate a buffer larger than ~2GB so this'll fail for large files.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this back in for now so this PR isn't blocked on WASM streaming interop. Will remove it in that PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this back in for now so this PR isn't blocked on WASM streaming interop. Will remove it in that PR.

Are you saying that, as of this PR, we still won't support >2GB files but it will nearly support it and then when you do the WASM piece it will support it on Server too?

If so that's totally fine - just want to check I understand.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As of this PR we'll support >2GB in server and up to the current ~2GB limit in WASM. With the new WASM PR, I have >2GB working in WASM as well (using this Blob.slice approach).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TanayParikh where can I find this "the new WASM PR" and information weather Blazor WASAM 6.0 supports >2GB upload or not?

I can't find any documentation or examples on uploading >2GB file in WASM to an API (preferably with progressive information for UI display)? The docs on Blazor file upload doesn´t event mention the <2GB limit

Its totally unclear if this works now in .net 6.0 , just like I think you are pointing out here.

The internet is littered with questions like this and this and no answers.

Hope you can point me in the right direction.

@TanayParikh TanayParikh Dec 20, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where can I find this "the new WASM PR"

#33986

weather Blazor WASAM 6.0 supports >2GB upload or not?

It does.

I can't find any documentation or examples on uploading >2GB file in WASM to an API (preferably with progressive information for UI display)?

The process is the same as you would take to upload files < 2GB. We recently added docs for showing progress for uploads in Blazor server (here), you should be able to use that as guidance for implementing something similar in Blazor WASM.

The docs on Blazor file upload doesn´t event mention the <2GB limit

The 2GB limit in .NET 5 and before was un-intended and un-documented behavior. This has been resolved in .NET 6. I've created an issue to add a notice for this limitation in .NET 5 to the docs. dotnet/AspNetCore.Docs#24381

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks for the update. This helped me allot deciding on my next steps

Comment thread src/Components/Web.JS/src/Platform/Circuits/CircuitStreamingInterop.ts Outdated
Comment thread src/Components/Web.JS/src/InputFile.ts Outdated
@Pilchie Pilchie added the area-blazor Includes: Blazor, Razor Components label Jun 28, 2021
@TanayParikh
TanayParikh marked this pull request as ready for review June 28, 2021 17:32
@TanayParikh
TanayParikh requested review from a team and Pilchie as code owners June 28, 2021 17:32
@TanayParikh TanayParikh changed the title Prototype Large File Upload Support Blazor Jun 28, 2021
Comment thread src/Components/Web.JS/src/InputFile.ts Outdated
Comment thread src/Components/Web.JS/src/InputFile.ts Outdated
Comment thread src/Components/Web.JS/src/Platform/Circuits/CircuitStreamingInterop.ts Outdated
Comment thread src/Components/Web/src/Forms/InputFile/BrowserFileStream.cs Outdated

@javiercn javiercn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

Only have a few comments left

Comment thread src/Components/Web.JS/src/InputFile.ts Outdated
Comment thread src/JSInterop/Microsoft.JSInterop.JS/src/src/Microsoft.JSInterop.ts
Comment thread src/JSInterop/Microsoft.JSInterop.JS/src/src/Microsoft.JSInterop.ts Outdated
@TanayParikh
TanayParikh requested a review from MackinnonBuck July 2, 2021 00:03
@TanayParikh
TanayParikh requested a review from a team July 2, 2021 00:03
@TanayParikh

Copy link
Copy Markdown
Contributor Author

@javiercn @SteveSandersonMS could I please get a review, hoping to get it in for Preview 7.

Comment thread src/Components/Web.JS/src/Platform/Circuits/CircuitStreamingInterop.ts Outdated

@SteveSandersonMS SteveSandersonMS left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! One or two minor comments added, but should be basically ready to go :)

@SteveSandersonMS

Copy link
Copy Markdown
Member

I think you spotted one of the comments - the other is at #33900 (comment) in case it gets lost.

@TanayParikh
TanayParikh enabled auto-merge (squash) July 9, 2021 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-blazor Includes: Blazor, Razor Components

5 participants