Web app large uploads and downloads
Folks (anyone working?) I've been asked to add a feature to a Blazor Webassembly app to allow uploads and downloads of possibly large numbers of files between the local file system and Blob storage. I'm not sure how to implement this feature in a browser hosted app. I wrote a WPF tool for "managers" which does high-performance bulk uploads and downloads with nice progress (the code is trivial on the desktop), but now they want the same feature for "normal" users in the Blazor app. Given how dumb and restricted browser hosted apps are, I don't know how to code this, or if it's even feasible. Are there some tools, techniques or tricks I can apply? Any ideas or suggestions anyone? Thanks, *Greg Keogh*
Hi Greg, Are you using any Blazor frameworks? Several of them seem to have pretty good file upload components. That might give the best experience. Regards, Greg Dr Greg Low 1300SQLSQL (1300 775 775) office | +61 419201410 mobile SQL Down Under | Web: https://sqldownunder.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc&e=> | About Greg: https://about.me/greg.low<https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk&e=> From: Greg Keogh via ozdotnet <ozdotnet@ozdotnet.com> Sent: Saturday, December 30, 2023 8:58 AM To: ozDotNet <ozdotnet@ozdotnet.com> Cc: Greg Keogh <gfkeogh@gmail.com> Subject: Web app large uploads and downloads Folks (anyone working?) I've been asked to add a feature to a Blazor Webassembly app to allow uploads and downloads of possibly large numbers of files between the local file system and Blob storage. I'm not sure how to implement this feature in a browser hosted app. I wrote a WPF tool for "managers" which does high-performance bulk uploads and downloads with nice progress (the code is trivial on the desktop), but now they want the same feature for "normal" users in the Blazor app. Given how dumb and restricted browser hosted apps are, I don't know how to code this, or if it's even feasible. Are there some tools, techniques or tricks I can apply? Any ideas or suggestions anyone? Thanks, Greg Keogh
How about this one? https://demos.devexpress.com/blazor/Upload On Sat, Dec 30, 2023 at 10:06 AM Dr Greg Low via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Hi Greg,
Are you using any Blazor frameworks? Several of them seem to have pretty good file upload components. That might give the best experience.
Regards,
Greg
Dr Greg Low
1300SQLSQL (1300 775 775) office | +61 419201410 mobile
SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>
*From:* Greg Keogh via ozdotnet <ozdotnet@ozdotnet.com> *Sent:* Saturday, December 30, 2023 8:58 AM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* Greg Keogh <gfkeogh@gmail.com> *Subject:* Web app large uploads and downloads
Folks (anyone working?)
I've been asked to add a feature to a Blazor Webassembly app to allow uploads and downloads of possibly large numbers of files between the local file system and Blob storage. I'm not sure how to implement this feature in a browser hosted app.
I wrote a WPF tool for "managers" which does high-performance bulk uploads and downloads with nice progress (the code is trivial on the desktop), but now they want the same feature for "normal" users in the Blazor app. Given how dumb and restricted browser hosted apps are, I don't know how to code this, or if it's even feasible.
Are there some tools, techniques or tricks I can apply? Any ideas or suggestions anyone?
Thanks,
*Greg Keogh* -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
Folks (welcome to the distant future of 2024) I'm pretty sure that there is no sensible way to provide a bulk Azure Blob upload facility in a web hosted app (Wasm, JS or whatever). There could be thousands of files with a total size up to hundreds of MB. Managers are currently using a WPF program I created for uploads and it feels like a perfectly natural process on the desktop, and it's pretty fast using streams on multi cores (I also optionally check for new or changed so only *diffs* are uploaded which often saves a lot of time). None of the components or controls I've seen are designed for huge uploads, and in any case, I've reported that it's technically and usability questionable to have "normal" users of the browser app doing this sort of thing. The boss of the app suite is now considering the bigger picture and the bulk upload feature may be delayed or moved to somewhere else in the flow, or the desktop program will suffice. So I'm happy the issue is on-hold for now. I think this is a good example of how the web browser should never have evolved into a host for business apps. I think the web browser, HTML, HTTP, REST, css and JS have diseased 21st century IT. *Greg K*
Yep we usually do this sort of thing with a batch process, particularly if there are large files. I don’t see any web app being used just to upload a bunch of files. For fun try asking ChatGPT or one of the others to see where they go. On Wed, 3 Jan 2024 at 11:13, Greg Keogh via ozdotnet <ozdotnet@ozdotnet.com> wrote:
Folks (welcome to the distant future of 2024)
I'm pretty sure that there is no sensible way to provide a bulk Azure Blob upload facility in a web hosted app (Wasm, JS or whatever). There could be thousands of files with a total size up to hundreds of MB. Managers are currently using a WPF program I created for uploads and it feels like a perfectly natural process on the desktop, and it's pretty fast using streams on multi cores (I also optionally check for new or changed so only *diffs* are uploaded which often saves a lot of time).
None of the components or controls I've seen are designed for huge uploads, and in any case, I've reported that it's technically and usability questionable to have "normal" users of the browser app doing this sort of thing. The boss of the app suite is now considering the bigger picture and the bulk upload feature may be delayed or moved to somewhere else in the flow, or the desktop program will suffice. So I'm happy the issue is on-hold for now.
I think this is a good example of how the web browser should never have evolved into a host for business apps. I think the web browser, HTML, HTTP, REST, css and JS have diseased 21st century IT.
*Greg K*
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
I notice that the Azure Storage Explorer desktop app nowadays defaults to using azcopy under the covers to copy files. That’s a much better option for files of any size. Probably need a component that does that but azcopy has a few dependencies. It’s on GitHub as open source so you could probably check out how it does what it does. It’s sure fast and filesize isn’t an issue. I’ve used azcopy on multi-terabyte files without issue. (Apart from the hosting provider for the site calling to find out what was going on, given the way it worked in parallel and flooded their network) Regards, Greg Dr Greg Low 1300SQLSQL (1300 775 775) office | +61 419201410 mobile SQL Down Under | Web: https://sqldownunder.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc&e=> | About Greg: https://about.me/greg.low<https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk&e=> From: DotNet Dude via ozdotnet <ozdotnet@ozdotnet.com> Sent: Wednesday, January 3, 2024 4:17 PM To: ozDotNet <ozdotnet@ozdotnet.com> Cc: DotNet Dude <adotnetdude@gmail.com> Subject: Re: Web app large uploads and downloads Yep we usually do this sort of thing with a batch process, particularly if there are large files. I don’t see any web app being used just to upload a bunch of files. For fun try asking ChatGPT or one of the others to see where they go. On Wed, 3 Jan 2024 at 11:13, Greg Keogh via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: Folks (welcome to the distant future of 2024) I'm pretty sure that there is no sensible way to provide a bulk Azure Blob upload facility in a web hosted app (Wasm, JS or whatever). There could be thousands of files with a total size up to hundreds of MB. Managers are currently using a WPF program I created for uploads and it feels like a perfectly natural process on the desktop, and it's pretty fast using streams on multi cores (I also optionally check for new or changed so only diffs are uploaded which often saves a lot of time). None of the components or controls I've seen are designed for huge uploads, and in any case, I've reported that it's technically and usability questionable to have "normal" users of the browser app doing this sort of thing. The boss of the app suite is now considering the bigger picture and the bulk upload feature may be delayed or moved to somewhere else in the flow, or the desktop program will suffice. So I'm happy the issue is on-hold for now. I think this is a good example of how the web browser should never have evolved into a host for business apps. I think the web browser, HTML, HTTP, REST, css and JS have diseased 21st century IT. Greg K -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
I think it's been using azcopy for a fair while now. Azcopy has improved dramatically over the last few years to have more switches and options. I recently used it to bulk copy a container in one storage account to another, then bulk upload all files with certain extensions (which wasn't originally possible). In each case it was harder than refining fissionable uranium, so I pasted the skeletons of the working commands into my "bag of tricks" text file. The Blazor bulk upload is mercifully on hold while they reconsider who should have the facility -- *Greg K* On Wed, 3 Jan 2024 at 16:27, Dr Greg Low via ozdotnet <ozdotnet@ozdotnet.com> wrote:
I notice that the Azure Storage Explorer desktop app nowadays defaults to using azcopy under the covers to copy files. That’s a much better option for files of any size. Probably need a component that does that but azcopy has a few dependencies.
It’s on GitHub as open source so you could probably check out how it does what it does. It’s sure fast and filesize isn’t an issue. I’ve used azcopy on multi-terabyte files without issue. (Apart from the hosting provider for the site calling to find out what was going on, given the way it worked in parallel and flooded their network)
Regards,
Greg
Dr Greg Low
1300SQLSQL (1300 775 775) office | +61 419201410 mobile
SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>
*From:* DotNet Dude via ozdotnet <ozdotnet@ozdotnet.com> *Sent:* Wednesday, January 3, 2024 4:17 PM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* DotNet Dude <adotnetdude@gmail.com> *Subject:* Re: Web app large uploads and downloads
Yep we usually do this sort of thing with a batch process, particularly if there are large files. I don’t see any web app being used just to upload a bunch of files.
For fun try asking ChatGPT or one of the others to see where they go.
On Wed, 3 Jan 2024 at 11:13, Greg Keogh via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Folks (welcome to the distant future of 2024)
I'm pretty sure that there is no sensible way to provide a bulk Azure Blob upload facility in a web hosted app (Wasm, JS or whatever). There could be thousands of files with a total size up to hundreds of MB. Managers are currently using a WPF program I created for uploads and it feels like a perfectly natural process on the desktop, and it's pretty fast using streams on multi cores (I also optionally check for new or changed so only *diffs* are uploaded which often saves a lot of time).
None of the components or controls I've seen are designed for huge uploads, and in any case, I've reported that it's technically and usability questionable to have "normal" users of the browser app doing this sort of thing. The boss of the app suite is now considering the bigger picture and the bulk upload feature may be delayed or moved to somewhere else in the flow, or the desktop program will suffice. So I'm happy the issue is on-hold for now.
I think this is a good example of how the web browser should never have evolved into a host for business apps. I think the web browser, HTML, HTTP, REST, css and JS have diseased 21st century IT.
*Greg K*
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
I haven't tried with hundreds of files uploaded in one batch, would need to start accounting for retry if they fail. But a few files of 100mb is no problem. On Wed, Jan 3, 2024 at 11:14 AM Greg Keogh via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Folks (welcome to the distant future of 2024)
I'm pretty sure that there is no sensible way to provide a bulk Azure Blob upload facility in a web hosted app (Wasm, JS or whatever). There could be thousands of files with a total size up to hundreds of MB. Managers are currently using a WPF program I created for uploads and it feels like a perfectly natural process on the desktop, and it's pretty fast using streams on multi cores (I also optionally check for new or changed so only *diffs* are uploaded which often saves a lot of time).
None of the components or controls I've seen are designed for huge uploads, and in any case, I've reported that it's technically and usability questionable to have "normal" users of the browser app doing this sort of thing. The boss of the app suite is now considering the bigger picture and the bulk upload feature may be delayed or moved to somewhere else in the flow, or the desktop program will suffice. So I'm happy the issue is on-hold for now.
I think this is a good example of how the web browser should never have evolved into a host for business apps. I think the web browser, HTML, HTTP, REST, css and JS have diseased 21st century IT.
*Greg K*
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
I don't know Blazor, but there are a ton of pure Js file uploaders. One I like is https://www.dropzone.dev/ which should be able to be configured to upload directly to blog storage, ie not have to send the files to your webserver first https://stackoverflow.com/questions/35456197/upload-file-directly-to-azure-b... On Sat, Dec 30, 2023 at 8:59 AM Greg Keogh via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Folks (anyone working?)
I've been asked to add a feature to a Blazor Webassembly app to allow uploads and downloads of possibly large numbers of files between the local file system and Blob storage. I'm not sure how to implement this feature in a browser hosted app.
I wrote a WPF tool for "managers" which does high-performance bulk uploads and downloads with nice progress (the code is trivial on the desktop), but now they want the same feature for "normal" users in the Blazor app. Given how dumb and restricted browser hosted apps are, I don't know how to code this, or if it's even feasible.
Are there some tools, techniques or tricks I can apply? Any ideas or suggestions anyone?
Thanks, *Greg Keogh* -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
I assume you have a client Blazor wasm app that uploads the file to a server app (ie web api) which would then upload to azure or wherever. Is this what you are asking? Thanks Tom On Sun, 31 Dec 2023 at 10:19, Craig vN via ozdotnet <ozdotnet@ozdotnet.com> wrote:
I don't know Blazor, but there are a ton of pure Js file uploaders. One I like is https://www.dropzone.dev/ which should be able to be configured to upload directly to blog storage, ie not have to send the files to your webserver first https://stackoverflow.com/questions/35456197/upload-file-directly-to-azure-b...
On Sat, Dec 30, 2023 at 8:59 AM Greg Keogh via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Folks (anyone working?)
I've been asked to add a feature to a Blazor Webassembly app to allow uploads and downloads of possibly large numbers of files between the local file system and Blob storage. I'm not sure how to implement this feature in a browser hosted app.
I wrote a WPF tool for "managers" which does high-performance bulk uploads and downloads with nice progress (the code is trivial on the desktop), but now they want the same feature for "normal" users in the Blazor app. Given how dumb and restricted browser hosted apps are, I don't know how to code this, or if it's even feasible.
Are there some tools, techniques or tricks I can apply? Any ideas or suggestions anyone?
Thanks, *Greg Keogh*
--
ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
It's not a direct answer, but I'd feel happier uploading to OneDrive and passing the server a link to that. Mike On Sat, 30 Dec 2023, 08:29 Greg Keogh via ozdotnet, <ozdotnet@ozdotnet.com> wrote:
Folks (anyone working?)
I've been asked to add a feature to a Blazor Webassembly app to allow uploads and downloads of possibly large numbers of files between the local file system and Blob storage. I'm not sure how to implement this feature in a browser hosted app.
I wrote a WPF tool for "managers" which does high-performance bulk uploads and downloads with nice progress (the code is trivial on the desktop), but now they want the same feature for "normal" users in the Blazor app. Given how dumb and restricted browser hosted apps are, I don't know how to code this, or if it's even feasible.
Are there some tools, techniques or tricks I can apply? Any ideas or suggestions anyone?
Thanks, *Greg Keogh* -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
participants (7)
-
Craig vN -
DotNet Dude -
Dr Greg Low -
Greg Keogh -
kirsten greed -
mike smith -
Tom P