More efficient way of assembling files from articles
Forum rules
Help us help you:
Help us help you:
- Are you using the latest stable version of SABnzbd? Downloads page.
- Tell us what system you run SABnzbd on.
- Adhere to the forum rules.
- Do you experience problems during downloading?
Check your connection in Status and Interface settings window.
Use Test Server in Config > Servers.
We will probably ask you to do a test using only basic settings. - Do you experience problems during repair or unpacking?
Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
More efficient way of assembling files from articles
Is it a bad idea to put the cache, temporary or completed folder on a USB flash memory stick? Why? Because the flash memory wears out with heavy use and the sticks are expensive?
Last edited by pobox on January 25th, 2009, 1:40 pm, edited 1 time in total.
Re: USB Flash stick
Should be fine for low-medium usage. USB Flash drives will alternate what blocks it writes to in order to prolong the life of a drive (leveling).
If you want to lessen the hard disk writing increase the article cache in config>general to say 70M
If you want to lessen the hard disk writing increase the article cache in config>general to say 70M
Re: More efficient way of assembling files from articles
I'm downloading a lot of posts that have 4MB article size and 400MB RAR size. The poster claims this is the future of Usenet.
I don't have enough memory. Memory article cache is 0 and I use disk cache.
I don't enjoy using my computer during the assembly phase of the 400MB RAR's even when the cache is on a different drive, and am looking for a downloader client that has on-the-fly assembly of the articles.
The Newsbin Pro, non-free, work-in-progress contraption pre-allocates the completed size of the files with .nb! filename extensions and fills them in as the articles are downloaded and decoded. The handling of the .nb! files gets more complicated if some of the articles are missing. That's kind of weird.
The no longer developed BNR2 program builds files on-the-fly with numerical extensions. If for example filename.r00 has 10 article segments and the fifth article is missing, the files assembled on-the-fly in the download folder are named filename.r00.01-04 and filename.r00.06-10, and the blocks in files named as such are recognized by quickpar.
Xnews assembles the articles and builds the files on-the-fly.
I don't know what NewsLeecher does.
I looked in the Feature Requests forum for assembly on-the-fly vs. assembly after all articles downloaded, and didn't find a discussion, only a discussion of decoding on-the-fly. I have been thinking of SABnzbd+ as pretty advanced, but the behavior of caching individual articles followed by the assembly phase seems to be less advanced.
I don't have enough memory. Memory article cache is 0 and I use disk cache.
I don't enjoy using my computer during the assembly phase of the 400MB RAR's even when the cache is on a different drive, and am looking for a downloader client that has on-the-fly assembly of the articles.
The Newsbin Pro, non-free, work-in-progress contraption pre-allocates the completed size of the files with .nb! filename extensions and fills them in as the articles are downloaded and decoded. The handling of the .nb! files gets more complicated if some of the articles are missing. That's kind of weird.
The no longer developed BNR2 program builds files on-the-fly with numerical extensions. If for example filename.r00 has 10 article segments and the fifth article is missing, the files assembled on-the-fly in the download folder are named filename.r00.01-04 and filename.r00.06-10, and the blocks in files named as such are recognized by quickpar.
Xnews assembles the articles and builds the files on-the-fly.
I don't know what NewsLeecher does.
I looked in the Feature Requests forum for assembly on-the-fly vs. assembly after all articles downloaded, and didn't find a discussion, only a discussion of decoding on-the-fly. I have been thinking of SABnzbd+ as pretty advanced, but the behavior of caching individual articles followed by the assembly phase seems to be less advanced.
Last edited by pobox on January 25th, 2009, 1:40 pm, edited 1 time in total.
Re: USB Flash stick
That's what we have the memory cache for.
How is your system going to cope with par2 repair of such huge downloads?
May I remind you that most of the mentioned programs are not very memory-friendly either.
BTW: the writing method you describe is not guaranteed to be efficient,
since you don't know how the operating system does its disk caching.
The fact that your system isn't very usable during assembly suggests you are
using Windows, which is notoriously bad at fairly distributing disk access over
different programs.
Anyway, we will definitely look into this for a future release.
Just don't expect miracles from such an improvement.
How is your system going to cope with par2 repair of such huge downloads?
May I remind you that most of the mentioned programs are not very memory-friendly either.
BTW: the writing method you describe is not guaranteed to be efficient,
since you don't know how the operating system does its disk caching.
The fact that your system isn't very usable during assembly suggests you are
using Windows, which is notoriously bad at fairly distributing disk access over
different programs.
Anyway, we will definitely look into this for a future release.
Just don't expect miracles from such an improvement.
Re: USB Flash stick
Your request has been filed as ticket #226.
https://trac2.assembla.com/SABnzbd/ticket/226
https://trac2.assembla.com/SABnzbd/ticket/226