Page 1 of 1
More efficient way of assembling files from articles
Posted: December 16th, 2008, 2:12 pm
by pobox
Is it a bad idea to put the cache, temporary or completed folder on a USB flash memory stick? Why? Because the flash memory wears out with heavy use and the sticks are expensive?
Re: USB Flash stick
Posted: December 16th, 2008, 2:28 pm
by switch
Should be fine for low-medium usage. USB Flash drives will alternate what blocks it writes to in order to prolong the life of a drive (
leveling).
If you want to lessen the hard disk writing increase the article cache in config>general to say 70M
Re: More efficient way of assembling files from articles
Posted: January 24th, 2009, 4:15 am
by pobox
I'm downloading a lot of posts that have 4MB article size and 400MB RAR size. The poster claims this is the future of Usenet.
I don't have enough memory. Memory article cache is 0 and I use disk cache.
I don't enjoy using my computer during the assembly phase of the 400MB RAR's even when the cache is on a different drive, and am looking for a downloader client that has on-the-fly assembly of the articles.
The Newsbin Pro, non-free, work-in-progress contraption pre-allocates the completed size of the files with .nb! filename extensions and fills them in as the articles are downloaded and decoded. The handling of the .nb! files gets more complicated if some of the articles are missing. That's kind of weird.
The no longer developed BNR2 program builds files on-the-fly with numerical extensions. If for example filename.r00 has 10 article segments and the fifth article is missing, the files assembled on-the-fly in the download folder are named filename.r00.01-04 and filename.r00.06-10, and the blocks in files named as such are recognized by quickpar.
Xnews assembles the articles and builds the files on-the-fly.
I don't know what NewsLeecher does.
I looked in the Feature Requests forum for assembly on-the-fly vs. assembly after all articles downloaded, and didn't find a discussion, only a discussion of decoding on-the-fly. I have been thinking of SABnzbd+ as pretty advanced, but the behavior of caching individual articles followed by the assembly phase seems to be less advanced.
Re: USB Flash stick
Posted: January 24th, 2009, 12:43 pm
by shypike
That's what we have the memory cache for.
How is your system going to cope with par2 repair of such huge downloads?
May I remind you that most of the mentioned programs are not very memory-friendly either.
BTW: the writing method you describe is not guaranteed to be efficient,
since you don't know how the operating system does its disk caching.
The fact that your system isn't very usable during assembly suggests you are
using Windows, which is notoriously bad at fairly distributing disk access over
different programs.
Anyway, we will definitely look into this for a future release.
Just don't expect miracles from such an improvement.
Re: USB Flash stick
Posted: January 24th, 2009, 1:07 pm
by shypike