[solved] Post-processing multiple downloads

[thejinx0r]

Hi,

Is there a way to post-process multiple downloads?
For me, the real issue is with the post processing scripts that I am running. The actual verifying +extraction step takes longer than my post-processing scripts Is there a way to start verifying+extract while the post-processing script is running?

I am running it in linux on a debian machine.

[shypike]

Of course not. The post-processing script is supposed to work with
the results of the verification/extraction process.
What does your PP script do?

[thejinx0r]

I guess I miss-phrased what I was trying to say.
My PP sends the downloaded files from my server to my nas through rsync.
And I'd like to have my PP run, and have another download get verified/extracted.
Ideally, I'd like to have multiple downloads get verified and extracted at the same time.

------------

solved:
What I had tried to do in the past was to have a crontab to just run my rsync file.
But, there was a bug where it would still run despite checking for a lock file. The issue was the lock file was being written somewhere not accessible. So, it would never be created. Now, I just write my lock file to /tmp and it works fine now.

[shypike]

The CPU and disk are maxed out by as single post-processing job,
running them in parallel is not useful.
If you want your own script to run in parallel,
then use an intermediate script that starts the copy-script asynchronously.
On Windows that would be:

@echo off
start c:\path\to\copy-script.cmd %1 %2 %3 %4 %5 %6 %7 %8 %9