Page 2 of 2

Re: Post processing scipts not working

Posted: January 26th, 2017, 9:34 am
by jcfp
Thanks for the namei output, ''unfortunately'' it all looks normal. Maybe python and mergerfs just aren't besties...?

Re: Post processing scipts not working

Posted: January 26th, 2017, 12:41 pm
by sander
I find it difficult to help with anything that has to do with mounted drives, but let's try two approaches for analysis:

Run this python script with argument /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py

Code: Select all

import stat
import time
import os
import sys

def mode_of_file(myfile):
	st = os.stat(myfile)
	mode, ino, dev, nlink, uid, gid, size, atime, mtime, ctime = st

	print oct(mode)
	if (int(mode) & 64):
		print "Executable"
	else:
		print "not executable"


try:
	mode_of_file(sys.argv[1])
except:
	print "Give an existing file or directory as argument"

And let's go really hard core: compile this C-program (see instructions below), and also run with as argument /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py

Code: Select all

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

int main(int argc, char **argv)
{
    printf("Hello World. You gave %d arguments.\n", argc);

    if (argc<2) {
        printf("No file as argument ...");
        exit(0);
    }

    const char *pathname = argv[1];
    printf("File %s\n", pathname);

    // (R_OK, W_OK, X_OK) or the existence test (F_OK).
    printf("F_OK: %d \n", access(pathname, F_OK));
    printf("R_OK: %d \n", access(pathname, R_OK));
    printf("W_OK: %d \n", access(pathname, W_OK));
    printf("X_OK: %d \n", access(pathname, X_OK));

}
... and post all output here.

PS:

Compiling and running the C-program:

Code: Select all

sander@Stream-13:~/test-rwx-bit$ gcc test-f-rwx-bit.c -o test-f-rwx-bit
sander@Stream-13:~/test-rwx-bit$ ./test-f-rwx-bit /bin/bash
Hello World. You gave 2 arguments.
File /bin/bash
F_OK: 0 
R_OK: 0 
W_OK: -1 
X_OK: 0 

Re: Post processing scipts not working

Posted: January 26th, 2017, 3:28 pm
by mcawley
Python code output

Code: Select all

0100777
Executable
c code output

Code: Select all

Hello World. You gave 2 arguments.
File /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py
F_OK: 0 
R_OK: 0 
W_OK: 0 
X_OK: -1 

Re: Post processing scipts not working

Posted: January 26th, 2017, 3:40 pm
by sander
mcawley wrote:Python code output

Code: Select all

0100777
Executable
c code output

Code: Select all

Hello World. You gave 2 arguments.
File /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py
F_OK: 0 
R_OK: 0 
W_OK: 0 
X_OK: -1 
Wow ...

So:
1) the alternative Python code sees the correct RWX-settings and sees it as executable
2) C's access() says NOT executable. So even at that level the problem exists. (And thus Python's and thus SAB's is kind-of correct, or at least consistent.). That is really beyond my expertise. And if even @jcfp can't solve it ... :o

Next Step ... I don't know. Replace the current python code with the other python code .... ???

Or, maybe easier: you put your script on your regular harddisk. Oh wait: /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py is some other disk, isn't it?

Re: Post processing scipts not working

Posted: January 27th, 2017, 2:20 am
by jcfp
sander wrote:Or, maybe easier: you put your script on your regular harddisk. Oh wait: /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py is some other disk, isn't it?
That filesystem seems to act as a kind of union of multiple underlying physical hds. Only suggestion I've left is talk to the devs of mergerfs and ask them about the inconsistent results when checking permissions.

Re: Post processing scipts not working

Posted: January 27th, 2017, 2:29 am
by sander
Only suggestion I've left is talk to the devs of mergerfs and ask them about the inconsistent results when checking permissions.
Is https://github.com/trapexit/mergerfs the correct source / location?
If so: I can't reproduce the problem, so the OP should report there.

FWIW:
I can only find this not-a-bug: http://bugs.python.org/issue14706
It's the opposite behaviour (sudo gives x-bit), but I'm wondering if this quote is useful:
So this seems to be a well-known portability problem accross Unix implementations. If you want to test the executable bits, just use os.stat().
So use os.stat() instead of os.access() as a workaround?

Re: Post processing scipts not working

Posted: January 27th, 2017, 5:16 am
by mcawley
Thanks for the support - I'll look at trying the alternate Python code for now, and I will raise this with the mergerfs devs.

Re: Post processing scipts not working

Posted: January 27th, 2017, 1:24 pm
by trapexit
Before submitting any bug reports for mergerfs please create a simple reproduction of the issue. Be sure that the policies being used aren't weird and multiple copies of the script on different drives (or explicitly chmod it through mergerfs with the `all` policy on chmod). Also be sure that mergerfs is running as root or at least that the file, directory, mergerfs, and the acting user are all the same.

Re: Post processing scipts not working

Posted: January 27th, 2017, 3:41 pm
by sander
@mccawley

Some code you could use.

Code: Select all

import os
import sys
filename = sys.argv[1]

# one-liner that tells if x-bit is set for user:
print (os.stat(filename)[0] & (1<<6)) > 0

# ... same same, but more clear what's going on:
# rwx rwx rwx
# 876 543 210 
userXbit = 1<<6
rwxbits = os.stat(filename)[0]
print (rwxbits & userXbit) > 0
@safihre:

To replace the code in sabnzbd/newsunpack.py

Code: Select all

if not os.access(command[0], os.X_OK):
what code you prefer:

Code: Select all

if not ((os.stat(command[0])[0] & (1<<6)) > 0):
or a bit more readable

Code: Select all

userXbit = 1<<6
rwxbits = os.stat(command[0])[0]
if not ( (rwxbits & userXbit) > 0 ):
or a seperate and thus much more readable:

Code: Select all

if not userxbitisset(command[0]) :
I prefer the last so that anybody can still understand it later on.

Re: Post processing scipts not working

Posted: January 27th, 2017, 5:04 pm
by safihre
A function for sure :)

Re: Post processing scipts not working

Posted: January 28th, 2017, 2:41 am
by sander
OK, done. New code is on https://github.com/sanderjo/sabnzbd/blo ... sunpack.py

It works for me: tested with python-file with and without x-bit set.

Code: Select all

Python script "/home/sander/sab-scripts/noexec.py" does not have execute (+x) permission set
Cannot run script /home/sander/sab-scripts/noexec.py
@mcawley: can you try that code? Use that github repository (branch "x-bit-checking"), or, easier: use https://raw.githubusercontent.com/sande ... sunpack.py to replace your existing sabnzbd/newsunpack.py. A "wget https://raw.githubusercontent.com/sande ... sunpack.py" will give you the code, then copy to correct directory.

Re: Post processing scipts not working

Posted: January 29th, 2017, 7:41 am
by mcawley
New code works perfectly, thank you.

Re: Post processing scipts not working

Posted: January 29th, 2017, 8:16 am
by sander
mcawley wrote:New code works perfectly, thank you.
PR sent: https://github.com/sabnzbd/sabnzbd/pull/800