Post processing scipts not working

[jcfp]

Thanks for the namei output, ''unfortunately'' it all looks normal. Maybe python and mergerfs just aren't besties...?

[sander]

I find it difficult to help with anything that has to do with mounted drives, but let's try two approaches for analysis:

Run this python script with argument /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py

Code: Select all

import stat
import time
import os
import sys

def mode_of_file(myfile):
	st = os.stat(myfile)
	mode, ino, dev, nlink, uid, gid, size, atime, mtime, ctime = st

	print oct(mode)
	if (int(mode) & 64):
		print "Executable"
	else:
		print "not executable"


try:
	mode_of_file(sys.argv[1])
except:
	print "Give an existing file or directory as argument"

And let's go really hard core: compile this C-program (see instructions below), and also run with as argument /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py

Code: Select all

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

int main(int argc, char **argv)
{
    printf("Hello World. You gave %d arguments.\n", argc);

    if (argc<2) {
        printf("No file as argument ...");
        exit(0);
    }

    const char *pathname = argv[1];
    printf("File %s\n", pathname);

    // (R_OK, W_OK, X_OK) or the existence test (F_OK).
    printf("F_OK: %d \n", access(pathname, F_OK));
    printf("R_OK: %d \n", access(pathname, R_OK));
    printf("W_OK: %d \n", access(pathname, W_OK));
    printf("X_OK: %d \n", access(pathname, X_OK));

}

... and post all output here.

PS:

Compiling and running the C-program:

Code: Select all

sander@Stream-13:~/test-rwx-bit$ gcc test-f-rwx-bit.c -o test-f-rwx-bit
sander@Stream-13:~/test-rwx-bit$ ./test-f-rwx-bit /bin/bash
Hello World. You gave 2 arguments.
File /bin/bash
F_OK: 0 
R_OK: 0 
W_OK: -1 
X_OK: 0 

[mcawley]

Python code output

Code: Select all

0100777
Executable

c code output

Code: Select all

Hello World. You gave 2 arguments.
File /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py
F_OK: 0 
R_OK: 0 
W_OK: 0 
X_OK: -1 

[sander]

Python code output

Code: Select all

0100777
Executable

c code output

Code: Select all

Hello World. You gave 2 arguments.
File /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py
F_OK: 0 
R_OK: 0 
W_OK: 0 
X_OK: -1 

Wow ...

So:
1) the alternative Python code sees the correct RWX-settings and sees it as executable
2) C's access() says NOT executable. So even at that level the problem exists. (And thus Python's and thus SAB's is kind-of correct, or at least consistent.). That is really beyond my expertise. And if even @jcfp can't solve it ...

Next Step ... I don't know. Replace the current python code with the other python code ....

Or, maybe easier: you put your script on your regular harddisk. Oh wait: /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py is some other disk, isn't it?

[jcfp]

Or, maybe easier: you put your script on your regular harddisk. Oh wait: /media/Data/downloads/scripts/nzbToMedia/nzbToSickBeard.py is some other disk, isn't it?

That filesystem seems to act as a kind of union of multiple underlying physical hds. Only suggestion I've left is talk to the devs of mergerfs and ask them about the inconsistent results when checking permissions.

[sander]

Only suggestion I've left is talk to the devs of mergerfs and ask them about the inconsistent results when checking permissions.

Is https://github.com/trapexit/mergerfs the correct source / location?
If so: I can't reproduce the problem, so the OP should report there.

FWIW:
I can only find this not-a-bug: http://bugs.python.org/issue14706
It's the opposite behaviour (sudo gives x-bit), but I'm wondering if this quote is useful:

So this seems to be a well-known portability problem accross Unix implementations. If you want to test the executable bits, just use os.stat().

So use os.stat() instead of os.access() as a workaround?

[mcawley]

Thanks for the support - I'll look at trying the alternate Python code for now, and I will raise this with the mergerfs devs.

[trapexit]

Before submitting any bug reports for mergerfs please create a simple reproduction of the issue. Be sure that the policies being used aren't weird and multiple copies of the script on different drives (or explicitly chmod it through mergerfs with the `all` policy on chmod). Also be sure that mergerfs is running as root or at least that the file, directory, mergerfs, and the acting user are all the same.

[sander]

@mccawley

Some code you could use.

Code: Select all

import os
import sys
filename = sys.argv[1]

# one-liner that tells if x-bit is set for user:
print (os.stat(filename)[0] & (1<<6)) > 0

# ... same same, but more clear what's going on:
# rwx rwx rwx
# 876 543 210 
userXbit = 1<<6
rwxbits = os.stat(filename)[0]
print (rwxbits & userXbit) > 0

@safihre:

To replace the code in sabnzbd/newsunpack.py

Code: Select all

if not os.access(command[0], os.X_OK):

what code you prefer:

Code: Select all

if not ((os.stat(command[0])[0] & (1<<6)) > 0):

or a bit more readable

Code: Select all

userXbit = 1<<6
rwxbits = os.stat(command[0])[0]
if not ( (rwxbits & userXbit) > 0 ):

or a seperate and thus much more readable:

Code: Select all

if not userxbitisset(command[0]) :

I prefer the last so that anybody can still understand it later on.

[safihre]

A function for sure

[sander]

OK, done. New code is on https://github.com/sanderjo/sabnzbd/blo ... sunpack.py

It works for me: tested with python-file with and without x-bit set.

Code: Select all

Python script "/home/sander/sab-scripts/noexec.py" does not have execute (+x) permission set
Cannot run script /home/sander/sab-scripts/noexec.py

@mcawley: can you try that code? Use that github repository (branch "x-bit-checking"), or, easier: use https://raw.githubusercontent.com/sande ... sunpack.py to replace your existing sabnzbd/newsunpack.py. A "wget https://raw.githubusercontent.com/sande ... sunpack.py" will give you the code, then copy to correct directory.

[mcawley]

New code works perfectly, thank you.

[sander]

New code works perfectly, thank you.

PR sent: https://github.com/sabnzbd/sabnzbd/pull/800