""" =============================================================== Run a Python script in an Android ForegroundService that automaticaly runs this file in an spawned processs, with command-line arguments passed in a JSON string, and output (including prints) redirected to a file. This allows the app's long-running asks to keep running when the app is left and paused; threads do too, but do not post a notification while running. It's also a wildly proprietary Android-only option that requires much Java-API support code. Mergeall's scripts can be run directly by subprocess.Popen on Android and will continue in the backgrouns (and they are, in the original tkinter GUI), but using a thread or service sidesteps the child-process killing potential in Android 12+. Neither threads, services, nor processes pause when the user leaves the app for another. This is wrapper is an alternative to coding+importing main() functions for all scripts' __main__ logic, and changing all print()s to use a custom object. This is also less brittle that the thread wrapper: while it too resets global system items for script duration, these resets affect only the spawned process, not the main app/GUI. There were multiple dev hurdles here - with thin or no docs: - Must add FOREGROUND_SERVICE to buildozer permissions, else service starts but dies after a few seconds; why not auto? - Broadcast receiver callback always gets two arguments; noted on docs, but terse, and the next point muted this - stdout/err in the service go nowhere: to debug, had to forcibly write 'i am here' msgs to file on phone (deleted) - Coding error muted by the prior point: SystemExit evades Exception catch, which meant send-msg never called - To send a msg with context in a service, must use service name from manifest in autoclass (not org.kivy name), AND .mService member of service class (not the usual .mActivty of activity object); mentioned nowhere in p4a's broadcast docs or online, and finding it required studying java code - In the end, the message id worked as guessed and seems arbitrary, but this was not documented anywhere either - on_pause/on_resume start/stop of receiver seems moot The service and broadcast support is really quite useful, but if you want people to use p4a/buildozer, please doc better! And please add support for FileProvider, now an Android staple. =============================================================== """ import sys, builtins, os, traceback, io, json, time from jnius import autoclass, cast from android.config import ACTIVITY_CLASS_NAME DEBUGGING = False def dump(msg=''): """ For prints to whatever stdout is at the time. This doesn't seem to show up in logcat, for reasons unknown (and uncared). Send debugging messages to an easily accessed file on the phone instead. """ if DEBUGGING: #print(msg, flush=True) log = open('/storage/emulated/0/Documents/PPUS-log.txt', 'a') log.write(msg + '\n') log.close() class Flusher: """ Wrap an output file to auto-flush all writes, so that output appears immediately in the file to be seen by file watchers. Also handles bad Unicode characters in output lines. Unlike the thread's flusher, the output file is reopened by name here, and the on_eof callback is not present: because this is a separate process, it can't be a callback in spawner. Alt: force unbuffered mode in file via os calls; portable? On write exceptions due to Unicode errors, retry with ascii() to escape all non-ascii chars. Else, a bad character in a filename can crash the run due to output-stream write errors, even for UTF8. Seen when printing a filename in diffall on Android: "'utf-8' codec can't encode character '\udccc'...: surrogates not allowed" (though how this decoded from the filesystem by os.listdir() on UTF8-based Android remains tbd). ascii() also escapes '\n' and adds enclosing quotes. Here: 'A\udcccB\n' => 'A\\udcccB\n', length 9, prints A\udcccB; 'ABñC\n' => 'AB\\xf1C\n', length 8, prints AB\xf1C [1.2.0] Note: mergeall.py calls isatty() to see if it should show help text to a console, in the unlikely event of a command-line error; okay here => isatty() is defined by proxy via __getattr__. [1.5] Added, but later disabled, a no-op flush() to try to avoid double OS flushes for the print() redefinition used when the mergeall.py script is run as a service by a Windows or Linux exe. This context is not possible, because these exes can never run mergeall.py in service mode--only in thread mode, which already ignores flush(). The final report in mergall.py may cause double flushes, but it's too minor and trivial to warrant optinization. See main.py's "[1.5] About print() and trace():" #4 for more. """ def __init__(self, filename): self.fileobj = open(filename, 'w', encoding='utf8') def write(self, text): try: self.fileobj.write(text) # even utf8 can fail except UnicodeEncodeError: # retry as ascii+\escapes try: putback = '\n' if text[-1] == '\n' else '' if putback: text = text[:-1] self.fileobj.write(ascii(text)[1:-1] + putback) except: self.fileobj.write('\n') self.fileobj.flush() # to file for tailers def close(self): # on script-code exit self.fileobj.close() # final flush moot """ def flush(self): # [1.5] moot, per above pass # already flushed by write() """ def __getattr__(self, attr): # all others: to file return getattr(self.fileobj, attr) # mostly optional here def run_script_service_wrapper( mainfilepath, # path to the script file to run in thread (rel|abs) argslist, # command-line arguments list for the script stdouterr, # where to route stdout+stderr: a file name, not object onexitid, # message id: signal script/process exit to app/GUI process appname, # 'usbsync' or 'usbsynctrial': full and trail apps' suffixes stdin=None, # a string to use to thread's stdin, '\n' for multiline trace=True): # if True, display state there """ Run a script file's code in a process, with system globals reset. This module's code runs in a ForegroundService, and a new process automatically spawned by Android (by virtue of a single ":" prefixed to its name; wildly ad-hoc, that!). It runs the Mergeall script as an exec() string with gobal mods. This allows long-running tasks to keep running on app leave/pause; threads do too, but this additionally opens an Android notification. Alternative to importing script's main function (in '.' or a package folder), for scripts whose main logic isn't coded as a function. A subprocess.Popen() process could work too (and does, in the pydroid3 tkinter GUI), but is much more likely to be killed by the android 'phantom' process killer in android12+. Formal service processes are (presumably) largely immune to kills. The thread wrapper's postop won't work here (a callable from the spawning process is invalid in the spawned process), but is instead wedged into the broadcastreceiver's callback in the app/gui process. Both trace and chdir are unneeded (prints in the GUI process still go to logcats, and changing directories here has no impact on the GUI process), but trace dumps state to the spawned process's stderr. stdouterr is automatic here: pass a filename for the action's logfile, which is reopened and closed here. A file object from the GUI process need not, and cannot, be passed to this process. There's no need to save/restore global state here: the process ends on script exit, and no state is shared with the main app/GUI process. Script exit is signaled to the UI by an Android BroadcastReceiver, which is supported in p4a. The app will receive this message both when it is foreground (visible), and when paused - in which case it displays its toast message when it's in the foreground again. UPDATE: +appname - the java folder name of the generated service class's code differs for the full and trial versions of this app! The message id is the same in both apps; it's not a folder name and assumed to pass in libs, and it's unlikely both are listening. """ def showstate(): dump( '=' * 4 + '\n' + os.getcwd() + '\n' + str(sys.path) + '\n' + str(os.listdir('.')) + '\n' + '=' * 4) if trace: showstate() # reopen logfile here, just created by app/gui process stdouterr = Flusher(stdouterr) # where/what to run mainfiledir, mainfile = os.path.split(mainfilepath) mainfileabs = os.path.abspath(mainfiledir) # change global state builtins.__name__ = '__main__' sys.argv = [mainfileabs] + argslist # command-line arguments sys.stdout = stdouterr # prints, writes (or reset .print) sys.stderr = stdouterr # catch/show exceptions in script sys.path = [mainfileabs] + sys.path # module imports ('.' fails, moot?) os.chdir(mainfiledir) # relative-path files (if any) if stdin != None: sys.stdin = io.StringIO(stdin) # provide canned script input() # in this process: new pvm and state, scope moot try: pycode = open(mainfile, 'r', encoding='utf8').read() globalscope = {} exec(pycode, globalscope) #testing: time.sleep(10) except SystemExit: # mergeall -report mode ends with sys.exit(0) # if uncaught: finally+reraise - skips broadcast! pass except Exception: # other exceptions in script code: to logfile traceback.print_exc() finally: # exc or not, close logfile (eof not a signal here) stdouterr.close() signal_script_exit(onexitid, appname) def signal_script_exit(messageid, appname): """ Broadcast script-exit to app/gui process. Alt: timer loop in app.gui process to poll for signal file; simpler and portable, but polling will pause along with the app on Android (broadcsts are delivered in pause state too?). UPDATE: +appname - java folder name differs for full/trial app! """ Intent = autoclass('android.content.Intent') intent = Intent() intent.setAction(messageid) #intent.putExtra('data', 'Main action process exit') # not required in this use case #intent.setComponent('com.quixotely.%s' % appname) # explicit: this app only/always # nope - doesn't work in a service #pythonact = autoclass(ACTIVITY_CLASS_NAME) #mActivity = pythonact.mActivity # service activity name from manifest - not the main app's! pythonact = autoclass('com.quixotely.%s.ServiceRunscript' % appname) mActivity = pythonact.mService activity = cast('android.app.Activity', mActivity) context = cast('android.content.Context', activity.getApplicationContext()) context.sendBroadcast(intent) # to BroadcastReceiver in app's gui process if __name__ == '__main__': dump('in service process') # at caller: argument = json.dumps('...') arguments_str = os.environ.get('PYTHON_SERVICE_ARGUMENT', '') arguments_dct = json.loads(arguments_str) # run the script's code in this process run_script_service_wrapper(**arguments_dct)