File: PC-Phone USB Sync/main.py
#!/usr/bin/env python3
"""
=========================================================================================
PC-Phone USB Sync
A standalone and Python-coded Android (and PC) app for Mergeall.
[Please note: this code was not meant to be published. If you
stumble onto it anyhow, ignore the rough edges and snarky bits.]
Copyright © 2024 quixotely.com. All rights reserved.
License: this file is provided without warranties of any kind, and
only for online viewing and vetting. It may not be copied, posted to
GitHub, modified, repackaged, or sold. It is a view-only resource.
This Python 3.X script file is run as an Android app; a macOS app;
Windows and Linux executables; and source code. Only the latter
requires Python and Kivy installs.
This script's GUI uses the portable Kivy library; the Android app
is built with buildozer (which uses p4a); its macOS app and Windows
and Linux executables are built with PyInstaller; and Python 3.9
(3.8 on some PCs) is used internally for both GUI and sync code.
buildozer builds use the command lines below, and PyInstaller
builds use pc-build/build.py on all three PC platforms.
On Android, this app runs on Android 8.0 (api 26) and later (tested
to A13), and targets Android 12 (api 31) due to Play store rules for
new apps. On PCs, this runs as source code broadly, and its 64-bit
exes are built on macOS Catalina (universal2), Windows 11, and Ubuntu
Linux. Mergeall's former tkinter GUI still works on earlier Androids
and other PCs, though their market share is rapidly approaching zero.
This app is a free executable on PCs, and runs on Windows, macOS, and
Linux; see https://quixotely.com for downloads. The PC versions were
added later, but also use the Kivy GUI, and this same code. The
Android app is for pay on the Google Play store at a modest cost,
with a free trial that expires after a preset number of app runs
(more or less; see handle_trial_counter() for subversions+caveats).
Dependency: Kivy currently requires Open GL ES 2.0, which has been
marked as deprecated by Apple in favor of its proprietary alternative
(yes, rudely!). Support for Open GL may also be murky on Windows.
The Kivy project has already begun moving to alternatives on macOS
(using a compatability layer developed by Google), but in the worst
case, Mergeall's own tkinter GUI can be used to run syncs on PCs too.
See https://github.com/kivy/kivy/issues/5789.
The underlying Mergeall system run by both PC executables and Android
apps is open source, and available at learning-python.com/mergeall.html.
This app/GUI is not open source, but may become so eventually; it was
first released as a premium app, to set it apart from free Android fare.
-----------------------------------------------------------------------------------------
ANDROID BUILD CHEAT SHEET
[see also ../_HOW.txt for the build saga]
[see also pc-build/build.py for PC builds: totally different, PyInstaller]
Start code must be main.py (this), can config folder in .spec
There is a .buildozer/ in both ~ and ., plus code installed in python /Frameworks
buildozer.spec is in . after running "buildozer init" => mod for app
Built apps show up in ./bin
Kivy ui def: "class Xxx" => file "xxx.kv"
MANAGE_EXTERNAL_STORAGE works if in manifest via ./bd.spec file: needs runtime perm
foreground services and file providers require extra steps: see notes here + bd.spec
MAIN BUILD COMMANDS:
# caution: folder name in below's .sh code changed for 1.2 (and 1.3)
# set vars here and in buildozer.spec to make 4 apps: full/trial & debug/release
# now set just REL* here/ahead for release version: no more trial app [1.3.0]
cd ~/Desktop/DEV-BD/apps/PC-Phone-USB-Sync-1.3
(find {., ~/.buildozer} -name ".DS_Store" -print -exec rm {} \;)
source _dev-misc/UPLOAD-KEY/set-env-vars.sh (set upload-key vars for signing 'release')
buildozer android debug (make apk: can sideload or stream install)
(mod REL* here/below for release=True)
buildozer android release (make aab uploadable instead of apk)
PLUS:
# stream install, logcat seperately (mind the version # in the filename!)
export ADB=~/.buildozer/android/platform/android-sdk/platform-tools/adb
$ADB install bin/usbsync-1.3.0-arm64-v8a_armeabi-v7a-debug.apk
$ADB install bin/usbsynctrial-1.3.0-arm64-v8a_armeabi-v7a-debug.apk
$ADB logcat | grep python
DEV:
# build+install+run+logcat:
buildozer android debug deploy run logcat | grep python
(add "clean" to reset app components: runs long)
DEV:
# build only (and to see more error msgs):
~/Desktop/DEV-BD/apps/PC-Phone-USB-Sync-1.3$
buildozer -v android debug
ETC:
buildozer -v android debug deploy run logcat
buildozer -v android debug deploy run logcat > my_log.txt
buildozer -v android deploy run logcat | grep python
ETC:
buildozer --help (for all options)
Android-SDK/cmdline-tools/latest/bin/apkanalyzer files list bin/usbsync-1.3.0-*-debug.apk
Android-SDK/cmdline-tools/latest/bin/apkanalyzer manifest print bin/usbsync-*-debug.apk
Android-SDK/build-tools/33.0.1/aapt dump badging bin/usbsync-1.3.0-*-debug.apk
-----------------------------------------------------------------------------------------
MORE DEV NOTES
**See ../PC-Phone-USB-Demo/main.py for more early notes trimmed here**
**And ../HOW.txt plus the ../../{DEV, DEV2}/HOW older files it names**
SUMMARY
As of rc2, a "clean" build requires 2 manual steps, + 1 if view files in Finder:
- [1.2.0] to install a newly targeted android API level (sdk):
1) change target-api setting in ./buildozer.spec
2) get new sdk in ~/.buildozer (buildozer doesn't (!))
~/.buildozer/android/platform/android-sdk/tools/bin$
export ANDROID_HOME=/Users/me/.buildozer/android/platform/android-sdk
./sdkmanager --sdk_root=${ANDROID_HOME} --install "platforms;android-33"
ls ../../platforms => android-31 android-33
3) mod the following TWO build files for new target-api level:
./.buildozer/android/platform/build-arm64-v8a_armeabi-v7a/dists/usbsync/project.properties
and likewise for ../usbsynctrial
else buildozer uses new api to compile, but doesn't reset it in manifest (!)
(avoids "clean" or "appclean" build that refetches new versions of everything)
(https://stackoverflow.com/questions/72746288/buildozer-android-api-doesnt-change)
- [1.2.0] to force python38: "source ~/use38"
and mod ~/.bash_profile to be sure
and "py3 main.py" to test app as source
- must "find {., ~/.buildozer} -name ".DS_Store" -print -exec rm {} \;"
else obscure for version# errs
- must edit jnius/env.py file, TWICE - in both arch builds (post make clean)
else JNIUS undef error early in build log
see ../HOW.txt
- must insert fileprovider code, TWICE - in TWO dists manifest templates (post make clean)
else Logs OPEN won't workin the app
see ./manifest-manual-*/xml
MORE BUILD/DEBUG HURDLES
- had to declare properties in Main() ref'd in .kv file
- had to add .txt to source.include_exts in .spec for help text files in apk folder
- had to set android.no-compile-pyo=True in .spec else mergeall config not editable
- no-compile-pyo => no-byte-compile-python Nov22 (but still in old spec files!!)
- adding '.' to sys.path fails, but os.getcwd() works
- the usual 2 JAVA_HOME edits after build clean (see _dev*/demo-app/main.py)
- [android:largeHeap="true"] in manifest doesn't help for file loads (java, not ndk)
- BIG: had to patch buildozer manifest template for <provider>: see do_logs_open()
- had to disable Back close: detecting service running on restarts deprecated+errorprone
- maxnumbackups was a mess: 'from' statement, function-arg default, del sys.modules
- BIG: mod mergeall.cpall to use forked shutil--skip chmod errs on Android for exFAT+FAT32
- two apps reuires two ids everywhere: file providers, foreground service, etc.
- cancel failing update actions for removable folders outside app folder on android 10-
- BIG: fix fatal kivy TextInput memory blowup by clearing its cache manually in TAIL+WATCH
- aab signing for play store: upload key in _dev-misc, poorly documented and onerous
- BIG: fix fatal kivy race condition for ToggleButton weakref dels in on_tab_switch()
- fix kivy black TextInput lines, loss of scrolling, slow loading, orange dots, and more
- plus lots of other kivy, buildozer, and pyinstaller workarounds not listed here
PC PORTS
macOS largely worked unchanged
x usb drive names => list /Volumes
x permissions (incl. full-disk-access) is weakly documented: downplay
Windows problems resolved:
x usb drive reports as drive_fixed (below)
x opens too small (set window size)
x Help takes 8secs to open (defer set to next cycle)
x weird persistent red/orange dot post btn touch, on macos too (obscure hack)
x checkboxes too bright (allow to vary at startup)
x emdash and copyright symbols fail in text (use utf encoding)
x unicode nfd variants propagated from mac=>win=>mac can fail on mac
Linux mostly worked post windows
x macos open => xdg-open
x join => osjoin (see gotcha below)
x usb drive names => scrape mount
x RunningOnLinux was also True for android (win size) => fix
GOTCHA: 3 "join()" calls in this file worked in the Android app, but NOT when run
as source code on PCs! The p4a wrapper code used to launch this file in Android
app imported "join" from os, but didn't delete it from __main__'s scope before
launching this code. Changed to use "osjoin()" everywhere here, but the wrapper
is error-prone and should be improved. The p4a startup code (in github, builds):
python-for-android/pythonforandroid/bootstraps/common/build/jni/application/src/start.c
PC BUILDS
macOS
buildozer was a fail
use pynstaller --onedir so opens fast sans splash screen
pyinstaller worked after add .kv to datas
use '.' for file --add-datas, else a subdir
need to set app title/icon and window icon here, else titles drops ' _', kivy icon
app seems to lose Document folder permissions on reinstall; see Settings
later: required .spec file for verion# BUNDLE arg available in spec file only
later: add prompt for full-disk-access perm on run1, similar to android a-f-a
later: built universal2 binary exe, using py 3.10 and pillow wheels combo on macbook
Windows
add special ctypes code for dpi scaling, else large/blurry (worse than tkinter)
need to use spec file with Tree()s for kivy-deps, else no window provider
add special code to close the splash screen, else never goes away
need to restore builtins.print in thread wrapper: mergeall.py resets for frozen exes
exes are suddenly >2G, killing build: exclude pc-build/ from tempsrc, copy its files up
add special code for kivy logging error: writes to newly None sys.std* if --windowed
Linux
does NOT use spec files and kivy-deps like Windows; Kivy docs are deceptive on this!
the .kv file is looked up ONLY in _MEI* temp unzip folder: must --add-data
windows looks up .ks in _MEI* too, but seems to add to exe automatically
lots of giant modules are pointless: excludes shrunk size form 200M to 40M
running in virtualenv to reduce size doesn't help: shot up with undef installs
plus all the Windows WSL2 Linux shenanigans; see Tech-Notes.html, feb24+.
-----------------------------------------------------------------------------------------
TODO (rough, now dated: see _dev-misc/todo2.txt for laters)
See also _dev-misc/todo.txt, Note20 memo
n run ma as imported pkg to avoid os.cdir()? [kivy impact unknown]?
x allow #logfiles-saved and #bkp-saved to be configs?
x help-message.txt
x icon
x recode gui...
x persist paths...
x threads in gui (exec or import/recode)...
? disable file selection only (what?)...
n animate disabled-image for Main buttons...
Button().background_disabled_normal = image
n bkgrnd and power-management issues, kills, and throttling
power setting? Foreground Service? WorkManager?...
thread & service run if left/pause and screen off states
x is wakelock really held?
yes: scripts run with screen off
x is thread really immune to process kills?
probably - it's not a child process
x does thread run if app paused?
yes: and at same speed as fg and termux
x resize gui for keyboard?...
https://stackoverflow.com/questions/36770050/properly-resize-main-kivy-window-when-soft-keyboard-appears-on-android#61037979
x why does icon/list swap work?....
=> because kivy keeps both views, swaps, and updates redundantly
x why doesn't ActionButton@Button work?...
=> because it clashes with a Kivy class name (!)
- disable app-spec content backup to cloud?...
- punt on app-specific altogether? (it's complicated, requires filename fixer for android)
- reconsider cruft-pattern edits? (complex: skip+keep lists + case, pattern syntax)
x fileprovider for open
x foreground service for action runs
x consider supporting older android with prior permissions scheme; usb differs
x config tab, fontsize etc
x catch/disable app exit if thread running? back button? subprocess differs?
did for back button if script (thread|process) running; recent swipe can't catch
x are mergeall modules compiled to bytecode?
YES - in __pycache__, with .cpython-39.opt-2.pyc suffix
x scrolled run output: \n dropped, ascii() quotes, mono font, scroll to end, speed
but now moot: abandned for animated gif + Log TAIL/WATCH on demand
x configs: color chooser, etc
x port to macos, windows, linux
x build pc exes
x fonts in icon view in filechooser: FIXED in .kv
n consider more responsive layout for landscape phones; PUNT (for now)
x userguide
x domain name, website
x signing/store
x trial version
(see _dev-misc/todo2.txt for laters)
=========================================================================================
"""
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
# IMPORTS1 + PC PLATFORM TWEAKS
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
# Python (see also GOTCHA above)
import sys, os
import pickle, glob, time, re, queue, webbrowser, json, datetime, threading, platform
osjoin, osexists = os.path.join, os.path.exists
# Python 3.X+ only
assert(int(sys.version[0]) >= 3)
"""
-----------------------------------------------------------
Platform: we mostly care about the app/exe being run here,
because a Python is bundled, and source is not distributed.
Per Python's sys.platform:
Linux means native, WSL on Windows, or Android
don't care about WSL: must run Windows or Linux exe
Cygwin means Windows, but using its own Python
don't care about Cygwin: must run Windows exe, not code
Linux must be reset to mean non-Android Linux only, else some
code would erroneously fire on Android too (e.g., window size).
Android detection prior to Python 3.7 is partly heuristic:
any(key for key in os.environ if key.startswith('ANDROID_')).
Python 3.7+'s sys.getandroidapilevel() is set at build time.
3.7+ req ok: app uses 3.9+ for Android (3.8+ among all PCs).
-----------------------------------------------------------
"""
RunningOnAndroid = hasattr(sys, 'getandroidapilevel') # py 3.7+, but Android uses 3.9+
RunningOnMacOS = sys.platform.startswith('darwin') # intel and apple m (rosetta?)
RunningOnWindows = sys.platform.startswith('win') # Windows py, may be run by Cygwin
RunningOnCygwin = sys.platform.startswith('cygwin') # Cygwin's own py, run on Windows
RunningOnLinux = sys.platform.startswith('linux') # native, Windows WSL, Android
RunningOnLinux = RunningOnLinux and not RunningOnAndroid # Linux ONLY: else true on Android too
# [1.1.0]+ (feb24) Windows WSL2 Linux only
wsl2sig = 'microsoft-standard-WSL2'
RunningOnWSL2 = RunningOnLinux and platform.uname().release.endswith(wsl2sig)
if RunningOnWindows:
"""
-----------------------------------------------------------
[1.1.0]+ (nov23) Move win32 imports to top of script so
they will run once at script startup, instead of later in
drive-name method storage_names_and_paths_removables().
This avoids import aborts if/when Windows 10+'s Storage
Sense wrongly deletes the PytInstaller --onedir _MEI*
temp folder prematurely (it was seen to do so once).
That abort reflects a nasty bug in Storage Sense: temp
files of still-running programs should NEVER be silently
removed! The abort was also exceedingly rare: it was
observed just once, when an odd system state caused the
abort in an instance that had not yet opened a folder
chooser before a restart. Still, the workaround here
is easier than explaining all this to users...
Note: mergeall/ scripts and imported modules are immune
to this, because they are loaded from the install unzip
folder (.exe's host), not the _MEI* per-run unzip folder.
Code ahead cds to the install folder during startup, and
mergeall/ is accessed from '.' therafter. Also note: win32
modules are included in the exe despite no --hidden-import.
-----------------------------------------------------------
"""
# get drive roots, network paths [after 'pip3 install pywin32' on windows]
import win32api, win32file, win32wnet
if RunningOnWindows:
"""
-----------------------------------------------------------
Like tkinter, Kivy does not do DPI saling well on Windows,
which makes GUIs open large and blurry! This code must be
run before importing kivy and addresses the issue, but is
provisional, pending a fix in SDL2 layers in the Kivy stack.
Note that this applies to python.exe for source-code runs,
but to the frozen executable for its run: per-process call.
Used for tkinter: windll.shcore.SetProcessDpiAwareness(1)
https://github.com/kivy/kivy/pull/7299
https://github.cwindll.user32.SetProcessDpiAwarenessContext(c_int64(-4))om/kivy/kivy/issues/3705
https://learning-python.com/post-release-updates.html#win10blurryguis
-----------------------------------------------------------
"""
from ctypes import windll, c_int64
windll.user32.SetProcessDpiAwarenessContext(c_int64(-4))
if hasattr(sys, 'frozen') and (RunningOnWindows or RunningOnLinux):
"""
---------------------------------------------------------------
Yet another Kivy open bug workaround! In --windowed mode, Kivy
logging writes to sys.stdout/err that are now set to None by
PyInstaller instead of dummy objects, thereby triggerring a
recursion-limit error. Doesn't happen if --windowed not used.
This must be called this early in the script, else logs happen.
Likely to be fixed very soon; but how wasn't this tested?...
https://github.com/kivy/kivy/issues/8074
https://github.com/pyinstaller/pyinstaller/issues/7329
---------------------------------------------------------------
"""
os.environ['KIVY_NO_CONSOLELOG'] = '1'
# this is call-stack depth, and defaults to 1000 on Windows
# NO: pyinstaller code was stuck in a loop, this didn't help
# sys.setrecursionlimit(2000)
if hasattr(sys, 'frozen') and (RunningOnWindows or RunningOnLinux):
"""
---------------------------------------------------------------
Tell the PyInstaller splash screen to close (else it doesn't!).
The splash screen isn't supported on macOS, but the --onedir
build for macOS opens quickly, unlike Windows/Linux --onefile.
Android apps get splash screens from buildozer that just work.
Nice feature, but why in the world doesn't this auto close?...
[1.1.0]+ (feb24) doc: the splash screen (only) uses tkinter,
which requires Tcl/Tk to be present. This causes problems on
Windows WSL2 Linux - users must manually install Tcl/Tk to use
this app there, because PyInstaller assumes they're present,
and they're not in WSL2 Ubuntu. There was also another obscure
issue on desktop Linux with Tcl/Tk version numbers. Plus all
the weirdness of not auto closing; is this really worth it?
---------------------------------------------------------------
"""
import pyi_splash
# Update the text on the splash screen?
pyi_splash.update_text('Loading program...') # currently does nothing...
# Close the splash screen. It does not matter when the call
# to this function is made; the splash screen remains open until
# this function is called or the Python program is terminated.
#
# pyi_splash.close() # see next function, called later
def close_pc_splash_screen():
# run this in App.on_start() instead, to wait for window
if hasattr(sys, 'frozen') and (RunningOnWindows or RunningOnLinux):
import pyi_splash
pyi_splash.close()
if hasattr(sys, 'frozen') and (RunningOnMacOS or RunningOnWindows or RunningOnLinux):
"""
---------------------------------------------------------------
Set up runtime cwd context for frozen PC executables.
PyInstaller unzips --onefile exes and their added data to a
temporary folder, _MEI*, which is deleted after app exit.
This differs from the app's install folder, where the download
is unzipped. In this context, this app must arrange access to
both persistent and non-persistent data items by cwd or other:
- shipped mergeall/ subdir
- modified mergeall/mergeall_configs.py module
- shipped help and about message files
- created run-counter and configs-pickle files
- shipped usbsync-anim.gif used for image in gui
- created logfiles and UNDO backups
- shipped .kv GUI-def file, not handled auto!
- shipped usbsync-pc/, PCs must set as icon in .py
Some of these these may not work in _MEI* temporary unzip dir,
because they are changed and must be retained between app runs:
- mergeall/ scripts are loaded as source code from its folder,
so cannot be .pycs; never change, so can be in either install
or _MEI*; and are already open source, so need not be hidden.
Some modules in mergeall/ are also imported here (backup, cpall).
- mergeall/mergeall_configs.py is auto-changed by this script for
GUI settings. It can be in _MEI* iff it is changed on _every_
Main-tab action run; else it must be in install (or elsewhere).
Because this must be accessible, all of mergeall/ must be too.
- Help and about files are read-only. They could be in install
or _MEI*, but might as well follow the same policy as mergeall/.
- Run-counter and configs-pickle files cannot be in _MEI*, because
they must be changed and retained between runs. They can be in
the install dir, except on macOS which maps them to ~/Library
because apps may not be able to write own folders (tbd). These
might also be mapped to ~/Documents everywhere, but are not (yet?).
- The gif is read only and can be in any cwd (install or _MEI*);
and the .kv is read only (and optional, if its code is in .py).
- usbsync.png is used at build time (buildozer, pyinstaller),
but also for .py icon calls required on PCs (else kivy icon).
UPDATE: PCs now use prebuilt icons in usbsync-pc/, not pyinstaller.
- logfiles are mapped to ~/Documents everywhere and so are moot.
Other: SYNC backups are in TO/, and so are also moot here.
To handle these cases, this app uses these policies for PC builds:
For macOS:
Since PyInstaller builds an app folder anyhow, use --onedir;
use --add-data to store all data items alongside the executable
in the app folder; and cd to sys.executable's dir (the install
unzip's <app>/Contents/MacOS) on startup for cwd-relative access
to mergeall/, help, about, gif, and .kv file.
The app dir persists between runs, and is assumed changeable by
app. This gives persistence for mergeall/mergeall_configs.py,
as well as access for help, about, and gif loads. The run-count
and configs-pickle files are mapped to ~/Library/appname. This
also avoids lags for startup unzips sans splash sceeen on macos.
For Windows and Linux:
Store data items alongside the --onefile single-file executable
in a manually created zipfile, and cd to sys.executable's dir
(the install's unzip) on startup for cwd-relative paths (like
Mergeall). These items will not unzip to _MEI*, and hence will
persist between runs as needed. There is a lag for the unzip
at app startup, but these platforms now post a splash screen.
Exception: Linux, ONLY, does not look for the .kv file in cwd,
but ONLY in the _MEI* temp unzip folder. On Linux alone, must
add it via --add-data and can skip copying it to install folder.
UPDATE: per more tests, Windows exes run if the .kv is not
present in cwd (install folder) and is not added via --add-data,
so it must be adding it to the exe at build time. macOS requires
that the .kv be added by --add-data on --onedir mode, else the
app won't open (and --onefile on macOS takes ~12 seconds to open
with no support for a splash screen, vs 1~2 for --onedir mode).
Kivy's folder-selection code is way too whack to plumb for
answers, and its handling of .kv files in frozen executables
is an undocumented, convoluted mess. PC builds work; punt!
macOS alt: there's a chance that the app folder won't be writeable
by the app. If so, use --onefile to build and unzip to _MEI*;
_always_ mod mergeall/ mergeall_configs.py for numbackups on each
Main action run in this context ONLY, to make its persistence moot;
and cd to __file__ (_MEI*) on startup for cwd-relative access to
mergeall/, help, about, and gif. The run-counter and configs-pkl
files are already mapped to ~/Libraries/appname for macOS only.
UPDATE: not required - app can mod mergeall/mergeall_configs.py.
The chdir here makes an empty __file__ dir map to install dir.
It precludes any relative paths in cmdline args to frozen exes,
but this is moot, as exes here don't access items in users' cwds.
Mods here aren't run for either source code or the Android app,
which both keep working normally and as is. For more background
and another PyInstaller example, see Mergeall's fixfrozenpaths.py.
SEE ALSO: pc-builds/build.py applies these policies in builds.
---------------------------------------------------------------
"""
# sys.executable is the frozen exe for --onefile and --onedir
# sys.argv[0] is the path used in the launch command, poss rel
# sys._MEIPASS is the TEMPORARY uzip folder for --onefile
# __file__ is sys._MEIPASS+script for --onefile, else install folder
exepath = sys.executable
exedir = os.path.dirname(os.path.abspath(exepath)) # install folder
os.chdir(exedir) # '.' for all extras
# not required: this + launcher mod path based on abs(cwd)
# sys.path.append(exedir) # for ma/configs .py import
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
# IMPORTS2
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
# app
from run_script_as_thread import run_script_thread_wrapper, Queuer, Flusher
from run_script_as_process import run_script_process_wrapper
# auto-run as a script by p4a service support: import for error check only
if RunningOnAndroid:
from run_script_as_service import run_script_service_wrapper
# kivy: GUI
from kivy.app import App
from kivy.uix.floatlayout import FloatLayout
from kivy.uix.popup import Popup
from kivy.uix.button import Button
from kivy.uix.label import Label
from kivy.uix.textinput import TextInput
from kivy.uix.togglebutton import ToggleButton
from kivy.uix.checkbox import CheckBox
from kivy.uix.boxlayout import BoxLayout
from kivy.properties import ObjectProperty, StringProperty, NumericProperty
from kivy.clock import mainthread, Clock
from kivy.core.window import Window # instance is app.root_window
import kivy.metrics
from kivy.config import Config
from kivy.cache import Cache
from kivy.uix.filechooser import FileSystemLocal # hangware [1.1.0]+ (feb24)
# pyjnius: Java interface
if RunningOnAndroid:
from jnius import autoclass, cast
# p4a: helpers
if RunningOnAndroid:
import android.storage
#from android.storage import app_storage_path
#from android.storage import primary_external_storage_path
from android.runnable import run_on_ui_thread
from android.config import ACTIVITY_CLASS_NAME
from android.broadcast import BroadcastReceiver
else:
run_on_ui_thread = lambda func: func # no-op decorator for PCs
# old-style storage perms
if RunningOnAndroid:
from android.permissions import request_permission, check_permission, Permission
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
# GLOBALS
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
# version also redundantly in buldozer.spec + pc-build/{_macOS*.spec, _winver*};
# version string for Play can be anything, x.y.z allows simple patches to x.y;
# buildozer (really, p4a) builds versionCode from it: '10'+sdkmin+shiftedVERSION;
# Search on [VERSION] to find mods in code; [VERSION]+ is select platforms only;
# History: [1.0.0] May 2023; [1.1.0] June 2023; [1.1.0]+ later; [1.2.0] March 2024;
# [1.3.0] September 2024: make free and drop trial version, update docs accordingly.
APPNAME = 'PC-Phone USB Sync' # used for assorted folder names, etc.
VERSION = '1.3.0' # versionName in manifest, inserted in About
PUBDATE = 'September 2024' # inserted in About by on_start() too [1.1.0]
# Android trial app: mod TRIAL_VERSION + buildozer.spec [title, package, presplash, icon]
# per handle_trial_counter(), app data clears and installs can subvert run counter;
# use 'buildozer android release' to make release aab of both, use '...debug' for apk;
# this works because this file's code is copied into app as it was at build time here;
TRIAL_VERSION = False # True for trial: limited #opens
TRIAL_APP_OPENS = 10 # 5 seems too few, but reinstalls should be a bit onerous
# distinguish android trail app where it matters (only) [1.1.0]
APPNAME_FULL_OR_TRIAL = APPNAME if not TRIAL_VERSION else APPNAME + ': Trial'
# release aab on Android: mod to disable trace print()s to logcat (per Play);
# change manually - can have release versions of _both_ full and trial app;
# if `buildozer android release` ensures this, it's a very well-kept secret;
# print()s in nested mergeall/ code are routed to logfile in all run modes;
# [1.3.0] trial version is gone, so this is the only per-build mod now required
RELEASE_VERSION = False # True for release: don't trace
# global font size: it's complicated; see set_font_size()
# [1.1.0]+ (oct23) add a fudge factor for linux - init fontsize is _very_
# small [caveat: this is risky - tested on just one linux device so far];
#
# also changed for linux in [1.1.0]+:
# - (sep23) scale initial window size and add fudge factor too, ahead
# - (oct23) executable rebuilt on ubuntu22 for lib-skew issue on 20
#
# per kivy docs: sp() for fonts (uses user font settings), else dp();
# note: RunningOnLinux constant means linux only, and not android;
lx = 5 if RunningOnLinux else 0 # [1.1.0]+, +3 still seems too small
FONT_SIZE_DEFAULT = int(kivy.metrics.sp(15+lx)) # kv default=15sp, scaled for device+user
# items in app-install folder (not app-private) (no user access on android)
HELP_FILE = 'help-message.txt'
ABOUT_FILE = 'about-template.txt'
TERMS_OF_USE_FILE = 'terms-of-use.txt'
MERGEALL_PATH = 'mergeall'
SERVICE_BREADCRUMB = 'latestservice.txt' # no longer used
# items in app-private folder (not app-install) (no user access on android)
SETTINGS_FILE = 'settings.pkl'
RUNCOUNT_FILE = 'runcounter.txt'
# logfiles in Documents folder (android: shared-storage, pcs: ~, user can access; !trial)
LOGS_SUBFOLDER = APPNAME # spaces ok+common; allow in OPEN/EXPLORE; .replace(' ', '_')
# [1.2.] now used for multiple greps (globs) in logfiles folders: ensure same
LOGFILE_NAME_PATTERN = 'date*-time*--*.txt'
# macos app-private folder in ~/Library (avoid app-bundle folder, user can access, !trial)
MACOS_APPPRIV_SUBFOLDER = APPNAME
# items in TO/__bkp__ sync backups folder (user can access)
UNDOABLE_FILE = '__undoable__' # see also '--UNDONE' renames
# values of root.script_running (0 is False, else True)
NOTHING_RUNNING = 0 # running as thread or service or process?: for app close
THREAD_RUNNING = 1 # as thread only: for trace() - don't print() to streams
SERVICE_RUNNING = 2 # as service only: for on_pause/resume() - mod msg receiver?
PROCESS_RUNNING = 3 # as service only: for giggles?
# thread coding choices (temp, probably)
STATUS_ANIMATION, STATUS_SCROLL = True, False
# save Main() instance in global (because self.trace is too much to type...)
guiroot = None
def trace(*args, **kargs):
"""
-----------------------------------------------------------
Don't print if thread running: streams sent to action's
logfile (formerly, line queue then logfile; thread mode
now uses thread Flusher which writes to logfile directly).
Prints go to adb logcat on Android, console on macOS?,
etc - for developer viewing only. Alt: logcat direct?
In addition: frozen Windows/Linux exes mod builtin print,
which raises excs for flush*2. Avoided by THREAD_RUNNING:
builtin print is changed by in-process mergeall.py code run
in a thread, but restored by thread wrapper on mergeall exit
before script_running is cleared (and before the clearing
code is scheduled to run!). See run_script_as_thread.py.
UPDATE: mergeall/mergeall.py now omits flush=True if one
is sent here, because excs still happened during tab-switch
callbacks run at odd times (thread/GUI timing is complex).
Moot if not THREAD_RUNNING: standard streams in different
process for foreground service (and unused simple process),
so app can print freely here and print reset has no impact.
Don't print trace messages in release builds for Android.
That means there's nothing to go on for errors, but most
users won't be able to run logcat anyhow, and Google Play
chides about trace messages (and is douchefully dogmatic).
-----------------------------------------------------------
"""
if RELEASE_VERSION:
return
if not guiroot.script_running == THREAD_RUNNING: # don't print to logfile
print('(PPUS)', *args, **kargs, flush=True) # don't flush*2 if frozen
def trace2(*args, **kargs):
"""
-----------------------------------------------------------
NOT CURRENTLY USED, but can be used for app debugging.
To trace while action thread runs; also pass trace=True
to run_script_thread_wrapper() so stderr is not rerouted
to line-queue Pipe. Don't double flush in frozen Windows
and Linux exes: they already reset builtin print to do so.
Don't print trace messages in release builds for Android.
[1.1.0]+ "RunningOnWindows or RunningOnLinux and hasattr(..)"
didn't look right and was corrected (py 'and' binds tighter
than 'or'), but this is moot and harmless - function unused,
and the need to omit flush=True was lifted (see trace() above;
was this why there were still unexplained print exceptions??);
-----------------------------------------------------------
"""
if RELEASE_VERSION:
return
if (RunningOnWindows or RunningOnLinux) and hasattr(sys, 'frozen'): # [1.1.0]+ (oct23)
print('(PPUS)', *args, **kargs, file=sys.stderr)
else:
print('(PPUS)', *args, **kargs, file=sys.stderr, flush=True)
class _PopupStack:
"""
===========================================================
A simple stack for popup dialogs that may nest.
Kivy coding structure makes it easiest to save
Popups for later closes, but Popups may nest here
(e.g., an info_message while a folderchooser is open,
though this may have been impossible in earlier code).
UPDATE: Button double-tap nonsense:
1) Pops of _empty_ stack triggered excs and app aborts
when dialog close buttons were double tapped on all
platforms. This specific issue is fixed by checking
for empty here, but double taps create other issues.
2) For one, double taps can close _two_ dialogs stacked
atop each other at once. To avoid this, all dismiss calls
now pass the Popup's content, and dismiss here skips the
call if passed contents do not match that of the stacked
Popup. This avoids closing overlaid Popup on double taps.
3) Now also uses auto_dismiss=False on all Popup() so
Popup not dismissed without a pop() here. Else, bogus
auto-dismissed entries remain stacked and are dismissed
on later dbl taps (though this seems to be harmless).
Could keep auto-dismiss by catching on_dismiss and doing
pop() in catcher, but taps outside multi-button popups
are ambiguous and seem error-prone in practice anyhow.
More generally, a double tap on _any_ button may trigger
its callback twice... This seems a Kivy misfeature (bug!),
but hasn't been an issue in testing. If can only matter
if event is fired because the widget still active; the
Popup case is probably timing - Button event beats the
dismiss() event. Alt: disable Button double-tap instead?
===========================================================
"""
def __init__(self):
self.popups = []
def push(self, popup):
self.popups.append(popup) # stack Popup object
def empty(self):
return not self.popups
def pop(self):
if not self.empty():
return self.popups.pop(-1) # remove+return top
def top(self):
if not self.empty(): # avoid dbl-tap excs
return self.popups[-1]
def open(self):
if not self.empty():
self.top().open() # display topmost Popup
def dismiss(self, content):
if not self.empty():
if self.top().content == content: # avoid dbl-tap closes
self.pop().dismiss() # close topmost Popup
class FileSystemLocalNoHang(FileSystemLocal):
"""
===========================================================
[1.1.0]+ (feb24) Wrap methods of the Kivy FileChooser's
FileSystemLocal interface class (os module calls, mostly)
in thread timouts. Else the chooser can stall (block and
possibly hang) as users navigate by tapping its UI.
Network drives, cameras, etc., may go offline, especially
on Windows, causing raw, unwrapped calls to block indefinitely.
This happens deep in the Kivy FileChooser code, but its
file_system hook supports mods. Code here fixes the chooser
widget; filesystem calls in the app itself were wrapped in
thread timeouts earlier (the 1.1.0 release in Jun23).
Coding notes: this doesn't call os functions directly, because
Kivy's FileSystemLocal.is_hidden() uses tight magic to access
and use a private win32 call on Windows. guiroot is the global
Main instance: see PCPhoneUSBSync.on_start(). This adds a
thread per chooser nav tap, but PCs and phones are fast.
TBD: is this useful only, or at least primarily, on Windows?
If so, could avoid one thread per os-module call elsewhere.
To date, this is the only platform where nav hangs have
been seen, though PCs and phones are both awfully fast...
===========================================================
"""
def listdir(self, filepath, warn=True):
"""
called for both test and result: warn user
"""
trace('NoHang.listdir')
listed = guiroot.try_any_nohang(super().listdir, filepath, onfail=None, log=False)
if listed != None:
return listed # empty or not (no exc or timeout)
else:
trace('Skipping dir: listdir')
if warn:
guiroot.info_message('Cannot access folder.', usetoast=True)
raise OSError # kill dir in filechooser (run in try)
def getsize(self, filepath, warn=False):
"""
may be used if is_dir fails, but rare
"""
trace('NoHang.getsize')
getsize = guiroot.try_any_nohang(super().getsize, filepath, onfail=None, log=False)
if getsize != None:
return getsize # dir or not (no exc or timeout)
else:
trace('Skipping file: getsize')
if warn:
guiroot.info_message('Cannot access file.', usetoast=True)
raise OSError # kill dir in filechooser (run in try)
def is_hidden(self, filepath, warn=False):
"""
win32 privates used in super
call numerous times: don't warn user
"""
#trace('NoHang.is_hidden')
ishidden = guiroot.try_any_nohang(super().is_hidden, filepath, onfail=None, log=False)
if ishidden != None:
return ishidden # hidden or not (no exc or timeout)
else:
trace('Skipping dir: is_hidden')
if warn:
guiroot.info_message('Cannot access folder.', usetoast=True)
raise False # kill dir in filechooser (run in try)
def is_dir(self, filepath, warn=False):
"""
called an insane number of times: don't warn user
"""
#trace('NoHang.is_dir')
isdir = guiroot.try_any_nohang(super().is_dir, filepath, onfail=None, log=False)
if isdir != None:
return isdir # dir or not (no exc or timeout)
else:
trace('Skipping dir: is_dir')
if warn:
guiroot.info_message('Cannot access folder.', usetoast=True)
return False # kill dir in filechooser
fileSystemLocalNoHang = FileSystemLocalNoHang() # set in chooser on create
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
# KIVY INTERFACE
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
class MainPathPickDialog(FloatLayout):
"""
=======================================================
The file-chooser dialog's callbacks.
Also defined with <> instance rule in .kv.
self argument implied in .kv file: root=instance.
Class and its properties _must_ by defined in .py.
=======================================================
"""
# these become self attrs
onpick = ObjectProperty(None) # refs callback in Main, run in .kv
oncancel = ObjectProperty(None) # all args to class _must_ be declared here
pickstart = StringProperty() # set here, used in nested fc: root.pickstart
rootpath = StringProperty() # same as dialog.ids.filechooser.rootpath
onviewmode = ObjectProperty() # icon|list from/to the Config tab (persistent)
class InfoDialog(FloatLayout):
"""
=======================================================
The info-message dialog.
Also defined with <> instance rule in .kv.
self argument implied in .kv file: root=instance.
Class and its properties _must_ by defined in .py.
=======================================================
"""
message = StringProperty() # passed here as args => root properties
oncancel = ObjectProperty(None) # refs bound callback in Main, run in .kv
class ConfirmDialog(FloatLayout):
"""
=======================================================
All in .kv, but must declare here too
=======================================================
"""
message = StringProperty()
onyes = ObjectProperty(None)
onno = ObjectProperty(None)
class ColorPickDialog(FloatLayout):
"""
=======================================================
All in .kv, but must declare here too
=======================================================
"""
onpick = ObjectProperty(None)
oncancel = ObjectProperty(None)
class TrialEndedDialog(FloatLayout):
"""
=======================================================
All in .kv, but must declare here too
=======================================================
"""
message = StringProperty()
oncancel = ObjectProperty(None)
class ConfirmNameDialog(FloatLayout):
"""
=======================================================
All in .kv, but must declare here too
=======================================================
"""
message = StringProperty()
onrunreport = ObjectProperty(None)
onrunupdate = ObjectProperty(None)
oncancel = ObjectProperty(None)
class ConfigCheckbox(CheckBox):
"""
=======================================================
Ref in .kv, def in .py: need to set color per patform
[r, g, b, a] for image tinting - too dim on macos only
Default is [1, 1, 1, 1], which works well elsewhere
=======================================================
"""
color = [1, 3, 5, 4] if RunningOnMacOS else [1, 1, 1, 1]
class TextInputNoSelect(TextInput):
"""
========================================================================
Workaround for a Kivy text selection bug on Android.
Or: The struggles to disable persistent handles and pointless selections.
Kivy currently has a nasty bug which makes selection handles linger on
in the GUI on Android - both after popup dismiss, and after tab switch.
This is an issue only for _readonly_ text, as handles go away on either
keyboard minimize or tab switch; chooser paths and log text are readonly.
To workaround, first disabled copy, handles, and buttons in .kv file.
This works, but user actions can still select the text pointlessly.
Selection was first addressed by cancel_selection on tab switch (popup
is auto by recreate), but it's still subpar to allow useless selections.
The final workaround overrides selection methods in the TextInput class
here (it didn't work in .kv) to disale selections in full. This makes
the earlier fixes moot, and brings an end to all the troubles...
========================================================================
"""
def __init__(self, **kwargs):
super().__init__(**kwargs)
# try 1: prevent changes, and selection handles and bubbles (.kv)
#self.readonly = True # else keyboard still covers (iff font?)
#self.allow_copy = False # else select handles on other tabs...
#self.use_handles = False # prevents handles hell...
#self.use_bubble = False # prevents action-bubble popups...
# nuclear option: prevent selections altogether (here)
def on_double_tap(self): pass # trace('on_double_tap\n')
def on_triple_tap(self): pass # trace('on_triple_tap\n')
def on_quad_touch(self): pass # trace('on_quad_touch\n')
def long_touch(self, dt): pass # trace('long_touch\n')
"""
NO LONGER USED: in tab
class LogfilePickDialog(FloatLayout):
""
=======================================================
All in .kv, but must declare here too
=======================================================
""
pickstart = ObjectProperty(None)
onpick = ObjectProperty(None)
oncancel = ObjectProperty(None)
"""
"""
# TBD
class SectionLabel(Label):
def __init__(self, **args):
super().__init__(**args)
self.padding = [8, 8]
valign = 'center'
self.text_size = root.width, None
self.size = self.texture_size
class InfoLabel(Label):
def __init__(self, **args):
super().__init__(**args)
#self.padding = [8, 8]
valign = 'left'
self.text_size = self.texture_size
class LeftButton(Button):
def __init__(self, **args):
super().__init__(**args)
"""
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
# MAIN (ROOT) GUI CLASS
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
class Main(FloatLayout):
"""
=======================================================
The main/start window's callbacks.
Corresponds by name to root rule in .kv.
self argument implied in .kv file: root=instance.
=======================================================
"""
# kivy properties
from_path = ObjectProperty(None) # from/to path display+edit fields in GUI
to_path = ObjectProperty(None)
logfile_path = StringProperty() # referenced in kv file during build()
about_message = StringProperty() # default initial value is '' (a False)
help_message = StringProperty()
if STATUS_SCROLL:
run_output = ObjectProperty(None) # bound to same name in .kv via TextInput's id
#=======================================================================
# MAIN-TAB FILE CHOOSER
#=======================================================================
def do_main_path(self, kind, kindid):
"""
----------------------------------------------------------------------
Run from Main's FROM|TO buttons to pick a content path.
kind = 'FROM'|'TO' for use in popup dialog label only.
kindid = 'frompath'|'topath', which gives access via self.ids[] to
pathfield = text display for the selected folder kind in Main tab.
The last of these is used to initialize the selection-area display:
the picker opens at the last Main pathfield setting for kindid, unless
it's not a valid dir, in which case the picker opens on shared (here).
Dynamically adds drive-name buttons after the standard storages;
on taps, reset_main_picker_path() changes picker content dynamically.
This just works in the folder chooser (but it also has multiple bugs).
ROOT is complicated. It's unclear if the app can simply access
'/' folders sans su commands on rooted phones (Termux "ls" required
su for USB), and there's no root support in the kivy FileChooser.
Enable iff Config, show '/' if access else api's /system, and avoid
permission excs by setting dialog.rootpath to avoid '..' navigation.
Illegal folders (e/g. '/', '/storage') throw excs if reached by '..'.
Changes in the chooser's viewmode in the popup modify the popup's
display, but also the Config tab's associated settings. The Config
tab's settings are used for later restarts and persistence saves.
----
UPDATE: fix icon-view clipping for bigger fonts by copying template
from kivy.style to .kv and modifying the label; this is awful, but
there are no ids or other hooks for doing this; improve me, Kivy!
UPDATE: make red storage-name buttons radio toggles, so it's clearer
to users what is being displayed. This is probably moot once users
get used to path names on their devices, but seems ambiguous at first.
UPDATE: use child-list crawls here to nuke the "Size" label in list
view, which is bogus for a folder chooser, and gets truncated to just
"e" on some phones and font sizes (!). The fix simply sets the label
to '' which may waste its space, but there are no builtin ids or hooks,
and a core-code copy+mod seems worse (see the icon-view font fix in .kv).
Children are reverse order of kivy-lang defs: see FileChooserListLayout
in https://github.com/kivy/kivy/blob/master/kivy/data/style.kv (builtin)
UPDATE [1.1.0]+ (feb24): subclass the chooser's file-system interface
class, to wrap its calls in thread timeouts to avoid blocks and hangs.
Else navigation taps can stall deep in the chooser widget itself.
----------------------------------------------------------------------
"""
#---------------------------------------------------------------------
# prelims: get storage name/paths, make dialog
#---------------------------------------------------------------------
# user may have passed on open or later
if not self.get_storage_permission(): # confirm permission or ask again
return # had to reask user: user must retry
# get storages; shared is primary, appspec is for android only
# CAUTION: label_path_string() mimics some of this logic [1.1.0]
shared = self.storage_path_shared()
appspec = self.storage_path_app_specific()
drives = self.storage_names_and_paths_removables() # do before root for Windows
root = self.storage_path_root()
docs = self.storage_path_documents() # TBD: not curently shown by chooser
# view-style utils
def initstyle():
"""
use Config tab style on first open (used ahead)
the popup sets config buttons, but not vice versa
"""
return 'icon' if self.ids.mainchoosericon.state == 'down' else 'list'
def changestyle(viewmode):
"""
mod Config tab style on chooser style button press
yes, both buttons must be set, despite radio group
note: these are Config-tab's buttons, not chooser's
"""
style = str(viewmode) # now moot
self.ids.mainchoosericon.state = 'down' if style == 'icon' else 'normal'
self.ids.mainchooserlist.state = 'down' if style == 'list' else 'normal'
# init display to from|to path in Main tab (or shared, if run1 or drive absent)
pathfield = self.ids[kindid]
# naive: pickstart = pathfield.text if os.path.isdir(pathfield.text) else shared
# avoid chooser exc: prior pick may have been dir but no permission;
# probably should check on Pick, but actions already do before runs
"""
try:
os.listdir(pathfield.text) # might hang...
except:
pickstart = shared # unmounted, permissions, edited
else:
pickstart = pathfield.text # prior pick still present and valid
"""
# same, but use timeouts to avoid lags for inaccessible drives [1.1.0]
listed = self.try_listdir_nohang(pathfield.text)
if listed:
pickstart = pathfield.text # accessible: prior pick still present and valid
else:
pickstart = shared # hung, or fail (unmounted, permission, manual edit)
# new chooser: see also .kv
picker = MainPathPickDialog(
pickstart=pickstart, # start navs here
rootpath=pickstart, # stop up-navs here
onviewmode=changestyle,
onpick=lambda path, popupcontent:
self.do_main_path_pick(kindid, path, popupcontent),
oncancel=self.dismiss_popup)
#-----------------------------------------------------------------------
# set up-navigation cutoff for initial path in chooser
#
# allow '..' nav from last-run start path copied from Main, but only
# up to its storage-type root, if any (+1 '..' for paths in app-spec);
# else '..' navs above roots trigger silent exceptions in chooser;
#-----------------------------------------------------------------------
# use absolute path comparisons: user may enter anything manually [1.1.0]
abspickstart = os.path.abspath(pickstart)
if pickstart == pathfield.text: # copied over?
topmost = None # no cutoff
#------------------------------------------------------------------------
# android appspec first: it's nested in shared, and ../.. throws exc;
# also sort drives by decr path len: usb nested in os on linux [1.1.0];
#
# [1.1.0]+ (feb24) what?; this may reflect an erroneous mount of C: to
# /media/me, which hid USB drives that ubuntu auto-mounts to /media/me/XXX:
# the effect nested USB in C:'s mount (!), and only one might be usable;
#
# either way, match prefix by longest = most specific/nested first,
# to handle nesting on all platforms as well as possible; user can
# always tap a root button to restart (and trigger similar code ahead);
#
# assumes no drive will be nested in shared=home on Unix (macos+linux):
# unikely, imposs on android+windows, and just a few extra ..s if wrong;
#
# a bit gray here: can't tell which storage was used to select prior path;
# here the effect may be to limit .. navs to a lower nested storage root,
# but this roughly corresponds to button preselect ahead: most specific;
#
# NIT: in hindsight, it seems that calling out appspec before shared is
# not needed here, because the decreasing-path-length sort would order
# these too that way anyhow (and is relied on to do so in similar code
# at tab preselect and popup label fetch ahead, which could be merged);
#------------------------------------------------------------------------
drives_sorted = sorted(drives, key=(lambda np: len(np[1])), reverse=True)
for storage in [appspec, shared] + [path for (name, path) in drives_sorted]:
if storage != None and abspickstart.startswith(storage):
topmost = storage if storage != appspec else storage + '/..'
break
picker.ids.filechooser.rootpath = topmost # same as picker.rootpath
# left align paths on open, can be scrolled after
def leftalign(dt):
picker.ids.pathname.cursor = (0, 0)
Clock.schedule_once(leftalign)
# Worst Kivy Hack Ever (or so far): nuke list-view "Size" label per above
listlayoutobj = picker.ids.filechooserlistlayout
try:
listsizelabel = listlayoutobj.children[0].children[1].children[1]
if listsizelabel.text == 'Size':
listsizelabel.text = '' # to test: 'NOPE'
except:
pass # it's not worth dying for...
# [1.1.0]+ (feb24) wrap chooser's os calls in timeouts to avoid nav-tap stalls;
# this sets a Kivy hook to an instance of the file-system subclass (see above)
picker.ids.filechooser.file_system = fileSystemLocalNoHang
#---------------------------------------------------------------------
# add scrollable device/storage buttons to top of picker
#---------------------------------------------------------------------
class StorageToggleButton(ToggleButton):
"""
==================================================================
red button storage-root toggles at top of Main-tab folder chooser:
it's possible to mod 'down' color via on_state(), but not worth it;
originally just buttons (not toggles) sans scrolling; they've grown;
the 'down' preselect uses simple '==' because app.startswith(shared)
on android, and everything.startwith(root) on pcs: broadest wins;
update: preselect now uses startswith() + decreasing len() sort ahead;
reset picker to button's storage root, even if already toggled down;
this is complicated, and basically reimplements toogles behavior,
but there seems no other way to catch a tab on selected toggles?;
the usual on_release callback doesn't get triggerred in this case;
on_touch events weirdly bubble through _all_ widgets that register
for them: must check touch position for intersection with self,
else this appears to always trigger on the same/wrong button (but
it's really the first button in .children - which is added last!);
note that this would work as a simple func or lambda assigned to
togglebtn.on_touch_down (like on_release), but was coded as a class
along the way due to a state-skew theory (before found bubbling);
most self attrs here would be found in enclosing method's scope,
but 'self' of enclosing method would be hidden: use all explicits;
this was first inline code, then nested func, then nested class;
==================================================================
"""
numtouch = 0
def __init__(self, togglelabel, togglepath, picker, devbar, root, navup=False):
super().__init__(text=togglelabel)
# this button's state
self.togglepath = togglepath # used by sort for preselect, etc
self.picker = picker # rest used on touch callbacks
self.devbar = devbar # picker = popup, devbar = buttons
self.root = root # root = self of enclosing method
self.navup = navup # used to allow 1 '..' for appspec
# stadard toggle button bits
self.background_color = 'red'
self.group = 'pickstorage'
self.allow_no_selection = False
# set ToggleButton size, so BoxLayout has size, so ScrollView scrolls
# self.padding_x = 144
self.padding_x = kivy.metrics.dp( # scale to density [1.1.0]
72 if not RunningOnAndroid else 55) # see docs at top of .kv
self.size_hint = (None, 1)
self.bind(texture_size=self.setter('size'))
def on_touch_down(self, coords):
self.__class__.numtouch += 1
if not self.collide_point(*coords.pos):
# it wasn't on self, but we get it anyhow
# trace('touchdown %d ignored=>' % self.numtouch, id(self))
return False # keep bubbling, you magnificent bastard
else:
# it's in this widget's screen area
# trace('touchdown %d hit==>' % self.numtouch, id(self))
if self.state == 'normal':
# was up: clear all buttons sans self, change picker
for child in self.devbar.children:
child.state = 'normal'
self.state = 'down'
self.root.reset_main_picker_path(self.picker, self.togglepath, self.navup)
else:
# was down: keep self down (only), change picker
# oddly, need to reset ~elsewhere first, else no-op
elsewhere = osjoin(self.picker.ids.pathname.text, '.')
self.root.reset_main_picker_path(self.picker, elsewhere)
self.root.reset_main_picker_path(self.picker, self.togglepath, self.navup)
return True # end processing: did all toggle behavior manually
#---------------------------------------------------------------------
# back to storage-buttons setup
#---------------------------------------------------------------------
# or .children[:], .remove_widget()
devscr = picker.ids.devicebuttonsscroll
devbar = picker.ids.devicebuttons
devbar.clear_widgets() # likely superfluous - it's a new popup
# shared: really primary on-device (formerly 'SHARED') (never None)
lshared = 'PHONE' if RunningOnAndroid else 'PC'
bshared = StorageToggleButton(lshared, shared, picker, devbar, self)
devbar.add_widget(bshared)
#--------------------------------------------------------------------------
# drives: usb + microsd + cd/bdr (and other usb; hubs work on android too!)
# should drivename be truncated for button width? no: devbar now scrolled
# note the drivepath=drivepath, else same path for all in gui (see lp5e!)
# prior point now moot: lambda here replaced with call to external class
#--------------------------------------------------------------------------
for (drivename, drivepath) in drives:
bdrive = StorageToggleButton(drivename, drivepath, picker, devbar, self)
devbar.add_widget(bdrive)
# app: optional, never on PCs; after usb, because less used
if appspec != None and self.ids.appfolderselections.active:
bappspec = StorageToggleButton('APP', appspec,
picker, devbar, self, navup=True) # allow 1 up-nav
devbar.add_widget(bappspec)
# root: optional, weak|None on android, usable on pcs (system drive on Widows)
if root != None and self.ids.rootfolderselections.active:
# iff Config, show '/' if access else /<other>
broot = StorageToggleButton('ROOT', root, picker, devbar, self)
devbar.add_widget(broot)
# test: to verify storage-button scrolling sans devices
teststoragescroll = False
if teststoragescroll:
for i in range(5):
ltest = 'BREAKME' + str(i)
btest = StorageToggleButton(ltest, shared, picker, devbar, self)
devbar.add_widget(btest)
##bshared.size_hint = (None, 1)
#bshared.text_size = devscr.width, None
#bshared.size = bshared.texture_size #'120px' #bshared.texture_size[1]
##bshared.bind(texture_size=bshared.setter('size'))
# tbd: show documents folder too? - it's a shared sub, and seems too much
#--------------------------------------------------------------------------
# preselect: set the button for the storage in which start path is located;
#
# some storages nest in others (e.g., APP in PHONE on Android, PC in ROOT
# on PCs), so match prefix by longest first = most specific/nested; this
# works because longer means more specific in hierarchical filesystems;
# no button is selected if no match: e.g., in '/' and root disabled on pcs;
#
# the sort also handles linux 'os' at /media/me and drives at /media/me/xxx
# [but see (feb24) note at similar code above: the 'os' mount os C: was bad],
# as well as the fact that linux mounts might be anywhere (a mount in home
# (PC) will be longer than home itself - though curr only greps in /media
# and /mnt: there will be no storage-type root for mounts in /home in the
# chooser, but users can manually enter anything (preselect may be anything);
#
# see also kin: label_path_string() [1.1.0], rootpath nav above [1.1.0]
# (some of these could probably have merged or reused code, in retrospect);
#---------------------------------------------------------------------------
# sort bar buttons by decreasing path length = longest (most specific) first
btnsbypathlen = sorted(devbar.children,
key=(lambda c: len(c.togglepath)), reverse=True)
for btn in btnsbypathlen:
if abspickstart.startswith(btn.togglepath): # absolute path [1.1.0]
btn.state = 'down'
break
# size the bar for its buttons, so it scrolls: => now in .kv
"""DEFUNCT
-------------------------------------------------------------
# output: 0 0, 100
##trace(devbar.minimum_height, devbar.minimum_width)
##trace(picker.ids.devicebuttonsscroll.width)
def setdevbarwidth(dt):
# after widgets are drawn: have sizes
devscr = picker.ids.devicebuttonsscroll
devbar = picker.ids.devicebuttons
# output: 0 692, 13920
##trace(devbar.minimum_height, devbar.minimum_width)
##trace(picker.ids.devicebuttonsscroll.width)
devbar.size_hint = (None, None)
devbar.height = devscr.height
##devbar.width = max(devbar.minimum_width, devscr.width)
devbar.width = devbar.minimum_width
##trace('devbar.width=>', devbar.width)
# it doesn't work yet here
Clock.schedule_once(setdevbarwidth) # more bind()s might help too
##picker.do_layout()
##devscr.bind(width=lambda i, v: setdevwidth(0))
-------------------------------------------------------------
DEFUNCT"""
#------------------------------------------------------------
# update: binds work better, though kivy scrolling is far too subtle;
# max() in defunct above triggered scroll long before buttons clipped;
# devbar.height = devscr.height <= needed iff size_hint=(None, None);
#
# THE KICKER: the next two lines can be done entirely in the .kv, and
# even before any buttons are added to the BoxLayout (though buttons'
# size still must be arranged as they are built above); .kv is easier
# way to setup bindings that will fire later; alas, Kivy is so tersely
# documented, that you have to do things wrong a few times first...
#
# now in the .kv file
##devbar.size_hint = (None, 1)
##devbar.bind(minimum_width = devbar.setter('width'))
#------------------------------------------------------------
#---------------------------------------------------------------------
# post that dialog
#---------------------------------------------------------------------
self._popups.push(Popup(title='Choose %s Folder' % kind,
content=picker,
auto_dismiss=False, # ignore taps outside
size_hint=(0.9, 0.9))) # need max space
self._popups.open()
#self._popups.top().border = [0, 0, 0, 0] # works iff background image
# this can't be set in the .kv spec: see the note there
# and can't be done till after return to kivy loop (omg)
def do_this_later(dt):
picker.ids.filechooser.view_mode = initstyle()
Clock.schedule_once(do_this_later) # asap after open
def try_any_nohang(self, func, *args,
waitsecs=0.25, onfail=None, log=True): # [1.1.0]+ (feb24)
"""
--------------------------------------------------------------------
[1.1.0] Run any system call with a timeout to avoid GUI hangs.
Returns onfail for exception or timeout, else func(*args)'s result.
For portability, runs func(*args) in a thread with a timed join.
Also wraps the call in a try handler to catch any/all exceptions.
Initially coded for os.listdir() only, then updated for any func.
UPDATE: os.isdir() hangs on Windows (but not macOS) too: generalize
this to call any folder-access function with a timeout. This now
returns onfail if func(*args) either raised an exc before timeout
or hung; else returns func result if it ended without exc or hang.
Still not universal (no **kargs), but handles this app's use cases.
Windows drive enumeration now also uses this, not a custom version.
Timeouts can also be had with signals on Unix, but not on Windows,
and they are also available in multiprocessing, but not on Android.
Threads always finish even if timeout: no simple way to kill, but
a shared result object per thread ensures that new results aren't
overwritten. Hung thread is left to finish (exit on target return)
and modify a result object that is ignored by the calling thread.
But set daemon=True so thread auto-killed at app exit if hung that
long (unlikely); else inherits False from app process and may delay
app exit (which _may_ have triggered a kivy logger loop on exit).
INITIAL: run an os.listdir() to verify folder access before ops
that rely on it, but run it in a thread with a timeout to avoid
lags/hangs for drives that are inaccessible - temporarily or
permanently. This includes mounted but disconnected network
drives, but also others (e.g., USB drives not yet fully mounted).
Returns False iff os.listdir() either failed immediately or hung.
Used on all platforms and all contexts - chooser start folder,
chooser storage tap/reset, and Main action folder prechecks.
Also used for an Android hang in storage_path_app_specific().
[1.1.0]+ (feb24) UPDATE: change waitsecs default timer duration
from 0.50 to 0.25 seconds; half a second is a bit jarring when a
Windows mapped network drive is offline (as they regularly are),
and a quarter of a second should be plenty to notice a ghost.
Also added logs=True param to allow callers to turn off traces
(traces can be too much, but never happen in released Android apps).
--------------------------------------------------------------------
"""
def try_any(func, args, result, onfail):
try:
got = func(*args) # run folder-access call: hangware
except:
result.append(onfail) # failure, immediate, or later if hung
else:
result.append(got) # success, now or later; got may = onfail
result = [] # new shared object for each thread
nohang = threading.Thread(
target=try_any, args=(func, args, result, onfail), daemon=True)
nohang.start()
nohang.join(timeout=waitsecs) # wait till thread finished or timeout
if nohang.is_alive():
if log: trace('try thread timeout')
sendback = onfail # let thread finish and set its ignored result
else:
if log: trace('try thread exit')
sendback = result[0] # thead finished and set result: ok or exc
return sendback # can be onfail for exc, timeout, or got
#----------------------------------------------------------------------------
# No-hang helpers
#
# try_listdir_nohang() => True IFF listable, empty or not: no list returned;
# use a try_any_nohang() instead if list is needed too (see examples ahead);
# try_isdir_nohang() => False if exc, timeout, or false (callers don't care);
#----------------------------------------------------------------------------
def try_listdir_nohang(self, path):
return self.try_any_nohang(os.listdir, path, onfail=None) != None
def try_isdir_nohang(self, path):
return self.try_any_nohang(os.path.isdir, path, onfail=False)
# Other try_any() clients: Windows drive enumeration, Android drive name, file-system
def reset_main_picker_path(self, pickdialog, pickpath, navup=False):
"""
--------------------------------------------------------------------
On device/storage name tap in Main folder chooser: reset its path + rootpath.
Changing the arg-based property changes the chooser's content (magically).
(Update: usually; toggle-down clicks resets have to change ~ elsewhere first.)
If navup, rootpath includes '..' after path, to allow nav 1 level up (only).
Run os.listdir here to make sure it's still there - USB may be removed,
though USB won't be in the devices list if user skipped All files access.
Storage permission is checked both before we get here for the folder
chooser, and before user confirmation for Main-tab actions. Folder
access is checked both here, and just after Main-tab action perm check.
----
UPDATE: storage-type-in-list-but-inaccessible is an error state common on
newer Androids for the first run (or later, if A-F-A perm is granted later).
This now auto-Cancels the chooser dialog on info-message Okay in this state.
Else, the dialog remains open with the failed button down; the display is
that of a prior type; and it's unclear which button to auto-select for calls.
Users must restart the app as instructed if permission is pending, though the
dialog could remain for mount pending with prior button still set if recode
callers to check listdir() _before_ this is called or the fail button is set.
Leaving the dialog open may be better for pending mounts, but seems worse for
pending permissions that require a full app restart - an immediate Back no
longer kills the app if popups are open, so there are two steps to close.
Closing the dialog is a compromise, with a trivial reopen for pending mounts.
----
NIT: chooser open has been seen to hang briefly (2~7 seconds) while a USB
drive finishes mounting on some devices - notably, a Fold4 which had just
been granted A-F-A without a restart, and a Note20U in the same state. This
might trigger ANR, but has not yet, and it's unknown where the hang occurs
(may be here or in the chooser's code). On the Fold4, later pending-mount
opens do not hang, and do not return the pending drive in the removables
list till it's fully mounted; the next note docs Note20U's odd behavior.
PUNT - rare, one time, harmless, very complex to address (thread?).
----
ALSO NIT: a Note20U (Android 10) adds a drive in pending-mount state to the
chooser list with inaccessible path '/dev/null' till mounted; and, worse,
leaves it in the list after drive unmount as inaccessible (to listdir)...
UNTIL the user physically removes the drive's cable from the phone. Both
cases yield error popups in this app, and now dialog auto-Cancels.
By contrast, a Fold4 (Android 13) correctly does NOT add drives to the list
till they are fully mounted; and correctly does NOT leave them in list after
they are unmounted - regardless of the cable's status. (The dialog refetches
the list on each dialog open, so all new states may require a new dialog.)
This is too convoluted to address - and likely bugs in older Androids or
Samsung add-ons. The only known fix for Note20U oddness is to catch USB
mount/unmount events, and this would be required only for the now-fading
Android 10 and earlier (presumably) user base. PUNT; users will have to
use some common sense when the now-augmented error dialog here appears...
----
UPDATE [1.1.0] hangs for not-yet-mounted drives on Android or elsewhere
will no longer happen here, because the listing call is run with a timeout.
This (plus similar timeouts elsewhere) addresses the fold4 case NIT above,
as well as network drives that have become inaccessible since dialog open.
UPDATE [1.1.0]: the Android run1 hang was eventually isolated, and resolved
with a timeout in storage_path_app_specific().
UPDATE [1.1.0]+ on macOS only, due to the elimination of hangs, listdir
(et al) now returns immediately if the system permission-verify popup is
opened (this formerly paused/hung until the popup was closed). Tweak
the message to clarify that an app restart is not needed (hopefully).
Also, qualify the opening as "on Android" because it won't apply
anywhere else: Windows+Linux have no such popups, macOS popups differ.
The opener might be omitted on PCs, but its second sentence applies.
For 1.1.0, Android+Windows_Winux don't need to care: it's moot or not.
This model is also used at verify_paths() and init_logfile_folder().
--------------------------------------------------------------------
"""
"""
try:
os.listdir(pickpath)
except:
# nested modal dialog
else:
# pickdialog.path bound to pickstart via creation arg
"""
# same, but use timeouts to avoids lags for inaccessible drives [1.1.0]
listed = self.try_listdir_nohang(pickpath)
if not listed:
# hung (inaccessible), or immediate fail (unmounted?)
# nested modal dialog
noaccessmsg = (
'Cannot access %s.' # [1.1.0] add dots to all line1
'\n\n'
'If you just granted storage permission on Android, ' # [1.1.0]+
'please try restarting this app now.'
'\n\n'
'Otherwise, retry after ensuring that drives '
'have been fully mounted and recognized.'
)
if RunningOnMacOS:
noaccessmsg += (
'\n\n'
'On macOS, you may be able to simply rerun the last '
'request if you just handled the permission popup.' # [1.1.0]+
)
if RunningOnAndroid and self.android_version() <= 10:
noaccessmsg += (
'\n\n'
'On older Androids, you may need to wait for '
'mounts to finish, or detach after unmounts.'
)
if not RunningOnAndroid: # [1.1.0] added
noaccessmsg += (
'\n\n'
'On PCs, also check folder permissions and locks, '
'and network-drive status.'
)
closechooser = lambda: self.dismiss_popup(pickdialog)
self.info_message(noaccessmsg % pickpath,
usetoast=False,
nextop=closechooser) # close chooser too
else:
# storage root accessible now
# pickdialog.path bound to pickstart via creation arg
up = '/..' if navup else '' # android app-specific only
pickdialog.pickstart = pickpath # reset chooser's content (ids.fc.path)
pickdialog.rootpath = pickpath+up # remove '..', avoid excs for bad dirs
# left align on reset, can be scrolled after
def leftalign(dt):
pickdialog.ids.pathname.cursor = (0, 0)
Clock.schedule_once(leftalign)
def do_main_path_pick(self, kindid, popuppath, popupcontent):
"""
On Pick, copy chosen path to Main tab, and close the picker
dialog (which shows the same path next time). Don't save
from/to paths until a Main-tab action is run (else may be a
bogus path, and would need to catch any manual changes too).
No need to clear selection: TextInputNoSelection disables it.
(Formerly: clear_selection() here doesn't fix lingering selection
handles, and prior selection auto disappears with new popup.)
"""
# now passed in from .kv file
# popuppath = self._popups.top().ids.pathname.text # brittle, that
mainpath = self.ids[kindid] # from|to, Main tab
mainpath.text = popuppath # from or to path
self.dismiss_popup(popupcontent)
def dismiss_popup(self, popupcontent):
"""
Used for all modal popups; this coding avoids setting
popup content's close handler after the popup is made
(popup needs content, but content close needs popup...).
Now just a convenience interface to the popups stack.
Kivy Popups also have an auto_dismiss, which runs dismiss()
on taps outside the dialog, and is enabled by default.
UPDATE: pass Popup's content to avoid closing an overlaid
dialog on double taps (see popups stack for more info).
Also set Kivy auto_dismiss=False to prevent taps and closes
outside scope of popups stack (plus ambiguous+twitchy).
"""
self._popups.dismiss(popupcontent) # topmost: no longer assumes only 1 modal
"""
UNUSED
def filter_dirs_by_permission(self, itemslist):
""
In Main filechooser, rule out folders that can't be accessed.
These don't open in the GUI, but trigger uncaught exceptions.
UNUSED: only folders accessed by ".." generate uncaught
(harmless) exceptions: avoid by setting chooser's rootpath.
""
def accessible(folder):
try:
os.listdir(folder) # caution: may hang
except:
return False
return True
return [item for item in itemslist
if not os.path.isdir(item) or accessible(item)]
"""
#=======================================================================
# MAIN-TAB ACTION - OPTIONS (DOC)
#=======================================================================
"""
This program runs tasks that are long running, and potentially
_very_ long running. These tasks cannot block the GUI, and so
must be run in parallel with it. To do so, there are a number
of options whose tradeoffs vary per plaform:
1) Simple child processes created by subprocess.Popen()
Continue to run if the app or program is left for another,
(which pauses the app on Android), and are a portable tool.
This is an option on PCs, but not Android.
On Android, child processes may be arbitrarily terminated
by the Android 12+ 'phantom' process killer, which makes
this a nonstarter. On PCs, there is neither pause equivalent
nor processes killer, and this is a standard option.
This is what was used for scripts and GUIs on Android in the
Termux and Pydroid 3 apps. Neither ever saw a process kill,
but this varies by phone usage patterns.
This mode requires an IPC scheme to signal script-process exit
to the GUI process, unless the GUI is polling for EOF on the
logfile read by a threas (see Mergeall's GUI launcher) or the
appearance of a sentinel file (see Frigcal's launcher).
The tkinter GUI did EOF polling with unbuffered output streams +
a consumer thread's blocking-read loop + a timer-polled thread
queue. This costs cpu and power on phones, but a similar scheme
can be used here on PCs to skip IPC altogether.
TBD: the next option may work just as well on PCs.
YES: thread scheme coded for Android adopted for all PCs too.
2) Threads spawned by the GUI process
Continue to run if the program or app is left/paused; are fully
portable to PCs and Android; are immune to Android process kills;
and require no IPC mechanism to signal their exit (an in-process
callback can be run directly in the thread-wrapper's exit code).
This is available on Android as an option, and was the first coding.
It's only downside seems to be that script exit cannot be reported
to users until the app is resumed.
Threads may incur a minor speed hit for thread-switching performed
within the Python virtual machine (see its global interpreter lock).
However, this is likely to be minor, and negligable in multi-minute
runs. Per benchmarks, there is _no_ noticable speed difference for
the same tasks run as process in Termux and thread in the app (and
later, service in the app). Indeed, Android pseudo-nondeterminism
makes processes as likely to lose the race.
3) Android-only Foreground services, which run scripts in auto processes
Continue to run if the app is left, and are unlikely to be killed
by Android, but are fully proprietary and nonportable, and require
a simlarly proprietary IPC mechanism to catch exits while paused.
This is used by default on Android, for its notifications.
The only advantage over threads is that notifications are posted
for tasks, which sufficces to report script status and exit to the
user while the app is paused. The app's toast is posted on script
exit but doesn't appear except in the app; global overlays are
possible, but intrusive (and possibly rude).
Service processes may require an IPC mechanism for exits, because
the script runs in a separate process (callbacks are out); EOF or
sentinel-file polling in the GUI may not detect a script exit while
the app is left/paused; and EOF polling requires setting stdout to
be unbuffered for the script's run. Unix IPC and pipes are unusable,
because the process is auto-spawned by Android.
As coded, script exit is coded as a proprietary BroadcastReceiver.
In hindsight, given that toasts don't overlay other apps and can't
appear intil the app is revisited, signal-file polling may suffice.
Complication: foreground services keep running if app killed (as
supported in kivy/p4a today), so must check if still running on app
restart, and reset in-progress state, as well as any postscr callback.
This is handled with a service breadcrumb file as coded here. The
alternative is killing the service with the app, and seems as hard.
UPDATE: punt - Back-button app kills now prohibited if any Main action
is in progress, as service-running detection is inaccurate and iffy.
Android services can also run a thread, which may avoid IPC, but
also requires from-scratch service coding, and has no clear benefit.
4) The multiprocessing module's processes
Cannot be used on Android, due to a missing semaphore call,
and seems overkill and seems likely to be slower in any event.
"""
#=======================================================================
# MAIN-TAB ACTION: PROCESS
#=======================================================================
def launch_script_process(self,
opname, script, cmdargs, logfilepath, stdin=None, postop=None):
"""
--------------------------------------------------------------------
On PCs, launch the script as a simple subprocess.Popen() child
process, with sentinel-file or EOF polling to detect script exit.
DEV: it's not clear that this would be any better than using the
portable thread scheme developed for Android intially...
UPDATE: now using threads on all PCs. This has not discernable
performance negative, but the future remains to be written...
--------------------------------------------------------------------
"""
return # TBD (sort of)
"""
# catch app close (prints okay: new process)
self.script_running = PROCESS_RUNNING
self.on_script_start(opname) # label, gif, buttons
self.on_script_exit(opname, logfilepath, logfile=None) # updates gui
"""
#=======================================================================
# MAIN-TAB ACTION: SERVICE+PROCESS
#=======================================================================
def launch_script_service(self,
opname, script, cmdargs, logfilepath, stdin=None, postop=None):
"""
--------------------------------------------------------------------
Run the Main -tab action's script as an Android ForegroundService,
in an automatically spawned process (but not 'sticky': restarts).
This allows the action to keep running when users leave the app
(like threads), and reports script status and exit with a normal
notification while paused (unlike threads).
Much of the java interface is automated by p4a (unlike FileProvider!),
including both a java service class and pyjnius broadcast-receiver
code for process-exit dispatch, though there are glaring seams (see
the hurdles list in the service runner). This scheme also requires:
1) buildozer.spec service=arg setting (below), which maps service
name to python file, and triggers creation of java service class.
Also: +FOREGROUND_SERVICE in permission settings (crucial, undoc!).
2) run_script_as_service.py, which has the main+wrapper code run in
the service process that unpacks args and runs the target script's
code. This file is auto run as a script on service start below.
3) Broadcast send in the closing code of #2, using an Intent and the
same message ID used to create the BroadcastReceiver here.
Docs (such as they are):
https://python-for-android.readthedocs.io/en/latest/services/
(ditto)/en/latest/apis/?highlight=broadcast#module-android.broadcast
https://github.com/kivy/python-for-android/... (for code spelunking)
Generated java service class(ES):
~/.../PC-Phone-USB-Sync/.buildozer/android/platform/
build-arm64-v8a_armeabi-v7a/dists/usbsync(TRIAL?)/
src/main/java/com/quixotely/usbsync(TRIAL?)/ServiceRunscript.java
Min: service.start(mActivity, argument)
Max: service.start(mActivity, 'small_icon', 'title', 'content' , argument)
Empty strings on three args pick reasonable defaults in generated Java code
--------------------------------------------------------------------
"""
# java dir varies by app: here (start + msgid), service wrapper (exit msg)
appsuffix = 'usbsync' if not TRIAL_VERSION else 'usbsynctrial'
# config gui and msg receiver in this process
msgid = self.service_in_progress_state(opname, logfilepath, postop, appsuffix)
# pass args as json string
argsdict = dict(mainfilepath=script,
argslist=cmdargs,
stdouterr=logfilepath,
onexitid=msgid,
appname=appsuffix,
stdin=stdin)
argumentstr = json.dumps(argsdict)
trace('json=>', argumentstr)
# start service+process (via generated java class: dirname differs per app!)
service = autoclass('com.quixotely.%s.ServiceRunscript' % appsuffix)
mActivity = autoclass(ACTIVITY_CLASS_NAME).mActivity
service.start(mActivity,
'', APPNAME_FULL_OR_TRIAL, 'Running %s action' % opname , argumentstr)
# and in buildozer.spec (not + ':sticky' = restart):
# services = runscript:run_script_as_service.py:foreground
# save breadcrumb file for app restarts while service still running
# no longer used: worked, but service-running test is deprecated
# and error prone; see check_for_running_service() for more docs
"""
svcrecord = open(SERVICE_BREADCRUMB, 'wb')
pickle.dump([opname, cmdargs], svcrecord)
svcrecord.close()
"""
def service_in_progress_state(self, opname, logfilepath, postop, appname):
"""
--------------------------------------------------------------------
A method, because used both at service launch, and by app
start-up code if service in progress post app kill+restart.
We either need to kill a service on app exit (kivy/p4a don't),
or detect and reset in-progress state to avoid allowing a new
action run. Threads don't need to care: they die with the
Android app. On restarts: opname unknown, and postop is MIA.
Nit: should msgid vary for full/trail apps? This may or may not
matter, users seem very unlikely to run both at once, and maybe
we don't need to care. App-name diffs are _forced_ for both the
foreground service itself (by the names of generated java-code
folders), and OPEN's file providers (by installer which checks
that provider name is not already used by an installed app).
UPDATE: appname (suffix) now used in broadcast message id here
too. It's unknown if this is required for message ids, but was
adopted here after installer mandated it for file providers...
--------------------------------------------------------------------
"""
# catch app close (prints okay: new process)
self.script_running = SERVICE_RUNNING
self.on_script_start(opname) # label, gif, buttons
# start script-exit broadcast receiver
msgid = 'com.quixotely.%s.SCRIPT_EXIT' % appname
def onexit(context, intent): # always 2 args!
self.on_service_process_exit(opname, logfilepath, postop)
self.breceiver = BroadcastReceiver(
callback=onexit, # run in this process, not service
actions=[msgid]) # intent action name, arbitrary
self.breceiver.start()
return msgid
def on_service_process_exit(self, opname, logfilepath, postop):
"""
--------------------------------------------------------------------
On receipt of the script-exit broadcast message from
service/process in the main app/gui process.
Same as for thread, but run postop in main app/gui process
here and now (the thread variant runs it in the thread
runner itself), and the service process closes the logfile.
This will be triggered whether the app is foreground or
paused, though the toast it generates doens't appear unless
and until the app is in the foreground (visible). There
are ways to force app overlays, but this is too intrusive.
The service does, however, post a notification while running.
This notification usually shows up after ~8 seconds on first
run (and faster thereafter); is not noticable for very short
runs; and is auto-removed on service exit. It also qualifies
as useful and ample user notice, and warrants using a service.
This callback apparently has 5 seconds to finish; bizarro!
UPDATE [1.2.0] there is a rumor (well, just 2 unrecreatable
aborts among a gazillion runs) that pyjnius code somewhere in
breceiver.stop() is prone to raise an uncaught exception that
closes the app. Try to address here, but this may be due to
holding Java objects too long (or not long enough), and is too
low in the libs to be sure (it's a hard crash, with Java stack
details that mean zit here). Such is life with a glitchy stack...
--------------------------------------------------------------------
"""
if postop:
postop()
self.on_script_exit(opname, logfilepath, logfile=None) # updates gui
try:
self.breceiver.stop()
except:
pass # [1.2.0] maybe recover?
trace('exiting broadcast receipt callback')
#=======================================================================
# MAIN-TAB ACTION: THREAD
#=======================================================================
def launch_script_thread(self,
opname, script, cmdargs,
logfilepath, logfile, stdin=None, postop=None, timersecs=0.10):
"""
--------------------------------------------------------------------
Create a cross-thread object queue, start the script thread,
and start a timer-event queue consumer in the main GUI thread.
All stdout and stderr in script thread goes to Pipe's queue.
Unicode: logs always UTF-8, GUI always run through ascii().
Threads keep running when the app is left/paused, but script
exit cannot be reported to the user until the app is resumed
on Android (unlike services, they do not post a notifcation).
Threads are used on Android as an option (services are default,
except on Android 8), and also on macOS, Windows, and Linux PCs.
----
UPDATE: scrolled run output proved too slow, and was replaced
with an animated GIF plus on-demand TAIL/WATCH on Logs tab. For
this new mode, STATUS_ANIMATION, there is no need for either a
timer loop or queue. Instead, use a simple stream flusher, and
pass a callback object to be invoked when the thread runner closes
the streams flusher after the script's code finishes.
This works because it's all one process: the callback is in-process,
and the script runner in the thread knows when the script exits.
The callback uses kivy's @mainthread to ensure GUI-thread execution
for GUI upates triggered by the callback.
There is also no need for running display text through ascii():
kivy displays unknown glyphs as rectangles, which is as good/bad.
--------------------------------------------------------------------
"""
# catch app close, disallow prints
self.script_running = THREAD_RUNNING
self.on_script_start(opname) # mod label, gif, buttons
if STATUS_SCROLL:
self.text_size = 0 # reset output dislay/size
self.text_line = 0 # num lines for scrolling
self.pending_lines = '' # initialize lines batch
self.pending_count = 0
self.run_output.text = '' # clear text area in gui
if STATUS_ANIMATION:
pipe = Flusher(logfile,
on_eof=lambda: self.on_script_exit(opname, logfilepath, logfile))
if STATUS_SCROLL:
pipe = Queuer()
thread = run_script_thread_wrapper( # spawn the script thread
script, # path to script file
cmdargs, # sys.argv[1:] for script
pipe, # logfile proxy + exit signal
postop, # postop run by wrapper
stdin, # canned input iff used
chdir=True, # goto script's folder
trace=False) # trace=True for trace2()
if STATUS_ANIMATION:
pass # no timer loop needed
if STATUS_SCROLL:
def callback(dt):
self.scroll_on_action_timer_callback(pipe, logfile)
self.timer_callback = callback # keep a ref, ignore dt
Clock.schedule_interval(callback, timersecs) # auto-rescheculed timer
@mainthread
def on_script_start(self, opname):
"""
Common script-start GUI code: thread, service, process.
Assumes animation coding; former scroll scheme won't work.
[1.1.0] Update: set partial wakelock for cpu if screen off.
"""
self.ids.statuslabel.text = opname # action label
self.ids.statusimg.anim_delay = 0.20 # animate gif: 5x/sec
# disable action buttons
for actionbutton in self.action_buttons:
self.ids[actionbutton].disabled = True
# acquire partial wake lock on action start [1.1.0]
if RunningOnAndroid and not self.wakelock.isHeld():
trace('acquiring wakelock')
self.wakelock.acquire()
@mainthread
def on_script_exit(self, opname, logfilepath, logfile=None):
"""
--------------------------------------------------------------------
When action exits in animation coding, reenable app closes and
prints, stop animation, close logfile, reenable Main buttons.
This is now used by thread and service+process/broadcast modes.
It's likely thread and broadcast safe: scheduled by @mainthread.
Update: also clear arrowheads with .reload() - marginally neater.
Nit: could re-deable NAME on Windows here, but punting on this.
[1.1.0] Also scrape logifile summary info, and release Android
partial wake lock (supports screen timeouts: see App.on_start()).
--------------------------------------------------------------------
"""
# reenable app close, prints (thread or not), stop WATCH
self.script_running = NOTHING_RUNNING
self.ids.statuslabel.text = 'Action Status' # back to generic label
self.ids.statusimg.anim_delay = -1 # stop Main-tab animation
self.ids.statusimg.reload() # reset: clear arrows
if logfile: logfile.close() # if thread, not service+process
# reenable action buttons (maybe undo)
for actionbutton in self.action_buttons:
self.ids[actionbutton].disabled = False
if self.undo_verboten:
self.ids.undobutton.disabled = True # and Windows NAME? - punt
# release partial wake lock on action exit [1.1.0]
if RunningOnAndroid and self.wakelock.isHeld():
trace('releasing wakelock')
self.wakelock.release()
# try to scrape and display crucial info from logfile [1.1.0]
summary = self.scrape_action_summary_info(opname, logfilepath)
self.info_message('%s action finished.%s' % (opname, summary),
usetoast=(not ('\n' in summary)))
def scrape_action_summary_info(self, opname, logfilepath, tailsize=(10 * 1024)):
"""
--------------------------------------------------------------------
[1.1.0]: For convenience, try to parse out and present the crucial
logfile info in the exit popup. It's arguably hard to view logs on
a narrow phone sans landscape rotate, and summmary bits extracted here
normaly suffice, especially when verifying counts against syncs on PCs.
This must avoid hangs/lags by reading just the end of file (like TAIL
and WATCH), and the parse can easily fail and must recover (logs can
end with errors sans summaries). Also changed to propagate logfilepath
from spawners (service +thread); it originates in Main's confirmed().
--------------------------------------------------------------------
"""
def commify(obj):
# add commas to str|int number for display
if isinstance(obj, str):
obj = int(obj) # str=>int
return '{:,}'.format(obj) # always returns str
try:
# read tail only: don't hang gui
logsize = os.path.getsize(logfilepath)
logfile = open(logfilepath, 'r', encoding='utf8', errors='replace')
logfile.seek(max(0, logsize - tailsize))
logtext = logfile.read()
logfile.close()
if opname in ['SYNC', 'UNDO']:
scrape = (
r'^Compared => files: (\d+), folders: (\d+), symlinks: (\d+)\n'
r'^Differences => .*?\n'
r'^Changed:\n'
r'^files => created: (\d+), deleted: (\d+), replaced: (\d+)\n'
r'^folders => created: (\d+), deleted: (\d+), replaced: (\d+)\n'
)
matched = re.search(scrape, logtext, re.MULTILINE)
if matched:
groups = matched.groups()
summary = (
'Items compared:\n'
' Files: %s\n'
' Folders: %s\n'
' Symlinks: %s\n\n'
)
totals = [int(text) for text in groups]
nummods = sum(totals[3:])
if nummods == 0:
groups = groups[:3]
summary += 'No changes were made.'
else:
summary += 'File changes: %s\n' % commify(sum(totals[3:6]))
summary += (
' Created: %s\n'
' Deleted: %s\n'
' Replaced: %s\n\n')
summary += 'Folder changes: %s\n' % commify(sum(totals[6:9]))
summary += (
' Created: %s\n'
' Deleted: %s\n'
' Replaced: %s')
elif opname == 'SHOW':
scrape = (
r'^Compared => files: (\d+), folders: (\d+), symlinks: (\d+)\n'
r'^Differences => '
r'samefile: (\d+), uniqueto: (\d+), uniquefrom: (\d+), mixedmode: (\d+)'
)
matched = re.search(scrape, logtext, re.MULTILINE)
if matched:
groups = matched.groups()
summary = (
'Items compared:\n'
' Files: %s\n'
' Folders: %s\n'
' Symlinks: %s\n\n'
)
totals = [int(text) for text in groups]
numdiffs = sum(totals[3:])
if numdiffs == 0:
groups = groups[:3]
summary += 'No differences were found.'
else:
summary += 'Differences found: %s\n' % commify(numdiffs)
summary += (
' Same file: %s\n'
' Unique in TO: %s\n'
' Unique in FROM: %s\n'
' Mixed kind: %s')
elif opname == 'COPY':
scrape = (
r'^Copied (\d+) files, (\d+) folders, (\d+) symlinks'
)
matched = re.search(scrape, logtext, re.MULTILINE)
if matched:
groups = matched.groups()
totals = [int(text) for text in groups]
summary = 'Copied items: %s\n' % commify(sum(totals))
summary += (
' Files: %s\n'
' Folders: %s\n'
' Symlinks: %s')
elif opname == 'DIFF':
scrape0 = (
r'^Files checked: (\d+), Folders checked: (\d+), '
r'Symlinks checked: (\d+), Files skipped: (\d+)\n'
r'(?:.*\n)?'
)
scrape1 = scrape0 + '^No diffs found\.'
scrape2 = scrape0 + r'^Diffs found: (\d+)'
summary = (
'Items checked:\n'
' Files: %s\n'
' Folders: %s\n'
' Symlinks: %s\n'
' Skipped: %s\n\n')
matched = re.search(scrape1, logtext, re.MULTILINE)
if matched:
groups = matched.groups()
summary += 'No differences were found.'
else:
matched = re.search(scrape2, logtext, re.MULTILINE)
if matched:
groups = matched.groups()
summary += 'Differences found: %s\n'
uniques = logtext.count('\n- items UNIQUE at [')
differs = logtext.count('\n- files DIFFER at [')
differs += logtext.count('\n- links DIFFER at [')
misses = logtext.count('\n- items MISSED at [')
summary += (
' Same file: %s\n'
' Has uniques: %s\n'
' Mixed kind: %s'
) % tuple(commify(tot) for tot in (differs, uniques, misses))
elif opname == 'NAME':
scrape0 = r'^Visited (\d+) files and (\d+) folders\n'
scrape1 = scrape0 + r'^Total nonportable names found but unchanged: (\d+)'
scrape2 = scrape0 + r'^Total nonportable names found and changed: (\d+)'
summary = (
'Items visited:\n'
' Files: %s\n'
' Folders: %s\n\n')
matched = re.search(scrape1, logtext, re.MULTILINE)
if matched:
groups = matched.groups()
summary += 'Names found but unchanged: %s.'
else:
matched = re.search(scrape2, logtext, re.MULTILINE)
if matched:
groups = matched.groups()
summary += 'Names found and changed: %s.'
if not matched:
summary = '\n\nCould not extract summary.\n\n'
else:
commags = tuple(commify(text) for text in groups)
summary = summary % commags
summary = '\n\n' + summary + '\n\n'
# spurious for name and diff - today
indicator = r'^\*\*There are (error messages|non-error notes)'
if re.search(indicator, logtext, re.MULTILINE):
summary += '**This run has errors or notes to inspect.\n\n'
summary += 'See Logs for more info.'
except:
trace('scrape exc:', sys.exc_info())
summary = '\n\nCould not extract summary: see Logs.\n\n'
pass # bail on all file or parse errors
return summary
def init_mergeall_globals(self):
"""
--------------------------------------------------------------------
UPDATE: this is not required for foreground service+process or
process run options, as the script's global state is its own.
See also update_mergeall_configs_maxbackups() for related mods.
A downside of running Mergeall scripts as strings in a thread: any
globals in imported modules aren't reinitialized per run, because
the script is rerun in the same process. For example, this causes
problems in backup.py, because it uses globals to trigger creation
of a backup-name timestamp once per run; if not reset, the same
backups folder is created for each mergeall.py run in the app; ouch.
The options. 1) Going back to subprocesses instead of threads fixes
the global issue, but makes app actions subject to the phantom-process
killer for child processes on Android 12+; this is a non-starter.
2) Importing modules with global state and resetting globals before
mergeall.py runs (something similar is already done for changing
mergell_configs.MAX_BACKUPS changes). 3) Removing the offending
modules from sys.modules to force reimports on each run. Of these,
2 seems least risky, given the many changes to imports in python 3.Xs.
Note that this pertains only to loaded modules; the top-level script
is recompiled from a string each time, and hence starts from scratch.
This is also required for mergeall.py only: on SYNC (for backups and
copies), and UNDO (for copies), but not on SHOW (which does neither).
NIT: this could avoid imports if modules are already in sys.modules,
but the import is required on first thread run, and trivial after.
NOTE: none of the attibutes reset here show up as function argument
defaults, which could make the resets useless - see maxbackups...
Running as a thread with exec also makes output tricky (all prints
go to the script's output pipe), and requires that the cwd is moot.
OTOH, random child-process kills are a Very Bad Thing in syncs...
--------------------------------------------------------------------
"""
# ./mergeall/ modules are loaded after a run - in just one form
# for mod in sys.modules.values():
# trace('$>', mod) # __name__, __file__
# no-op if not running a thread: process imports anew
# TBD: also no-op for simple process, iff used on pcs
# as coded, this assumes either service else thread,
# probably better to call from launch_script_thread()
if RunningOnAndroid and self.ids.runactionasservice.active:
return
else:
# for thread runs of mergeall.py only: android or pc
trace('ma=> globals reset: backup, cpall')
os.chdir('mergeall') # in app-install (unzip) folder
sys.path.append(os.getcwd()) # .pyc not text, '.' fails here
import backup, cpall # identical to mergeall.py runs
backup.datetimestamp = None # run in subfolder, same process
backup.pruned = False # reset globals: as if new process
backup.firstbkpmsg = True
cpall.anyErrorsReported = False
os.chdir('..') # back to app's cwd, and import path
sys.path.pop()
# NO LONGER USED
"""UNUSED
def scroll_on_action_timer_callback(self, pipe, logfile, linesbatch=8):
""
On timer event: consume next batch of output lines queued by
action thread, if any, and route each line to the line handler.
Returning False here means don't reschedule callback in Kivy.
This structure avoids blocking main GUI thread while script runs.
""
try:
line = pipe.linequeue.get(block=False)
except queue.Empty:
return True # reschedule timer loop
# process a batch of lines
batchnum = 0
while True:
if line == pipe.eofsignal:
self.scroll_on_output_line(None, logfile)
return False # end of consuming loop
else:
batchnum += 1
self.scroll_on_output_line(line, logfile) # add to log and gui
if batchnum == linesbatch:
break # back to top of timer loop
elif pipe.linequeue.empty():
break # back to top of timer loop
else:
line = pipe.linequeue.get(block=False) # back to top of batch loop
return True # reschedule next callback
UNUSED"""
# NO LONGER USED
"""UNUSED
def scroll_on_output_line(self, line, logfile, guimaxchars=(256 * 1024), guibatch=100):
""
Handle next output line from action thread.
Run from on_action_timer_callback() for lines in queue.
This avoids overflflow in the text widget by halving text (TBD).
This avoids Unicode issues in the GUI with ASCII mapping (TBD).
UPDATE: despite tweaking timer and queue batch settings and halving
of text size, scrolling line by line was too slow on phones - thread
runtimes > doubled, and runs occasionally hung or terminated.
Tried changing to send text to GUI only in guibatch chunks here,
This helped with speed, but a few random crashes were still seen.
PUNT: on realtime scrolling, and use an animated GIF for run
status, along with tail/watch on-demand options in the Logs tab.
NIT: the logfile write() here should really catch Unicode errors and
use ascii() like the Flusher class does, but we're punting anyhow...
""
if line == None:
# eof at thread exit
# same as animation endthread (plus image)
self.on_script_exit(?, logfile) # has changed...
# show final lines batch in gui
self.run_output.text += self.pending_lines
self.ids.runoutputscroll.scroll_y = 0
self.pending_lines = ''
self.pending_count = 0
else:
# next output line from thread
try:
#trace2('>>', repr(line))
logfile.write(line) # <= should catch Unicode errors
#logfile.flush() # to tail file (slows slightly)
except:
self.info_message('Cannot write line to logfile.', usetoast=True)
line = ascii(line[:-1]) # avoid Unicode probs in GUI, keep \n
line = line[1:-1] # strip quotes added by ascii()
line = line + '\n' # lines always have a \n at end
linelen = len(line) # len after any ascii() expansion
if self.text_size + linelen >= guimaxchars: # too big: chop in half
trace2('Halving text', self.text_size)
fulltext = self.run_output.text # else text widget hangs
fulllen = len(fulltext)
halflen = fulllen // 2
halftext = fulltext[halflen:]
self.run_output.text = halftext
self.text_size = fulllen - halflen
self.text_line = halftext.count('\n')
self.text_size += linelen
self.text_line += 1
self.pending_lines += line
self.pending_count += 1
if self.pending_count == guibatch:
# show batch of text in gui now
self.run_output.text += self.pending_lines # insert_text(s, from_undo=False)
#self.run_output.cursor = (0, self.text_line) # scroll to line added? - no
# textinput.cursor doesn't hscroll (kivy bug): use scrollview wrapping it
self.ids.runoutputscroll.scroll_y = 0
self.pending_lines = ''
self.pending_count = 0
# gui redrawn auto now or on return to timer loop
# alt: force redraws with self.canvas.ask_update())
UNUSED"""
#=======================================================================
# MAIN-TAB ACTIONS
#=======================================================================
# All Main-tab action button ids in .kv file
action_buttons = [
'syncbutton',
'showbutton',
'undobutton',
'copybutton',
'diffbutton',
'namebutton'
]
def confirm_action(self,
message, # text for confirm popup, None=preconfirmed
opname, # text of button pressed
script, # path to main script file
cmdargs, # list of args for script
postcon=None, # run iff confirm, before script, in gui thread
postscr=None, # run iff confirm, after script, in gui|script thread
stdin=None, # string to pipe into thread's stdin
popupcontent=None # propagated from callback by preconfirmed callers
):
"""
--------------------------------------------------------------------
Confirm action with message.
Launch script thread/service/process iff confirmed.
Also save settings now: configs and from/to paths.
Split into steps/continuations to force modal dialogs.
Formerly started a GUI timer loop; now just status animation,
and wait for thread exit or broadcast message. Simple process
mode may start a polling loop for pipe reads; tbd.
postcon: called iff confirmed, before script, in the GUI thread.
postscr: called iff confirmed, after script, in script thread for
thread mode, and in GUI thread in service mode on message receipt.
postscr is complicated by fact that app may be restarted while
a service is still running. To support app restarts, save opname
to a file, and handle UNDO's postscr callback specially on restarts.
UPDATE: punt - service-running detection is deprecated and iffy.
UPDATE: if message is None, it's assumed that the action has been
preconfimed with a still-open popup - skip the normal confirmation
dialog, and run the action as if Yes had been pressed. Added for
the NAME report|update rewrite ahead; a bit of a kludge, but it's
simpler than refactoring confirmed() at this point in the project.
UPDATE [1.2.0]: this is now 3 steps instead of 2, to allow for up
to 2 modal dialogs before action run: new Android 13+ notification
permission infobox + dialog, and/or prior user-confirmation dialog.
--------------------------------------------------------------------
"""
def confirm_confirmed(popupcontent):
"""
----------------------------------------------------------
[confirm_action step #3]
On user confirmation of any Main-tab action.
Uses multiple variables in the enclosing scope.
Storage perms now checked in verify, before this.
Notification perm resolved before this, if needed.
Pruning + new logfile here will cause the Logs tab
to reload its logfile list when switch to Logs tab.
[1.2.0] Pruner now filters out new logfile by name
passed here, else may prune newly made logfile if
host's date/time is old. See make_new_logfile().
----------------------------------------------------------
"""
# close run confirmation
self.dismiss_popup(popupcontent)
# always log, but auto-clean folder (Logs tab will reload)
logfileandpath = self.make_new_logfile(opname)
if not logfileandpath:
return
logfile, logfilepath = logfileandpath # bailed if failed
# prune for new log on each run (Logs tab will reload)
self.auto_clean_logfiles(logfilepath)
# [1.2.0] save impending run's valid logfile path for WATCH
self.current_run_logfile_path = logfilepath
# save Main paths on each run
self.save_persisted_settings_Main()
# if-confirmed op in this thread and process, pre-run
if postcon:
postcon()
# allow non-update actions to be killed by Back button?
# no: requires detecting others in-progress on restart
# self.update_action = (opname not in ('SHOW', 'DIFF'))
doservice = self.ids.runactionasservice.active
if RunningOnAndroid and doservice:
# Android only: foreground service + process (default)
# broadcast receiver for script-exit signal
logfile.close()
self.launch_script_service(
opname, script, cmdargs, logfilepath, stdin, postscr)
elif RunningOnAndroid or not RunningOnAndroid: # Shakespearean, that
# Android and PCs: posix thread in the app's process (original)
# run if paused, script-exit via thread wrapper
self.launch_script_thread(
opname, script, cmdargs, logfilepath, logfile, stdin, postscr)
else:
# PCs only: child process via subprocess.Popen? (TBD)
# sentinel-file polling for script-exit signal
self.launch_script_process(
opname, script, cmdargs, logfilepath, logfile, stdin, postscr)
def confirm_askuser():
"""
----------------------------------------------------------
[confirm_action step #2]
[1.2.0] On dismissal (or no-show) of notifications info.
Uses multiple variables in the enclosing scope.
Appears under system perm dialog on first service action,
but this seem better than both convolutinf first open,
and pre-andr13 api perm dialog while first action runs.
Exception: NAME will run while perm dialog is open; rare.
----------------------------------------------------------
"""
if message == None: # preconfirmed, propagate content
confirm_confirmed(popupcontent) # uses args in enclosing scope
else:
confirmer = ConfirmDialog(message=message,
onyes=confirm_confirmed,
onno=self.dismiss_popup)
self._popups.push(Popup(title='Confirm Action Run',
content=confirmer,
auto_dismiss=False, # ignore taps outside
size_hint=(0.9, 0.8))) # wide for phones
self._popups.open()
#----------------------------------------------------------
# [confirm_action, step #1]
#
# On all Main-tab actions, post checks and preps.
#----------------------------------------------------------
# [1.2.0] verify or ask first, on android 13+ for services
self.check_notification_permission(nextop=confirm_askuser)
def check_notification_permission(self, nextop):
"""
--------------------------------------------------------------------
[1,2,0] Android 13 (API level 33) mods notifications permission:
it must be called out in the manifest (via buildozer.spec), and
apps themselves are now responsible for asking the user to enable
the permission with code below (it used to be fully automatic, and
asked once at first foreground-service start). Because Play
requires API level <= 1 year old for app changes, supporting this
was required to release a new version with fixes. Yes, grrr.
This is _almost_ automated by p4a's permissions lib, but it
should really ask just once, and bail if the user said no (or
yes) in the past; unlike storage permission, this is not a core
req, and applies iff an action is run as a foreground service.
Also: the Android request call simply posts an old-style runtime
permission dialog, instead of opening Settings' notification
toggles page. This is far less custom than All Files Access.
Ref: https://stackoverflow.com/questions/72310162/
how-do-i-request-push-notification-permissions-for-android-13
----
Coding/timing dilemma:
1) If this is posted on first service run, the infobox will be
overlaid by the end-run summary for a fast action, and the system
dialog doesn't appear till the infobox is dismissed (a four-level
interaction: confirm, summary, infobox, dialog; yikes). The
perm also won't apply to the first action run (as before).
2) If this is posted at app first open, it won't be lost under
first action run, but there's no reason to ask unless and until
an action is run as a service (users may select threads instead,
especially if future Androids deprecate fgservice for bkp/sync).
#1 is basically how this auto worked before Andr13, except that
the infobox keeps the system dialog from appearing while covered.
It was confusing before, is probably more confusing with the new
infobox, and means no notification for first action.
Should either:
A- Ask at first open always (like/after a-f-a)
B- Drop the infobox to emulate pre-13 behavior
C- Make infobox modal with another continuation, like confirm dialog
#A is coherent, but overkill if just threads, and convolutes startup UI
#B is still confusing when a first action finishes before dialog closed
#C is best but complicates code: requires another continuation in confirm
Either way, should ask on first _service_ run (not threads) and only
if haven't already asked (granted or not). #A doesn't have to check
if already asked (and may be able to skip checking if granted with
the bizarre+iffy android api method, though unclear for upgrades),
but it requests the permission pointlessly if user will run threads.
Went with #C. Made infobox here a first modal step of confirm, so
that system perm dialog overlays action confirm popup, and must
be handled+dismissed before action starts to run. This makes
the permission active for the first action too, and defers action
run until perm is handled.
Caveat: NAME still runs before perm dialog is handled, but it's
better than pre-13 (the infobox here gets focus first); the odds
of NAME running first are very slim; and this is just a one-time
event, after all (users probably won't care).
--------------------------------------------------------------------
"""
# not for pcs or threads
doservice = self.ids.runactionasservice.active
if not (RunningOnAndroid and doservice):
trace('np=> not relevant platform or mode')
nextop(); return
# not for androids 8..12
if self.android_version() < 13: # a.k.a. API level 33
trace('np=> not relevant android')
nextop(); return # work the way it used
# monkey patch: kivy android.permissions grew this later
if not hasattr(Permission, 'POST_NOTIFICATIONS'):
Permission.POST_NOTIFICATIONS = "android.permission.POST_NOTIFICATIONS"
# already got one?
if check_permission(Permission.POST_NOTIFICATIONS):
trace('np=> Has post_notification permission')
nextop(); return
# welcome to trial-and-error theater
#actcompat = autoclass('android.ActivityCompat')
#Manifest = autoclass('android.Manifest')
activity = self.get_android_activity()
permname = Permission.POST_NOTIFICATIONS
# don't ask again if user previously denied; yes, True=denied (!)
if activity.shouldShowRequestPermissionRationale(permname):
trace('np=> User denied post_notification')
nextop(); return
# explain the system dialog first, then open it
msg = ('This app uses Android\'s notifications permission for services.'
'\n\n'
'In the following dialog, please grant this app this '
'permission. This will allow this app to post a notification '
'that lets you know when a foreground service is running '
'to process a Main-tab action.'
'\n\n'
'The service will work whether you grant this permission or not, '
'and you can change this permission option in your phone\'s '
'Settings panels at any time.')
self.info_message(msg,
nextop=lambda:
(request_permission(Permission.POST_NOTIFICATIONS), nextop()) )
def verify_paths(self, frompath, topath):
"""
--------------------------------------------------------------------
Paths are editable and drives can be removed: verify that both
name existing directories, and are accessible to the app.
Either can be None: NAME uses only FROM, UNDO uses just TO.
Storage permission is checked both here by Main-tab actions, and
prior to opening popup by folder chooser. Folder access is checked
both here (post permission) and on folder-chooser storage buttons.
Subtle: permission is now checked here instead of post confirmation,
else the listdir here may fail due to no permission without reasking
for permission, and the restart prompt must follow granting perms.
Though should happen only for MANUALLY typed paths: the chooser asks
for perms before opening its popup too...
UPDATE [1.1.0] avoid lags/hangs for inaccessible paths prior to Main
action runs. This may arise for network or other drives unmounted
since a chooser pick, as well as arbitrary paths entered manually.
UPDATE [1.1.0]+ tweak the message to clarify that an app restart is
not needed on macOS (probably), and qualify the opening as "on Android"
because it won't apply anywhere else. See reset_main_picker_path().
----
UPDATE [1.2.0] On Android only, avoid exposing the app's source code
for a manually entered path in FROM. Do likewise to avoid harming
app source code for manual entries in TO. Else, for example, a
SYNC/COPY with a manual FROM='..' copies the app's code in full,
and the same actions with TO='..' destroys the app.
Formally, both absolute and relative paths may refer to the app's code
when input manually (i.e., not by chooser pick). Because the app first
chdir()s into the ./mergeall subfolder when actions are run in both
thread and service modes on Android, the following relative paths apply:
-=works, *=prevented by simple abspath==abspath, x=inaccessible
- name is a subfolder in mergeall/ (okay to expose)
- '.' is mergeall/ (okay to expose, THOUGH a mod version)
* '..' is the app's full code (bad)
* '../..' is a folder with app/ (the code: bad), runcounter, and settings
x '../../..' is inaccessible, and issues an error popup
And the following absolute paths apply both post- and pre-chdir()
(checked pre-chdir() here, when '.' is still the app's code folder):
- (mergeall) /data/data/com.quixotely.usbsync/files/app/mergeall
* (. = cwd) /data/data/com.quixotely.usbsync/files/app [works]
* (..) /data/data/com.quixotely.usbsync/files [works]
- (../..) /data/data/com.quixotely.usbsync [works! =rel ../../..]
x (../../..) /data/data [inaccessible]
BUT: all the abs paths also work with /data/user/0 instead of /data/data,
and this form is not detected by a simple comparison to abspath('.'),
and Python's os.path.realpath() does NOT map either form to the other,
and os.path.relpath() does NOT return the same string for each form.
THOUGH: os.path.samefile(form1, form2) is True on Unix (incl Android)
per inode testing (via stat()) that is immune to all path-form diffs.
HENCE => must either:
(1) Use [hangable] samefile(input-path, *code-folders) instead of ip == cf
(2) Punt on '..'s and fail if split(abspath))[1] == split(appprivate)[1]
GO WITH #1, because #2 is hampered by the fact that different filesystem
roots may lead to the same place independently (split[1] isn't enough).
#1 requires try-any_nohang() to avoid unlikely hangs for the stat call,
as well as except catches for inaccessible roots above code (may vary);
it may also burn time on paths that don't make sense, but phones are fast.
Caveat: try_any_nohhang() will return False if exc, timeout, or false.
The exc and false cases are what we want, but a timeout may occur for
timeout when the path refers to a private folder (a false negative).
Punt; this check seems hardly worth the effort already put into it...
See also storage_path_app_private() for more on the folder structure;
per logcat: (PPUS) app-priv: /data/user/0/com.quixotely.usbsync/files.
Defunct note: this also assumes that os.path.abspath() won't hang:
it runs only os.getcwd() plus purely syntactic path transforms.
Alt: disallow _all_ relative paths, but this seems too extreme,
and doesn't handle absolute-path refs to the code folder or parents.
This is only an issue on Android, because Buildozer apps don't do
anything about hiding source automatically: .py files are in the
run folder, which is app private, but completely open to the app
itself (and thus exposed to users via unchecked pathname entries!).
PC apps are built with PyInstaller, which normally bundles just
bytecode in formatted exes or files, and ".." is just the app
unzip folder (Windows+Linux) or application folder (macOS)...
with the following glaring caveat.
Also in [1.2.0], avoided exposing the app's source-code files in the
Windows PyInstaler exe, per this from pc-build/(windows-spec-file):
#
# [ML] [1.2.0] Tighten up source-code visibility with excludes=[...]
# in added Tree(), so raw .py source-code files are not exposed in
# PyInstaller _MEI* Temp unzip folders (an undocumented side-effect
# of a Tree() spec-file insert required+prescribed by Kivy!).
#
# This applies only to the GUI's code (Mergeall's code used for
# file ops is fully open source); and only to Windows (Linux and
# macOS exe/dir don't add a Tree('.') and so don't include .pys,
# and Andoid's Buildozer has different source-code policies).
#
# PyInstaller used to have a '--key' encrypted-bytecode option,
# but dropped it in 6.0 because it was too weak: keys were shipped
# in PI exes, and Github has tools to extract+decrypt+decompyle.
# Encryption or not, PyInstaller packages code, but hides it from
# casual eyes only; this might encourage open source and free.
# (Ref: https://github.com/pyinstaller/pyinstaller/pull/6999).
#
# Also closed up a code leak for manually input paths on Android.
# In the end, this all may be a moot point: it's probaby easier
# to make a workalike app than steal+mod+rebuild this one's vast
# Python code, and Play payola and audience naiveté remain hurdles.
#
UPDATE [1.2.0]: also changed to catch invalid relative paths in the
GUI, instead of letting them reach and fail in mergeall.py. This
can happen for paths like '../app' which are valid in the cwd here,
but not after chdir to mergeall/ to run the action: must check in
the context if the mergeall/ forlder for relative paths. Such
paths are not kicked out as app-private, because they are invalid
in mergeall/, and hence skipped by app-private tests. Current
coding runs os.chdir() > once, but this is too fast to matter.
--------------------------------------------------------------------
"""
# confirm permission or ask again before listdir, force retry if ask
if not self.get_storage_permission():
return False
# verify access next
checks = [('FROM', frompath), ('TO', topath)]
# check from and to
for (kind, path) in checks:
if path == None:
continue # skip if unused by action
# [1.2.0] checks vary
is_abs_path = os.path.isabs(path) # check cwd does not matter
is_rel_path = not is_abs_path # check relative to mergeall/
# check isdir
# os.path.isdir(path) can also hang on Windows [1.1.0]
if is_rel_path:
os.chdir('mergeall') # [1.2.0]
isdir = self.try_isdir_nohang(path)
if is_rel_path:
os.chdir('..') # [1.2.0]
if not isdir:
self.info_message('%s is not a valid folder path.' % kind,
usetoast=True) # [1.1.0] expanded
return False # cancel action
# check app-private
# [1.2.0] disallow app's code folder as FROM or TO
if RunningOnAndroid:
exposed = False
samefile = os.path.samefile
if is_abs_path:
# chdir won't matter
for p in ['mergeall', '.', '..', '../..']:
if self.try_any_nohang(samefile, path, p, onfail=False):
exposed = True
break
else:
# chdir will matter
os.chdir('mergeall')
for p in ['.', '..', '../..', '../../..']:
if self.try_any_nohang(samefile, path, p, onfail=False):
exposed = True
break
os.chdir('..')
if exposed:
self.info_message('%s cannot be app private.' % kind,
usetoast=True)
return False # cancel action
# check listdir
"""
try:
os.listdir(path)
except:
"""
# same, but use timeouts to avoids lags for inaccessible drives [1.1.0]
if is_rel_path:
os.chdir('mergeall') # [1.2.0]
listed = self.try_listdir_nohang(path)
if is_rel_path:
os.chdir('..') # [1.2.0]
if not listed:
# hung (inaccessible), or immediate fail (unmounted?)
# differs from chooser: about folder, not drives list
noaccessmsg = (
'%s is not accessible.'
'\n\n'
'If you just granted storage permission on Android, ' # [1.1.0]+
'please try restarting this app now.'
'\n\n'
'Otherwise, retry after ensuring that your '
'device has finished mounting your drive in full.'
)
if RunningOnMacOS:
noaccessmsg += (
'\n\n'
'On macOS, you may be able to simply rerun the last '
'request if you just handled the permission popup.' # [1.1.0]+
)
if not RunningOnAndroid: # [1.1.0] added
noaccessmsg += (
'\n\n'
'On PCs, also check folder permissions and locks, '
'and network-drive status.'
)
self.info_message(noaccessmsg % kind, usetoast=False)
return False # cancel action
return True # all passed: proceed with action, if user confirms
def check_older_android_removeable_updates(self, updatepath):
"""
--------------------------------------------------------------------
Because Android's SAF sucks in full.
(And the Android 10- audience is shrinking rapidly.)
((And most users will sync only PC-to-phone anyhow.))
Prohibit non-appfolder removable-drive updates in
Android 10- for SYNC, UNDO, COPY, and NAME, which
would all fail with error messages in the logfile.
Called on action picks: this can't be addressed in
the chooser, because it's okay to read non-app
folders on removables on all Androids, the action
selection is not known until the popup is closed,
and users can manually type/paste paths into Main.
The chooser also doesn't preset paths to app folder
paths instead of drive roots for removables on 10-,
because it's valid and normal to choose other folders
on these drives for reads (e.g., PC=>phone syncs).
--------------------------------------------------------------------
"""
message = ('CAUTION: you have selected a removable-drive folder that '
'does not support updates on your version of Android, '
'and a Main-tab action that may modify the folder.'
'\n\n'
'Per this app\'s docs, Androids 11 and later allow reads and '
'updates anywhere on removable drives. Androids 10 an earlier '
'allow reads in any removable-drive folder, but writes are '
'supported only in the drive\'s Android/data/com.quixotely.%s.'
'\n\n'
'Because any updates outside this folder will fail with a '
'logfile error message, this action must be cancelled. '
'Please relocate your content on the drive as needed.')
if RunningOnAndroid and self.android_version() <= 10: # through 10/Q, api 29
appsuffix = 'usbsync' if not TRIAL_VERSION else 'usbsynctrial'
appfolder = '/Android/data/com.quixotely.' + appsuffix
removables = self.storage_names_and_paths_removables()
for (rname, rpath) in removables:
if updatepath.startswith(rpath):
# removable
if updatepath.startswith(rpath + appfolder):
return True # proceed: in app folder
else:
self.info_message(message % appsuffix, usetoast=False)
return False # cancel: outside app folder
return True # proceed: does not apply or not removable
def label_path_string(self, path):
"""
--------------------------------------------------------------------
[1.1.0] Format path with its volume label. Now shown in action
confimation popups, both to remind users of paths, and associate
logical label with physical path.
This was a compromise: it's important to remind users of the FROM
and TO subjects in the popup, and useful to show the label with
the path. But displaying the label in Main tab FROM and TO fields
is problematic, because these are user-editable fields, and there
are few delimeters that would work for all platforms in path strings.
E.g., "<label> path" seems ok, but requires special rules and docs
for manual edits, and "<> " are valid in Unix file and folder names.
Confirmation dialogs are not editable, and require no iffy parses.
Uses a most-specific storage label: may != chooser pick for ROOT.
Mimic Main chooser's path preselect logic: pick storage by fist
prefix match among all storages sorted by decreasing lenth - which
orders most-specific first: see do_main_tab()'s preselect for docs.
Here, must build all-storages list, because chooser buttons don't
exist. In fact, the chooser may not have been run or used at all
(the user can enter anything, manually), so cannot cache chooser's
latest results to avoid network lags on Windows.
A former coding that tried sorted removables, then app, then shared,
then root, may have guessed poorly on Windows (where ROOT=system drive
is removed from drives, but PC (home) might not be nested in ROOT).
The global sort handles all cases, including arb nested Linux mounts.
[1.1.0]+ dec23: drop enclosing name parens if present, because caller
always adds more; else Windows '(Removable)' shows as odd '((Removable))',
and likewise for '(Remote)' for network drives and WSL2 Linux mounts.
[1.1.0]+ (feb24) recoded to be more robust, based on the following ops:
>>> x = '(Removable)'
>>> x[1:-1] if x.startswith('(') and x.endswith(')') else x
'Removable'
>>> x = 'T7SSD'
>>> x[1:-1] if x.startswith('(') and x.endswith(')') else x
'T7SSD'
>>> x = '((Hmm))'
>>> x[1:-1] if x.startswith('(') and x.endswith(')') else x
'(Hmm)'
>>> x.strip('()')
'Hmm'
>>> x = ''
>>> x[1:-1] if x.startswith('(') and x.endswith(')') else x
''
>>> x[1:-1] if x[0] == '(' and x[-1] == ')' else x
IndexError: string index out of range
>>> x = '()'
>>> x[1:-1] if x.startswith('(') and x.endswith(')') else x
''
>>> x[1:-1] or x if x.startswith('(') and x.endswith(')') else x
'()'
Risk: thus will clash with another drive here (only) if a user labels
a drive 'Removable' (but so will 'Untitled' on macOS, and very unlikely),
and may drop outer parens coded by users (but why would they do that?).
--------------------------------------------------------------------
"""
# Get all storage types
shared = self.storage_path_shared()
appspec = self.storage_path_app_specific()
drives = self.storage_names_and_paths_removables() # before root for Windows
root = self.storage_path_root() # we're punting on Docs
# Build list of all names+paths to mimic choose's storage buttons
# start with removables, sans system drive on windows
# not in APP|PHONE on Android nor PC on Windows|macOS, may be in PC on Linux
storagesbypathlen = drives
# app is moot for pc, nested in phone on android
if appspec != None:
storagesbypathlen.append(('APP', appspec))
# shared storage on android, home (account) on pcs, never None
storagesbypathlen.append((('PHONE' if RunningOnAndroid else 'PC'), shared))
# subsumes all on unix and android, system drive on windows, poss None on android
if root != None:
storagesbypathlen.append(('ROOT', root))
# sort storage types by decreasing path length = longest (most specific) first
storagesbypathlen.sort(key=lambda store: len(store[1]), reverse=True)
# use absolute path comparison: user may enter anything
abspath = os.path.abspath(path)
# Select storage type by matching path prefix, caller formats
for (stname, stpath) in storagesbypathlen:
if abspath.startswith(stpath):
if stname.startswith('(') and stname.endswith(')'): # [1.1.0]+ dec23+
stname = stname[1:-1] or stname # unless is '()'
return (stname, path)
# [1.2.0] for allowed android /data app-private: __pycache__ in ./mergeall/
return ('?', path) # punt: unexpected manual entry outside model?
"""DEFUNCT----
# removables: not nested in app or phone app on android, nor in in pc (home)
# on windows pcs, but might be mounted/nested in pc on linux - test first;
# sort by decr path len on linux: 'os' at /media/me, drives at /media/me/xxx;
# windows: root=system drive is removed from drives, so won't do pc=home here;
# see also preselect in chooser dialog, and origins in names-and-paths call:
if RunningOnLinux: # linux only, not android
drives.sort(key=lambda drive: len(drive[1]), reverse=True) # longest first
for (remname, rempath) in drives:
if path.startswith(rempath):
return display % (remname, path)
# app next: nested in phone=shared on android, moot on pcs
if appspec != None and path.startswith(appspec):
return display % ('APP', path)
# user account (home) on pcs, shared storage on android
if path.startswith(shared):
return display % ('PHONE' if RunningOnAndroid else 'PC', path)
# root last: subsumes all on unix and android, system drive on windows
if root != None and path.startswith(root):
return display % ('ROOT', path)
DEFUNCT----"""
def format_paths(self, frompath=None, topath=None):
"""
--------------------------------------------------------------------
[1.1.0] Action confirm-display helper.
Nit: this can't use unicode arrows in info-msg font.
Also nit: path wrapping on narrow phones can look a bit
subpar, but no other format seemed better, and manual
splits are right out. E.g., indenting name+path on new
line is ideal sans wrapping but looks jumbled wrapped;
and uppercase names seem lost in both name+path on new
line sans indent, and only name on FROM/TO line sans
parens or spaces with path unindented on new line.
Also tried "Path:" on new path line: busy, redundant.
The Format used calls out name well, and suffices on both
larger screens like PCs and foldables, and narrow phones.
--------------------------------------------------------------------
"""
display = ''
if frompath:
display += 'FROM: (%s)\n%s\n\n' % self.label_path_string(frompath)
if topath:
display += 'TO: (%s)\n%s\n\n' % self.label_path_string(topath)
return display
# SYNC-------------------------------------------------------------
def toggle_undo(self, saving_backups, topath):
"""
On user confirmation in SYNC confirm-run dialog.
Disable UNDO if confirm run sans backups, renable if bkps on.
Changing the TO field in the GUI renables UNDO button via code
in the .kv file: different dir makes UNDO's current state moot.
The .kv handler fires for both manual and chooser mods to TO.
[1.1.0] But not if self.script_running, else UNDO will come on
while an action is in progress, and user may start > 1 action!
Instead, on path-text edit, the .kv file reenables UNDO immediately
iff no action is running, and sets undo_verboten to False always
so UNDO will be enabled on script exit (and therafter).
"""
self.undo_verboten = not saving_backups # toggle in running app
try:
undoname = osjoin(topath, '__bkp__', UNDOABLE_FILE) # flag file for next run
if not saving_backups and osexists(undoname): # saved in in TO/__bkp__
os.remove(undoname)
elif saving_backups and not osexists(undoname):
bkpspath = osjoin(topath, '__bkp__')
if not osexists(bkpspath): # if never saved backups
os.mkdir(bkpspath)
undofile = open(undoname, 'w') # empty file: flag only
undofile.close()
except:
self.info_message('Cannot manage undoable file.', usetoast=True)
def do_sync(self, frompath, topath):
trace('do_sync')
if not self.verify_paths(frompath, topath):
return
if not self.check_older_android_removeable_updates(topath):
return
message = ('About to run a SYNC action...'
'\n\n'
'%s'
'This will change your TO content folder to be the same '
'as FROM, by updating TO only for changes in FROM.'
'\n\n'
'%s'
'\n\n'
'This may change TO, but not FROM.'
'\n\n\n'
'Continue with run?')
optionals = []
if self.ids['backupscheckbox'].active: optionals.append('-backup')
if self.ids['skipcruftscheckbox'].active: optionals.append('-skipcruft')
# warn about runs sans backups
saving_backups = self.ids['backupscheckbox'].active
warnbkps = ('Backups for changes made by this SYNC will be saved to your '
'TO/__bkp__ and can be rolled back by UNDO.'
if saving_backups else
'CAUTION: this sync cannot be rolled back with UNDO '
'because backups are turned off in the Config tab.')
# if confirm: setup undo, globals for threads, maxbackups for thread|process
self.confirm_action( # show labels+paths [1.1.0]
message = message % (self.format_paths(frompath, topath), warnbkps),
opname = 'SYNC',
script = osjoin('mergeall', 'mergeall.py'),
cmdargs = [frompath, topath, '-auto', '-quiet'] + optionals,
postcon = lambda: (self.toggle_undo(saving_backups, topath),
self.init_mergeall_globals(),
self.update_mergeall_configs_maxbackups()
))
# SHOW-------------------------------------------------------------
def do_show(self, frompath, topath):
trace('do_show')
if not self.verify_paths(frompath, topath):
return
message = ('About to run a SHOW action...'
'\n\n'
'%s'
'This will report FROM/TO differences quickly, '
'by using timestamps, sizes, and structure, '
'instead of full content.'
'\n\n'
'This will not change anything in FROM or TO.'
'\n\n\n'
'Continue with run?')
optionals = []
if self.ids['skipcruftscheckbox'].active: optionals.append('-skipcruft')
self.confirm_action(
message = message % self.format_paths(frompath, topath),
opname = 'SHOW',
script = osjoin('mergeall', 'mergeall.py'),
cmdargs = [frompath, topath, '-report', '-quiet'] + optionals)
# UNDO-------------------------------------------------------------
def discard_latest_backup_folder(self, latestbkp):
"""
Postop call in thread: rename latest __bkp__ so N UNDOs go back in time.
Each later UNDO will find and rollback the next-most-recent in __bkp_.
A special case, but this would otherwise require a complex Mergeall mod.
The rename tacks a prefix onto the latest __bkp__ subfolder's name.
This removes this backup from future consideration here, but also keeps
it in the set of subfolders auto-cleaned in mergeall's backup.py.
Could be run on EOF in UI thread instead, by passing to timer callback.
UPDATE: confirm message now has both postcon (in gui thread, before
script) and postscr (in gui|script thread, after script). Run file ops
in script thread if possible (can't for service) to avoid pausing gui.
Why not nested func: also called by app restart if service still running.
This must be called after script exit, and won't survive app kill+restart.
UPDATE: no longer used for app restart - see check_for_running_service().
"""
try:
os.rename(latestbkp, (latestbkp + '--UNDONE'))
except:
self.info_message('Cannot rename latest backup.', usetoast=True)
def do_undo(self, topath):
trace('do_undo')
if not self.verify_paths(None, topath):
return
if not self.check_older_android_removeable_updates(topath):
return
message = ('About to run an UNDO action...'
'\n\n'
'%s'
'This will roll back your TO content folder '
'to be what it was before its latest SYNC.'
'\n\n'
'UNDO assumes SYNCs were run with backups enabled in '
'the app, and is generally prohibited if not.'
'\n\n'
'Successive UNDOs roll back successively older syncs, '
'but only if prior syncs saved backups.'
'\n\n'
'You can undo an UNDO by simply rerunning the SYNC '
'that was rolled back.'
'\n\n'
'This may change TO, but not FROM (FROM is ignored).'
'\n\n\n'
'Continue with run?')
# find latest backup folder
tobkp = osjoin(topath, '__bkp__') # backups in TO
bkppatt = 'date??????-time??????' # ignore '--UNDONE's here
allbkps = glob.glob(osjoin(tobkp, bkppatt)) # mergeall uses 'date*-time*'
# bail if no backups - first, else undofile test fires
if not allbkps:
self.undo_verboten = True
self.ids.undobutton.disabled = True
self.info_message('No usable backups in TO.', usetoast=True)
return
# bail if TO's latest sync did not save backups (and save flag file);
# for first UNDO in session: never get here after a SYNC sans bkps;
# nothing can be done for older bkps: user may mod folder arb;
# changing the TO field in the GUI renables UNDO button via .kv;
undofile = osjoin(topath, '__bkp__', UNDOABLE_FILE)
if not osexists(undofile):
self.undo_verboten = True
self.ids.undobutton.disabled = True
self.info_message('No backups for latest sync in TO.', usetoast=True)
return
# bail if bkp not a dir (user can mod folder)
latestbkp = sorted(allbkps)[-1] # last=newest
if not os.path.isdir(latestbkp):
self.undo_verboten = True
self.ids.undobutton.disabled = True
self.info_message('No usable latest backup in TO.', usetoast=True)
return
optionals = []
if self.ids['skipcruftscheckbox'].active: optionals.append('-skipcruft')
# no -backup: breaks next UNDO, and SYNC == UNDO of an UNDO
self.confirm_action(
message = message % self.format_paths(topath=topath),
opname = 'UNDO',
script = osjoin('mergeall', 'mergeall.py'),
cmdargs = [latestbkp, topath, '-restore', '-auto', '-quiet'] + optionals,
postcon = self.init_mergeall_globals,
postscr = lambda: self.discard_latest_backup_folder(latestbkp))
# COPY-------------------------------------------------------------
def do_copy(self, frompath, topath):
trace('do_copy')
if not self.verify_paths(frompath, topath):
return
if not self.check_older_android_removeable_updates(topath):
return
# make cpall always create from's root as a subfolder in to;
# cpall allows it to be made manually and already exist, but
# this can't be distinguished from overwrites that will fail,
# and to must aleady exist to pass verify_paths() checks here;
# also must avoid same-folder copies, else copies fall into
# infinite loops and die (copying self into self); cpall.py
# does this already, but we're modding the path early here;
# abspath()==abspath() nor os.path.realpath() may suffice;
if hasattr(os.path, 'samefile'):
samepath = os.path.samefile(frompath, topath) # inode: Unix+Android
else:
samepath = os.path.abspath(frompath) == os.path.abspath(topath)
if samepath:
self.info_message('Cannot copy a folder into itself.', usetoast=True)
return
# [1.1.0] actually, need to test for path prefix match of FROM in
# TO, not just samefile, else recursive (until path limit reached):
# copying A/B parent into A/B, A/B/C, A/B/C/D, or A/B/* will loop!
# samefile test above is mostly subsumed, but it may check inodes;
if os.path.abspath(topath).startswith(os.path.abspath(frompath)):
self.info_message('Cannot copy a folder into its own subfolder.',
usetoast=True)
return
origtopath = topath # save for confirm display
contentroot = os.path.split(frompath)[-1] # same as basename
topath = osjoin(topath, contentroot) # okay to do pre-confirm
# and don't allow if appended TO exists: overwrites fail on some
# platforms, and this also catches the case of copying A/B into A;
# this test also catches from and appended to being the same path
if osexists(topath):
self.info_message('Cannot copy to an existing folder.', usetoast=True)
return
message = ('About to run a COPY action...'
'\n\n'
'%s'
'This will copy your FROM content folder into TO completely.'
'\n\n'
'FROM\'s root folder, %s, will be created as a new subfolder '
'inside TO automatically.'
'\n\n'
'Because this copies in full, it may run slowly.'
'\n\n'
'This will change TO, but not FROM.'
'\n\n\n'
'Continue with run?')
optionals = []
if self.ids['skipcruftscheckbox'].active: optionals.append('-skipcruft')
# TBD: -v dirs or -vv dirs+files; no -u: flush=no-op
self.confirm_action(
message = message % # can't %= in kw args
(self.format_paths(frompath, origtopath), contentroot),
opname = 'COPY',
script = osjoin('mergeall', 'cpall.py'),
cmdargs = [frompath, topath, '-vv'] + optionals)
# DIFF-------------------------------------------------------------
def do_diff(self, frompath, topath):
trace('do_diff')
if not self.verify_paths(frompath, topath):
return
message = ('About to run a DIFF action...'
'\n\n'
'%s'
'This will compare your FROM content folder to TO deeply, '
'using byte-for-byte comparisons of each common item.'
'\n\n'
'Because this compares in full, it may run slowly.'
'\n\n'
'This will not change anything in FROM or TO.'
'\n\n\n'
'Continue with run?')
optionals = []
if self.ids['skipcruftscheckbox'].active: optionals.append('-skipcruft')
# no -u: flush=no-op here
self.confirm_action(
message = message % self.format_paths(frompath, topath),
opname = 'DIFF',
script = osjoin('mergeall', 'diffall.py'),
cmdargs = [frompath, topath, '-quiet'] + optionals)
# NAME-------------------------------------------------------------
def do_name(self, frompath):
"""
Original conception was a configurable prestep for SYNC and COPY,
but it's difficult to wedge two scripts into the thread launcher
code, and fixer's output must be called out from the others'.
TBD: fixer's report-only mode would be nice, but it's console.
As is, running NAME seems a bit scary (even for its creator!).
UPDATE: this action now uses a custom confimation dialog that lets
users run the target script in report or update mode, or cancel.
This convolutes the confim call (it now allows preconfirms, else
confirmed() must be unnested), but there is no other good option:
a Config-tab report|update setting doesn't make sense, because
this may be chosen on every NAME run (other Config action settings
rarely, if ever, change), and the GUI's too cramped in any event.
"""
trace('do_name')
if not self.verify_paths(frompath, None):
return
if not self.check_older_android_removeable_updates(frompath):
return
message = ('About to run a NAME action...'
'\n\n'
'%s'
'This action fixes any nonportable filenames in FROM '
'to ensure that your content arrives intact in TO.'
'\n\n'
'In update mode, all nonportable characters in FROM\'s '
'filenames will be replaced with an underscore, so they '
'can be transferred to other devices.'
'\n\n'
'In report mode, nonportable filenames in FROM are '
'displayed in the logfile, but not changed. An initial '
'report-mode run is advised, as this action has no undo.'
'\n\n'
'Update mode may change FROM, but not TO (TO is ignored).'
'\n\n\n'
'Please select a run mode to continue.')
def confirmed_report(popupcontent):
self.confirm_action(
message = None, # preconfirmed
opname = 'NAME',
script = osjoin('mergeall', 'fix-nonportable-filenames.py'),
cmdargs = [frompath, '-report'],
stdin = 'y\n',
popupcontent=popupcontent)
def confirmed_update(popupcontent):
self.confirm_action(
message = None, # preconfirmed
opname = 'NAME',
script = osjoin('mergeall', 'fix-nonportable-filenames.py'),
cmdargs = [frompath],
stdin = 'y\n',
popupcontent=popupcontent)
confirmer = ConfirmNameDialog(message=message % self.format_paths(frompath),
onrunreport=confirmed_report,
onrunupdate=confirmed_update,
oncancel=self.dismiss_popup)
self._popups.push(Popup(title='Choose NAME Mode',
content=confirmer,
auto_dismiss=False, # ignore taps outside
size_hint=(0.9, 0.8))) # wide for phones
self._popups.open()
#=======================================================================
# UTILITIES
#=======================================================================
def get_android_activity(self):
"""
Common preamble for pyjnius java-api code: the main activity (screen)
In build code:
self.activity_class_name = u'org.kivy.android.PythonActivity'
'ACTIVITY_CLASS_NAME': self.ctx.activity_class_name
"""
pythonact = autoclass(ACTIVITY_CLASS_NAME)
mActivity = pythonact.mActivity
activity = cast('android.app.Activity', mActivity)
return activity
def get_android_context(self, activity=None):
"""
Common preamble for pyjnius java-api code: the app's context (info)
Alts:
from android.config import ACTIVITY_CLASS_NAME
PythonActivity = autoclass(ACTIVITY_CLASS_NAME)
activity = PythonActivity.mActivity
currentActivity = cast('android.app.Activity', activity)
context = cast('android.content.ContextWrapper', currentActivity.getApplicationContext())
PythonActivity = autoclass('org.kivy.android.PythonActivity')
currentActivity = cast('android.app.Activity', PythonActivity.mActivity)
context = cast('android.content.Context', currentActivity.getApplicationContext())
"""
if activity == None:
activity = self.get_android_activity()
context = cast('android.content.Context', activity.getApplicationContext())
return context
@mainthread
def info_message(self, pmsg, usetoast=False, longtime=False, nextop=None):
"""
--------------------------------------------------------------------
Show an info message as an Android Toast, or in-GUI overlay Popup.
pmsg is the text to show+scroll in the popup: a Python string in toast.
Uses an Android Toast display if usetoast on Android: 2-lines max.
@mainthread is required to run graphics updates in main/GUI thread only.
@run_on_ui_thread is apparently just for Android things like Toast.
It's unclear why we're not in the main/GUI thread here like show_pick(), but...
Pass nextop to chain one next dialog after the info popup: any callable.
Caution: this doesn't support >1 overlapping modal dialogs as is (was).
UPDATE: yes it does - Popups are now stacked for overlaps, and also allow
for double taps without silently closing covered popups (see _PopupStack).
Also now uses a shorter popup if usteoast and not on Android - no reason
to take up most of the window for 1 cryptic line. This might also use
macOS slide-down popups, but Kivy doesn't support them? (kivyMD?; tbd).
UPDATE: the text is now scrollable in .kv, so go a bit smaller everywhere.
UPDATE [1.1.0]: delay the setting of the popup's text content until the
next frame, to avoid glitchy lag on slower phones. Else, the text displays
as a scrunched single column initially, before being drawn in full. This
was glaring on only one older/slower test phone (2018 Note 9), but may be
an issue on lower-end phones in general. The delayed set has no noticeable
impact on faster phones, and is the same solution used for larger texts in
the Help and About tabs: see on_start() and on_tab_switch().
UPDATE [1.1.0] scale the popup per message size. Especially for narrow
phones, 90% wide (link confirmation+chooser) where warranted will help.
--------------------------------------------------------------------
"""
if usetoast and RunningOnAndroid:
# the arguably silly Android popup
self.show_toast(pmsg, longtime)
else:
# toast is limited: make me an in-GUI modal popup
if not nextop:
canceller = self.dismiss_popup # close Info: modal, basically
else:
canceller = (lambda popupcontent:
(self.dismiss_popup(popupcontent), nextop()))
# shorter for one-liners
# and wider for phones if 'big' (most are tall or wide) [1.1.0]
# sizer = (0.8, 0.5) if usetoast else (0.8, 0.8)
tallmsg = pmsg.count('\n') > 3
widemsg = any(len(line) > 30 for line in pmsg.splitlines())
swidth = 0.9 if widemsg else 0.8
sheight = 0.8 if tallmsg else 0.5
sizer = (swidth, sheight)
content = InfoDialog(oncancel=canceller) # not message=pmsg [1.1.0]
self._popups.push(Popup(title='Info',
content=content,
auto_dismiss=False, # ignore taps outside
size_hint=sizer)) # sized per message, (x, y)
self._popups.open()
def set_info_text(dt):
content.message=pmsg # closure: enclosing scope
Clock.schedule_once(set_info_text) # delay to avoid lag [1.1.0]
@run_on_ui_thread
def show_toast(self, pmsg, longtime=False):
"""
Show an info message as an Android Toast.
Highly limited popup for short messages only.
These can overlap poorly if longtime=True.
Call this directly, or info_message(usetoast=True).
See also: from kivymd.toast import toast
"""
# 2-line max (and should be narrow)
pmsg = pmsg.strip().split('\n')
pmsg = '\n'.join([pmsg[0], pmsg[-1] if len(pmsg) > 1 else ''])
# Android API bit
context = self.get_android_context()
String = autoclass('java.lang.String')
jmsg = cast('java.lang.CharSequence', String(pmsg))
Toast = autoclass('android.widget.Toast')
duration = Toast.LENGTH_LONG if longtime else Toast.LENGTH_SHORT
Toast.makeText(context, jmsg, duration).show()
def open_web_page(self, url):
"""
The Android Intent is automated by android package:
https://github.com/kivy/python-for-android/blob/develop/
pythonforandroid/recipes/android/src/android/_android.pyx
What webbrowser invokes:
Intent = autoclass('android.content.Intent')
Uri = autoclass('android.net.Uri')
intent = Intent()
intent.setAction(Intent.ACTION_VIEW)
intent.setData(Uri.parse(url))
activity = self.get_android_activity()
activity.startActivity(browserIntent)
Alt scheme from pydroid 3 guis (where it fails):
cmd = 'am start --user 0 -a android.intent.action.VIEW -d %s' % HELPURL
os.system(cmd)
"""
webbrowser.open(url) # yes, that's it (for PCs too)
def android_version(self):
"""
Get version of Android on device running the app.
Cached for speed, though likely relatively trivial.
This is android.os.Build.VERSION.RELEASE in Java;
SDK_INT may be botched if rooted and nonstandard (?).
Subtly, VERSION is a nested class, not a Build field:
developer.android.com/reference/android/os/Build
developer.android.com/reference/android/os/Build.VERSION
Note: Python's sys.sys.getandroidapilevel() is the _build_
time API, not _run_ time, and won't suffice for checking
the host device's version. E.g., Termux is always 24 today,
on Android 10 and 13 phones. This seems useless for most
apps' use cases, but host #s require pyjnius code below.
"""
if hasattr(self, 'cache_android_version'):
return self.cache_android_version
else:
VERSION = autoclass('android.os.Build$VERSION')
apiversion = VERSION.SDK_INT # api/sdk version number, e.g., 31, 33
andversion = VERSION.RELEASE # android version number, e.g., 12, 13
trace('Host Android:', apiversion, andversion)
# str may be x, x.y, x.y.z, or ? - drop all but x or x.y
while andversion.count('.') > 1:
andversion = andversion[:andversion.rfind('.')]
numandversion = float(andversion)
self.cache_android_version = numandversion
return numandversion
#=======================================================================
# STORAGE PERMISSIONS: ALL FILES ACCESS (11+), LEGACY (~10), MACOS
#=======================================================================
def get_storage_permission(self):
"""
--------------------------------------------------------------------
Verify or request All Files Access if running on Android 11+, else
old-style legacy. Called on each startup to verify|request permission,
and again by Main filechoosers to verify permission or request it again,
as well as confirmed Main actions in case the user types paths manually.
Returns True if permission already held, else False if the permission
request Setting screen or dialog was opened to ask the user to grant.
If False, the reask cancels the Main tab's filechooser or action; this
forces users to retry both, and ensures that choosers and actions will
never open sans permission. Return value is ignored at startup.
Manually entered paths are verified for being a directory and accessible
first, so no invalid paths should ever reach Main actions; but ask to
allow permission grants anyhow in the unlikely event that the user types
a manual path that's accessible sans permissions (probably not, but...).
This is not called by logfile operations: if the app cannot create or
access the logfile folder, log ops will fail with toasts, but don't
ask for permissions until a Main chooser or action, or an app rerun.
The return value is ignored when invoked on startup from the App
class via self.root, which refers to the .root class created by the
kv file's code automatically. This might be staticmethod to skip
root, though self is used for the initial info popups.
Storage permission details:
- This app targets Android 12/S (api 31), because this is the
base requirement of the Play store today (not for sideloading).
- This app's minimum support is Android 8/Oreo (api 26), because
file modtimes didn't work before that.
- Permissions use All Files Access on Android 11/R (api 30) and
later, but legacy permissions from Android 8/oreo..10/Q (api 29).
- Permission vary slightly: Android 11+ can access Downloads/ sans
user grants, but pre-11 cannot (this impacts logfile operations).
Permissions requires both old and new permissions in the manifest
(per buildozer.spec), and run-time checks and requests here. Both
permissions require user interaction; A-F-A is a prompted Settings
page, old-style is a system dialog, and either may denied by users.
This dual-permission support assumes that the USB list from
storage_names_and_paths_removables() works before 11 too (it
does, but equires different methdod before and after 11).
There is no All Files Access permission or Setting before 11,
and the 11+ /mnt/media_rw/uuid may not work before 11 either.
All Files Access restores POSIX and file-path access for shared
and USB storage, and is roughly the same as per-11 legacy storage
permission (more info ahead). What a convoluted mess, eh?
NOTE: permmission results cannot be cached for speed, because
the user can revoke them at anytime in Settings - even while app
is running. Must check anew before every op that needs them.
UPDATE: macOS may require similar permission settings too,
but this app prompts the user just once, because, unlike
Android, cannot verify the setting accurately on macOS.
LATER: macOS Full Disk Access now seems pointless for user
folders; the o/s asks users for specific-folder permission.
====
UPDATE: Android USB storage permissions' effect varies by version:
- In Android 11+, All files access grants read+write access to shared
stroage, as well as the entire USB drive (though mergeall.cpall had
to be patched here to ignore a spurious chmod exception in Pythn's
shutil.coptstat for exFAT and FAT32 drives on these Andoids only).
- In Android 10-, READ+WRITE (or WRITE only) grants access to shared
storage, as well as read access to the entire USB drive, but *NOT*
write access to the full USB drive. The app gets USB write access
only to its own Android/data/appname folder on USB drives, and cannot
write to other USB folders except with Storage Access Framework (SAF),
a horribly slow, convoluted, and proprietary Java/Android library
that requires content URIs instead of file-path and POSIX access.
WRITE probably doesn't enable anything on 10- except shared storage
(Mergeall's GUI can write in pydroid3's USB app folder sans perms),
though it's still required by this app for that. Run1 message updated.
Since SAF is a nonstarter here, support full bidirectional syncs on
Android 11+, but liited support on Anddroids 10-: both to-phone syncs
from arbitrary folders from-phone syncs in app USB folders only.
--------------------------------------------------------------------
"""
if not RunningOnAndroid: # moot on PCs (and don't re-ask on macOS)
return True
else:
if self.android_version() >= 11:
return self.get_all_files_access_permission_11plus()
else:
return self.get_legacy_storage_permission_pre11()
def get_legacy_storage_permission_pre11(self):
"""
On phones running Android 8..10, ask for the old-style
storage permission, if not already held.
"""
# old-style pyjnius code is automated in android.permissions here
if check_permission(Permission.WRITE_EXTERNAL_STORAGE):
# already got one...
trace('sp=> Has legacy-storage permission')
return True
else:
# explain the system dialog first, then open it
# drop "both your USB drive, as well as" per above
msg = ('This app requires Android\'s general storage permission.'
'\n\n'
'In the following dialog, please grant this app this '
'storage permission. This will allow this app to '
'access your content in shared storage.'
'\n\n'
'Whether granted or not, your content will never leave your devices '
'in this app. See PRIVACY POLICY in the About tab for more info.'
'\n\n'
'You can change this permission option in your phone\'s Settings '
'panels at any time.'
'\n\n'
'Note: you may have to restart this app just once to activate this '
'permission; you\'ll be prompted to do so if required.')
self.info_message(msg,
nextop=lambda: request_permission(Permission.WRITE_EXTERNAL_STORAGE))
# retry choosers+runs - had to reask user, result unknown sans callback
return False
def get_all_files_access_permission_11plus(self):
"""
--------------------------------------------------------------------
Check if this app has been granted All Files Access (a.k.a.
MANAGE_EXTERNAL_STORAGE) in Settings, and direct user to the
settings screen if not. Run me before folder choosers, and on
each app startup - paths can be typed manually sans choosers,
and the permission may be disabled in Settings at any time.
All Files Access gives POSIX and file-path access to shared
storage, as well as the root of the USB drive. This restores
the sort of access Android allowed prior to Android 11, though
this permission requires both Android 11's API or later, as well
as approval for the Play store (but sync apps should get it, and
sideloading doesn't need it). File-API access is also slowed by
wrappers in 11+, but the hit is unavoidable; generally acceptable
on phones; and soon to be >= halved by UFS-4 storage chips. More:
https://developer.android.com/training/data-storage/manage-all-files
NOTE that, unlike some permissions, this dialog is NOT required:
A-F-A is a persistent system setting that can be made by the user
anytime and need not be persisted by the app. Turning it on in
Settings suffices, but this sends the user there as a convenience.
----
UPDATE: runtime permissions may not be activated until the app is
RESTARTED, at least on some Androids/phones. The "fix" for this is
to either recode the app radically to not do anything until a Java
callback is run for the grant, or programmatically restart the app.
The former seems far too much work just for the glitchy Android,
and both of these are woefully undocumented by Google and rely on
Internet SO/GH gossip that proposes codings which don't work across
all Androids and can even vary per vendor.
That is: PUNT!! Instead, add this to the docs, note it in the
permission dialog, and ask for a restart in an info popup if/when
access fails. A single manual restart iff needed is not a whole lot
to ask users of a $2.99 app that's free on PCs. And WTH doesn't
Android auto-restart apps in this context?! Unless this was meant
as a joke, it's difficult to understand...
----
UPDATE: the restart-required issue can currently be recreated only on
a Fold4 running Android 13; test devices running Androids 8 through 10
do not exhibit the problem after uninstall+restart+reinstall (though
it's not impossible that some app info may be cached anyhow). Android
11 and 12 cannot be tested (except on a Pixel 4a, which seems to wholly
botch POSIX USB access even with permissions), but the current best
guess is that this was a fixed bug in 8..10, but ongoing in 11+. This
app also doesn't need to care; until Android grows more stable (and
less advertising based!), it is what it is, and we can expect no more.
LATER: Android 12 on Pixel4a does indeed require a post-perm restart too,
to access a fat32 drive; it's likely the norm on ALL Androids 11+. This
phone required 11=>12 upgrade to support USB drives, and then just fat32.
Maybe this is why people don't use Android for anything important; yet?
----
UPDATE [1.2.0] Kivy/p4a's android.permissions module has added the
MANAGE_EXTERNAL_STORAGE permission since this code was written (and
this app is still using the former Kivy). It's unknown if using this
module to check/request this permission works the same as the custom
code here, but it seems doubtful; AFA is a custom permission, though
Android 13's notification permission above uses this module in part
(probably; see check_notification_permission()).
--------------------------------------------------------------------
"""
"""
in Java (and for Android 11+ only):
if (!Environment.isExternalStorageManager()) {
Intent intent = new Intent(Settings.ACTION_MANAGE_APP_ALL_FILES_ACCESS_PERMISSION);
Uri uri = Uri.fromParts("package", getPackageName(), null);
intent.setData(uri);
startActivity(intent);
}
"""
Environment = autoclass('android.os.Environment')
if Environment.isExternalStorageManager():
trace('sp=> Has All-files-access permission')
return True
else:
# explain the setting screen first, then open Settings for app
msg = ('This app requires Android\'s All files access permission.'
'\n\n'
'In the next screen, please grant this app the All files access '
'permission by turning its toggle on. This allows this app to '
'access your content in both your USB drive and shared storage.'
'\n\n'
'Whether granted or not, your content will never leave your devices '
'in this app. See PRIVACY POLICY in the About tab for more info.'
'\n\n'
'The toggle screen is part of your phone\'s Settings, and can also '
'be accessed at any time by searching for "All files access" there.'
'\n\n'
'Note: you may have to restart this app just once to activate this '
'permission; you\'ll be prompted to do so if required.'
#
# [1.2.0] not so much anymore (a temp state issue on 1 phone?)...
# '\n\n'
# 'Also note: for reasons unknown, the toggle may take a dozen '
# 'seconds to change on some phones; please wait for the change.'
)
self.info_message(msg,
nextop=self.open_afa_settings)
# retry choosers+runs - had to reask user, result unknown sans callback
return False
def open_afa_settings(self):
"""
Part 2, get_all_files_access: open Settings for app
after info popup. This is a persistent-Setting screen.
This has never been seen to fail on 6 phones running
Androids 8~13, but user can still do manually if excs.
"""
try:
activity = self.get_android_activity()
context = self.get_android_context(activity)
Uri = autoclass('android.net.Uri')
Settings = autoclass('android.provider.Settings')
Intent = autoclass('android.content.Intent')
intent = Intent(Settings.ACTION_MANAGE_APP_ALL_FILES_ACCESS_PERMISSION)
intent.setData(Uri.fromParts('package', context.getPackageName(), None))
activity.startActivity(intent)
# and it's up to the user: no callback here to check - recheck on choosers
except:
excmsg = 'Cannot auto-open All files access: open manually in Settings.'
self.info_message(excmsg, usetoast=False)
def macos_ask_for_full_disk_access(self):
"""
--------------------------------------------------------------------
On fist app run on macOS, ask the user to enable Full Disk Access
in Settings, and route them there automatically. This is similar
to All files access on Android 11+, but, unlike Android, there's no
way to check if the permission is granted (short of access failures
that may have multiple causes), so ask only on the first run, not
again later. This is available on macOS Mojave and later, but the
macOS app supports only Catalina and later so version is moot. It
may also be largely optional, but Apple docs seem to be nonexistent.
TBD: why does Mergeall's py2app app not require this permission to
access USB drives? There must be a diff in signing or entitlements
in py2app (manifest diffs look trivial), but this seems out of this
project's control and scope. Mergeall, and this app sans F-D-A, asks
for individual folder perms (e.g., Documents, Removables) on access,
and lists these in Settings. This app, however, did not ask for
Documents and so couldn't write logfiles; Docs is in Settings later
for manual enables, but it's safer to prompt the broader F-D-A access
up front. Even so, the app has been seen to ask for Document and
Removables permission when accessed _with_ F-D-A too. As usual,
the rules are convoluted, the docs are thin, and time is limited!
Editorial: macOS has also instituted extreme signing requirements that
deprecate independent developers. Why are device makers so bent on
stifling innovation--and fun--with security lockdowns, for the sake
of a clueless straw-person audience that does not exist? The most
likely explanations seem paranoia and monetizable points of control.
Between this, cruft files, and NFD-Unicode dogma, it's difficult to
continue pretending that macOS is an open Unix system...
UPDATE: macOS Full Disk Access now seems pointless for user-created
content folders. This has been watered down to a weak suggestion
in the popup and docs. macOS always asks for individual folder perms
on first access, whether FDA is held or not. FDA seems to be for
app data and admin files only, though it's stupidly undocumented.
UPDATE [1.1.0]+: due to the elimination of hangs, macoOS permission
popups may overlay folder-inaccessible info_message popups. Add text
to clarify that restarts not required, here and in the all info_message.
macOS shouldn't intercept listdir sans app knowledge this way, but...
--------------------------------------------------------------------
"""
# explain the setting screen first, then open Settings screen
msg = ('About macOS storage permissions.'
'\n\n'
'As you use this app, macOS will ask you to grant permissions to '
'folders like Documents and Removable drives when this app first '
'accesses them. If you don\'t grant these permissions when asked, '
'you\'ll need to enable them for this app later in Settings. '
'You do not need to restart the app after each permission popup.' # [1.1.0]+
'\n\n'
'Though optional for typical use, this app may also benefit '
'from macOS\'s Full Disk Access (FDA) permission. FDA enables '
'access to private app data like Mail and Messages, as well as '
'files related to admin settings. This is not normally required '
'for content folders of your own making, but may be useful in '
'some scenarios.'
'\n\n'
'Toggles for both specific-folder and FDA permissions are in your '
'PC\'s Settings, Security & Privacy, Privacy (or similar); scroll '
'down to the entries for FDA or Files and Folders. As a convenience, '
'this app will open this screen once after this dialog closes.'
'\n\n'
'Whether permissions are granted or not, your content will never '
'leave your devices in this app. See PRIVACY POLICY in the About '
'tab for more info.')
self.info_message(msg,
nextop=self.macos_open_full_disk_access)
def macos_open_full_disk_access(self):
"""
This seems a bit iffy, but works in all tests run so far...
TBD: unclear if should use Privacy or Privacy_AllFiles: the
latter may not work on all, but probably does on Catalina+;
Update: this has been seen to fail with longer suffix; punt!
"""
try:
webbrowser.open(
'x-apple.systempreferences:com.apple.preference.security?Privacy')
except:
excmsg = 'Cannot auto-open privacy Settings: open manually as needed.'
self.info_message(excmsg, usetoast=False)
# and it's up to the user: no way to verify, so don't ask again
#=======================================================================
# STORAGE PATHS
#=======================================================================
# Originally written for Android and its [paths], generalized for PCs.
# Hence the odd coding structures and focus on Android in initial docs.
# This presents a logical model to users which downplays physical paths,
# but still shows them for verification and allows them to be input manually.
# With PCS, "shared" is now preferred storage for platform; "app_specific"
# is moot on PCs; "root" is system-drive root (e.g., '/' on Unix, "C:\"
# on Winsows, something useful on Android if any be); and removable-drive
# name lookup is inherently platform specific be run for all devices.
def storage_path_app_private(self):
"""
--------------------------------------------------------------------
[/data/user/0/com.quixotely.usbsync/files, a.k.a /data/data/com...]
Not for user content: fully private to app (sans SAF), nuked on
uninstall (unless check the toggle added by hasFragileUserData).
Used internally to store config settings, run counts, and more?.
This is Context.getFilesDir() in Java => the files/ subfolder of
app private. The app install is in its (hidden) app/ subfolder.
It has at least 2 names on Samsung phones: /data/data, /data/user/0
(and Python tools don't map the two to a common canonical form...).
There is also Context.getDataDir(); and Environment.getDataDirectory()
for a home folder, which seems to be the parent folder of
Context.getFilesDir(), and shouldn't be used (supposedly).
--------------------------------------------------------------------
"""
if not RunningOnAndroid:
if RunningOnMacOS:
# tbd: macos may not allow access to app's own folder; use ~/Library
# https://learning-python.com/post-release-updates.html#macosquarantine
# okay if this exposes the app's run counter: pc versions are free
applib = osjoin(os.path.expanduser('~'), 'Library',
MACOS_APPPRIV_SUBFOLDER)
if not osexists(applib):
os.makedirs(applib) # make Library too if needed, vs mkdir
return applib
else:
return '.' # Windows, Linux: cwd, which is app install/unzip folder
# original Android code
path = android.storage.app_storage_path()
trace('app-priv:', path)
return path
def storage_path_shared(self):
"""
--------------------------------------------------------------------
[/storage/emulated/0, a.k.a. /sdcard, a.k.a. main|phone|internal]
Preferred for content: fully accessible by POSIX+path with All Files
Access, (except Android/*: use APP for this app's subfolder), and
never auto-removed on app uninstall. Slow, but 2-4X less with UFS-4.
This is Environment.getExternalStorageDirectory() in Java. There's
also call getDataDirectory(), but this seems to be mostly unused by apps.
See also:
plyer: autoclass('android.os.Environment').getExternalStorageDirectory().getAbsolutePath()
plyer: https://github.com/kivy/plyer/blob/master/plyer/platforms/android/storagepath.py
android: https://github.com/kivy/python-for-android/blob/develop/pythonforandroid/recipes/android/src/android/storage.py
--------------------------------------------------------------------
"""
if not RunningOnAndroid: # probably in C:\ = ROOT on Windows, but?
return os.path.expanduser('~') # home: preferred content path on all PCs
# original Android code
path = android.storage.primary_external_storage_path()
trace('shared:', path)
return path
def storage_path_app_specific(self):
"""
--------------------------------------------------------------------
[/storage/emulated/0/Android/data/com.quixotely.usbsync/files]
Optional for content: in Android/data, mostly accessible, fast, nuked on
uninstall (unless check the toggle added by hasFragileUserData in manifest).
This creates the app-spec folder if it does not exist, and returns its path.
Folder cannot be created by the app manually (os.mkdir) in current Androids.
See also: getExternalFilesDirs() returns app-spec on all mounted devices.
----
UPDATE [1.1.0] the first-run hang on Android was finally isolated: there is
a ~10 second hang when picking FROM or TO, but only on the first run after
a fresh install (post uninstall if needed), and only if a chooser is opened
just after attaching a removable drive that has not yet finished mounting.
In this case only, Android's getExternalFilesDir() call can hang, on both
Androids 10 and earlier, and Androids 11 and later. Surprisingly, this has
NOTHING to do with any of the API calls in the removable-drive name fetch
ahead. It happens when trying to make the app-specific folder here.
This is rare and unlikely in the extreme. It doesn't happen for app runs 2
and beyond, and doesn't happen unless a drive mount is in progress when the
chooser is opened, and many users in this case may opt to open a suggested
explorer to handle the drive and wait ill it registers there as mounted.
The odds of a user attaching a USB in run 1 and opening a chooser without
waiting for the mount to finish are _very_ slim. Still, it's a rude welcome.
To address, add a timeout to the hanging call here and return None on hangs.
This will omit APP storage from the chooser, but only when this happens:
chooser opens to and later on run 1 will work fine as will all runs 2+.
--------------------------------------------------------------------
"""
if not RunningOnAndroid:
return None # moot on PCs: do not list in chooser
# original Android code
# from Java docs: getApplicationContext().getExternalFilesDir("")
context = self.get_android_context()
"""
trace('prehang ', time.ctime())
appfile = context.getExternalFilesDir('') # hangware!
trace('posthang', time.ctime())
"""
# get path, else None if exc or hang [1.1.0]
appfile = self.try_any_nohang(context.getExternalFilesDir, '')
if appfile == None:
return None # won't appear in choosers till hang over
apppath = appfile.getAbsolutePath()
trace('app-spec:', apppath, appfile)
return os.path.normpath(os.path.abspath(apppath))
def storage_names_and_paths_removables(self, maxnamelen=32):
r"""
--------------------------------------------------------------------
['drivename' + /mnt/media_rw/xxx-xxxx]
Fully accessible by POSIX and file path with All Files Access
permission, and never auto-removed on uninstalls.
Return attached USB drive's name and /mnt/media_rw/uuid mount path.
The xxxx-xxxx UUID is needed to access the path: can't list media_rw.
Returns [(drive-name, drive-path)], which includes USBs and microSDs.
Main-tab choosers list all returned removable storage names for picks.
The name makes user selection easier: most won't know drive's UUID
(this requires a supporting explorer), and many may not grok paths.
On Android, the list will have multiple USB drives if >1 in a hub, and
may also include removable microSD cards (if any still be). On PCs,
the list will include any mounted (mapped on Windows) device. This
app special-cases Android "Internal storage" as "PHONE" (formerly
"SHARED"), and treats home ("PC") and root ("ROOT") specially on PCs.
Each drive name is truncated at maxnamelen to avoid blowing up GUI.
UPDATE: storage row is now hscrolled, so we don't need to trunc much.
SUBTLE: volume.getDirectory() was None after USB unmount: don't
chain volume.getDirectory().getAbsolutePath(), else craches app.
ALSO SUBTLE: the calls used on Windows can pause the chooser open
for inaccessible network drives and others; meh - ANR is unlikely
(this was addressed for network drives in 1.1.0: see update below).
SEE ALSO: Androids 10 and earlier differ in their behavior around
drive mount and unmount times (and are arguably buggy!); see the
Main tab's folder-chooser dialog docs above for more info. Androids
11+ may hang until the drive is mounted too; a user issue for now.
UPDATE [1.1.0]: this hang was resolved in storage_path_app_specific().
----
WINDOWS UPDATE [1.1.0]: (temp) on Windows, skip the drive-name query
for mapped network ('remote') type drives, because it can hang for
seconds if the drive is inaccessible. Per rumors on the web,
win32wnet.WNetGetConnection() might fetch a network-drive name
(in form '\\server\share') sans lag, but '(Remote)' may be as
good: this is about only networks mapped to drive letters.
----
ALL UPDATE [1.1.0]: the prior update's mod was backed out, and replaced
with thread-based timeout code here, which was also added at later
calls to os.listdir() in chooser open, chooser storage-tap reset,
and Main action verify (see try_listdir_nohang()). The timeout
code here uses try_any_nohang(), and covers all drive-name fetches,
including chooser opens and Main-action confirmation dialogs.
Timeouts were also added for one Android name call, preemptively.
This is a more global/inclusive solution. Forcing '(Remote)' here
helped in this one case but inaccessible network drives can still
hang in many other contexts, and this is not limited to Windows
(macOS hung on dropped network drives for ~5 seconds, and Windows
for ~15), nor network drives (Android has no notion of mounted network
drives, but choosers were known to hang prior to USB-mount completion;
this was resolved with a timeout in storage_path_app_specific()).
On Windows, also tweaked the logic here to better handle network
drives: use WNetGetConnection() to fetch and use \\server\share UNC
path instead of drive letter, because any click in the chooser morphs
the path into this anyhow (due to a change to os.path.realpath() in
Python 3.8+ used by Kivy, which may have also broke '..' to UNC root).
GetVolumeInformation() still used for Windows name, because we're doing
timeouts. An tested router drive is named 'T_Drive' with pathname
'\\readyshare\T_drive' (not '(Remote)' and 'Z:\').
On all PC platforms, network drives are usable in the app iff they are
mounted on Unix or mapped to drive letters on Windows. macOS Finder
mounts at '/Volumes/T_drive'; Linux may mount at '/media/me' or '/mnt'.
----
LINUX UPDATE [1.1.0]+ dec23: unlabeled removables - fix plus docs. Change
Linux mount-output match+extract so unlabeled drives appear in folder chooser.
On Ubuntu 22: no '[label]' at end, and mounts at '/media/me/xxxx-xxxx' UUID.
Example: an SDD (labeled T7) and camera (unlabeled) attached by USB cable:
Result for both (works for none, one, or both):
[('T7', '/media/me/T7'), ('9C33-6BBD', '/media/me/9C33-6BBD')]
Mount output: [~$ mount -l | grep "/media/$USER\|/mnt"]
/dev/sdb1 on /media/me/T7 type exfat (rw,nosuid,nodev,relatime,
uid=1000,gid=1000,fmask=0022,dmask=0022,iocharset=utf8,errors=remount-ro,
uhelper=udisks2) [T7]
/dev/sda1 on /media/me/9C33-6BBD type exfat (rw,nosuid,nodev,relatime,
uid=1000,gid=1000,fmask=0022,dmask=0022,iocharset=utf8,errors=remount-ro,
uhelper=udisks2)
# see also later C: mount below
Prior [scraper = re.compile('^.*? on (.*?) type .*? \(.*?\) \[(.*?)\]\n')]
worked for labeled drives, but failed to match unlabeleds (e.g., camera).
User can always nav down from ROOT, but folder chooser is more friendly.
On [Win, macOS, Andr] name = ['(Removable)', 'Untitled', 'MATSHITA USB Drive'];
app makes up a name on Windows, Android auto makes up a name from device type.
----
LINUX UPDATE [1.1.0]+ (feb24): Windows WSL2 Linux mounts a bunch of spurious
crap in /mnt; tighten up the parsing here to avoid it, as well as similar on
other distros. Caveat: removables still only noticed at WSL2 startup, and some
devs never mount. With a T7 SSD and an unlabeled camera both attached by USB:
On dual-boot (standalone) Ubuntu:
# camera attached => shows up in chooser as "xxxx-xxxx" (and path /media/me/xxxx-xxxx)
# t7 ssd attached => shows up as "T7" (and path /media/me/T7)
# windows c: manually mounted (properly) => shows up as "OS" (and path /media/me/C)
~$ mount -l | grep "/media/$USER\|/mnt"
/dev/nvme0n1p3 on /media/me/C type fuseblk (ro,relatime,user_id=0,group_id=0,allow_other,blksize=4096) [OS]
/dev/sda1 on /media/me/T7 type exfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,iocharset=utf8,errors=remount-ro,uhelper=udisks2) [T7]
/dev/sdb1 on /media/me/9C33-6BBD type exfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,iocharset=utf8,errors=remount-ro,uhelper=udisks2)
~$
~$ mount -l | grep "/media"
<same as above>
~$
~$ mount -l | grep "/mnt"
~$
On WSL2 Ubuntu (after "wsl --shutdown" + restart, while camera visible in Windows)
# camera not mounted, mia in app (but "(Removable)" (D:\) in Windows app)
# C: (win sys) and E: (T7) => "c" (/mnt/c) and "e" (/mnt/e); no "T7" label mia
# bad "wsl", "wslg", "distro", "versions.txt", "doc" (= ROOT) tabs in chooser pre fix
# bad "doc" from "portal" after open gedit
~$ mount -l | grep "/media/$USER\|/mnt"
none on /mnt/wsl type tmpfs (rw,relatime)
none on /mnt/wslg type tmpfs (rw,relatime)
/dev/sdc on /mnt/wslg/distro type ext4 (ro,relatime,discard,errors=remount-ro,data=ordered)
none on /mnt/wslg/versions.txt type overlay (rw,relatime,lowerdir=/systemvhd,upperdir=/system/rw/upper,workdir=/system/rw/work)
none on /mnt/wslg/doc type overlay (rw,relatime,lowerdir=/systemvhd,upperdir=/system/rw/upper,workdir=/system/rw/work)
C:\ on /mnt/c type 9p (rw,noatime,dirsync,aname=drvfs;path=C:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=65536,trans=fd,rfd=5,wfd=5)
E:\ on /mnt/e type 9p (rw,noatime,dirsync,aname=drvfs;path=E:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=65536,trans=fd,rfd=5,wfd=5)
portal on /mnt/wsl/.../doc ...
~$
~$ mount | grep "/media"
~$ mount | grep "media"
~$
~$ mount | grep "/mnt"
<same as above>
To improve mount parsing, drop items for which word #1 is "none", and skip
"distro" on WSL2 (only) because it's identical to ROOT ('/') already shown.
Prior [scraper = re.compile(r'^.*? on (.*?) type .*? \(.*?\)(?: \[(.*?)\])?\n')]
shows unlabeled drives by serial#, but opens a pandora's box now partly closed.
This doesn't hurt standalone Ubuntu Linux, and may or may not improve some
other contexts; alas, Linux is TOO flexible for its users' good at times...
TBD: should the regex allow mounts in /media in addition to /media/$USER?
No distro is known to use this, Ununtu auoto-mounts into the latter by def,
there have been no user reports, and the app mount requirement is documented.
But testing has been limited to date (just ubuntu and wsl); and it's Linux.
UPDATE: DONE; all /media (really, /media*, including /media_rw) now checked
for mounted drives; this should be more inclusive and harmless; but it's Linux.
----
LINUX UPDATE [1.1.0]+ (feb24):: alas, opening gedit adds yet another junk
/mnt mount: "doc" from source "portal" (WSL2 only: mounts to /run elsewhere).
For better or worse, mounts clearly have multiple non-drive roles, so the
rules here were changed again per their Tech Notes description:
'''
Mounts: the app's parsing of Linux mount command output was tightened up to
discard non-disk interlopers in the app's folder-chooser dialog. Specifically:
mounts are now recognized in both /media* and /mnt*, but only if their source
is /dev* (a physical device) or a Windows drive letter pattern like C:\ (used
in WSL2). Any other mounts are accessible by navigating from the ROOT tab in
the chooser.
Before this mod, odd WSL2 mounts in /mnt created bogus storage-root tabs in
the dialog. This included a "distro" that was identical to ROOT (/); multiple
items nested in a "wsl" root; a "version.txt" that wasn't a folder at all; and
a "doc" from source "portal" that showed up after launching gedit. All cropped
up in /mnt, but this folder cannot be simply skipped: unlike Ubuntu on
stand-alone Linux, this is where WSL2 auto-mounts Windows and removable drives
(when it mounts them at all: see the next section).
The new mount parser drops the WSL2 junk, but retains true storage roots
mounted at conventional locations on all Linuxes. It errs on the side of
being conservative, because ROOT provides access to the entire filesystem,
and mounts have numerous roles. Linux flexibility is both asset and liability.
'''
TBD: it might also work to drop the /mnt or /media requirement now that we
require source = /dev (or X:\). This would recognize mounts in /home, etc.
This app is hesitant to do so, because Linux is wildly variable; norms
should be encouraged; testing each distro requires a Linux wipe+install;
and there hasn't been a single Linux user feedback in close to a year.
"""
"""
ANDROID: never-finished alt:
UsbInfo = autoclass('android.hardware.usb.UsbDevice')
trace(UsbInfo)
UsbInst = UsbInfo #()
usb_uuid = UsbInst.getSerialNumber()
use_name = UsbInst.getProductName()
trace('USB:', usb_uuid)
trace('USB:', usb_name)
"""
"""
In Java:
StorageManager storage = (StorageManager) getSystemService(STORAGE_SERVICE);
if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.N) {
List<StorageVolume> volumes = storage.getStorageVolumes();
for (StorageVolume volume :volumes) {
Log.d("STORAGE", "Device name:" + volume.getDescription(this));
}
}
--------------------------------------------------------------------
"""
# if not RunningOnAndroid: # don't nest: there be significant PC code here
if RunningOnMacOS:
#-------------------------------------------------------------------
# this feels wrong (custom mounts?), but works so far...
# grep for removable drives auto-mounted in /Volumes/xxx
# this is where removable drives are auto-mounted by macOS
#
# drive mounts are something macOS arguably does better:
# it combines determinism of Windows' drive letters with
# the generality of Linux's filesystem hierarchy; drives
# are an odd special case in Windows, and at unpredictable
# locations in Linux; neither is true in macOS
#-------------------------------------------------------------------
return [(name, '/Volumes/' + name) # 'M.HD' = ROOT
for name in os.listdir('/Volumes') # mounted here
if name not in ['Macintosh HD', 'Recovery']] # complete?
elif RunningOnWindows:
#-------------------------------------------------------------------
# this also seems bad, and was replaced by pywin32 code below...
#
# possroots = [chr(ord('A') + i) + ':' for i in range(26)] # 'A:'...'Z:'
# usedroots = [root for root in possroots if osexists(root)]
# return [(getnamefromshellcommand?(root), root) for root in usedroots]
#
# do the convoluted and proprietary Windows-API thing via pywin32
# win32 api requires pywin32 (or ctypes) - adds a dependency, but not much
#
# ----
# [1.1.0]+ (feb24) note only => PUNT
# unlabeled drives are displayed as "(Removable)" or "(Remote)"
# which may be subpar if > one each; this could instead use the
# serial number returned in GetVolumeInformation()[3];
#
# otoh: serial number in Windows is a simple long int, not the
# nice Linux xxxx-xxxx uid (though it could be: see below); and
# is a meaningless number really better than "(Removable)" when the
# Windows drive letter is shown on tap? multiple unlabeleds seems
# rare (and meaningless serial numbers won't help either way), and
# macOS's "Untitled" mount label in /Volumes" does no better;
#
# it seems beter to assume that users will label their drives;
# cameras that don't support this, will be the only "(Removable)".
# bonus - to xlate a 32-bit long to ax xxxx-xxxx hex id:
#
# >>> x = 2 ** 32 - 1
# >>> y = ('%X' % x)
# >>> y[:4]+ '-' +y[4:]
# 'FFFF-FFFF'
#-------------------------------------------------------------------
# get drive roots, network paths [after 'pip3 install pywin32' on windows]
# [1.1.0]+ (nov23) moved to top of script: avoid abort if temp folder deleted
# import win32api, win32file, win32wnet
# 'C:\\\x00D:\\\x00Z:\\\x00' => ['C:\\', 'D:\\', 'Z:\\']
#
allroots = win32api.GetLogicalDriveStrings().split('\0')[:-1]
# assoc root with type, so can skip network drive names (now moot) [1.1.0]
# volroots = [root for root in allroots
# if win32file.GetDriveType(root) in usetypes]
# drive types we care about - iff mapped to letter
usetypes = (win32file.DRIVE_REMOVABLE, # usb ssd, flash, microsd, etc
win32file.DRIVE_FIXED, # system drive, other
win32file.DRIVE_CDROM, # bdr or cdrom, verified
win32file.DRIVE_REMOTE) # network or vm host
# filter by type [t7 usb ssd reports as fixed, weirdly]
volroots = []
for rootpath in allroots:
roottype = win32file.GetDriveType(rootpath)
if roottype in usetypes:
volroots.append((rootpath, roottype))
trace('volroots:', volroots)
# get volume names [microSD reports 2 roots, one fails GVI()]
namesandpaths = []
for (rootpath, roottype) in volroots:
# use timouts to avoid lags for inaccessible network drives [1.1.0]
def getnameandpath(rootpath, roottype):
name = win32api.GetVolumeInformation(rootpath)[0] # hangware
if roottype != win32file.DRIVE_REMOTE:
path = rootpath
else:
# use unc for path, not letter
path = win32wnet.WNetGetConnection(rootpath.rstrip('\\'))
return (name, path)
name_path = self.try_any_nohang(
getnameandpath, rootpath, roottype, onfail=(None, None))
# skip drive if name=None: exc or timeput; else rootpath=unc for net
name, rootpath = name_path
if name != None:
if name == '':
vtype = roottype # was win32file.GetDriveType() [1.1.0]
makenames = {win32file.DRIVE_REMOVABLE: '(Removable)',
win32file.DRIVE_FIXED: '(Fixed)',
win32file.DRIVE_CDROM: '(Optical)',
win32file.DRIVE_REMOTE: '(Remote)'}
name = makenames.get(vtype, '?')
name = name[:maxnamelen]
namesandpaths.append((name, rootpath)) # see also confirm: strip parens!
trace('namesandpaths:', namesandpaths)
# set+discard ROOT=system drive [may be named 'OS', 'WINDOWS', ?]
magicvar = 'SYSTEMDRIVE'
if magicvar in os.environ:
# we got ROOT
self.windows_os_root = os.environ[magicvar]
if not self.windows_os_root.endswith('\\'):
self.windows_os_root += '\\' # used by storage_path_root
# don't list system drive > once, regardless of name
namesandpaths = [np for np in namesandpaths
if np[1] != self.windows_os_root]
else:
# go fish: probably never required, but...
osnames = [np for np in namesandpaths if np[0].upper() == 'OS']
winames = [np for np in namesandpaths if np[0].upper() == 'WINDOWS']
if len(osnames) == 1:
self.windows_os_root = osnames[0][1] # use OS first, if just 1
namesandpaths.remove(osnames[0])
elif len(winames) == 1:
self.windows_os_root = winames[0][1] # else try WINDOWS, if just 1
namesandpaths.remove(winames[0])
else:
self.windows_os_root = 'C:\\' # punt: too convoluted to try
trace('root+namesandpaths:', self.windows_os_root, namesandpaths)
return namesandpaths
elif RunningOnLinux: # linux (and wsl2) only, not android
#-------------------------------------------------------------------
# yep, there's a command line for that... but it's brittle...
# parse out the salient bits for chooser's removable-storage tabs
#
# includes native linux, wsl on windows, lx virtual machines
# result+chooser includes 'os' = windows drive if dual boot
# note: removables may be in os: drive=/media/me/yyy, os=/media/me
#
# 'mount -l' adds [label], '/media/$USER' is where USB appears;
# command output and re groups:
#
# /dev/xxx on /media/me/yyy type zzz (xx,xx,xx,xx) [T7SSD]\n'
# 11111111 2222222222222 33333
#
# assume mount commands don't hang [1.1.0]
#
# UPDATE [1.1.0]: grep for mounts in _both_ /media/$USER (original,
# and where system-mounted things like USB drives and the Windows
# system drive show up), and /mnt (new, and where some Linux docs
# suggest other manually mounted external storage like network
# drives should be hosted). /media can be used for all, of course,
# and mounts can really be anywhere; this is opinionated+variable.
#
# UPDATE [1.1.0]+ dec23: match unlabeled drives too by generalizing
# the mount regex to make label optional, and extract UUID for name
# from end of path; see docstring at top of this method for more info.
# Prior: re.compile('^.*? on (.*?) type .*? \(.*?\) \[(.*?)\]\n')
#
# UPDATE [1.1.0]+ (feb24): discard spurious mounts on Windows WSL2
# (and other) Linux - ignore if source (word #1) is not "/dev*" or
# "X:\", or name via path tail is "distro" on WSL2 (= ROOT); see above.
# ALSO: grep all /media/* (users), not just /media/$USER (auto-mounts).
# Formerly:
# shellout = os.popen('mount -l | grep "/media/$USER\|/mnt"').readlines()
# scraper = re.compile(r'^.*? on (.*?) type .*? \(.*?\)(?: \[(.*?)\])?\n')
# scraper = re.compile(r'^(.*?) on (.*?) type .*? \(.*?\)(?: \[(.*?)\])?\n')
#
# UPDATE [1.2.0]: also check that parsed mount points are folders;
# likely moot, but better to be defensive/cautious on this stuff.
#-------------------------------------------------------------------
shellout = os.popen('mount -l | grep "/media\|/mnt"').readlines()
scraper = re.compile(r'^(.*?) on (.+?) type .*? \(.*?\)(?: \[(.*?)\])?\n')
wslmount = re.compile(r'[A-Z]:\\.*')
namesandpaths = []
for line in shellout:
match = scraper.match(line)
if match:
msource, mpath, mlabel = match.groups()
# [1.1.0]+ (feb24) skip non-drives
if not (msource.startswith('/dev') or wslmount.match(msource)):
continue
# [1.2.0] skip non-folders # omit in choosers
if not self.try_isdir_nohang(mpath): # exc, timeout, false
continue
if mlabel != None: # labeled: use [xxx]
name = mlabel or '(Unlabeled)' # unless it's empty
else: # unlabeled: try path
name = mpath.split('/')[-1] # use end of path
if RunningOnWSL2 and name == 'distro': # [1.1.0]+ (feb24)
continue # skip distro==ROOT
name = name[:maxnamelen]
namesandpaths.append((name, mpath)) # label|end, path
trace(namesandpaths)
return namesandpaths
#-----------------------------------------------------------------------
# original Android code: refetch on each call (state may change)
# do the convoluted and proprietary android-api thing via pyjnius
#
# [1.1.0] assume only getDescription() can hang (plus later system calls);
# Android hangs are intermittent+unrepeatable, but timeouts are harmless;
# the self.try_any_nohang(F) returns returns None if F hangs or raises exc;
# also in 1.1.0, skip drive if exc or timeout, don't label as '(Removable)';
#
# UPDATE [1.1.0]: per tracing, the run1 hang doesn't happen here - it was
# finally made repeatable and isolated to the app-specific code above, in
# getExternalFilesDir(), where it was fixed with a timeou. But keep the
# timeout here: it's harmless, and being proactive is shrewd on Android...
#-----------------------------------------------------------------------
context = self.get_android_context()
storage = context.getSystemService(context.STORAGE_SERVICE)
volumes = storage.getStorageVolumes()
namesandpaths = []
for i in range(volumes.size()): # use java iterator
volume = volumes.get(i)
if self.android_version() < 11: # through 10/Q, api 29
pypath = volume.getPath() # why did this go away? (bugs?)
if not pypath: # aosp MediaStore bigotry?
continue
else:
jvfile = volume.getDirectory() # for android 11+
if jvfile == None: # no getDirectory() in 10
continue # None/null post unmount
pypath = jvfile.getAbsolutePath()
"""TEMP
trace(volume.getUuid(), # xxxx-xxxx or None
volume.isPrimary(), # False for usb
volume.isRemovable(), # True for usb
volume.getDescription(context), # name: T7, Internal storage
volume.getDirectory() # java.io.file, None post unmount
if self.android_version() >= 11
else volume.getPath(), # a simple path string
pypath, # posix file-path nirvana
sep=' -- ')
TEMP"""
if (not volume.isPrimary()) and volume.isRemovable():
# get name, else None if exc or hang [1.1.0]
name = self.try_any_nohang(volume.getDescription, context)
if name != None:
# tbd: skip if /dev/null=mounting on android 10-
# this won't help for android 11+: access fails
name = name or '(Removable)'[:maxnamelen] # fake if name is ''; see confirm
namesandpaths.append((name, pypath)) # skip if exc or timeout
trace(namesandpaths)
return namesandpaths
def storage_path_documents(self):
"""
--------------------------------------------------------------------
[/storage/emulated/0/Documents]
Generally accessible and persistent. Used for logfiles on
Android, but not added to Main chooser (or called out in
confirmation dialogs [1.1.0]): it's a sub of shared.
Available in plyer too (though not in android), but it's
simple enough to use the Java-equivalent here explicitly.
[1.1.0]+ (feb24) ~/Documents folder is now created if needed
in init_logfile_folder() (e.g., for WSL2 Linux). This could
be done here instead, but docs folder is used only for logfiles
in this app today, and hence isn't needed until Logs-tab open
or Main-tab action. If it's ever added to folder choosers,
move the create code here instead (and ensure timing is okay).
--------------------------------------------------------------------
"""
if not RunningOnAndroid:
# same on all 3 PCs
return osjoin(os.path.expanduser('~'), 'Documents')
# original Android code
Environment = autoclass('android.os.Environment')
docsname = Environment.DIRECTORY_DOCUMENTS
documents = Environment.getExternalStoragePublicDirectory(docsname)
documents = documents.getAbsolutePath()
if not osexists(documents):
os.mkdir(documents)
return documents
def storage_path_root(self):
"""
--------------------------------------------------------------------
[root is /, but android api returns /system or /storage]
Generally useful only on rooted devices - the chooser adds
a ROOT button iff Config button enables it.
Root is /, which is not accessible on unrooted phones, but the api
calls here return /system (the os mount point) or /storage (storage
sources, which may be more useful).
Allow ROOT to be turned on/of via Config, but show /storage if /
fails, and avoid uncaught excs by setting dialog.rootpath in
kivy FileChooser to prevent ".." navigation altogether.
UPDATE: Environment.getStorageDirectory() is devices mount point,
which reports as /storage on the test device with All Files Access,
and gives access to SAF providers (including Termux's app-private).
UPDATE: os.listdir('/storage') fails; fallback to useless /system
(and consider punting ROOT in full: may require su for listdirs).
--------------------------------------------------------------------
"""
if not RunningOnAndroid:
# neither of these span all drives directly, but useful;
# Windows assumes already set by names_and_paths: call first
if RunningOnWindows: # from env var or volume names
return self.windows_os_root # probably c:\, but not always
else: # it's macOS or Linux or ?
return '/' # system drive root, see also os=/media
# original Android code
Environment = autoclass('android.os.Environment')
roots = ['/', # canonical
Environment.getStorageDirectory().getAbsolutePath(), # /storage
Environment.getRootDirectory().getAbsolutePath(), # /system
]
for root in roots:
try:
os.listdir(root) # assume won't hang [1.1.0]
except:
pass
else:
return root
return None # all failed
#=======================================================================
# SETTINGS PERSISTENCE
#=======================================================================
def init_setting_defaults(self):
"""
--------------------------------------------------------------------
Paths and Config-tab settings: map .kv-file GUI ids => field defaults.
Now a method (not class attributes), to use self in setting values.
Main-tab paths:
Loaded on start, auto-saved on Main runs, not saved/reset on Config btns
Config-tab configs:
Loaded on start, manually saved/reset on Config btns, not auto-saved on Main runs
For Main-tab paths, fail in default kivy font: ⇦ ← ⇐ ⬅; go DejaVuSans
for these two fields in the .kv file (better appearance anyhow).
NOTE: because these defaults are reloaded into the GUI on Config-tab's
'Restore' and fetched by code from the GUI, all presets tweaks must happen
here only; else a Restore may mod a setting in a way triggers erroneous
actions, even if the corresponding Config-tab widget is disabled.
--------------------------------------------------------------------
"""
self.config_ids_Main = dict(
# main-tab paths, auto-save on run
frompath = '⇦ Tap to select FROM, or enter here',
topath = '⇦ Tap to select TO, or enter here',
)
self.config_ids_Config = dict(
# config-tab settings, manual save/restore
# toggles
backupscheckbox = True,
skipcruftscheckbox = True,
runactionasservice = RunningOnAndroid and self.android_version() >= 9,
appfolderselections = RunningOnAndroid, # moot on PCs
rootfolderselections = not RunningOnAndroid, # start True on PCs
keepscreenon = True, # applies to android+pcs [1.1.0]
# inputs
maxnumbackups = 25,
maxnumlogfiles = 30,
maxlogstailsize = 10 * 1024, # was 32k: save graphics mem
# appearance
logsbackgroundcolor = '#000038',
logsforegroundcolor = 'white',
globalfontsize = FONT_SIZE_DEFAULT, # '15sp' to abs: set_font_size()
mainchoosericon = ['down'],
mainchooserlist = ['normal'], # last two are one selection
# defunct
#logfilepathname = 'Select or enter...' # always log, same folder
#logfilescheckbox = True,
#fixfilenamescheckbox = True, # moved to Main action
)
self.config_ids_All = self.config_ids_Main.copy()
self.config_ids_All.update(self.config_ids_Config) # i.e., Main + Config
def load_settings_from_file(self):
"""
Load ALL settings from dict in app-private pickle file.
"""
apppriv_path = self.storage_path_app_private() or ''
settings_path = osjoin(apppriv_path, SETTINGS_FILE)
if not osexists(settings_path):
settings = self.config_ids_All.copy() # first run: defaults
else:
try:
settings_file = open(settings_path, 'rb')
settings = pickle.load(settings_file)
settings_file.close()
except:
settings = self.config_ids_All.copy()
self.info_message('Cannot load settings: please report.', usetoast=True)
return settings
def save_settings_to_file(self, settings):
"""
Save ALL settings to dict in app-private pickle file.
"""
try:
apppriv_path = self.storage_path_app_private() or ''
settings_path = osjoin(apppriv_path, SETTINGS_FILE)
settings_file = open(settings_path, 'wb')
pickle.dump(settings, settings_file)
settings_file.close()
except:
self.info_message('Cannot save settings: please report.', usetoast=True)
def update_GUI_from_settings(self, settings):
"""
Copy ALL loaded settings into the GUI's fields.
"""
for (id, value) in settings.items():
try:
if type(value) == str:
self.ids[id].text = settings[id] # str text
elif type(value) == bool:
self.ids[id].active = settings[id] # checkbox
elif type(value) == int:
self.ids[id].text = '{:,}'.format(settings[id]) # int text
elif type(value) == list:
self.ids[id].state = settings[id][0] # radio btn
except KeyError:
pass # old app version + new settings file - skip new key [1.1.0]
def update_settings_from_GUI(self, settings, config_ids):
"""
Update SOME settings in-place from GUI fields: config_ids=Main or Config.
"""
for (id, default) in config_ids.items():
if type(default) == str:
settings[id] = self.ids[id].text
elif type(default) == bool:
settings[id] = self.ids[id].active
elif type(default) == int:
settings[id] = int(self.ids[id].text.replace(',', ''))
elif type(default) == list:
settings[id] = [self.ids[id].state]
def reset_persisted_settings_Config(self):
"""
Restore all CONFIG settings to original "factory" defaults.
This resets Config-tab settings only, NOT Main-tab paths;
the latter would be a suprise, as paths are on another tab.
TBD: should this also write them to persistence file now?
"""
# gui fields
for (id, default) in self.config_ids_Config.items():
if type(default) == str:
self.ids[id].text = default
elif type(default) == bool:
self.ids[id].active = default
elif type(default) == int:
self.ids[id].text = '{:,}'.format(default)
elif type(default) == list:
self.ids[id].state = default[0]
# internal copy
self.settings.update(self.config_ids_Config) # Main paths never reset
# apply Config's font size now, else user must tap Apply after
# Restore, even though the Config tab has been reset to the default;
# this is grayer than startup (user edits aren't auto applied either),
# but it's arguably user friendlier to make the GUI match the restore
self.set_font_size(self.ids.globalfontsize.text)
def load_persisted_settings(self):
"""
--------------------------------------------------------------------
On startup, fetch saved configs+paths, and use them to intialize
the GUI's fields. This includes all settings on the Config tab,
and all selectable paths (i.e., FROM and TO).
Settings start with the defaults above before any saves. Paths
are saved automatically on each Main-tab action run, and configs
are saved manually on each Config-tab save button path tap.
The Config-tab restore button restores only configs, not paths,
because paths are on a different tab. The Config tab includes
both run and non-run settings, but auto-saves are too presumptive.
Stored in app-private storage (not app install), in a pickle-file
dict, where the .kv file's IDs are used as dict keys in settings.
App-private is app's data folder, can't be seen by explorers or
other apps, and goes away on app uninstall unless user toggles
the save option in uninstall popup (per manifest fragile setting).
UPDATE [1.1.0] copy any new items in defaults to settings loaded
from file, so that they will be set in GUI and added to file on
later saves. Else first value comes from .kv, not defaults here,
which is fairly harmless, but makes for redundant code/logic.
Note that this makes new versions of the app backward compatible
with existing settings files, but not vice vera: running an old
version of the app with a new settings file that has new keys
will fail when updating the GUI; catch excs in GUI load for this.
--------------------------------------------------------------------
"""
# Set up presets once now
self.init_setting_defaults()
# Internal copy, kept for later saves
self.settings = self.load_settings_from_file()
# Update loaded settings for anything new in defaults [1.1.0]
for key in self.config_ids_All:
if key not in self.settings:
self.settings[key] = self.config_ids_All[key]
# Copy all loaded settings into the GUI's fields
self.update_GUI_from_settings(self.settings)
# Extras call moved to App.on_start() to make it more explicit
def save_persisted_settings_Main(self):
"""
On Main-tab action run: update Main-tab paths in persistence file.
Nit: it's inefficient to save all each time, but the set is small.
"""
# Get Main settings from the GUI's fields
self.update_settings_from_GUI(self.settings, self.config_ids_Main)
# Save full settings dict to pickle file
self.save_settings_to_file(self.settings)
def save_persisted_settings_Config(self):
"""
On Config-tab Save: update Config-tab settings in persistence file.
maxbkps is a mergeall config-module edit in app-install subfolder,
and subtly also requires updating the imported module: see startup.
"""
# Get Config settings from the GUI's fields
self.update_settings_from_GUI(self.settings, self.config_ids_Config)
# Save full settings dict to pickle file
self.save_settings_to_file(self.settings)
def startup_gui_and_config_tweaks(self):
"""
Assorted mods to gui and configs for platform diffs/etc, post load
of persisted setings. Now called explicitly from App.on_start(),
after load_persisted_settings has initialized or loaded settings.
"""
# Mod filename fixer, now a Main tab action, not a config
# Update: PUNT - this looks odd, wasn't re-disabled on action exit,
# and it's unknown if non-Windows filesystems on Windows need NAME
if RunningOnWindows:
# Nonportable names not possible on Windows or Android shared
"""
self.ids.namebutton.disabled = True
self.ids.namelabel.disabled = True
"""
pass
"""
TBD: Android appspec allows nonportables, but not advised; init only
elif RunningOnAndroid and
not self.ids['frompath'].text.startswith(self.storage_path_app_specific()):
# yes, but need to reset on path changes - even manual edits; punt
"""
# On android 8.x (only), foregroundservice works but broadcastreceiver does
# not, for reason that 8's market share no longer justifies uncovering; punt!
def disable_android_service_gui():
# must do in defaults: self.ids.runactionasservice.active = False
self.ids.runactionasservice.disabled = True
self.ids.runactionasservicelabel.disabled = True
if RunningOnAndroid:
if self.android_version() < 9: # i.e., android 8~8.1, api 26~27
disable_android_service_gui()
else:
# on 9+, resume in-progress state if kill+restart while service runs
# UPDATE: PUNT - deprecated and inaccurate; see the method for docs
# self.check_for_running_service()
pass
else:
# PCs - foreground service and notifications moot
disable_android_service_gui()
if not RunningOnAndroid:
#
# no locked-down/auto-removed app storage on pcs
# must do in defaults: self.ids.appfolderselections.active = False
#
self.ids.appfolderselections.disabled = True
self.ids.appfolderselectionslabel.disabled = True
if not RunningOnAndroid:
#
# scrolling is too slow on PCs (only)
#
scrollviews = ('helptextscroll', 'abouttextscroll',
'picklogscroll', 'logfilescroll')
for scroll in scrollviews:
if RunningOnWindows or RunningOnLinux:
self.ids[scroll].scroll_wheel_distance = 20
self.ids[scroll].smooth_scroll_end = 20
else:
self.ids[scroll].scroll_wheel_distance = 10 # macos is special
self.ids[scroll].smooth_scroll_end = 60
if not RunningOnAndroid:
#
# kill the BIZARRO persistent orange circles on double clicks (kivy bug!)
# seen on windows+macos; not documented, and wth hasn't this been fixed?
#
Config.set('input', 'mouse', 'mouse,multitouch_on_demand')
if RunningOnWindows or RunningOnLinux:
# --------------------------------------------------------------------
# window size
#
# The default size opens too small on these platforms only;
# doing this earlier at imports does not avoid resize twitch
#
# no effect: Config.set('graphics', 'width', 1000)
# no effect: Config.set('graphics', 'height', 900)
# Config.write() may help, but applies to all apps always
#
# ----
# [1.1.0]+ (sep23): use dp() density-scaled pixels here too, else
# preset size was okay on a low-res (2k) test machine, but too small
# to use on a high-res (4k) screen sans a manual resize on each open;
# this mod was released in the Windows+Linux executabes _only_ (not
# Android or macOS: unused); on Windows, the new dp() setting here
# looks same as 'original' setting on low-res display (for screenshots);
#
# note: _not_ setting Windows.size (always, or on runs 2+) doesn't
# help - Windows does not remember/restore the prior run's size, so
# the window will always open at the too-small-on-low-res default;
#
# also: dp() seems iffy on Linux - add a fudge factor else too small
# [this is risky: has been tested on just one linux device to date];
# on linux test dev, dp() does nothing... which means scale factor
# is 1... which is distubing, because dp() matters for same 1920x1200
# display on Windows in a dual-boot setup; does scaling really work?;
#
# linux has lotsa problems: default fonts start painfully small too
# (this was tweaked in [1.1.0]+ (oct23) above), and it's unknown if
# the linux executable works beyond test devs (the [1.1.0]+ (oct23)
# rebuild on ubuntu22 fixes a lib-skew issue in u20; others likely);
# note: RunningOnLinux constant means linux only, and not android;
#--------------------------------------------------------------------
initial_center = Window.center # save for recenter next [1.1.0]+ (oct-24-23)
lx = 150 if RunningOnLinux else 0 # was 100 (oct23)
Window.size = (kivy.metrics.dp(748+lx), kivy.metrics.dp(612+lx)) # (w, h), [1.1.0]+
#Window.size = (748, 612) # different: too small on low-res dell
#Window.size = (1120, 920) # original: too small on high-res yoga
if RunningOnWindows: # not Linux: see ahead
#--------------------------------------------------------------------
# window position
#
# [1.1.0]+ (oct-22-23): Windows' initial window position is lousy:
# users must move the window immediately to see tha bottom part
# of the GUI which comes up offscreen. This has been seen on both
# 2K and 4K displays. E.g., default = [300,560] on 1920x1200,
# but 560+scaled(612) is offscreen (scaled size is [1122,918]
# after code below, and 560+918=1418 > 1200 screen height).
#
# Force startup near upper left corner here, and scale for screen
# density, but don't place any furter down/right than the default.
# E.g., on 1920x1200, scaled height 612 = 918, scaled pos 85 = 127,
# and 127+918=1045: onscreen. Scaling will be higher on 4k and
# lower on 1K (e.g., 85 + 612 = 697 < any usable display's height).
#
# Position can be set before size (ahead) and doesn't impact
# the splashscreen here. Note: env var SDL_VIDEO_WINDOW_POS
# is supposed to set position, but doesn't (anymore? on Win?).
# This also isn't an issue on macOS, which centers windows well,
# nor on Linux, where there's ample space below a smaller window.
#
# Subtly: this may be a timing issue in Kivy. Per testing, it
# appears that window position is set by Kivy just _once_ and for
# the _default_ size. This explains why, sans the pos/size sets
# here, the spashscreen and window are the same size on Windows,
# and both center well (and why macOS centers well with default
# size). With the required size setting below, the splashscreen
# stays at the original size for default window. This is arguably
# a bug in kivy: setting size should auto update position auto,
# especially before the window is opened.
#
# With the pos/size sets here, the splashscreen is smaller than
# the window and slighly to the right/down (because it's centered),
# but the skew is trivial. Running the size set alone and earlier
# (e.g., at top of script) might auto center, but could also make
# the splashscreen huge (TBD). In any event, the window must be
# resized, because the default is too small to be usable on Windows.
# Kivy also does not allow Windows to auto-position new windows
# (which also pushes content off-screen eventually in tkinter).
#
# ====
# UPDATE [1.1.0]+ (oct-24-23): this now _recenters_ the window on
# the display, and _after_ it has been resized, instead of just
# positioning it at scaled(85, 85) pixels before resizing.
# This is marginally better aesthetics (and avoids bringing up
# the window too far away from the splash screen), though the window
# is slightly lower than the 85px original (by single-digit pixels),
# and the window is slightly higher than center (by the size of the
# Windows taskbar - the app's OpenGL window itself is centered,
# but the Windows taskbar at the top grabs more space above it).
#
# There is no known direct way in kivy to get physical display size
# in pixels for manual centering calcs. Instead, move position
# per new post-resize Window.center point, but not beyond the
# (0,0) point at top left (else may be off-screen again?), and not
# so high that Windows taskbar off-screen (hopefully: a guess). The
# Window.center gives a pixels value portably, and can be use to
# do positions indirecty. Doing only resize in App.build() or at top
# of script had no effect (see tries ahead): kivy doesn't auto-recenter
# on size mods, and probably shouldn't, _except_ before window opened.
# As is, kivy auto-centers just once, and for a default (small) size
# (and this drama is all because kivy default size on Windows sucked).
# Skip on Linux: position okay, and Wayland may disallow repositions?
#--------------------------------------------------------------------
"""ORIGINAL
dflttop, dfltleft = Window.top, Window.left
Window.top = min(dflttop, kivy.metrics.dp(85))
Window.left = min(dfltleft, kivy.metrics.dp(85))
"""
# recenter instead of pinning to abs location
new_center = Window.center # center after resize above
diffx = new_center[0] - initial_center[0] # how much did center change?
diffy = new_center[1] - initial_center[1]
Window.left = max(5, Window.left - diffx) # adjust top-left for change
Window.top = max(25, Window.top - diffy) # but not past top-left corner
trace('Repos:', initial_center, new_center) # @2k: (400.0, 300.0) (561.0, 459.0)
# apply Config's font size now, else user must tap Apply each
# run, even though Config has been set to the last size saved;
# see also restore-defaults: similarly auto-applies value set;
self.set_font_size(self.ids.globalfontsize.text)
# Force left-align text in Main tab's path fields (else pseudo-random)
# weirdly hard, but halign and scroll_x in .kv had no effect, for reasons tbd
# scheduling separately helps, but still fails on 1 of 6 test devs (andr9)
# UPDATE: this is related to font size: moving below that setting here
# fixed andr9 too; clock scheduling is still required, but just once...
def leftalign(field):
self.ids[field].cursor = (0, 0)
Clock.schedule_once(lambda dt: (leftalign('frompath'), leftalign('topath')))
#Clock.schedule_once(lambda dt: leftalign('frompath'))
#Clock.schedule_once(lambda dt: leftalign('topath'))
"""
now done in class def, so picked up at each instance build
if RunningOnMacOS:
# [r, g, b, a] for image tinting - too dim on macos only
ConfigCheckbox.check_box_color = [1, 3, 5, 4] # else default = [1, 1, 1, 1]
"""
# [1.1.0] initialize sdl2 screen-stays-on switch from configs;
# this applies to both pc screen saver and android screen timeout
Window.allow_screensaver = not self.settings['keepscreenon'] # see on_start()
def update_mergeall_configs_maxbackups(self):
"""
--------------------------------------------------------------------
On Main-tab SYNC runs only, apply the Config tab's setting
maxnumbackups to mergeall/mergeall_configs.py such that it
will be used by mergeall sync runs. This needs no changes
in Mergeall's base code (e.g., a new command arg), but must
be implemented in ways that vary by mergeall.py run mode:
+ For processes, both foreground-service (Android) and simple
(PC?), it suffices to update the config module's source code,
because mergeall runs in its own process and imports the
config module from its changed .py anew on each run.
+ For threads (Android + PC), this must update the imported
configs module object, because mergeall runs in the same
single app process, in which configs is only ever imported
once, even if its source code is changed for processes here.
A module reload works too, but setting an attribute in the
sudbir module imported at startup suffices.
Since we don't know if mergeall will be run as processes,
threads, or any mix of these, we must accommodate both run
modes here. Threads need only imported-module mods, and
processes use only source, but either can be run any time.
Because this always updates for the current Config-tab setting,
there is no potential for source/setting skew. On app start,
the Config tab will be reloaded with the last setting saved.
On each SYNC run, the source code and imported module will be
updated for whatever is in the Config tab at the time.
App startup initially picks up the setting last saved to the
source code via import, but SYNCs always use and apply the
setting in the Config tab, loaded on app start. Hence, if a
Config-tab setting was used for a SYNC but not saved, the
latest save will override the source on subsequent SYNCs.
That is: module object (for threads) and code (for processes)
always agree on SYNCs, and reflect the current value in Config,
which may be persisted across runs or not.
----
UPDATE: it gets subtler - this originally reset the max setting
in the mergeall_configs module, but backup.py gets the setting
with a "from" instead of an "import." Hence, backup's value
won't be updated if mergeall_configs is changed here: mergeall
imoorts backup which imports mergeall_configs, but backup's
"from" gets the setting, not its module, and backup itself
will never be imported again by mergeall in the app process.
Thus: this must mod backup, not mergeall_configs, else the
update won't be noticed when SYNC runs as a thread (!).
It's also simple to access backup in sys.modules, which is
shared by app and script threads; it must still be imported
initially, however, to get its value to detect diffs for both
thread and process runs. mergeall_configs' MAXBACKUPS is used
nowhere else in mergeall (and has no other app configurables).
See also init_mergeall_globals(), which is similarly run on
SYNCs to reset global state in some of mergeall's imports,
but for same-process threads only (here, we mod for both).
For threads ONLY, that imports backup before this is called.
Subtle: this need not be called for UNDO, as -backup not used.
----
UPDATE: and it gets worse - resetting MAXBACKUPS in the backup
module shared by app and script threads still has no effect,
because the setting is a function-argument default in backup:
def prunebkpdirs(toroot, maxbackups=MAXBACKUPS):
The setting in this is evaluated once at _function creation time_
and the result is attached to the function object. Hence, while
changing MAXBACKUPS in the shared module works, it has no impact
on this function, which is ultimately what mergeall.py calls.
So... as a third rewrite here (!), instead remove the backup and
mergeall_configs modules from sys.modules in full, to force imports
in the thread to reload from the changed source code. Probably,
a change in backup.py and/or a new command arg in mergeall.py for
maxbackups is warranted now, but the goal was no Mergeall changes.
None of init_mergeall_globals()'s attr mods are functions args.
Forcing reloads here makes that function moot for backup.py, but
ONLY when the Config-tab's numbackups setting differs from the
code on a SYNC, and that function isn't called for process runs.
Note: mergeall.py prunes backups only for SYNCs with backups
enabled that actually make changes (thereby triggering a new
backup folder). Lowering the max setting will have no effect
until this happens. By contrast, logfiles are pruned each run.
----
UPDATE+CAVEAT: the change made here to mergeall/mergeall_configs.py
in the install folder will be undone if the user clears this app's
data with Settings=>Apps=>appname=>Storage=>Clear data. This is
officially expected behaviour (though an arguable misfeature in
Android!). Since this also wipes/resets all Config-tab user settings,
the reset to the .py seems reasonable (and fixes are too extreme).
See handle_trial_counter() for a related uninstall-wipe drama.
WEIRDLY, this is not an issue for reinstalls: the .py mod here (along
with configs and all other files in app-private/install folders) is
retained by and survives both an adb streamed reinstall (which is
presumably what Play store updates do), and an uninstall+install if
the unsintall option to keep app data is checked. Hence, it's JUST
Settings' clear-app-data action that resets in full - inconsistently!
--------------------------------------------------------------------
"""
if 'backup' not in sys.modules:
# first script run: import backup module
trace('ma=> importing backup') # detect diffs, for threads + processes
os.chdir('mergeall') # in app-install (unzip) folder
sys.path.append(os.getcwd()) # '.' fails here, use real path
import backup # threads run in subfolder, same process
os.chdir('..') # services run in separate process
sys.path.pop()
configbkps = int(self.ids.maxnumbackups.text) # Configs-tab setting
mergeallbkps = sys.modules['backup'].MAXBACKUPS # value imported into module by "from"
if configbkps != mergeallbkps:
# 1) Source code - for processes (and future app runs)
trace('ma=> configs code update')
try:
maconfigspath = osjoin('mergeall', 'mergeall_configs.py')
maconfigsfile = open(maconfigspath, 'r', encoding='utf8')
maconfigstext = maconfigsfile.read()
maconfigsfile.close()
replace = 'MAXBACKUPS = %d' % configbkps
pattern = '^MAXBACKUPS = [0-9]*'
maconfigstext = re.sub(pattern, replace, maconfigstext, 1, re.MULTILINE)
maconfigsfile = open(maconfigspath, 'w', encoding='utf8')
maconfigsfile.write(maconfigstext)
maconfigsfile.close()
except:
self.info_message('Cannot change max backups.', usetoast=True)
# 2) Imported module - for threads
trace('ma=> backup module update')
# reset mod attr: fails if func arg default
#sys.modules['backup'].MAXBACKUPS = configbkps
# force import of new code in script thread
del sys.modules['backup'] # mergeall => backup => mergeall_configs
del sys.modules['mergeall_configs'] # or dict.pop()
#=======================================================================
# CONFIG AND HELP TABS
#=======================================================================
def pickcolor(self, onpick, kind):
"""
On Pick in bg or fg color in Configs tab: open chooser.
The chooser's Pick in turn passes color to do_ handler.
In detail: onpick is passed in from the Config-tab button
and sent to the popup, and is one of the do_x_color below
(plus self); it's not called until the popup's on_release.
"""
picker = ColorPickDialog(onpick=onpick,
oncancel=self.dismiss_popup)
self._popups.push(Popup(title='Choose %s Color' % kind,
content=picker,
auto_dismiss=False, # ignore taps outside
size_hint=(0.9, 0.9))) # max space
self._popups.open()
def do_bg_color(self, color, hexcolor, popupcontent):
"""
Log tab's text color now set automatically via cross-tab ref in
the .kv file to Config tab color display. Else Log tab is not
reset on defaults restore in Config tab.
Config tab colors must be readonly, else app crashes as user types
and so must be TextInputNoSelect to avoid lingering handles; blah.
"""
self.dismiss_popup(popupcontent)
self.ids.logsbackgroundcolor.text = hexcolor # Config tab (hex str)
#self.logfiletext.background_color = color # Logs tab (rgba list)
def do_fg_color(self, color, hexcolor, popupcontent):
"Ditto"
self.dismiss_popup(popupcontent)
self.ids.logsforegroundcolor.text = hexcolor # or tie to 1 prop? -> yes
#self.logfiletext.foreground_color = color # too many ways to do it
def set_font_size(self, fontsize):
"""
--------------------------------------------------------------------
On Apply for font size in Config: setting the App instance's
dynamic_font_size here automatically changes font size for
EVERY wdget in GUI, because Widget font_size is bound to it
in the .kv file. Powerful (and arguably scary). Otherwise,
GUIs can bind font_size to this property in selected widgets.
This size is in absolute pixels as coded, but could append size
string with 'dp' or 'sp' to use density and user+density scaling:
see kivy.org/doc/stable/api-kivy.metrics.html#kivy.metrics.sp.
This seems arbitrary (a relative # is a relative #), but the
preset defalt may need to be scaled per device on startup.
Kivy's default is '15sp' which maps to '39' pixels on a Note20.
----
UPDATE: use 'sp' pixels to avoid the scaling issues of preset...
Downside: this provides less granularity then absolute pixels
(1 sp pixel is about 2.56absolute pixels on screens like the
Note20's), and could probably specialize preset by platform (TBD).
UPDATE: use the fontsize from the GUI as a string and allow
input per float filter, instead of assuming an int(). This
allows fraction sp pixels to compensate for the granularity.
UPDATE: for better granularity, use absolute int pixels in GUI,
but set default/start to '15sp' equivalent per kivy.metrics (39
on Note20). This scales per device and user settings on first
run, though it won't auto-update for user settings till rerun.
Config layout note: font is not readonly, so it doesn't need
TextInputNoSelect: select handles go away when keyboard is
minimized. Likewise for max number logfiles and backups.
UPDATE: this is now also run on startup to set fontsize in
configs saved by user; else user must click Apply each run.
--------------------------------------------------------------------
"""
# there be much magic here...
App.get_running_app().dynamic_font_size = fontsize # abs, not + 'sp'
# _not_ needed to force Log's filelist minimum_width to
# recalc for scrolling - happens auto when font changed;
# see the Log's tab's code in tab-change handler ahead
##self.width = self.width
def do_explore_backups(self):
"""
On Help tab's 'explore TO SYNC backups.' This uses the
TO setting on Main tab, but it's too tight to put there.
TO may be anything at this point if was entered manually.
"""
topath = self.ids.topath.text # path on another tab
if not self.try_isdir_nohang(topath): # hangware on win [1.1.0]
self.info_message('TO on Main tab is not a folder.', usetoast=True)
elif not os.path.isdir(osjoin(topath, '__bkp__')):
self.info_message('TO on Main tab has no __bkp__ folder.', usetoast=True)
else:
self.launch_file_explorer(osjoin(topath, '__bkp__'))
def on_backups_clicked(self, toggleon):
"""
Warn users of the UNDO consequences of toggling off SYNC
backups in the Config tab. Because it's a major risk,
and people don't read docs, right?...
"""
trace('backups on_touch_up')
if not toggleon:
self.info_message(
'CAUTION: backups disabled.'
'\n\n'
'Disabling SYNC backups may save '
'some storage space and make SYNCs run faster, '
'but also makes it impossible to use UNDO to '
'roll back changes made by SYNCs run while '
'this toggle is off.'
'\n\n'
'Please use with care, and see the user guide '
'for more details.',
usetoast=False)
def on_keepscreenon_clicked(self, toggleon):
"""
[1.1.0] Users decide if screen should timeout or be kept on.
This toggles the SDL2 flag flavor; for this to work, p4a's
redundant wakelock alt must be disabled in buildozer.spec.
Could do this in the .kv too, but parallels backups tap.
Note: False disables both Android timeouts and PC screensavers.
This might be limited to Android via p4a, but there's no simple
way to disable p4a's wakelock code as there is for sdl2's alt.
See App.on_start()s windows section for more background.
"""
# True/on disables PC screensaver + Android screen timout
trace('keep-sceen-on:', toggleon)
Window.allow_screensaver = not toggleon
# See also: chooser icon/list style configs handled in the popup's code
#=======================================================================
# LOGS-TAB ACTIONS
#=======================================================================
def log_ops_preface(self, picklogitems):
"""
Fetch and verify path of selected logfile in Logs tab.
Never get here unless there are logfiles to choose (?),
which implies that log folder created and accessible.
"""
# btnsgroup is a single-selection list
btngroup = picklogitems.children
btndowns = [btn for btn in btngroup if btn.state == 'down']
logfilepath = osjoin(self.logfile_path, btndowns[0].text) if btndowns else None
# it's possible to deselect the sole radio-buttons selection
# update: no longer possible after added allow_no_selection=False
# UPDATE: but this test still fires if there are no logfiles yet (?)
if not logfilepath:
self.info_message('Please select a logfile.', usetoast=True)
return None
# this seems very unlikely with custom chooser and name globs, but...
if not os.path.isfile(logfilepath): # assume won't hang [1.1.0]
self.info_message('Logfile path is not a file.', usetoast=True)
return None
return logfilepath
def truncate_long_lines(self, text, maxline=400): # was 512 [1.1.0]
"""
Kivy text bug: displays a black box for very long lines (!).
To work around, truncate long longs, at some cost in speed
(though this is minor, because text is just the tail here).
Actual maxline limit is unknown; constant works in practice.
Kivy's code tries to trim lines, but fails: please fix this.
UPDATE [1.1.0]: at the original maxline=512, one black line
has been seen in the wild - among many thousands, but it's
a glitch. Unfortunately, there is no absolute cutoff: per
testing, lines > 512 (and up to 600+) worked with blackouts,
so this must depend on context (or fate?), and scaling lines
back further may discard useful info. As a compromise, change
to maxline=400 to reduce blackouts risk further, but this is
still a hueristic guess. Kivy: fix me!!
"""
return ''.join(
[line + '\n' if len(line) <= maxline else line[:maxline] + '...\n'
for line in text.splitlines()])
# TAIL-------------------------------------------------------------
def purge_the_stupid_kivy_textinput_cache(self):
"""
--------------------------------------------------------------------
Empty the TextInput widget's cache, which causes fatal
memory explosion on most Android phones tested if the
text content is changed regularly. Called by TAIL and
WATCH whenever the widget's text is reset. There's no
way this misfeature should be so hidden; it cost days of
hardcore dev time. See WATCH for more info (and rant).
If this cache has any benefit, it's hardly universal.
Note [1.1.0]: kivy.uix.textinput._textinput_clear_cache()
runs the two removes here, but also gets rid of some
cached TI weakrefs. Per macOS testing, this call has no
extra effect: the app (which uses the code here) is always
~15M larger than source - whether source uses the code here
or the alternative. This holds true both at startup and
after running the same ops, and both app and source start
near 100M and hover around 150~200M after running many ops
and logfile views. Kivy requires a lot of memory...
--------------------------------------------------------------------
"""
# stop the #$@%* memory blowup!
#from kivy.cache import Cache
Cache.remove('textinput.label')
Cache.remove('textinput.width')
def do_logs_tail(self, picklogitems, prefacesize=512):
"""
--------------------------------------------------------------------
Show the last part of a selected logfile on demand.
Scroll to the end (where summaries are) for ease,
and add a brief preamble from the start of the log.
This is usually enough info, and avoids full loads
of large logfiles that may exceed kivy limitations
or require an Android Intent multi-step interaction.
Caution: seeking to a random byte position in a text file
can return partial/split Unicode characters, which will
fail to decode and raise exceptions (seen in the wild).
Both here and in WATCH, user errors=replace to replace
the partial-character bytes (with '?', probably).
We don't need to run text through ascii() for GUI display,
as Kivy shows Unicode code points sans glyphs as a rectangle.
We do need to re-enable hscroll by resetting text size, on
text sets (here+), and width changes (phone rotate, pc resize).
UPDATE: scaled tailsize preset config down from 32k to 10k
due to an odd graphics-memory drain; see WATCH's update ahead.
Also set max size to 256k; the prior 512k seems a bit iffy.
----
Note [1.1.0]: the auto-scrolls here and in WATCH do nothing
if a prior manual scroll's ending animation has not finished.
Neither moving the a-s code past enable_hscroll() nor running
it with Clock.schedule_once() help. Tweaking scroll effects
may, but this is as much usage issue as bug, and the required
tweaks (if any be) have zero docs and wildly obfuscated code...
UPDATE [1.1.0]: the prior Note's auto-scrolls bug has finally
been fixed, by setting 'velocity' to 0 in the the auto-created
x/y instances of the effect_cls set in the .kv file. This is
stupidly hidden knowledge, which required numerous excursions
in Kivy's source code, and trail-and-error guesswork in the end.
The docs on this are so lean and bad that they qualify as rude!
--------------------------------------------------------------------
"""
logfilepath = self.log_ops_preface(picklogitems)
if not logfilepath:
return
try:
# now in configs tab, not arg
tailsize = int(self.ids.maxlogstailsize.text.replace(',', ''))
if tailsize > (256 * 1024):
self.info_message('TAIL size too big in Configs: try OPEN.',
usetoast=True)
return
logsize = os.path.getsize(logfilepath)
logfile = open(logfilepath, 'r', encoding='utf8', errors='replace')
logfile.seek(max(0, logsize - tailsize))
logtext = logfile.read()
trace('logsize:', logsize)
if logsize > tailsize:
# drop partial line1, unless it's very long
eoln1 = logtext.find('\n')
if eoln1 != -1 and eoln1 < 512:
logtext = logtext[eoln1+1:]
# add either trim note at top, or preface + trim note in middle
if prefacesize > (logsize - tailsize):
logtext = '...LOG TRIMMED ABOVE HERE...\n' + logtext
else:
logfile.seek(0)
preface = logfile.read(prefacesize)
eolnN = preface.rfind('\n') # drop partial line at end
if eolnN != -1:
preface = preface[:eolnN+1]
logtext = preface + '\n...LOG TRIMMED HERE...\n\n' + logtext
# unlike Help/About tabs, it doesn't help to Clock.schedule_once()
# this code: its lag is minor, and it still crashes if text too big
logfile.close()
logtext = self.truncate_long_lines(logtext) # avoid black box of death
self.ids.logfiletext.text = logtext # replace gui text (not +=)
# stop the #$@%* memory blowup!
self.purge_the_stupid_kivy_textinput_cache()
# auto-scroll to bottom left: see note above
# textinput.cursor doesn't scroll (kivy bug): use scrollview wrapping it
# stop prior scroll's animation, else auto-scroll is a no-op [1.1.0]
self.ids.logfilescroll.effect_x.velocity = 0
self.ids.logfilescroll.effect_y.velocity = 0
# auto-scroll for new text
self.ids.logfilescroll.scroll_y = 0 # content bottom on bottom edge
self.ids.logfilescroll.scroll_x = 0 # content left on left edge
# required on all text sets, phone rotations, pc width resizes
self.enable_logs_text_hscroll()
except Exception as E:
self.info_message('Cannot load logfile.\n\n' + 'Internal error: ' + str(E))
# WATCH------------------------------------------------------------
def do_logs_watch(self, picklogitems,
tailsize=(2 * 1024), secspertail=1.0, autostopticks=60):
"""
--------------------------------------------------------------------
Sample the end of a logfile once per second. This may bog
down the runing thread (though see NOTE), but is useful as
a simple status check that's more than the Main animation.
Use TAIL or OPEN after the run exits for a better look,
or TAIL for a one-time look that won't auto-update.
WATCH is disallowed if the file selection is bad, or no action
is running (there's nothing to watch; use TAIL); and is auto
cancelled if still running at action exit (else it burns CPU
and battery pointlessly thereafter if the user leaves it on).
WATCH is also disallowed if the logfile selected is not the
latest at the top (i.e., the new logfile created by the action
running). This normally won't be an issue, as the latest log
is preselected. Forcing this log sans select may be confusing.
=> this is moot in 1.2.0: auto uses current run's log
WATCH relies on the fact that script output is imediately
flushed. Flushes may add some runtime, but testing so far
says it's no worse than piping output to a file in Termux's
shell. See also errors=replace notes in TAIL callback above.
NOTE: in testing, using WATCH while a Main action is running
does NOT degrade action speed, and MAY even make it faster,
due to Android's "squeaky-wheel-gets-the-grease" scheduler.
A normally 198-sec test often took just 168 secs with WATCH...
UPDATE: WATCH has been seen to slow actions by 10% on WIndows.
====
UPDATE: very-long-running actions (e.g., 1~1.5hours) usually
work fine, but two have been seen to fail on an Android 9 Note10
when WATCH was left on: the redrawn screen looked briefly like a
bad acid trip, and the GUI soon hung/died with a black screen -
though the foreground service kept running. Per days of research
and logcats, this was caused by the GPU running out of memory it
shares with the CPU for buffers and such (not the GPU's own memory).
The logcat after errors had 100s (or 1000s) of lines like this:
... 9581 W Adreno-GSL: <sharedmem_gpuobj_alloc:2706>:
sharedmem_gpumem_alloc: mmap failed errno 12 Out of memory
... 9581 E Adreno-GSL: <gsl_memory_alloc_pure:2270>:
GSL MEM ERROR: kgsl_sharedmem_alloc ioctl failed.
... 9581 I python : (PPUS) In text hscroll enabler
The best guesses are that Kivy's TextInput leaks memory when text
is reset frequently; and/or there is a glitch in the GPU driver
(or Android graphics-memory manager) which sometimes fails to get
a clue and run garbage collection. It's happened in Unity too...
This is also intermittent and unreproducible, unfortunately: two
1.5-hour WATCHes the next day worked fine with the same device,
actions, and content. [Later: TAIL was seen to explode shared
graphics memory on each tap too, and more reiably crashed the GUI.]
A handful of coding fixes were tried here, including forced python
gc.collect(); logfiletext.canvas.clear(); setting .text='' to clear;
disabling all text-handling code; scheduling the set with Clock;
the private textinput._textinput_clear_cache(); and tweaking the
timer callback - don't save ref, move to top-level func to avoid
closure, and use schedule_once vs schedule_interval(). [Callback
is moot for TAIL which still grows shared graphics memory on each
tap, and doesn't explain why the Fold4 doesn't have the issue].
None had any effect on graphics memory growth. Per Android Studio's
Profile attached to the running app, memory kept increasing linearly
on each WATCH tick, but leveled off at 1~2G when no error occurred.
Each TAIL seemed to spike up graphics memory too, though this isn't
perpetual like WATCH. And Kivy's TextInput source code is about as
convoluted as it gets: it's all opaque graphics ops at the bottom.
The only things that DID make an impact were turning off WATCH (this
somehow triggers a graphics-memory reclaim); and replacing live
file text with a few static lines (this doesn't blow up memory
badly, presumably because the text size was below some threshold).
Given that actions won't run 1.5 hours for most users' content (that
was for 200G, 160k files, and 14k subfolders), this seems unlikely to
be much of an issue in practice, and can generally be cured by running
the action again (or letting the service finish). And really, after
working around a truckload of Kivy bugs, this seems one too far.
Hence, this is a preliminary PUNT, with the following tweaks:
- Set android:largeHeap="true in the Android manifest. This won't
help if the GPU/other isn't getting a low-memory signal, or the
java heap that this mod increases is moot for native/Kivy code.
[UPDATE: dropped this - it may increase chances of a process kill.
That would apply more to a cached GUI process than a foreground-
service process, but it risks one kill to try heading off another:
developer.android.com/topic/performance/memory-overview#SwitchingApps
Also: TAIL gm growth rapidly hits device ram size; more won't help!]
- Scale down the sizes (tailsize) of both WATCH and TAIL text. WATCH
was cut from 3k to 1k (you could see only 1/3 of it without very fast
scrolls anyhow); and TAIL's default was changed from 32k to 10k (just
the end matters, and scrolls to the top were tedious as it was).
[UPDATE: WATCH was bumped up to 2k after the solution ahead, but TAIL
stayed at 10k because 32k takes more time/space and is generally TMI.]
- WATCH will now be turned off automatically at 60 ticks (seconds).
There's no reason to leave it on perpetually (battery use alone is
a compelling argument), and it's not worth a GUI hang; the Main tab's
animation should be enough for status, and WATCH can be restarted.
[UPDATE: this was kept despite solution below: minimize battery draw,
and CPU use for other apps in general.]
- The prior two mods will be applied on all platforms, even though
problems were only seen on Android; the scope is unclear, and
bugs that can't be recreated or fixed merit extra caution.
[UPDATE: fast TAIL taps later proved to be a reliable crasher,
but only if used atypically - the cache cleared itself in time.]
- It _may_ help to add android:hardwareAccelerated="false" to the
Android manifest, but this is tbd and unknown (and unverifiable!).
buildozer/p4a set this to "true" automatically, so maybe not.
[NO: this had no effect on Note10 or Note20U (and required manual
manifest template edits - can't have both true and false for attr).]
Go with these for 1.0.0 and wait to see if any bug reports come in.
If they do, WATCH could be further neutered to automatically turn
off even sooner (10 seconds?). But at some point, this probably
is a red herring; time to gui death may be > time to battery empty...
====
UPDATE: the above did not help on Note9, Note20U, or S23U: all explode
graphics memory for TAIL taps rapidly until it hits device's available
memory, then freeze/crash - along with other apps open on the machine.
=> BUT not the fold4: uses 'other' memory, and no memory explosion
=> WHY? - it's not because hardware acceleration graphics is off
Final tries:
- Recoded to remove/buid/add TextInput widget dynamically:
NO EFFECT, and made scrolling and colore config difficult
- Recoded as Label: worked on macOS, FAILED on Android, with logcat:
I python : (PPUS) logsize: 765339
I python : diffall 3.3 starting
I python : -------------------
I python : Exception: Unable to allocate memory for texture (size is -1949184896)
I python : Exception ignored in: 'kivy.graphics.texture.Texture.allocate'
I python : Traceback (most recent call last):
I python : File "/.../kivy/core/window/__init__.py", line 1627, in on_draw
I python : Exception: Unable to allocate memory for texture (size is -1949184896)
F DEBUG : .../kivy/core/text/_text_sdl2.so (offset 0x4000)
====
**SOLVED**
See purge_the_stupid_kivy_textinput_cache() for the fix. Called both here
and in TAIL when logfile text display is reset to new content
TextInput STUPIDLY saves prior displays' state in a kivy.cache.Cache, with
a 60-second timeout. That's why only WATCH triggered the issue before:
you have to tap TAIL in rapid succession to cause an issue. Simply purging
the Cache before or after each text reset clears enough memory to make this
a non-issue: memory in the profiler spikes up momentarily, but immediately
goes back down to a reasonable level.
Why in the world this isn't called out explicitly with a disable option in
the widget is anyone's guess. Sans manual purge, it makes Kivy text all but
unusable on most Android devices except for trivial static display...
The Fold4's diff remains an unsolved mystery; an opengl version skew?
====
NOTE: [1.2.0] Now ensures that new logfiles are not auto-pruned when
the device's clock is far in the past, but time changes may make
them not appear at the top of the sorted file list; cannot WATCH,
because this currently checks that watched file is top/latest
(and more likely to try to watch a truly older file than this).
INTERIM [1.2.0] Resolve the preceding by changing the old-file
test to simply issue a popup (with toast on Android) to alert the
user, instead of prohibiting WATCH altogether. This allows users
to scroll down to older name and still watch run if time changes.
UPDATE [1.2.0] In hindsight, it's weird to require a logfile pick
in the GUI for WATCH, when it only makes sense if the current run's
file is piked, and the app already knows what that is! HENCE, 1.2.0
redesignd this to save the known logfile only at action start, and
use it in WATCH instead of GUI picks. This elinates odd error checks
for picks, and obviates the issue of newest logfile != current run
(and users don't have to root around to find it). TAIL and OPEN
still use the GUI pick, but WATCH and EXPLORE do not.
TBD [1.2.0] This could also auto select and scroll to the run's
logfile used, but this is perilous in Kivy (see tab switch!), and
why do here only and not for tab switch? In the end, run's log will
always be at list top, except for rare and temporary time changes.
--------------------------------------------------------------------
"""
def ontimer(dt):
if self.logs_timer_counter == autostopticks:
offmsg = 'WATCH was turned off to save resources: press WATCH to restart.'
self.ids.logfiletext.text = offmsg
self.ids.logfilescroll.scroll_x = 0 # scroll to left: 1 line
self.enable_logs_text_hscroll() # user can hscroll if need
self.ids.logfilewatch.state = 'normal' # turn off watch btn auto
endwatch() # and end current watch loop
else:
self.logs_timer_counter += 1
logsize = os.path.getsize(logfilepath) # current size, func scope
logfile = open(logfilepath, 'r', encoding='utf8', errors='replace')
logfile.seek(max(0, logsize - tailsize))
logtext = logfile.read()
logfile.close()
# drop partial line1, unless it's very long
eoln1 = logtext.find('\n')
if eoln1 != -1 and eoln1 < 512:
logtext = logtext[eoln1+1:]
logtext = self.truncate_long_lines(logtext) # avoid black box of death
self.ids.logfiletext.text = logtext # replace gui text (not +=)
# stop the #$@%* memory blowup!
self.purge_the_stupid_kivy_textinput_cache()
# auto-scroll to bottom (and left on exit): see [1.1.0] note in TAIL
# enable-hscroll is required on text sets, rotations, width resizes
# stop prior scroll's animation, else auto-scroll is a no-op [1.1.0]
self.ids.logfilescroll.effect_x.velocity = 0
self.ids.logfilescroll.effect_y.velocity = 0
# auto-scroll to bottom (only: user may want to see end of paths)
self.ids.logfilescroll.scroll_y = 0 # scroll to end (but not left!)
self.enable_logs_text_hscroll() # user can hscroll long lines
if not self.script_running: # ended since last ontimer()
self.ids.logfilescroll.scroll_x = 0 # scroll to left (only) now
self.ids.logfilewatch.state = 'normal' # turn off watch btn auto
endwatch() # and end current watch loop
def startwatch():
# nit: open and explore could be left on
for logbtnid in ('logfiletail', 'logfileopen', 'logfileexplore'):
self.ids[logbtnid].disabled = True
# start auto-rescheculed timer loop
self.logs_timer_counter = 0
self.logs_timer_callback = ontimer # keep a ref (moot)
self.clockevent = Clock.schedule_interval(ontimer, secspertail)
def endwatch():
# reenable other actions
for logbtnid in ('logfiletail', 'logfileopen', 'logfileexplore'):
self.ids[logbtnid].disabled = False
# end auto-rescheduled timer loop
if not hasattr(self, 'clockevent'): # if btn stuck on (it happens)
pass # ignore: toggle may bounce
#trace('Watch is on unexpectedly')
else:
self.clockevent.cancel() # end timer loop
del self.clockevent # remove event hook
def latest_logfile_selected(logfilepath):
# NO LONGER USED
# never called if no logfiles in list, but test just in case
# [1.2.0] this test is bogus if device's date/time is askew...
return (self.latest_logs and
self.latest_logs[0] == logfilepath) # both have self.logfile_path
#-------------------------------------
# do_logs_watch: on watch button press
#-------------------------------------
trace('toggle state:', self.ids.logfilewatch.state)
if self.ids.logfilewatch.state == 'down': # toggled up=>down
# [1.2.0] use run's logfile, not gui pick
if not self.script_running: # watch is moot
self.ids.logfilewatch.state = 'normal' # turn off auto
self.info_message('No running action to watch.', usetoast=True)
else:
logfilepath = self.current_run_logfile_path # run's, not gui pick
startwatch() # that's it; use global
elif self.ids.logfilewatch.state == 'normal': # toggled down=>up
endwatch() # end loop, leave hscroll
#-------------------------------------
# pre-1.2.0-final version (temp)
#-------------------------------------
"""
logfilepath = self.log_ops_preface(picklogitems) # in func scope
if not logfilepath: # invalid path
self.ids.logfilewatch.state = 'normal' # turn off auto
elif not self.script_running: # watch is moot
self.ids.logfilewatch.state = 'normal' # turn off auto
self.info_message('No running action to watch.', usetoast=True)
#elif not latest_logfile_selected(logfilepath): # watch is bogus
# self.ids.logfilewatch.state = 'normal' # turn off auto
# self.info_message('Cannot watch prior-run logfile.', usetoast=True)
else:
# [1.2.0] weaken to cautionary popup only: time skew
if not latest_logfile_selected(logfilepath):
self.info_message('Watching an old logfile.', usetoast=True)
startwatch()
"""
# OPEN-------------------------------------------------------------
def do_logs_open(self, picklogitems):
"""
--------------------------------------------------------------------
Open logfile in another app, to sidestep size limitations in
the kivy text widget.
This is simple on PCs, but horrifically convoluted on Android,
since Android 11 (which dropped API support for file:// URIs).
Requires a FileProvide and reams of proprietary Java and manifest
code, even though the files are in shared storage that apps can
already access. Why did Android turn so corporate and nazi?...
[1.1.0]+ (feb24): on Linux, background xdg-open via &, else it
blocks (and may hang) GUI on Windows WSL2 Linux + Ubuntu (only).
xdg-open also requires a 'sudo apt install xdg-utils' on WSL2
Ubuntu (only). The '&' is harmless in other Linux contexts.
OPEN might run explorer.exe for Windows editors (else must also
install gedit/other in WSL2), but this doesn't open in the logfiles
dir for EXPLORE (and users alreayd have to instal tcl/tk too).
"""
"""
-------------------------------------------------------------
Try #1: simple loads do not work, even with [android:largeHeap="true"]
in manifest. Files ~512k oky, but 1M+ take long to load, cannot scroll,
and crash app. Alas, this seems to be a limitation in kivy or the
TextInput widget, not android per se, but is a full stop fpr open().
# sys._debugmallocstats()
# trace(sys.getandroidapilevel())
try:
# open full with boosted heap size?
logfile = open(logfilepath, 'r', encoding='utf8') # no errors
logtext = logfile.read()
logfile.close()
self.ids.logfiletext.text = logtext
# textinput.cursor doesn't scroll (kivy bug): use scrollview wrapping it
self.ids.logfilescroll.scroll_y = 0 # content bottom on bottom edge
self.ids.logfilescroll.scroll_x = 0 # content left on left edge
# required now AND on rotations
self.enable_logs_text_hscroll()
except Exception as E:
self.info_message('Cannot open logfile: %s.' % E)
-------------------------------------------------------------
"""
"""
-------------------------------------------------------------
Try #2: Nope (a frustrated Unix last resort)...
os.chmod(popuppath, 0o777)
webbrowser.open(popuppath)
-------------------------------------------------------------
"""
"""
-------------------------------------------------------------
Try #3: almost, but triggered file:// uri exception in logcat
"exposed beyond app through Intent.getData() android.os.FileUriExposedException"
even though the file is in shared storage and already accessible to
the handler apps. Editorial: Android is fully retarded on this...
activity = self.get_android_activity()
#context = self.get_android_context(activity)
Intent = autoclass('android.content.Intent')
Uri = autoclass('android.net.Uri')
File = autoclass('java.io.File') # filenames only
intent = Intent()
intent.setAction(Intent.ACTION_VIEW)
intent.setData(Uri.fromFile(File(logfilepath)))
activity.startActivity(intent)
-------------------------------------------------------------
"""
"""
-------------------------------------------------------------
Try #4: the one that WORKED...
Using Adroid FileProvider, and 'secure' content:// uris
Other parts of this:
1) ./manifest-manual-application-postattrs.xml
<provider> def, but insert location not supported by buildozer
this must be in <application attrs>X</application>
buildozer does only <application attrs X></application>
and <application> cannot be repeated in <manifest>:
https://developer.android.com/guide/topics/manifest/manifest-intro.html#filec
2) ./fileprovider_paths.xml
<paths> folder def ref'd by <provider> in #1 (wth is this separate?)
3) ./buildozer.spec
includes #2 (but not #1), via recent additions from github
and enables androidx package, where FileProvider lives
and sets gradle_dependecy to load androidx's package (no quotes, undoc)
https://developer.android.com/jetpack/androidx/releases/core#core-1.9.0
4) ~/.../PC-Phone-USB-Sync/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a/dists/usbsync/templates/AndroidManifest.tmpl.xml
patched this file (only) to add #1 as a literal; YUCK
https://github.com/kivy/python-for-android/issues/1964 (2019...)
=> LATER: and must manually REPATCH if this goes away after changing build args or "clean" (craps)
=> LATER: and must patch in .../usbsynctrial/... branch in dists too, for trial app (!!)
Later:
- need to add androidx pkg to build, for FileProvider
- enabling androidx in bd.spec not enough - need gradle_dependency too
- cannot find core lib in repositories searched (don't use quotes)
- tried manually fetching androidx, to no avail (removed) (also triggered old .ds_store version# err)...
- removing quotes in bd.spec seemed to help, but core 1.9.0 needs api33+; use 1.8.0...
- located proper insert file: see #4 above, removed other tries
- rm junk ctrl chars on <meta-data> from copy/paste off google doc page
- ditto for something else copied off a google docs page
- an indentation error was obscured by lack of logcat output; run twice
- tried coding own FileProvider subclas in java: not required, use androidx's in manifest+autoclass()
- java subclass example has ctrl characters, and lacked a required ";"...
- <path> setting proved wonky - use undec <root-path> and full path to file
-------------------------------------------------------------
"""
"""
-------------------------------------------------------------
in java:
File imagePath = new File(Context.getFilesDir(), "my_images");
File newFile = new File(imagePath, "default_image.jpg");
Uri contentUri = getUriForFile(getContext(), "com.mydomain.fileprovider", newFile);
from jnius import JavaClass, MetaJavaClass
class MyFileProvider(JavaClass, metaclass=MetaJavaClass):
__javaclass__ = 'androidx/core/content/FileProvider'
__javaconstructor__ == (
'()V',
'(Ljava/lang/String;)V',
)
package com.quixotely;
import androidx.core.content.FileProvider;
public class MyFileProvider extends FileProvider {
public MyFileProvider() {
super(R.xml.file_paths)
}
}
-------------------------------------------------------------
"""
"""
-------------------------------------------------------------
late step: don't need Java subclass in ./src/c/l/u/.java: no diff
manifest => android:name="androidx.core.content.FileProvider"
not "com.quixotely.FileProvider" (doesn't exist in java realm)
#FP = autoclass('com.quixotely.usbsync.MyFileProvider')
last step: <path> in paths file worked with 1st, not 2nd\
<root-path name="logfiles" path="." />
<external-path name="logfiles" path="Documents/PC-Phone_USB_Sync" />
no idea why the 2nd doesn't work, but life's too short to care...
#trace('file=>', file, flush=True)
#trace('path=>', file.getAbsolutePath(), flush=True)
#trace('FP=>', FP, flush=True)
#trace('uri=>', furi)
#fp = FP()
#trace('fp()=>', fp)
TBD: also set 'text/plain' mime type with setDataAndType()?
Android docs strongly suggest that data (content URI) is
enough, and mime type is inferred from it autmatically.
-------------------------------------------------------------
"""
logfilepath = self.log_ops_preface(picklogitems)
if not logfilepath:
return
# pcs are easy
if not RunningOnAndroid:
if RunningOnWindows:
os.startfile(logfilepath)
elif RunningOnMacOS:
os.system('open "%s"' % logfilepath) # allow spaces: app name!
elif RunningOnLinux:
os.system('xdg-open "%s" &' % logfilepath) # likewise; [1.1.0]+ (feb24)
return
# android is not!
activity = self.get_android_activity()
context = self.get_android_context(activity)
Intent = autoclass('android.content.Intent')
Uri = autoclass('android.net.Uri')
File = autoclass('java.io.File') # filenames only
String = autoclass('java.lang.String') # optional
FP = autoclass('androidx.core.content.FileProvider')
# furi _must_ vary for full and trial apps: here and in manifest inserts
# see service_in_progress_state() for more; also true for service names
appsuffix = 'usbsync' if not TRIAL_VERSION else 'usbsynctrial'
file = File(logfilepath)
furi = FP.getUriForFile(context,
String('com.quixotely.%s.fileprovider' % appsuffix),
file)
intent = Intent()
intent.setAction(Intent.ACTION_VIEW)
intent.setData(furi)
intent.addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION |
Intent.FLAG_GRANT_WRITE_URI_PERMISSION)
activity.startActivity(intent) # and user picks registered handler app
# or just os.startfile() on Windows, os.system('open %s') on macos...
# EXPLORE----------------------------------------------------------
def launch_file_explorer(self, folder):
"""
Used for both logs-tab explore, and help-tab explore backups
Java:
Uri selectedUri = Uri.parse(Environment.getExternalStorageDirectory() + "/myFolder/");
Intent intent = new Intent(Intent.ACTION_VIEW);
intent.setDataAndType(selectedUri, "resource/folder");
if (intent.resolveActivityInfo(getPackageManager(), 0) != null)
{ startActivity(intent); }
else
{ // no file explorer app installed on device }
[1.1.0]+ (feb24): on Linux, background xdg-open via &, else it
blocks and may hang GUI on Windows WSL2 Linux (only); see OPEN;
"""
# pcs are easy
if not RunningOnAndroid:
if RunningOnWindows:
os.startfile(folder)
elif RunningOnMacOS:
os.system('open "%s"' % folder) # allow spaces: app name
elif RunningOnLinux:
os.system('xdg-open "%s" &' % folder) # ditto; [1.1.0]+ (feb24)
return
# android is not!
activity = self.get_android_activity()
context = self.get_android_context(activity)
Uri = autoclass('android.net.Uri')
uri = Uri.parse(folder) # assumed absolute
Intent = autoclass('android.content.Intent')
intent = Intent(Intent.ACTION_VIEW)
intent.setDataAndType(uri, 'resource/folder')
try:
activity.startActivity(intent)
except Exception as E:
self.info_message('No usable file-explorer app.', usetoast=True)
# this and resolveActivity() always None, but intent can be started anyhow
"""
if intent.resolveActivityInfo(context.getPackageManager(), 0):
activity.startActivity(intent)
else:
self.info_message('No usable file-explorer app.', usetoast=True)
"""
def do_logs_explore(self):
# no need for a gui selection here, but ensure folder access?
# tbd: seems gray - maybe it's up to explorers to grok the path?
if not self.init_logfile_folder():
return
logfilefolder = self.logfile_path # logfilepath's (if set) folder
self.launch_file_explorer(logfilefolder) # abs(<shared>/Documents/<appname>)
#=======================================================================
# LOGFILE UTILITIES
#=======================================================================
# See also maxnumlogfiles in Config tab
def current_logfiles_by_descending_name(self):
"""
--------------------------------------------------------------------
This is now used in three places: ensure same.
Sorted by descending filename; because logfiles are named per host
system's date/time, this is the same as descending creation time.
Assumes self.logfile_path has been set, exists, and is listable
(init_logfile_folder(), called before this, ensures all three),
and returns a list of full paths, not basenames (files).
--------------------------------------------------------------------
"""
logpatt = LOGFILE_NAME_PATTERN
globlogs = glob.glob(osjoin(self.logfile_path, logpatt))
sortlogs = sorted(globlogs, reverse=True)
return sortlogs
def on_tab_switch(self):
"""
--------------------------------------------------------------------
On tab switches to the Logs tab, reload its logsfilechooser so
it shows any newly-added or in-progress logs (e.g., for Wait),
iff the set of logfiles has changed (e.g., for a new run).
This might be done in make_new_logfile(), but this wouldn't
handle manual folder changes made by the user, or auto-prunes.
TabbedPanel provides current_tab, which can be monitored
for changes by [on_current_tab: <this>]. Nothing else
used here is documented (the code was largely discovered
by trace prints), and the _update_files call is _private...
Subtle: it's impossible for a pruned logfile to appear in the
Logs tab's file list, because the delete triggers a reload here
if the user switches from Main (action start+prune) to Logs tab.
Caveat [now moot]: the chooser update causes a momentary
flash in the file list whenever switching to Logs tab, but
detecting new logs via state+filesystem may be just as slow.
----
UPDATE: as an optimization, only update the chooser's file
list if new logs have arrived since last update or init.
This avoids the flash in full, except after a Main action.
Check needs to see if entire list of logfiles has changed
(not just most recent file, currlogs[-1]), and update if so.
Else, will be junk entries at end if logs have been trimmed.
UPDATE: now also forces chooser's selection to most-recent
logfile, if the folder has changed. This is a convenience;
users will almost always want latest for WATCH and TAIL (and
WATCH could simply use the latest always, but it works as is).
----
UPDATE: recoded with a custom scrolled file-selection widget.
This was initially coded to use a stock kivy FileChooser with
FileChooserListLayout, but this was a major pain in the app...
autoselect was elusive, prior selections were not cleared after
_update_files() (autoselect or not), icon-view mode didn't
show full filename, and a popup requires pointess extra taps.
See _dev-misc/todo.txt for the filechooser coding abondoned.
NIT: the current coding for the selection list assumes that all
lognames are the same size in a mono-width font. Changing this
in kivy to be more flexible seems the stuff of madness.
NIT [now moot, per UPDATE below]: the selection list doesn't
hscroll if user sets a font so big that it overflows screen.
But names are fine on a Note20U at settings already large
enough to break (former) filecooser's iconview.
(Editorial: Kivy's MVC filechooser is unneccessarily convoluted,
functionally weak, and just plain buggy. And why is there no
basic selection-list widget as in tkinter? Improve me, please.
Sizing for scrolling also seems a nonorthogonal mess; it smacks
of CSS, but with a built-in N-minute app-build delay per tweak.)
NOTE: no longer needs to forcibly cancel selection on the
logs display: TextInputNoSelect now disables selection in full.
(Formerly: despite all settings, it's still possible to select
text, and there seems no UI deselect. clear_selection() here
removes selection because it won't auto disappear on a tab,
but this doesn't fix lingering selection handles issue/bug!).
----
UPDATE: Logs' custom file toggle-button list IS now horizontally
scrolled too (in addition to vertically). This matters for very
large fonts and/or very narrow displys (see Fold4 cover screen).
The code to make this happen is arcane in the extreme; but works.
----
UPDATE: this now also works around the lag in Help and About text
display, by simply setting the text on the next GUI cycle. This
is not documented in kivy docs, and the advice on the web is way
off base (it suggest recyvleviews, and there was an optimization
for this in kivy that largely failed). Without this workaround,
there is a momentary twitch on all platforms, and a horrific delay
of ~8 seconds for 15k Help on Windows that's all kivy widget time,
not file read time (a bug!). Note that this has to be done at
tab-switch time; the lags occurred when text was set on startup,
and Clock scheduling the sets at startup didn't help. In general,
the code here would have to be used whenever about to display text.
Note: this now sets help/about text just once, but this seems moot:
if reset on tab reopens, there is no load pause or jump to top-of-text.
Perhaps TextInput is smart enough to detect 'is' equality on resets?
Note [1.1.0]: delayed text-content sets are now also employed to
avoid glitchy lag in info-message popups on some slower phones;
see info_message().
----
UPDATE [1.1.0]: for Logs tab, this now avoids a Kivy race-condition
crash for ToggleButtons, by setting all .group to None before removing
from children lists. This ensures that Kivy's TB button-press code
won't try to .remove a TB's weakref from its group list after an
async garbage-collection callback has already .removed them.
Without this workaround, a Logs toggle-button press can crash the
GUI with an uncaught exc for Kivy's .remove. A rare Kivy bug to be
sure, but it was witnessed once on macOS, and perhaps aggrevated by
action threading on PCs: async GC callbacks may be more likely when
thread switching is intense. Deleted TBs may be GCd at any time,
including in the middle of the TB button-press handler code.
Setting .group also invokes a .remove for the prior group's list
in Kivy TB code, but this won't crash because the buttons' are still
held in child lists, thereby preventing GC and async GC callbacks.
It's not fully clear why Kivy uses the GC callback to .remove from
groups in general; this is not thread safe. Kivy also builds a
bogus group for buttons changed to None, but this seems harmless.
----
UPDATE [1.1.0]: catch double and triple taps on logfile buttons,
and auto-run just TAIL for double, and TAIL+OPEN for triple. Kivy
fires double on the way to triple, so running just OPEN is stateful.
Implementation: it suffices to register for just the last button
added: all logfile buttons get the event, and the selected button's
text is fetched from the 'down' button in the action callback run.
Don't need [logfilebtn.collide_point(*touch.pos)]: any tap triggers.
Need [ToggleButton.on_touch_down(logfilebtn, touch)]: to select last.
Taps bubble through all logfile buttons (only): add trace()s to see.
ALSO [1.1.0]: less padding here on Windows and Linux else buttons
oddly large, and less spacing on all PCs in .kv else gaps too big.
ALSO ALSO [1.1.0]: prior point is superseded by new density-scaled
pixels globally - here, chooser buttons, and in .kv (see its docs).
----
Note: [1.2.0] ensures that new logfiles are not auto-pruned when
the device's clock is far in the past, but time changes may make
them not appear at the top of the sorted file list; user must find.
TBD [1.2.0]: the top of the file list is not necessarily the most
recent if host clock moved back, and auto-selecting this may not be
useful. Might set it to the last run's logfile save at run launce
(and used in WATCH), but this seems iffy and dangerous: the last
run may have had issues, and the Kivy bugs worked around in code
here makes auto select+scroll a very perilous business. As is,
top WILL be most recent the vast majority (or all) of the time.
--------------------------------------------------------------------
"""
trace('tab switch')
if self.ids.toptabs.current_tab == self.ids.helptab:
trace('help tab')
def sethelptext(dt):
self.help_message = self.stage_help_message
if not self.help_message:
Clock.schedule_once(sethelptext)
elif self.ids.toptabs.current_tab == self.ids.abouttab:
trace('about tab')
def setabouttext(dt):
self.about_message = self.stage_about_message
if not self.about_message:
Clock.schedule_once(setabouttext)
elif self.ids.toptabs.current_tab == self.ids.logstab:
trace('logs tab')
# make if needed, bail if fail
if not self.init_logfile_folder():
return
# refresh logsfilechooser iff needed
currlogs = self.current_logfiles_by_descending_name()
if currlogs != self.latest_logs:
trace('logs changed')
pickscroll = self.ids.picklogscroll # won't scroll sans size
pickitems = self.ids.picklogitems # btns on a layout in a scrollview
# avoid Kivy race-condition crash [1.1.0]
for logfilebtn in pickitems.children:
logfilebtn.group = None # else exc if GC during press
pickitems.clear_widgets() # delete lfbtns, GC now or later
# not in .kv (but would work there too)
pickitems.size_hint = (None, None)
pickitems.bind(minimum_width =pickitems.setter('width')) # set width on minwidth
pickitems.bind(minimum_height=pickitems.setter('height')) # mins calc sizes auto
#pickitems.halign = 'center' # use AnchorLayout
for path in currlogs:
# add logfile radio button to v+h scolled list
# log_ops_preface() expands basename text on Logs actions
logfilebtn = ToggleButton(text=os.path.basename(path))
logfilebtn.group = 'picklog'
logfilebtn.state = 'normal'
logfilebtn.font_name = 'RobotoMono-Regular'
logfilebtn.padding = (
# initial tweak, superseded [1.1.0]
# (24, 24) if RunningOnWindows or RunningOnLinux else (32, 32))
# scale to screen density, not platform or device [1.1.0]
(kivy.metrics.dp(14), kivy.metrics.dp(14)))
logfilebtn.allow_no_selection = False
logfilebtn.size_hint = (None, None)
logfilebtn.halign = 'center'
logfilebtn.valign = 'middle'
logfilebtn.bind(texture_size=logfilebtn.setter('size')) # set size on texture_size
pickitems.add_widget(logfilebtn)
if currlogs: # possibly []
numlogs = len(currlogs)
latest = pickitems.children[-1] # default adds at ix=0 (!)
latest.state = 'down' # auto select latest log
# and scroll to selected file at top - needed despite rebuild!
pickscroll.scroll_y = 1 # content top on scroll's top edge
# register touch callback for multitaps [1.1.0]
# see update above for more on this implementation
def on_logfile_down(touch):
ToggleButton.on_touch_down(logfilebtn, touch) # last added
if (touch.is_double_tap and
self.ids.logfilewatch.state == 'normal'):
# auto TAIL, if not WATCH
self.do_logs_tail(pickitems)
if touch.is_triple_tap:
# auto OPEN, always
self.do_logs_open(pickitems)
return False # bubble to other logfile buttons
logfilebtn.on_touch_down = on_logfile_down # last added only
self.latest_logs = currlogs # save for next switch to Logs
# DEFUNCT (if currlogs)
# set pickscroll.width = pickitems.width
# else scroll width=100px default, but disables hscroll
# no longer needed or correct - see the UPDATE below
"""
pickitems.bind(minimum_width=pickscroll.setter('width'))
"""
# enables filelist hscroll, but only after a window resize,
# and so misses both startup and apply for huge font sizes;
# punt - small windows and large fonts that hclip Log's
# filelis are mostly unusable in lots of other ways too;
#
# UPDATE: equivalent now done in .kv file, where bindings
# nested in expressions are easier, and fire automatically;
# this also obviates the manual scroll-width binding above,
# which kept this alternaive from working for font sizes;
"""
def set_pickscroll_width(root, rootwidth=...):
pickscroll.width = min(pickitems.minimum_width, rootwidth)
self.bind(width=set_pickscroll_width)
"""
# END DEFUNCT
def init_logfile_folder(self):
"""
--------------------------------------------------------------------
Make the logfile folder if it doesn't yet exist, and save its
path in instance and GUI (a kivy property). Also verify access,
but don't request permission here; seems too much nagging.
Return True iff logfile folder exists and is listable, whether
made anew or not. Called for all Androids on each Main action run,
as well as on switches to the Logs tab in the GUI (but no longer
on startup, as the next para explains).
Documents/ works sans permission in Android 11+, but the os.mkdir()
won't work on pre-11 sans runtime permission - which users can deny,
and which won't yet be given when the nonmodal permission dialog is
posted in App.on_start() (where this call failed on pre-11 Androids).
It's unlikely that logfile access failures reflect storage permission
issues (Documents/ on Android is always accessible?), but prompt for
a restart just in case. This seems moot for PCs, including macOS.
UPDATE: maybe not - macOS pops up permission dialogs on first access.
UPDATE [1.1.0]+ tweak the message to clarify that an app restart is
not needed on macOS (probably), and qualify the opening as "on Android"
because it won't apply anywhere else. See reset_main_picker_path().
UPDATES [1.1.0]+ (feb24): auto-create the ~/Documents folder if it
does not already exist; required for Windows WSL2 Linux + Ubuntu (only);
could also use os.makedirs() to make all path parts, but it's overkill;
--------------------------------------------------------------------
"""
docspath = self.storage_path_documents()
if not osexists(docspath):
try:
os.mkdir(docspath) # [1.1.0]+ (feb24)
except:
pass # and fail below
self.logfile_path = osjoin(docspath, LOGS_SUBFOLDER) # used multiple places
if not osexists(self.logfile_path):
try:
os.mkdir(self.logfile_path)
except:
self.info_message('Cannot make logfiles folder.'
'\n\n'
'If you just granted storage permission on Android, '
'please try restarting this app now.' # [1.1.0]+
'\n\n'
'Otherwise, retry after ensuring that '
'drives have been mounted and recognized.'
+
(
'' if not RunningOnMacOS else # [1.1.0]+
'\n\n'
'On macOS, you may be able to simply rerun the last '
'request if you just handled the permission popup.'
),
usetoast=False)
return False
# osexists() doesn't imply accessibility
try:
os.listdir(self.logfile_path) # assume won't hang [1.1.0]
except:
self.info_message('Cannot access logfiles folder.'
'\n\n'
'If you just granted storage permission on Android, '
'please try restarting this app now.' # [1.1.0]+
'\n\n'
'Otherwise, retry after ensuring that '
'drives have been mounted and recognized.'
+
(
'' if not RunningOnMacOS else # [1.1.0]+
'\n\n'
'On macOS, you may be able to simply rerun the last '
'request if you just handled the permission popup.'
),
usetoast=False)
return False
return True # it's there, and we can access it
def make_new_logfile(self, opname):
"""
--------------------------------------------------------------------
On Main action runs. Ensure log folder each time: may fail
on pre-11 android if no runtime permission granted by user.
New logfile is closed+reponened in subprocess action modes.
----
UPDATE [1.2.0]: ensure that a newly created logfile is at top of
the logfiles list ordered by descending names. Else the new log
won't be considered newest by later sorts: alert the user, bail.
Logiles are labeled according to the host machine's date and
time, and then sorted by name for display and pruning. This
can fail badly if the host's date/time is wrong. In one case,
a device that couldn't connect to a network to get the latest
time thought it was many years in the past (2020 vs 2024) - new
logfiles were oldest by name, and runs pruned their own new
logfiles, leaving no record of anything happening (even though
actions ran and potentially made changes).
The glitch requires threads (services reopen the logfile post
prune), but threads are always used on PCs and Android 8 (the
erring device in question), and display order will still be
wrong for services.
TO ADDRESS, check that the new logfile is latest by a name sort
(which suffices to detect name-label/creation-time skew), and alert
the user and cancel the action if not true to avoid stealth runs.
The sort here mimics both on_tab_switch() and auto_clean_logfiles(),
which handle displays and prunes.
The alternative is to always sort by file modtimes instead of names,
but this seems risky: it requires additional filesystem access (that
may hang), and modtimes are notoriously wonky on some filesystems.
And really, this WOULD NOT HELP: if the system thinks it's 5 years ago,
that's what it will stamp in new file creation/modification times!
Logfile names reflect creation time the same as filesystem times.
----
EXCEPT [1.2.0]... the preceding scheme renders that app unusable
for one hour once a year, after clocks fall back for DST changes.
It might also preclude usage for awhile after flying to a new
timezone that's earier by more than the duration of the flight.
Labeling files with UTC time (or sorting by same in the filesystem)
would help with DST and timezone skew, but this would make logfile
names not very useful to human readers, and would NOT HELP if the
device's time or timezone settings were wrong - the labels or sorts
would still be off (in the fail, 2020 is still < 2024, UTC or not).
[Python's time.strftime() defaults to local time. It also accepts
a UTC time.gmtime(), which normally returns non-decreasing values,
but won't be as meaningful to users (GMT time in logfile names?),
and can still return a lower value than a previous call if the system
clock has been set back between the two calls. Py's time.monotonic()
cannot go backwards and is not affected by system clock updates, but
only the difference in #seconds between the results of two calls is
valid, and only during a given single process - #seconds since start
would be meaningless in filenames, and this resets to 0 on each run!]
HENCE, punt on the warning+cancel here, but use simpler option
of just avoiding a prune of the new logfile. This will still
display the new file at the end of the list (and may prune it
on next run, and botches WATCH - till final 1.2.0, at least),
but the run will at least work and keep its logfile long enough
for users to spot device-time issue. In the case of DST and
timezone changes, the new logfile likely won't be at the end of
the sorted list, so it won't be removed on next run.
ALSO issue a caution popup (which may be covered by summary popup
for fast runs, but this is very rare) when order is askew by DST,
timezone, or bad clock. AND ensured that Mergeall itself was NOT
pruning the new run's __bkp__ sync back folder for an old date/time
(an identical issue, formerly avoided by pruning before creating).
--------------------------------------------------------------------
"""
if not self.init_logfile_folder():
return False
logdir = str(self.logfile_path)
logname = time.strftime('date%y%m%d-time%H%M%S--%%s.txt') % opname
logpath = osjoin(logdir, logname)
try:
logfile = open(logpath, 'w', encoding='utf8')
except:
self.info_message('Cannot create logfile.'
'\n\n'
'Run cancelled - please resolve and rerun.',
usetoast=False)
return False
# [1.2.0] is new logfile at top of name-ordered descending sort?
currlogs = self.current_logfiles_by_descending_name() # >= 1: new log made
if logname != os.path.basename(currlogs[0]):
self.info_message('Caution: logfile naming-order skew.'
'\n\n'
'Logfiles are named with your device\'s date/time when '
'they are created. This run\'s logfile is not newest by '
'name, which may reflect a normal time change, but can '
'also be a symptom of your device\'s date/time being '
'improperly set too far in the past.'
'\n\n'
'This causes new logfiles not to show up at the top in '
'the Logs tab. They may also be pruned sooner than '
'expected, leaving no record of actions\' changes. '
'SYNC backups with dates in the past may similarly '
'be pruned too soon, making UNDO impossible.'
'\n\n'
'Please verify that date/time is okay on your device.',
usetoast=False)
return (logfile, logpath) # worked, okay, proceed with action
def auto_clean_logfiles(self, newlogpath):
"""
--------------------------------------------------------------------
Just before each Main-tab action run, prune the logfiles folder
to keep only the most recent 'maxlogs' logfiles to avoid taking
up excess space. There's one log for every Main action run.
Never called unless logfile folder is known to exist.
Don't do on startup: would not handle new logs in session.
Don't prune by age only: may not SYNC for weeks/months.
Retains maxlogs (not -1), because the next log has been added.
For user ease: always log, but auto-clean folder this way.
TO backup folders are auto-cleaning too, but are pruned by
mergeall.py on SYNCs having any backups (and configured there).
Note: it's impossible for a pruned logfile to appear in the Logs
tab's file list, because that list will be reloaded with the
current set when the user switches from Main to Logs tabs in the
GUI (the logfiles change triggers a reload: see on_tab_switch()).
----
[1.2.0] This now filters out the new run's logfile, newlogpath,
from the list of files to prune: it may be there because the host's
date/time was set far in the past. Else, an action would run but
leave no record of changes in logfile, and summary popup can't
extract (for threads; services reopen log). See make_new_logfile().
Nit: this will throw off the number of logs kept by +1, but this is
temp and rare. We could simply run this _before_ make_new_logfile(),
so that the new long will never wind up in prunes even if date/time is
old (this is how Mergeall skirts the issue for its __bkp__ backup
folder prunes); but that would have to dup much init_logfile_folder()
code, else calling it here could trigger its popups twice in caller.
Alternative coding - sets lose name-order sort:
prunes = list(set(prunes) - set([newlogpath]))
prunes.sort(reverse=True)
--------------------------------------------------------------------
"""
maxlogs = int(self.ids.maxnumlogfiles.text) # from Config tab
logroot = str(self.logfile_path) # set on run startup
if osexists(logroot): # none on first run
# oldest last, per sort, known to be listable here
currlogs = self.current_logfiles_by_descending_name() # full paths
prunes = currlogs[maxlogs:]
# [1.2.0] drop new logfile path if in prunes: date/time skew
if newlogpath in prunes:
trace('filtering newlogpath from prunes')
prunes = [p for p in prunes if p != newlogpath]
anyfailed = False
for prunee in prunes: # globs have full paths
trace('Pruning log:', prunee) # normally 0 or 1, unless failed
try:
os.remove(prunee) # skip Windows FWP: path short
except Exception:
anyfailed = True
trace('This prunee failed, but pruning continued')
trace('%s %s' % (sys.exc_info()[0], sys.exc_info()[1]))
def enable_logs_text_hscroll(self, *args):
"""
--------------------------------------------------------------------
Workaround for one of about a dozen kivy bugs: hscroll lost on rotate.
TBD: is this still needed (originally for Main scrolled run output)?
YES: logs text still has no hscroll both initially and after rotations.
NEW: call on logfile load AND scroll's on_height, NOT text's on_text.
NIT: Window.on_rotate() may have worked too (and runs less often?).
BUT: Window.on_rotate() seems to never fire despite docs; see on_start().
Workaround: on both scrollview's on_height and textinput's on_text,
enable horizontal+vertical scrolling of program-run output in Main tab.
This output may also be trimmed due to size limits in kivy TextInput:
users should view full logs in Logs Open, opened with Android Intents.
UPDATE: no longer called on textinput's on_text, only on Logs-tab text
scrollview height (from .kv) and Logs-tab text sets (from this .py).
HACK: there is no documented way to make a TextInput scroll horizontally
when [do_wrap: False]. The TI scrolls only vertically by itself, and
its width must be set to enable horiz scrolls when wrapped in a ScrollView.
But there seems no way to do this short of the following brittle code; WHY?
ALSO must run after phone rotation, else hscroll lost: Scrollview.on_height.
----
UPDATE: this code now also tries to works around tabs bar rarely not drawing
after a phone rotate to landscape on Android 12L. This failed in on_start(),
which docs it further; since code here is called on rotates too (but only for
the Logs tab: text-scroll size), do tabs-bar redraw here. This is overkill
in calls for text sets, but might work, and is harmless and unnoticeable.
UPDATE: PUNT - this didn't apply to Help/About/Config tabs, and didn't help
for the Logs tab with either coding below; and it's too rare to fret...
UPDATE [1.1.0]: a variation of this was also tried--and also failed--in the
.kv file's code, to Clock.schedule a do_layout() for the TabbedPanel from
GridLayout, or ask_update() for the whole root display, on panel width
changes on Android (which catch phone rotations). Is this an SDL2 glitch?
UPDATE [1.1.0]: yes, it is - this reflects a bug in SLD2, Kivy, or Samsung
Android, which drops the event having correct screen size sans Android 12L+
taskbar. See the window section of App.on_start() ahead for more info.
The app can't work around this, because no correct display size exists.
UPDATE [1.1.0]: after globally converting absolute to display-scaled pixels
using dp(), an absolute 50 for padding here was not enough on phones, and
longest lines were truncated on the right. This seems to be because the new
[dp(12), dp(12)] padding in the .kv when scaled was > the old absolute 50 here.
The padding's former absolute [24, 24] jived with absolute 50, and the size
setting here must manually include the .kv's padding (the 50 wasn't an rpad
fudge factor, it was to accomodate all horizontal padding). Changing to
dp(25) here foo fixed the truncation on all devices, without leaving space
on the right. LESSON: once you go scaled, you must go all the way!...
--------------------------------------------------------------------
"""
trace('In text hscroll enabler')
widthcalc = self.ids.logfilescroll.width
for linelabel in self.ids.logfiletext._lines_labels:
widthcalc = max(widthcalc, linelabel.width + kivy.metrics.dp(25)) # 50=pad
self.ids.logfiletext.width = widthcalc # [1.1.0]
# work around tabs bar rarely not drawing after rotate ~landscape
##self.ids.toptabs.canvas.ask_update()
##self.canvas.ask_update() # whole screen
"""
DEFUNCT - now in-tab
def do_logfile_path(self):
picker = LogfilePickDialog(
pickstart=str(self.logfile_path),
onpick=self.do_log_path_pick,
oncancel=self.dismiss_popup)
self._popups.push(Popup(title='Choose Logfile to View',
content=picker,
auto_dismiss=False, # ignore taps outside
size_hint=(0.9, 0.9)))
self._popups.open()
def do_log_path_pick(self, logfilepath, popupcontent):
""
On Pick, close the picker dialog, and spawn an Android intent
to view the logfile. TBD: could just scroll the file's text
in this tab, but Kivy TextInput might hang for large files.
""
if not os.path.isfile(popuppath):
self.info_message('Logfile path is not a file.')
return
else:
# TBD: or try showing in a kivy textinput widget?
# TBD: intent seems same as webbrowser.open()/open_web_page()?
"""
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
# APP CLASS
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
class PCPhoneUSBSync(App):
"""
=======================================================
Auto builds Main root widget in ./pcphoneusbsync.kv.
Locates file by class name (alt: pass as a string).
Could be built manually, but avoids add_widget, etc.
Or manual + build(), or string + Builder.load_string().
Subtlety: 'root' refers to the top-level Main() widget,
event though it's built auto from .kv file, not here.
=======================================================
"""
# bound to Widget font_size in .kv: changes update ALL widgets
dynamic_font_size = NumericProperty(FONT_SIZE_DEFAULT) # set before build()
"""UNUSED
Works, but not required:
def build(self):
self.dynamic_font_size = FONT_SIZE_DEFAULT
return super().build()
UNUSED"""
"""UNUSED
Does not auto recenter, whether set size before or after super().build(),
or in class def before App.build() is ever called; order also doesn't
impact the momentary resize twitch in the gui, and doing this at the
top of this script doesn't either; see startup_gui_and_config_tweaks();
def build(self):
result = super().build()
if RunningOnWindows or RunningOnLinux:
lx = 150 if RunningOnLinux else 0 # was 100 (oct23)
Window.size = (kivy.metrics.dp(748+lx), kivy.metrics.dp(612+lx)) # (w, h), [1.1.0]+
return result
#return super().build()
if RunningOnWindows or RunningOnLinux:
lx = 150 if RunningOnLinux else 0 # was 100 (oct23)
Window.size = (kivy.metrics.dp(748+lx), kivy.metrics.dp(612+lx)) # (w, h), [1.1.0]+
UNUSED"""
def on_start(self):
"""
Run by kivy .run() after intitialization - after build()
has built the GUI, but before app has started running.
"""
global guiroot
super().on_start()
root = self.root # GUI root: for attrs in Main()
guiroot = root # export to global for trace()
# once a global, now an attr
root.script_running = NOTHING_RUNNING
# divide runs in logcat, requires script_running
trace('@' * 80)
trace('app.on_start')
# allow Popups to nest
root._popups = _PopupStack()
# SPLASH
# close splash for Windows and Linux frozen executables now: window open;
# must do this _before_ missing-files error popup, else splash covers it!
if not RunningOnAndroid:
close_pc_splash_screen()
# VITALS
# warn the user if supporting runtime files are absent:
# likely moved exe out of unzip folder (in docs, but...)
vitals = (HELP_FILE, ABOUT_FILE, TERMS_OF_USE_FILE)
vitals+= (MERGEALL_PATH, 'usbsync-anim.gif') # not .png
vitals+= ('usbsync-pc',) if not RunningOnAndroid else ()
if any(not osexists(v) for v in vitals):
root.info_message(
'Fatal error: missing app files.'
'\n\n'
'This app cannot run because it cannot find one '
'or more of its supporting files. Did you move the '
'executable without its folder? That doesn\'t work.'
'\n\n'
'The app will close itself to avoid exceptions. Please '
'see the User Guide\'s App-Packages coverage for more info.',
usetoast=False,
nextop=self.shutdown_app)
return # show message and die on Okay
# ICONS
# pcs seem to need help on these beyond pyinstaller build
# the .ico doesn't work on Windows with Kivy (cause unknown)
if not RunningOnAndroid:
self.title = APPNAME # no pc trial
pciconname = ('icon-round.png' if RunningOnWindows else # not .ico
'icon-round.icns' if RunningOnMacOS else
'icon-round.gif')
pciconpath = osjoin('usbsync-pc', pciconname)
self.icon = pciconpath # rounded-corners version
Window.icon = pciconpath # else kivy icon displayed
# PRELIMS
# preset configs+paths
root.load_persisted_settings()
# tweak gui+configs for platforms, etc.
root.startup_gui_and_config_tweaks()
# fetch+update run counter: macos perms, terms of use, trial version
# [1.1.0]+ (oct23) catch permission errors: e.g., Windows unzips to
# C:\Program Files; could specialize msg per platform, but very rare;
# don't continue on excs, else may always repeat startup message;
try:
runnum = self.get_run_counter(root)
except:
root.info_message(
'Fatal error: cannot write admin files.'
'\n\n'
'This app cannot run because it does not have permission '
'to write its admin files. On Windows, this is often caused '
'by unzipping the package in C:\\Program Files; please use '
'any other location. In other cases, please ensure that the '
'app\'s data folder is available and allows reads and writes.'
'\n\n'
'The app will close itself to avoid exceptions. Please '
'see the User Guide\'s App-Packages coverage for more info.',
usetoast=False,
nextop=self.shutdown_app)
return # show message and die on Okay
# PERMISSIONS
# ask user to enable macOS full disk access on first run only
if RunningOnMacOS and runnum == 1:
root.macos_ask_for_full_disk_access()
# verify|request on each Android startup: all files access 11+, else old style
if RunningOnAndroid:
root.get_storage_permission()
# everything past this may not (yet) have storage permission (nonmodal
# system dialogs); check/ask again on folder-chooser popups and restarts
# TBD: ask for battery-optimzatin disable here (see wake locs ahead)?
# LOGS, UNDO
# fails here for pre-11 android: runs before permission dialog dismissed
# root.init_logfile_folder()
# init latest logs now for tab switches
root.latest_logs = None # trigger first load on first Logs tab view
# initially allowed, till sync sans bkp or undo sans __undoable__
root.undo_verboten = False
# TBD: make __undoable__ in TO/__bkp__s if countruns==1 (first run)?
# but can't know if true: pre-app syncs may or may not have saved bkps
# HELP+ABOUT
# load terms-of-use file, for About and popup
tou_file = open(TERMS_OF_USE_FILE, 'r', encoding='utf8') # [1.3.0] mods
tou_text = tou_file.read()
tou_file.close()
# fill text displayed in tabs
with open(HELP_FILE, 'r', encoding='utf8') as help_file: # [1.3.0] mods
root.stage_help_message = help_file.read()
with open(ABOUT_FILE, 'r', encoding='utf8') as about_file: # [1.3.0] mods
# because it's easy to forget to change the file
trialnote = ('' if not TRIAL_VERSION else
'\nFree trial: %d of %d opens\n' % (runnum, TRIAL_APP_OPENS))
platname = ('Android' if RunningOnAndroid else
'Windows' if RunningOnWindows else # no cigwin: a windows exe
'macOS' if RunningOnMacOS else
'Linux' if RunningOnLinux else '(Unknown)')
"""
# no: this is runtime, not buildtime!...
monthyear = datetime.datetime.now().strftime('%B %Y') # or .date.today()
copyright = '2023'
dateparts = monthyear.split()
if len(dateparts) == 2 and dateparts[1] != '2023':
copyright = '2023-%s' % dateparts[1]
"""
"""
# no: these are both now from new PUBDATE at top of file [1.1.0]
# change me manually on rebuilds (or mod build scripts?)
monthyear = 'May 2023'
copyright = '2023'
"""
# add underlines to uppers
tou_text2 = ''.join([line + '\n' if not line.isupper() else
line + '\n' + '=' * len(line) + '\n'
for line in tou_text.splitlines()])
# not yet seen, but...
androidpower = ('' if not RunningOnAndroid else
'\n\nANDROID NOTE\n'
'============\n\n'
'Some phones kill apps to save battery power more '
'aggressively than they perhaps should. This app runs its '
'long-running actions as Android foreground services by default '
'to avoid this risk, and no automatic kills have been seen or '
'reported, even for multi-hour runs and very large content. '
'Should you ever see an action killed anyhow, please disable '
'battery optimizations for this app in your Apps Settings. '
'This may also avoid temporary app stops on some phones.\n') # [1.1.0]
about_message = (about_file.read() %
dict(VERSION = VERSION,
PUBDATE = PUBDATE, # [1.1.0]
PLATFORM = platname,
COPYYEAR = PUBDATE.split()[1], # [1.1.0]
TRIALNOTE = trialnote, # [1.3.0] now unused
TERMSOFUSE = tou_text2,
ANDROIDPWR = androidpower))
root.stage_about_message = about_message
# DEFUNCT
# kivy TextInput is wildly slow (pauses everywhere, Help ~8secs on Windows)
# (no effect) predraw Help tab to avoid half-second resize on first open?
# root.ids.helptab.canvas.ask_update()
# (also no effect) with or without a kivy property (and scrolls poor) punt!
# root.ids.helptext.text = root.help_message
# FIXED: lag was eliminated by scheduling text set => see on_tab_switch()
# WINDOW+WAKELOCK
# on android, pan iff needed so input target is above onscreen keyboard;
# default '' = don't pan or resize; also 'pan' (always), 'resize' (no-op)
Window.softinput_mode = 'below_target'
# catch Back button: route user to Main tab, or close if in Main tab;
# also ignore Back/Escape if any dialog open: Back too easy in android
Window.bind(on_keyboard=self.catch_back_button)
# warn user if tries to close app while Main action is running;
# called on 'back' when app foreground, but NOT on recents swipe;
Window.bind(on_request_close=self.on_request_close)
#-----------------------------------------------------------------------------
# [1.1.0] Android wake locks, 2.0
#
# Summary: allow screen timeouts (and savers) to be toggled by the user.
# This requires a manual partial wakelock on Android - p4a's screen-only
# wakelock must be disabled, and didn't ensure cpu on screen-off anyhow.
#
# On Android, when "wakelock" is enabled in buildozer.spec, it enables
# kivy=>buildozer=>p4a's wakelock, which is just SCREEN_BRIGHT_WAKE_LOCK.
# This has has been deprecated since API level 15 (2011's Android 4!),
# and only keeps the screen on while the app has focus (it's released
# and required by p4a in on_pause/resume). This doesn't keep the CPU on
# in sleep ("doze") state; is cancelled during screen off for power-button
# presses; and prevents screen timeouts while in foreground, which can
# drain battery if the app is left in foreground sans the power button
# and user Settings push screen-off out for a long time.
#
# Ref: https://developer.android.com/reference/android/os/
# PowerManager#SCREEN_BRIGHT_WAKE_LOCK
#
# Code:
# in kivy's p4a code on github, at pythonforandroid/bootstraps/sdl2/
# build/src/main/java/org/kivy/android/PythonActivity.java.
#
# In addition, kivy exposes a call in SDL2 for enabling/disabling
# screensavers - which weirdly means screen timeouts on Android. It
# defaults to off (disabled), and on Android uses FLAG_KEEP_SCREEN_ON,
# which is an alternative that seems preferred in Android's docs.
# On Android, this seems *WHOLLY REDUNDANT* with p4a's "wakelock".
# Enabling either option keeps the screen on; both disable timeouts;
# and neither ensure that the CPU will keep running when the screen
# is turned off (if they do, it's undocumented and buried deep in
# Android's source code). The only diff: sdl2's flavor is switchable
# and also applies to PC screensavers; p4a's is fixed and Android only.
#
# Proabably, forcibly keeping the screen with either scheme ensures
# CPU by preventing the device from going into sleep mode (Android's
# "doze" circus). But this is just a side-effect; won't work as soon
# as the user presses the power button, or switches to another app
# that allows the screen to timeout; and drains the user's battery
# badly - and needlessly vs partial wake locks.
#
# Code:
# in sdl2's code on github, at: SDL/android-project/app/src/main/
# java/org/libsdl/app/SDLActivity.java
#
# That said, actions in testing finish with the screen off in both
# thread and service mode, and Android's docs on this are pathetic
# See "doze" and "lightweight doze"; sleep is as opaque as it can be
# (and may even vary by device!), and battery-optimization stops make
# it worse. This story has grown convoluted over Android's history,
# and it's unclear how wakelocks play with doze, foreground services,
# and spawned threads and processes. But wakelocks probably don't hurt.
#
# FOR 1.1.0: turn off p4a's keep-screen-on wakelock in builds; allow
# sdl2's alternative to be enabed/disabled with a toggle in Confgs tab;
# and manually acquire/release an Android PARTIAL WAKE LOCK (PWL) via
# pyjnius when actions start/exit. This reinstates screen timeouts
# on demand, and PWL is the only API tool documented to do anything for
# CPU usage directly. The wakelock is held only while and action runs,
# but is not released in on_pause cecause it must appply to app switches
# too. This requires create/acquire/release code in .py (here), plus
# manifest permission in buildozer.spec; "wakelock" in buildozer.spec
# must be disabled else screen-on is constant and unswitchable.
#
# Ref: https://developer.android.com/training/scheduling/.
# Ref: https://developer.android.com/reference/android/os/
# PowerManager#PARTIAL_WAKE_LOCK
#
# There remains battery-optimization kill+stop concerns. Running a
# foreground service elevates priority to make kills unlikely, but
# stops are more gray. This might be addressed by running an intent
# ACTION_REQUEST_IGNORE_BATTERY_OPTIMIZATIONS to direct users to
# disable power optimizations, but this has Play-store consequences
# that are unclear and undocumented (yes, a pattern), and we don't
# need the drama. For now, a note in the About tab and other docs
# will have to suffice; TBD.
#-----------------------------------------------------------------------------
# init in startup_gui_and_config_tweaks() from persisted settings
# Window.allow_screensaver = True # True = PC screensaver + Android screen timout
if RunningOnAndroid:
# create a wake lock to use later [1.1.0]
activity = root.get_android_activity()
Context = autoclass('android.content.Context')
PowerManager = autoclass('android.os.PowerManager')
# works on both activity and context (!)
powermgr = activity.getSystemService(Context.POWER_SERVICE)
wakelock = powermgr.newWakeLock(
PowerManager.PARTIAL_WAKE_LOCK,
'usbsync%s::actionrunning' % ('trial' if TRIAL_VERSION else ''))
root.wakelock = wakelock # and .aquire()/.release() around action runs
#-----------------------------------------------------------------------------
# [1.1.0] Rotate glitch, reloaded (docs only - cannot work around)
#
# Try to work around ya kivy bug: on an Android 12L fold4, the tabs bar
# is occasionally not redrawn when rotating to landscape; force a redraw?
# Alas, Window on_rotate() never fires; try in enable hscroll, then etc...
#
# Fail: Window.bind(on_rotate=lambda: self.root.ids.toptabs.canvas.ask_update())
# Fail: Window.on_rotate = lambda rotation: print('window on_rotate')
# Fail: self.root_window.update_viewport(); also tried code in .kv file
#
# LATER [1.1.0] findings (and punt: can't change sdl2 events sans more info):
# the event catchers below often get fired twice when going to landscape
# WHEN IT WORKS: the event fires first for full screen size, and then again
# for full sceen size minus Android 12L taskbar size. But it fires just
# once for full size WHEN TABS BAR IS MIA; it never fires twice when going
# to portrait; and often fires just once with correct size for landscape.
# Either SDL2 or Kivy is dropping correct-display-size events, or Android
# (or Samsung's flavor of it) sends them badly. Keyboard opens don't fix
# this because no correct size (sans Android 12L+ taskbar) ever exists.
# The app cannot work around this for the same reason: display size wrong.
"""
From logcat:
# okay: tabs bar drawn in landscape (with visible glitch)
(PPUS) window on_resize, (PPUS) sizes: (2176, 1749) [2176, 1749]
(PPUS) window on_resize, (PPUS) sizes: (2176, 1623) [2176, 1623]
(PPUS) window redraw
(PPUS) window redraw
# okay: tabs bar drawn in portrait
(PPUS) window on_resize, (PPUS) sizes: (1812, 1968) [1812, 1968]
(PPUS) window redraw
# ==>FAIL<==: tabs bar missing in landscape
(PPUS) window on_resize, (PPUS) sizes: (2176, 1749) [2176, 1749]
(PPUS) window redraw
# okay: tabs bar drawn in portrait
(PPUS) window on_resize, (PPUS) sizes: (1812, 1968) [1812, 1968]
(PPUS) window redraw: (1812, 1968)
# okay: tabs bar drawn in landscape
(PPUS) window on_resize, (PPUS) sizes: (2176, 1623) [2176, 1623]
(PPUS) window redraw: (2176, 1623)
"""
# Event-catcher alts:
# works, but futile: system_size wrong due to dropped events
"""
def force_redraw_in_two_seconds():
trace('sizes:', self.root_window.size, self.root_window.system_size)
if True or RunningOnAndroid:
Clock.schedule_once(
lambda dt: (trace('window redraw:', self.root_window.size),
self.root_window.update_viewport()),
2.0)
"""
# possibly overkill: fires for PC height changes too
"""
def window_resize(width, height):
trace('window on_resize', end=', ')
real_resize(width, height)
force_redraw_in_two_seconds()
real_resize = self.root_window.on_resize
Window.on_resize = window_resize
"""
# occasionally fires twice for android rotates - but see above
"""
def window_width(*args):
trace('window width', end=', ')
force_redraw_in_two_seconds()
Window.bind(width=window_width)
"""
#-----------------------------------------------------------------------------
# TERMS OF USE
# popup terms-of-use blurb on first run (also added to About tab above);
# on all platforms; onerous, but makers need to protect themselves too;
# popup ~last so appears on top (not displayed if trial counter at max);
if runnum == 1:
root.info_message('Welcome to %s. Before you get started, please '
'take a moment to read and agree to the following.'
'\n\n\n'
'%s'
'\n\n'
'For reference, you can find a copy of all these '
'statements in the app\'s About tab.'
% (APPNAME_FULL_OR_TRIAL, tou_text), usetoast=False)
# TRIAL
# check for max-runs expired in full-but-trial version of app (no ads!!)
# truly last, so android perms dialog won't cover (run #1 popups never do)
if RunningOnAndroid:
self.handle_trial_counter(runnum, root)
"""DEAD
# get mergeall's config module for maxbkps checks+changes (code+object)
# value imported here may be trounced by the value loaded into Config
# this is complicated: see root.update_mergeall_configs_maxbackups()
# UPDATE: must mod backups, not mergeall_configs - backups uses "from"
# see update_mergeall_configs_maxbackups(), now run at SYNC time only
os.chdir('mergeall') # in app-install (unzip) folder
sys.path.append(os.getcwd()) # .pyc not text, '.' fails here
trace('ma=>', os.getcwd())
trace('ma=>', os.listdir(os.getcwd()))
import mergeall_configs # threads run in subfolder, same process
assert hasattr(mergeall_configs, 'MAXBACKUPS') # services run in separate process
root.mergeall_configs = mergeall_configs
os.chdir('..')
sys.path.pop()
DEAD"""
def check_for_running_service(self):
"""
--------------------------------------------------------------------
[NO LONGER USED; ref in startup_gui_and_config_tweaks()]
On restarts, detect running service and reset in-progress
state, else user could start a new action, which breaks
the actions model and yields odd states. The alternative
is to kill a service along with the app, but this doesn't
seem very well (if at all) suported in kivy/p4a today (see
on_request_close()). Threads die with the app, and need
no in-progress check. Services die only on Recents upswipe.
Nit: getRunningServices() is marked as deprecated in Android
docs, but says it will stick around for same-app use. If
it ever goes away, a sentinel file may work just as well,
sans unexpected kills by the o/s. A field in the service
class may work too (though it's java, and ditto on kills).
UPDATE: PUNT - this works on all Androids supported, but it
is based on a deprecated API call that may go away any time,
and is inherently inaccurate and error prone: the service
might exit in the time between the check here and the restore
of GUI state and message receiver, leaving the app waiting
for the end of a service that's gone. Though rare, the app
could go in-progress forever, requiring restart; ugly, that.
A service-breadcrumb file may address API deprecation, but
wouldn't be accurate if the service was foribly killed by
Android, and suffers from the same timing issue as the API.
TO DO BETTER, the Back button handler now prohibits app close
if a Main action (service or thread) is in progress. This
is applied for all Main actions, not just update actions:
show and diff are okay to kill, but would still require
in-progress detection on restarts. This policy is also
better for threads, which are killed outright on app close.
Users can still kill the app along with its running action
by an upswipe in Recents; this cannot be intercepted in
kivy/p4a (afaik), but seems a reasonable last-resort op.
--------------------------------------------------------------------
"""
"""
In java:
private boolean isMyServiceRunning(Class<?> serviceClass) {
ActivityManager manager = (ActivityManager) getSystemService(Context.ACTIVITY_SERVICE);
for (RunningServiceInfo service : manager.getRunningServices(Integer.MAX_VALUE)) {
if (serviceClass.getName().equals(service.service.getClassName())) {
return true;
}
}
return false;
}
"""
"""DEFUNCT
root = self.root
activity = root.get_android_activity()
context = root.get_android_context(activity)
appsuffix = 'usbsync' if not TRIAL_VERSION else 'usbsynctrial'
svcname = 'com.quixotely.%s.ServiceRunscript' % appsuffix
manager = activity.getSystemService(context.ACTIVITY_SERVICE)
services = manager.getRunningServices(2 ** 31 - 1)
for i in range(services.size()):
service = services.get(i)
if service.service.getClassName() == svcname:
svcrunning = True
break
else:
svcrunning = False
if svcrunning:
# reset gui, restart msg receiver
trace('service still running')
# read service breadcrumb file in app-install folder
svcrecord = open(SERVICE_BREADCRUMB, 'rb')
opname, cmdargs = pickle.load(svcrecord)
svcrecord.close()
if opname != 'UNDO':
postop = lambda: None
else:
# restore postscr callback for undos
# assume undo all conditions still hold
latestbkp = cmdargs[0]
postop = lambda: root.discard_latest_backup_folder(latestbkp)
root.service_in_progress_state(opname, ?, postop) # and hope didn't just stop!
DEFUNCT"""
def catch_back_button(self, window, key, scancode, codpoint, modifier):
"""
--------------------------------------------------------------------
On keypress 27 - Escape, which also means Back button on
Android - either goto the Main tab automatically, or allow
Back to close the app if it's already on the Main tab.
This is a bit custom, but tabs feel a lot like windows,
and it's otherwise too easy to exit app unintentionally.
This might also force pause mode in Main, but convoluted.
UPDATE: on Android, this now also prohibits app kills if
any Main action is running as either a service or a thread.
Users can still kill both forcibly by Recents swipes instead
(which seems uncatchable in the kivy/p4a realm), but a Back
app kill would kill a thread outright, and service-running
detection on restarts is deprecated + inaccurate (see above).
Related: it might be nice to change tabs on SWIPE gestures
automatically, but they are far too low-level to integrate:
https://duckduckgo.com/?q=kivy+change+screen+on+swipe+gesture
Kivy's a powerful framework, but could use more docs and dev.
UPDATE: on Android (Back) and PCs (Escape), now ignores the
event if any modal dialog is open, rather than going to Main
or closing the app. Back is way too easy to hit on Android.
This jives with recent auto_dismiss=False mod in all popups.
--------------------------------------------------------------------
"""
if key != 27:
return False # continue processing key
else:
trace('back button press')
root = self.root
if not root._popups.empty():
# ignore if any modal dialog open
return True # end key processing
elif root.ids.toptabs.current_tab != root.ids.maintab:
# switch to Main tab and end key processing
root.ids.toptabs.switch_to(root.ids.maintab, do_scroll=True)
return True
else:
# Back in Main tab
if not root.script_running:
# close now - sans hanging!
self.shutdown_app()
else:
# prohibit close, on Android or PC
message = (
# [1.1.0] this seems TMI in hindsight, and didn't say
# what is and is not safe to close: rewrite in full;
#
#'This app cannot be closed while a Main-tab '
#'action is running.'
#'\n\n' '
#'This ensures your content\'s safety. An app close '
#'would kill the action outright if running as a '
#'thread, and it\'s impossible for restarts to '
#'accurately detect in-progress services. In both '
#'cases, content updates might fail.'
'You are trying to close this app while a Main-tab '
'action is running.'
'\n\n'
'It\'s safe to close this app while a non-update SHOW '
'or DIFF is running, but closing while any other '
'actions are running may stop updates in progress '
'perilously and require action reruns.'
'\n\n'
'If you really want to kill both this app and its '
'running action, please instead use either a swipe '
'in your Recents display on Android, or the window '
'close button on PCs.')
root.info_message(message, usetoast=False)
return True # end key processing
def shutdown_app(self, popupcontent=None):
"""
The quest for a hang-free app close on Android.
Outside PCs, now used by the Back handler above.
See on_request+close below for the full saga.
popupcontent is optional and ignored: may be called
from a popup or not, but this is shutdown, right?
"""
trace('os._exit: bye')
os._exit(0) # this feels wrong, but it works...
"""
hangware...
trace('app.stop')
self.stop() # end the Android activity (sort of)
trace('window.close')
Window.close() # just in case (same if on_stop); reached
trace('bye') # doesn't reach this on a hang...
"""
def on_request_close(self, *args, **kargs):
"""
--------------------------------------------------------------------
Warn user if tries to close app while Main action is running.
Called on 'back' when app foreground, but NOT on recents swipe.
HANGS: kivy closes were also seen to hang sporadically in initial
Android app dev, with no useful info in the log ("Leaving application
in progress..." was displayed, but the GUI didn't close).
As a workaround, force the close with either self.stop(), sys.exit(),
or os._exit(). These may dork up kivy state (and sys.exit may be
caught/disabled by kivy), but the app run is over anyhow. The
atexit module's handlers are not run for os._exit, so this seems
the last resort.
----
UPDATE: on Android, self.stop() ends the Android activity, and seems
to have fixed the issue. Still tbd: nothing can be done if a Recents
swipe closes hang (short of kivy code spelunking...).
UPDATE: back-button exits have been seen to hang again, but it may
be due to low-memory on phone, and a reset cleared this up. Just
in case, add a Window.close() after App.stop() - per the web, this
seems to have been a perennial issue in kivy sans framework fix...
----
NOPE: still hanging sporadically, even after reset, and even without
running any actions. Fallback to the nuclear os_exit() option; which
is probably bad, but closes are a convoluted mess, routed though kivy,
p4a, sdl2, and java bootstrap code. os._exit() skips the drama...
Known downside: os._exit precludes shrink-back-into-app-screen-icon
animation, though hangs are worse, and this anim only fires if never
navigate away to another app (else the app screen is no longer under
the app for the animation; Android state is wack with poobrain).
NEVERMIND: no other p4a or buildozer app built does this animation...
NOTE: for all exits tried here, a foreground service will keep running
if the app is killed by a Back button; the required manifest code to
prevent this seems unsupported, though an intent might work (tdb);
https://duckduckgo.com/?t=ffab&q=android+kill+foreground+service
----
UPDATE: this will now be used on PCs only, when a GUI close request
is triggered by the user. On Android, Back-button closes are now
prohibited during Main action runs (above), and never reach this code.
Curiously, this is still invoked on Android for Recents-swipe closes,
which trigger on_pause, on_resume, this, and on_stop when a service
is running (and the first three of these with no service - sometimes),
but cannot be disabled by a True return here in any event. App close
in kivy/p4a could be a bit more coherent than it currently seems.
----
NOTE [1.1.0]: after thousands of runs, a source-code run launched by
PyEdit was seen to hang for ~6 seconds on close-button press, when a
chooser popup was open, on macOS, after PyEdit itself was killed,
and when the GUI was allowed to remain minimized for days.
The hang was due to Kivy logger error and looping: it filled a 3.7M
and 92k-line logfile with ""[WARNING] stderr: --- Logging error ---"
mostly (after "[INFO ] Base: Leaving application in progress...")
before the app finally closed. This hang is harmless and cannot be
recreated (and thus was never verified as fixed), but it's a puzzler.
Logging might be disabled everywhere with KIVY_NO_FILELOG and/or
KIVY_NO_CONSOLELOG (see top of file), but they're useful on errors.
Though now 5 months and many details ago, the comment near the top
of this docstring strongly suggests that a logger loop was not the
cause of earlier, initial Kivy hangs on Android which elicited the
os._exit() nuclear option. Then again, Kivy is wonky business...
ALSO NOTE: in the possibly related column, the try_any_nohang()
timeout tool now sets daemon to True in the threads it spawns,
so they will not delay app exit (they formerly inherited False).
This may matter if these threads hang forever, but it's unikely;
the False still shouldn't have sent the Kivy logger into overdrive;
and the Main-tab action thread initially used on Android was already
using daemon=True and hence should have exited on action finish.
--------------------------------------------------------------------
"""
trace('app request close')
root = self.root # GUI root: Main() attrs
if RunningOnAndroid: # not reached on Back (intercepted above)
return False # but is reached for Recents swipe (oddly!)
if not root.script_running:
# pick your closer...
#return False # do normal kivy close - hangs sporadically
#self.stop() # end the Android activity (really)
#sys.exit() # nuclear option: exit now (really!)
#os._exit(0) # nuclear option: exit now (really!!)
self.shutdown_app()
else:
message = (
# [1.1.0] never get here anymore if RunningOnAndroid,
# and a service is auto killed on android swipes anyhow;
# also note the show and diff are okay to close early
# here and for android above (could be detected auto);
#
#'Unless the action is running as an Android service with '
#'notifications, '
'You are trying to close this app while a Main-tab '
'action is running.'
'\n\n'
'Non-update actions SHOW and DIFF can be stopped safely. '
'For all other actions, any updates in progress will be '
'stopped perilously and may require action reruns.'
'\n\n\n'
'Continue with close?')
confirmer = ConfirmDialog(message=message,
onyes=self.shutdown_app, # trigger normal close
onno=root.dismiss_popup) # don't close app
root._popups.push(Popup(title='Confirm App Close',
content=confirmer,
auto_dismiss=False, # ignore taps outside
size_hint=(0.9, 0.8))) # wide for phones? (moot)
root._popups.open()
return True # don't close the window (on PCs, at least)
def get_run_counter(self, root):
"""
--------------------------------------------------------------------
Read+update the run-cunter file. Was in handle_trial_counter(),
but this is now used on all platforms, and not just for the
Android trial version: also used to prompt on macOS for perms,
and show terms of use on run #1 everywhere.
Also refactored to ensure trial-expired is the _last_ on_start()
popup created, so it's not covered by others: the macOS and
t-o-u popups won't cover it because they appear only in run #1
(and the trial expire is later), but the Android storage perms
popup may appear too if the user never granted permissions.
Since there's no way out of trial-expired, perm dialog is moot.
UPDATE [1.1.0]+ (oct23): the write() here can easily fail if users
unzip the Windows package in C:\Program Files. Catch this at call
and issue a popup with tips, and close. User should unzip anywhere
else; permission mods (once) and run-as-admin (always) are harder.
This was per an Oct23 user report; the same user was also trying
to sync over MTP, and both issues merited Tips in App-Packages.
The write() may fail for other reasons too, but it's wildly rare.
--------------------------------------------------------------------
"""
# get run counter on all plaforms
apppriv = root.storage_path_app_private() or ''
countpath = osjoin(apppriv, RUNCOUNT_FILE)
if not osexists(countpath):
countruns = 1
else:
countfile = open(countpath, 'r') # in app-private, not install
countruns = int(countfile.read()) + 1
countfile.close()
# save new run counter on all platforms
countfile = open(countpath, 'w')
countfile.write(str(countruns))
countfile.close()
trace('#Runs:', countruns)
return countruns # for mac perms, terms of use, trial version
def handle_trial_counter(self, countruns, root):
"""
--------------------------------------------------------------------
Update run counter: may be used for trial version (not ads or
feature limits!). Counter _may_ begin at 2 on android, after
all-file-access dance (but hasn't since early codings). This
scheme has evolved; skip ahead to CONCLUSION for the outcome.
INITIAL: yes, this scheme can be subverted by simply uninstalling
and reinstalling the app every N runs (the counter is in app-private
which is wiped), but if people want to work that hard just to avoid
paying $2.99, we really don't want to take money from them anyhow.
Also okay: a sideloaded trial version on website, and/or just an
annoying popup sans full close. This may wind up free, and won't
net much money given Android's ecosystem (free or ignored, Google
takes 15%, and $5 is expensive). Let's not be douchey here, eh?
----
UPDATE: a sideloaded trial version on website is not okay - a simple
remame+unzip+untar exposes the full source code, and, alas, this is
not a domain where trust makes a whole lot of sense.
UPDATE: app-private storage may be wiped by an uninstall+reinstall,
but always is by 'Clear app data' in Settings=>Apps=>App=>Storage.
This seems a misfeature (it zaps settings, preferences, and more in
lots of apps), and will erase this app's run-counter file. This is
still inconvenient after every N runs, but easier than a reinstall.
To close this loophole, store the run counter in '.' (CWD), which
is the app's install folder, not its wipeable app-private space.
If this doesn't work, a '.ppus' hidden file might be used as a
doppleganger in app-specific ("external"), which is wiped only on
app uninstall. CWD seems less intrusive, and is already known to
be writeable for changes to mergeall_configs.py.
----
UPDATE: nope... app install folder is wiped of additions too, on a
simple clear-app-data (google's security obsession is not right).
Fall back on checking for a redundant .hiddenfile in the app-specific
storage folder if the counter is missing in app-private. app-spec
is cleared on unistalls too (sans the keep-data toggle on 10+),
but not on a user clear-app-data (?), and is not intrusive - users
can store content there, but it's likely to become inaccessible soon.
Clear-app-data also resets the nested mergeall/mergeall_configs.py,
which means it's not just top-level, and a nested file won't subvert
it. By contrast (and INCONSISTENTLY!), both app update/reinstall
and app uninstall with keep-app-data clicked do not wipe either app
private/install folders or mergeall_configs.py mods. Yes, really.
And any solution that relies on Google's cloud for auto backups and
requires users to be signed in to a Google account is right out!
See update_mergeall_configs_maxbackups() for a related fiasco.
----
UPDATE: nope... app-specific ("external") storage _is_ cleared on
Settings' clear-app-data too. It's also cleared on app uninstall,
but only if users don't click the 10+ option to retain app storage.
Clear-app-data has no such options. The uninstall toggle must be
meant only for uninstall/reinstall sequences, though it's not clear
why clear-app-data exists at all; why not just uninstall? So: the
only remaining options are shared storage (which is indeed intrusive),
or network/cloud schemes which are nonstarters here.
CONCLUSION: Settings' clear-app data wipes app-private, app-install,
and app-specific; unsintall retains all if insstructed to do so only.
The only remaining options are shared storage (which is indeed
intrusive), or network/cloud schemes which are nonstarters here.
So: this app went with a full trial version with a limited number runs,
implemented with a run counter in app-private storage - as initially.
This can be subverted by both Settings' clear-app-data and (less easily)
app reinstall, but both are annoying enough to serve as incentives to buy
the full app. The popup nag is just a polite request for reimbursment.
Decent users will do so; we can't help people who are just cheap; and
we shouldn't take money from people who truly can't afford $2.99.
Update: see get_run_counter(), split off from here because its scope
is now broader, and order matters: trial popup opens last == on top.
----
UPDATE: after a year, the app may be made free after all, because
Play is a payola, and its users demand apps for free.
- Play: apps have no visibility without paying for promo ads
- Users: apps won't be installed when seen unless they are free
- Apps: there's no way to fund promo ads sans in-app revenue
Because this app will not do in-app ads or subscriptions, and
cannot do in-app purchases for tech reasons (unsupported in
Kivy and too much pyjnius code), this is a game-ended for paid.
The only way to increase installs is $0 (free) sales that are
promoted with ads; that's a complete loss, and just not worth it.
Alas, Google point-of-control model on Android+Play wins; today...
--------------------------------------------------------------------
"""
"""UNUSED
def _teststorages():
# NOT USED IN PRODUCTION
apppriv = root.storage_path_app_private() or ''
appinstall = os.getcwd()
appspec = root.storage_path_app_specific()
stores = (('a-s:', appspec), ('a-p:', apppriv), ('a-i:', appinstall))
for (name, path) in stores:
trace(name, path)
trace('===>', os.listdir(path))
sprobe = osjoin(appspec, RUNCOUNT_FILE)
pprobe = osjoin(apppriv, RUNCOUNT_FILE)
iprobe = osjoin(appinstall, RUNCOUNT_FILE)
probes = (('pprobe', pprobe), ('iprobe', iprobe), ('sprobe', sprobe))
for (name, path) in probes:
if not osexists(path):
trace(name, 'absent')
open(path, 'w').write('1')
else:
prior = open(path).read()
trace(name, 'present:', prior)
open(path, 'w').write(str(int(prior) + 1))
if False and RunningOnAndroid: _teststorages(); return
UNUSED"""
#
# check for trial expired on android only and in trial version only;
# no run1 popups (permission, terms-of-use) will cover this: #runs > 1
# and android permissions is opened first if user never granted perms;
#
if RunningOnAndroid and TRIAL_VERSION and countruns > TRIAL_APP_OPENS:
message = ('Sorry, but you\'ve now opened this app as many times as '
'this trial version allows.'
'\n\n'
'To continue using this app, please visit its Play-store '
'page with the link below to get its full non-trial version. '
'The non-trial version works the same, but does not limit the '
'number of opens (and has no "Trial" in its name or icon).'
'\n\n'
'Thanks for trying this app, and thanks for your support.')
trialended = TrialEndedDialog(message=message,
oncancel=self.stop)
root._popups.push(Popup(title='Trial Ended',
content=trialended, # self.stop can be run here
size_hint=(1, 1))) # may make auto_dismiss moot
# alt: popup.bind(on_dismiss=lambda: True)
root._popups.top().auto_dismiss = False # the Back btn kills app too
root._popups.open() # now all: auto_dismiss=False
# and there's no way out but app exit (with a possible visit to Play)
#"""
def on_pause(self):
"""
--------------------------------------------------------------------
On user leaving the running app by picking another in Recents
or 12L taskbar. Not triggered when up-swipe to kill app in
Recents (maybe: see note above). Foreground service keeps
running, notification erased on exit. Thread keeps running,
no notification ever posted. Service stopped only on Recents
upswipe, not Back stop/kill.
Don't stop self.breceiver if app gets messages while paused?
TDB - docs say do this, but this seems a bit misleading.
Can the message be missed if sent while app pause?
YES, though toast doesn't appear except in the app.
ALSO, there seems no ill effect for not stop/restarting.
Polling for a signal file in '.' or app-private won't
detect exit while paused, but will when resumed; there's
really no reason for a service then aprt from notifications
(threads run on pause too), though this seems ample cause.
--------------------------------------------------------------------
"""
trace('app.on_pause')
if self.root.script_running == SERVICE_RUNNING:
#self.root.breceiver.stop()?
pass
return True # True=pause, don't stop now (default? docs seem askew)
def on_resume(self):
"""
--------------------------------------------------------------------
On user returning to the paused app by picking it in Recents
or 12L taskbar. Not triggered if repick app in either Recents
or Apps after killing it with a Back button in Main or Recents
upswipe: on_start comes after both, with unpacking app msgs.
Don't stop breceiver if app gets messages while paused?
--------------------------------------------------------------------
"""
trace('app.on_resume')
if self.root.script_running == SERVICE_RUNNING:
#self.root.breceiver.start()?
pass
def on_stop(self):
trace('app.on_stop')
#"""
if __name__ == '__main__':
PCPhoneUSBSync().run()