"""
===============================================================================
CALENDAR-FOLDER STORAGE (part of Frigcal)

About the SAF...

This app's code is bifurcated for Android's permissions rules:

• On all PCs, it uses the usual, portable, and interoperable filepaths 
  and POSIX calls for the calendars folder.  No permissions are required.

• On Android, it uses Android's proprietary SAF picker for the calendars 
  folder, which grants required permission but also forces the app to use 
  content URIs and the arguably tortuous SAF API for storage processing 
  (except where file descriptors can be had at the bottom of the code).

Using SAF for the calendars folder on Android roughly _doubles_ the amount
of storage code just for the sake of Android and obscures this app's
implementation substantially.  Moreover, SAF's API is wildly convoluted 
compared to POSIX/filepath equivalents (see ahead), and the SAF is widely 
known to be very slow, which matters even more on phones.

By contrast, quixotely.com's PPUS sync app uses Android's All Files Access
(a.k.a. MANAGE_EXTERNAL_FILES) permission because it requires both broad
storage access and optimal storage speed.  This permission also allows a 
single version of code based on POSIX and filepaths to be used on all 
platforms, thereby avoiding code bifurcation and its inherent costs.
There is some extra code for Android storage (e.g., asks) but it's minor.

In the end, All Files Access was deemed overkill here because, unlike PPUS,
this app uses only one folder in storage, and its speed requirements are
negligible (loads and saves are rare events, not core behavior).  Also 
unlike PPUS, using All Files Access would likely preclude distributing this
app on the Play store, which is officially a downside, though the rules 
that Play regularly imposes on developers may qualify as a negative too.

The SAF itself may have been an interesting idea in theory - accessing 
both physical and non-physical storage the same way - but it doesn't 
seem to survive contact with reality.  It's bloated, functionally limited,
and almost non-sensical at times (again, see the code ahead); there is 
no compelling reason to support non-physical storage in some apps (clouds 
and networks can be a Bad Thing, especially for private calendar data); 
and POSIX/filepath code is both much simpler and works on PCs too. 

As it stands, developers must choose between supporting Android or PCs, 
or budgeting for double the effort to support both.  The former is a 
choice that naturally benefits monopolistic agendas and the latter is 
an added cost that Google seems unlikely to reimburse.  Can't we do better?

Related: main.py's storage_path_app_private() is a platform-specific path
for storage of admin files, not calendars, which also varies by PC kind.
===============================================================================
"""


from common import *                                       # shared collector module
from hangless import try_any_nohang, try_listdir_nohang    # hang-free IO calls

# ICS-file parser/generator (3rd-party lib, uses pytz, both shipped with app)
try:
    import icalendar
except:
    # should never happen, but...
    print('Required and included icalendar or pytz packages not present')
    sys.exit(1)

from icsfiletools import *                            # global tables, icalendar objects
from icalendar.parser import unescape_char            # currently unused
from icalendar.parser import escape_char, foldline    # for Unicode file text




#==============================================================================
# 🗓  ICS events data structures: internal format of calendars and events
#==============================================================================



"""
-------------------------------------------------------------------------------
Global calendar and event data structures

ICS calendar files:

- Are loaded into the following in-memory data structures on calendar loads,
  which occur on app startup and calendar-folder switches

- Are converted back to iCalendar text from this form and written to ICS 
  files on main-menu Save, which is prompted on app exit and folder switches 

The GUI is drawn from the in-memory form, and event updates are applied to 
it until user Save requests.  The in-memory form is three global tables:

- CalendarsTable embeds icalendar-library Calendar objects created by 
  ICS-file parses and used for ICS-file generations.  Calendar objects 
  may also be created by main-menu New.  

- CalendarsDirty designates user changes and is separate to minimize code
  changes in earlier Frigcal vesions.  

- EventsTable is an index that combines all ICS files' events and uses 
  two-level dictionary table nesting: [date][uid].  

All three are largely built in storage.py's calendar parsing.  Structure:

CalendarsTable =               # Parsed file's raw calendar data (all data)
    {icsfilename:              # key: basename of .ics file parsed (with '.ics')
         icalendar.cal         # icalendar.cal Calendar object, from parser or New   
    }

CalendarsDirty =
    {icsfilename:              # key: basename of .ics file parsed
         Boolean               # True=calendar changed since startup or latest save
    }

EventsTable =                  # Union of all events in all ics files (indexed data)
    {(m, d, y):                # key: date object for event start (and display) date
        {uid:                  # key: event's required unique id, for fast deletes
            (.calendar,        # ics file basename, link to icalendar in CalendarsTable
             .summary,         # event summary text (aka Title in GUI)
             .description,     # event description text (aka Note in GUI)
             .category,        # category (just 1/first used)
             .uid,             # event's required unique id, for easy access
             .orderby,         # creation (or else mod) date: display order sort
             .veventobj)       # reference link to vevent in icalendar.cal object
        }
    }

More about EventsTable:

Events from all calendars are stored in a 2-level table, indexed by date and
unique id (UID) for quick access to both displayed and parsed data.   Because
events in the table are stamped with their calendar name, there is no need to
segregate by a 3rd calendar dimension (an initial design) - the GUI displays
the _union_ of all calendar files' events, and a single combined table still
supports display grouping by calendar, via sorts.

Events in the table also have by-name links back to their parsed icalendar object
for fast in-place updates in memory (files are regenerated only on Save), and GUI
month windows keep UID-indexed tables of displayed events for quick deletion.
This model assumes events have UIDs, but the iCalendar standard requires these. 

Subtlety: IN-PLACE CHANGE to mutable objects shared by multiple references is
crucial in this model.  EventsTable-index objects are referenced from registered
GUI event handlers, and there is just one copy of each CalendarsTable parsed
icalendar array, referenced from index objects and used at file regeneration time.
-------------------------------------------------------------------------------
"""



# loaded calendar/event tables, per above (singletons)
# changed in memory, transferred to/from external ICS files on demand
# reset in-place before loads for Settings-screen folder changes 

CalendarsTable = {}    # {icsfilebasename: icalendar.cal.Calendar()}

CalendarsDirty = {}    # {icsfilebasename: Boolean} [1.1]

EventsTable = {}       # {Edate(): {uid: EventData()} ]



class Edate:

    """
    -------------------------------------------------------
    Encapsulate and name event date numbers.
    != ViewDate, used to manage GUI day in monthgui.py.
    -------------------------------------------------------
    """

    def __init__(self, month, day, year):
        self.month = month                          # a namedtuple may do some of this, but not all
        self.day = day
        self.year = year

    @staticmethod                                   # copy datetime.{date, datetime} object attrs 
    def from_datetime(dateobject):                  # staticmethod optional if 3.X class calls only
        return Edate(dateobject.month, dateobject.day, dateobject.year)

    def to_datetime_date(self):
        return datetime.date(**self.__dict__)       # copy attrs to a datetime.date object

    def as_tuple(self):
        return (self.month, self.day, self.year)

    def as_string(self):
        return '%02d/%02d/%4d' % self.as_tuple()    # formatted display, 'MM/DD/YYYY'

    def as_nice_string(self): 
        # [4.0] month-labeled and locale-neutral 'mmm D, YYYY'
        # don't zero-pad day with %d: users aren't computers
        return self.to_datetime_date().strftime(f'%b {self.day}, %Y')    # 'Nov 8, 2025'

    # support use in dictionary key, comparisons, sort key
    
    def __hash__(self):
        return hash(self.as_tuple())                # dictionary key (else by addr)
    
    def __eq__(self, other):
        return self.as_tuple() == other.as_tuple()  # comparisons (else by addr) 

    def __lt__(self, other):                        # sort keys (else fails)
        """
        [1.4] order events on dates by year first (y-m-d), not month;
        but keep as_tuple() format (m-d-y): it's also used for displays;
        was originally: return self.as_tuple() < other.as_tuple()
        """
        return ((self.year,  self.month,  self.day) <
                (other.year, other.month, other.day))



class EventData:

    """
    -------------------------------------------------------
    Maximal attributes set for an event.
    Used for both EventsTable index and GUI widget values.
    -------------------------------------------------------
    """

    def __init__(self,
        uid='', 
        calendar='', summary='', description='', category='', 
        orderby='',  veventobj=None):
        
        self.uid = uid                      # event unique id, required by iCalendar std
        self.calendar = calendar            # calendar file's basename (not path)
        self.summary = summary              # for event summary in month, elsewhere
        self.description = description      # for extra text in footer, edit dialog
        self.category = category            # for colorizing event summary entries
        self.orderby = orderby              # for ordering events entries initially
        self.veventobj = veventobj          # a reference to icalender object: updates, deletes

        # or: for (name, value) in kargs.update(defaults): setattr(self, name, value)
        
    def copy(self):
        return EventData(**self.__dict___)  # use attrs dict for keyword args
    
    def __eq__(self, other):
        return self.__dict__ == other.__dict__  # comparisons (else by addr) 




#==============================================================================
# 💾  BASE CLASS -- common code for both PCs and Android
#==============================================================================



class CalendarsStorage:

    """
    ------------------------------------------------------------------
    Implement file IO ops per the hosting platform's permission rules.
    One instance of one sublcass, created by and stored in App instance.
 
    Base class with common/shared methods for Android and PCs subclasses.
    Example usage in App:

    self.storage = \
        CalendarsStorage_Android(self) if RunningOnAndroid else 
        CalendarsStorage_PCs(self)
    
    self.storage.run_with_calendars_folder(
        self.storage.load_all_calendar_files, forcedask=?)

    self.storage.run_with_calendars_folder(
        self.storage.backup_andsave_all_changed_calendar_files)        
    ------------------------------------------------------------------
    """



    def __init__(self, app, maxbackups):
        self.app = app                           # link to state+ops in main.py's App()
        self.maxbackups = maxbackups             # from App.settings, changed by Settings
        self.folder_details_shown = False        # show dialog extra info once pre run
        self.any_file_ops_since_empty = False    # warn about saves sans folder access



    def clear_calendar_and_event_tables(self):

        """
        ------------------------------------------------------------------
        Clear global tables of loaded calendars and events to be empty
        prior to loads, for Settings-screen folder changes.  This must 
        be an in-place change.  It cannot simply assign {}s to these names,
        because they are imported with 'from' by other modules, which 
        share objects and would not notice name changes here (see LP6E!).
        ------------------------------------------------------------------
        """

        for table in (CalendarsTable, CalendarsDirty, EventsTable): 
            table.clear()    # same object in this module and others 



    def ask_calendars_folder_info(self, nextop, forcedask, rawnextop):
        
        """
        ------------------------------------------------------------------
        Folder ask step 1: called on all platforms before file ops when 
        there is no access to a calendars folder.  Popup info about the 
        ask, then run a platform-specific asker, which later runs the 
        nextop platform-specific load or save (three levels).  Context:

        - forcedask is True only for Settings screen's calendar-folder mod.
        - rawnextop is the true next step after ask, for save+chdir caution.

        This step is generic enough to be the same for Android and PCs 
        and multiple ops, but the message must be tailored for Android: 
        see default_calendar_folder_path().

        FORMATTING:

        Caveat: on wsl2, [size=20sp] seems not to scale to font size as it does 
        in Help/About, likely due to default font size diffs and a wsl2 glitch;
        it _should_ scale per Kivy's 2.3.1 code, which calls dpi2px() for 'Nsp'
        to scale (despite the fact the Google's AI said it is not supported...).

        Update: [size=20sp] was dropped because it also doesn't change if the 
        user changes globalfontsize in the Settings screen (there's no way to
        bind it to a property).  Instead, headings now use color, here and in
        Help and About text.  This is subtle: color must work in both Dark and 
        Light mode, alt fonts (e.g, DejaVuSans) makes text look weak, and 
        italic and underscores are already used much in non-header text.
        This oddball need not match Help/About style per se, but it does.

        DETAILS EXTRA:

        The details part of this message might be limited to first-run asks, 
        when self.app.runnum == 1, because it's mostly just for new users 
        setting the folder initially; after that, there is similar info in 
        Help's calendars coverage.

        Update: as a compromise, show details just once per run, instead of
        either always or on first run only.  Probably a useful tip on future
        Settings folder mods, but they may be annoying if > once in a run.

        Nit: the spacing for details is less than in first-run terms of use
        and Help/About, but it should match the caution up next and we don't 
        want the caution to be missed by users.  Trivial but significant.

        SAVES EXTRA:

        rawnextop is used to detect calls from run_with* for a main-menu 
        Save, when there are calendar changes and the prior calendar folder 
        cannot be accessed due to a delete, rename, or app-permission removal 
        after a load while app is running.  This is a dicey UX boundary case.
        There are three contexts that invoke run_with_calendars_folder() and 
        its asks:
        
        - Initial calendar loads, where nextop is load_all_ics_files()
        - On-demand calendar-folder changes, where nextop is load_all_ics_files()
        - On-demand calendar saves, where nextop is backup_and_save_all_changed_ics_files()
        
        The first two already load all calendars next by design.  Saves do not 
        reload calendars, because doing so would wipe out the very changes that 
        the user is trying to save, but not reloading means that the GUI won't 
        reflect the folder after Save.  Reloading post saves would load only 
        calendars that were saved in the new folder, and that's not right either.

        The best Save can do in this case is to save changes to a newly picked
        folder, and leave it to the user to sort out the aftermath.  This is also
        so unlikely as to be either impossible or attributable to user error. 
         
        But to minimize risk, run_with* asks now rely on this method to detect
        this rare use case and augment the pre-select info_message so the user 
        is aware of the consequences.  The save proceeds in the new calendar 
        folder if the user picks one, however, to save the calendar changes.
        See main.py's on_menu_save for parallel notes and context.

        Subtlety: we're interested only in Save sans folder access when the
        in-memory calendars reflect a real folder, such that saves will create
        skew.  To detect this, we must check rawnextop for the save method but
        also whether there were prior calendar loads since either startup or 
        the last folder change.  In either case, it's okay to not yet have 
        folder access on Save if calendars have been added but not loaded or 
        saved to files (e.g., after adding new calendars on first run or 
        switching to a new empty folder in Settings).  This cannot check
        CalendarsTable, because new calendars are added there pre load/save.
        Instead, any_file_ops_since_empty flags file ops since the last 
        empty-folder state, so we can know if in-memory calendars reflect a 
        real folder's content and thus may yield skew on Save to a new folder.
    
        Update: this moved from separate popup to augmentation here, to 
        avoid popups overload (three is too much).

        Postscript: the save-sans-folder state might have been avoided 
        if Save saved _all_ calendars, instead of just changed calendars.  
        The new folder would then always be complete.  But partial saves
        are by design because saving all calendars on every save might 
        be noticeably slow, especially on slower phones.  In the end, 
        speed is more important than avoiding a wildly unlikely UI state.  
        ------------------------------------------------------------------
        """

        first_run_extra = save_caution_extra = ''

        # provide extra details once per run only, info in Help later

        if not self.folder_details_shown:
            self.folder_details_shown = True

            first_run_extra = (
            '\n\n\n'
            '[H2]Details[/H2]'
            '\n\n' 
            'This app creates and uses calendars formatted per the '
            'iCalendar standard and stores them in a folder as portable ICS '
            'files with extension .ics.  Calendar [i]loads[/i] combine all the '
            '.ics files in this folder, and calendar [i]saves[/i] update all '
            'changed .ics files there after creating backup copies.'
            '\n\n'
            'To share your calendars with other programs and devices, simply '
            'sync or manually copy .ics files to or from your chosen calendars '
            'folder.  Some other calendar apps may require imports or exports '
            'for ICS files, but this app uses them directly on all platforms.'
            )


        # alert user about picking a new folder for saves, not a separate popup

        if (self.any_file_ops_since_empty and
            rawnextop == self.app.storage.backup_and_save_all_changed_ics_files):

            save_caution_extra = (
            '\n\n\n'
            '[u][H2]Caution[/H2][/u]: save sans prior-folder access[/color][/b]'
            '\n\n'
            'The external calendars folder to which your changed calendars '
            'correspond is no longer accessible.'
            '\n\n'
            '[i]Proceeding[/i] with this save by choosing a new folder in the '
            'following chooser will result in only your changed calendars being '
            'saved to the new folder, and the app\'s GUI will not be reloaded '
            'to reflect the new folder\'s contents.  You may have to reconcile '
            'prior/new folder differences manually after the save.'
            '\n\n'
            '[i]Cancelling[/i] this save in the following chooser will avoid folder '
            'and GUI skew but will not save your calendar changes.  If you removed '
            'app permissions or deleted or renamed your folder since this app was '
            'started, you may wish to undo your mods now and retry the save.'
            )
       

        reason = (
            'You have asked to change the calendars folder' 
            if forcedask else 
            'This app does not have access to a calendars-storage folder'
        )

        suggested = (
            '' if RunningOnAndroid else
            'For convenience, a Frigcal subfolder in your home '
            'folder is available as a suggested default.'
        )

        self.app.info_message(

            '[b]Calendar folder selection[/b]'
            '\n\n'
           f'{reason}.  '
            'In the next screen, please select the folder to be used for '
            'storing your calendar files.'
            '\n\n'
            'You can select any folder that you can access on your device, '
            'including one that is part of content you normally sync with '
           f'other devices.  {suggested}'
           f'{first_run_extra}'
           f'{save_caution_extra}'

            , usetoast=False, nextop=nextop
        )

        # and the folder chooser comes up next, followed by nextop



    def no_calendar_files_message(self, folderPathOrUri):

        """
        ------------------------------------------------------------------
        Shared by POSIX/filepath and SAF/content-uri calendar loaders.
        folderPathOrUri is a py str: POSIX pathname or SAF content URI.
        Called only before load if have calendars-folder access by a 
        prior op or a newly provided chooser reply, and folder is empty.
        This can happen on startup and on later Settings folder changes.

        Clears any_file_ops_since_empty because this is a known state in
        which internal calendars do not correspond to an external folder,
        and a Save cannot write partial folder contents.  All later loads 
        and saves set this flag because internal reflects external.

        TBD: automatically add a default calendar file if none in folder,
        or expect the user to add one with New?  Either policy may be seen
        as onerous by some users: manually deleting an unwanted calendar 
        versus manually adding a new one.  

        CHOICE: assume user must run New run next, and show this info_message
        with tips.  Further assume that user will run a Save to write the 
        calendar's ICS file if and when desired.  No default cal required.

        UPDATE: but must not skip calendars load if no calendars in folder,
        because this may occur after a change-folder in Settings, and the
        GUI should reflect the new empty folder.  Else, a Save would save 
        the prior folder's calendars in the new and unrelated folder.
        This is acheived by coding in the load methods, not here.
        ------------------------------------------------------------------
        """

        self.any_file_ops_since_empty = False

        self.app.info_message(
            '[b]No calendars in folder[/b]'
            '\n\n'
           f'....{folderPathOrUri}'
            '\n\n'
            'If you are using this folder for the first time, either:'
            '\n\n'
            '1) Add a [i]first calendar[/i] to this folder now using the '
            'main menu\'s New Calendar, and use its Save Calendars to '
            'create the new calendar\'s file when desired.'
            '\n\n'
            '2) Copy [i]existing calendar files[/i] to this folder, and '
            'load them by either restarting this app or changing to the '
            'same folder again in the Settings screen.'
            '\n\n'         
            'To use a different folder altogether, see Settings.')

        # and end UI interaction for calendar loads (after reload!)



    def parse_one_ics_file(self, pyFileObject, icsfilebasename):

        r"""
        ------------------------------------------------------------------
        ICS events file parser: creates/returns in-memory data from a .ics file.
        Load events at startup from ICS file whose open file object is passed to
        pyFileObject.  icsfilebasename is the ICS file's basename (a py str), 
        regardless if it's from a pathname or URI, and this is run within a 
        non-GUI thread by its caller so it cannot block the GUI.

        Run by a platform-specific version of loading code, which uses POSIX/paths
        on all PCs and SAF on Android.  Backups and saves instead have a platform- 
        neutral driver that calls out to platform-specifc parts; which is better?

        The caller finds, opens, and closes pyFileObject in platform-specific code.
        Don't reinitialize tables here: they are already imported by object via 'from'!
        Any errors here are caught and displayed by the caller: the data may be bad.

        VTEXT is a str subclass, and works as is: no extra str() conversion is needed.
        icalendar uses the UTF8 iCalendar default Unicode encoding scheme throughout.
        from_ical parse auto-unescapes any "\X" icalendar sequences and does Unicode decoding.
        to_ical generate auto adds "\X" escapes if needed and does Unicode encoding.
        See also the note about reading all at once here, in generate_ics_files() ahead.

        Unicode NB: this formerly used icalendar.parser.DEFAULT_ENCODING for calendar
        files' Unicode encodings, with a note about not hardcoding UTF-8.  In FC 4.0, 
        both callers hardcode UTF-8 because it's used by iCalendar per spec.  This 
        may be an issue for non-conforming calendar tools that use other encodings, 
        but there is no source code to be changed in frozen apps/executables, and 
        making this a user setting may not help (encoding might vary per ICS file).
        Bytes works in icalendar API too, but it's not clear how this would help.
        ------------------------------------------------------------------
        """

        # TBD: make calendar/event tables attrs of this self, not globals?

        global CalendarsTable, CalendarsDirty, EventsTable          # all filled here
        trace('Parsing', icsfilebasename)

        # read ics text, caller handles open, close, exceptions
        icstext = pyFileObject.read()                               # all at once: small files
        
        # parse ics text: 3rd party package(s), huge dependency
        icsarray = icalendar.cal.Calendar.from_ical(icstext)        # .cal optional but explicit
        CalendarsTable[icsfilebasename] = icsarray                  # add to global calendars table
        CalendarsDirty[icsfilebasename] = False                     # no changes yet at startup
        self.any_file_ops_since_empty = True

        # extract data used here, index by start date and uid
        for event in icsarray.walk('VEVENT'):                       # for all events in parsed calendar

            # icalendar.prop.vDDDTypes: .dt is datetime.{date or datetime} with .m/d/y
            startdate = event['DTSTART']                # required start date
            datestamp = event['DTSTAMP']                # required modtime: display order if no created
            createdon = event.get('CREATED', None)      # optional createtime: for display order

            # icalendar.prop.vText (VText): .to_ical() is bytes, str() decodes auto
            uniqueid  = event['UID']                    # required id, for update searches
            labeltext = event.get('SUMMARY', '')        # for main label and view/edit dialog
            extratext = event.get('DESCRIPTION', '')    # for view/edit dialog, main text
            category  = event.get('CATEGORIES', '')     # for coloring and view/edit dialog
 
            if isinstance(category, list):
                category = category[0]                  # if multiple, use just one (first) for coloring

            # vdates: .dt is a datetime.date/datetime object
            # transfer to custom date type for better control here
            eventdate = Edate.from_datetime(startdate.dt)
            if createdon:
                orderdate = Edate.from_datetime(createdon.dt)
            else:
                orderdate = Edate.from_datetime(datestamp.dt)

            # vtext: see above--ok as is, but undo odd '\X' escapes in some existing data
            labeltext = labeltext.replace('\\"', '"')
            extratext = extratext.replace('\\"', '"')            # '\"' -> '"'
           
            # [1.4] undo =5C's from file, restoring '\'; see unescape function ahead
            labeltext = backslash_unescape(labeltext)
            extratext = backslash_unescape(extratext)       # '=5C' -> '\'
            
            # index this file's event data for date by start m/d/y
            icsdata = EventData(uid=uniqueid,
                                calendar=icsfilebasename,
                                summary=labeltext,
                                description=extratext,
                                category=category,
                                orderby=orderdate,
                                veventobj=event)
            
            if eventdate not in EventsTable:                   # .keys() optional, eitbti?
                EventsTable[eventdate] = {}                    # first event for this date
            EventsTable[eventdate][uniqueid] = icsdata         # add to global events union table



    def backup_and_save_all_changed_ics_files(self, folderPathOrUri):

        """
        ------------------------------------------------------------------
        Backup and save the ICS files of all calendars that have been 
        changed since loaded or last saved.  This is a two-step combo 
        that is run in a thread as one unit to avoid blocking the GUI.
        Never called if no calendars have changes: caller ensures.

        folderPathOrUri is a py str on PCs or a java Uri on Android.
        Called by run_with_calendars_folder() only if the app already 
        has calendar-folder access or it's given in an popup precursor.
        Runs platform-specific code in subclasses for actual file IO.

        Subtle: if the user was asked for a new folder before this ran
        due to permission/folder changes, this does _not_ load calendars 
        from the new folder, because doing so could lose any unsaved 
        changes in the in-memory calendars.  This will simply save 
        changed calendars to the new folder. 	For more info on this, 
        see SAVES EXTRA in ask_calendars_folder_info().
        ------------------------------------------------------------------
        """

        # backup+save files in new thread to avoid blocking gui
        def backup_and_save_ics_files_thread():

            # two-step combo
            backupfails = self.backup_all_changed_ics_files(folderPathOrUri)
            if not backupfails:
                savefails = self.save_all_changed_ics_files(folderPathOrUri)

            # on thread exit, update gui in gui thread
            def on_backup_and_save_thread_exit(dt):          # blocker is in enclosing scope,

                #self.app.dismiss_popup(blocker)             # despite the different threads
                self.app.finalizeBusyPopup(blocker)          # else too fast?, blocker in scope

                if backupfails:
                    self.app.info_message(
                        '[b]Backups failed - all saves skipped[/b]'
                        '\n\n'
                        'Errors occurred while backing up these ICS files:\n\n'
                       f'  {backupfails}\n\n'
                        'Due to the errors, no saves were performed.  Other files '
                        'were backed up, and the prior versions of all ICS files '
                        'were retained.')

                elif savefails:
                    self.app.info_message(
                        '[b]Saves failed - some files not saved[/b]'
                        '\n\n'
                        'Errors occurred while saving these ICS files:\n\n'
                       f'  {savefails}\n\n'
                        'Due to the errors, these files\' saves were aborted, '
                        'but all other files\' saves were successful and all '
                        'ICS files were backed up.')
 
            Clock.schedule_once(on_backup_and_save_thread_exit)    # same as @mainthread

        # post modal popup, start function in thread
        blocker = self.app.postBusyPopup(kindtext='saving')
        threading.Thread(
            target=backup_and_save_ics_files_thread,    # don't block gui thread 
            args=(),                                    # uses locals in scopes 
            daemon=True).start()                        # DON'T close thread if app closed: writes
        # and continue in gui thread with popup



    def backup_all_changed_ics_files(self, folderPathOrUri):

        r"""
        ------------------------------------------------------------------
        Backup ICS file(s) that have changes on main-menu Save, as 
        a precursor to file saves.  

        folderPathOrUri is a py str on PCs or a java Uri on Android.
        A true result denotes errors as a list of failed filenames
        that cancels saves.  Unlike loads, this runs a platform-specific
        method in subclasses along the way to minimize some redundancy.  
        It's run within its caller's thread, so it cannot block the GUI. 
        ------------------------------------------------------------------
        """

        global CalendarsTable    # used here

        fails = []
        icsfoldername = self.app.settings.calendarsfolder    # str: POSIX path or SAF uri
        assert icsfoldername == \
            (folderPathOrUri.toString() if RunningOnAndroid else folderPathOrUri)
        trace('Backing up to', icsfoldername)

        # get|make bkp subdir: (str or Uri) or None
        backupsSubfolder = self.backups_get_subfolder(folderPathOrUri)
        if not backupsSubfolder:
            fails.append('<Could not make backups folder>')   # too rare to do more
        else:
            for icsfilename in CalendarsTable:
                if not CalendarsDirty[icsfilename]:
                    # no changes since startup or save: don't backup/write this file
                    continue

                try:
                    # trim backups folder if needed
                    # keeps most recent N-1 of each, based on sort order of bkp names

                    # glob returns pathname strs or DocumentFile objs
                    backuppatt  = 'date*-time*--' + icsfilename
                    currbackups = self.backups_glob_dir(backupsSubfolder, backuppatt)
                    currbackups.sort(reverse=True, 
                        key=lambda x: x if not RunningOnAndroid else x.getName())

                    prunes = currbackups[(self.maxbackups - 1):]               # earliest last
                    for prunee in prunes:                                      # globs have paths
                        trace('pruning', prunee)
                        try:                                                   # file deletes can fail:
                            self.backups_remove_file(prunee)                   # but don't cancel bkp/save
                        except:
                            trace(f'Error while pruning: skipped {prunee}')    # too rare to do more
                            traceback.print_exc()                    

                    # transfer current file to backup folder/name
                    # note: move is quick and may be less error prone,
                    # but copy retains original file in case save fails

                    datetimestamp = time.strftime('date%y%m%d-time%H%M%S')          # add date/time prefix
                    backupname    = f'{datetimestamp}--{icsfilename}'
                    self.backups_backup_file(
                        folderPathOrUri, icsfilename, backupsSubfolder, backupname)
                
                except:
                    # on any and first failure for this file
                    fails.append(icsfilename)
                    trace(f'Error while backing up: skipped {icsfilename}')
                    traceback.print_exc()
                    # tbd: break to stop now since saves will be cancelled?
    
        return fails   # [failed filenames] or []=False:OK



    def save_all_changed_ics_files(self, folderPathOrUri):

        r"""
        ------------------------------------------------------------------
        Store events in ICS file(s) on main-menu Save, but only after 
        successful backups.  

        folderPathOrUri is a py str on PCs or a java Uri on Android.
        A true result denotes errors as a list of failed filenames.
        Unlike loads, this runs a platform-specific method in subclasses
        along the way to minimize some redundancy.  It's run within its
        caller's thread, so it cannot block the GUI. 
    
        Coding note: calendar files are written here, and read in the
        parse function, all at once with .read() and .write(all), not 
        in chunks via .read(N) and .write(part).  These files are small 
        (app proprietor's largest is just 2M for 20 years), all usable 
        devices have gigabytes of memory today, and arbitrary buffer 
        sizes should work on all platforms under Python 3.X today.  An 
        old Windows issue precluded writing buffers > 64M on network  
        drives but is fixed in recent libs (and 64M equates to 640 
        comparable years of events for the app proprietor's usage).
        ------------------------------------------------------------------
        """

        global CalendarsTable    # used here

        fails = []
        icsfoldername = self.app.settings.calendarsfolder    # str: POSIX path or SAF uri
        assert icsfoldername == \
            (folderPathOrUri.toString() if RunningOnAndroid else folderPathOrUri)
        trace('Generating to', icsfoldername)

        for icsfilename in CalendarsTable:
            if not CalendarsDirty[icsfilename]:
                # no mods to this file since startup or Save: don't backup/write
                continue
        
            try:
                icscalendar = CalendarsTable[icsfilename]
                if not icscalendar.walk('VEVENT'):    # a list
                     # if no events after New or deletes, add a required one now (rare but true)
                    vevent = new_vevent(                                        # fix name error [1.5]
                        dtstart=None,
                        uid=icalendar_unique_id() + '-' + icsfilename,          # make unique if > one [1.5]
                        summary=f'{APPNAME}-generated event',
                        description='Required sole event generated.\n\n'        # add tips text [1.5]
                                    'You can delete this event at any time '
                                    'by tapping it within the app.\n\n'
                                    'To completely remove this event\'s calendar, '
                                    'delete its ".ics" file from your calendars folder.',
                        category='')                                            # color? (else white) [2.0]
                    icscalendar.add_component(vevent)                           # was 'system internal'

                icstext = icscalendar.to_ical()                                 # bytes, major dependency
                self.save_one_ics_file(icsfilename, folderPathOrUri, icstext)   # platform-specific subs
                CalendarsDirty[icsfilename] = False                             # no changes after save
                self.any_file_ops_since_empty = True

            except:
                # skip file but keep going
                fails.append(icsfilename)
                traceback.print_exc()
                trace(f'Error while generating ICS file: {icsfilename}')

        return fails   # [failed filenames] or []=False:OK




#==============================================================================
# 💻  PC SUBCLASS -- factored-out code for PCs only: POSIX/filepaths
#==============================================================================



class CalendarsStorage_PCs(CalendarsStorage):

    """
    ------------------------------------------------------------------
    Subclass to factor out PC-specific calendars behavior.
    On all PCs, use the portable POSIX toolset and file pathnames.
    Don't use these on Android to avoid All Files Access permission.
    ------------------------------------------------------------------
    """



    def pprintFolder(self, pathnameStr):
     
        """
        ------------------------------------------------------------------
        Nothing to see here (show POSIX pathnames as they are).
        ------------------------------------------------------------------
        """

        return pathnameStr



    def default_calendar_folder_path(self):

        r"""
        ------------------------------------------------------------------
        The user's home folder on all PCs: app always has access.
        Path: macOS=/Users/me, Windows=C:\Users\me, Linux=/home/me
        For suggested Frigcal calendars-folder default on PCs - only.
        ------------------------------------------------------------------
        """

        return os.path.expanduser('~')



    def run_with_calendars_folder(self, nextop, forcedask=False):

        r"""
        ------------------------------------------------------------------
        PCs version: verify in filesystem or use folder chooser.
        forcedask is True for folder changes in Settings.
        nextop callable is run with pick after folder selection.

        [See the Android subclass's version for common overview]

        ABOUT WINDOWS/WSL2 PATH CONVERSIONS FOR OPENS:

        This does not attempt to manually convert the priorPath 
        from Windows to WSL (Linux in Windows) format on WSL, 
        nor from WSL to Windows format on Windows.  

        It's possible to do so with a "wslpath" command lines in both
        platforms (e.g., /home/me to \\wsl.localhost\Ubuntu\home\me on
        Windows and C:\Users\me to /mnt/c/users/me on WSL2.  See 
        main.py's launch_file_explorer() for code that does this
        because it's required for file explorers.  For file opens
        here, this conversion happens automatically in Windows but 
        not in WSL2, for reasons with no known documentation.

        Applying manual conversions here are more dubious.  Detecting 
        platform-specific paths is iffy, and this seems a use case 
        with no users: it's possible only if Windows and WSL2 share
        a common settings file, and this can only occur when running
        a common source-code package on both.  It's unclear that a
        full source-code package will be released, and even if it is,
        users will normally instead use separate Windows and Linux 
        executable packages that have their own settings files.
        ------------------------------------------------------------------
        """

        haveaccess = False
        priorPath = self.app.settings.calendarsfolder    # None or prior ask
        trace(f'{priorPath=}')

        if priorPath != None:
            try:
                listed = try_listdir_nohang(priorPath)
                if not listed:
                    # hung (inaccessible), or immediate fail (unmounted?)
                    haveaccess = False
                else:
                    haveaccess = True
            except:
                pass

        if haveaccess and not forcedask:
            nextop(priorPath)                                        # pass py str
        else:
            self.ask_calendars_folder_info(                          # step 1: info popup
                nextop=lambda: self.ask_calendars_folder(nextop),    # modal asks
                forcedask=forcedask,                                 # if Settings chdir
                rawnextop=nextop)                                    # for save caution



    def ask_calendars_folder(self, nextop):
 
        """
        ------------------------------------------------------------------
        Folder ask step 2: on all PCs, ask user to pick a calendars
        folder in the Kivy folder chooser.

        This varies completely from the Android version, which must use 
        the Android-only SAF chooser for permissions.  PPUS's chooser has
        far too much functionality to reuse here (e.g., removable-drives),
        and Kivy's base chooser is really just a chooser-construction kit.  
        Instead, use the simple KivyMD chooser.  Downsides: non-native and
        limited (e.g., no new-folder op, though the app makes one in ~).
        The chooser is inline code here, not .kv code (a past lesson).

        Alternative: bundle tkinter and use its folder choosers from Kivy?
        Per a web search ("bundle tkinter with a kivy app [on android]"),
        this seems feasible on PCs but impossible on Android.  We 
        don't care here for PCs, but there is a showstopper: tkinter's 
        chooser blocks the Kivy GUI: it won't redraw, resize, minimize...
        It might work in a thread, but PyInstaller packaging is tricky.
        tkinter's chooser also has known issues on some macOS (see PyEdit).

        Alternative: Kivy's plyer also has a file chooser that tries to be 
        platform native, but it's unusable on Android (the SAF chooser is
        required to get and persist permissions), and its folder chooser 
        is lousy on Windows (a shell hack that posts a blurry 1990s-ish
        box that's so bad that plyer itself recommends alternatives).
        The weak Windows support is a full showstopper for PCs here.
        ------------------------------------------------------------------
        """
        
        # open in ~/Frigcal suggested default created by App.on_start()
        trace('ask_calendars_folder')

        def on_exit_folderchooser(*args):
            trace('fm exit')
            folderchooser.close()
            self.app.folderchooser_open = False

        def on_select_folderchooser(path):
            trace('fm select:', path)
            on_exit_folderchooser()
            self.on_selection(path, nextop)

        dohiddens = self.app.settings.chooserShowHiddenFiles    # unchangeable (tbd)
        startpath = self.app.defcalspath
        
        # chooser silently refuses to open if the default path has been renamed
        # we have limited access on Android, but this is run on PCs only (else SAF) 

        if not os.path.isdir(startpath):
            startpath = self.default_calendar_folder_path()     # ~ sans subfolder

        folderchooser = MDFileManager(
            search='dirs',                             # folders only
            show_hidden_files=dohiddens,               # tbd: Settings?
            exit_manager= on_exit_folderchooser,
            select_path=on_select_folderchooser
        )
        folderchooser.show(startpath)
 
        self.app.folderchooser = folderchooser         # for routing Back taps
        self.app.folderchooser_open = True             # back=>up in folderchooser



    def on_selection(self, pickedPath, nextop):

        """
        ------------------------------------------------------------------
        Folder ask step 3: after user has interacted with Kivy calendar-folder 
        chooser on PCs, run the nextop with the selected URI, if any.
        ------------------------------------------------------------------
        """

        trace('on_selection', pickedPath)
        if pickedPath:

            # update folder setting (only) for pick
            self.app.settings.update_and_save_settings(calendarsfolder=pickedPath)

            # update Settings screen for pick
            self.app.settingsgui.screen.ids.calendarsfolder.text = pickedPath
            
            nextop(pickedPath)    # pass selected path's py pathname str

            # TBD: reload calendar files and redraw gui, unless nextop is load_all_calendars?
            # NO: see ask_calendars_folder_info() - reload would delete changes



    def load_all_ics_files(self, folderPath):

        """
        ------------------------------------------------------------------
        thread(load calendars from folderPath), draw gui
        Uses chosen POSIX pathname str, app has folder permission+access.

        Called by run_with_calendars_folder() only if the app already 
        has calendar-folder access or it's given in an popup precursor.
        Runs files-load loop in a thread to avoid blocking the GUI.
        Runs superclass's generic/common parsing code in the thread.
        See no_calendar_files_message() for TBD on policy used here.
        ------------------------------------------------------------------
        """

        global CalendarsTable, CalendarsDirty, EventsTable    # all reset in-place here

        # TBD: add default calendar file if none in folder?
        # NO: expect user to run New, plus Save for file/changes

        trace('loading from', folderPath)
        if not folderPath: 
            return

        listed = try_any_nohang(os.listdir, folderPath, onfail=None)
        if not listed:
            # hung (inaccessible), or immediate fail (unmounted?)
            calslist = []
        else:
            calslist = [filename for filename in listed                         # i.e., glob.glob
                            if filename.lower().endswith(ICS_FILE_EXTENSION)    # [4.1] ignore case
                            and not filename.startswith('._')]                  # [4.1] skip AppleDoubles

        trace(f'{calslist=}')    # list of .ics filenames or []
        if not calslist:
            # alert user but load anyhow: clear prior folder's events, if any
            self.no_calendar_files_message(folderPath)

        # clear global tables for Settings-screen folder change, in-place
        self.clear_calendar_and_event_tables()

        # load files in new thread to avoid blocking gui
        def load_all_ics_files_thread():
            fails = []
            for calname in calslist:
                calpath = osjoin(folderPath, calname)
                try: 
                    fileobj = open(calpath, 'r', encoding='utf8')        # open can fail
                except:
                    fails.append(calname)
                    trace(f'{calname=} skipped:\n{sys.exc_info()=}')
                else: 
                    try:
                        self.parse_one_ics_file(fileobj, calname)        # *common ops super
                        trace(f'{calname=} parsed')                      # read/parse can fail
                    except:
                        fails.append(calname)
                        trace(f'{calname=} skipped:\n{sys.exc_info()=}')
                    finally:
                        fileobj.close()                                  # always close object

            # (load thread) on thread exit, update gui in gui thread
            def on_load_thread_exit(dt):
                #self.app.dismiss_popup(blocker)                         # may be too fast: button 
                self.app.refill_month_widgets()                          # update gui, [4.1] unscrolls days
                self.app.finalizeBusyPopup(blocker)                      # blocker in scope

                if fails:
                    self.app.info_message(
                        '[b]Some files not loaded[/b]'
                        '\n\n'
                        'The following ICS files had loading errors '
                        'and were skipped:\n\n'
                       f'  {fails}\n\n'
                        'These files\'s events will not be shown.')

            # (load thread) schedule gui update
            Clock.schedule_once(on_load_thread_exit)                     # == @mainthread

        # show modal popup, start function in thread
        blocker = self.app.postBusyPopup(kindtext='loading')
        threading.Thread(
            target=load_all_ics_files_thread,      # don't block gui thread 
            args=(),                               # uses locals in scopes 
            daemon=False).start()                  # close thread if app closed: reads
        # and continue in gui thread with popup



    def save_one_ics_file(self, icsfilename, icsfolder, icstext):

        """
        ------------------------------------------------------------------
        Save one (changed) calendar to its ICS file, by POSIX + pathname.
        Run as part of a thread by super's common ICS-file maker loop.

        - icsfilename is the 'xxx.ics' name of the calendar being saved.
        - icsfolder is the POSIX folder filepath str chosen by the user.  
        - icstext is the bytes object created for ICS text, made by the 
          icalendar lib for the in-memory calendar.
         ------------------------------------------------------------------
        """

        icsfilepath = osjoin(icsfolder, icsfilename)
        trace('writing', icsfilepath, '...')
        if not osexists(icsfilepath):
            trace(f'creating new ics file: {icsfilename}')
        try:                                                # write or overwrite
            fileobj = open(icsfilepath, 'wb')               # text is bytes, not str
        except:
            trace(f'open fail: {sys.exc_info()=}')
        else:
            try:
                fileobj.write(icstext)                      # all at once: small files
            except:
                trace(f'write fail: {sys.exc_info()=}')
            finally:
                fileobj.close()                             # flush, exc or not (!with: eibti)



    def backups_get_subfolder(self, icsfolderPath):

        """
        ------------------------------------------------------------------
        This and its 3 followers are called by super's common backups loop
        and implement the POSIX/filepath versions of the required ops.
        All are run from the caller's thread, so they cannot block GUI.
        This method returns pathname str or falsy None.
        ------------------------------------------------------------------
        """

        backupdir = osjoin(icsfolderPath, BACKUPS_SUBDIR_NAME)
        try:
            if not os.path.exists(backupdir):
                os.mkdir(backupdir)
            assert os.path.exists(backupdir) and os.path.isdir(backupdir)
            return backupdir
        except:
            trace('backupDir make fail or already a non-dir')
            return None


    def backups_glob_dir(self, backupsSubfolderPath, backuppatt):
        return glob.glob(osjoin(backupsSubfolderPath, backuppatt))    # result has paths


    def backups_remove_file(self, pruneePath):
        os.remove(pruneePath)


    def backups_backup_file(self, folderPath, icsfilename, backupsSubfolder, backupname):
        # any exception caught by caller
        frompath = os.path.join(folderPath, icsfilename)
        topath   = os.path.join(backupsSubfolder, backupname)

        # skip: new in-memory calendar that doesn't yet have a file  
        if not osexists(frompath):
            trace(f'no file to backup: {icsfilename}')
            return
        else:
            trace(f'saving "{frompath}" to "{topath}"')
            shutil.copy2(frompath, topath)                   # copy content+info (write)

 


#==============================================================================
# 📱  ANDROID SUBCLASS -- factored-out code for Android only: SAF/URIs
#==============================================================================



class CalendarsStorage_Android(CalendarsStorage):
 
    """
    ------------------------------------------------------------------
    Subclass to factor out Android-specific calendars behavior.
    On Android, use the SAF toolset and content URIs for permission.

    We can't use POSIX/pathnames on Android without All Files Access,
    and the SAF chooser returns content URIs that provide permission
    but are not filepaths and tightly bind file ops to the SAF API.
    ------------------------------------------------------------------
    """


    # till loaded from config or asked
    calendarsFolderURI = None



    def pprintFolder(self, escapedUriStr):
     
        """
        ------------------------------------------------------------------
        Content URIs received from the SAF chooser may have a few 
        embedded %3A escapes (for ':') and %2F (for '/') that app
        users shouldn't have to grok (content URIs are iffy as is). 
        ------------------------------------------------------------------
        """

        # no: java, via pyjnius
        #URLDecoder = autoclass('java.net.URLDecoder')
        #return URLDecoder.decode(encodedUriStr, 'UTF-8')

        # yes: python, direct
        from urllib.parse import unquote_plus 
        return unquote_plus(escapedUriStr, encoding='utf-8', errors='ignore')



    def default_calendar_folder_path(self):

        """
        ------------------------------------------------------------------
        The internal-storage root on Android: requires user permission.
        Path: /storage/emulated/0 or its /sdcard synonym on some devices.
        This is general shared storage, not app-specific nor app-private.
        For creating a suggested-default calendars folder on first run. 

        UNUSED: shared storage cannot be used for a suggested calendars-
        folder default on Android because it requires a user permission 
        granted by either All Files Access or single-folder choice:

        - All Files Access would allow POSIX APIs but is unused by this 
          app because it would likely preclude Play store distribution.
  
        - Single-folder choice, used for the calendar folder in general,
          would require opening the SAF folder chooser just to access a 
          known calendar default folder, and the chooser returns a  
          content URI which essentially bind ops to the SAF API.

        The SAF chooser also comes with a poorly thought-out catch-22 for
        the initial folder path: a content URI is required to point users 
        to a suggested folder in the chooser, but apps cannot get a 
        suggested folder's URI without using the chooser.  IOW, apps 
        must use the SAF chooser to get a URI to pass to the SAF chooser!  
 
        More broadly, there is no good path to use for either calendars 
        or settings out of the box.  Besides shared storage:

        - A Documents/ URI might be forged, but it's iffy, and still
          requires user permission as for all of shared storage.

        - An app-specific (aka "external") folder located at:
          /storage/emulated/0/Android/data/com.quixotely.frigcal/files
          is no longer accesisble to most file-explorer apps as of 2025.
          App-specific is still available in Cx (oddly), but it generally 
          requires the Files app (which runs AOSP's explorer) or root 
          emulation with the shizuku app (which may not be available).

        - An app-private (aka "internal") folder in /data is never 
          accessible to other apps, including explorers and syncs.

        Both app-specific and app-private can be served to other apps by 
        coding a file provider, but explorer-app support is limited, and 
        both app-specific and app-private are deleted on app uninstalls. 

        Hence, PUNT: there is no good option for this default in Android 
        scoped storage, so users must make their own folder.  See also 
        App.storage_path_app_private()'s similar dilemma for settings;
        Android's lockdowns convolute development and limit users - badly.
        ------------------------------------------------------------------
        """

        return android.storage.primary_external_storage_path()



    # arbitrary number that ids a specific Android intent (how 1980s...)
    REQUEST_ID_OPEN_DOCUMENT_TREE = 987654



    def run_with_calendars_folder(self, nextop, forcedask=False):

        """
        ------------------------------------------------------------------
        Android version: verify by SAF or use SAF chooser

        Overview: Called on all calendar loads and saves to verify or 
        ask for the calendars folder.  Run by loads for app startup, 
        saves for main-menu's Save Calendars (including after change
        alerts), and loads for calendar-folder changes in the Settings 
        screen.  forcedask is True only for the latter, and nextop is 
        a callable that's run with the pick after a folder is selected.

        Note that users can remove either a folder or the app's permission 
        to a folder at any time, so the app must check before each action,
        even if already asked and persisted.

        On Android, there is a fundamental schism between POSIX filepaths 
        and SAF content URIs - they use very different APIs, and converting 
        between the two is iffy ('file://' URIs are no longer supported).
        Fetch POSIX file descriptors from the SAF API where possible, and
        assume that the SAF API calls used cannot hang (see hangless.py).
        ------------------------------------------------------------------
        """

        haveaccess = False
        Uri = autoclass('android.net.Uri')
        DocumentFile = autoclass('androidx.documentfile.provider.DocumentFile')
        priorUriString = self.app.settings.calendarsfolder    # None or prior ask
        trace(f'{priorUriString=}')
 
        if priorUriString != None:
            priorUri = Uri.parse(priorUriString)              # py str => java Uri
 
            # check uri in persisted permissions
            activity = self.app.get_android_activity()
            context  = self.app.get_android_context()

            persistedUriPermissions = \
                activity.getContentResolver().getPersistedUriPermissions()
            hasUriPermission = any(
                priorUriString == uritry.getUri().toString() 
                    for uritry in persistedUriPermissions)

            trace('perms:')
            trace([uritry.toString() for uritry in persistedUriPermissions])
            trace(f'{hasUriPermission=}')

            if hasUriPermission:
                # try listing: it may not be accessible now
                try:
                    documentsTree = DocumentFile.fromTreeUri(context, priorUri)
                    if documentsTree:
                        for documentFile in documentsTree.listFiles():
                            break
                        haveaccess = True
                except:
                    pass

        if haveaccess and not forcedask:
            nextop(priorUri)                                         # pass java Uri
        else:
            self.ask_calendars_folder_info(                          # step 1: info popup
                nextop=lambda: self.ask_calendars_folder(nextop),    # modal asks
                forcedask=forcedask,                                 # if Settings chdir
                rawnextop=nextop)                                    # for save caution



    def ask_calendars_folder(self, nextop):

        """
        ------------------------------------------------------------------
        Folder ask step 2: on Android, ask user to pick a calendars
        folder in Android's SAF UI so permissions are granted.  The
        content URI returned requires later processing to use SAF too,
        unless file descriptors can be fetched/used (not for folders).

        On Android, the SAF picker is effectively modal (blocking)
        because the app is paused until the user closes the picker
        (App on_pause() and on_resume() run on picker open/close):
        '''
        While it is technically a separate system activity and not a 
        modal dialog in the traditional programming sense, it has the
        practical effect of being a modal experience from the user's 
        perspective, as it blocks interaction with your app until the 
        task is complete.
        ''' 
        Hence, the app can chain actions assuming the picker will send
        a result to its callback before anything else happens in the app.

        The missing magic bits: "p4a_activity" uses pyjnius to make a 
        Java class with a callback method named onActivityResult() in 
        Python, which is registered to Java through the app's Android 
        activity and simply calls the Python callable that we pass to 
        p4a_activity.bind() here when the Intent finishes:

        https://github.com/kivy/python-for-android/blob/develop/
            pythonforandroid/recipes/android/src/android/activity.py

        SUBTLE: this must also _unbind_ any prior binds, here or in the 
        results handler, or arrange to bind just once (e.g., in __init__).
        Else, N prior runs and binds here means N+1 calls to the result 
        handler instead of 1 (yes, really!).  Barring user folder or 
        permission changes, this crops up only for the Settings screen's 
        calendar-folder change but is an arguably perilous API choice.  
        See the above activity.py; its code seems the only docs on this... 

        Tbd: Intents sent to and from Android have putExtra()/get*Extra() to 
        pass data on to the result handler, but it's Java strongly typed, and 
        it's unclear how to send a Python callback object to/from this API.
        PUNT: the current coding works as is.

        TBD: folder-ask intent also has an initial folder to open the popup
        at, but converting the default filepath to a URI is iffy~impossible
        [intent.putExtra(DocumentsContract.EXTRA_INITIAL_URI, uriToLoad)]
        _May_ work, possibly with file:// prefix:
        if (Build.VERSION.SDK_INT >=  Build.VERSION_CODES.N)
            fileUri = Uri.parse(self.app.defcalspath);
        else
            fileUri = Uri.fromFile(new File(self.app.defcalspath));
        Uri.parse(new File("/sdcard/Documents/Frigcal").toString())
        PUNT: this is just how the AOSP/SAF icker works.
        ------------------------------------------------------------------
        """

        self.nextop = nextop    # post-user-choice action

        # python-for-android's activity class: for on_activity_result listener
        from android import activity as p4a_activity

        # use SAF chooser for Android permissions, not kivy widget

        # must use mActivty, not the casted app activity
        mActivity = self.app.get_android_activity(docast=False)

        # ask user, run action after UI result appears later
        Intent = autoclass('android.content.Intent')
        intent = Intent()
        intent.setAction(Intent.ACTION_OPEN_DOCUMENT_TREE)

        # required: unbind prior bind if any, else N callbacks for N binds! 
        p4a_activity.unbind(on_activity_result=self.on_ask_calendars_folder_result)

        # bind for results callback via p4a activity
        p4a_activity.bind(on_activity_result=self.on_ask_calendars_folder_result)
        mActivity.startActivityForResult(intent, self.REQUEST_ID_OPEN_DOCUMENT_TREE)

        # now the SAF picker popup appears and the app waits for the user's response



    def on_ask_calendars_folder_result(self, requestCode, resultCode, intent):

        """
        ------------------------------------------------------------------
        Folder ask step 3: after user has interacted with SAF calendar-folder 
        chooser on Android, run the nextop with the selected URI, if any.

        This is called by the Java API (by way of the Kivy Activity class) when
        the user dismisses the folder picker.  The app is blocked until the 
        user selects a folder or cancels the picker, so it's effectively modal.
        The selected URI is in the intent and is a java Uri, not a py str.

        This could convert URI to filepath (sort of; see web), but that would 
        limit the app to physical storage.  We may or may not need to care, but 
        using content URIs allows for virtual storage like clouds, which may be 
        useful but are also innately perilous from a privacy/safety perspective.
        ------------------------------------------------------------------
        """

        trace('saf picker callback')
        activity = self.app.get_android_activity()
        Activity = autoclass('android.app.Activity')
        Intent   = autoclass('android.content.Intent')

        if requestCode != self.REQUEST_ID_OPEN_DOCUMENT_TREE:
            trace('folder picker - irrelevant intent result')
        else:
            if resultCode == Activity.RESULT_CANCELED:
                trace('folder picker - cancelled')

            elif resultCode != Activity.RESULT_OK:
                trace('folder picker - unknown resultCode:', resultCode)
                
            else: 
                # OK: persist permission, save its uri, run nextop
 
                # chosen uri is in passed intent
                pickedUri = intent.getData()         # content uri, not posix pathname
                if not pickedUri:                    # assume it's a folder & accessible
                    trace('folder picker - no URI')
                    return    # possible?

                # save to settings as a str, Uri.parse(str) undoes
                pickedUriString = pickedUri.toString()              # java Uri => py str
                trace('folder picker results:', pickedUriString)

                # update setting in app-private storage, auto backed up
                self.app.settings.update_and_save_settings(
                    calendarsfolder=pickedUriString)

                # update Settings screen for pick: must be run in gui thread
                def setum(dt):
                    guifield = self.app.settingsgui.screen.ids.calendarsfolder
                    guifield.text = pickedUriString
                Clock.schedule_once(setum)

                # save persistable URI permission for long-term access
                activity.getContentResolver().takePersistableUriPermission(
                    pickedUri,
                    Intent.FLAG_GRANT_READ_URI_PERMISSION | 
                    Intent.FLAG_GRANT_WRITE_URI_PERMISSION
                )

                # run dependent action: ensure run on main gui thread if needed;
                # same as calling a @mainthread decorated callback func now/here;
                # passes the picked folder as the java Uri, not the py str;

                Clock.schedule_once(lambda dt: self.nextop(pickedUri))

                # TBD: reload calendar files and redraw gui, unless nextop is load_all_calendars?
                # NO: see ask_calendars_folder_inf() - reload would delete changes


 
    def load_all_ics_files(self, folderUri):

        """
        ------------------------------------------------------------------
        thread(load calendars from folderUri), draw gui.
        Uses chosen SAF folder URI, app has folder permission+access.
        folderUri is the java Uri from asker, not the py str.

        Called by run_with_calendars_folder() only if the app already 
        has calendar-folder access or it's given in an popup precursor.
        Runs files-load loop in a thread to avoid blocking the GUI.
        Runs superclass's generic/common parsing code in the thread.
        See no_calendar_files_message() for TBD on policy used here.

        About DocumentFile: it's a wrapper around a URI that represents 
        a document or directory managed by a SAF ContentProvider, which 
        provides file and directory access sans ContentResolver queries.
        Requires androidx.documentfile.documentfile:1.1.0 in bd.spec.

        About fds: this reads clendars in Python via os file descriptor
        fetched from SAF, coded per Java examples + Android docs (yuck!).
        Do not also run afd.close() (parcel file descriptor): closefd=True
        by default.  Can also get file uri from its name this way:

            Uri = autoclass('android.net.Uri')
            baseuri = Uri.parse(folderUri)
            itemUri = Uri.withAppendedPath(baseUri, calname)
        ------------------------------------------------------------------
        """

        global CalendarsTable, CalendarsDirty, EventsTable    # all reset in-place here

        trace('loading from', folderUri and folderUri.toString() or '?')
        if not folderUri: 
            return

        # fetch a list of the *.ics files in the calendars folder
        activity = self.app.get_android_activity()
        context  = self.app.get_android_context()
        DocumentFile = autoclass('androidx.documentfile.provider.DocumentFile')

        # get DocumentFile objects for the selected directory
        calslist = []
        documentsTree = DocumentFile.fromTreeUri(context, folderUri)
        if documentsTree:
            for documentFile in documentsTree.listFiles():
                if documentFile.isFile():
                    documentName = documentFile.getName()
                    if (documentName and documentName.lower().endswith(ICS_FILE_EXTENSION)
                        and not documentName.startswith('._')):    # [4.1] skip AppleDoubles
                        calslist.append((documentName, documentFile.getUri()))
   
        trace(f'{calslist=}')    # list of .ics filenames or []
        if not calslist:
            # alert user but load anyhow: clear prior folder's events, if any
            self.no_calendar_files_message(self.pprintFolder(folderUri.toString()))
            
        # clear global tables for Settings-screen folder change, in-place
        self.clear_calendar_and_event_tables()

        # read calendars via os file descriptor fetched from SAF
        contentResolver = activity.getApplicationContext().getContentResolver()

        # load files in new thread to avoid blocking gui
        def load_all_ics_files_thread():
            fails = []
            for (calname, caluri) in calslist:
                afd = contentResolver.openFileDescriptor(caluri, 'r')
                if not afd:
                    fails.append(calname)
                    trace(f'{calname=} skipped: no fd')
                else:
                    try:
                        fd = afd.detachFd()                                 # back in py (ahh)
                        fileobj = os.fdopen(fd, 'r', encoding='utf8')       # open can fail
                    except:
                        fails.append(calname)
                        trace(f'{calname=} skipped: open {sys.exc_info()=}')
                    else:
                        try:
                            self.parse_one_ics_file(fileobj, calname)       # *common ops super 
                            trace(f'{calname=} parsed')                     # read/parse can fail
                        except:
                            fails.append(calname)
                            trace(f'{calname=} skipped: parse {sys.exc_info()=}')
                        finally:
                            fileobj.close()                                 # always close object

            # (load thread) on thread exit, update gui in gui thread
            def on_load_thread_exit(dt):
                #self.app.dismiss_popup(blocker)                         # may be too fast: button 
                self.app.refill_month_widgets()                          # update gui, [4.1] unscrolls days
                self.app.finalizeBusyPopup(blocker)                      # blocker in scope

                if fails:
                    self.app.info_message(
                        '[b]Some files not loaded[/b]'
                        '\n\n'
                        'The following ICS files had loading errors '
                        'and were skipped:\n\n'
                       f'  {fails}\n\n'
                        'These files\'s events will not be shown.')

            # (load thread) schedule gui update
            Clock.schedule_once(on_load_thread_exit)                        # == @mainthread

        # show modal popup, start function in thread
        blocker = self.app.postBusyPopup(kindtext='loading')
        threading.Thread(
            target=load_all_ics_files_thread,      # don't block gui thread 
            args=(),                               # uses locals in scopes 
            daemon=False).start()                  # DO close thread if app closed: just reads
        # and continue in gui thread with popop



    def save_one_ics_file(self, icsfilename, icsfolder, icstext):

        """
        ------------------------------------------------------------------
        Save one (changed) calendar to its ICS file, by Android SAF.
        Run as part of a thread by super's common ICS-file maker loop.

        - icsfilename is the 'xxx.ics' str name of the calendar being saved.
        - icsfolder is the SAF folder content URI chosen by the user: 
          a java Uri here, not a py str.
        - icstext is the bytes object created for ICS text, made by the 
          icalendar lib for the in-memory calendar. 
         
        TBD: documentDir.createFile is supposed to be called sans extension,
        on the assumption that the provider adds one per MIME type.  This
        seems gray in practice, but try type/calendar for a .ics instead 
        of text/plain which may add .txt (alt: Application/octet-stream?).
        UPDATE: this works as coded. 

        ABOUT 'stupid_saf_mode': Android 10 bizarrely changed the open 
        mode for contentResolver.open* such that the standard 'w' mode 
        no longer truncates the file to zero bytes.  The net effect can 
        leave garbage bytes at the end of the file if its new content is 
        shorter in the new write.  This is wildly proprietary and was 
        woefully undocumented (and comes off as naive and arrogant).

        THE FIX: Mode 'wt' can force truncation but apparently failed for 
        Google Drive's SAF provider.  Mode 'rwt' used here, and recommeded
        as a workaround by Google/Android, truncates too and works in GD.
        The garbage bytes seemed sporadic so it's not certain that this 
        fixes the problem, but all signs point to yes.

        FOR MORE on this Android 10 breakage (and a weak Google reply):
        https://issuetracker.google.com/issues/180526528
        or search: "android saf open for writing does not truncate file"

        Python's file.truncate(0) also truncates, and writing the bytes 
        in smaller chunks may help (fails were seen on a 2.3M file), but
        a single write() works outside Android, and neither of these 
        should be required.  An earlier flush() fix attempt is now moot.
        ------------------------------------------------------------------
        """

        trace(f'writing {icsfilename} in {icsfolder.toString()} ...')
        activity = self.app.get_android_activity()
        context  = self.app.get_android_context() 
        DocumentFile = autoclass('androidx.documentfile.provider.DocumentFile')
 
        documentDir = DocumentFile.fromTreeUri(context, icsfolder)
        if not documentDir:
            trace('No documentDir')
        else:
            documentFile = documentDir.findFile(icsfilename)
            if not documentFile:
                # may be first save for a new calendar
                trace(f'creating new ics file: {icsfilename}')
                documentFile = documentDir.createFile('text/calendar', icsfilename)

            if not documentFile:
                trace('No documentFile')
            else:
                # write as bytes via file descriptor
                stupid_saf_mode = 'rwt'                # see above
                icsfileuri = documentFile.getUri() 
                contentResolver = activity.getApplicationContext().getContentResolver()
                afd = contentResolver.openFileDescriptor(icsfileuri, stupid_saf_mode)
                if afd:
                    try:
                        fd = afd.detachFd()                                 # back in py (ahh)
                        fileobj = os.fdopen(fd, 'wb')                       # open+write can fail
                    except:
                        trace(f'{icsfilename=} skipped: {sys.exc_info()=}')
                    else:
                        try:
                            fileobj.write(icstext)                          # actually, it's bytes
                        except:
                            trace(f'{icsfilename=} skipped: {sys.exc_info()=}')
                        finally:
                            #fileobj.flush()                                # might matter for fd
                            fileobj.close()                                 # always close object



    def backups_get_subfolder(self, icsfolderUri):

        """
        ------------------------------------------------------------------
        Verify or create backups subfolder.
        -icsfolderUri here is a java Uri: not python str or DocumentFile.

        This and its 3 followers are called by super's common backups loop
        and implement the Android SAF/URI versions of the required ops.

        Returns a DocumentFile wrapping a SAF content URI or falsy py None,
        not a Uri or py str.  Could convert result to Uri with DF.getUri() 
        but that would require extra reconversions later and Uri not used.
         ------------------------------------------------------------------
        """

        trace(f'{icsfolderUri=}')
        context = self.app.get_android_context() 
        DocumentFile = autoclass('androidx.documentfile.provider.DocumentFile')
       
        documentDir = DocumentFile.fromTreeUri(context, icsfolderUri)
        if not documentDir:
            trace('No documentDir')
            return None
        else:
            try:
                backupDir = documentDir.findFile(BACKUPS_SUBDIR_NAME)
                if not backupDir:
                    trace('making backup dir')
                    backupDir = documentDir.createDirectory(BACKUPS_SUBDIR_NAME)
                trace('have backup dir')                 
                assert backupDir and backupDir.exists() and backupDir.isDirectory()
                return backupDir
            except:
                trace('backupDir make fail or already a non-dir')
                return None



    def backups_glob_dir(self, backupsSubfolderUri, backuppatt):

        """
        ------------------------------------------------------------------
	Returns a list of DocumentFile objects, for "*.ics" in Backups.
        - backupsSubfolderUri is a DocumentFile, from backups_get_subfolder.
        It wraps a Uri but isn't one itself, despite the early name.
        ------------------------------------------------------------------
        """

        trace(f'{backupsSubfolderUri=}')
        context = self.app.get_android_context()        
        DocumentFile = autoclass('androidx.documentfile.provider.DocumentFile')

        # vs glob.glob: SAF Java API is gross
        calslist = []
        ##documentsTree = DocumentFile.fromTreeUri(context, backupsSubfolderUri)
        documentsTree = backupsSubfolderUri

        if documentsTree:
            for documentFile in documentsTree.listFiles():
                if documentFile.isFile():
                    documentName = documentFile.getName()
                    if documentName and fnmatch.fnmatch(str(documentName), backuppatt):
                        calslist.append(documentFile)    # path implied, or .getUri()  
        return calslist



    def backups_remove_file(self, pruneeDoc):

        """
        ------------------------------------------------------------------
        Remove a file, the SAF way.
        - pruneeDoc is a DocumentFile, from backups_glob_dir.
        Or use a list of documentFile.getUri() results and:
        assert context.getContentResolver().delete(pruneeUri, None, None) > 0
        ------------------------------------------------------------------
        """

        assert pruneeDoc.delete()   # a DocumentFile



    def backups_backup_file(self, folderUri, icsfilename, backupsSubfolderUri, backupname):

        """
        ------------------------------------------------------------------
        Read+write calendars via os file descriptors fetched from SAF.
        - backupsSubfolderUri is a DocumentFile, from backups_get_subfolder.
        - folderUri is a java Uri (not py str) and requires a fromTreeUri.
        Vs joins + [shutil.copy2 or .read() + .write()]: SAF is horrific!
        TBD: copying file metadata like timestamps seem impossible in SAF?
        ------------------------------------------------------------------
        """

        activity = self.app.get_android_activity()
        context  = self.app.get_android_context()    
        DocumentFile = autoclass('androidx.documentfile.provider.DocumentFile')

        # read FROM any exception caught by caller
        documentDir = DocumentFile.fromTreeUri(context, folderUri)
        assert documentDir
        documentFileFrom = documentDir.findFile(icsfilename)
 
        # skip: new in-memory calendar that doesn't yet have a file  
        if not documentFileFrom:
            trace(f'no file to backup: {icsfilename}')
            return    

        else:
            trace(f'saving "{icsfilename}" to "{backupname}"')
            contentResolver = activity.getApplicationContext().getContentResolver()

            # read as text in py native code
            afd = contentResolver.openFileDescriptor(documentFileFrom.getUri(), 'r')
            assert afd
            fd = afd.detachFd()
            fileobj = os.fdopen(fd, 'r', encoding='utf8')
            fromtext = fileobj.read()
            fileobj.close()

            # create+write TO, any exception caught by caller
            ##documentDir = DocumentFile.fromTreeUri(context, backupsSubfolderUri)
            documentDir = backupsSubfolderUri
            assert documentDir
            documentFileTo = documentDir.createFile('text/calendar', backupname)
            assert documentFileTo

            # write as text in py native code ('rwt' not needed here)
            afd = contentResolver.openFileDescriptor(documentFileTo.getUri(), 'w')
            assert afd
            fd = afd.detachFd() 
            fileobj = os.fdopen(fd, 'w', encoding='utf8')
            fileobj.write(fromtext)
            fileobj.flush()
            fileobj.close()
