Cache rewrite

CHECKSUMS.txt

@@ -0,0 +1 @@
+"file" "checksum" "filesize" "algorithm"

DESCRIPTION

@@ -19,8 +19,8 @@ SystemRequirements: 'unrar' (Linux/macOS) or '7-Zip' (Windows) to work with '.ra
 URL: 
     https://reproducible.predictiveecology.org,
     https://github.com/PredictiveEcology/reproducible
-Date: 2024-12-11
-Version: 2.1.2
+Date: 2025-01-13
+Version: 2.1.2.9004
 Authors@R: 
     c(person(given = "Eliot J B",
              family = "McIntire",
@@ -97,6 +97,8 @@ Roxygen: list(markdown = TRUE)
 RoxygenNote: 7.3.2
 Collate: 
     'DBI.R'
+    'messages.R'
+    'GPT2.R'
     'cache-helpers.R'
     'cache-internals.R'
     'robustDigest.R'
@@ -107,7 +109,6 @@ Collate:
     'convertPaths.R'
     'copy.R'
     'download.R'
-    'messages.R'
     'exportedMethods.R'
     'gis.R'
     'helpers.R'

NAMESPACE

@@ -48,6 +48,7 @@ export(.tagsByClass)
 export(.unwrap)
 export(.wrap)
 export(Cache)
+export(Cache2)
 export(CacheDBFile)
 export(CacheDBTableName)
 export(CacheDigest)
@@ -61,6 +62,7 @@ export(Filenames)
 export(asPath)
 export(assessDataType)
 export(basename2)
+export(cacheId)
 export(cc)
 export(checkAndMakeCloudFolderID)
 export(checkPath)
@@ -165,9 +167,11 @@ importFrom(data.table,rbindlist)
 importFrom(data.table,set)
 importFrom(data.table,setDF)
 importFrom(data.table,setDT)
+importFrom(data.table,setcolorder)
 importFrom(data.table,setindex)
 importFrom(data.table,setkeyv)
 importFrom(data.table,setnames)
+importFrom(data.table,setorderv)
 importFrom(digest,digest)
 importFrom(fpCompare,"%==%")
 importFrom(fs,is_absolute_path)

NEWS.md

@@ -1,3 +1,28 @@
+# reproducible 3.0.0
+
+* many edge cases were found that were not correctly Cached. This resulted in 2 major changes: 
+rewrite and simplification of `Cache` 
+and modified `digest` of the arguments. These changes are not backwards compatible. Details next.
+* near complete rewrite of `Cache` so it is simpler and more robust. 
+The main function is now 130 lines, instead of almost 700. 
+* In addition to full rewrites, numerous simplifications throughout code that is still being used;
+* Fixes to several ongoing "edge cases" that were difficult to address, mostly focused around deeply nested objects 
+that are file-backed with pointers, such `terra::SpatRaster` class;
+* `digest` changes include the following fixes:
+  - lists would be digested without their top-level names; thus two lists with 
+  exactly the same elements but with and without names would have the same digest
+  - file-backed objects were not correctly unique in the cache as they did not have
+  the `cacheId` in the filename; now they will have the 
+  `cacheId` prefixed on the file (so they sort alongside the main cache file). This 
+  `cacheId` prefix is removed on recovery from the cache, overwriting 
+  any files with the same name.
+  - extracting the functionName from a function had several edge cases did not work; these now work
+* `useMemoise` would work with file-backed objects, but only if the file-backed object 
+did not change after the caching (the pointer to the file was intact, but the file changed). 
+Now, memoising will copy file-backed information from disk
+each time it "retrieves a file-backed object from memeory". This will result in slower
+memoising than previously. However, it will be robust to downstream changes to the file.
+
 # reproducible 2.1.2
 
 * remove `PackedStatExtent` class, releasing it for `terra`; `reproducible`