volatilityfoundation
Tentative fix for Issue #599. Fix `--clear-cache` functionality.
Fix issue #599:
- Delete all the cache files
- Make volshell reuses this functionality instead of re-implementing it.
I'm happy making volshell use the same functionality.
Wiping out all the cache files means that expensive schema checks (which are keyed against the schema hash) may need to be redone for all ISF files. I know a lot of people don't necessarily have jsonschema installed, but for those that do it's a serious time sync when running volatility. The linux/mac banners will be wiped out too, but these should be recalculated whenever there are changes to the available linux/mac ISF files. If they're not, I'd prefer to improve that mechanism, or increase the incomplete cache clear-out to include the banner files, rather than tinkering with the cache clearing one. That's why the cache clearing only cleared out data_ ones in the first place...
It sounds good. I saw other tickets where you have been working in cache improvements. I also noticed that, and that was the reason I needed to wipe the cache and ended up finding this issue, as the cache files are in the user home, if you have multiple working directories and different symbols on each [wd]/volatility3/symbols/ folder.. it seems the cache doesn't take the latest one per each kernel. For instance, if you clone the vol3 repo in a new directory, create a new ISF symbols file with some modification on it for a kernel version that was already cached, those changes are not used. As per my tests, the volatility instance running in this new working directory still uses the cached version from the older working directory. I know you can change the cache directory, but it would be nice that the cache take the time of these files into consideration. Hope that helps.
Ah, I see, so this isn't about the JSON files, it's about linux/mac ones in particular? The banner files are a dictionary of kernel banners each of which points to a list of ISF files that says it has that banner. File timestamps aren't really a good way of keep track of which is which, but we could alert when two files say they have the same banner? We really wouldn't expect an ISF for a particular banner to change, because we try not to work with partial ISF files. The timestamp also could change without the file path changing, which means we'd need to check the timestamp of every linux/mac ISF every time a plugin was run, which I think for such a corner case isn't ideal.
I might consider adding the banner files to the clear-cache option, but you'll likely still run into the issue of having two with the same banner and having to decide which to use. So for that I'll probably warn when there's more than one option for a particular image. There won't be a way of choosing which to use though...
I've ported over the sync between volshell and the CLI in commit c8dd8d0, but I'm still leaving the full clear out as never called (but present in the code incase someone wants to build off it). My hope is that #609 will resolve the issue, but give it a try and let me know. I'll leave this open until you let me know how the other PR works out...
#609 got merged, but this is probably still dependent on #630 as the means of resolution. I need to check how --clear-cache will work once we move over to sqlite3 banner caching. I need to think through the implications of where the schema verification lives.
Pretty sure this has gone stale since we moved to sqlite3, and volshell does now call to clear_cache. Closing it off.