StrumentiResistenti
prevent file-managers from indexing when deleting "folder" under store before @
Would it be possible to make deleting folders under /store more difficult or to prevent file-managers from indexing all folders under /store? I accidentally deleted folders under /store a couple of times (thinking I would be under /tags)... Guess what happened: My CPU got really hot, because indexing all sub-folders and sub-files obviously takes incredible time, as there are faculty_of(number_of_tags) sub-folders, so e.g. for 50 tags, we would have 50! = 3,0414093201713 x 10^64 possible sub-folders (that's a 3 with 64 zeroes afterwards). Any other suggestion welcome
The +/ operator forces Tagsistant to list all the tags endlessly. I don't see a way to prevent an indexer or a file manager from scanning them again and again. Moreover, when you delete a directory inside store/, the file manager first lists all its contents and recursively apply the same approach to each entry. So, even excluding the +/ operator, deleting a tag would anyway cause the removal of the other tags.
Do you have a proposal to overcome this problem?
Maybe you can limit artificially for how many tags you can search simultaneously and how many tags you can attach to one file? For example 30. This could then be made a setting in ~/.tagsistant/repository.ini .
@conrad-heimbold why cannot you add the dir where tagsistant is mounted to ignore list of your indexer?
Like, in Plasma 5/KDE 5, from what I use Dolphin file manager, has the settings for that kind of stuff - it allows you to set which folders should be ignored when indexing is done.
@StrumentiResistenti @conrad-heimbold Limit makes sense too as a solution. I would say it can be a default, as long as there is clear error message when it's reached and it's clear where you can increase that if you need.
A limit on the tags shown in store/ is a limit for everyone: for the indexer you would like to not iterate endlessly, but also for the user composing its legitimate query. In my opinion, the user would prefer to be presented with all the available tags, not only of a selection arbitrarily chosen by Tagsistant to limit a recursive algorithm, because this could be misleading: are all the tags available shown to me? What about the tag I've just created? Why it does not show up?
The only way to really solve the recursive problem is to bring Tagsistant back to the traditional FS behaviour. When you list store/tag1/, you see it empty. You can then manually write another piece of the query, like in store/tag1/tag2/, still empty, and then store/tag1/tag2/@/: here are your results. In this way, when a file manager on an indexer hits store/, it just finds the list of the tags presented at depth 1, the way Tagsistant does the tags/ dir, but nothing less. However, wouldn't you perceive this behaviour as a worsening in Tagsistant usage experience for the human user while writing queries, running queries and tagging files?
Just let me add that disabling tag listing on incomplete queries (that's queries without the ending @/ operator) is quite easy and could be even bond to a command line argument. We can do this as a test.
@StrumentiResistenti I guess the @conrad-heimbold was referring to the "depth" of search. So, basically, with default limit of, let's say, 5 tags, you cannot search for 6 or more tags. This will effectively limit the depth of filestystem from the point of view of tools that are not aware of Tagsistant.
To be honest, I wouldn't consider this to be an issue of Tagsistant at all. I mean, tagsistant is not responsible for external indexing software and any indexing software should implement "ignore list" of some kind, because that's a VERY common issue - for instance, you might have an archive of some big open-source projects source files (kernel, let's say) and indexing that for file search is probably pretty much pointless, but may take a LOT of time and produce huge CPU loads.
Limit is just a possible workaround on Tagsistant side.
I see, seems like I've misunderstood @conrad-heimbold proposal. Thanks @ivan-kolmychek. Limiting the depth of the search instead of the width of the returned results is definitely better. However it does not solve the problem yet. If you limit the depth to 30 levels, and suppress the +/ operator, you'll anyhow get 30! possible combinations (performance will be affected the same way) and the whole tag space would be traversed because it would be listed at depth 2.
I agree with @ivan-kolmychek: limiting an indexer is not a Tagsistant issue. Any indexer should provide a blacklist mechanism for privacy and performance reasons. Think to sensitive information you may want to prevent from indexing or remotely mounted file systems (NFS, Samba, ...).
The other drawback (accidental deletion of files and tags from store/ by the user) is unfortunately very difficult to prevent, if not impossible. What the user perceives as a whole operation (the recursive deletion of a tag and its contents) is instead formed by a list of discrete operations that Tagsistant can't recognize as a deletion of the whole repository. Tagsistant is basically stateless. It manages every request as atomic and unrelated to other operations, the way a file system is supposed to behave. That's why I'm currently set to impossible to avoid.
@StrumentiResistenti that's why I specified something like 5 in depth limit. Basic use cases are still all right with that ("pictures" + "star_wars" + "starship", "photos" + "vaction" + "me" + "2016").
Anyway, indexing that will still produce a lot of CPU load given even moderate amount of tags because of combinatorial explosion. Also indexing tool will probably not understand that "star_wars" + "starship" is the same as "starship" + "star_wars".
So I would say that using indexing tool on Tagsistant is probably not a good idea and provides more problems than actual value, with or without the limit. Blacklisting may be the best way to deal with it.