openage icon indicating copy to clipboard operation
openage copied to clipboard

Create file watching subsystem

Open TheJJ opened this issue 10 years ago • 9 comments

Currently, the inotify support is pretty proof-of-concept only. It's directly integrated into the assetmanager, which should be abstracted in the future.

I'm thinking of creating a libopenage/watch/ subsystem which provides some sane api for registering a watch on a folder, file, etc with requested events.

When the watched events happen, generic callback functions should be called so we can use the watches for any file watching makes sense.

In the background, the "sane api" calls the plattform-specific kernel api, but openage can use just one uniform interface. This way we can support OS X, BSD, etc.

TheJJ avatar Mar 29 '15 14:03 TheJJ

This would be a step towards #203, and if the new implementation is somehow optimized, #260 will be resolved as well.

TheJJ avatar Mar 29 '15 14:03 TheJJ

Users should be able to request file-like objects from the asset system by specifying an asset path, and optionally pass a callback method that would be invoked when the file has changed.

The filewatch subsystem should provide FileWatcher objects that monitor a single file; I propose the following interface:

openage::watch::FileWatcher::FileWatcher(FSMonitor *monitor, const char *filename, function_ptr callback);

FileWatcher objects would invoke the callback whenever their file changes. FileWatcher may contain internal (non-exposed) optimizations that reduce the number of inotify watches. Optionally, a DirectoryWatcher may be exposed as well.

If a FileWatcher cannot be instantiated due to lack of OS support/exhausted watches/..., an exception shall be thrown.

All FileWatchers would be maintained in a FSMonitor object, which does event polling every engine tick, via a ::poll() or ::invoke_callbacks() member. The engine will have a FSMonitor as a member; the FSMonitor should have a noexcept constructor (i.e. work even ob platforms where no file watching is implemented).

Once the asset loader is extended to support data packs, it may use DirectoryWatchers to monitor whether an overriding asset has been created in a different datapack.

mic-e avatar Mar 29 '15 18:03 mic-e

https://code.google.com/p/simplefilewatcher/

franciscod avatar Mar 30 '15 01:03 franciscod

That code will certainly be helpful when it comes to writing the OSX and win32 code.

mic-e avatar Mar 30 '15 02:03 mic-e

Once #217 is done, we could use butter. If not using py, then at least for Linux we should switch to fanotify.

TheJJ avatar May 28 '15 11:05 TheJJ

done in #287, still using inotify though.

TheJJ avatar Aug 12 '15 05:08 TheJJ

the integration in #396 must still be done.

TheJJ avatar Oct 21 '15 10:10 TheJJ

look at dis https://pypi.python.org/pypi/watchdog

Supported Platforms

Linux 2.6 (inotify)
Mac OS X (FSEvents, kqueue)
FreeBSD/BSD (kqueue)
Windows (ReadDirectoryChangesW with I/O completion ports; ReadDirectoryChangesW worker threads)
OS-independent (polling the disk for directory snapshots and comparing them periodically; slow and not recommended)

franciscod avatar Jan 18 '16 16:01 franciscod

Inspiration for cross plattform support:

http://trac.wildfiregames.com/browser/ps/trunk/source/lib/sysdep/dir_watch.h http://trac.wildfiregames.com/browser/ps/trunk/source/lib/sysdep/os/win/wdir_watch.cpp

https://github.com/emcrisostomo/fswatch

TheJJ avatar Feb 01 '16 20:02 TheJJ