tfrecord
tfrecord copied to clipboard
vahidk
TFRecord reader for PyTorch
The first alteration fixes the bug in reading zipped tfrecords. The library computed the size of a zipped file instead of unarchived data, because of which only a part of...
Hey, I just added the functionality that you can pass a directory to the index creating method and an index file is created for every tfrecord in that directory.
Hi, Thanks for making this tool! I've got a warning message from pytorch when loading tfrecords using `MultiTFRecordDataset`: ``` UserWarning: The given NumPy array is not writeable, and PyTorch does...
if i use CVAT create tfrecord, when i read it will work, but some tfrecord, will got wrong. https://github.com/vahidk/tfrecord/blob/5034c32f5acb91d72d5707e99521b6bf0aea1ca3/tfrecord/reader.py#L121 then i find sometime it (value) will be a empty list,...
It would be great if writer supports generating compressed gzip TFRecords!
reader now supports .send() syntax for specifying index, MapDataset sets up iterator and uses it with .send() retry of https://github.com/vahidk/tfrecord/pull/96 I added .send() functionality to reader to let it seek...
python3 -m tfrecord.tools.tfrecord2idx ./test_tfrecord ./index_file/ /home/star/anaconda3/envs/pytorch_hjf/lib/python3.9/runpy.py:127: RuntimeWarning: 'tfrecord.tools.tfrecord2idx' found in sys.modules after import of package 'tfrecord.tools', but prior to execution of 'tfrecord.tools.tfrecord2idx'; this may result in unpredictable behaviour warn(RuntimeWarning(msg)) problem...
Hey @vahidk – thanks a ton for this library! I lean on tfrecord in several projects, but ran into an interoperability snag whenever another dependency (e.g. 🤗 Transformers, Keras, TF-Agents)...
