hive icon indicating copy to clipboard operation
hive copied to clipboard

HiveError: Cannot read, unknown typeId: 64. Did you forget to register an adapter?

Open Bringoff opened this issue 5 years ago • 22 comments

There are multiple issues with similar problems, but no actual solution or explanation.

Steps to Reproduce

  • Create an object to save in Hive
  • Register generated adapter
  • write list of objects to Hive
  • reload the app and try to read written objects

And I get an error HiveError: Cannot read, unknown typeId: 64. Did you forget to register an adapter?. But I don't have that much types registered. Adapters are registered in main function before Hive.initFlutter() (if I do init before registering adapters, it doesn't change anything).


#0      BinaryReaderImpl.read (package:hive/src/binary/binary_reader_impl.dart:322:11)
#1      BinaryReaderImpl.readList (package:hive/src/binary/binary_reader_impl.dart:202:17)
#2      BinaryReaderImpl.read (package:hive/src/binary/binary_reader_impl.dart:314:16)
#3      BinaryReaderImpl.readFrame (package:hive/src/binary/binary_reader_impl.dart:273:26)
#4      FrameHelper.framesFromBytes (package:hive/src/binary/frame_helper.dart:17:26)
#5      FrameIoHelper.framesFromFile (package:hive/src/io/frame_io_helper.dart:41:12)
<asynchronous suspension>
#6      StorageBackendVm.initialize (package:hive/src/backend/vm/storage_backend_vm.dart:82:30)
<asynchronous suspension>
#7      BoxBaseImpl.initialize (package:hive/src/box/box_base_impl.dart:90:20)
#8      HiveImpl._openBox (package:hive/src/hive_impl.dart:106:22)
<asynchronous suspension>
#9      HiveImpl.openBox (package:hive/src/hive_impl.dart:135:18)
#10     MasterDataCacheStorage._openBox (package:ic_camera_3/data/db/master_data_cache_storage.dart:47:34)
#11     MasterDataCacheStorage._retrieve (package:ic_camera_3/data/db/master_data_cache_storage.dart:40:25)
#12     MasterDataCacheStorage.getAgent (package:ic_camera_3/data/db/master_data_cache_storage.dart:17:37)
#13     MasterDataGateway.retrieveFavoriteStores (package:ic_camera_3/data/master_data_gateway.dart:81:29)
#14     HomeCubit.fetch (package:ic_camera_3/modules/home/cubit/home_cubit.dart:32:45)
#15     _StoreListPageState.initState (package:ic_camera_3/modules/home/pages/store_list_page.dart:37:16)
#16     StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4765:58)
#17     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4601:5)
#18     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3569:14)
#19     Element.updateChild (package:flutter/src/widgets/framework.dart:3327:18)
#20     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4652:16)
#21     Element.rebuild (package:flutter/src/widgets/framework.dart:4343:5)
#22     ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:4606:5)
#23     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4601:5)
#24     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3569:14)
#25     Element.updateChild (package:flutter/src/widgets/framework.dart:3327:18)
#26     SingleChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:6118:14)
#27     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3569:14)
#28     Element.updateChild (package:flutter/src/widgets/framework.dart:3327:18)
#29     SingleChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:6118:14)
#30     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3569:14)
#31     Element.updateChild (package:flutter/src/widgets/framework.dart:3327:18)
#32     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4652:16)
#33     Element.rebuild (package:flutter/src/widgets/framework.dart:4343:5)
#34     ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:4606:5)
#35     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4601:5)
#36     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3569:14)
#37     Element.updateChild (package:flutter/src/widgets/framework.dart:3327:18)
#38     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4652:16)
#39     Element.rebuild (package:flutter/src/widgets/framework.dart:4343:5)
#40     ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:4606:5)
#41     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4601:5)
#42     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3569:14)
#43     Element.updateChild (package:flutter/src/widgets/framework.dart:3327:18)
#44     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4652:16)
#45     StatefulElement.performRebuild (package:flutter/src/widgets/framework.dart:4800:11)
#46     Element.rebuild (package:flutter/src/widgets/framework.dart:4343:5)
#47     ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:4606:5)
#48     StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4791:11)
#49     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4601:5)
#50     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3569:14)
#51     Element.updateChild (package:flutter/src/widgets/framework.dart:3327:18)
#52     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4652:16)
#53     Element.rebuild (package:flutter/src/widgets/framework.dart:4343:5)
#54     ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:4606:5)
#55     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4601:5)
#56     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3569:14)
#57     Element.updateChild (package:flutter/src/widgets/framework.dart:3327:18)
#58     SliverMultiBoxAdaptorElement.updateChild (package:flutter/src/widgets/sliver.dart:1158:36)
#59     SliverMultiBoxAdaptorElement.createChild.<anonymous closure> (package:flutter/src/widgets/sliver.dart:1143:20)
#60     BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2683:19)
#61     SliverMultiBoxAdaptorElement.createChild (package:flutter/src/widgets/sliver.dart:1136:11)
#62     RenderSliverMultiBoxAdaptor._createOrObtainChild.<anonymous closure> (package:flutter/src/rendering/sliver_multi_box_adaptor.dart:350:23)
#63     RenderObject.invokeLayoutCallback.<anonymous closure> (package:flutter/src/rendering/object.dart:1883:59)
#64     PipelineOwner._enableMutationsToDirtySubtrees (package:flutter/src/rendering/object.dart:915:15)
#65     RenderObject.invokeLayoutCallback (package:flutter/src/rendering/object.dart:1883:14)
#66     RenderSliverMultiBoxAdaptor._createOrObtainChild (package:flutter/src/rendering/sliver_multi_box_adaptor.dart:339:5)
#67     RenderSliverMultiBoxAdaptor.addInitialChild (package:flutter/src/rendering/sliver_multi_box_adaptor.dart:423:5)
#68     RenderSliverFixedExtentBoxAdaptor.performLayout (package:flutter/src/rendering/sliver_fixed_extent_list.dart:197:12)
#69     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#70     RenderSliverEdgeInsetsPadding.performLayout (package:flutter/src/rendering/sliver_padding.dart:132:12)
#71     _RenderSliverFractionalPadding.performLayout (package:flutter/src/widgets/sliver_fill.dart:170:11)
#72     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#73     RenderViewportBase.layoutChildSequence (package:flutter/src/rendering/viewport.dart:507:13)
#74     RenderViewport._attemptLayout (package:flutter/src/rendering/viewport.dart:1561:12)
#75     RenderViewport.performLayout (package:flutter/src/rendering/viewport.dart:1470:20)
#76     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#77     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#78     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#79     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#80     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#81     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#82     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#83     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#84     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#85     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#86     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#87     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#88     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#89     MultiChildLayoutDelegate.layoutChild (package:flutter/src/rendering/custom_layout.dart:171:12)
#90     _ScaffoldLayout.performLayout (package:flutter/src/material/scaffold.dart:498:7)
#91     MultiChildLayoutDelegate._callPerformLayout (package:flutter/src/rendering/custom_layout.dart:243:7)
#92     RenderCustomMultiChildLayoutBox.performLayout (package:flutter/src/rendering/custom_layout.dart:402:14)
#93     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#94     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#95     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#96     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#97     _RenderCustomClip.performLayout (package:flutter/src/rendering/proxy_box.dart:1308:11)
#98     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#99     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#100    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#101    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#102    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#103    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#104    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#105    RenderStack.performLayout (package:flutter/src/rendering/stack.dart:560:15)
#106    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#107    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#108    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#109    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#110    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#111    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#112    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#113    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#114    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#115    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#116    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#117    RenderOffstage.performLayout (package:flutter/src/rendering/proxy_box.dart:3220:14)
#118    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#119    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:113:14)
#120    RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
#121    _RenderTheatre.performLayout (package:flutter/src/widgets/overlay.dart:685:15)
#122    RenderObject._layoutWithoutResize (package:flutter/src/rendering/object.dart:1634:7)
#123    PipelineOwner.flushLayout (package:flutter/src/rendering/object.dart:884:18)
#124    RendererBinding.drawFrame (package:flutter/src/rendering/binding.dart:436:19)
#125    WidgetsBinding.drawFrame (package:flutter/src/widgets/binding.dart:914:13)
#126    RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:302:5)
#127    SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:1117:15)
#128    SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:1055:9)
#129    SchedulerBinding._handleDrawFrame (package:flutter/src/scheduler/binding.dart:971:5)
#130    _rootRun (dart:async/zone.dart:1190:13)
#131    _CustomZone.run (dart:async/zone.dart:1093:19)
#132    _CustomZone.runGuarded (dart:async/zone.dart:997:7)
#133    _invoke (dart:ui/hooks.dart:251:10)
#134    _drawFrame (dart:ui/hooks.dart:209:3)

Code sample

@HiveType(typeId: TypeIds.store)
class StoreEntity extends Equatable {
  StoreEntity({
    this.uuid,
    this.serverId,
    this.customerId,
    this.name,
    this.taskUuids,
    this.address,
    this.latitude,
    this.longitude,
    this.isEnabled,
    this.description,
    this.retailChainName,
    this.logoUrl,
    this.routeKey,
    this.weekDays,
  });

  @HiveField(0)
  final String uuid;
  @HiveField(1)
  final int serverId;
  @HiveField(2)
  final String customerId;
  @HiveField(3)
  final String name;
  @HiveField(4)
  final List<String> taskUuids;
  @HiveField(5)
  final String address;
  @HiveField(6)
  final double latitude;
  @HiveField(7)
  final double longitude;
  @HiveField(8)
  final bool isEnabled;
  @HiveField(9)
  final String description;
  @HiveField(10)
  final String retailChainName;
  @HiveField(11)
  final String logoUrl;
  @HiveField(12)
  final String routeKey;
  @HiveField(13)
  final String weekDays;

  ...
}

...

Future<void> putStores(List<StoreEntity> stores) =>
      _store(_storeBoxKey, stores);

Future<void> _store(String key, dynamic value) async {
    try {
      final box = await _openBox();
      await box.put(key, value);
    } on HiveError catch (error) {
      throw StorageException('Error storing $value', cause: error);
    }
  }

Future<Box> _openBox() => Hive.openBox('some_key');

Version

  • Platform: iOS, Android
  • Flutter version: 1.22.5
  • Hive version: 1.4.4+1

Bringoff avatar Dec 18 '20 11:12 Bringoff

Hive adds 32 to your typeId (the first 32 are reserved for Hive types). So it might actually be typeId 32 that causes the problem?

wietsebuseyne avatar Apr 07 '21 09:04 wietsebuseyne

No, typeId was 5 here.

Bringoff avatar Apr 07 '21 15:04 Bringoff

Did you specify the generics parameters when registering the type adapters? Could you share the code where you are registering the adapters?

wietsebuseyne avatar Apr 07 '21 18:04 wietsebuseyne

The problem is it was 4 months ago and we already went away with SQLite. But as I remember we fixed that problem by declaring separate type CollectionOfStores with separate typeId and a single field that held List<StoreEntity>.

Bringoff avatar Apr 09 '21 06:04 Bringoff

I don't use object, I just use normal with key and value . and now i get that error too.

Kimsoer avatar Apr 13 '21 16:04 Kimsoer

Basically this issue happens when:

  1. Create an adapter with specific type id
  2. Write to the database (box) using that adapter
  3. Remove the adapter or change typeId

So when hive reads database, it detects that something was there with typeid of X and tries to deserialize (convert binary into runtime object) using registered adapter that's typeid is X. But hive couldn't found the adapter because it's removed / typeid is changed. So hive returns that error because it couldn't deserialize the data.

This part was written in Updating a class section of hive documentation.

To fix the issue you have to do one of the given solutions:

  • Register the previous adapter with the previous typeId
  • Remove the box file
  • Use Hive.ignoreTypeId<MyType>(X)

themisir avatar Apr 13 '21 17:04 themisir

@TheMisir basically, that’s not the case 🙂 I’ve described a situation that appears without touching typeId. Register an adapter for an object, and then put a list of these objects to a box with a single key.

Bringoff avatar Apr 13 '21 19:04 Bringoff

Oh, thanks for letting me know. I'll investigate it ASAP.

themisir avatar Apr 13 '21 20:04 themisir

I have the same issue I have 3 HiveType 1, 2 and 3 with their adapters type 2 contain one type 1 when I save one type 2 and restart the app throw me Unhandled Exception: HiveError: Cannot read, unknown typeId: 132. Did you forget to register an adapter? just saving 1 item not a list

oscarshaitan avatar Jun 04 '21 23:06 oscarshaitan

I think I have the same issue, I have several hive objects but it happens only in one that has the type List<String>. and not on the first or second run of the program (using dart native not flutter).

guyluz11 avatar Nov 01 '22 20:11 guyluz11

Hi, I Have the same issue too. In my case, I have a lot of types but it happens when I added a type which encapsulate a List<int> field. Before that I had a lots of type which have fields of this type or List<String> type.

E/flutter (25895): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: HiveError: Cannot read, unknown typeId: 73. Did you forget to register an adapter?

My typeId for this new type is 19. Hope this can help.

Thanks in advance.

Melkius avatar Jan 29 '23 13:01 Melkius

Having the same issue here

AliEasy avatar Aug 05 '23 13:08 AliEasy

Same here, attached the firebase log: image

I'm never used adapter in my code, the hive works fine for long time, but one day (today!) just stopped and I saw the crash in firebase.

here's my code snippet:

      AndroidOptions _getAndroidOptions() => const AndroidOptions(encryptedSharedPreferences: true);
      FlutterSecureStorage secureStorage = FlutterSecureStorage(aOptions: _getAndroidOptions());
      String? encryptionKeyString = await secureStorage.read(key: 'key');
      if (encryptionKeyString == null) {
        final key = Hive.generateSecureKey();
        await secureStorage.write(
          key: 'key',
          value: base64UrlEncode(key),
        );
      }

      String? key = await secureStorage.read(key: 'key');
      final encryptionKeyUint8List = base64Url.decode(key!);

      Directory dir = await getApplicationSupportDirectory();
      await Hive.initFlutter(dir.path);
      pref = await Hive.openBox<Object>(PREF_BOX, encryptionCipher: HiveAesCipher(encryptionKeyUint8List)); <---  (prefs.dart:60)

xOldeVx avatar Dec 18 '23 08:12 xOldeVx

Hi @simc, @themisir,

We're having exactly the same problem as @xOldeVx: we only have one box, encrypted, all keys with simple string values, no adapters. Many of our users get the error message "HiveError: Cannot read, unknown typeId: 120. Did you forget to register an adapter?" with different values for typeId. They say they can use the app for a while, then after a few uses they get the error. The issues have been raised on iOS as well as Android.

We use : hive: ^2.2.3 hive_flutter: ^1.1.0 flutter: ^3.7.12

Here's an extract from our code:

class CacheStorage implements Storage {
  dynamic _secureKey;
  final String _secureKeyKey = 'encrypted-key';
  late Box _box;

  late FlutterSecureStorage _secureStorage;

  AndroidOptions get _androidOptions => const AndroidOptions(encryptedSharedPreferences: true);
  IOSOptions get _iosOptions => const IOSOptions(accessibility: KeychainAccessibility.unlocked_this_device);

  Future<void> init() async {
    try {
      await Hive.initFlutter();
      _secureStorage = FlutterSecureStorage(aOptions: _androidOptions);
      await _ensureSecureKey();
      _box = await Hive.openBox('cache', encryptionCipher: _secureKey != null ? HiveAesCipher(_secureKey) : null);
    } catch (e, stack) {
      Sentry.captureException(e, stackTrace: stack);
    }
  }

  /// Retrieve or create an encryption key to secure the Hive box and its data.
  /// The key must be stored outside of the Hive's box but still securely, therefore flutter_secure_storage is used to cache it.
  Future<void> _ensureSecureKey() async {
    final existingKey = await _secureStorage.read(key: _secureKeyKey, aOptions: _androidOptions, iOptions: _iosOptions);
    if (existingKey != null) {
      _secureKey = base64Url.decode(existingKey);
    } else {
      _secureKey = Hive.generateSecureKey();
      await _secureStorage.write(key: _secureKeyKey, value: base64UrlEncode(_secureKey), aOptions: _androidOptions, iOptions: _iosOptions);
    }
  }

...

}

And here is the stack trace:

HiveError: HiveError: Cannot read, unknown typeId: 120. Did you forget to register an adapter?
  File "binary_reader_impl.dart", line 325, in BinaryReaderImpl.read
  File "binary_reader_impl.dart", line 342, in BinaryReaderImpl.readEncrypted
  File "binary_reader_impl.dart", line 278, in BinaryReaderImpl.readFrame
  File "frame_helper.dart", line 21, in FrameHelper.framesFromBytes
  File "frame_io_helper.dart", line 42, in FrameIoHelper.framesFromFile
  File "<asynchronous suspension>"
  File "storage_backend_vm.dart", line 86, in StorageBackendVm.initialize
  File "<asynchronous suspension>"
  File "hive_impl.dart", line 111, in HiveImpl._openBox
  File "<asynchronous suspension>"
  File "hive_impl.dart", line 142, in HiveImpl.openBox
  File "<asynchronous suspension>"
  File "cache_storage.dart", line 29, in CacheStorage.init
  File "<asynchronous suspension>"
  File "app.dart", line 48, in appSetup
  File "<asynchronous suspension>"
  File "main_prod.dart", line 28, in main.<fn>
  File "<asynchronous suspension>"
  File "<asynchronous suspension>"

Using Hive.deleteBoxFromDisk is not an option for us or users will have to reconnect their account again and again.

Please help us, thank you very much in advance for your time and your help.

camillealbert avatar Jan 05 '24 10:01 camillealbert

Did anyone solve this issue?

LaxmikanthMadhyastha avatar Mar 06 '24 14:03 LaxmikanthMadhyastha

In our case Cannot read, unknown typeId Issue was caused by keys that were longer than the allowed 256 chars (cause our code to get the keys was stupid). Cause length is only checked in assert, longer keys will weird behaviour in release build.

test0terter0n avatar Mar 06 '24 17:03 test0terter0n

I am getting this error as well, unexpectedly since 2 days now and fails my android app's startup sequence.

"HiveError: Cannot read, unknown typeId: 64. Did you forget to register an adapter?"

I have registered all my custom adapters. In my case, I have a Map<MyEnum, int> in my custom class that is throwing this error when it tried to read in the adapter code. MyEnum adapter is also registered and no change was made to all this code. It was working fine for months.

The problem began some time after I added a new field (a List<MyAnotherEnum>) in the custom class. It still breaks even if i undo the field. The weird thing is its breaking on Map<MyEnum, int> ! Not directly on the new field as any one would have expected. I have tried clearing up storage, uninstalling the app on my test device multiple times and even tried on a brand new emulator. I also tried changing the new field to just MyAnotherEnum instead of a List of the same. But it still fails on the Map<MyEnum, int> .

Any help would be appreciated. The alternative is to write a bunch of serialization deser code to workaround all these fields that were implemented with Maps and Lists, causing a lot of maintenance overhead.

This thread has been open since 2020 -- any hints/ideas/resolutions so far?

rvpzen avatar Mar 20 '24 20:03 rvpzen

--- Update: I tried converting the probably offending fields into standard types by flattening and string-ifying with custom code. Still getting the same error unfortunately. And now I have run out of ideas because its not clear what is causing this. Debugger doesnt seem to help trace anything much either.

@themisir , are you still involved in this project to help take a look?

rvpzen avatar Mar 21 '24 22:03 rvpzen

Ok, I may have figured out how to fix it for my case but not clear what the root cause is in the hive library. I have tested for android app on multiple devices now.

For this particular custom class I had to:

  • Remove all Maps, Lists etc and flatten them out into Strings
  • use only String, bool, simple enum in the @HiveField types
  • Rewrote my class TypeAdapter to not use any of the writeInt or readBool functions, but instead ONLY use the read() write() methods for all fields. I found no comments saying mixed types were not allowed.
  • moved my initFlutter to be after all my type adapters are registered (not sure how much this helped, but I hesitate to change it back 😢 )

Unpredictable behavior 😞 cost me days.

I dont know why, but this is working. I also dont know how it was working earlier when i used mixed types in this class. I also dont know when the other classes in this app will break since mixed types continue to work fine there.

I hope the above fix helps anyone else facing similar issues too. If anyone finds a real solution please post back.

rvpzen avatar Mar 22 '24 00:03 rvpzen

Rewrote my class TypeAdapter to not use any of the writeInt or readBool functions, but instead ONLY use the read() write() methods for all fields. I found no comments saying mixed types were not allowed.

My guess is this one fixed the problem. The problem usually occurs when the serializer is implemented wrongly or you change serializer implementation without considering backwards compatibility (you try to read a value that's not previously written). This is the case for both custom adapters and adapters generated using hive-generator.

Unfortunately this is one of the hard problems in terms of binary serialization, you just have to be careful. If you don't want to deal with it in any way, you can prefer to use databases with high level schemas: sqlite3, isar, and more I can't currently remember the name.

themisir avatar Mar 22 '24 12:03 themisir

Thanks @themisir ... As I pointed out earlier, the code has been working as expected for months. There was no change made to the existing fields or adapters. I added 1 new Enum field at the end and that broke the re-open Hive box path during app startup sequence. And reverting that enum also did not help anymore. So I doubt its because serializer was implemented the wrong way or was incompatible with anything there. I currently still have the flexibility to kill the DB and try out etc.. but not for long. So this makes me hesitate about using the lib in production. Based on the thread above there are cases where this happens and is usually with data structures like lists/maps. Its not about registering custom adapters.

This library so far has been a great experience for me to build my app and iterate fast.. It would be helpful if the library could offer better error messages, and info on what type is actually breaking. It took me over a day to realize the problem was not in my new field but was coming from an older unchanged field :) This kills dev productivity.

rvpzen avatar Mar 22 '24 17:03 rvpzen

I understand your concerns. It is just not really possible to provide any further details on the error message. At least the current standard for writing ".hive" files doesn't contain any debugging information or field size [^1] details in order for us to consistently read fields or provide helpful error messages. That has been done to reduce disk space (no need to store additional metadata) use and improve performance (we can just read fields sequentially without having to worry about additional metadata processing).

This is not to say it is totally users' fault when the above error happens. I remember there were an internal bug that was causing hive to misbehave in certain scenarios, so it could be possible that there's a similar bug, but without ability to properly reproduce the bug it is unlikely to be found.

Honestly I would suggest using a proper RDBMS like sqlite3 if you are intending to store complex data structures, and use hive for simple key value storage (ie: to store user preferences or other miscellaneous stuff). While that would increase development complexity, it would pay off on a long run.

[^1]: Hive has field size metadata written for dynamically sized fields (strings, lists, raw bytes, custom adapters). However, this is not enough to consistently read fields. For example if you change type of an int field into bool, hive will read 1 byte instead of 4. And then it will assume the next 3 bytes are part of the following field.

themisir avatar Mar 22 '24 19:03 themisir