Basic IndexedDB Functionality

[rasviitanen]

This PR consists of basic functionality for IndexedDB. The database engine used is Rkv, and might not be the most optimal engine to use, so I created a pluggable engine. This makes it easy to change for eventual benchmarks etc. Recently, Rkv has raised warnings for using it in production, and the plan seems to be to change the backend to sqlite instead. If we expect that IndexedDB is finished after the issues in Rkv has been fixed, we should be able to use it after all. The ergonomics of Rkv is really good, and suits IndexedDB quite well.

What works:

• Basic Open
• Basic Put
• Basic Add
• Basic Get
• Basic Remove

Some things that don't work:

• Key generators
• Key ranges
• Aborting/Reverting transactions, including upgrades
• Closing/Removing a store/db
• Injection of keys
• Proper scheduling of transactions (they are started immediately atm)
• IDBIndex
• IDBCursor
• Manually committing transactions
• Some Error handling is missing
• ... lots of other stuff

Implementation details worth noting:

• Transactions run in a custom thread pool. read and write transactions use the same pool, which in the future probably should be separate.
• IndexedDB is disabled by default, as it has a lot of troubles

I am making this PR pre-prematurely, as it is part of a course I take in school (and it's soon due date), so I would be very happy if reviews can be done rather quickly.

---

• [X] ./mach build -d does not report any errors
• [X] ./mach test-tidy does not report any errors
• [X] These changes fix (PARTLY) #6963

• [X] These changes do not require tests because: There are many WebIDL tests that we can run to test the code. I was not able to run wpt tests with logging on windows, so test metadata is currently missing. I have, however, run some tests manually, and we can expect some of the tests to PASS. Some examples of passing tests: idbobjectstore_put.htm idbobjectstore_put2.htm idbobjectstore_put3.htm idbobjectstore_put5.htm idbobjectstore_put7.htm idbobjectstore_put9.htm, idbobjectstore_put10.htm idbobjectstore_put11.htm ... idbfactory_open.htm idbfactory_open2.htm idbfactory_open3.htm idbfactory_open4.htm ...

[highfive]

Thanks for the pull request, and welcome! The Servo team is excited to review your changes, and you should hear from @asajeffrey (or someone else) soon.

[highfive]

Heads up! This PR modifies the following files:

• @asajeffrey: components/script/dom/webidls/IDBTransaction.webidl, components/script/dom/idbrequest.rs, components/script/dom/mod.rs, components/script/dom/window.rs, components/script/dom/idbobjectstore.rs and 14 more
• @kichjang: components/script/dom/webidls/IDBTransaction.webidl, components/net/lib.rs, components/script/dom/idbrequest.rs, components/script/dom/mod.rs, components/net/indexeddb/mod.rs and 24 more

[highfive]

Warning

• These commits modify unsafe code. Please review it carefully!
• These commits modify net and script code, but no tests are modified. Please consider adding a test!

[jdm]

Please add a __dir__.ini to the indexeddb metadata directory to enable the new preference. This will make it possible for us to run the tests on CI and get the updated metadata you're looking for!

[jdm]

@bors-servo try=wpt

[bors-servo]

:hourglass: Trying commit f0626939c99be7e5b63a4814f1e018c7e8cc579a with merge e720b9808381616e378479ac1b1c796021d79751...

[bors-servo]

:broken_heart: Test failed - status-taskcluster

[jdm]

@bors-servo try=wpt

[bors-servo]

:hourglass: Trying commit afbe66e6e5dcfb8834a8b5d20f3cb6e396e30f90 with merge be4d861ba98897b1026dd06a5d0566764874a473...

[bors-servo]

:broken_heart: Test failed - status-taskcluster

[jdm]

Run wget https://community-tc.services.mozilla.com/api/queue/v1/task/P8Qk0aigQUKtiQRajoTYzg/runs/0/artifacts/public/test-wpt.log and then ./mach update-wpt test-wpt.log to update the metadata automatically. Don't forget to git add tests/wpt/metadata/IndexedDB to pick up any new ini files.

[rasviitanen]

Huh, those are not the test results I was expecting. Looking at the test harness and databases that are created for each test, it looks like we must be able to create multiple databases with different names.

dbname = (dbname ? dbname : "testdb-" + new Date().getTime() + Math.random() );

It looks like this doesn't work at the moment, and everything essentially operates on the same database. I assume this is why e.g. idbobjectstores_add.htm fails. This probably explains why I have been able to get a lot more passes when running one test at a time. I think I just forgot to add a call to Open and it shouldn't be too difficult to fix.

Hopefully, we'll get more sensical test results once it's fixed.

[jdm]

@bors-servo try=wpt

[bors-servo]

:hourglass: Trying commit ef8bbaece553a6bfdf6a94e64541bf25addfaa2f with merge 086a05b400bd327977df73e37eb675382254c495...

[bors-servo]

:broken_heart: Test failed - status-taskcluster

[jdm]

You can run wget https://community.taskcluster-artifacts.net/VZJOQ2QZT02QrGOjmc4Qvw/0/public/test-wpt.log followed by ./mach update-wpt test-wpt.log to automatically update the metadata for the indexeddb tests.

[jdm]

@bors-servo try=wpt

[bors-servo]

:hourglass: Trying commit eb0e848ab3b2170b1399eef56b73febab8a89cbb with merge 02002e86bca4f893628bf91ad7b2d6263a1e6d5e...

[rasviitanen]

Sorry, I forgot to push a line that fixed some flaky behavior. Hopefully, it is working as it should now :) let's see what bors says

[bors-servo]

:broken_heart: Test failed - status-taskcluster

[rasviitanen]

Now it looks much more like I expected, test metadata is now updated.

[jdm]

@bors-servo try=wpt

[bors-servo]

:hourglass: Trying commit 70cf65542ebcf105e7060393617fd5f4c1b87e99 with merge e01035455ade7b58c44182f8fe7166fbf8bfa916...

[bors-servo]

:broken_heart: Test failed - status-taskcluster

[jdm]

@bors-servo try=wpt

[bors-servo]

:hourglass: Trying commit 58ce3a72ac2f9465ac3e9c1e964a48192509a6ef with merge 092bf9a308e6d95c2df4119961a5f85dcae52f59...

[bors-servo]

:broken_heart: Test failed - status-taskcluster

[jdm]

16 unexpected results that are NOT known-intermittents:
  â–¶ OK [expected CRASH] /IndexedDB/blob-delete-objectstore-db.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/blob-delete-objectstore-db.any.worker.html:
  │ FAIL [expected PASS] Deleting an object store and a database containing blobs doesn't crash.
  └   → assert_unreached: deleteDatabase should succeed Reached unreachable code


  â–¶ OK [expected CRASH] /IndexedDB/blob-valid-before-commit.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/blob-valid-before-commit.any.worker.html:
  │ FAIL [expected PASS] Blobs can be read back before their records are committed.
  └   → assert_unreached: deleteDatabase should succeed Reached unreachable code


  â–¶ OK [expected CRASH] /IndexedDB/idb-explicit-commit-throw.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit-throw.any.worker.html:
  │ FAIL [expected PASS] Any errors in callbacks that run after an explicit commit will not stop the commit from being processed.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"


  â–¶ OK [expected CRASH] /IndexedDB/blob-valid-after-deletion.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/blob-valid-after-deletion.any.worker.html:
  │ FAIL [expected PASS] Blobs stay alive after their records are deleted.
  └   → assert_unreached: deleteDatabase should succeed Reached unreachable code


  â–¶ OK [expected CRASH] /IndexedDB/blob-contenttype.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/blob-contenttype.any.worker.html:
  │ FAIL [expected PASS] Ensure that content type round trips when reading blob data
  └   → assert_unreached: deleteDatabase should succeed Reached unreachable code


  â–¶ OK [expected CRASH] /IndexedDB/idbindex_keyPath.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/idbindex_keyPath.any.worker.html:
  │ FAIL [expected PASS] IDBIndex's keyPath attribute returns the same object.
  └   → assert_unreached: deleteDatabase should succeed Reached unreachable code


  â–¶ OK [expected CRASH] /IndexedDB/structured-clone-transaction-state.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/structured-clone-transaction-state.any.worker.html:
  │ FAIL [expected PASS] Transaction inactive during structured clone in IDBObjectStore.add()
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/structured-clone-transaction-state.any.worker.html:
  │ FAIL [expected PASS] Transaction inactive during structured clone in IDBObjectStore.put()
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/structured-clone-transaction-state.any.worker.html:
  │ FAIL [expected PASS] Transaction inactive during structured clone in IDBCursor.update()
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"


  â–¶ OK [expected CRASH] /IndexedDB/idbindex_reverse_cursor.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/idbindex_reverse_cursor.any.worker.html:
  │ FAIL [expected PASS] Reverse cursor sees update from separate transactions.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idbindex_reverse_cursor.any.worker.html:
  │ FAIL [expected PASS] Reverse cursor sees in-transaction update.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"


  â–¶ OK [expected CRASH] /IndexedDB/key-generators/reading-autoincrement-indexes-cursors.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-indexes-cursors.any.worker.html:
  │ FAIL [expected PASS] IDBIndex.openCursor() iterates over an index on the autoincrement key
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-indexes-cursors.any.worker.html:
  │ FAIL [expected PASS] IDBIndex.openKeyCursor() iterates over an index on the autoincrement key
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-indexes-cursors.any.worker.html:
  │ FAIL [expected PASS] IDBIndex.openCursor() iterates over an index not covering the autoincrement key
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-indexes-cursors.any.worker.html:
  │ FAIL [expected PASS] IDBIndex.openKeyCursor() iterates over an index not covering the autoincrement key
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"


  â–¶ OK [expected CRASH] /IndexedDB/key-generators/reading-autoincrement-store.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-store.any.worker.html:
  │ FAIL [expected PASS] IDBObjectStore.getAll() for an autoincrement store
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-store.any.worker.html:
  │ FAIL [expected PASS] IDBObjectStore.getAllKeys() for an autoincrement store
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-store.any.worker.html:
  │ FAIL [expected PASS] IDBObjectStore.get() for an autoincrement store
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"


  â–¶ OK [expected CRASH] /IndexedDB/key-generators/reading-autoincrement-indexes.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-indexes.any.worker.html:
  │ FAIL [expected PASS] IDBIndex.getAll() for an index on the autoincrement key
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-indexes.any.worker.html:
  │ FAIL [expected PASS] IDBIndex.getAllKeys() for an index on the autoincrement key
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-indexes.any.worker.html:
  │ FAIL [expected PASS] IDBIndex.get() for an index on the autoincrement key
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-indexes.any.worker.html:
  │ FAIL [expected PASS] IDBIndex.getAll() for an index not covering the autoincrement key
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-indexes.any.worker.html:
  │ FAIL [expected PASS] IDBIndex.getAllKeys() returns correct result for an index not covering the autoincrement key
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-indexes.any.worker.html:
  │ FAIL [expected PASS] IDBIndex.get() for an index not covering the autoincrement key
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"


  â–¶ OK [expected CRASH] /IndexedDB/idbcursor-request.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/idbcursor-request.any.worker.html:
  │ FAIL [expected PASS] cursor.request from IDBObjectStore.openCursor
  └   → assert_unreached: deleteDatabase should succeed Reached unreachable code

  â–¶ Unexpected subtest result in /IndexedDB/idbcursor-request.any.worker.html:
  │ FAIL [expected PASS] cursor.request from IDBObjectStore.openKeyCursor
  └   → assert_unreached: deleteDatabase should succeed Reached unreachable code

  â–¶ Unexpected subtest result in /IndexedDB/idbcursor-request.any.worker.html:
  │ FAIL [expected PASS] cursor.request from IDBIndex.openCursor
  └   → assert_unreached: deleteDatabase should succeed Reached unreachable code

  â–¶ Unexpected subtest result in /IndexedDB/idbcursor-request.any.worker.html:
  │ FAIL [expected PASS] cursor.request from IDBIndex.openKeyCursor
  └   → assert_unreached: deleteDatabase should succeed Reached unreachable code


  â–¶ OK [expected CRASH] /IndexedDB/idb-explicit-commit.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] Explicitly committed data can be read back out.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] commit() on a version change transaction does not cause errors.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] A committed transaction becomes inactive immediately.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] A committed transaction is inactive in future request callbacks.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] Puts issued after commit are not fulfilled.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] Calling commit on an aborted transaction throws.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] Calling commit on a committed transaction throws.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] Calling abort on a committed transaction throws and does not prevent persisting the data.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] Calling txn.commit() when txn is inactive should throw.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] Transactions with same scope should stay in program order, even if one calls commit.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] Transactions that explicitly commit and have errors should abort.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/idb-explicit-commit.any.worker.html:
  │ FAIL [expected PASS] Transactions that handle all errors properly should behave as expected when an explicit commit is called in an onerror handler.
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"


  â–¶ OK [expected CRASH] /IndexedDB/idbobjectstore_keyPath.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/idbobjectstore_keyPath.any.worker.html:
  │ FAIL [expected PASS] IDBObjectStore's keyPath attribute returns the same object.
  └   → assert_unreached: deleteDatabase should succeed Reached unreachable code


  â–¶ OK [expected CRASH] /IndexedDB/key-generators/reading-autoincrement-store-cursors.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-store-cursors.any.worker.html:
  │ FAIL [expected PASS] IDBObjectStore.openCursor() iterates over an autoincrement store
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/key-generators/reading-autoincrement-store-cursors.any.worker.html:
  │ FAIL [expected PASS] IDBObjectStore.openKeyCursor() iterates over an autoincrement store
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"


  â–¶ OK [expected CRASH] /IndexedDB/transaction-relaxed-durability.tentative.any.worker.html

  â–¶ Unexpected subtest result in /IndexedDB/transaction-relaxed-durability.tentative.any.worker.html:
  │ FAIL [expected PASS] Committed data can be read back out: case 0
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/transaction-relaxed-durability.tentative.any.worker.html:
  │ FAIL [expected PASS] Committed data can be read back out: case 1
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/transaction-relaxed-durability.tentative.any.worker.html:
  │ FAIL [expected PASS] Committed data can be read back out: case 2
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/transaction-relaxed-durability.tentative.any.worker.html:
  │ FAIL [expected PASS] Committed data can be read back out: case 3
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/transaction-relaxed-durability.tentative.any.worker.html:
  │ FAIL [expected PASS] Committed data can be read back out: case 4
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

  â–¶ Unexpected subtest result in /IndexedDB/transaction-relaxed-durability.tentative.any.worker.html:
  │ FAIL [expected PASS] Invalid durability option throws a TypeError
  └   → assert_equals: Expected success event, but got error event instead expected "success" but got "error"

Finally, a much more reasonable set of results!

[jdm]

@bors-servo r+

[bors-servo]

:pushpin: Commit fa8a94e has been approved by jdm