[DynamoDB online store] Disable entity key serialization and allow the entity_id to be configurable

[Mustahsan1]

Hi,

we have DynamoDb online store and we are using DynamoDBOnlineStoreConfig, where we want to get online features from, but the problem is that it does not allows the custom primary keys to be used, and it always uses the entity_id as the key and also hashes the value as well. Is there a way to change this behavior?

[franciscojavierarceo]

can you elaborate on what you are doing? Maybe show examples of your feature view and retrieval code?

[mozupwork]

@franciscojavierarceo thanks for your reply.

here's our Feature View definition:

guided_search_freelancer_fv = FeatureView(
    name="guided_search_freelancer",
    description="Feature view for guided search freelancer profiles",
    entities=[guided_search_freelancer_entity],
    schema=[
        Field(name="CONTRACTOR_UID", dtype=String),
        Field(name="L3_UIDS", dtype=Array(String))
    ],
    source=guided_search_freelancer_source,
    ttl=timedelta(days=365),
    online=True,
) 

entity:

guided_search_freelancer_entity = Entity(name="guided_search_freelancer_entity", join_keys=["CONTRACTOR_UID"], value_type=ValueType.STRING)

and we try to fetch features from the online server using the rest endpoint.

curl --location 'https://<host>/get-online-features' \
--header 'Content-Type: application/json' \
--data '{
  "entities": {
    "CONTRACTOR_UID": ["1888589087087573111"]
  },
  "features": [
    "guided_search_freelancer:L3_UIDS"
  ]
}'

but the response is an error:

"An error occurred (ValidationException) when calling the BatchGetItem operation: The provided key element does not match the schema"

its because the feast online server sends the request to dynamodb like this:

'body': b'{"RequestItems": {"ddb-feast-fs-staging-guided_search_freelancer": {"Keys": [{"entity_id": {"S": "4eb6acf3158080aab7df0c44358dae16"}}], "ConsistentRead": false}}}'

so it uses the entity_id for the key and also serializes the value, while in DynamoDB the data is put in plain format without serialization

[mozupwork]

this is our feature.yaml for online-server:

project: feast-umx
provider: local
registry:
  registry_type: remote
  path: <path>
online_store:
  type: dynamodb
  region: us-west-2
  table_name_template: ddb-feast-fs-staging-{table_name}
entity_key_serialization_version: 3
auth:
  type: kubernetes

[nanohanno]

Could you solve the issue? To me, it seems the entity should just be CONTRACTOR_ID in your case. You do not have to add it as a Field to the schema if that is basically the key of the feature, right? Or do you want to have the the key named CONTRACTOR_ID instead of entity_id? If you use the Feast API to write features into the DynamoDB table, it takes care of value serialization for you.