Failed on fetching historical feature values with get_historical_features python API

[tsenhungwu]

Expected Behavior

get_historical_features should return a joined result with feature view or views as expected but I am unable to see the results.

Current Behavior

get_historical_features throws out an error "ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]String or binary data would be truncated in table 'feature-store.dbo.feast_entity_df_3dada85cfda44050b3d22b896579ba2f', column 'customer_id'. Truncated value: 'D'. (2628) (SQLExecDirectW)")"

Steps to reproduce

1. Create an entity with "customer_id" as a string type.
2. Create a feature view that links to the above entity.
3. Feast apply in a shell script.
4. Execute get_historical_features.

Specifications

• Version: Python==3.8, feast==0.28.0
• Platform: Azure
• Subsystem: Both registry and offline store are Azure MSSQL and the online store is Azure cache for Redis

Possible Solution

1. I looked into the source code and if I understand the backend correctly, when a user triggers "get_historical_features", at a very high level, Feast executes three steps: 1) create a temp table 2) insert row(s) based on what was submitted in one's entity_df parameter 3) ready for point-in-time correctness join with feature view data.
2. Root cause was found post the temp table formation from step 1. For a string type of data or entity in my case, Feast transfers the data type into "varchar" without inferring its length. Therefore, from the DDL pov, SQL server would interpret it as varchar(1) and this is why steps 2 and 3 both failed afterward.
3. Proposed resolutions: In the file "type_map.py", function "pa_to_mssql_type" line 601 is the root cause. I would propose two solutions, either make a large number within varchar like varchar(max) or perform some dynamic length checks upfront. I kinda prefer the former as performing some additional checks would take more time and further delay the following operation.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[renziver]

This issue still persists. Hard to work with FEAST on Azure.