superlinked
Movie Recommendation using VectorDB
Hi @vipul-maheshwari thank you for contributing.
Before I dive into this, I noticed you say "production grade" in the summary - what for you makes this example qualify for "production grade"?
Hi @vipul-maheshwari thank you for contributing.
Before I dive into this, I noticed you say "production grade" in the summary - what for you makes this example qualify for "production grade"?
Hey Hi @svonava-superlinked , Well the retrieval and the similarity checks with the genre embeddings makes it more robust in terms of the recommendation! If you think I need to change this, let me know!
Thanks for the requested changes @morkapronczay , I am going to update the branch as per the requested changes and feedback. Pinging you soon
Changes to the Article Outline
Hello @morkapronczay,
I am making some changes to the article outline for a few reasons:
-
Using embeddings from the hidden layer: Initially, we planned to use embeddings from the hidden layer for vector search. However, this approach seems too advanced given our basic neural network with only a couple of layers. The results may not be as effective as we hoped.
-
New approach: Instead of using embeddings, I will use the classified genres. I have already implemented pre-filtering with LanceDB. Combining the classified genres and pre-filtering will help us provide better recommendations.
-
Not using Movie overviews : Leaving overview part out of the training data as it turns out to be noise for the genre classification later.
Architectural Overview
Here is the updated architectural overview:
- Classified Genres: Use the neural network to classify genres for the given movie title.
- Pre-filtering with LanceDB: Use LanceDB to pre-filter the classified genres.
- Combine Both: Combine the results from the classified genres and pre-filtering to give recommendations.
This new approach is more effective for our current setup.
Looking goooood! 🚀 Thanks @robertdhayanturner @morkapronczay , we did this together! More to come