ibarrond
Implementing pyfhel with MNIST using simple CNN architecture.
Hello sir, I tried to implemet pyfhel to implement HE using bfv scheme in 5 layers simple CNN architecture using MNIST dataset, i managed to code the following: 1- 28 x 28 MNIST dataset is preprocessed by average pooling process and minimize to 14 x 14, then padding process takes a place (Minimize the number of trainable parameters).
2- Building CNN simple model using Keras API with MNIST dataset (accuracy approximately 98%)
3- Due to the HE atomic construction the model is built again from scratch using matrices (weights and biases are the same in keras model restored after being saved) and the exact Keras model accuracy is retrieved (accuracy approximately 98%).
4- The real values of input data and weights and biases are encoded (transformed to integers) since BFV scheme deals with integer values and the accuracy is degraded about 3%. Guessing this is acceptable since the values are now discrete not real as in Keras (accuracy about 95% -96%).
5- The encoded data is now encrypted using the BFV scheme through pyfhel package in python but the accuracy is now {9.5% to 10%} after it’s was approximately 95%.
6- The building, encoding, encryption are working, but the main issue a value (t) is large prime number and its (t-1) must be Divisable by 2^N where n is 8192 or could be 4096 the same size of batch size.
7- (t) value is large prime number for like 13 digits or even bigger.
The goal is to train the model over plaintext MNIST dataset, then classification is done over encrypted data.
The Problem: what is the right value of t that gives me higher accuracy and close to the original one?
I am having a lack of technical support and experiance in this matter.. Any ideas? thanks in advance.
Hi, I don't think you'll have much success with BFV here. Try to use the CKKS scheme, which supports a kind of "floating point like" approximate computation that works much better for machine learning.
@AlexanderViand Thanks for responding, I thought that at first but i found some code in GitHub https://github.com/MarzioMonticelli/python-cryptonet which i coulding manage to run properly, so i started based on analyzing and understanding the main ideas to build my own code. just today i found this published artical https://www.mdpi.com/2410-387X/6/3/34 using Pyfhel and bfv scheme but it works over medical images and larger image sizes than MNIST (Published July 1st), will investigate this one for sure. Still want to know is there some Helper function to give me initial paramter's estimation? i found this https://morfix.io/sandbox but not quite sure about it.
Following up on Alex's suggestion, this paper might help you out to implement a CKKS-based CNN: https://eprint.iacr.org/2018/1041
Closing this issue with all the provided links as references to check out. Feel free to open new issues for individual operations inside the CNN (e.g., convolution), and we will do our best to guide you implementing them with the latest version of Pyfhel. It now (v3.3.0+) supports Scalar product with simple semantics!:
https://github.com/ibarrond/Pyfhel/blob/5e0a456050a02f6d79d995d4d224aee688d31cb6/examples/Demo_7_ScalarProd.py#L171))