kerNET

kerNET implements a modular training method for deep image classifiers. In addition, kerNET can be used for implementing radial basis function networks or as a helpful wrapper that simplifies the training of classifiers.

See Getting Started for how to use kerNET. See References for the papers that proposed this modular learning method.

Table of Contents

1. Installation
2. Testing
3. Getting Started
4. License
5. References

Installation

pip install -r requirements.txt
pip install .

# or if you want developer install
pip install -e .

Testing

We recommend using pytest for testing. To run the test suites with pytest, do

pip install pytest
pytest test/

Note that some of the tests are computationally intensive as they involve training/testing networks and should therefore be executed on GPUs.

Getting Started

kerNET is primarily for

• modular learning for deep classifiers.

In the case where the network is trained as two modules, our modular learning method amounts to (1) training the input module with a special objective function called a "proxy objective", and then (2) freezing the input module and training the output module with a usual classification loss such as cross-entropy. The optimality of this method has been proved in certain (pretty general) settings in our papers (see References).

kerNET is flexible. In addition to the main functionality, kerNET

• provides a memory-efficient implementation of radial basis function network;
• can be used as a lightweight wrapper for classifier training (modular or end-to-end) that gives you access to a flexible, powerful pipeline via a command line interface.

We currently support the following datasets and models.

• Datasets
  • MNIST
  • Fashion-MNIST
  • CIFAR-10
  • CIFAR-100
  • SVHN
• Models
  • Classic kernel method-based models
    • Radial basis function networks, kernel machines
  • Neural networks
    • Fully-connected networks
    • LeNet-5
    • ResNets
  • Kernel method-based connectionist models
    • Kernel networks

You can add dataset and model by modifying kernet/datasets and kernet/models, respectively.

License

©Copyright 2020 University of Florida Research Foundation, Inc. All rights reserved.

Licensed under the CC BY-NC-SA 4.0 license.

The code is released for academic research use only.

References

The modular learning method implemented here is from our following two papers. BibTeX entries available in links below.

• Modularizing Deep Learning via Pairwise Learning With Kernels (IEEE Transactions on Neural Networks and Learning Systems, to appear)
• On Kernel Method-Based Connectionist Models and Supervised Deep Learning Without Backpropagation (Neural Computation, 2020)