Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

6-2023

Abstract

Binarized Neural Networks (BNNs) are receiving increasing attention due to their lightweight architecture and ability to run on low-power devices, given the fact that they can be implemented using Boolean operations. The state-of-the-art for training classification BNNs restricted to few-shot learning is based on a Mixed Integer Programming (MIP) approach. This paper proposes the BeMi ensemble, a structured architecture of classification-designed BNNs based on training a single BNN for each possible pair of classes and applying a majority voting scheme to predict the final output. The training of a single BNN discriminating between two classes is achieved by a MIP model that optimizes a lexicographic multi-objective function according to robustness and simplicity principles. This approach results in training networks whose output is not affected by small perturbations on the input and whose number of active weights is as small as possible, while good accuracy is preserved. We computationally validate our model using the MNIST and Fashion-MNIST datasets using up to 40 training images per class. Our structured ensemble outperforms both BNNs trained by stochastic gradient descent and state-of-the-art MIP-based approaches. While the previous approaches achieve an average accuracy of on the MNIST dataset, the BeMi ensemble achieves an average accuracy of when trained with 10 images per class and when trained with 40 images per class.

Keywords

Binarized neural networks, Mixed-integer linear programming, Structured ensemble of neural networks

Discipline

Artificial Intelligence and Robotics | OS and Networks

Research Areas

Intelligent Systems and Optimization

Publication

Learning and Intelligent Optimization: 17th International Conference, LION, Nice, France, 4-8 June 2023: Proceedings

Volume

14286

First Page

443

Last Page

458

ISBN

9783031445057

Identifier

10.1007/978-3-031-44505-7_30

Publisher

Springer

City or Country

Cham

Additional URL

https://doi.org/10.1007/978-3-031-44505-7_30

Download

Share

COinS