Don't fear peculiar activation functions: EUAF and beyond.

 0 Người đánh giá. Xếp hạng trung bình 0

Tác giả: Feng-Lei Fan, Hengtao Guo, Qianchao Wang, Zhaoheng Xie, Dong Zeng, Tieyong Zeng, Shijun Zhang

Ngôn ngữ: eng

Ký hiệu phân loại:

Thông tin xuất bản: United States : Neural networks : the official journal of the International Neural Network Society , 2025

Mô tả vật lý:

Bộ sưu tập: NCBI

ID: 643223

In this paper, we propose a new super-expressive activation function called the Parametric Elementary Universal Activation Function (PEUAF). We demonstrate the effectiveness of PEUAF through systematic and comprehensive experiments on various industrial and image datasets, including CIFAR-10, Tiny-ImageNet, and ImageNet. The models utilizing PEUAF achieve the best performance across several baseline industrial datasets. Specifically, in image datasets, the models that incorporate mixed activation functions (with PEUAF) exhibit competitive test accuracy despite the low accuracy of models with only PEUAF. Moreover, we significantly generalize the family of super-expressive activation functions, whose existence has been demonstrated in several recent works by showing that any continuous function can be approximated to any desired accuracy by a fixed-size network with a specific super-expressive activation function. Specifically, our work addresses two major bottlenecks in impeding the development of super-expressive activation functions: the limited identification of super-expressive functions, which raises doubts about their broad applicability, and their often peculiar forms, which lead to skepticism regarding their scalability and practicality in real-world applications.
Tạo bộ sưu tập với mã QR

THƯ VIỆN - TRƯỜNG ĐẠI HỌC CÔNG NGHỆ TP.HCM

ĐT: (028) 36225755 | Email: tt.thuvien@hutech.edu.vn

Copyright @2024 THƯ VIỆN HUTECH