Abstract
One of the most widely applied tasks in security and computer vision applications today is object detection, by which different categories of objects can be detected and located from images and videos. The advances in Deep Learning (DL) have presented us with new ways to detect objects and develop efficient mechanisms for object detection. The state-of-the-art research field shows that different activation functions in different layers and structures of the DL network significantly impact accuracy; however, they are mainly focused on selecting the most suitable activation functions for different layers from a local optimization perspective. In this work, we investigate the combination of activation functions across different layers from a global perspective as an optimization problem, applying orthogonal experiments to optimize the combined activation functions. Also, we design orthogonal matrices using different layers in the object detection deep learning model and several standard activation functions. We achieved a Mean Average Precision (mAP) of 83.34% on the PASCAL VOC 2007 dataset and 84.62% on the PASCAL VOC 2012 dataset. Experiments on two classical object detection datasets demonstrate that globally optimized results show significant improvements with statistical significance. The code has been made publicly available at https://github.com/54yc/OCA.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Diao, C., Zhang, D., Liang, W., Li, K.-C., Hong, Y., Gaudiot, J.-L.: A novel spatial-temporal multi-scale alignment graph neural network security model for vehicles prediction. IEEE Trans Intell Transp Syst 24(1), 904–914 (2023)
Long, J., Liang, W., Li, K.-C., Wei, Y., Marino, M.D.: A regularized cross-layer ladder network for intrusion detection in industrial internet of things. IEEE Trans Industr Inf 19(2), 1747–1755 (2023)
Hashemzadeh, M., Asheghi, B., Farajzadeh, N.: Content-aware image resizing: an improved and shadow-preserving seam carving method. Signal Process 155, 233–246 (2019)
Zhang, L., Xu, W., Shen, C., Huang, Y.: Vision-based on-road nighttime vehicle detection and tracking using improved hog features. Sensors 24(5), 1590 (2024)
Huang, X., Li, Z., Jin, Y., Zhang, W.: Fair-adaboost: extending adaboost method to achieve fair classification. Expert Syst Appl 202, 117240 (2022)
Zou, Z., Chen, K., Shi, Z., Guo, Y., Ye, J.: Object detection in 20 years: a survey. Proc IEEE 111(3), 257–276 (2023)
Asheghi, B., Salehpour, P., Khiavi, A.M., Hashemzadeh, M.: A comprehensive review on content-aware image retargeting: from classical to state-of-the-art methods. Signal Process 195, 108496 (2022)
Meng, X., Liang, W., Xu, Z., Li, K., Khan, M.K., Kui, X.: An anonymous authenticated group key agreement scheme for transfer learning edge services systems. ACM Trans Sen Netw 20(3), 1–23 (2024)
Lee, Y.-H., Kim, Y.: Comparison of CNN and yolo for object detection. J. Semicond. Disp. Technol. 19(1), 85–92 (2020)
Yadgar, G., Gabel, M., Jaffer, S., Schroeder, B.: SSD-based workload characteristics and their performance implications. ACM Trans Stor 17(1), 1–26 (2021)
Asheghi, B., Salehpour, P., Khiavi, A.M., Hashemzadeh, M., Monajemi, A.: Dasod: detail-aware salient object detection. Image Vis. Comput. 148, 105154 (2024)
Cheng, T., Song, L., Ge, Y., Liu, W., Wang, X., Shan, Y.: Yolo-world: Real-time open-vocabulary object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16901–16911 (2024)
Sivapriya, M., Suresh, S., Fathimal, P.M.: Aggregate channel features and fast RCNN for ship, iceberg classification (2023)
Xu, X., Zhao, M., Shi, P., Ren, R., He, X., Wei, X., Yang, H.: Crack detection and comparison study based on faster R-CNN and mask R-CNN. Sensors 22(3), 1215 (2022)
Bi, X., Hu, J., Xiao, B., Li, W., Gao, X.: Iemask R-CNN: Information-enhanced mask R-CNN. IEEE Trans. Big Data 9(2), 688–700 (2022)
Zhou, Z., Yu, X., Wang, X.: Object detection in drone video based on recurrent motion attention. Pattern Recogn. Lett. 183, 56–63 (2024)
Liang, W., Li, Y., Xie, K., Zhang, D., Li, K.-C., Souri, A., Li, K.: Spatial-temporal aware inductive graph neural network for c-its data recovery. IEEE Trans. Intell. Transp. Syst. 24(8), 8431–8442 (2023)
Hu, N., Zhang, D., Xie, K., Liang, W., Li, K., Zomaya, A.: Multi-graph fusion based graph convolutional networks for traffic prediction. Comput. Commun. 210, 194–204 (2023)
Yin, X., Goudriaan, J., Lantinga, E.A., Vos, J., Spiertz, H.J.: A flexible sigmoid function of determinate growth. Ann. Bot. 91(3), 361–371 (2003)
Kou, Y., Chen, Z., Chen, Y., Gu, Q.: Benign overfitting in two-layer relu convolutional neural networks. In: International Conference on Machine Learning, pp. 17615–17659 (2023). PMLR
Frei, S., Vardi, G., Bartlett, P., Srebro, N.: Benign overfitting in linear classifiers and leaky relu networks from kkt conditions for margin maximization. In: The Thirty Sixth Annual Conference on Learning Theory, pp. 3173–3228 (2023). PMLR
Singh, B., Patel, S., Vijayvargiya, A., Kumar, R.: Analyzing the impact of activation functions on the performance of the data-driven gait model. Results Eng. 18, 101029 (2023)
Lee, M.: Mathematical analysis and performance evaluation of the gelu activation function in deep learning. J. Math. 2023(1), 4229924 (2023)
Nawaz, M., Saleem, S., Masood, M., Rashid, J., Nazir, T.: COVID-ECG-RSNet: COVID-19 classification from ECG images using swish-based improved ResNet model. Biomed. Signal Process. Control 89, 105801 (2024)
Manessi, F., Rozza, A.: Learning combinations of activation functions. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 61–66 (2018). IEEE
Hu, X.-M., Zhang, J., Li, Y.: Orthogonal methods based ant colony search for solving continuous optimization problems. J. Comput. Sci. Technol. 23(1), 2–18 (2008)
Fernandes, A.M., Utkin, A.B., Chaves, P.: Automatic early detection of wildfire smoke with visible-light cameras and efficientdet. J. Fire Sci. 41(4), 122–135 (2023)
Chen, S., Sun, P., Song, Y., Luo, P.: Diffusiondet: Diffusion model for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19830–19843 (2023)
Ye, J., Wu, Y., Peng, D.: Low-quality image object detection based on reinforcement learning adaptive enhancement. Pattern Recogn. Lett. 182, 67–75 (2024)
Sharma, S., Sharma, S., Athaiya, A.: Activation functions in neural networks. Towards Data Sci. 6(12), 310–316 (2017)
Li, G., Liu, G., Jin, B., Wang, W., Fang, X.: A new artificial lateral line attitude perception method based on mixed activation function-multilayer perceptron (maf-mlp). Ocean Eng. 288, 116100 (2023)
Zhang, X., Chen, X., Yao, L., Ge, C., Dong, M.: Deep neural network hyperparameter optimization with orthogonal array tuning. In: Neural Information Processing: 26th International Conference, ICONIP 2019, Sydney, NSW, Australia, December 12–15, 2019, Proceedings, Part IV 26, pp. 287–295 (2019). Springer
Rankovic, N., Rankovic, D., Ivanovic, M., Lazic, L.: Cosmic fp method in software development estimation using artificial neural networks based on orthogonal arrays. Connect. Sci. 34(1), 185–204 (2022)
Jiaqiang, E., Zeng, Y., Jin, Y., Zhang, B., Huang, Z., Wei, K., Chen, J., Zhu, H., Deng, Y.: Heat dissipation investigation of the power lithium-ion battery module based on orthogonal experiment design and fuzzy grey relation analysis. Energy 211, 118596 (2020)
Tong, K., Wu, Y.: Rethinking PASCAL-VOC and MS-COCO dataset for small object detection. J. Vis. Commun. Image Represent. 93, 103830 (2023)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2016)
Wang, C.-Y., Bochkovskiy, A., Liao, H.-Y.M.: Yolov7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7464–7475 (2023)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liao, J., Yu, C., Jiang, L. et al. A method for composite activation functions in deep learning for object detection. SIViP 19, 362 (2025). https://doi.org/10.1007/s11760-025-03938-7
Received:
Revised:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1007/s11760-025-03938-7