Path-Based Computation Encoder for Neural Architecture Search

Abstract

Recently, neural architecture search (NAS) has received increasing attention as it can replace human experts indesigning the architecture of neural networks for different tasks and has achieved remarkable results in manychallenging tasks. In this study, a path-based computation neural architecture encoder (PCE) was proposed. Our PCE first encodes the computation of information on each path in a neural network, and then aggregatesthe encodings on all paths together through an attention mechanism, simulating the process of informationcomputation along paths in a neural network and encoding the computation on the neural network instead ofthe structure of the graph, which is more consistent with the computational properties of neural networks. Weperformed an extensive comparison with eight encoding methods on two commonly used NAS search spaces(NAS-Bench-101 and NAS-Bench-201), which included a comparison of the predictive capabilities of performancepredictors and search capabilities based on two search strategies (reinforcement learning-based andBayesian optimization-based) when equipped with different encoders. Experimental evaluation shows that PCEis an efficient encoding method that effectively ranks and predicts neural architecture performance, therebyimproving the search efficiency of neural architectures.