Eager Data Transfer Mechanism for Reducing Communication Latency in User-Level Network Protocols
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Clusters have become a popular alternative for building high-performance parallel computing systems. Today’s high-performance system area network (SAN) protocols such as VIA and IBA significantly reduce user-to-user communication latency by implementing protocol stacks outside of operating system kernel. However, emerging parallel applications require a significant improvement in communication latency. Since the time required for transferring data between host memory and network interface (NI) make up a large portion of overall communication latency, the reduction of data transfer time is crucial for achieving low-latency communication. In this paper, Eager Data Transfer (EDT) mechanism is proposed to reduce the time for data transfers between the host and network interface. The EDT employs cache coherence interface hardware to directly transfer data between the host and NI. An EDT-based network interface was modeled and simulated on the Linux-based, complete system simulation environment, Linux/SimOS. Our simulation results show that the EDT approach significantly reduces the data transfer time compared to DMA-based approaches. The EDT-based NI attains 17% to 38% reduction in user-to-user message time compared to the cache-coherent DMA-based NIs for a range of message sizes (64 bytes ~ 4 Kbytes) in a SAN environment.
Abstract
Clusters have become a popular alternative for building high-performance parallel computing systems. Today’s high-performance system area network (SAN) protocols such as VIA and IBA significantly reduce user-to-user communication latency by implementing protocol stacks outside of operating system kernel. However, emerging parallel applications require a significant improvement in communication latency. Since the time required for transferring data between host memory and network interface (NI) make up a large portion of overall communication latency, the reduction of data transfer time is crucial for achieving low-latency communication. In this paper, Eager Data Transfer (EDT) mechanism is proposed to reduce the time for data transfers between the host and network interface. The EDT employs cache coherence interface hardware to directly transfer data between the host and NI. An EDT-based network interface was modeled and simulated on the Linux-based, complete system simulation environment, Linux/SimOS. Our simulation results show that the EDT approach significantly reduces the data transfer time compared to DMA-based approaches. The EDT-based NI attains 17% to 38% reduction in user-to-user message time compared to the cache-coherent DMA-based NIs for a range of message sizes (64 bytes ~ 4 Kbytes) in a SAN environment.
참고문헌 (31)
- "Intel, Compaq and Microsoft Corporations, Virtual Interface Architecture Specification, Version 1.0" (1997)[URL]
- M. Banikazemi, "Design Alternatives for Virtual Interface Architecture (VIA) and an Implementation on IBM Netfinity NT Clusters", Proc. of the Int'l Parallel and Distributed Processing Symposium (2000)
- "M-VIA: A High Performance Modular VIA for Linux"[URL]
- "Infiniband Architecture Specification, Vol. 1"[URL]
- S.S. Mukherjee, "Making Network Interfaces Less Peripheral", IEEE Computer, 31(10), pp.70~76 (1998)
- S.S. Mukherjee, "Coherent network Interfaces for Fine-Grain Communication", Proceedings of the 23rd International Symposium on Computer Architecture (ISCA) (1996)
- S.S. Mukherjee, "The impact of Data Transfer and Buffering Alternatives on Network Interface Design", Proceedings of the 4th International Symposium on High-Performance Computer Architecture (HPCA) (1998)
- R.A.F. Bhoedjang, "Design Issues for User-Level Network Interface Protocols on Myrinet", IEEE Computer, 31(11), pp.53~60 (1998)
- R.A.F. Bhoedjang, "Reducing Data and Control Transfer Overhead through Network-Interface Support", First Myrinet User Group Conference (MUG) (2000)
- Won, C. ., "Linux", SimOS - A Simulation Environment for Evaluating High-Speed Communication Systems, 2005, ~
- H. Hellwagner, "Exploring the Performance of VI Architecture Communication Features in the Giganet Cluster LAN", Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications(PDPTA2000) (2000)
- F. Briggs, "The Intel 870 Family of Enterprise Chipsets", Proc. of the Hot Chips XIII (2001)
- "Pentium(R) Processor Family Developer’s Manual"[URL]
- M. Rosenblum, "Using the SimOS Machine Simulator to study Complex Computer Systems", ACM Transactions on Modeling and Computer Simulations, 7(1) (1997)
- J. R. Thorpe, "A Machine Independent DMA Framework for NetBSD", USENIX 1998 Annual Technical Conference (1998)
- "MIPS R1000 Microprocessor User’s Manual, Version 2.0"[URL]
- C. May, "The PowerPC Architecture: A Specification for a New Family of RISC Processors" (1994), Morgan Kaufmann Publishers, Inc.
- "Pentium Processor Family Developer’s Manual"[URL]
- Message Passing Interface Forum, "MPI: A Message Passing Interface Standard", The International Journal of Supercomputer Applications and High Performance Computing, 8, pp.994~
- "MVICH: MPI for Virtual Interface Architecture, Berkeley Lab"[URL]
- T. Mehlan, "Providing a High-Performance VIA-Module for LAM", MPI, 2004, ~
- M. Bertozzi, "Design of a VIA based communication protocol for LAM", MPI Suite, 2001, ~
- R. Dimitrov, "An efficient MPI implementation for Virtual Interface Architecture -- enabled cluster computing", Proc. of the 3rd MPI developer's and user's conference (1999)
- "MPICH-A Portable Implementation of MPI"[URL]
- "Second Version of MPICH"[URL]
- "MVAPICH: MPI for InfiniBand on VAPI Layer"[URL]
- H. Tezuka, "Pin-down Cache: A Virtual Memory Management Technique for Zero-copy Communication", 12th International Parallel Processing Symposium, Orlando, FL (1998)
- J. Liu, "Design and Implementation of MPICH2 over InfiniBand with RDMA Support", International Parallel and Distributed Processing Symposium(IPDPS 04) (2004)
- R. Grabner, "An MPICH2 Channel Device Implementation over VAPI on InfiniBand", Proc. of CAC'04, Workshop on Communication Architecture for Clusters held in conjunction with IPDPS 2004 (2004)
- N.J. Boden, "Myrinet: A gigabit-per-second local area network", IEEE Micro, 15(1), pp.29~36 (1995)