저장 시스템의 논리 파티션을 이용한 파일 중복 제거
File Deduplication using Logical Partition of Storage System
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
In traditional target-based data deduplication system, all of the files should be chunked and compared for reducing duplicated data blocks. One of the critical problem of this system arises as the number of files are increasing. The system suffers from computational delay for calculating hash value and processing metadata for handling each file. To overcome this problem, in this paper, we propose a novel data deduplication system using logical partition of storage system. The system applies data deduplication scheme to each logical partition not each file. Experiment result shows that the proposed system is more efficient compared with traditional deduplication scheme where the logical partition is full of files by 50% in terms of deduplication capacity and processing time.
Abstract
In traditional target-based data deduplication system, all of the files should be chunked and compared for reducing duplicated data blocks. One of the critical problem of this system arises as the number of files are increasing. The system suffers from computational delay for calculating hash value and processing metadata for handling each file. To overcome this problem, in this paper, we propose a novel data deduplication system using logical partition of storage system. The system applies data deduplication scheme to each logical partition not each file. Experiment result shows that the proposed system is more efficient compared with traditional deduplication scheme where the logical partition is full of files by 50% in terms of deduplication capacity and processing time.
참고문헌 (11)
- 최은정, "퍼스널 클라우드 서비스를 위한 임의의 단말간 컨텐츠 동기화 방법", 대한임베디드공학회논문지, 6(6), pp.377~382 (2011), 대한임베디드공학회
- D.T. Meyer, "A study of practical deduplication", Proceedings on the 9th USENIX conference on File and stroage technologies (FAST) (2011)
- S. Quinlan, "Venti: a new approach to archival storage", Proceedings on the FAST 2002 Conference on File and Storage Technologies, 4 (2002)
- A. Muthitacharoen, "A low-bandwidth network file system", ACM SIGOPS Operating System Review, 35(5), pp.174~187 (2001)
- S. Annapureddy, "Shark: Scaling file servers via cooperativecaching", Proceedings on the 2nd Symposiumon Networked Systems Design andImplementation (NSDI), pp.129~142 (2005)
- F. Douglis, "Application-specific Delta-encoding via Resemblance Detection", Proceedings on 2003 USENIX Technical Conference, pp.113~126 (2003)
- P. Kulkarni, "Redundancy Elimination Within Large Collections of Files", Proceedings on 2004 USENIX Technical Conference (2004)
- B. Zhu, "Avoiding the disk bottleneck in the data domain deduplication file system", Proceedings on the Seventh USENIX Conference on File and Storage Technologies (FAST), pp.269~282 (2008)
- A. Broder, "Network Applications of Bloom Filters: A Survey", Internet Mathematics, 1(4), pp.485~509 (2002)
- M. Lillibridge, "Sparse Indexing, Large Scale, Inline Deduplication Using Sampling and Locality", Proceedings on the Seventh USENIX Conference on File and Storage Technologies (FAST) 2009 (2009)
- D. Harnik, "Estimation of deduplication ratios in large data sets", Proceedings on IEEE 28th Symposium on Mass Storage Systems and Technologies, pp.1~11 (2012)