关键词: 小文件, 低延迟高可用, 分布式存储

Abstract: SNS (social networking services) and E-commerce services developed rapidly. Such services need store numerous small files like pictures, music files and macro blog texts. Traditional distributed storage systems, such as HDFS (Hadoop distributed file system), are designed for large files, which will have problems such as too much overhead with metadata and high latency when dealing with large number of small files. This paper analyzes the architecture and read-write flow of TFS (Taobao file system), and finds that TFS has to build several network connections when writing or reading a small file, which increases the read-write latency. Aiming at the challenge of storing numerous small files and the problems of TFS, this paper proposes SFFS (small-file file system), a low-latency high availability small-file-oriented distributed storage. The performance experiments show that the write latency of SFFS decreases 76.6%, and the read latency of SFFS decreases about 10% compared with TFS. SFFS also has a higher availability than TFS since the center node in SFFS has lighter load and can recover more quickly.

Key words: small file, low-latency high availability, distributed storage