Filer Stores-SeaweedFS Wiki-面试哥

Comparing Storage Options
- Switching between different Stores
Extending Storage Options

Comparing Storage Options

The Filer Store persists all file metadata and directory information.

Filer Store Name	Lookup	number of entries in a folder	Scalability	Renaming	TTL	Note
memory	O(1)	limited by memory	Local, Fast			for testing only, no persistent storage
leveldb	O(logN)	unlimited	Local, Very Fast			Default, fairly scalable
leveldb2	O(logN)	unlimited	Local, Very Fast, faster than leveldb			Similar to leveldb, part of the lookup key is 128bit MD5 instead of the long full file path
Redis	O(1)	limited	Local or Distributed, Fastest		Yes	one directory's sub file names are stored in one key~value entry
Cassandra	O(logN)	unlimited	Local or Distributed, Very Fast		Yes
MySql	O(logN)	unlimited	Local or Distributed, Fast	Atomic		Easy to manage
Postgres	O(logN)	unlimited	Local or Distributed, Fast	Atomic		Easy to manage
MemSql	O(logN)	unlimited	Distributed, Fast	Atomic		Scalable
TiDB	O(logN)	unlimited	Distributed, Fast	Atomic		Scalable
CockroachDB	O(logN)	unlimited	Distributed, Fast	Atomic		Scalable
Etcd	O(logN)	~10GB	Distributed, 10,000 writes/sec			No SPOF. High Availability.
TiKV	O(logN)	unlimited	Local or Distributed, Fast	Atomic		Easy to manage

Switching between different Stores

It is easy to switch between different filer stores.

For example:

# first save current filer meta data
 
$ weed shell
> fs.cd   http://filerHost:filerPort/
> fs.meta.save
...
total 65 directories, 292 files
meta data for http://localhost:8888/ is saved to localhost-8888-20190417-005421.meta
> exit
 
# now switch to a new filer, and load the previously saved metadata
$ weed shell
> fs.meta.load localhost-8888-20190417-005421.meta
...
total 65 directories, 292 files
localhost-8888-20190417-005421.meta is loaded to http://localhost:8888/

Extending Storage Options

For any new storage option, please implement the FilerStore interface. It should be fairly straight forward to implement. Welcome to contribute back.