FastdDFS入门一环境搭建

1.1、依赖搭建

安装FastDFS服务需要安装四个软件：

libfastcommon：fastDFS与FastDHT中提取的公用的C函数的库。
fastdfs： fastDFS软件
fastdfs-nginx-module：当前主流web服务器的扩展模块(包括nginx/apache)，其用意在于利用web服务器直接对本机storage数据文件提供http服务，以提高文件下载的性能
nginx服务器

因为fastDFS是C语言编写的，因此需要fastDFS需要C语言的编译环境。首先需要安装编译环境

yum -y install cmake make gcc-c++

FastDFS还依赖一个libevent库，因此还需要安装libevent

yum -y install libevent

1.2、libfastcommon安装

把所有的fastDFS都安装到/usr/local/fastdfs目录下：

mkdir /usr/local/fastdfs

把解压的libfastcommon拷贝到/usr/local/fastdfs目录下

cd /usr/local/fastdfs/libfastcommon-master
./make.sh
./make install
?

执行完上述命令之后，会打印如下内容：

mkdir -p /usr/lib64
mkdir -p /usr/lib
mkdir -p /usr/include/fastcommon
install -m 755 libfastcommon.so /usr/lib64
install -m 644 common_define.h hash.h chain.h logger.h base64.h shared_func.h pthread_func.h ini_file_reader.h _os_define.h sockopt.h sched_thread.h http_func.h md5.h local_ip_func.h avl_tree.h ioevent.h ioevent_loop.h fast_task_queue.h fast_timer.h locked_timer.h process_ctrl.h fast_mblock.h connection_pool.h fast_mpool.h fast_allocator.h fast_buffer.h skiplist.h multi_skiplist.h flat_skiplist.h skiplist_common.h system_info.h fast_blocked_queue.h php7_ext_wrapper.h id_generator.h char_converter.h char_convert_loader.h common_blocked_queue.h multi_socket_client.h skiplist_set.h uniq_skiplist.h fc_list.h locked_list.h json_parser.h buffered_file_writer.h server_id_func.h fc_queue.h sorted_queue.h fc_memory.h shared_buffer.h thread_pool.h fc_atomic.h array_allocator.h sorted_array.h /usr/include/fastcommon

其作用就是把libfastcommon.so 安装到/usr/lib64目录下，并把一些头文件安装到 /usr/include/fastcommon目录下。

因此FastDFS引用的库的目录是/usr/local/lib，因此需要把之前安装的主库中的一些库目录做软链接，或者拷贝到/usr/local/lib目录下。

ln -s  /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so

1.3、FastDFS安装

把下载的后的安装包解压到/usr/local/fastdfs目录

cd /usr/local/fastdfs/fastdfs-master

在fastDFS目录下的make.sh中有如下的代码：

ENABLE_STATIC_LIB=0
ENABLE_SHARED_LIB=1
TARGET_PREFIX=$DESTDIR/usr
TARGET_CONF_PATH=$DESTDIR/etc/fdfs
TARGET_SYSTEMD_PATH=$DESTDIR/usr/lib/systemd/system

TARGET_PREFIX属性是用来指定fastDFS的默认安装路径，在集群情况下不要修改，在单节点情况下可以修改。

改成

TARGET_PREFIX=$DESTDIR/usr/local

在源码目录下执行:

./make.sh
./make.sh install

FastDFS的安装位置：

/usr/local/bin是可执行文件所在的位置，默认在/usr/bin中
/etf/fdfs 配置文件所在位置，默认也是这个位置
/usr/local/lib64主程序代码所在位置，默认在/usr/bin中
/usr/local/include/fastdfs包含的一些插件组所在的位置，默认在/usr/include/fastdfs中。

在之前解压源码的目录/usr/local/fastdfs/目录中的主程序中的init.d目录下有如下两个文件：

-rwxr-xr-x. 1 root root 961 9月  14 10:40 fdfs_storaged
-rwxr-xr-x. 1 root root 963 9月  14 10:40 fdfs_trackerd

上述两个脚本是启动服务脚本。如果在安装过程中没有把这两个脚本安装到/etc/init.d/目录下，则把源码中的这两个脚本拷贝到/etc/init.d/目录下。

在/etc/fdfs/目录下的配置文件如下：

-rw-r--r--. 1 root root  1909 11月 23 23:55 client.conf
-rw-r--r--. 1 root root 10246 11月 23 23:55 storage.conf
-rw-r--r--. 1 root root   620 11月 23 23:55 storage_ids.conf
-rw-r--r--. 1 root root  9138 11月 23 23:55 tracker.conf

如果当前安装节点作为storage节点，就修改storage.conf配置文件。

如果当前节点作为tracker节点，就修改tracker.conf配置文件。但是注意备份。

在/usr/local/bin目录下的命令就是fastdfs的命令

-rwxr-xr-x. 1 root root  369360 11月 23 23:55 fdfs_appender_test
-rwxr-xr-x. 1 root root  369144 11月 23 23:55 fdfs_appender_test1
-rwxr-xr-x. 1 root root  356064 11月 23 23:55 fdfs_append_file
-rwxr-xr-x. 1 root root  355448 11月 23 23:55 fdfs_crc32
-rwxr-xr-x. 1 root root  356088 11月 23 23:55 fdfs_delete_file
-rwxr-xr-x. 1 root root  356824 11月 23 23:55 fdfs_download_file
-rwxr-xr-x. 1 root root  356752 11月 23 23:55 fdfs_file_info
-rwxr-xr-x. 1 root root  376472 11月 23 23:55 fdfs_monitor
-rwxr-xr-x. 1 root root  356312 11月 23 23:55 fdfs_regenerate_filename
-rwxr-xr-x. 1 root root 1298008 11月 23 23:55 fdfs_storaged
-rwxr-xr-x. 1 root root  379240 11月 23 23:55 fdfs_test
-rwxr-xr-x. 1 root root  378456 11月 23 23:55 fdfs_test1
-rwxr-xr-x. 1 root root  522640 11月 23 23:55 fdfs_trackerd
-rwxr-xr-x. 1 root root  357016 11月 23 23:55 fdfs_upload_appender
-rwxr-xr-x. 1 root root  358032 11月 23 23:55 fdfs_upload_file

1.4、Tracker配置

FastDFS的配置文件默认在/etc/fdfs/目录下：

-rw-r--r--. 1 root root  1909 11月 23 23:55 client.conf
-rw-r--r--. 1 root root 10246 11月 23 23:55 storage.conf
-rw-r--r--. 1 root root   620 11月 23 23:55 storage_ids.conf
-rw-r--r--. 1 root root  9144 11月 24 22:04 tracker.conf

client.conf配置客户端

storage.conf配置存储节点

tracker.conf配置追踪节点

tracker.conf

安装完成FastDFS之后，如果配置tracker.conf配置文件，则此节点未tracker，如果配置了storage.conf，则此节点为storage.

在tracker.conf的配置文件中的配置如下；

# is this config file disabled
# false for enabled
# true for disabled
disabled = false
?
# bind an address of this host
# empty for bind all addresses of this host
bind_addr =
?
# the tracker server port
port = 22122
?
# connect timeout in seconds
# default value is 30
# Note: in the intranet network (LAN), 2 seconds is enough.
connect_timeout = 5
?
# network timeout in seconds for send and recv
# default value is 30
network_timeout = 60
?
# the base path to store data and log files
base_path = /home/yuqing/fastdfs
?
# max concurrent connections this server support
# you should set this parameter larger, eg. 10240
# default value is 256
max_connections = 1024
?
# accept thread count
# default value is 1 which is recommended
# since V4.07
accept_threads = 1
?
# work thread count
# work threads to deal network io
# default value is 4
# since V2.00
work_threads = 4
?
# the min network buff size
# default value 8KB
min_buff_size = 8KB
?
# the max network buff size
# default value 128KB
max_buff_size = 128KB
?
# the method for selecting group to upload files
# 0: round robin
# 1: specify group
# 2: load balance, select the max free space group to upload file
store_lookup = 2
?
# which group to upload file
# when store_lookup set to 1, must set store_group to the group name
store_group = group2
?
# which storage server to upload file
# 0: round robin (default)
# 1: the first server order by ip address
# 2: the first server order by priority (the minimal)
# Note: if use_trunk_file set to true, must set store_server to 1 or 2
store_server = 0
?
# which path (means disk or mount point) of the storage server to upload file
# 0: round robin
# 2: load balance, select the max free space path to upload file
store_path = 0
?
# which storage server to download file
# 0: round robin (default)
# 1: the source storage server which the current file uploaded to
download_server = 0
?
# reserved storage space for system or other applications.
# if the free(available) space of any stoarge server in 
# a group <= reserved_storage_space, no file can be uploaded to this group.
# bytes unit can be one of follows:
### G or g for gigabyte(GB)
### M or m for megabyte(MB)
### K or k for kilobyte(KB)
### no unit for byte(B)
### XX.XX% as ratio such as: reserved_storage_space = 10%
reserved_storage_space = 20%
?
#standard log level as syslog, case insensitive, value list:
### emerg for emergency
### alert
### crit for critical
### error
### warn for warning
### notice
### info
### debug
log_level = info
?
#unix group name to run this program, 
#not set (empty) means run by the group of current user
run_by_group=
?
#unix username to run this program,
#not set (empty) means run by current user
run_by_user =
?
# allow_hosts can ocur more than once, host can be hostname or ip address,
# "*" (only one asterisk) means match all ip addresses
# we can use CIDR ips like 192.168.5.64/26
# and also use range like these: 10.0.1.[0-254] and host[01-08,20-25].domain.com
# for example:
# allow_hosts=10.0.1.[1-15,20]
# allow_hosts=host[01-08,20-25].domain.com
# allow_hosts=192.168.5.64/26
allow_hosts = *
?
# sync log buff to disk every interval seconds
# default value is 10 seconds
sync_log_buff_interval = 1
?
# check storage server alive interval seconds
check_active_interval = 120
?
# thread stack size, should >= 64KB
# default value is 256KB
thread_stack_size = 256KB
?
# auto adjust when the ip address of the storage server changed
# default value is true
storage_ip_changed_auto_adjust = true
?
# storage sync file max delay seconds
# default value is 86400 seconds (one day)
# since V2.00
storage_sync_file_max_delay = 86400
?
# the max time of storage sync a file
# default value is 300 seconds
# since V2.00
storage_sync_file_max_time = 300
?
# if use a trunk file to store several small files
# default value is false
# since V3.00
use_trunk_file = false 
?
# the min slot size, should <= 4KB
# default value is 256 bytes
# since V3.00
slot_min_size = 256
?
# the max slot size, should > slot_min_size
# store the upload file to trunk file when it's size <=  this value
# default value is 16MB
# since V3.00
slot_max_size = 1MB
?
# the alignment size to allocate the trunk space
# default value is 0 (never align)
# since V6.05
# NOTE: the larger the alignment size, the less likely of disk
#       fragmentation, but the more space is wasted.
trunk_alloc_alignment_size = 256
?
# if merge contiguous free spaces of trunk file
# default value is false
# since V6.05
trunk_free_space_merge = true
?
# if delete / reclaim the unused trunk files
# default value is false
# since V6.05
delete_unused_trunk_files = false
?
# the trunk file size, should >= 4MB
# default value is 64MB
# since V3.00
trunk_file_size = 64MB
?
# if create trunk file advancely
# default value is false
# since V3.06
trunk_create_file_advance = false
?
# the time base to create trunk file
# the time format: HH:MM
# default value is 02:00
# since V3.06
trunk_create_file_time_base = 02:00
?
# the interval of create trunk file, unit: second
# default value is 38400 (one day)
# since V3.06
trunk_create_file_interval = 86400
?
# the threshold to create trunk file
# when the free trunk file size less than the threshold,
# will create he trunk files
# default value is 0
# since V3.06
trunk_create_file_space_threshold = 20G
?
# if check trunk space occupying when loading trunk free spaces
# the occupied spaces will be ignored
# default value is false
# since V3.09
# NOTICE: set this parameter to true will slow the loading of trunk spaces 
# when startup. you should set this parameter to true when neccessary.
trunk_init_check_occupying = false
?
# if ignore storage_trunk.dat, reload from trunk binlog
# default value is false
# since V3.10
# set to true once for version upgrade when your version less than V3.10
trunk_init_reload_from_binlog = false
?
# the min interval for compressing the trunk binlog file
# unit: second, 0 means never compress
# FastDFS compress the trunk binlog when trunk init and trunk destroy
# recommand to set this parameter to 86400 (one day)
# default value is 0
# since V5.01
trunk_compress_binlog_min_interval = 86400
?
# the interval for compressing the trunk binlog file
# unit: second, 0 means never compress
# recommand to set this parameter to 86400 (one day)
# default value is 0
# since V6.05
trunk_compress_binlog_interval = 86400
?
# compress the trunk binlog time base, time format: Hour:Minute
# Hour from 0 to 23, Minute from 0 to 59
# default value is 03:00
# since V6.05
trunk_compress_binlog_time_base = 03:00
?
# max backups for the trunk binlog file
# default value is 0 (never backup)
# since V6.05
trunk_binlog_max_backups = 7
?
# if use storage server ID instead of IP address
# if you want to use dual IPs for storage server, you MUST set
# this parameter to true, and configure the dual IPs in the file
# configured by following item "storage_ids_filename", such as storage_ids.conf
# default value is false
# since V4.00
use_storage_id = false
?
# specify storage ids filename, can use relative or absolute path
# this parameter is valid only when use_storage_id set to true
# since V4.00
storage_ids_filename = storage_ids.conf
?
# id type of the storage server in the filename, values are:
## ip: the ip address of the storage server
## id: the server id of the storage server
# this paramter is valid only when use_storage_id set to true
# default value is ip
# since V4.03
id_type_in_filename = id
?
# if store slave file use symbol link
# default value is false
# since V4.01
store_slave_file_use_link = false
?
# if rotate the error log every day
# default value is false
# since V4.02
rotate_error_log = false
?
# rotate error log time base, time format: Hour:Minute
# Hour from 0 to 23, Minute from 0 to 59
# default value is 00:00
# since V4.02
error_log_rotate_time = 00:00
?
# if compress the old error log by gzip
# default value is false
# since V6.04
compress_old_error_log = false
?
# compress the error log days before
# default value is 1
# since V6.04
compress_error_log_days_before = 7
?
# rotate error log when the log file exceeds this size
# 0 means never rotates log file by log file size
# default value is 0
# since V4.02
rotate_error_log_size = 0
?
# keep days of the log files
# 0 means do not delete old log files
# default value is 0
log_file_keep_days = 0
?
# if use connection pool
# default value is false
# since V4.05
use_connection_pool = true
?
# connections whose the idle time exceeds this time will be closed
# unit: second
# default value is 3600
# since V4.05
connection_pool_max_idle_time = 3600
?
# HTTP port on this tracker server
http.server_port = 8080
?
# check storage HTTP server alive interval seconds
# <= 0 for never check
# default value is 30
http.check_alive_interval = 30
?
# check storage HTTP server alive type, values are:
#   tcp : connect to the storge server with HTTP port only, 
#        do not request and get response
#   http: storage check alive url must return http status 200
# default value is tcp
http.check_alive_type = tcp
?
# check storage HTTP server alive uri/url
# NOTE: storage embed HTTP server support uri: /status.html
http.check_alive_uri = /status.html
?
?

base_path用来存储数据与日志，Tracker服务启动成功后，会在base_path下创建data、logs两个目录。目录结构如下：

${base_path}
  |__data
  |   |__storage_groups.dat：存储分组信息
  |   |__storage_servers.dat：存储服务器列表
  |__logs
  |   |__trackerd.log： tracker server 日志文件

修改tracker启动脚本

在/etc/init.d/目录下的fdfs_trackerd脚本是tracker的启动脚本，修改这个启动脚本。

fastDFS默契情况下会把可执行文件放在/usr/bin目录下，但是我们在安装时修改了默认配置。

把make.sh中的

TARGET_PREFIX=$DESTDIR/usr

改成了

TARGET_PREFIX=$DESTDIR/usr/local

因此启动脚本也需要修改，让启动脚本可以找到可执行文件把

PRG=/usr/bin/fdfs_trackerd
CONF=/etc/fdfs/tracker.conf

改成

PRG=/usr/local/bin/fdfs_trackerd
CONF=/etc/fdfs/tracker.conf

启动tracker

./fdfs_trackerd start
./fdfs_trackerd status
fdfs_trackerd (pid 1854) 正在运行...
?
./fdfs_trackerd stop

也可以采取

ps -ef | grep fdfs 命令查看是否启动成功。

如果想要设置开机启动，可以在/etc/rc.d/rc.local目录下添加如下代码：

/etc/init.d/fdfs_trackerd start

1.5、Storage配置

storage.conf

在/etc/fdfs/目录下storage.conf就是Storage节点的配置文件。打开这个配置文件：

# is this config file disabled
# false for enabled
# true for disabled
disabled = false
?
# the name of the group this storage server belongs to
#
# comment or remove this item for fetching from tracker server,
# in this case, use_storage_id must set to true in tracker.conf,
# and storage_ids.conf must be configured correctly.
group_name = group1
?
# bind an address of this host
# empty for bind all addresses of this host
bind_addr =
?
# if bind an address of this host when connect to other servers 
# (this storage server as a client)
# true for binding the address configured by the above parameter: "bind_addr"
# false for binding any address of this host
client_bind = true
?
# the storage server port
port = 23000
?
# connect timeout in seconds
# default value is 30
# Note: in the intranet network (LAN), 2 seconds is enough.
connect_timeout = 5
?
# network timeout in seconds for send and recv
# default value is 30
network_timeout = 60
?
# the heart beat interval in seconds
# the storage server send heartbeat to tracker server periodically
# default value is 30
heart_beat_interval = 30
?
# disk usage report interval in seconds
# the storage server send disk usage report to tracker server periodically
# default value is 300
stat_report_interval = 60
?
# the base path to store data and log files
# NOTE: the binlog files maybe are large, make sure
#       the base path has enough disk space,
#       eg. the disk free space should > 50GB
base_path = /home/yuqing/fastdfs
?
# max concurrent connections the server supported,
# you should set this parameter larger, eg. 10240
# default value is 256
max_connections = 1024
?
# the buff size to recv / send data from/to network
# this parameter must more than 8KB
# 256KB or 512KB is recommended
# default value is 64KB
# since V2.00
buff_size = 256KB
?
# accept thread count
# default value is 1 which is recommended
# since V4.07
accept_threads = 1
?
# work thread count
# work threads to deal network io
# default value is 4
# since V2.00
work_threads = 4
?
# if disk read / write separated
##  false for mixed read and write
##  true for separated read and write
# default value is true
# since V2.00
disk_rw_separated = true
?
# disk reader thread count per store path
# for mixed read / write, this parameter can be 0
# default value is 1
# since V2.00
disk_reader_threads = 1
?
# disk writer thread count per store path
# for mixed read / write, this parameter can be 0
# default value is 1
# since V2.00
disk_writer_threads = 1
?
# when no entry to sync, try read binlog again after X milliseconds
# must > 0, default value is 200ms
sync_wait_msec = 50
?
# after sync a file, usleep milliseconds
# 0 for sync successively (never call usleep)
sync_interval = 0
?
# storage sync start time of a day, time format: Hour:Minute
# Hour from 0 to 23, Minute from 0 to 59
sync_start_time = 00:00
?
# storage sync end time of a day, time format: Hour:Minute
# Hour from 0 to 23, Minute from 0 to 59
sync_end_time = 23:59
?
# write to the mark file after sync N files
# default value is 500
write_mark_file_freq = 500
?
# disk recovery thread count
# default value is 1
# since V6.04
disk_recovery_threads = 3
?
# store path (disk or mount point) count, default value is 1
store_path_count = 1
?
# store_path#, based on 0, to configure the store paths to store files
# if store_path0 not exists, it's value is base_path (NOT recommended)
# the paths must be exist.
#
# IMPORTANT NOTE:
#       the store paths' order is very important, don't mess up!!!
#       the base_path should be independent (different) of the store paths
?
store_path0 = /home/yuqing/fastdfs
#store_path1 = /home/yuqing/fastdfs2
?
# subdir_count  * subdir_count directories will be auto created under each 
# store_path (disk), value can be 1 to 256, default value is 256
subdir_count_per_path = 256
?
# tracker_server can ocur more than once for multi tracker servers.
# the value format of tracker_server is "HOST:PORT",
#   the HOST can be hostname or ip address,
#   and the HOST can be dual IPs or hostnames seperated by comma,
#   the dual IPS must be an inner (intranet) IP and an outer (extranet) IP,
#   or two different types of inner (intranet) IPs.
#   for example: 192.168.2.100,122.244.141.46:22122
#   another eg.: 192.168.1.10,172.17.4.21:22122
?
tracker_server = 192.168.209.121:22122
tracker_server = 192.168.209.122:22122
?
#standard log level as syslog, case insensitive, value list:
### emerg for emergency
### alert
### crit for critical
### error
### warn for warning
### notice
### info
### debug
log_level = info
?
#unix group name to run this program, 
#not set (empty) means run by the group of current user
run_by_group =
?
#unix username to run this program,
#not set (empty) means run by current user
run_by_user =
?
# allow_hosts can ocur more than once, host can be hostname or ip address,
# "*" (only one asterisk) means match all ip addresses
# we can use CIDR ips like 192.168.5.64/26
# and also use range like these: 10.0.1.[0-254] and host[01-08,20-25].domain.com
# for example:
# allow_hosts=10.0.1.[1-15,20]
# allow_hosts=host[01-08,20-25].domain.com
# allow_hosts=192.168.5.64/26
allow_hosts = *
?
# the mode of the files distributed to the data path
# 0: round robin(default)
# 1: random, distributted by hash code
file_distribute_path_mode = 0
?
# valid when file_distribute_to_path is set to 0 (round robin).
# when the written file count reaches this number, then rotate to next path.
# rotate to the first path (00/00) after the last path (such as FF/FF).
# default value is 100
file_distribute_rotate_count = 100
?
# call fsync to disk when write big file
# 0: never call fsync
# other: call fsync when written bytes >= this bytes
# default value is 0 (never call fsync)
fsync_after_written_bytes = 0
?
# sync log buff to disk every interval seconds
# must > 0, default value is 10 seconds
sync_log_buff_interval = 1
?
# sync binlog buff / cache to disk every interval seconds
# default value is 60 seconds
sync_binlog_buff_interval = 1
?
# sync storage stat info to disk every interval seconds
# default value is 300 seconds
sync_stat_file_interval = 300
?
# thread stack size, should >= 512KB
# default value is 512KB
thread_stack_size = 512KB
?
# the priority as a source server for uploading file.
# the lower this value, the higher its uploading priority.
# default value is 10
upload_priority = 10
?
# the NIC alias prefix, such as eth in Linux, you can see it by ifconfig -a
# multi aliases split by comma. empty value means auto set by OS type
# default values is empty
if_alias_prefix =
?
# if check file duplicate, when set to true, use FastDHT to store file indexes
# 1 or yes: need check
# 0 or no: do not check
# default value is 0
check_file_duplicate = 0
?
# file signature method for check file duplicate
## hash: four 32 bits hash code
## md5: MD5 signature
# default value is hash
# since V4.01
file_signature_method = hash
?
# namespace for storing file indexes (key-value pairs)
# this item must be set when check_file_duplicate is true / on
key_namespace = FastDFS
?
# set keep_alive to 1 to enable persistent connection with FastDHT servers
# default value is 0 (short connection)
keep_alive = 0
?
# you can use "#include filename" (not include double quotes) directive to 
# load FastDHT server list, when the filename is a relative path such as 
# pure filename, the base path is the base path of current/this config file.
# must set FastDHT server list when check_file_duplicate is true / on
# please see INSTALL of FastDHT for detail
##include /home/yuqing/fastdht/conf/fdht_servers.conf
?
# if log to access log
# default value is false
# since V4.00
use_access_log = false
?
# if rotate the access log every day
# default value is false
# since V4.00
rotate_access_log = false
?
# rotate access log time base, time format: Hour:Minute
# Hour from 0 to 23, Minute from 0 to 59
# default value is 00:00
# since V4.00
access_log_rotate_time = 00:00
?
# if compress the old access log by gzip
# default value is false
# since V6.04
compress_old_access_log = false
?
# compress the access log days before
# default value is 1
# since V6.04
compress_access_log_days_before = 7
?
# if rotate the error log every day
# default value is false
# since V4.02
rotate_error_log = false
?
# rotate error log time base, time format: Hour:Minute
# Hour from 0 to 23, Minute from 0 to 59
# default value is 00:00
# since V4.02
error_log_rotate_time = 00:00
?
# if compress the old error log by gzip
# default value is false
# since V6.04
compress_old_error_log = false
?
# compress the error log days before
# default value is 1
# since V6.04
compress_error_log_days_before = 7
?
# rotate access log when the log file exceeds this size
# 0 means never rotates log file by log file size
# default value is 0
# since V4.02
rotate_access_log_size = 0
?
# rotate error log when the log file exceeds this size
# 0 means never rotates log file by log file size
# default value is 0
# since V4.02
rotate_error_log_size = 0
?
# keep days of the log files
# 0 means do not delete old log files
# default value is 0
log_file_keep_days = 0
?
# if skip the invalid record when sync file
# default value is false
# since V4.02
file_sync_skip_invalid_record = false
?
# if use connection pool
# default value is false
# since V4.05
use_connection_pool = true
?
# connections whose the idle time exceeds this time will be closed
# unit: second
# default value is 3600
# since V4.05
connection_pool_max_idle_time = 3600
?
# if compress the binlog files by gzip
# default value is false
# since V6.01
compress_binlog = true
?
# try to compress binlog time, time format: Hour:Minute
# Hour from 0 to 23, Minute from 0 to 59
# default value is 01:30
# since V6.01
compress_binlog_time = 01:30
?
# if check the mark of store path to prevent confusion
# recommend to set this parameter to true
# if two storage servers (instances) MUST use a same store path for
# some specific purposes, you should set this parameter to false
# default value is true
# since V6.03
check_store_path_mark = true
?
# use the ip address of this storage server if domain_name is empty,
# else this domain name will ocur in the url redirected by the tracker server
http.domain_name =
?
# the port of the web server on this storage server
http.server_port = 8888
?
?

在此配置文件中，我们需要配置如下的内容：

base_path = /fdfs/storage/base 配置信息与日志目录

store_path0 = /fdfs/storage/store 存储文件的目录

tracker_server = 192.168.209.121:22122 追踪服务器的地址

创建配置的目录

[root@localhost fdfs]# mkdir -p /fdfs/storage/base
[root@localhost fdfs]# mkdir -p /fdfs/storage/store

修改storage的启动脚本

同理，因此我们修改了fastDFS的默认位置，因此还是需要修改启动脚本的。

把

?
PRG=/usr/bin/fdfs_storaged

修改成

?
PRG=/usr/local/bin/fdfs_storaged

启动storage

[root@localhost init.d]# ./fdfs_storaged start
Starting FastDFS storage server: 
[root@localhost init.d]# ./fdfs_storaged status
fdfs_storaged (pid 1979) 正在运行...
[root@localhost init.d]# ps -ef |grep  fdfs
root       1854      1  0 23:01 ?        00:00:00 /usr/local/bin/fdfs_trackerd /etc/fdfs/tracker.conf
root       1979      1  8 23:40 ?        00:00:03 /usr/local/bin/fdfs_storaged /etc/fdfs/storage.conf
root       2000   1650  0 23:41 pts/0    00:00:00 grep --color=auto fdfs

1.6、客户端配置

客户端配置是为了用命令行测试我们的fastDFS，如果我们不准备用命令行测试我们的fastDFS，我们可以不用配置这些。

客户端配置既可以放在tracker服务器上，也可以放在storage服务器上。

client.conf

# connect timeout in seconds
# default value is 30s
# Note: in the intranet network (LAN), 2 seconds is enough.
connect_timeout = 5
?
# network timeout in seconds
# default value is 30s
network_timeout = 60
?
# the base path to store log files
base_path = /home/yuqing/fastdfs
?
# tracker_server can ocur more than once for multi tracker servers.
# the value format of tracker_server is "HOST:PORT",
#   the HOST can be hostname or ip address,
#   and the HOST can be dual IPs or hostnames seperated by comma,
#   the dual IPS must be an inner (intranet) IP and an outer (extranet) IP,
#   or two different types of inner (intranet) IPs.
#   for example: 192.168.2.100,122.244.141.46:22122
#   another eg.: 192.168.1.10,172.17.4.21:22122
?
tracker_server = 192.168.0.196:22122
tracker_server = 192.168.0.197:22122
?
#standard log level as syslog, case insensitive, value list:
### emerg for emergency
### alert
### crit for critical
### error
### warn for warning
### notice
### info
### debug
log_level = info
?
# if use connection pool
# default value is false
# since V4.05
use_connection_pool = false
?
# connections whose the idle time exceeds this time will be closed
# unit: second
# default value is 3600
# since V4.05
connection_pool_max_idle_time = 3600
?
# if load FastDFS parameters from tracker server
# since V4.05
# default value is false
load_fdfs_parameters_from_tracker = false
?
# if use storage ID instead of IP address
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# default value is false
# since V4.05
use_storage_id = false
?
# specify storage ids filename, can use relative or absolute path
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# since V4.05
storage_ids_filename = storage_ids.conf
?
?
#HTTP settings
http.tracker_server_port = 80
?
#use "#include" directive to include HTTP other settiongs
##include http.conf
?
?

主要配置如下两个配置

base_path = /fdfs/client 存储日志的目录

tracker_server = 192.168.0.197:22122 tracker服务器

测试客户端

配置完客户端之后，开始测试。

进入可执行命令的目录下，在本案例中是/usr/local/bin目录

在此目录下执行如下的命令：

[root@localhost bin]# ./fdfs_upload_file /etc/fdfs/client.conf ~/fastdfs-master.zip 
group1/M00/00/00/wKgCiGGeYG-ARQWSAA3yQ6X14YY529.zip

./fdfs_upload_file 是要执行的命令

/etc/fdfs/client.conf 是要采用的配置文件

~/fastdfs-master.zip 是要上传的文件

group1/M00/00/00/wKgCiGGeYG-ARQWSAA3yQ6X14YY529.zip 是上传文件后返回的file_id

最后在/fdfs/storage/store/data/00/00目录下可以找到这个文件。

文件路径解析

group1:这个值是在/etc/fdfs/storage.conf中的group_name决定的。

M00是一个虚拟目录，类似于一个快捷方式，这个M00的引用就是引用到/fdfs/storage/store/data目录

1.7、nginx安装

FastDFS是文件存储服务器，我们上传的一些图片等文件需要预览，因此需要一个http服务器，nginx可以作为一个http服务器用来提供预览服务。

把fastdfs-nginx-module-master.zip解压到/usr/local/fastdfs目录。并进入源码目录。

[root@localhost src]# ll
总用量 84
-rw-r--r--. 1 root root 43507 11月 29 2019 common.c
-rw-r--r--. 1 root root  3995 11月 29 2019 common.h
-rw-r--r--. 1 root root   848 11月 29 2019 config
-rw-r--r--. 1 root root  3725 11月 29 2019 mod_fastdfs.conf
-rw-r--r--. 1 root root 28668 11月 29 2019 ngx_http_fastdfs_module.c

其中mod_fastdfs.conf是配置文件。config也是配置文件。

fastdfs nginx模块安装

ngx_addon_name=ngx_http_fastdfs_module
?
if test -n "${ngx_module_link}"; then
    ngx_module_type=HTTP
    ngx_module_name=$ngx_addon_name
    ngx_module_incs="/usr/local/include"
    ngx_module_libs="-lfastcommon -lfdfsclient"
    ngx_module_srcs="$ngx_addon_dir/ngx_http_fastdfs_module.c"
    ngx_module_deps=
    CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"/etc/fdfs/mod_fastdfs.conf\"'"
    . auto/module
else
    HTTP_MODULES="$HTTP_MODULES ngx_http_fastdfs_module"
    NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_fastdfs_module.c"
    CORE_INCS="$CORE_INCS /usr/local/include"
    CORE_LIBS="$CORE_LIBS -lfastcommon -lfdfsclient"
    CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"/etc/fdfs/mod_fastdfs.conf\"'"
fi

因此安装时，我们修改过配置文件，因此需要修改一下配置文件

 CORE_INCS="$CORE_INCS /usr/local/include/fastdfs /usr/include/fastcommon"

如果安装fastDFS时，没有修改过配置，只需要把CORE_INCS中的local删除掉就行了。

配置文件修改好，并不是安装好，这个模块的安装是在nginx安装时，把这个模块添加上就行了。

nginx安装

安装nginx的依赖

yum install gcc-c++
yum install -y pcre pcre-devel
yum install -y zlib zlib-devel
yum install -y openssl openssl-devel

将nginx的源码拷贝到/usr/local/目录下，并创建一个/var/temp/nginx目录。进入源码目录，执行配置命令

mkdir -p /var/temp/nginx
./configure \
--prefix=/usr/local/nginx \
--with-http_gzip_static_module \
--http-client-body-temp-path=/var/temp/nginx/client \
--http-proxy-temp-path=/var/temp/nginx/proxy \
--http-fastcgi-temp-path=/var/temp/nginx/fastcgi \
--http-uwsgi-temp-path=/var/temp/nginx/uwsgi \
--http-scgi-temp-path=/var/temp/nginx/scgi \
--add-module=/usr/local/fastdfs/fastdfs-nginx-module-master/src

nginx安装后的配置

 nginx path prefix: "/usr/local/nginx"
  nginx binary file: "/usr/local/nginx/sbin/nginx"
  nginx modules path: "/usr/local/nginx/modules"
  nginx configuration prefix: "/usr/local/nginx/conf"
  nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
  nginx pid file: "/usr/local/nginx/logs/nginx.pid"
  nginx error log file: "/usr/local/nginx/logs/error.log"
  nginx http access log file: "/usr/local/nginx/logs/access.log"
  nginx http client request body temporary files: "/var/temp/nginx/client"
  nginx http proxy temporary files: "/var/temp/nginx/proxy"
  nginx http fastcgi temporary files: "/var/temp/nginx/fastcgi"
  nginx http uwsgi temporary files: "/var/temp/nginx/uwsgi"
  nginx http scgi temporary files: "/var/temp/nginx/scgi"

--add-module此配置信息是为了指定安装Nginx时需要加载的模块，如果未指定，则在nginx安装时，无法加载fastdfs-nginx-module-master模块，后续的工作无法开展。

后面执行

make
make install

修改配置

拷贝mod_fastdfs.conf到/etc/fdfs/目录下。

mod_fastdfs.conf配置文件如下：

# connect timeout in seconds
# default value is 30s
connect_timeout=2
?
# network recv and send timeout in seconds
# default value is 30s
network_timeout=30
?
# the base path to store log files
base_path=/tmp
?
# if load FastDFS parameters from tracker server
# since V1.12
# default value is false
load_fdfs_parameters_from_tracker=true
?
# storage sync file max delay seconds
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# since V1.12
# default value is 86400 seconds (one day)
storage_sync_file_max_delay = 86400
?
# if use storage ID instead of IP address
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# default value is false
# since V1.13
use_storage_id = false
?
# specify storage ids filename, can use relative or absolute path
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# since V1.13
storage_ids_filename = storage_ids.conf
?
# FastDFS tracker_server can ocur more than once, and tracker_server format is
#  "host:port", host can be hostname or ip address
# valid only when load_fdfs_parameters_from_tracker is true
tracker_server=tracker:22122
# the port of the local storage server
# the default value is 23000
storage_server_port=23000
?
# the group name of the local storage server
group_name=group1
?
# if the url / uri including the group name
# set to false when uri like /M00/00/00/xxx
# set to true when uri like ${group_name}/M00/00/00/xxx, such as group1/M00/xxx
# default value is false
url_have_group_name = false
?
# path(disk or mount point) count, default value is 1
# must same as storage.conf
store_path_count=1
?
# store_path#, based 0, if store_path0 not exists, it's value is base_path
# the paths must be exist
# must same as storage.conf
store_path0=/home/yuqing/fastdfs
#store_path1=/home/yuqing/fastdfs1
?
# standard log level as syslog, case insensitive, value list:
### emerg for emergency
### alert
### crit for critical
### error
### warn for warning
### notice
### info
### debug
log_level=info
?
# set the log filename, such as /usr/local/apache2/logs/mod_fastdfs.log
# empty for output to stderr (apache and nginx error_log file)
log_filename=
?
# response mode when the file not exist in the local file system
## proxy: get the content from other storage server, then send to client
## redirect: redirect to the original storage server (HTTP Header is Location)
response_mode=proxy
?
# the NIC alias prefix, such as eth in Linux, you can see it by ifconfig -a
# multi aliases split by comma. empty value means auto set by OS type
# this paramter used to get all ip address of the local host
# default values is empty
if_alias_prefix=
?
# use "#include" directive to include HTTP config file
# NOTE: #include is an include directive, do NOT remove the # before include
#include http.conf
?
?
# if support flv
# default value is false
# since v1.15
flv_support = true
?
# flv file extension name
# default value is flv
# since v1.15
flv_extension = flv
?
?
# set the group count
# set to none zero to support multi-group on this storage server
# set to 0  for single group only
# groups settings section as [group1], [group2], ..., [groupN]
# default value is 0
# default value is 0
# since v1.14
group_count = 0
?
# group settings for group #1
# since v1.14
# when support multi-group on this storage server, uncomment following section
#[group1]
#group_name=group1
#storage_server_port=23000
#store_path_count=2
#store_path0=/home/yuqing/fastdfs
#store_path1=/home/yuqing/fastdfs1
?
# group settings for group #2
# since v1.14
# when support multi-group, uncomment following section as neccessary
#[group2]
#group_name=group2
#storage_server_port=23000
#store_path_count=1
#store_path0=/home/yuqing/fastdfs

这边有几个配置项需要修改。

1、tracker_server 修改成跟踪服务器的ip地址。

2、url_have_group_name url中是否包含组名，改成true

3、store_path0 改成storage服务器存储的位置，在本例中是/fdfs/storage/store

4、进入fastdfs源码目录下的conf目录，在本例中是/usr/local/fastdfs/fastdfs-master/conf ，拷贝http.conf、 mime.types拷贝到/etc/fdfs/目录下。

5、因为nginx在启动时，默认在/usr/lib64目录下查找需要的.so文件，如果在安装FastDFS时，修改了make.sh文件中的TARGET_PREFIX参数，必须创建此软链接。

ln -s /usr/local/lib64/libfdfsclient.so  /usr/lib64/libfdfsclient.so

6、创建网络访问存储服务的软连接

ln -s /fdfs/storage/store/data/  /fdfs/storage/store/data/M00

在文件上传到FastDFS后，FastDFS会返回 group1/M00/00/00/xxxxxxxxxxxxxxxxx.xxx，其中group1是卷名，在mod_fastdfs.conf配置文件中已经配置了url_have_group_name，以确保URL解析正确。而其中的M00是FastDFS保存数据时使用的虚拟目录，需要将这个虚拟目录定位到真实数据目录上。

7、配置Nginx服务器

 server {
        listen       8888;
        server_name  localhost;
?
        location ~/group[0-9]/M00{
                ngx_fastdfs_module;
        }
?
        #charset koi8-r;
?
        #access_log  logs/host.access.log  main;
?
        location / {
            root   html;
            index  index.html index.htm;
        }
?
        #error_page  404              /404.html;
?
        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
}

监听8888端口，并把以group0 到group9的开头的请求转发到ngx_fastdfs_module。

测试

[root@localhost ~]# /usr/local/bin/fdfs_upload_file /etc/fdfs/client.conf java.jpg 
group1/M00/00/00/wKgCiGGftZiAXdL1AABlsYgsPvE177.jpg
[root@localhost ~]# systemctl stop firewalld.service

http://192.168.2.136:8888/group1/M00/00/00/wKgCiGGftZiAXdL1AABlsYgsPvE177.jpg

关闭防火墙，通过ip地址预览

fastDFS

FastdDFS入门一环境搭建

1.1、依赖搭建

1.2、libfastcommon安装

1.3、FastDFS安装

1.4、Tracker配置

tracker.conf

修改tracker启动脚本

启动tracker

1.5、Storage配置

storage.conf

修改storage的启动脚本

启动storage

1.6、客户端配置

client.conf

测试客户端

文件路径解析

1.7、nginx安装

fastdfs nginx模块安装

nginx安装

修改配置

测试

相关

docker-compose fastdfs

C#集成FastDFS断点续传

分布式文件服务器FastDFS

FastDFS-分布式存储集群-部署

FastDFS-分布式存储集群-部署

如何在服务器上手动部署FastDFS

FastDFS（distributed file system）工作流程讲解

分布式文件系统FastDFS动态扩容

FastDFS

手把手教你搭建FastDFS集群（上）

基于Django+FastDFS文件管理系统搭建

SpringBoot + fastDFS 的简单总结

标签

FastdDFS入门一 环境搭建

1.1、依赖搭建

1.2、libfastcommon安装

1.3、FastDFS安装

1.4、Tracker配置

tracker.conf

修改tracker启动脚本

启动tracker

1.5、Storage配置

storage.conf

修改storage的启动脚本

启动storage

1.6、客户端配置

client.conf

测试客户端

文件路径解析

1.7、nginx安装

fastdfs nginx模块安装

nginx安装

修改配置

测试

相关

FastdDFS入门一环境搭建