FreeNAS (FreeBSD) Shell rsync & cp Commands

How do you copy files (and subordinate files/folders) from one folder/volume in FreeNAS to another folder/volume using the rsync command from shell?

rsync Command - Copy files and directories.


Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time. Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates that the file's data does not need to be updated.

One FreeNAS Server - Copy files and folders to a new location in FreeNAS

Run the following command from shell inside FreeNAS:

rsync [options] [source] [destination]


rsync -Whrtv  --progress --stats /mnt/VOLUMENAME1/FOLDER1/FOLDER2/ /mnt/VOLUMENAME2/FOLDER3/FOLLDER4/

OR if you need/want to copy file permissions use:


rsync -Whav  --progress --stats /mnt/VOLUMENAME1/FOLDER1/FOLDER2/ /mnt/VOLUMENAME2/FOLDER3/FOLLDER4/


This will copy the contents of "folder2" into "folder4".  Note that "folder4" must exist prior to running the command.  If your folder name contains a space include the entire path between apostrophes, e.g. 'folder 2'.



Useful rsync commands:
-W, --whole-file copy files whole (w/o delta-xfer algorithm)
-h, --human-readable     output numbers in a human-readable format
-a, --archive     archive mode; equals -rlptgoD (no -H,-A,-X)
     -r, --recursive     recurse into directories
     -l, --links     copy symlinks as symlinks
     -p, --perms     preserve permissions
     -t, --times     preserve modification times
     -g, --group     preserve group
     -o, --owner     preserve owner (super-user only)
     -D     same as --devices --specials
          --devices     preserve device files (super-user only)
          --specials     preserve special files
-v, --verbose     increase verbosity
-P     same as --partial --progress
     --partial     keep partially transferred files
     --progress     show progress during transfer
--stats     give some file transfer stats
-e, --rsh=COMMAND     specify rsh replacement
-z, --compress     compress file data during the transfer
-n, --dry-run     perform a trial run with no changes made
--delete     delete extraneous files from dest dirs

Long explanations of commands:
-W, --whole-file
      With this option rsync's delta-transfer algorithm  is  not  used
      and  the whole file is sent as-is instead.  The transfer may be
      faster if this option is used when  the  bandwidth  between  the
      source  and destination machines is higher than the bandwidth to
      disk  (especially  when  the  "disk"  is actually  a  networked
      filesystem).   This is the default when both the source and des-
      tination are specified as local paths.

-h, --human-readable
      Output  numbers in a more human-readable format. This makes big
      numbers output using larger units, with a K, M, or G suffix.  If
      this  option  was  specified  once,  these units are K (1000), M
      (1000*1000), and G (1000*1000*1000); if the option is  repeated,
      the units are powers of 1024 instead of 1000.

-a, --archive
      This  is equivalent to -rlptgoD. It is a quick way of saying you
      want recursion and want to preserve almost everything  (with  -H
      being  a notable  omission).   The  only exception to the above
      equivalence is when --files-from is specified, in which case  -r
      is not implied.
      Note that -a does not preserve hardlinks, because finding multi-
      ply-linked files is expensive.  You must separately specify  -H.

-v, --verbose

      This  option increases the amount of information the daemon logs
      during its startup phase.  After the client connects,  the  dae-
      mon's verbosity level will be controlled by the options that the
      client used and the "max verbosity" setting in the module's con-
      fig section.

--partial

      By  default, rsync will delete any partially transferred file if
      the transfer is interrupted. In some circumstances  it  is  more
      desirable  to keep partially transferred files. Using the --par-
      tial option tells rsync to keep the partial  file  which should
      make  a subsequent transfer of the rest of the file much faster.

--progress

      This  option  tells  rsync  to  print  information  showing  the
      progress of  the transfer. This gives a bored user something to
      watch.

      This option is normally combined with  -v.  Using  this option

      without  the  -v option will produce weird results on your dis-
      play.

--stats

      This  tells  rsync  to  print a verbose set of statistics on the
      file transfer, allowing you to  tell  how  effective  the  rsync
      algorithm is for your data.

-e, --rsh=COMMAND

      This option allows you to choose an  alternative  remote  shell
      program  to  use for communication between the local and remote
      copies of rsync. By default, rsync will use  rsh,  but  you  may
      like to instead use ssh because of its high security.

      You can also choose the remote shell program using the RSYNC_RSH

      environment variable.

      See also the --blocking-io option  which is  affected  by  this

      option.


----------------------------------------------------------------------


Two FreeNAS Servers - Copy files between two different FreeNAS servers

Turn on SSH service of destination FreeNAS server.  Destination dataset must be set to UNIX share type.  Run the following command from shell inside FreeNAS:


rsync -Whav --progress --stats /mnt/VOLUMENAME1/FOLDER1/FOLDER2/  [email protected]:/mnt/VOLUMENAME2/FOLDER3/FOLDER4/


"x.x.x.x" is the IP of the destination FreeNAS server.  This will copy the contents of "folder2" into "folder4".  Note that "FOLDER4" must exist prior to running the command.  You will get approximately 50 MB/s transfer speed.



----------------------------------------------------------------------


cp Command - Copy files and directories.

In the first synopsis form, the cp utility copies the contents of the source_file to the target_file.  In the second synopsis form, the contents of each named source_file is copied to the destination target_directory. The names of the files themselves are not changed.  If cp detects an attempt to copy a file to itself, the copy will fail.

One FreeNAS Server - Copy files and folders to a new location in FreeNAS

Run the following command from shell inside FreeNAS:

cp [options] [source] [destination]

cp -iav /mnt/VOLUMENAME1/FOLDER1/FOLDER2/ /mnt/VOLUME NAME2/FOLDER3/FOLLDER4/


OR if you do not want to copy file permissions (will not copy times):

cp -iRv /mnt/VOLUMENAME1/FOLDER1/FOLDER2/ /mnt/VOLUME NAME2/FOLDER3/FOLLDER4/


Useful cp commands:

-i     Interactive.  Prompt you if overwriting any files.
-a     Archive mode.  Same as -RpP.
     -R     Recursive. Will copy sub-directories.
     -p     Preserve file attributes like date, time, etc.
     -P          If the -R option is specified, no symbolic links are followed.  This is the default.
-v     Verbose. Show you each file as it is copied in the command line console.

This will copy the contents of "folder2" into "folder4".  Note that "folder4" must exist prior to running the command.


Long explanations of cp commands:

-i   Cause cp to write a prompt to the standard error output before
  copying a file that would overwrite an existing file.  If the
  response from the standard input begins with the character `y' or
  `Y', the file copy is attempted.  (The -i option overrides any pre-
  vious -f or -n options.)

-p   Cause cp to preserve the following attributes of each source file

  in the copy: modification time, access time, file flags, file mode,
  ACL, user ID, and group ID, as allowed by permissions.

  If the user ID and group ID cannot be preserved, no error message

  is displayed and the exit value is not altered.

-R   If source_file designates a directory, cp copies the directory and

  the entire subtree connected at that point. If the source_file
  ends in a /, the contents of the directory are copied rather than
  the directory itself.  This option also causes symbolic links to be
  copied, rather than indirected through, and for cp to create spe-
  cial files rather than copying them as normal files. Created
  directories have the same mode as the corresponding source direc-
  tory, unmodified by the process' umask.

-v   Cause cp to be verbose, showing files as they are copied.


--------------------------------------


Transfer rates for each that I experienced:


1st - cp one server - .51 GB/s

2nd - rsync one server - 0.18 GB/s
3rd - rsync two server - 0.06 GB/s




Popular Posts