Ruby net-scp cannot scp multiple files with asterisk(*)

net-ssh/net-scp is a Ruby gem which can be used to scp files between different *nix machines. It's similar to how the *nix scp command. It can be used to scp a file or a directory. However, it seems it has some problem to scp multiple files using pattern * .

For example, below script is supposedto download allfiles from remote directory to local directory:

require 'net/scp'
host = 'testmachine'
login = 'testaccount'
password = "testpassword"
remote_path = '/tmp/remote_dir/*'
Net::SCP.start(host, login, :password => password) do |scp|
scp.download(remote_path, '.')
end

The execution result of this script will produce some SCP error.

/home/kepi/.rvm/gems/ruby-2.2.2/gems/net-scp-1.2.1/lib/net/scp.rb:366:in `block (3 levels) in start_command': SCP did not finish successfully (1): scp: /tmp/remote_dir/*: No such file or directory (Net::SCP::Error)

It says that no file can be found, but actually there arefiles present.

The root cause of this issue isactually being that * is escaped when the command is being built. Basically it's in the start_command method. When shellescape method is being called, * will be escaped. See below code in scp.rb .

str.gsub!(/([^A-Za-z0-9_\-.,:\/@\n])/n, "\\\\\\1")

Since * is escaped, the actual command being executed would look like

scp -f /tmp/remote_dir/\\*

This command will fail. Actually the issue should be resolved by updating the shellescape method to not escape * .

str.gsub!(/([^A-Za-z0-9_\-.,:\/@\n \* ])/n, "\\\\\\1")

But before this issue can be actually fixed in the official gem, there are a few workarounds for this issue. The first one is to define your own shellescape method for the path which contains asterisk. Since the shellescape in scp.rb looks like

# Imported from ruby 1.9.2 shellwords.rb
def shellescape(path)
# Convert path to a string if it isn't already one.
str = path.to_s
# ruby 1.8.7+ implements String#shellescape
return str.shellescape if str.respond_to? :shellescape
# An empty argument will be skipped, so return empty quotes.
return "''" if str.empty?
str = str.dup
# Process as a single byte sequence because not all shell
# implementations are multibyte aware.
str.gsub!(/([^A-Za-z0-9_\-.,:\/@\n])/n, "\\\\\\1")
# A LF cannot be escaped with a backslash because a backslash + LF
# combo is regarded as line continuation and simply ignored.
str.gsub!(/\n/, "'\n'")
return str
end

Above code shows that if a str has shellescape defined, that method will be called and returned. So in your own script, you can do something

require 'net/scp'
host = 'testmachine'
login = 'testaccount'
password = "testpassword"
remote_path = "/tmp/remote_dir/*"
def remote_path.shellescape
p "In self defined method " + self.to_s
self
end
Net::SCP.start(host, login, :password => password) do |scp|
scp.download(remote_path, '.')
end

With this, multiple files can be downloaded.

Another alternative you can do is to download the whole directory if you want to download all files in a directory. The code would look like.

scp.download! "#{source_dir}", "#{target_dir}", :recursive => true

This method will create a directory in target_dir and download all files from source_dir to the newly created dir.