Home Projects Articles About Us 6 Ways to Run Shell Commands in Ruby

Often times we want to interact with the operating system or run shell commands from within Ruby. Ruby provides a number of ways for us to perform this task.
Exec

Kernel#exec (or simply exec) replaces the current process by running the given command For example:


$ irb
>> exec 'echo "hello $HOSTNAME"'
hello nate.local
$

Notice how exec replaces the irb process is with the echo command which then exits. Because the Ruby effectively ends this method has only limited use. The major drawback is that you have no knowledge of the success or failure of the command from your Ruby script.
System

The system command operates similarly but the system command runs in a subshell instead of replacing the current process. system gives us a little more information than exec in that it returns true if the command ran successfully and false otherwise.


$ irb
>> system 'echo "hello $HOSTNAME"'
hello nate.local
=> true
>> system 'false'
=> false
>> puts $?
256
=> nil
>>

system sets the global variable $? to the exit status of the process. Notice that we have the exit status of the false command (which always exits with a non-zero code). Checking the exit code gives us the opportunity to raise an exception or retry our command.
Note for Newbies: Unix commands typically exit with a status of 0 on success and non-zero otherwise.

System is great if all we want to know is “Was my command successful or not?” However, often times we want to capture the output of the command and then use that value in our program.
Backticks (`)

Backticks (also called “backquotes”) runs the command in a subshell and returns the standard output from that command.


$ irb
>> today = `date`
=> "Mon Mar 12 18:15:35 PDT 2007\n"
>> $?
=> #
>> $?.to_i
=> 0

This is probably the most commonly used and widely known method to run commands in a subshell. As you can see, this is very useful in that it returns the output of the command and then we can use it like any other string.

Notice that $? is not simply an integer of the return status but actually a Process::Status object. We have not only the exit status but also the process id. Process::Status#to_i gives us the exit status as an integer (and #to_s gives us the exit status as a string).

One consequence of using backticks is that we only get the standard output (stdout) of this command but we do not get the standard error (stderr). In this example we run a Perl script which outputs a string to stderr.


$ irb
>> warning = `perl -e "warn 'dust in the wind'"`
dust in the wind at -e line 1.
=> ""
>> puts warning

=> nil

Notice that the variable warning doesn’t get set! When we warn in Perl this is output on stderr which is not captured by backticks.
IO#popen

IO#popen is another way to run a command in a subprocess. popen gives you a bit more control in that the subprocess standard input and standard output are both connected to the IO object.


$ irb
>> IO.popen("date") { |f| puts f.gets }
Mon Mar 12 18:58:56 PDT 2007
=> nil

While IO#popen is nice, I typically use Open3#popen3 when I need this level of granularity.
Open3#popen3

The Ruby standard library includes the class Open3. It’s easy to use and returns stdin, stdout and stderr. In this example, lets use the interactive command dc. dc is reverse-polish calculator that reads from stdin. In this example we will push two numbers and an operator onto the stack. Then we use p to print out the result of the operator operating on the two numbers. Below we push on 5, 10 and + and get a response of 15\n to stdout.


$ irb
>> stdin, stdout, stderr = Open3.popen3('dc')
=> [#, #, #]
>> stdin.puts(5)
=> nil
>> stdin.puts(10)
=> nil
>> stdin.puts("+")
=> nil
>> stdin.puts("p")
=> nil
>> stdout.gets
=> "15\n"

Notice that with this command we not only read the output of the command but we also write to the stdin of the command. This allows us a great deal of flexibility in that we can interact with the command if needed.

popen3 will also give us the stderr if we need it.


# (irb continued...)
>> stdin.puts("asdfasdfasdfasdf")
=> nil
>> stderr.gets
=> "dc: stack empty\n"

However, there is a shortcoming with popen3 in ruby 1.8.5 in that it doesn’t return the proper exit status in $?.


$ irb
>> require "open3"
=> true
>> stdin, stdout, stderr = Open3.popen3('false')
=> [#, #, #]
>> $?
=> #
>> $?.to_i
=> 0

0? false is supposed to return a non-zero exit status! It is this shortcoming that brings us to Open4.
Open4#popen4

Open4#popen4 is a Ruby Gem put together by Ara Howard. It operates similarly to open3 except that we can get the exit status from the program. popen4 returns a process id for the subshell and we can get the exit status from that waiting on that process. (You will need to do a gem instal open4 to use this.)


$ irb
>> require "open4"
=> true
>> pid, stdin, stdout, stderr = Open4::popen4 "false"
=> [26327, #, #, #]
>> $?
=> nil
>> pid
=> 26327
>> ignored, status = Process::waitpid2 pid
=> [26327, #]
>> status.to_i
=> 256

A nice feature is that you can call popen4 as a block and it will automatically wait for the return status.


$ irb
>> require "open4"
=> true
>> status = Open4::popen4("false") do |pid, stdin, stdout, stderr|
?> puts "PID #{pid}"
>> end
PID 26598
=> #
>> puts status
256
=> nil

Rake task vs script/runner

The difference between them is that script/runner boots Rails whereas a
Rake task doesn't unless you tell it to by making the task depend on
:environment, like this:

task :some_useful_task => :environment do
# do some useful task
end

Since booting Rails is expensive, it might be worth skipping if you can
avoid it.

Other than that, they are roughly equivalent.

Ruby - DUP vs CLONE

Both DUP & CLONE can be used to create shallow copy of an object. Both copies the instance variables of obj. But we need to be selective in their usage.

Few difference between these are

1) CLONE copies both FROZEN and TAINTED state of an object, where as DUP only copies TAINTED state of an object.

2) With CLONE you can copy any singleton methods of an object but DUP does not support this.

CLONE is used to duplicate an object, including its internal state, DUP typically uses the class of the descendent object to create the new instance.

I had some bitter experience while using DUP for duplicating an ActiveRecord row, this ended up in losing the original one the same worked fine with CLONE.

Frozen state:

The object can not be modified any more.

Example


a = [ "a", "b", "c" ]
a.freeze
a << "z"

Will return

`<<': can't modify frozen array

Ruby: for vs each

The difference is that #each scopes block variable inside the block, whereas for/in scopes it outside the block.

Following demonstration will clear this:

for i in [1,2,3]
# ...
end
puts i # => 3

[1,2,3].each do |j|
# ...
end
puts j # => NameError: undefined local...

Lets take one more example to demonstrate this:
loop1 = []
loop2 = []

calls = ["one", "two", "three"]

calls.each do |c|
loop1 << Proc.new { puts c }
end

for c in calls
loop2 << Proc.new { puts c }
end

loop1[1].call #=> "two"
loop2[1].call #=> "three"

Collect vs Map

Collect vs Map in ruby

I’ve been using collect in all my programs up till now, and I recently discovered that the map function is equivalent.

My question was what is the difference between them and which one is better to use. I investigated the net and found that it is a matter or convention more than a difference:
- collect is recommended to be used when the list is not changed:


ruby-1.8.7-p302 :009 > ["one", "two", "three"].collect{|item| item.length == 3}
=> [true, true, false]

- map is recommended to be used when the elements of the list have to be processed:

["one", "two", "three"].map{|item| process(item)}

delayed_job method caching issue

In recent I have faced weird behavior of delayed_jobs. I had some failed enqueued jobs in delayed_jobs table. I seen last error field and corrected them by and pushed code.

But jobs were still failing as delayed_job caches method definition also with method call so it was calling that old buggy method only even though I have updated that method.

So I have restarted delayed job server again and its started working as expected.

delayed_job vs resque

Resque vs DelayedJob

How does Resque compare to DelayedJob, and why would you choose one over the other?

Resque supports multiple queues
DelayedJob supports finer grained priorities
Resque workers are resilient to memory leaks / bloat
DelayedJob workers are extremely simple and easy to modify
Resque requires Redis
DelayedJob requires ActiveRecord
Resque can only place JSONable Ruby objects on a queue as arguments
DelayedJob can place any Ruby object on its queue as arguments
Resque includes a Sinatra app for monitoring what's going on
DelayedJob can be queried from within your Rails app if you want to add an interface

If you're doing Rails development, you already have a database and ActiveRecord. DelayedJob is super easy to setup and works great. GitHub used it for many months to process almost 200 million jobs.

Choose Resque if:

You need multiple queues
You don't care / dislike numeric priorities
You don't need to persist every Ruby object ever
You have potentially huge queues
You want to see what's going on
You expect a lot of failure / chaos
You can setup Redis
You're not running short on RAM

Choose DelayedJob if:

You like numeric priorities
You're not doing a gigantic amount of jobs each day
Your queue stays small and nimble
There is not a lot failure / chaos
You want to easily throw anything on the queue
You don't want to setup Redis

In no way is Resque a "better" DelayedJob, so make sure you pick the tool that's best for your app.