Category: Ruby

Rails MemCacheStore and dalli gem

We can use a memcached server for centralized caching in our application. This server is handling all caching mechanisms. Rails is using the gem memcache-client by default. But here I am using the dalli gem. Because it is better than memcache-client, but one thing to say is I am not experienced this. By reading documents, I have concluded that. You just go through this dalli gem link, https://github.com/mperham/dalli. This is a super caching mechanism by Rails in production environments. The Only thing we need to do is

  1. Say you want caching in your application, by adding the following in your environments, add the following in your production.rb file, if you are running in production mode and you are using production.rb file as your environment configuration file.
    2. config.action_controller.perform_caching = true
  2. Just install dalli gem

    3. Add the following in your gem file and do bundle install

    $ gem 'dalli', '2.0.2'
$ bundle install
  3. Install memcached server
    4. $ sudo apt-get install memcached
  4. Start the memcached server if not started
    5. $ sudo /etc/init.d/memcached start
  5. Configure in your environments, add the following in your production.rb file 
    6. config.cache_store = :dalli_store, 'YOUR_PRODUCTION_PUBLIC_DNS:11211',
    { :namespace => "app-YOUR_HOST", :expires_in => 1.day, :compress => true }

    Here ‘YOUR_PRODUCTION_PUBLIC_DNS:11211’ is saying where your memcached server is running. Public DNS and the port number of the memcached server.

Take the rails console and experiment something like the following to know the MemCacheStore cache mechanism is working. If you are in localhost then do

ruby-1.9.2-p290 :003 > cache = Dalli::Client.new('localhost:11211')
 => #0}, @ring=nil> 
ruby-1.9.2-p290 :004 > cache.set('testing', 1234)
 => true
ruby-1.9.2-p290 :006 > cache.get('testing')
 => 1234

Now you are ready to go

Read Files From Amazon s3 with Expiry

Suppose you have a need that is to download a file from amazon s3, that stored in http://s3.amazonaws.com//file.doc, if it is not accessable to public you will not get.

You can get an idea about Authenticated read by reading the following
(Reference: http://www.bucketexplorer.com/documentation/amazon-s3–access-control-list-details.html)

ACL and its Workings

Amazon S3 allows users to store their objects in Buckets. All Buckets and Objects are associated with Access control policies. ACL is a mechanism which decides who can access what. ACL is the set of permissions of read,write and update on Object as well as Bucket on the basis of these ACLs user can perform operation of upload new files, delete existing objects.

Bucket ACLs are completely independent of Object ACLs. It means that ACLs set on a bucket can be different of ACLs set on any object, contained in bucket.

Types of ACL provided by Amazon S3:

With reference to Bucket:

  • Read: Authorized user can list the file names, their size and last modified date from a bucket.
  • Write: Authorized user can upload new files in your bucket. They can also delete files on which they don’t have permission. Someone with write permission on a bucket can delete files even if they don’t have read permission to those files.
  • Read ACP: Authorized users can check ACL of a bucket.
  • Write ACP: Authorized user can update ACL of the bucket.

With reference to Object:

  • Read: Authorized user can download the file.
  • Write: Authorized user can replace the file or delete it.
  • Read ACP: Authorized user can list ACL of that file.
  • Write ACP: Authorized user can modify the ACL of the file.

Who can Access and How?

Amazon grants permission to four types of users:

  1. Owner (Account Holder): Person who holds Amazon s3 Account is also known as owner of the service. By default owner has full permission. Owner can create access and delete objects. She can also view and modify ACLs of each and every Bucket and its object(s).
  2. Amazon S3 Users (by Adding Amazon.com email address or Canonical Id)
    If owner wants to share or allow another AmazonS3 user to access her bucket, then owner should know the email address of the invitee, email address only works if invitee has registered her Amazon s3 account with that email address.
  3. Authenticated User (Sharing globally with all Amazon s3 Users)
    Anyone with a valid S3 account is a member of “Authenticated Users” group.If Owner wants to share her bucket globally with all Amazon’s s3 users then she can give read permission to authenticated user see the objects and can give write permission to update existing and upload new objects.
  4. Non Authenticated Users (All Users)
    If owner wants to make public her bucket and objects with all internet users, then she needs to give the appropriate permissions to ALL USERS. Now any user will be able to access the object provided name of the bucket.

Amazon s3 Request Url without expiry

So if you want private files from Amazon s3 access by, giving the correct url by giving the access key id and secret access key.

Eg:http://s3.amazonaws.com//file.docAWSAccessKeyId=EOKJGAKIAIHHMD3HP5OLLME5N4A&Signature=0ipwRz3sss6xnfAbebtigAGNOysdfdf1sDpKCl0%3D

Expire the Amazon s3 Request Url

If anyone access this url they can get the files. So here comes the use of expiring a request url. Create a url with access key id and secret access key and expires this after some seconds say 10 seconds.

Eg: http://s3.amazonaws.com//file.doc?AWSAccessKeyId=EOKJGAKIAIHHMD3HP5OLLME5N4A&Expires=1325481379&Signature=0ipwRz3sss6xnfAbebtigAGNOysdfdf1sDpKCl0%3D

Ruby gem aws-s3 and the Class AWS::S3::Base

aws-s3 is a Ruby library for Amazon’s Simple Storage service’s (S3) REST API. AWS::S3::Base is the abstract super class of all classes who make requests against S3.

Establishing a connection with the Base class is the entry point to using the library:

  AWS::S3::Base.establish_connection!(:access_key_id => '...', :secret_access_key => '...')
The :access_key_id and:secret_access_key are the two required connection options.

Create, write and read a file and also print the first line of a file in ruby

Create a file: Open a file in write mode

ruby-1.9.2-p290 :058 > f = File.open(“my_file.txt”, “w”)
=>

Write something to that file

ruby-1.9.2-p290 :059 > f.write(“This is the first line of my file\n”)
=> 34

Close the file

ruby-1.9.2-p290 :060 > f.close
=> nil

Open the file for reading

ruby-1.9.2-p290 :061 > f = File.open(“my_file.txt”, “r”)
ruby-1.9.2-p290 :062 > f.read
=> “This is the first line of my file.\n”

Print the first line of a file

ruby-1.9.2-p290 :063 > lines = IO.readlines(“my_file.txt”)
ruby-1.9.2-p290 :063 > puts lines.first
This is the first line of my file.

A useful link about ruby File manupulation is given below

Open ods, xls, xlsx files in ruby by using ‘roo’

Install ‘roo’ gem
$ sudo gem install ‘roo’
take the ruby console
$ irb
for example take an xlxs file and try to open

ruby-1.9.2-p290 :003 > require ‘roo’
if you get an error like
/home/prescience-toshiba/.rvm/gems/ruby-1.9.2-p290@rails31rc/gems/roo-1.10.0/lib/roo.rb:6: warning: already initialized constant VERSION
/home/prescience-toshiba/.rvm/gems/ruby-1.9.2-p290@rails31rc/gems/roo-1.10.0/lib/roo.rb:7: warning: already initialized constant LIBPATH
/home/prescience-toshiba/.rvm/gems/ruby-1.9.2-p290@rails31rc/gems/roo-1.10.0/lib/roo.rb:8: warning: already initialized constant PATH
LoadError: no such file to load — zip/zipfilesystem
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from /home/prescience-toshiba/.rvm/gems/ruby-1.9.2-p290@rails31rc/gems/roo-1.10.0/lib/roo/openoffice.rb:3:in `’
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from /home/prescience-toshiba/.rvm/gems/ruby-1.9.2-p290@rails31rc/gems/roo-1.10.0/lib/roo.rb:71:in `’
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from (irb):3
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/bin/irb:16:in `’
ruby-1.9.2-p290 :004 >

Then install the ‘rubyzip2’ gem

$ gem install rubyzip2

and open the file
ruby-1.9.2-p290 :003 > file = Excelx.new(“/home/prescience-toshiba/Documents/abhilash.raw.xlsx”)

output:
=> {[1, 1]=>”Name”, [1, 2]=>”Email Id”, [1, 3]=>”Alternate Number”, [1, 4]=>”Date of Birth”, [1, 5]=>”Mobile No.”, [1, 6]=>”Functional Area”, [1, 7]=>”Area of Specialization”, [1, 8]=>”Industry”, [1, 9]=>”Resume Title”, [1, 10]=>”Key Skills”, [1, 11]=>”Work Experience”, [1, 12]=>”Current Employer”, [1, 13]=>”Previous Employer”, [1, 14]=>”Current Salary”, [1, 15]=>”Level”, [1, 16]=>”Current Location”, [1, 17]=>”Preferred Location”, [1, 18]=>”Course(Highest Education)”, [1, 19]=>”Name”, [1, 20]=>”Email Id”, [1, 21]=>”Alternate Number”, [1, 22]=>”Date of Birth”, [1, 23]=>”Mobile No.”, [1, 24]=>”Functional Area”, [1, 25]=>”Area of Specialization”, [1, 26]=>”Industry”, [1, 27]=>”Resume Title”, [1, 28]=>”Key Skills”, [1, 29]=>”Work Experience”, [1, 30]=>”Current Employer”, [2, 1]=>”Rahel Robert”, [2, 2]=>”rtrobert@yahoo.com”, [……………………..

Ruby gem roo error: LoadError: no such file to load — zip/zipfilesystem

ruby-1.9.2-p290 :003 > require ‘roo’
/home/prescience-toshiba/.rvm/gems/ruby-1.9.2-p290@rails31rc/gems/roo-1.10.0/lib/roo.rb:6: warning: already initialized constant VERSION
/home/prescience-toshiba/.rvm/gems/ruby-1.9.2-p290@rails31rc/gems/roo-1.10.0/lib/roo.rb:7: warning: already initialized constant LIBPATH
/home/prescience-toshiba/.rvm/gems/ruby-1.9.2-p290@rails31rc/gems/roo-1.10.0/lib/roo.rb:8: warning: already initialized constant PATH
LoadError: no such file to load — zip/zipfilesystem
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from /home/prescience-toshiba/.rvm/gems/ruby-1.9.2-p290@rails31rc/gems/roo-1.10.0/lib/roo/openoffice.rb:3:in `’
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from /home/prescience-toshiba/.rvm/gems/ruby-1.9.2-p290@rails31rc/gems/roo-1.10.0/lib/roo.rb:71:in `’
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require’
from (irb):3
from /home/prescience-toshiba/.rvm/rubies/ruby-1.9.2-p290/bin/irb:16:in `’
ruby-1.9.2-p290 :004 >

Instead of roo do

ruby-1.9.2-p290 :004 > require ‘roo/excel’

This works fine. If you want zip gem then install it.

$ gem install rubyzip2

Then do

ruby-1.9.2-p290 :001 > require ‘roo’

=> true

Installing rails 3.1 and ruby 1.9.2-p180 using rvm and unicorn setup

Install rvm

Open your .bashrc file.
And replace the following line
[ -z “$PS1” ] && return
with
if [[ -n “$PS1” ]]; then
Now add this to the last line of the file
if [[ -s $HOME/.rvm/scripts/rvm ]] ; then source $HOME/.rvm/scripts/rvm ; fi

fi
This loads RVM into a shell session.

If already have rvm then
Update rvm

$ rvm get latest

if this shows error like ‘downloaded does not match it’s md5 checksum’

then run

$ rvm get head
$ rvm reload
$ rvm get latest

For ruby 1.9.2 we want the packages it lists under the MRI & ree section. Let’s install those now.

$ sudo aptitude install build-essential bison openssl libreadline5 libreadline-dev curl git-core zlib1g zlib1g-dev libssl-dev vim libsqlite3-0 libsqlite3-dev sqlite3 libreadline-dev libxml2-dev git-core subversion autoconf

Setting up Ruby

Do rvm list known to know all ‘Ruby implementation made available through RVM’

$ rvm list known

Then install latest stable release of ruby

$ rvm install 1.9.2

You can see the ruby versions installed and currently using ruby version by

$ rvm list rubies

To list default ruby

$ rvm list default

You can change it by

$ rvm use ruby-1.9.2-p180

for default : rvm –default 1.9.2-p180

Using Gemsets

Install rails stable release and then rails 3.1
Start by creating our gemsets

$ rvm gemset create rails307 rails31

See available gem sets by

$ rvm gemset list

If any confusion with rails31 gemset delete it

$ rvm gemset delete rails31

And create rails31rc

$ rvm gemset create rails31rc

As a best practice, remember to always use one gemset per project*.

Installing Rails

Now we have multiple gem set installed. Now set the different gem set for ruby versions.

$ rvm use 1.9.2-p180@rails307

Install Rails 3.0.7

$ gem install rails [-v 3.0.7] [–no-rdoc]

Select the gemset for Rails 3.1

$ rvm use 1.9.2-p180@rails31rc

for default $ rvm use 1.9.2-p180@rails31rc –default

Install Rails 3.1

$ gem install rails -v “>=3.1.0rc”

if not found

$ gem install rails -v “>=3.1.0.rc4”

rvmrc file

Switching from one project to another, from a client to a personal project, from testing the release candidate to developing using the latest stable version, always having to manually switch from using a gemset to another can impact productivity. The project .rvmrc files can increase the development speed by setting up our project’s ruby environment when we switch to the project root directory.

The rule of thumb here is to use a .rvmrc file for each project, for both development and deployment.*

.rvmrc file :

rvm –rvmrc use 1.9.2-p180@rails31rc

if you go to your app folder ‘$ cd myapp’ then you can see like





There are three types of .rvmrc files
1) system : on /etc/rvmrc
2) user : $HOME/.rvmrc
3) Project : .rvmrc

Create a new rails App

$ rails new myapp

with mysql as database

$ rails new myapp -d mysql
$ rails server -p4000

If u get error like

“”” home/user/.rvm/gems/ruby-1.9.2-p180@rails31rc4/gems/execjs-1.1.2/lib/
execjs/runtimes.rb:43:in `autodetect’: Could not find a JavaScript
runtime. See https://github.com/sstephenson/execjs for a list of
available runtimes. (ExecJS::RuntimeUnavailable) “””

Then you need a javascript engine to run your code…

try installing rubyracer (Embeds the V8 Javascript interpreter into Ruby.)

in your Gemfile add

gem ‘therubyracer’
and do bundle install

Unicorn Installation

$ rvm use default # Just to make sure
$ gem install unicorn

Start unicorn server

$ unicorn_rails

A Program for finding all files in a nested folder that contains a specific string : Ruby Vs Python

This program has a major method ‘search in file’ that accepts a folder path as a parameter. Next initialize whatever variables we want.

Ruby
Python

def search_in_file(folder_path)
      @file_array = Array.new
      files_has_string = Array.new
      erb_files_has_no_string = Array.new
      rb_files_has_no_string = Array.new
      unknown_files = Array.new
      iterate_through_dir(folder_path, @file_array)
      ———
      ———

import os
import re
file_array = []
def search_in_file(folder_path):
      files_has_string = []
      erb_files_no_string = []
      rb_files_no_string = []
      unknown_files = []
      iterate_through_dir(folder_path)
      ———
      ———

Then calls a method ‘iterate_through_dir’ which has two parameters in ruby and one in python, ‘folder path’ and instance variable ‘file_array’. In python we initialized a gobal variable ‘file_array’. But in ruby ‘@file_array’ is an instance variable. This method finds file paths of files included in the folder we have given, say ‘sample_folder’. Using the class method ‘glob’ in ruby and os module’s listdir in python we can find all the files and folders that inside sample_folder. Next check it is a file or a folder. If it is a folder we need to iterate through that folder(use recursion ) and get the file paths of files inside the folder. Else it is a file push that file path into our instance array ‘file_array’.

Ruby
Python

def iterate_through_dir(dir_path, file_array)
      file_paths = Dir.glob(dir_path + “/*”)
      file_paths.each do |f|
         if File.file?(f)
            file_array.push(f)
         else
            iterate_through_dir(f, file_array)
         end
      end
end

def iterate_through_dir(dir_path):
      file_paths = os.listdir(dir_path)
      for i in file_paths:
         path_to_dir = dir_path + “/” + i
         if os.path.isdir(path_to_dir) == True:
            iterate_through_dir(path_to_dir)
         else:
            file_array.append(path_to_dir)

Now we got all the file paths of files inside the nested folder sample_folder in @file_array / file_array. Take each element in the @file_array / file_array, and open that file in read mode. Call ‘f.read’ in ruby and ‘f.read()’ in python to get the file contents as a string.

Ruby
Python

      ———
      @file_array.each do |file_path|
      f = File.open(file_path, ‘r’)
      file_contents_in_string = f.read
      ———
      ———

      ———
      for i in file_array:
      f = open(i, ‘r’)
      file_content = f.read()
      ———
      ———

Then call ruby-‘.match’, python – ‘re.search'(import regular expression module first -import re) on file content string to find the string inside the file. If you find the string then put it in an array. If you can’t find then put it in an another array. Before that find the file extensions to get the different type of files and append that file paths to different arrays.

Ruby
Python

———
   if (file_contents_in_string[0..500].match(“your string”))
      files_has_string << file_path
      else
        file_extension = File.basename(file_path).split(‘.’)[1]
        if file_extension == ‘rb’
           rb_files_has_no_string << file_path
        elsif file_extension == ‘erb’
           erb_files_has_no_string << file_path
        else
           unknown_files << file_path
      end
end
end

———
   if re.search(‘find_this_string’, file_content[0:500]):
      files_has_string.append(i)
      else:
       &  file_extension = os.path.basename(i).split(‘.’)[1]
       & if if file_extension == ‘rb’:
           rb_files_no_string.append(i)

        elif file_extension == ‘erb’:
         erb_files_no_string.append(i)
      else:
           unknown_files.append(i)

At last call the print_results method to print results.

Ruby
Python

——–
      print_results(@file_array, rb_files_has_no_string, erb_files_has_no_string, unknown_files)
end

print search_in_file(“/your/path/to/nested/folder”)

——–
      print_results(rb_files_no_string, erb_files_no_string, unknown_files)

print search_in_file(“/your/path/to/nested/folder”)

Output : Ruby

Output : Python

search_a_string
The method ‘print_results’ accepts four arrays and prints the results accordingly.

Install Rails 3 on Ubuntu 10.04 Lucid Lynx Using RVM with Mysql and sphinx

First install ‘curl’ and ‘git’

  • step 1: install curl
    $ sudo apt-get install curl
  • step 2: install git
    $ sudo apt-get install git-core

    • Then install RVM

  • step 3: install rvm
    $ bash < <( curl http://rvm.beginrescueend.com/releases/rvm-install-head )
  • step 4: edit .bashrc file to load rvm into the shell session
    $ gedit .bashrc
    Add the following line to the end of the file:
    if [[ -s “$HOME/.rvm/scripts/rvm” ]] ; then source “$HOME/.rvm/scripts/rvm” ; fi
  • step 5 : restart linux
    $ sudo reboot
  • step 6 : type
    $ rvm notes
    In the notes RVM tells you what packages you’re gonna need to install for various flavors of Ruby. Since we’re going with 1.9.2 we want the packages it lists under the MRI & ree section. Let’s install those now.
    $ sudo aptitude install build-essential bison openssl libreadline5 libreadline-dev curl git-core zlib1g zlib1g-dev libssl-dev vim libsqlite3-0 libsqlite3-dev sqlite3 libreadline-dev libxml2-dev git-core subversion autoconf

    • Install Ruby 1.9.2

  • step 7 : install ruby 1.9.2 through rvm
    $ rvm list known
    $ rvm install 1.9.2-head
    for 64bit:
    $ rvm install 1.9.2
  • step 8 : make ruby 1.9.2 as default
    $ rvm –default ruby-1.9.2-head
    for 64bit:
    $ rvm –default ruby-1.9.2
    check ruby version
    $ ruby -v

    • Install Rails 3

  • step 9 : Install rails 3
    $ gem install i18n tzinfo builder memcache-client rack rack-test rack-mount erubis mail text-format thor bundler
    $ gem install rails –pre

    • Select your database

  • step 10 : Install database
    For Sqlite3
    $ sudo apt-get install sqlite3 libsqlite3-dev
    $ gem install sqlite3-ruby
    (OR)
    $ sudo apt-get update
    For Mysql goto synaptic package manager and type
    mysql-server, mysql-client and install and set password.
    Install mysql client libraries :
    $ sudo aptitude install libmysqlclient16-dev
    or
    $ sudo apt-get install libmysql-ruby libmysqlclient-dev

    • Install Sphinx

  • If you want sphinx, install Sphinx
    $ wget http://sphinxsearch.com/downloads/sphinx-0.9.8.1.tar.gz
    $ tar xzvf sphinx-0.9.8.1.tar.gz
    $ cd sphinx-0.9.8.1/
    $ ./configure
    $ make
    $ sudo make install

    • To know about sphinx type
    $ search

If you are getting error when starting sphinx make sure that the ‘indexer’,’searchd’,’search’ is under /usr/local/bin or under where sphinx specifies.
You are now ready to go…