Our problem with micro-services using AWS ECS

We are into a startup and as a trending technology our predecessors used micro-services for our new website.

It is a good move if we see the benefits of micro-services like

  • scaling a website gets much easier
  • loosely coupled
  • easy for developing
  • deployment can focus on each service
  • can be used different technologies for each service
  • testing can be concentrated on one service

Here we are considered the problems we may face in future and developed the application thinking about the benefits of micro-services.

Point to be noted: We also have to think about – does these problems will be a big issue for us compared to the disadvantages of the micro-services?

NOTE: Our Website is a less traffic website and we are getting our clients one by one. It is not a heavily used website. Keep this in mind while reading this article.

We faced issues following issues with micro-services:

  • Complexity – more complex for deployment and maintenance
  • Cost – AWS ECS costs more for all the micro-service hosting for a less traffic website
  • Even though these are micro-services it has some dependency between them that we cannot avoid. Be ready to deal with this.
  • Sometimes one service (AWS Container) goes offline and starting over! So we get the No Service issue in the website.
  • Very difficult to go back to monolith application even clubbing 3-4 services together, as they use different software or software versions.

So be careful if you are planning to use micro-service architecture for a less used website or as a starting of your business. Take your time and think if micro-service architecture suits your business and your current situation.

The fact: It should not take that much cost for a website which has less traffic. Be ready to pay minimum amount for hosting with micro-services.

Set up:

  • Used AWS ECS (ec2 launch type) with services and task definitions defined
  • 11 Micro-services, 11 containers are spinning
  • Cost: Rs.12k ($160) per month


  • Consider using AWS Fargate type but not sure these issues get resolved
  • Deploy all the services in one EC2 Instance without using ECS

Read Files From Amazon s3 with Expiry

Suppose you have a need that is to download a file from amazon s3, that stored in http://s3.amazonaws.com//file.doc, if it is not accessable to public you will not get.

You can get an idea about Authenticated read by reading the following
(Reference: http://www.bucketexplorer.com/documentation/amazon-s3–access-control-list-details.html)

ACL and its Workings

Amazon S3 allows users to store their objects in Buckets. All Buckets and Objects are associated with Access control policies. ACL is a mechanism which decides who can access what. ACL is the set of permissions of read,write and update on Object as well as Bucket on the basis of these ACLs user can perform operation of upload new files, delete existing objects.

Bucket ACLs are completely independent of Object ACLs. It means that ACLs set on a bucket can be different of ACLs set on any object, contained in bucket.

Types of ACL provided by Amazon S3:

With reference to Bucket:

  • Read: Authorized user can list the file names, their size and last modified date from a bucket.
  • Write: Authorized user can upload new files in your bucket. They can also delete files on which they don’t have permission. Someone with write permission on a bucket can delete files even if they don’t have read permission to those files.
  • Read ACP: Authorized users can check ACL of a bucket.
  • Write ACP: Authorized user can update ACL of the bucket.

With reference to Object:

  • Read: Authorized user can download the file.
  • Write: Authorized user can replace the file or delete it.
  • Read ACP: Authorized user can list ACL of that file.
  • Write ACP: Authorized user can modify the ACL of the file.

Who can Access and How?

Amazon grants permission to four types of users:

  1. Owner (Account Holder): Person who holds Amazon s3 Account is also known as owner of the service. By default owner has full permission. Owner can create access and delete objects. She can also view and modify ACLs of each and every Bucket and its object(s).
  2. Amazon S3 Users (by Adding Amazon.com email address or Canonical Id)
    If owner wants to share or allow another AmazonS3 user to access her bucket, then owner should know the email address of the invitee, email address only works if invitee has registered her Amazon s3 account with that email address.
  3. Authenticated User (Sharing globally with all Amazon s3 Users)
    Anyone with a valid S3 account is a member of “Authenticated Users” group.If Owner wants to share her bucket globally with all Amazon’s s3 users then she can give read permission to authenticated user see the objects and can give write permission to update existing and upload new objects.
  4. Non Authenticated Users (All Users)
    If owner wants to make public her bucket and objects with all internet users, then she needs to give the appropriate permissions to ALL USERS. Now any user will be able to access the object provided name of the bucket.

Amazon s3 Request Url without expiry

So if you want private files from Amazon s3 access by, giving the correct url by giving the access key id and secret access key.


Expire the Amazon s3 Request Url

If anyone access this url they can get the files. So here comes the use of expiring a request url. Create a url with access key id and secret access key and expires this after some seconds say 10 seconds.

Eg: http://s3.amazonaws.com//file.doc?AWSAccessKeyId=EOKJGAKIAIHHMD3HP5OLLME5N4A&Expires=1325481379&Signature=0ipwRz3sss6xnfAbebtigAGNOysdfdf1sDpKCl0%3D 

Ruby gem aws-s3 and the Class AWS::S3::Base

aws-s3 is a Ruby library for Amazon’s Simple Storage service’s (S3) REST API. AWS::S3::Base is the abstract super class of all classes who make requests against S3.

Establishing a connection with the Base class is the entry point to using the library:

  AWS::S3::Base.establish_connection!(:access_key_id => '...', :secret_access_key => '...')
The :access_key_id and:secret_access_key are the two required connection options.