Implementing Ruby Camping REST Services With RESTstop
Intro
It is pretty common nowadays for web applications and sites to expose a developer API to allow consumers to integrate their data with other application. Examples: Twitter, LinkedIn, Bit.ly, etc.
Although there are various options for implementing web services, as of 2010, the REST protocol seems to have achieved a comfortable lead over other approaches for Internet-based resource-oriented services.
RESTstop is a Ruby library making it easy to implement REST services
on top of the Camping framework.
Web Services Implementation Options
Before jumping to RESTstop (if you insist click here), let’s do a brief review of the few options,
developers have historically leveraged to build web services:
- SOAP services: the API is formally defined using the WSDL format. API messages use a specific XML representation.
Additional services can be composed on top of the basic protocol to provide authentication, authorization, etc.
Although powerful, SOAP has proven too complex and too heavy for web developer APIs .
Also they require a full SOAP-compatible client stack for making them more difficult to call especially in the case of AJAX applications.
As a consequence, SOAP services tend to have been relegated to intra-enterprise cross-application services. - XML services: the API is not formally defined but end-points and their inputs and outputs are usually documented on a developer site.
Each service has a given url and uses XML to represent messages. As such an XML parser is required to provide robust processing
especially if the data structures are complex and rely on namespaces. - JSON services: similarly to the XML services the API is also not formally defined.
Messages are represented as JSON structures.
JSON services lend themselves very well to AJAX calls from browser applications since
JSON structures can be easily processed using basic Javascript. - APIs for web sites mostly need to expose “resources” as opposed to behaviors (e.g. : CRUD operations on Posts)
Camping for Web Services
Since Camping is a generic Ruby web application framework, you can define service endpoints as url routes,
each mapped to a given controller.
An HTML-based view is not necessary obviously, so instead one option is to render the model
using the desired format: XML or JSON.
Example:
gem 'camping' , '>= 2.0' %w(rubygems active_support active_support/json active_record camping camping/session markaby erb ).each { | lib | require lib } Camping.goes :CampingJsonServices module CampingJsonServices include Camping::Session def CampingJsonServices.create end module CampingJsonServices::Controllers class APIDateToday < R '/datetoday' def get @result = {:today=>Date.today.to_s} @headers['Content-Type'] = "application/json" @result.to_xml(:root=>'response') end end class APITimeNow < R '/timenow' def get @result = {:now=>Time.now.utc.to_s} @headers['Content-Type'] = "application/json" @result.to_json end end end end
Such as an approach is fine for function-oriented APIs but if we want to provide CRUD access to resources,
then a REST approach provides more structure. You can of course implement a REST-style API on your own
by overriding the route dispatch mechanism and mapping the HTTP verbs such as PUT and DELETE
to the appropriate route controller methods.
RESTstop, a REST plugin for Camping
Matt Zukowski wrote the RESTstop library to easily provide access to REST resources using the underlying Camping route engine and controllers.
The idea was powerful yet simple, as a developer you would create a controller per resource
and specify the REST nature of the route like so:
module Blog::Controllers extend Reststop::Controllers class Posts < REST 'posts' end end
Then similarly to the standard get and post methods of Camping controllers, you would create CRUD-style action methods
such as: create, read, update, delete, and list:
class Posts < REST 'posts' # POST /posts def create end # GET /posts/1 # GET /posts/1.xml def read(post_id) end # PUT /posts/1 def update(post_id) end # DELETE /posts/1 def delete(post_id) end # GET /posts # GET /posts.xml def list end end
Then once you had a controller, you would implement a special type of view appropriate for the format of the data you wanted to expose.
For example you could create views to render the data as XML or JSON or any other format you liked.
module Blog::Views extend Reststop::Views module HTML include Blog::Controllers include Blog::Views def index if @posts.empty? p 'No posts found.' else for post in @posts _post(post) end end p { a 'Add', :href => R(Posts, 'new') } end end module XML include Blog::Controllers include Blog::Views def index @posts.to_xml(:root => 'blog') end end module JSON include Blog::Controllers include Blog::Views def index @posts.to_json end end
The Inner Plumbing Of RESTstop
RESTstop is an interesting plugin to dissect as it is a good illustration of meta-programming in Ruby.
Matt took the following approach:
- Alias and override the base service method to save off the HTTP verb
- Add a new route creation method called REST to replace the base r route creation method.
The new method will register all the expected REST resource routes and wire them up to the controller. - Default or decode the rendering format specified in the url (e.g. /posts.xml vs. /posts.json)
- Structure the Views module according to the various formats to specify: HTML vs. XML vs. JSON
- Render the controller output using the specified format
I would definitely recommend that your read the plugin's source. At 449 lines with many comments
it is pretty easy to see how it actually works. This will also make it easier for you to troubleshoot your code when needed.
Let's Create The REST Version Of The Blog Sample
Here is an outline of the approach:
# | To Do | Area (Module) |
---|---|---|
1 | Plug in RESTstop into the app | Main app |
2 | Plug in RESTstop into the Base | Base |
3 | Plug in RESTstop into the Controllers | Controllers |
4 | Create the REST Sessions controller | Controllers |
5 | Plug in RESTstop into the Helpers | Helpers |
6 | Plug in RESTstop into the Views | Views |
7 | Finish the REST Sessions controller | Controllers |
8 | Add JSON support | Views |
9 | Create the REST Posts controller | Controllers |
10 | Add an HTML view module for Posts | Views |
11 | Add an XML view module for Posts | Views |
12 | Add a JSON view module for Posts | Views |
1. Plug In RESTstop
Our first step is to install the gem and add a require for RESTstop:
gem install reststop
Now let's create a new Ruby source file and name it camping-svcs.rb.
Then let's require RESTstop and include its main module in the blog's main module.
require 'reststop' Camping.goes :Blog module Blog include Camping::Session include Reststop end
2.Plug in RESTstop into the Base
Now we need to enhance the Base module, so let's specify it.
This is the place where we'll add most of the extensions code taking care of:
- Aliasing 3 methods: render, lookup, and service so RESTstop can "insert" its magic
- Including Reststop::Base in the Base
- Adapt the Camping lookup method to look for view methods inside the RESTstop-specific Views::HTML module
So add the following code:
module Blog::Base alias camping_render render alias camping_lookup lookup # @techarch: required if camping > 2.0 alias camping_service service include Reststop::Base alias service reststop_service alias render reststop_render # Overrides the new Tilt-centric lookup method In camping # RESTstop needs to have a first try at looking up the view # located in the Views::HTML module. def lookup(n) T.fetch(n.to_sym) do |k| t = Blog::Views::HTML.method_defined?(k) || camping_lookup(n) end end end
3. Plug in RESTstop into the Controllers
Here we'll add an extend for the Reststop::Controllers.
module Blog::Controllers extend Reststop::Controllers end
This will add 2 new methods
- determine_format : will extract the service data format from the request url - e.g json from /posts.json
- REST : will allow definition of REST resources in our controller declarations - e.g. class Posts < REST 'posts'
4. Create the REST Sessions controller
The Camping Blog example has a simple login system built on top of the Camping session.
For our REST blog, we'll want to support both an interactive web app login (like in the original blog)
as well as a service-based login so that only authenticated applications can call our services.
So we'll expose a new REST resource called Session. Applications will be able to create and delete sessions -
to mirror the login and logout behavior. So let's create our first REST controller and name it Sessions:
class Sessions < REST 'sessions' end
Now we just need to declare only a subset of the standard CRUD REST methods: create, and delete.
For now, let's just create the shell:
# POST /sessions def create end # DELETE /sessions def delete end
For the create method, we will lookup the user based on the user id and password.
If not found we'll return a 401 (unauthorized) error with a message.
If found, we'll create a Camping session and render a view we'll create in a bit called user.
Here is the code:
# POST /sessions def create @user = User.find_by_username_and_password(input.username, input.password) if @user @state.user_id = @user.id render :user else r(401, 'Wrong username or password.') end end
Let's test the error scenario. For that we'll use the following 2 tools:
- The restr gem. This is another nice library created by Matt.
gem install restr
We'll use the library from IRB as a simple REST client.
- Pocket Soap's TcpTrace, a tool to inspect the contents of Tcp messages.
This will really help us troubleshoot the various interactions between client and service.
Ok so let's start our camping app on the default 3301 port:
camping camping-svcs.rb
Now let's run TcpTrace and start a new trace listening on port 8080 and forwarding to port 3301.
Note: we'll set our IRB-based client to post to port 8080).
Then start IRB and run the following statements to invoke create on the Sessions resource:
gem 'xml-simple' gem 'restr' require 'restr' # Set the url of our resource (via port 8080 for TcpTrace) u0 = "http://localhost:8080/sessions.xml" # Set the user name and incorrect password o = { :username=>'admin', :password=>'wrong'} # Post to the Sessions resource p0 = Restr.post(u0,o)
In TcpTrace you should see the request and the response:
In IRB you should also see the 401 error:
So now we have successfully tested the error scenario!
Let's go back to our code and continue setting up our RESTstop code so we can then define our user view.
5. Plug in RESTstop into the Helpers
We'll explicitly define our Helpers module so we can:
- alias the R method since RESTstop will need to intercept it to provide a way to create routes to resources (e.g. R(Posts/1)
- include the Reststop::Helpers module
module CampingRestServices::Helpers alias_method :_R, :R remove_method :R include Reststop::Helpers end
6. Plug in RESTstop into the Views
In the Views module, we'll first add an extend for the RESTstop Views module
module CampingRestServices::Views extend Reststop::Views end
Then we'll create 3 sub-modules, one for each type of format we'll support: HTML, XML, and JSON.
We'll also indicate that HTML is the default format.
For the moment we'll just create shells:
module CampingRestServices::Views extend Reststop::Views module HTML end module XML end module JSON end default_format :HTML end
Since we needed to create a user view when successfully creating a Session resource,
let's create a user view method in the XML submodule. The code will just serialize the @user object to XML.
module XML def user @user.to_xml(:root => 'user') end end
We're now ready to test the happy path!
So go back to your IRB session and evaluate the following statements:
# Set the user name and correct password o = { :username=>'admin', :password=>'camping'} # Post to the Sessions resource p0 = Restr.post(u0,o)
In TcpTrace you should see the request and the response containing the serialized user:
In IRB you should also see the returned user instance, deserialized from XML
7. Finish the REST Sessions controller
The remaining part of our Sessions controller is the delete method.
Delete will terminate the session and return a 200 message to that effect:
class Sessions < REST 'sessions' # ... # DELETE /sessions def delete @state.user_id = nil r(200, 'Session terminated') end end
Let's test this out from IRB by evaluating the following statement to DELETE the current session:
p0=Restr.delete(u0,{})
In TcpTrace you should see the request and the response containing the session termination message:
In IRB you should also see the same message:
So here is a recap for what we have accomplished so far:
- We have plugged in RESTstop in the following modules:
- We have created our first REST controller: Sessions implementing the create and delete methods
- We have created an XML submodule in our Views module to render the user XML view
- We have monitored the message protocol using TcpTrace
- We tested the API using the restr client library
8. Add JSON Support
Our Session resource can currently only be accessed via an XML REST interface.
So let's add JSON support by adding a user view to the JSON submodule (within our Views module):
module JSON def user @user.to_json end end
Let's go back to your IRB session and evaluate the following statements:
# Set the url for the JSON Resource format u0b = "http://localhost:8080/sessions.json" # Post to the Sessions resource p0=Restr.post(u0b,o)
In TcpTrace you should see the request and the response containing the serialized user in JSON format:
In IRB you should also see the returned user instance, deserialized from JSON:
At this stage, our service can provide authentication via the Sessions resource using either an XML or JSON protocol.
I will leave the HTML implementation for the reader. The approach would include re-using the existing Login controller,
but the login view would be moved to the HTML submodule and could be modified to POST to the Sessions resource.
So we now have the basic building authentication block for the rest of the functionality for our service.
Although this would work fine, this approach is not very robust for industrial-strength services.
I would recommend using OAuth as it provides more features such as:
- per-user consumer app registration
- token-based authorization
- explicit user-based authorization of tokens
I recently created an OAuth plugin for Camping so check out the details on this post here.
9. Create the REST Posts controller
We will model the Posts REST controller after the Sessions controller, with a few new twists such as the
read, update and list methods.
Currently, in the Camping Blog example, there are several controllers responsible for managing posts: Index, PostNew, PostN, and Edit.
We will combine them into a brand new REST controller called Posts. So let's define a Posts controller with a REST route of 'posts':
class Posts < R 'posts' end
Now we just need to declare the standard CRUD REST methods: create, read, update, delete, and list.
For now, let's just create the shell:
class Posts < R 'posts' # POST /posts def create end # GET /posts/1 or # GET /posts/1.xml or # GET /posts/1.json def read(post_id) end # PUT /posts/1 def update(post_id) end # DELETE /posts/1 def delete(post_id) end # GET /posts or # GET /posts.xml or # GET /posts.json def list end end
Let's start with the simplest method: list.
We will just take the code from the old Index controller's get method
(and later we'll move the index view method into the HTML submodule).
# GET /posts # GET /posts.xml def list @posts = Post.all(:order => 'updated_at DESC') render :index end
Now for the create method, we will start with the code from the old PostNew controller's post method.
But we will tweak the Post.create parameters list to accept either input.post_title (when coming from an HTML form)
or input.title (when coming from a REST service call). We'll do the same for the body input parameter.
Then we will adjust the redirect call to use the RESTstop-enhanced R route creation method as opposed
to redirecting to the old PostN controller.
So all in all the code will now look like this:
# POST /posts def create require_login! @post = Post.create :title => (input.post_title || input.title), :body => (input.post_body || input.body), :user_id => @state.user_id redirect R(@post) end
The read method is easy as it is a straight copy paste from the old PostN controller's get method
(and later we'll move the view view method into the HTML submodule).
# GET /posts/1 # GET /posts/1.xml def read(post_id) @post = Post.find(post_id) render :view end
For the update method, we'll use the same technique as for the create method:
we'll start with the code from the old Edit controller's post method and allow for the
variations in the input parameters (depending on whether we come from an HTML post or a REST post).
We'll also adapt the redirect to use the new R method. So the code should look like this:
# PUT /posts/1 def update(post_id) require_login! @post = Post.find(post_id) @post.update_attributes :title => (input.post_title || input.title), :body => (input.post_body || input.body) redirect R(@post) end
The original Camping blog does not have code to delete a post, but the code for our delete method
is pretty straightforward:
# DELETE /posts/1 def delete(post_id) require_login! @post = Post.find post_id if @post.destroy redirect R(Posts) else _error("Unable to delete post #{@post.id}", 500) end end
We will need a route to create new blog entries via HTML, so let's add a new action method
to instantiate a new Post and render the add view method:
# GET /posts/new def new require_login! @user = User.find @state.user_id @post = Post.new render :add end
Then finally we will need a route to edit an existing blog entry via HTML, so let's add an edit action method
to lookup an existing Post by id and render it using the edit view method:
# GET /posts/1/edit def edit(post_id) require_login! @user = User.find @state.user_id @post = Post.find(post_id) render :edit end
To finish up the application, there a few more "utility" controllers we'll need such as:
- an Index controller for the "home"/"index" page
- a Login controller
- a Logout controller
So first let's copy but change the original Index controller so that it just redirects to the /posts route
since the logic is now in the list method of the new Posts REST controller.
class Index def get redirect '/posts' end end
Then, we need a Login controller to render the login HTML view:
class Login < R '/login' def get render :login end end
And we also need a Logout controller to render the logout HTML view:
class Logout < R '/logout' def get render :logout end end
You can also migrate the original Styles controller, as well as the CSS declarations (located at the end of the original Camping blog source).
Before we start working on the views, let's define a couple login-related helper methods in our CampingRestServices::Helpers module:
- logged_in? : to test if we have a logged in user
- require_login! : to force a redirect to the login page if we don't have a valid user session
module CampingRestServices::Helpers # ... def logged_in? !!@state.user_id end def require_login! unless logged_in? redirect(R(CampingRestServices::Controllers::Login)) throw :halt end end end
We're now officially done with our Controllers module! :-)
10. Add an HTML view module for Posts
OK now that we have a functional Posts REST controller, we need to create the corresponding views
in the appropriate Views submodule (HTML vs. XML vs. JSON).
Let's start by migrating the old HTML views by copying the contents of the old Views module into
the new HTML submodule
module CampingRestServices::Views extend Reststop::Views module HTML # PLACE THE OLD Views CODE HERE end # ... default_format :HTML end
We will leave the layout view method unchanged.
Our first edit will be in the index method where we'll change the code to
create a route for the "add one" link to route to the '"new" action of our new Posts REST resource
as opposed to routing to the old PostNew controller.
def index if @posts.empty? h2 'No posts' p do text 'Could not find any posts. Feel free to ' a 'add one', :href => R(Posts, 'new') text ' yourself. ' end else @posts.each do |post| _post(post) end end end
The next view method to change is login. In the original Camping version the form will post to the Login controller.
We now need it to post to the new Sessions REST resource. So let's adapt the code as follows:
def login h2 'Login' p.info @info if @info form :action => R(Sessions), :method => 'post' do input :name => 'to', :type => 'hidden', :value => @to if @to label 'Username', :for => 'username' input :name => 'username', :id => 'username', :type => 'text' label 'Password', :for => 'password' input :name => 'password', :id => 'password', :type => 'password' input :type => 'submit', :class => 'submit', :value => 'Login' end end
Once the user has logged in we need to render the user HTML view
(if you remember we created an XML and JSON version but not an HTML one).
So let's add a quick view to greet the user and provide a link to the Posts page:
def user h2 "Welcome #{@user.username}!" a 'View Posts', :href => '/posts' end
And we need to create a logout view too, with a form which once submitted
would invoke the DELETE HTTP verb on the Sessions REST controller:
def logout h2 'Logout' form :action => R(Sessions), :method => 'delete' do input :type => 'submit', :class => 'submit', :value => 'Logout' end end
For the add view, we'll only adjust the posting route
from the old PostNew controller to the new Posts REST controller:
def add _form(@post, :action => R(Posts)) end
For the edit view, we'll also adjust the posting route
from the old Edit controller to an item-specific action (using the post id) on the
new Posts REST controller using the PUT HTTP verb:
def edit _form(@post, :action => R(@post), :method => :put) end
We will copy the view view as-is:
def view _post(@post) end
For the _post partial view, we'll adjust the h2 title link
from the old PostN controller to an item-specific action (using the post id) on the
new Posts REST controller. And we'll add a statement to display the post's body:
def _post(post) h2 { a post.title, :href => R(Posts, post.id) } p { post.body } p.info do text "Written by #{post.user.username} " text post.updated_at.strftime('%B %M, %Y @ %H:%M ') _post_menu(post) end text post.html_body end
The next partial view to update is _admin_menu, where we'll adjust the link
from the old PostNew controller to the new action of the new Posts REST controller:
def _admin_menu text [['Log out', R(Logout)], ['New', R(Posts, 'new')]].map { |name, to| capture { a name, :href => to} }.join(' – ') end
The last partial view to update is _post_menu, where we'll adjust the edit link
from the old Edit controller to an item-specific edit action (using the post id) on the
new Posts REST controller. And we'll also add a link to delete the blog post using
an item-specific delete action.
def _post_menu(post) if logged_in? a '(edit)', :href => R(Posts, post.id, 'edit') span ' | ' a '(delete)', :href => R(Posts, post.id, 'delete') end end
The _form partial view will most remain the same, except for a few visual tweaks:
def _form(post, opts) form({:method => 'post'}.merge(opts)) do label 'Title:', :for => 'post_title' input :name => 'post_title', :id => 'post_title', :type => 'text', :value => post.title br label 'Body:', :for => 'post_body' textarea post.body, :name => 'post_body', :id => 'post_body' br input :type => 'hidden', :name => 'post_id', :value => post.id input :type => 'submit', :class => 'submit', :value => 'Submit' end end
We're now officially done with our Views::HTML module!
So we should be able to test it out: start the Camping server and navigate to the app.
You should get the home/index page.
At this point you should be able to "exercise" all aspects of the app, from login to add, edit, delete, and logout.
11. Add REST XML and JSON view modules for Posts
The HTML submodule was the most complicated believe it or not! Adding the XML view support is very straightforward and consists of:
- a layout method whose only job is to yield the content
- a user view which we already created earlier when testing the Sessions controller
- an index view to return the list of posts formatted as XML
- a view view to return the details of a given posts formatted as XML
The submodule will look like this:
module CampingRestServices::Views extend Reststop::Views # ... module XML def layout yield end def user @user.to_xml(:root => 'user') end def index @posts.to_xml(:root => 'blog') end def view @post.to_xml(:root => 'post') end end # ... end
To test the API you can either:
- Navigate to:
http://localhost:3301/posts.xml
- Or use IRB and the RESTR client, by evaluating the following statements:
u1="http://localhost:8080/posts.xml" # Get all Posts resources p1 = Restr.get(u1,o)
Here are a few other examples to try out from IRB to test the create, read, update, delete methods:
p2={ :title=>'Brand new REST-issued post', :body=>'RESTstop makes it happen!!!'} # Create a new resource p2b=Restr.post(u1,p2) # ----------------------------------------------------- u3 = "http://localhost:8080/posts/1.xml" p3 = Restr.get(u3,o) # Modify the title p3['title']='UPDATED: ' + p3['title'] # Update the resource p3b = Restr.put(u2,p3,o) # ----------------------------------------------------- # Delete a resource p4=Restr.delete(u3)
You have now successfully implemented the XML rendering!
12. Add a JSON view module for Posts
The JSON submodule will be as simple as the XML submodule and will look near identical except for the JSON serialization code:
module CampingRestServices::Views extend Reststop::Views # ... module JSON def layout yield end def user @user.to_json end def index @posts.to_json end def view @post.to_json end end # ... end
Again, to test the JSON API you can either:
- Navigate to the JSON version of the url:
http://localhost:3301/posts.json
- Or use IRB and the RESTR client, by evaluating the following statements:
u1="http://localhost:8080/posts.json" # Get all Posts resources p1 = Restr.get(u1,o)
You have now successfully implemented the JSON rendering!
Overall Recap
So when you need to implement your own REST Camping app, here are the basic steps to remember:
# | To Do | Area (Module) |
---|---|---|
1 | Plug in RESTstop into the app | Main app |
2 | Plug in RESTstop into the Base | Base |
3 | Plug in RESTstop into the Controllers | Controllers |
4 | Plug in RESTstop into the Helpers | Helpers |
5 | Plug in RESTstop into the Views | Views |
6 | Create your REST resource controller | Controllers |
7 | Add an HTML view module for your controller | Views |
8 | Add an XML view module for your controller | Views |
9 | Add a JSON view module for your controller | Views |
10 | Either create a REST Sessions controller or implement OAuth | Controllers |
So What?
Although you can create REST services "by hand" using either basic Ruby web server or using the Camping framework,
RESTstop brings you the following benefits:
- makes it easy to define REST controllers using minimal syntax like:
class Posts < REST 'posts' end
- provides a simple convention for CRUD resource operations
- delegates the API output format to the appropriate format-specific view submodule
- fits nicely within the overall Camping framework architecture
So if you have been hesitant to provide a REST api in your Camping web application due to the anticipated complexity,
or if you want to create REST API separate from your web app, RESTstop is the right solution for you!
The only additional suggestion I would have is to consider using OAuth authorization framework
in conjunction with your REST API. This will increase the robustness and security of your service.
References and Resources
REST
- REST Definition on Wikipedia
- REST resource
- Roy Fielding's dissertation on REST
- Ryan Tomayko's "How I explained REST to my wife"
- JSON
Camping
- Ruby Camping Framework
- Original Camping Blog example
- Camping controller routes
- JSON
- RESTstop library on GitHub
- Camping-OAuth repository on GitHub
Tools
Contributors/Ruby-ists
- Magnus Holm (@judofyr) - Camping, ...
- Matt Zukowski - RESTstop, RESTr, ...
Metaprogramming
- Include vs. Extend in Ruby (by John Nunemaker)
- [Module] Mixins (in Programming Ruby)
- Extending Objects (in Programming Ruby)
- Adding module behavior with module_eval (in Programming Ruby)
- Fun with Ruby’s instance_eval and class_eval (by Brian Morearty)
My Other Related Posts:
- Easily Transform Your Ruby Camping App Into An OAuth Provider
- Camping light (nosql) with MongoDB
- Visualize Application Metrics on NewRelic for your Ruby Camping Application
- Running the Camping Microframework on IronRuby
Full Source Of The RESTstop Camping REST Services App
Here is the full source (you can also download it from GitHub here)