The "Tech. Arch."

Camping light (nosql) with MongoDB

Intro

For the last two years or so I have been following the NoSQL “movement” from a far, i.e. the new alternatives to the traditional SQL relational model
such as SimpleDB, BigTable, Cassandra, CouchDB, and MongoDB.
I recommend a few excellent podcasts on Cassandra, CouchDB and MongoDB to get a sense of what this is all about.
Recently, MongoDB has received a lot of attention due to the following factors:

1. availability on many platforms
2. rich language support: C, C++, C#, Java, Javascript, Perl, PHP, Python, Ruby
3. binary json for efficient storage
4. equivalent of Javascript “stored procedures”
5. community development of higher-level framework like DataMapper

Although I could have started earlier, Heroku’s recent announcement of support for MongoDB accelerated my plans.
So it was time for me to start creating a small prototype with my favorite Ruby micro-framework: Camping.

If you want to skip straight to the full source, click here

Getting Started

To get started with the prototype I followed these steps:

1. I downloaded the latest version from http://www.mongodb.org/display/DOCS/Downloads.
   In my case, I chose the 32 bit Windows version for my laptop.
2. After installing it, I had to create the data folder under \data\db. Note that this is important since the folder does not get created automatically.
3. Then I navigated to the bin folder of my MongoDB installation, opened up a console and started mongodb:

   mongod
   

   

4. I installed the base ruby MongoDB driver:

   gem install mongodb
   

5. I installed MongoMapper

   gem install mongo_mapper
   

6. I installed the latest version of Camping:

   gem install camping
   

Et voilà! :-)

Spinning Up Mongo In IRB

I always like to prototype in a very incremental and dynamic manner so I first started with the Interactive Ruby console: IRB and entered the following:

Since we’re just “playing”, we don’t need to worry about compiling the bson extension.

Ok so now let’s open a connection on a new database we’ll call “test”

					m = Mongo::Connection.new.db('test')

At this point Mongo has not yet created the physical database yet though so if you go to the \data\db folder no content has been created yet.

Content is typically organized in “collections”, similarly to tables in the relational world,
but with the important difference that a collection does not have schema and can store arbitrary objects of any shapes.
So let’s create a collection of “widgets”

					w = m.collection('widgets') 
					
					# m.collection(...) can also be abbreviated as m[...]

Now let’s add some widgets. We’ll start with a simple Ruby “hash” object.

					w1 = { :code => 'A101', 
								:description => '1/2 in. brass nut', 
								:price => 0.15 }
					w << w1 

MongoDB returned the object id of the hash we just stored in the widgets collection.
Now we can see that data files were created in the \data\db folder:

Ok now let's add one more widget so we can later do a query over more than one object!

					w2 = { :code => 'B201', 
								:description => '1/2 in. brass bolt', 
								:price => 0.2 }
					w << w2

We can now find and iterate over all objects in the collection using find and each:

	 w.find.each { |x| puts x.inspect }

We can write more simple queries on a given attribute like:

	 w.find({ :code => 'A101' }).each { |x| puts x.inspect }

Or we can use a comparison operator like:

	 w.find({ :price => { '$gt' => 0.19 } })
		.each { |x| puts x.inspect }

Of course you can include multiple attributes in the criteria, limit the number of results, filter the list of attributes returned, etc.
Check out the full description of the find method here
or review the full Ruby Mongo API.

Going Camping With MongoDB

Now that we have experimented with the basic features of MongoDB in IRB,
we can step up to a prototype web app using the Ruby Camping framework. We will also incorporate the MongoMapper data access layer.

A. Creating A Skeletal Camping App

I personally like using Camping as it allows me to structure my app with the model-view-controller (MVC) pattern
all while keep all source in one file (I can always split it up later). The Camping server will also automatically reload the app
as I add features incrementally. Camping is also lightweight and much simpler than Rails. So let's get started:

1. Open up your favorite text editor (e.g. NotePad++ or TextMate)
2. Create a new Ruby file named mongo-camping-test.rb
3. Add the gem and require statements for camping and mongo

gem 'camping' , '>= 2.0'
gem 'mongo', '= 1.0' #Using 1.0 since MongoMapper explicitly needs 1.0
gem 'mongo_mapper'

%w(rack camping mongo mongo_mapper).each { | r | require r}

Our app will be contained in a module named MongoTest.
So first, let's add the statement to tell Camping to start our MongoTest module.
The main MongoTest module will have a create method used to initialize the application so this is where we will place
the logic to configure MongoMapper in terms of connection and database.

Camping.goes :MongoTest

module MongoTest
	def self.create
		MongoMapper.connection = Mongo::Connection.new
		MongoMapper.database = 'test'
	end
end	

MongoTest.create

Our main MongoTest module will contain a submodule for each facet of our MVC pattern: Models, Controllers, and Views.
So let's define these modules with the very basic minimum to allow a default route and view for the application:

module MongoTest
	# ...
	
	module Models
	end
	
	module Controllers
		class Index
			def get 
				render :index
			end
		end
	end
	
	module Views
		def index
			h1 'My MongoDB App'
			div 'To be continued ...'
		end
	end
end

Now we have a skeletal Camping app that we can run as follows:

camping mongo-camping-test.rb

Access the app from the browser at the following url:

http://localhost:3001/

B. Creating The Model

In the skeleton we had created a Models module. This is where we will add our Widget class.
Unlike a traditional Camping or Rails model, we will not have this class inherit from an ActiveRecord descendant class.
Instead we will just include the MongoMapper::Document module.
This module will turbo charge our model with all the needed Mongo features we need.

	module Models
		class Widget
			include MongoMapper::Document
		end
	end

Now we will define the attributes of our Model: code, description, and price.
Note that unlike in Camping and Rails we don't need a migration!
MongoMapper also provides validators like ActiveRecord, so we'll define a numeric validator and a custom validator
for the price attribute.
The only "gotcha" we need to work-around is that if we don't override the collection name auto-generated for Widget, the table name will include the fully qualified model class name, containing the module hierarchy names! So we'll explicitly force the collection name.

	module Models
		class Widget
			include MongoMapper::Document
			
			key :code, String
			key :description, String
			key :price, Float
			
			validates_numericality_of :price
			validates_true_for :price, 
				:logic=> lambda { | x | x.price >= 0.05 && x.price <= 9999.99 }, 
				:message=>"must be in the following range: 0.05 to 9999.99"
			
			set_collection_name 'widgets'
		end
	end

C. Creating The "Home/Index" Controller

Now that we have a model, we can flesh-out our first controller.
Our initial skeleton has an Index (i.e. home) controller and
a matching index view rendered when an HTTP GET is
received by Camping on the default "index" route i.e. for the / path.
Let's modify the get method of our Index controller so we can retrieve the list of widgets using all method on our Widget model:

		class Index
			def get
				@widgets = Widget.all

				render :index
			end
		end

D. Creating The "Home/Index" View

By default, Camping uses a Ruby-syntax based templating engine called Markaby.
It allows declaration of HTML elements using Ruby statements:

	h1 "Welcome #{@your_name}"

Nested HTML elements are represented by Ruby blocks:

	table do
		tr { th 'First'; th 'Last'}
		tr { td 'John'; th 'Doe'}
	end

For more details on Markaby, see here.
Note: the upcoming 2.1 version of Camping uses Tilt to let you choose your favorite templating engine (e.g. ERB, Haml, ...).

So, let's flesh-out our index view method to render a table of widgets.

	module Views
		def index
			h1 "Widgets"

			if @widgets 
				table do
						tr { th 'id'; th 'Code'; th 'Description'; th 'Price'; }
					@widgets.each do | w |
						tr {  	td w._id; 
								td w.code; 
								td w.description; 
								td w.price;  }
					end
				end
			else
				p "No widgets yets"
			end
		end
	end

Let's refresh our page (since Camping will auto-reload our code on save in the editor):

E. Add Widget Feature

Now let's add the ability to create and save new widgets.
We'll start by creating a new AddWidget controller in the Controllers module
and register a new route for /widget/new.
We'll instance (not create) a new Widget model and render it in a widget_form (to be created soon):

		class AddWidget < R '/widget/new'
			def get
				@widget = Widget.new(:code=>'', :description=>'', :price=>0)
				render :widget_form
			end
		end

Ok now we can create our widget_form and allow a POST submit
to the route - using the R() Camping function - for our new AddWidget controller.
Note: we will add an optional section at the top of the form to report any model validation errors:

		def widget_form
			h2 "New Widget"

			if @errors
				h3 "Errors:"
				ol do
					@errors.each do | e |
						li { div "#{e[0]}: #{e[1].join(' ')}" }
					end
				end
			end
			
			form :action => R(AddWidget), :method => 'post' do
			  label 'Code:', :for => 'widget_code'; br
			  input :name => 'widget_code', :type => 'text', :value => @widget.code; br

			  label 'Description:', :for => 'widget_description'; br
			  input :name => 'widget_description', :type => 'text', :value => @widget.description; br

			  label 'Price:', :for => 'widget_price'; br
			  input :name => 'widget_price', :type => 'text', :value => @widget.price.to_s; br
			  
			  input :type => 'submit', :value => 'Save'
			end
		end

Access the new widget form from the browser at the following url:

http://localhost:3001/widget/new

Now we can add the handler for the HTTP POST to our AddWidget controller.
After validating the user entered some actual data, we'll create (not instantiate)
a new Widget model instance with the user's data and call the save method
to add it to the widgets collection.
We'll redirect back to the form if the model did not pass validation rules or if the save failed.
Then we'll re-query the list of widgets and render the index view to show the new data.

		class AddWidget < R '/widget/new'
			# ...
			
			def post
				return(redirect('/index')) unless input.widget_code && input.widget_description && input.widget_price
				
				w = Widget.create(:code => input.widget_code,
												:description => input.widget_description,
												:price => input.widget_price.to_f)
				
				begin
					w.save!
					@widgets = Widget.all
					return(render(:index))
				rescue Exception => ex
					@errors = w.errors
					return(render(:widget_form))
				end
			end
		end

F. Search Widget Feature

Finally let's add the ability to search for a specific widget based on a widget code.
We'll start by creating a new SearchWidgetByCode controller in the Controllers module
and register a new route for /widget/search.

		class SearchWidgetByCode < R '/widget/search'
			def get
				render :search_form
			end
		end

Ok now we can create our search_form and allow a POST submit
to the route - using the R() Camping function - for our new SearchWidgetByCode controller:

		def search_form
			h2 "Search"
			p @info if @info
			
			form :action => R(SearchWidgetByCode), :method => 'post' do
			  label 'Code', :for => 'widget_code'; br
			  input :name => 'widget_code', :type => 'text'; br
			  input :type => 'submit', :value => 'Search'
			end
		end

Access the widget search form from the browser at the following url:

http://localhost:3001/widget/search

Now we can add the handler for the HTTP POST to our SearchWidgetByCode.
We'll use the traditional dynamic finder method like in ActiveRecord: find_by_code method and pass a criteria based on the code attribute
The results will be rendered using the index view.

		class SearchWidgetByCode < R '/widget/search'
			# ...
			
			def post
				return(redirect('/index')) unless input.widget_code
				
				w = Widget.find_by_code input.widget_code
				@widgets = [ ] 
				@widgets << w if w
				
				render :index
			end
		end

G. Exercises Left For The Reader

Now that we have scratched the surface of the possibilities offered by MongoDB and MongoMapper,
we could extend our prototype to include the following features:

1. Add indexes to the database
2. Limit the main view to include only the first 12 items, and add paging
3. Add model validation rules to Widget model
4. Add more attributes to the Search criteria
5. Add an Edit page to leverage the save(object) feature of a collection
6. Add embedded models/document to Widget (e.g. TechnicalSpecification, etc.)

You can now leverage the power of Camping, Markaby and REPL to add these features on your own. :-).
Note: you can mix and match templating engine in Camping 2.1 using Tilt - which allows you to use ERB, Haml, etc.

So What?

1. The combination of MongoDB and MongoMapper makes it very easy to implement SQL-less persistence
2. We could work in a schema-less mode while using using hashes and the base Mongo Ruby Driver
3. But we can adopt migration-less models and have more structure
4. MongoMapper draws a lot of features from ActiveRecord making the transition easy
5. Since Camping is lightweight yet powerful, writing solid MongoDB apps can be done easily

So hopefully this post will have given you a feel for what development with MongoDB/Ruby and Camping feels like.
Happy experimentations!!!

References and Resources

• Cloud storage in a post-SQL world
• Cassandra - "Dude where is my database?" Techzing Live podcast
• CouchDB - Stack Overflow podcast
• MongoDB - Mike Dirolf's Interview on Techzing Live podcast
• MongoDB - official site
• MongoDB - Mike Dirolf's Intro Slides
• MongoHQ
• MongoDB Ruby API
• MongoMapper Overview Presentation by John Nunemaker
• Mongo Mapper - Data Access Layer
• Ruby Camping Framework
• Markaby - Ruby Templating Engine
• Markaby Cheat Sheet
• Tilt - Templating Engine Adapter
• Haml - Ruby Templating Engine
• Heroku Cloud Hosting for Ruby
• NoSQL catalog of products

My Other Related Posts:

1. Visualize Application Metrics on NewRelic for your Ruby Camping Application
2. Running the Camping Microframework on IronRuby

Full Source Of Our MongoDB Camping Web App

Here is the full source (you can also download it from Gist here)

gem 'camping' , '>= 2.0'
gem 'mongo', '= 1.0' #Using 1.0 since MongoMapper explicitly

%w(rack camping mongo mongo_mapper).each { | r | require r}

Camping.goes :MongoTest

module MongoTest
	def self.create
		MongoMapper.connection = Mongo::Connection.new
		MongoMapper.database = 'test'
	end

	module Models
		class Widget
			include MongoMapper::Document
			
			key :code, String
			key :description, String
			key :price, Float
			
			validates_numericality_of :price
			validates_true_for :price, 
				:logic=> lambda { | x | (0.05..9999.99) === x.price }, 
				:message=>"must be in the following range: 0.05 to 9999.99"

			set_collection_name 'widgets'
		end
	end

	module Controllers
		class Index
			def get
				@widgets = Widget.all

				render :index
			end
		end
		
		class AddWidget < R "/widget/new"
			def get
				@widget = Widget.new(:code=>'', :description=>'', :price=>0)
				render :widget_form
			end
			
			def post
				return(redirect('/index')) unless input.widget_code && input.widget_description && input.widget_price
				
				@widget = Widget.create(:code => input.widget_code,
												:description => input.widget_description,
												:price => input.widget_price)
				
				begin
					@widget.save!
					@widgets = Widget.all
					return(render(:index))
				rescue Exception => ex
					@errors = @widget.errors || [ ex.to_s ]
					return(render(:widget_form))
				end

			end
		end
		
		class SearchWidgetByCode < R '/widget/search'
			def get
				render :search_form
			end
			
			def post
				return(redirect('/index')) unless input.widget_code
				
				w = Widget.find_by_code input.widget_code
				@widgets = [ ] 
				@widgets << w if w
				
				render :index
			end
		end
	end
	
	module Views
		def layout
			html do
				head do					   
					style  :type => 'text/css' do
	<<-STYLE				
	body {
		padding:0 0 0 0;
		margin:5px;
		font-family:'Lucida Grande','Lucida Sans Unicode',sans-serif;
		font-size: 0.8em;
		color:#303030;
		background-color: #fbf9b5;
	}

	a {
		color:#303030;
		text-decoration:none;
		border-bottom:1px dotted #505050;
	}

	a:hover {
		color:#303030;
		background: yellow;
		text-decoration:none;
		border-bottom:4px solid orange;
	}

	h1 {
		font-size: 14px;
		color: #cc3300;
	}

	table {
		font-size:0.9em;
		width: 400px;
		}

	tr 
	{
		background:lightgoldenRodYellow;
		vertical-align:top;
	}

	th
	{
		font-size: 0.9em;
		font-weight:bold;
		background:lightBlue none repeat scroll 0 0;
		
		text-align:left;	
	}

	.url
	{
		width: 300px;
	}
		
	STYLE
					end
				end
				
				body do
					self << yield

					a "Home", :href=>"/"
					div.footer! do
						hr
						span.copyright_notice! { "Copyright © 2010   -  #{ a('Philippe Monnet (@techarch)', :href => 'https://blog.monnet-usa.com/') }  " }
					end
				end
			end
		end
	
		def index
			h1 "Widgets"

			if @widgets 
				table do
						tr { th 'id'; th 'Code'; th 'Description'; th 'Price'; }
					@widgets.each do | w |
						tr {  	td w._id; 
								td w.code; 
								td w.description; 
								td w.price;  }
					end
				end
			else
				p "No widgets yets"
			end

			ul do
				li { a "Search", :href => "/widget/search" }
				li { a "Create", :href => "/widget/new" }
			end
		end
		
		def widget_form
			h2 "New Widget"

			if @errors
				h3 "Errors:"
				ol do
					@errors.each do | e |
						li { div "#{e[0]}: #{e[1].join(' ')}" }
					end
				end
			end
			
			form :action => R(AddWidget), :method => 'post' do
			  label 'Code:', :for => 'widget_code'; br
			  input :name => 'widget_code', :type => 'text', :value => @widget.code; br

			  label 'Description:', :for => 'widget_description'; br
			  input :name => 'widget_description', :type => 'text', :value => @widget.description; br

			  label 'Price:', :for => 'widget_price'; br
			  input :name => 'widget_price', :type => 'text', :value => @widget.price.to_s; br
			  
			  input :type => 'submit', :value => 'Save'
			end
		end
		
		def search_form
			h2 "Search"
			p @info if @info
			
			form :action => R(SearchWidgetByCode), :method => 'post' do
			  label 'Code', :for => 'widget_code'; br
			  input :name => 'widget_code', :type => 'text'; br
			  input :type => 'submit', :value => 'Search'
			end
		end
		
	end
end

MongoTest.create

May 13th, 2010 Posted by techarch | mongodb, Ruby Camping | 2 comments