Ruby: Caching Technique

In this article, we’re going to explore some of the issues involved in caching as they apply generically to programming. Then we’re going to suggest a novel solution, build some proof of concept code, and then finally implement the caching code. We’ll implement the caching in stages so that you can clearly understand the ruby constructs involved. Actually, you could think of this whole article as a vehicle by which to demonstrate the following concepts in ruby:

• code blocks
• thread local variables
• nested hashes
• accessing the stack trace
• using bindings

The best approach to understanding these concepts would probably be to create a single ruby file and add the code snippets to it as we go. This way, you can run the tests and follow along with the development process. You can also twiddle with the code during the various stages to see how things work.

So, let’s dig in…

Why cache?

Often when accessing network services from a webapp, we find that our code is making more calls against the slow service than is necessary. The rendering of a single web page may even be making a surprising number of duplicate requests. For example, a detail screen may contain multiple instances of a drop-down which is generated from the results of a service request.

Our first instinct would be to simply create the drop-down on startup and simply re-use it going forward. This would be great. But, sometimes the drop-down may be made up of data that changes dynamically. In this situation, we are pretty much forced to lookup the data each time.

We realize that we should implement some sort of caching mechanism or maybe create the drop-downs once at the beginning of the page request and then re-use them. However, a caching mechanism seems like too much trouble and pre-generating re-used collections and components on each request leads to ugly code.

Wouldn’t it be cool if there was a way to simply tell ruby that you want to cache the results of a block of code like this?

This is a simple finder method of my model which has been over-ridden to retrieve data from a webservice. The really interesting part is the cache { ... }. It would be really cool if we could simply enclose a block of code whose results we want to cache.

Let’s see if this is possible. Upon further reflection, this can’t be as easy as it looks. What happens if the ‘offset’ or ‘row_count’ args are different on the next call? What about keeping the caches separate for the find_all_paged() methods of the Address model and the Person model? What about caching the results of the find_all_paged() separately from the find_all() method?

Alright. So, now we know that this caching mechanism has to somehow keep track of the caches based on the class ‘Address’, method ‘find_all_paged’, and the args ‘offset’ and ‘row_count’. This is all starting to sound pretty complicated. Also, we’ll probably have to pass args to our cache to use as keys to keep our cached results separate. It seems as if we aren’t going to be able to do something as clean as a simple cache { ... }. Should we just give up?

Create proof of concept code

Luckily, we are using ruby. And we have confidence that it will let us implement almost anything in any way we want. Therefore, we are going to forge ahead and setup a small proof of concept.

We’ll start by creating a new empty file and adding a mock Soap class to it that simply pretends to return values from some soap service as follows:

I included ‘puts’ statements within each method so that later we can tell if the Soap method was actually called or if the results were pulled from cache.

Next, we’ll add a model class to our file immediately following the Soap class. The model’s finder methods will be calling the Soap class for data. Therefore, we have to be sure that the Soap class is defined before the Address class.

Next we’ll add some test code at the bottom of the file that makes calls against the finders of the Address class and prints the results.

This test code makes multiple calls against the finders with identical args. Let’s run it and see what the output looks like without any caching:

Address.find_all
*  pretending to retrieve records from a webservice
=> ["result1", "result2", "result3", "result4"]

Address.find(1,3,5)
*  pretending to retrieve records from a webservice
=> ["value for 1", "value for 3", "value for 5"]

Address.find(1,3,5)
*  pretending to retrieve records from a webservice
=> ["value for 1", "value for 3", "value for 5"]

Address.find(0,1,2,3)
*  pretending to retrieve records from a webservice
=> ["value for 0", "value for 1", "value for 2", "value for 3"]

Address.find(1,3,5)
*  pretending to retrieve records from a webservice
=> ["value for 1", "value for 3", "value for 5"]

Address.find_all
*  pretending to retrieve records from a webservice
=> ["result1", "result2", "result3", "result4"]

Address.find_all
*  pretending to retrieve records from a webservice
=> ["result1", "result2", "result3", "result4"]

Address.find_all_paged(20, 10)
*  pretending to retrieve records from a webservice
=> ["page1", "page2", "page3", "page4"]

Address.find_all_paged(20, 10)
*  pretending to retrieve records from a webservice
=> ["page1", "page2", "page3", "page4"]

Address.find(20, 10)
*  pretending to retrieve records from a webservice
=> ["value for 20", "value for 10"] 

As you can see, the “pretending to retrieve…” message appears on every single finder call. If caching were implemented, we would have had only 5 hits against the webservice instead of 9 since there are only 5 unique calls being made.

Building the cache solution

We can do this. We’re going to build a caching solution that works exactly as depicted at the top of this post. Not only that, but along the way, we are going to learn about code blocks, thread local variables, nested hashes, accessing the stack trace, and using bindings! So, sit back, take a deep breath and plunge in!

We’re going to implement our cache as a module which can be mixed-into our model classes. We’ll start with the assumption that we want its usage to look like the cache { ... } syntax in the example at the top of this article. This implies that we are going to be passing a block to a method named ‘cache’. Let’s start with that. Add the following code to our ruby file somewhere above the Address class. This module must be defined above the Address class since we are going to be referencing the ModuleCache module from the Address class.

First of all, we’ve declared a module named ModelCache. This module will need to be mixed-in to any models that want to use the cache feature. Right now our cache method is setup to call the code block with the ‘yield’ statement and return the results of that block. Since ‘yield’ is the last expression of our cache method, it’s value is returned as if you typed “return yield”. And, by the way, you are correct that there is no caching taking place yet. We are simply proving in our mechanism for wrapping a code block.

Now, let’s modify our Address model to make use of our new caching mechanism. We’ll have to mix-in our module and then wrap the guts of each of our finder methods in our new ‘cache’ method:

Notice that instead of using the ‘include’ statement to mix-in our ModelCache methods, we used ‘extend’. The difference between ‘include’ and ‘extend’ is that ‘include’ mixes-in the module methods as instance methods where ‘extend’ mixes them in as class methods. Since all our finders are class methods, I used ‘extend’.

When we run it, we basically get identical output as before except that we’ve added a “cache called” before each “pretending to retrieve…” line.

Address.find_all
cache called
*  pretending to retrieve records from a webservice
=> ["result1", "result2", "result3", "result4"]

Address.find(1,3,5)
cache called
*  pretending to retrieve records from a webservice
=> ["value for 1", "value for 3", "value for 5"]

Address.find(1,3,5)
cache called
*  pretending to retrieve records from a webservice
=> ["value for 1", "value for 3", "value for 5"]

Address.find(0,1,2,3)
cache called
*  pretending to retrieve records from a webservice
=> ["value for 0", "value for 1", "value for 2", "value for 3"]

Address.find(1,3,5)
cache called
*  pretending to retrieve records from a webservice
=> ["value for 1", "value for 3", "value for 5"]

Address.find_all
cache called
*  pretending to retrieve records from a webservice
=> ["result1", "result2", "result3", "result4"]

Address.find_all
cache called
*  pretending to retrieve records from a webservice
=> ["result1", "result2", "result3", "result4"]

Address.find_all_paged(20, 10)
cache called
*  pretending to retrieve records from a webservice
=> ["page1", "page2", "page3", "page4"]

Address.find_all_paged(20, 10)
cache called
*  pretending to retrieve records from a webservice
=> ["page1", "page2", "page3", "page4"]

Address.find(20, 10)
cache called
*  pretending to retrieve records from a webservice
=> ["value for 20", "value for 10"] 

So, our wrapping mechanism works. Let’s add caching to it. In our particular implementation, we have a very dynamic set of data that is used to populate all our drop-downs and such. Therefore, we want to repopulate everything upon each request to render the page while not wanting to repeatedly make the same service requests while rendering that single page. Thus, we will be implementing our cache by storing service request results in a thread local variable as follows:

Ruby makes thread local variables simple. The ruby Thread class implements the [] method (square brackets operator) and []= method such that it works like the corresponding operators of the Hash class. This essentially lets you read and assign thread local variables using the super simple syntax of “a = thread[:person]” or “thread[:count] = 12”. In our case we want to associate our variables with the current thread that our web page is being rendered upon. Therefore, we call the Thread.current method to get that thread and then tack on the [:cache] to access our cached value.

We’re also making use of ruby’s lazy instantiation operator ||= (‘or’ ‘equals’). This operator essentially checks to see if the left-side operand is nil or undefined then assigns the right-side operand to it. If the left-side operant is non-nil then the assignment is skipped and the value of the left-side operand is returned as the expression value. Thus the following line from the code above:

is equivalent to:

At any rate, let's run it and see what we get:

Address.find_all
cache called
*  pretending to retrieve records from a webservice
=> ["result1", "result2", "result3", "result4"]

Address.find(1,3,5)
cache called
=> ["result1", "result2", "result3", "result4"]

Address.find(1,3,5)
cache called
=> ["result1", "result2", "result3", "result4"]

Address.find(0,1,2,3)
cache called
=> ["result1", "result2", "result3", "result4"]

Address.find(1,3,5)
cache called
=> ["result1", "result2", "result3", "result4"]

Address.find_all
cache called
=> ["result1", "result2", "result3", "result4"]

Address.find_all
cache called
=> ["result1", "result2", "result3", "result4"]

Address.find_all_paged(20, 10)
cache called
=> ["result1", "result2", "result3", "result4"]

Address.find_all_paged(20, 10)
cache called
=> ["result1", "result2", "result3", "result4"]

Address.find(20, 10)
cache called
=> ["result1", "result2", "result3", "result4"] 

Whoa! Every single test case returns the same exact result! Notice that the Soap class’s “pretending to retrieve…’ message is only printed on our very first call. If you look at our cache() method, this sortof makes sense. We are storing the results of all cached blocks into the same thread local variable (‘cache’).

In order to keep our cache results separate, we will have to take into consideration the uniquely identifying factors that we worked-out at the top of this post: class, method, and args. Let’s change our ‘cache’ method so that it stores the results into a Hash instead. We can use the uniquely identifying factors as keys.

This presents us with a bit of a dilemma, however. How do we use 3 factors as a key? The ruby Hash works just like the HashMap in Java – it requires the keys to implement both the ‘hash’ and ’==’ methods. We could create a new class whose instances could be used for our hash keys – where the ‘hash’ method performs some calculation based on the ‘hash’ of each of the 3 factors and the ’==’ method of our key class similarly checks that all 3 factors are equivalent. But, this is a very Java-like approach. So, let’s slap ourselves and take a moment and reflect on why we decided to learn ruby.

Instead, why don’t we just implement a nested hash. Huh? How is this easier? Well, let’s look at what the usage of a nested hash looks like:

Hey, that’s pretty clean. We have our hash variable followed by each of our 3 factors. But, won’t we have to create some mutant hash class that defaults all the values of any newly accessed keys to a new hash? Well, yeah. But this is actually already supported by the Hash class. Ruby’s Hash class lets you define the default value for any arbitrary key. For example, examine the following:

Note how the second form lets us specify a default value to be assigned when the new key :bird is accessed that doesn’t already exist. What would happen if we had it return a new empty hash by default?

Hey! It returned a hash. Ok, what happens if we access elements of that second hash?

It worked! We have a 2 dimensional nested Hash. The first dimension automatically defaults its elements to new empty hashes. The second dimension is just a regular hash which defaults its values to nil. We can assign values to that second dimension and then retrieve them in the normal fashion. Since the second dimension defaults to nil, we can take advantage of that and use the lazy instantiation operator to assign values to the cache by yielding to (executing) the bock of code whose value is to be cached – but only if that hash element has not already been assigned (cached).

Alright, let’s apply what we’ve learned here and add the first 2 factors (class, method) to our ‘cache’ method:

Ok. caller[0] =~ what?!!! This is just a little regular expression magic that I use to get the name of the method which called our ‘cache’ method. The ‘caller’ method returns the stack trace in an array where element 0 is the method which called the current method. The contents of each element of the trace looks something like this:

/Users/danlynn/Desktop/caching_block.rb:45:in `find’

The first two parts are the source file and line number. The last part is the name of the method. I used the regular expression ‘match’ operator ”=~” to apply that crazy regular expression to the first element in the stack trace. This expression allows me to pull out the filename, line, and method into the variables $1, $2, and $3. These actually correspond to the parts of the expression that are in parenthesis. Even tho these 3 variables start with a dollar sign, they are not global – they are actually thread local variables (just like our cache!) The dollar sign notation is just a carry-over from perl.

The model class is easy to determine. I can simply use the ‘self’ reference. This is very similar to Java’s ‘this’. However, in ruby, it refers to the context in which it is used. For example, in our case, the ‘cache’ method is being mixed into a class as a class method. Therefore, ‘self’ refers to that class (Address) – not the ModelCache module – not an instance of Address – but, the actual class Address. Once the Address class has been defined, it is associated with the global constant ‘Address’. I know that sounds strange – but, really, this is what happens in Java and just about any other language. Well, by using ‘self’ as our first hash dimension, we are essentially using the unique name of the model class as the first dimension.

Other than adding the classname and method to the “cache called from…” message, the only other big change here is that we are lazy instantiating our thread local variable named “cache” with a 2 dimensional hash and storing a reference to that thread local variable in the local variable named ‘cache’.

Let’s run it and see what we get:

Address.find_all
cache called from: Address.find_all()
*  pretending to retrieve records from a webservice
=> ["result1", "result2", "result3", "result4"]

Address.find(1,3,5)
cache called from: Address.find()
*  pretending to retrieve records from a webservice
=> ["value for 1", "value for 3", "value for 5"]

Address.find(1,3,5)
cache called from: Address.find()
=> ["value for 1", "value for 3", "value for 5"]

Address.find(0,1,2,3)
cache called from: Address.find()
=> ["value for 1", "value for 3", "value for 5"]

Address.find(1,3,5)
cache called from: Address.find()
=> ["value for 1", "value for 3", "value for 5"]

Address.find_all
cache called from: Address.find_all()
=> ["result1", "result2", "result3", "result4"]

Address.find_all
cache called from: Address.find_all()
=> ["result1", "result2", "result3", "result4"]

Address.find_all_paged(20, 10)
cache called from: Address.find_all_paged()
*  pretending to retrieve records from a webservice
=> ["page1", "page2", "page3", "page4"]

Address.find_all_paged(20, 10)
cache called from: Address.find_all_paged()
=> ["page1", "page2", "page3", "page4"]

Address.find(20, 10)
cache called from: Address.find()
=> ["value for 20", "value for 10"] 

Well, this is sortof better. It is only calling the actual Soap finder methods 3 times (count the “pretending…” messages). This is because we are only making calls to ‘cache’ from 3 different methods and all from the same 1 class. It is nice to see the class and method names in our output from the ‘cache’ method tho.

Alright. Let’s figure out how we’re going to get our third factor, the method arguments, to play a part in the cache hash key. We could simply modify our ‘cache’ method to accept multiple arguments and then pass the args in parens to each cache usage. Let’s see what that would look like:

Yuck! I demand perfection! It MUST look like the following – or no dice:

How can we do this? Can we in some magical ruby fashion reach back up a level in our stack trace and pluck the args from our method call? To tell the truth, I don’t know. I didn’t even look into this because I realized that we not only need the args of the method which called our ‘cache’ method – but also any local variables. This is because there could conceivably be some code above our ‘cache’ call which is setting up local variables which are being used within the code block that is passed to our ‘cache’ method.

I’ve played with a lot of ruby’s meta programming tricks before. Therefore, I already knew that I can get the list of local variables like this:

Adding this code into a new ruby file and running it outputs:

[“a”, “b”, “c”, “x”, “y”]

Ok. great! But, we’re also going to need the values of all these variables in order to generate that third factor of our 3 dimensional hash cache. Since these are strings, let’s try a little ruby introspection to get their values:

Which outputs the following:

["a", "b", "c", "x", "y"] [10, 20, 30, 1, 2]

The ‘eval’ statement takes a string and executes it within the current context (being the foo method in this case). The ‘collect’ method iterates over the elements of the array returned by the ‘local_variables’ method. For each element, in that array, the ‘collect’ starts building a new array whose values are the result of executing the code in the closure “eval(var)”. The ”|var|” is simply how we associate a name with the arg passed into the code block by the ‘collect’ method representing the current element.

Well, this proves that we can get the list of all the variables in a method and then build an array out of their values. But… how do we reach into the calling method’s context from our ‘cache’ method to do this? Well, this is where bindings come into play. Look closely at the following code:

You’ll notice that the code block following ‘cache’ has access to the context of the outer self.find_all_paged() method – including its arguments which are local variables. This may seem natural – until you note that this block is not executing until it is being called by our ‘cache’ method via the ‘yield’ statement. Therefore, somehow, this context is being passed around and made available to our ‘yield’. This, my friends, is referred to as a binding.

According to the pick-axe book, an instance of a binding will “encapsulate the execution context at some particular place in the code and retain this context for future use”.

I’ll take a moment here and wait while your mind reels at the possibilities. Imagine, execution state can be easily stored and then picked back up at a later time. This could be used for such things as automagically applying state to stateless webapps to make their code into a top-down or event-based program like a regular client application. Wow. And in fact, this is exactly what is at the core of a new breed of application servers called “continuation servers”. Look into them sometime.

Anyways, back to the tutorial. It turns out that code blocks which follow method calls are passed as an additional un-named argument to that method. This invisible argument is what is used by the ‘yield’ statement. In fact, you can test to see if this invisible arg was passed by calling ‘block_given?’ in your method. How to we get hold of this block? Well, ruby lets you specify a special last argument in your method definition’s argument list which is designated to have this block assigned to it by preceding its name with an “&”. This argument must be the very last arg.

When ruby see’s the ‘proc’ arg defined as follows, it converts the code block into a Proc instance and assigns it to that argument.

The Proc class has some useful elements. The one we want is its ‘binding’ method. This method returns the binding (execution context) from which ‘cache’ was called and in which ‘proc’ is to be executed. Well, now that we have access to this binding from within our ‘cache’ method, we can make use of it to execute some code within its context to get the list of local variables as follows:

We’ve now added the third factor (local variables) into the mix. We get the list of local variables by calling ‘eval’ and passing the binding to it as an optional arg. The ‘eval’ will execute the code passed in as a string within the context of that binding. Then, to make our output pretty, I make a superfluous second evaluation against that binding to build a list of name/value pairs to appear in our output as something like: “offset = 20, row_count = 10”. This lets us produce the fancy cache messages like this:

cache called from: Address.find_all_paged(offset = 20, row_count = 10)

Next, we added an additional dimension to our nested hash by changing this:

to this:

Note that we could implement a new NestedHash class that automagically allowed infinite depth - but that would distract from the gist of this article. Besides the NestedHash deserves an article of its own.

Then, to finish things out, we add the third factor to our cache hash usage:

We could have also replaced ‘yield’ with “proc.call” – but ‘yield’ looks nicer.

Now, let’s give it a run:

Address.find_all
cache called from: Address.find_all()
*  pretending to retrieve records from a webservice
=> ["result1", "result2", "result3", "result4"]

Address.find(1,3,5)
cache called from: Address.find(ids = [1, 3, 5])
*  pretending to retrieve records from a webservice
=> ["value for 1", "value for 3", "value for 5"]

Address.find(1,3,5)
cache called from: Address.find(ids = [1, 3, 5])
=> ["value for 1", "value for 3", "value for 5"]

Address.find(0,1,2,3)
cache called from: Address.find(ids = [0, 1, 2, 3])
*  pretending to retrieve records from a webservice
=> ["value for 0", "value for 1", "value for 2", "value for 3"]

Address.find(1,3,5)
cache called from: Address.find(ids = [1, 3, 5])
=> ["value for 1", "value for 3", "value for 5"]

Address.find_all
cache called from: Address.find_all()
=> ["result1", "result2", "result3", "result4"]

Address.find_all
cache called from: Address.find_all()
=> ["result1", "result2", "result3", "result4"]

Address.find_all_paged(20, 10)
cache called from: Address.find_all_paged(offset = 20, row_count = 10)
*  pretending to retrieve records from a webservice
=> ["page1", "page2", "page3", "page4"]

Address.find_all_paged(20, 10)
cache called from: Address.find_all_paged(offset = 20, row_count = 10)
=> ["page1", "page2", "page3", "page4"]

Address.find(20, 10)
cache called from: Address.find(ids = [20, 10])
*  pretending to retrieve records from a webservice
=> ["value for 20", "value for 10"] 

Whoop! We got it!! We have our caching working correctly (5 hits against webservice) and we were able to keep our beautiful cache { ... } notation. Not only that, but if you ignore our superfluous ‘puts’ code, we did it with only a 5 line cache method!!! ...and about 500 lines of discussion ;-)

You just gotta love ruby.

As always, I thrive on the comments I get on these articles. So, please indulge me by asking all the questions you want or by simply commenting below.

Full code example

Oh, and one more thing. Since you got this far in a rather lengthy ruby posting, I’ll go ahead and give you a link to the full ruby file that you can simply download and run. Feel free to experiment and let me know if you find better ways to write those 5 lines. :-)

caching_technique.rb