CacheGen Proof Of Concept

So, I finally have CacheGen to where I can probably integrate it into this website. I did some rough concurrency testing (spawning 60 threads accessing the cache with random clearing). It's a rough test, but it does show that there isn't anything obviously wrong with it at least.

So, the code it generates is brilliantly simple as well. Some good use cases for this:

  • Keep all your cache settings in one place
  • Statically typed and named! No more remembering manual casts or magic strings
  • Make your caching logic testable! It generates code against an easily mockable interface
  • Switch out your caching layer with ease.

Now, I'm only going to elaborate on the last point. "Why would I ever want to change out my caching layer!?"

Here's why. You built Bookface 1.0 and a few dozen users are on it. People start talking though and suddenly you have a few thousand(or more). You page response times have crept up into the seconds range. Something must be done. After upgrading servers, and expanding some of the hardware, you find the bottleneck. Your web server's caches are being cleared too often. There isn't anything you can do though, the memory is maxed out as it is. So, obvious choice: Use something like memcached for distributed caching on a dedicated server or two.

What's makes using memcached or something so hard? It requires code changes! Luckily for you, you used CacheGen though. Why? All of your caching is in one place, and your interface to the caching method(CacheMechanism) is in one single simple class. It's trivial to implement a two-level cache between ASP.Net and memcached at this point and all of your code relying on your cache will just magically work without being changed.

This is what I think makes CacheGen especially awesome. It manages your caching settings, makes everything statically typed, AND lets you have an almost unreal amount of flexibility.

It's not quite ready for primetime yet. I've proved that it should work, the thing now to do is clean up the API some and add some more unit testing to see if I can catch more bugs.

Anyway, I don't expect this process to take too long. I plan to tag an alpha release for this relatively soon (within the month)

Posted: 12/8/2012 7:21:17 AM

Making Caching Awesome Again

So, I've recently began a new project. It's no where near stable, or even usable. But I think it's a good enough time to introduce it.

It's called CacheGen and it's hosted at bitbucket and BSD licensed.

Basically, it's a T4 template that generates really awesome helpers for caching. It's geared at ASP.Net, but my goal is to make it so it can be used in other ways as well. So, how does it work? You give it a list of specifications for something you want to cache. Let's say you have a string that's really expensive to generate, but doesn't belong in a database. To go on my theme of markdown being slow, let's call it MarkdownTranslation

You tell the T4 template to create a cache item for it like this:

var tmp=new CacheObject
{
  Name="MarkdownTransform",
  ValueType="string";
}
var cache=new CacheGen("Earlz.Example.MyCache");
cache.Items.Add(tmp);
Write(cache.ToString();

And then, you can use the MyCache class like this:

var text=MyCache.MarkdownTransform ?? (MyCache.MarkdownTransform=Markdown.Translate(foobar));

Much much easier. But, there is a flaw in this. It's possible that we could consume double the server resources required for this. This wouldn't be a problem in this case, but imagine a very heavy SQL query or something. This isn't quite what we want when things are really expensive. So, let's use some other fancy syntax

var text=MyCache.MarkdownTransformCache.LockGetOrLoad(()=>Markdown.Translate(foobar));

What this will do instead is raise a flag so that requests for MarkdownTransform will block and during this time, it will execute the expensive code and other threads will sleep. This way, it only gets executed once. And, when it finally gets the results back, the cache will be loaded with it and the other threads will be able to access it. So, instead of computing the expensive thing multiple times, instead we just hold out other requests for a little while so the expensive thing is done only once.

Sure, you can do this same thing with ASP.Net's cache, but how much code would it require for each time you did this? Hence why it's T4. Also, it's completely statically typed. No casting!

Now, what if you wanted to cache the markdown for multiple posts? Or just want to cache entire post objects? Well, I thought of this too:

You'd make the CacheObject like this in T4:

Name="Transforms";
KeyType="ObjectId";
ValueType="BlogPostData";

Then, you could apply the same lock-first type behavior like so:

var post=MyCache.TransformsCache.LockGetOrLoad(objectid, ()=>LoadPost());

Or, if you prefer the (probably safer) possibly execute twice behavior:

var post=MyCache.Transforms[objectid] ?? (MyCache.Transforms[objectid] = LoadPost());

These are the ideas anyway. None of it is actually implemented yet, and a lot more research must be done to ensure that this is a sane way to go about it. But, you can expect to get something similar to this.

Posted: 12/6/2012 5:17:43 AM