Catching Data for Performance


I got a question the other day from a gentleman who is writing an application for YouTube.  His problem is that he’s gathering data from the YouTube web site and redisplaying it in his web application.  Once he has it, he then allows the user to page it and sort it.  His question was, “Is there any way of caching the data so that I don’t have to go back to the YouTube web site each time?”

There are actually several ways of dealing with this issue.


If you are looking for an automatic way of handling this problem.  Look no further than the ObjectDataSource.  There are several properties hanging off of this that can help you:

EnableCaching Set this to true to enable caching at the ObjectDataSource level.
CacheDuration Length of time, in seconds, to cache the data.  You can also specify “Infinite” if you want to hold the data forever.
CacheExpirationPolicy Absolute or Sliding.  Controls if duration is from first call to the server (absolute) or last call to the server (sliding)
CacheKeyDependancy Allows you to programmatically expire they cache based on some key.


This method stores the data on the server and associates it with the page.  Somewhat memory intensive as is the next method.


Session Variables

You could also do everything programmatically and push the DataSet into a session variable.  To do this, you’ll want to create a key for specific search conditions, if any and store the DataSet as the value.

You may be thinking at this point, “this is more work, and just as memory intensive as using the ObjectDataSource.”  And you would be right if you used an InProc session variable to do this.  But, if you used a Session Server, specifically the SQL Session Server, to do this, you can off load the memory onto the database but still have the advantage of caching your data.


Store It

Depending on the data you are caching and how long it is good for, you might want to consider storing the data into your own database.  In the case of the question that was asked of me, this might be the best solution.  You could retrieve the data once and use it for every visitor for hours after.  While implementing Caching on the ObjectDataSource and using Session Variables are both methods that only benefit one user at a time.  Storing the data into a database benefits all of the users.  The obvious down side to mirroring the data like this is that it takes considerably more effort to code and the first request will take considerably longer to process.  But, there are ways of dealing with those problems as well.


Depending on the data you are trying to deal with, one of these methods should work for you.


Other places talking about ObjectDataSource:


Other places talking about Sessions:

  • how to clear the session while loging out(ASP.Net,C#)

    - Hi, I would like to know how to clear the session while logging out from the application…ie even if i press back button it should say “Session Expired”….

  • Handling Session Timeout

    - The first time you put anything into Session State, ASP.NET creates the Session cookie called ASP.NET_SessionId. When Session Timeout occurs the ASP.NET_SessionId cookie is deleted BUT the Session itself is not. …

Most Commented Post

Comments are closed.