Catching Data for Performance

 

I got a question the other day from a gentleman who is writing an application for YouTube.  His problem is that he’s gathering data from the YouTube web site and redisplaying it in his web application.  Once he has it, he then allows the user to page it and sort it.  His question was, “Is there any way of caching the data so that I don’t have to go back to the YouTube web site each time?”

There are actually several ways of dealing with this issue.

ObjectDataSource

If you are looking for an automatic way of handling this problem.  Look no further than the ObjectDataSource.  There are several properties hanging off of this that can help you:

EnableCaching Set this to true to enable caching at the ObjectDataSource level.
CacheDuration Length of time, in seconds, to cache the data.  You can also specify “Infinite” if you want to hold the data forever.
CacheExpirationPolicy Absolute or Sliding.  Controls if duration is from first call to the server (absolute) or last call to the server (sliding)
CacheKeyDependancy Allows you to programmatically expire they cache based on some key.

 

This method stores the data on the server and associates it with the page.  Somewhat memory intensive as is the next method.

 

Session Variables

You could also do everything programmatically and push the DataSet into a session variable.  To do this, you’ll want to create a key for specific search conditions, if any and store the DataSet as the value.

You may be thinking at this point, “this is more work, and just as memory intensive as using the ObjectDataSource.”  And you would be right if you used an InProc session variable to do this.  But, if you used a Session Server, specifically the SQL Session Server, to do this, you can off load the memory onto the database but still have the advantage of caching your data.

 

Store It

Depending on the data you are caching and how long it is good for, you might want to consider storing the data into your own database.  In the case of the question that was asked of me, this might be the best solution.  You could retrieve the data once and use it for every visitor for hours after.  While implementing Caching on the ObjectDataSource and using Session Variables are both methods that only benefit one user at a time.  Storing the data into a database benefits all of the users.  The obvious down side to mirroring the data like this is that it takes considerably more effort to code and the first request will take considerably longer to process.  But, there are ways of dealing with those problems as well.

 

Depending on the data you are trying to deal with, one of these methods should work for you.

 

Other places talking about ObjectDataSource:

  • Using ASP.NET Dynamic Data with ObjectDataSource

    - ObjectDataSource: your page uses an ObjectDataSource to have ASP.NET interact with your entity and business layer classes. You tell the data source exactly which method to call for each CRUD operation. …

  • Building Custom Paging with LINQ, ListView, DataPager and …

    - To be able to use ObjectDataSource in proper way, I made a simple Business Class that wrap the NorthwindDataContext -Northwind LINQ to SQL Class- and called it NorthwindProducts. This class basically contain 2 static methods one that …

  • Exporting to Excel directly from ObjectDataSource?

    - Summary: I have a gridview with an objectdatasource. … Dim tw As New IO.StringWriter … Dim frm As HtmlForm = New HtmlForm … I dislike that I have to pull from the Gridview at all. …

 

Other places talking about Sessions:

  • how to clear the session while loging out(ASP.Net,C#)

    - Hi, I would like to know how to clear the session while logging out from the application…ie even if i press back button it should say “Session Expired”….

  • Handling Session Timeout

    - The first time you put anything into Session State, ASP.NET creates the Session cookie called ASP.NET_SessionId. When Session Timeout occurs the ASP.NET_SessionId cookie is deleted BUT the Session itself is not. …

Related Post

  • .NET Image Scaling in CSharp.NET Image Scaling in CSharp One feature of .NET that I use regularly is image scaling.  I typically use this on web sites that need image upload capabilities.  I assume the user is going to send me an […]
  • Git, Gitosis, Putty and WindowsGit, Gitosis, Putty and Windows I was recently sold on the idea of replacing SubVersion as my version control of choice and moving to GIT, or at least installing GIT and seeing for myself if it is that much better than […]
  • Upload a File via WebRequest Using CSharpUpload a File via WebRequest Using CSharp I got this question a couple of weeks ago but just never had the time to put into answering fully.  But today I have some extra time due to the fact that I’m under-booked with […]
  • DotNetNuke Skins – Skin ObjectsDotNetNuke Skins – Skin Objects There are a number of DotNetNuke controls, which they call objects, that are available to the skin designer. This adds functionality to the page such as a link to login/logout, date/time, […]
  • ASP.NET/VB.NET – Video TrainingASP.NET/VB.NET – Video Training Master the leading internet Web application programming languages Includes 5 levels of ASP.NET focusing on everything from server side […]

About Dave Bush

Dave Bush is a .NET programmer and Certified ScrumMaster who is passionate about managing risk as it relates to developing software. When he is not writing or speaking about topics related to Application Lifecycle Risk Management (ALRM), he is an example to his peers as he develops web sites in the ASP.NET environment using industry best practices.

Awards & Certs