We've recently been working on an inherited project that's got some interesting "features", one of which is how they handle URL rewriting.

There are a number of HTTPHandlers that can be plugged into your application or ISAPI filters to enable URL rewriting in IIS (and now routes in ASP.Net 4 etc) but a fairly classic/old method was to handle it within your Global.asax's Application_BeginRequest method.

A Simple Example

The example below rewrites a path such as "/article/123.aspx" and transforms it to "/articledisplay.aspx?id=132"

<%@ Application Language="C#" %>
<script RunAt="server">
    /// <summary>
    /// Begins the application request
    /// </summary>
    /// <param name="sender">The source of the event</param>
    /// <param name="e">A System.EventArgs that contains the event data</param>
    void Application_BeginRequest(Object sender, EventArgs e)
    {
        // Get the current path (this will be an root relative link e.g. /article/123.aspx)
        string path = Request.Path.ToLower();

        // Work out if we want to transform it
        if (path.Contains("/article/"))
        {
            string id = path.Replace("/article/", String.Empty).Replace(".aspx", String.Empty);
            Context.RewritePath(String.Concat("~/articledisplay.aspx?id=", id), false);
        }
    }
</script>

Ignoring the pro's and cons of using this method to rewrite the paths, one thing you should be aware of is that Context.RewritePath does not end the code execution.

Why's that important to know?

The problem with this is simple, if you have additional rules later in the code, they will also run -which could end up causing quite a lot of confusion.

Consider the following example Application_BeginRequest method (overlook the semantics of the code):

// Get the current path (this will be an root relative link e.g. /category/123.aspx)
string path = Request.Path.ToLower();

// It's a category request
if (path.Contains("/category/"))
{
    // Use the id to perform some form of database lookup i.e. the category
    int id = Category.GetIdByUrl(path);

    // Redirect the user to the page to display the category
    Context.RewritePath(String.Concat("~/rewritepath.aspx?type=category&id=", id), false);
}

// It's a product request
if (path.Contains("/product/"))
{
    string id = path.Replace("/product/", String.Empty).Replace(".aspx", String.Empty);

    // Redirect the user to the page to display the category
    Context.RewritePath(String.Concat("~/rewritepath.aspx?type=product&id=", id), false);
}

Now think about where the visitor would end up if they go to: /category/product/123.aspx is it /rewritepath.aspx?type=category&id=123 or /rewritepath.aspx?type=product&id=123?

Due to the way Context.RewritePath works, it executes /rewritepath.aspx?type=product&id=123 so where's the issue?

The problem is that although the pages behind it don't actually get executed until the end of the Application_BeginRequest method, because the url satisfies both the url rules, the database call will happen -even though in reality, it's not needed. The result is that you could end up with a massive (and unnecessary) overhead to every request hitting the ASP.Net engine (and if you have wildcard mapping enabled bare in mind that's every request -including images/css/javascript).

So if you feel the need to include additional rules (or additional processing e.g. database calls), consider the order of your rules and return from the method as soon as you know your rules are getting fulfilled, the above example would then be written as:

// Get the current path (this will be an root relative link e.g. /category/123.aspx)
string path = Request.Path.ToLower();

// It's a product request
if (path.Contains("/product/"))
{
    string id = path.Replace("/product/", String.Empty).Replace(".aspx", String.Empty);

    // Redirect the user to the page to display the category
    Context.RewritePath(String.Concat("~/rewritepath.aspx?type=product&id=", id), false);

    // We know that the user is going to the right place so no more rules need to be executed -return
    return;
}

// It's a category request
if (path.Contains("/category/"))
{
    // Use the id to perform some form of database lookup i.e. the category
    int id = Category.GetIdByUrl(path);

    // Redirect the user to the page to display the category
    Context.RewritePath(String.Concat("~/rewritepath.aspx?type=category&id=", id), false);

    // There are no more rules so no need to return
}

Other things to note

Using my somewhat simplistic example, there are a number of other things you could do to improve the code's maintainability and performance:

• Consider using "StartsWith" instead of contains to make the match more specific (the code we had used "IndexOf() != -1" throughout -but then also had a couple of "IndexOf() == -1" to spice it up). There may be a minor overhead doing this however it makes it a lot easier to understand what you want to achieve.
• Try not to do anything other than "forward" the request on as quickly as possible i.e. no database calls!
• Add known exclusions at the start of the code where you know you don't need to rewrite the path i.e. the folders for CSS, images and JavaScript. You could also approach this by wrapping all rules in an inclusion i.e. if(path.StartsWith("category") || path.StartsWith("product"))
• You should write the rules not only in order of importance/process but while doing so, give consideration to which will be processed most frequently e.g. if you have a rule that's only use by 1 in 1000 calls as the first rule, this will be checked before the relevant rules 999 times in 1000 -so what may be a very small overhead could become a large bottleneck as soon as your site starts to grow in popularity
• Document each rule clearly i.e. explaining what url formats should trigger it
• Where rules are mutually exclusive, use "else if" rather than just "if" to make it clear that you don't expect the others to be run if the first is valid.

Simple tip this afternoon. You may have got the following error when sending emails through ASP.Net’s built in mail server before:

From: postmaster@YourWebserversName [mailto:postmaster@YourWebserversName] 
Sent: 25 June 2010 13:22
To: sender@sendingdomainname.com
Subject: Delivery Status Notification (Failure)

This is an automatically generated Delivery Status Notification.

Delivery to the following recipients failed.

      recipient@recievingdomainname.com

Reporting-MTA: dns;YourWebserversName
Received-From-MTA: dns;YourWebserversName
Arrival-Date: Fri, 25 Jun 2010 13:21:30 +0100

Final-Recipient: rfc822;recipient@recievingdomainname.com
Action: failed
Status: 5.5.0
Diagnostic-Code: smtp;504 <YourWebserversName>: Helo command rejected: need fully-qualified hostname

The fix is easy:

1. Open IIS
2. View the properties of you Default SMTP Virtual Server
3. Go to the “Delivery” tab
4. Click the “Advanced” button (in the bottom right corner)
5. Under “Fully-qualified domain name” enter a domain name that points to the server
6. Click Ok until you’re back to IIS

If you're not configuring Umbraco through a web installer, you've had your installs in place for years and never checked the permissions or whoever set the permissions up was lazy and gave IIS write access to the entire folder, there will come a time when you want to restrict modify access to just those user(s) who should have access.

You can find a (pretty) complete list of the files/folders that the Umbraco install should have access to here but assigning them across 101 different installs is a PITA . Thanks to a helpful PowerShell script to set folder permissions from PowerShell.nu you can easily automate the process.

For those of you not familiar with PowerShell (like me) complete instructions are below. For the rest, here's the command:

Get-ChildItem -path ##PATH TO YOUR INSTALL## 
| Where { $_.name -eq "Bin" -or $_.name -eq "Config" -or $_.name -eq "Css" -or $_.name -eq "Data" -or $_.name -eq "Masterpages" -or $_.name -eq "Media" -or $_.name -eq "Scripts" -or $_.name -eq "Umbraco" -or $_.name -eq "Umbraco_client" -or $_.name -eq "UserControls" -or $_.name -eq "Xslt" } 
| ForEach {./SetFolderPermission.ps1 -path $_.Fullname -Access "NETWORK SERVICE" -Permission Modify}

Instructions:

1. Save the SetFolderPermission.ps1 script to your server
2. Open your PowerShell console (I think it's installed by default if not, you can download PowerShell here)
3. Copy the above PowerShell command into notepad
4. Update "##PATH TO YOUR INSTALL##" to your Umbraco install
5. If your IIS install doesn't use NETWORK SERVICE as the default user, update it to your user
6. Make sure it's all on a single line
7. Copy/Paste/Run in PowerShell

Bonus

If you're uber lazy and just have a web folder of Umbraco installs you can set the path to the folder of Umbraco installs and use:

Get-ChildItem -path ##PATH TO YOUR FOLDER## -recurse
| Where { $_.name -eq "Bin" -or $_.name -eq "Config" -or $_.name -eq "Css" -or $_.name -eq "Data" -or $_.name -eq "Masterpages" -or $_.name -eq "Media" -or $_.name -eq "Scripts" -or $_.name -eq "Umbraco" -or $_.name -eq "Umbraco_client" -or $_.name -eq "UserControls" -or $_.name -eq "Xslt" } 
| ForEach {./SetFolderPermission.ps1 -path $_.Fullname -Access "NETWORK SERVICE" -Permission Modify}

I've not tried this mind you and can't recommend it but hey, it's there if you want it ;)

Exactly a year ago today I posted a little application that output the sites in IIS to a text file and as a few days ago Lars asked for the source, I thought it would be a nice thing to release it exactly a year later.

I didn't plan it that way, it just happened! Cool :)

Identify IIS Sites and Log File locations for WWW and FTP source

using System;
using System.DirectoryServices;
using System.IO;
using System.Collections;

namespace IISSites

{...}

{
    class Program

    {...}

    {
        static string fileToWrite = String.Empty;

        [STAThread]
        static void Main(string[] args)

        {...}

        {
            fileToWrite = String.Format("IISExport{0:dd-MM-yyyy}.txt", DateTime.Today);
            if (args != null && args.Length > 0)

            {...}

            {
                fileToWrite = args[0];
            }

            SortedList www = new SortedList();
            SortedList ftp = new SortedList();
            try

            {...}

            {
                const string FtpServerSchema = "IIsFtpServer"; // Case Sensitive
                const string WebServerSchema = "IIsWebServer"; // Case Sensitive
                string ServerName = "LocalHost";
                DirectoryEntry W3SVC = new DirectoryEntry("IIS://" + ServerName + "/w3svc", "Domain/UserCode", "Password");

                foreach (DirectoryEntry Site in W3SVC.Children)

                {...}

                {
                    if (Site.SchemaClassName == WebServerSchema)

                    {...}

                    {
                        string LogFilePath = System.IO.Path.Combine(
                            Site.Properties["LogFileDirectory"].Value.ToString(),
                            "W3SVC" + Site.Name);
                        www.Add(Site.Properties["ServerComment"].Value.ToString(), LogFilePath);
                    }

                }

                DirectoryEntry MSFTPSVC = new DirectoryEntry("IIS://" + ServerName + "/msftpsvc");
                foreach (DirectoryEntry Site in MSFTPSVC.Children)

                {...}

                {
                    if (Site.SchemaClassName == FtpServerSchema)

                    {...}

                    {
                        string LogFilePath = System.IO.Path.Combine(
                            Site.Properties["LogFileDirectory"].Value.ToString(),
                            "MSFTPSVC" + Site.Name);
                        ftp.Add(Site.Properties["ServerComment"].Value.ToString(), LogFilePath);
                    }

                }

                int MaxWidth = 0;
                foreach (string Site in www.Keys)

                {...}

                {
                    if (Site.Length > MaxWidth)
                        MaxWidth = Site.Length;
                }

                foreach (string Site in ftp.Keys)

                {...}

                {
                    if (Site.Length > MaxWidth)
                        MaxWidth = Site.Length;
                }

                OutputIt("Site Description".PadRight(MaxWidth) + "  Log File Directory");
                OutputIt("".PadRight(79, '='));
                OutputIt(String.Empty);
                OutputIt("WWW Sites");
                OutputIt("=========");
                foreach (string Site in www.Keys)

                {...}

                {
                    string output = Site.PadRight(MaxWidth) + "  " + www[Site];
                    Console.WriteLine(output);
                    OutputIt(output);
                }

                if (ftp.Keys.Count > 0)

                {...}

                {
                    OutputIt(String.Empty);
                    OutputIt("FTP Sites");
                    OutputIt("=========");
                    foreach (string Site in ftp.Keys)

                    {...}

                    {
                        string output = Site.PadRight(MaxWidth) + "  " + ftp[Site];
                        OutputIt(output);
                    }

                }

            }

            // Catch any errors
            catch (Exception e)

            {...}

            {
                Console.WriteLine("Error: " + e.ToString());
            }

            finally

            {...}

            {
                Console.WriteLine();
                Console.WriteLine("Press enter to close/exit...");
                //Console.Read();
            }

        }

        static void OutputIt(string lineToAdd)

        {...}

        {
            Console.WriteLine(lineToAdd);

            if (!String.IsNullOrEmpty(fileToWrite))

            {...}

            {
                StreamWriter SW;
                SW = File.AppendText(fileToWrite);
                SW.WriteLine(lineToAdd);
                SW.Close();
            }

            else

            {...}

            {
                Console.WriteLine("locationToOutput is Null or String.Empty please supply a value and try again.");
            }

        }

    }

}

Doug Setzer posted this comment in response to my recent "A seriously elegant SQL Injection" post and I thought it may be of interest to others so have promoted it to a post...

Well, I'll step up and say that I am the "mate" who had this done.  Tim's right - *always* sanitize your inputs.  In my defence, this was a site that I inherited from a previous contractor.  I'm not entirely absent of blame, I still should have done a security sweep through the code.

I'd like to document the steps that I went through once this was identified to try and avoid this kind of thing in the future.

1. Edit every web page that executes a query to sanitize any parameters that are passed in.  Since the site was classic ASP, I used my "SQLStringFieldValue" function:
   www.27seconds.com/kb/article_view.aspx?id=50
2. Modify the DB user account that is used to have *read only* access to the database
3. Modify the pages that DO write to the database to have *read/write* access to the specific tables that are being changed.  This limits the number of places that SQL Injection can occur to a smaller set than was previously possible.  I still sanitize all of my input, but I'm extra spastic in these database calls.
4. Add database auditing (triggers writing to mirror tables with audit event indicator & date/time) to see when data changes occur.  This is still problematic with the pages that have "write" permissions to the tables, but again- that footprint is much smaller.

My future plans are to move to a view/stored procedure based architecture.  I can then limit write permissions to just the stored procedures and read permissions to just the views.  My grand gusto plans are to move to using command objects & parameters, but I'd sooner re-write the entire site.

Although Doug's attack wasn't the same nihaorr1.com attack that's going around atm it was similar so I would imagine other's will find this useful.

It still amazes me how many developers still fail to sanitise strings, only last week I came across another site (in PHP) that was allowing simple SQL injections to be used to log into their administration system. It was down to a problem with the sanitization string, but why not at least check your site before it goes live? It takes 2 minutes and even less to fix...

For those of you who need a few pointers, there's a good discussion or two about sanitising strings on the 4 Guys From Rolla site.

As with my previous post, we upgraded the AJAX framework on the weekend which broke a few things, but one control in particular that broke was our TextChangedTextBox which is based on Pete Kellner's timed postback control. Since updating we were receiving a "'debug' is undefined" error on line 1409 (which was in one of the JavaScript include files).

Having had this issue before I updated the TextChangedBehavior.js but that didn't sort it, I have the latest version of the Futures on the server too so I was lost. Turns out I had an old version of the AJAX Futures DLL within the Bin folder of the project.

So as with my post on the ASP.Net forums before -make sure you update your AJAX Futures when updating your Microsoft AJAX framework!

When we got our own dedicated server we needed to start working out a fair number of processes and decide upon a structure that was replicable, scaleable and manageable on a large scale, although the solution we've ended up adopting may not be the best, it certainly works for us.

One thing that has been bugging me however is the location and folder naming convention of the log files -for both the web hits and FTP hits. Typically, shared hosting solutions place the log files under the same folder as the one your website's root is situated but as we had no plans on giving our clients access to these logs this was an unnecessary task so we left them collecting in the default folder.

Leaving the log files in the default folder meant downloading them was very simple, all I needed to do was point our download script at the main folder and that was it, all would be included, the catch however was that the folders weren't named logically* instead they seemed to include some form of ID that was relevant to and assigned by IIS i.e. W3SVC1.

*By this I mean human readable i.e. domainname.com

Until recently I've not worried about analysing the log files beyond one or two clients whom I could manage fairly easily but now with the inclusion of a host of other domains on the server I needed a way of quickly and easily identifying the folders and which domains they related to.

Historically when I needed to know which domain the log folder related to I would log onto the server, open IIS, open the properties of the domain, click on the log file properties and below the folder directory would be the folder name, that's fine if it's only a handful of domains but what when it's say 20? That's 2mins each (with cross referencing etc) so that's 40minutes. I needed an automated system!

As it turns out, Microsoft have been kind enough to provide us with an interface we can easily code against in .Net so after a little Google-ing I wrote a number of little helper applications.

This little console application simply loops through all the domain names on the server it's being run on (the default instance of IIS) and outputs the relevant log file and folder path into a handy text file. I'll post in another post about how I use this file.

For convenience's sake I have this run on a nightly basis and the text file output to the root of the log file directory, that way when I download the logs during the next day I get the latest update of log file locations and domain names :)

Download the IIS WWW and FTP log file location exporter.

1 Year Update: I've posted the source for the IIS WWW and FTP log file location exporter here.

Another post from DougSetzer from 27Seconds.com :)

At my "day job", the systems guys are building new Windows 2003 servers to upgrade our aging Windows 2000 servers.  The plan is to:
 - Build the new Windows 2003 server
 - Install IIS
 - Install .NET
 - Run the IIS migration tool from the old Win2k server

That all went as well as could go - little things got mixed up and had to be corrected.  But, the server would let you request plain HTML files and ASPX files, but classic ASP pages were returned blank.  In poking around Google and the server, we came to find that we had to enable ASP content via:
 - IIS Manager
 - Web Services Extensions
 - Specifically allow Active Server Pages

But, we were still having the same issues.  Stopping and restarting IIS didn't help. Nor did a server reboot.

I found a blog post that mentioned checking that the ASP ISAPI has the correct path.  It tried a random thought that Microsoft has changed the default name of the "Windows"/"Winnt" folder -- Windows NT4, 2000, etc. all use "Winnt", where as Windows 2003 uses the "Windows" folder.  Sure enough, double checking the path to the ASP ISAPI had the wrong path and fixing this path fixed our issues with classic ASP files.

This is a really useful little VBS script that I’ve been meaning to post for a while now (along with a couple of other little applications I’ve written for log file analysis). I don’t think I wrote this script but at the same time can’t recall where it came from.

It basically traverses the FSO finding files with the designated extension and assuming the match the standard IIS date format, checks whether they’re older than x days, if they are deletes them. Running it is simple, place somewhere obvious on the server and just double click it. Alternatively if you want to read the output, run it from CMD. For safety’s sake, the first time you run it I would leave it just printing out the files that will be deleted.

Personally I don’t schedule this script as although automation is great, I’ll probably have it delete the logs before I’ve had a chance to download them so what I tend to do is download the logs and then after that (or the next time I’m on RDC ) I run it, I find that way I ensure I get all the log files i.e. if I go on holiday.

I’ve got two other applications that I’ll post shortly, one outputs the location of the log files for each domain name within IIS and the other combines the log files into one for analysis –it also takes the exported file/folder locations and names the combined log files with the domain’s name –saves a ton of time!

Download the VBS script as a ZIP file

Option  Explicit

Dim intDaysOld, strObjTopFolderPath, strLogFIleSuffix, ObjFS, ObjTopFolder 
Dim ObjDomainFolder, ObjW3SvcFolder, ObjSubFolder, ObjLogFile, ObjFile

intDaysOld        = 5        'Number of days to retain on the server
strObjTopFolderPath    = ""        'The location of your log files
strLogFIleSuffix    = ".log"    'The suffix of your log files

Set ObjFS = CreateObject("Scripting.FileSystemObject")
Set ObjTopFolder = ObjFS.GetFolder(strObjTopFolderPath)

For Each ObjDomainFolder in ObjTopFolder.SubFolders
WScript.Echo("Folder: " & ObjDomainFolder.name)
    For Each ObjW3SvcFolder in ObjDomainFolder.SubFolders
        WScript.Echo("  Folder: " & ObjW3SvcFolder.name)
        Set ObjSubFolder = ObjFS.GetFolder(ObjW3SvcFolder)
            For each ObjLogFile in ObjSubFolder.files
                Set ObjFile = ObjFS.GetFile(ObjLogFile)
                If datediff("d",ObjFile.DateLastModified,Date()) > intDaysOld and lcase(right(ObjLogFile,4))=strLogFIleSuffix then
                    '*****************************************************
                    'DON'T UNCOMMENT THIS UNTIL YOU KNOW IT WORKS PROPERLY!!!
                    WScript.Echo("    Will delete " & ObjSubFolder.name & "\" & ObjFile.name)
                    'WScript.Echo("    Deleted " & ObjSubFolder.name & "\" & ObjFile.name)
                    'ObjFile.Delete
                    '*****************************************************
                End If
                Set ObjFile = nothing
            Next
        Set ObjSubFolder = nothing
    Next
Next

Set ObjTopFolder = nothing
Set ObjFS = nothing

This is a useful Regex that Craig wrote today, it pulls out various info from an apache log file.

private  static Regex __Regex = null;  
internal static Regex _Regex  

{...}

{  
    get  

    {...}  

    {  
        if (__Regex == null)  

        {...}  

        {  
            __Regex = new Regex(@"(?<remoteHost>[^\ ]+?)\ (?<remoteIdent>[^\ ]+?)\ (?<remoteUs"
                + @"er>[^\ ]+?)\ \[(?<requestTime>[^\]]+?)\]\ \""(?<request>(?<r"  
                + @"equestMethod>[^\ ]+?)?\ ?(?<requestPath>[^\ ]+?)?\ ?(?<reque"
                + @"stProtocol>[^\ ]+?)?)\""\ (?<statusCode>[^\ ]+?)\ (?<sizeByt"
                + @"es>[^\ ]+?)\ \""(?<referer>[^\""]*?)\""\ \""(?<userAgent>[^\""]"
                + @"+?)\""\r?\n?", 
                RegexOptions.Compiled & RegexOptions.IgnoreCase & RegexOptions.IgnorePatternWhitespace & RegexOptions.CultureInvariant);
        }  

        return __Regex;  
    }  

}

Update: Craig's finally started writing about it, you can read the article here: Apache Log Fun