Monday, March 19, 2007

How to use Phil's error reporting code

I’ve done a number of posts now on Phil Whinstanley’s error reporting class and this blog appears to be getting a lot of hits because of that which is pretty neat, as a result I’ve had a couple of people write to me asking similar questions about the code so I thought it would be an idea to write a little summary.

Where can I download the code?

It would appear that most of the old copies of Phil’s code have disappeared from the web, I’m not sure why so I’ve uploaded the versions I’ve got below. For convenience I have compiled the code into DLLs for those that don’t know/want to do this and I’ve also included the Visual Studio solutions. I’m not sure if these are based on the original codebase but I don’t think I’ve made any major alterations to these versions:

• The original WebException v2 (62KB)
• WebException v4¹ (20KB)

¹ This is a version I was sent as his original including changes and example email was lost...

DLLs only:

• ASP.Net 1.1 DLL only (17KB)
• ASP.Net 2.0 DLL only (16KB)

If you have Visual Studio:

• ASP.Net 1.1 (Visual Studio 2003) version without DLL (24KB)
• ASP.Net 2.0 (Visual Studio 2005) version without DLL (26KB)
• ASP.Net 1.1 (Visual Studio 2003) version with DLL (131KB)
• ASP.Net 2.0 (Visual Studio 2005) version with DLL (59KB)

If you don’t have Visual Studio you can either download one of the above projects and delete the solution/project files or download the original WebException code. Ok, now you have the files :) -FWIW I can accept no responsibility for any of the files or the code, I just zipped them!

How to do I use the WebException class?

I’m now using a slightly modified version of the code to enable error reporting within AJAX (see: Reporting errors from AJAX using the WebException Class) which I’ll try and upload later but whichever version of the code you choose the use is pretty much the same.

Once you have referenced the DLL in your project (see: Importing/Referencing DLLs in Visual Studio) you will be able to use the WebException. As I’ve covered what you need to do to use the code from within an AJAX application in another post (see: Reporting errors from AJAX using the WebException Class) I’ll just cover how to use it to report global errors. To capture and respond to all application errors you will need to place this code within the global.asax, your project should automatically have one, if it doesn’t then you will need to add one.

Using the global.asax file, the first thing you need to do is add a reference to the DLL at the top of your code (this will allow you to call the methods and access the properties):

<%@ Import Namespace="ErrorReporting" %>

Next locate the Application_Error event handler, this is the method that handles all errors within the application (with exception of those thrown from within an AJAX application, read this post to report errors from within an AJAX application). Now replace your Application_Error and Application_PreRequestHandlerExecute handlers with (for more information on what I'm doing here see: ASP.Net WebException and Error Reporting useful code):

void Application_Error(object sender, EventArgs e)

{...}

{
    bool reportErrors = Convert.ToBoolean(System.Configuration.ConfigurationManager.AppSettings["SendErrors"]);

    if (reportErrors)

    {...}

    {
        Exception currentError = Server.GetLastError();

        Deal with 404's

        #region Deal with 404's

        //Redirect the user to a friendly page
        if(CheckForErrorType(currentError, "FileNotFound"))
            RedirectToFriendlyUrl("");

        #endregion

        Deal with Spambots

        #region Deal with Spambots

        if (CheckForErrorType(currentError, "System.FormatException"))

        {...}

        {
            if (HttpContext.Current.Request.Form.Count > 0)

            {...}

            {
                foreach (string key in HttpContext.Current.Request.Form)

                {...}

                {
                    if (key.IndexOf("_VIEWSTATE") > 0 && HttpContext.Current.Request.Form[key].ToString().IndexOf("Content-Type") > 0)
                        return;
                }

            }

        }

        #endregion

        //Enable the trace for the duration of the error handling
        TraceContext t = HttpContext.Current.Trace;
        bool bCurrentState = t.IsEnabled;
        t.IsEnabled = true;

        Handle the Exception

        #region Handle the Exception

        ErrorHandling.WebException WE = new ErrorHandling.WebException();
        WE.CurrentException = Server.GetLastError();
        WE.MailFrom = "you@yourdomain.com";
        WE.MailTo = "you@yourdomain.com";
        WE.MailAdmin = "you@yourdomain.com";
        WE.Site = "Your Site's Name or URL";
        WE.SmtpServer = "localhost";
        WE.FloodCount = 10;
        WE.FloodMins = 5;

        #endregion

        Choose what you're interested in

        #region Choose what you're interested in

        WE.ReturnCache = true;
        WE.DrillDownInCache = true;
        WE.IncludeApplication = true;
        WE.IncludeBrowser = true;
        WE.IncludeEnvironmentVariables = true;
        WE.IncludeForm = true;
        WE.IncludeProcess = true;
        WE.IncludeQueryString = true;
        WE.IncludeRequestCookies = true;
        WE.IncludeRequestHeader = true;
        WE.IncludeResponseCookies = true;
        WE.IncludeServerVariables = true;
        WE.IncludeSession = true;
        WE.IncludeTrace = true;
        WE.IncludeVersions = true;
        WE.IncludeAuthentication = true;

        #endregion

        WE.Handle();

        //Return the trace to its original state
        t.IsEnabled = bCurrentState;

        //Redirect the user to a friendly page
        RedirectToFriendlyUrl("");
    }

}

protected void Application_PreRequestHandlerExecute(Object sender, EventArgs e)

{...}

{
    if (Context.Handler is IRequiresSessionState || Context.Handler is IReadOnlySessionState)
        ErrorReporting.SessionTracker.AddRequest("Pre Request Handler Execute", true, true, false);
}

private bool CheckForErrorType(Exception ex, string errorText)

{...}

{
    if (ex != null)

    {...}

    {
        //Check the exception
        if (ex.GetType().ToString().IndexOf(errorText) > 0)
            return true;
        else
            return CheckForErrorType(ex.InnerException, errorText);
    }

    else

    {...}

    {
        return false;
    }

}

private void RedirectToFriendlyUrl(string Url)

{...}

{
    if (!String.IsNullOrEmpty(Url) && (Request.Url.Host.IndexOf("localhost") < 0))
        Response.Redirect(Url);
}

This will create a new instance of the WebException object, assign the various properties accordingly (you will need to configure these) and then finally handle the error.

That’s it! That’s all you really need to do to have super error reporting instantly installed in your application! If that wasn't enough it's overloaded with a couple of filters for you :). I recommend you read one of my previous posts I’ve added which overviews a few simple tips and tricks when using the WebException class to that improves on its functionality (see: ASP.Net WebException and Error Reporting useful code).

All that's left to do is to test it works (see below).

What should I get from it?

That’s the million dollar question! Once the WebException class has been added to your application you should receive an email every time the application throws an error (which of course means you’ll never get an email from the system!)

View an example of the email you’ll get with all outputs set to true.

More tips/Warnings!

Ok so it’s installed and you’re getting no errors through (because your codes perfect) but there are a couple of other little tweaks I would make to the WebException class to make it a little more useable.

Create a centralised class for it

A while ago I posted a set of “useful” tips for reducing the number of spambot related emails, redirecting the user etc (see: ASP.Net WebException and Error Reporting useful code). That’s fine until you start including the WebException class into multiple projects, managing tweaks to the codebase gets a little cumbersome (i.e. adding the spambot check to all our projects that use the WebException meant a couple of hours of copying and pasting). The work around for me was to wrap it all up into a central static method (see: Reporting errors from AJAX using the WebException Class). I did this rather than fiddling with Phil’s WebException class itself incase he ever got around to releasing another version which would mean a bunch of changes etc.

Limit the page request log

If you have a site where every user is likely to have a high page visit count with most of the pages involving some form of form submission then it may be worth limiting the number of request’s stored as we have found that without limiting these we start receiving very large emails (some topping 10MB).

The reason this is happening is because the session tracker logs all the form elements for the request so if you had i.e. a CMS that submits a page of content every other page request all that data will be stored in the tracker, sticking with the idea of a CMS, your typical text word is around 10bytes (see: How many bytes for...), so say the user writes 500 words per page (which isn’t really a lot) that’s 4.9Kb per form submission plus on the re-display of the page you've got ViewState... That’s just the data submitted by the user, around that, you’ve got all the form fields, field names, session info, query string etc, see how it starts to add up?

The solution is fairly straight forward, what you need to do is alter SessionTracker.cs¹:

¹I thought I'd done this in a project already but cannot find the source so this may not work.

public class SessionTracker

{...}

{
    public static void AddRequest(string Comments, bool DoForm, bool DoQueryString, bool DoCookies)

    {...}

    {
        Request R = new Request();
        R.Time = DateTime.Now;
        R.Comments = Comments;
        
        if (System.Web.HttpContext.Current != null)

        {...}

        {
            R.Path = System.Web.HttpContext.Current.Request.Path.ToString();
            if (System.Web.HttpContext.Current.Request.UrlReferrer != null)

            {...}

            {
                R.Referrer = System.Web.HttpContext.Current.Request.UrlReferrer.ToString();
            }

            if (DoForm)

            {...}

            {
                R.Form = System.Web.HttpContext.Current.Request.Form;
            }

            if (DoQueryString)

            {...}

            {
                R.QueryString = System.Web.HttpContext.Current.Request.QueryString;
            }

            if (DoCookies)

            {...}

            {
                R.Cookies = System.Web.HttpContext.Current.Request.Cookies;
            }

        }

        if (System.Web.HttpContext.Current.Session["RequestCollection"] != null)

        {...}

        {
            RequestCollection RC = ((RequestCollection)System.Web.HttpContext.Current.Session["RequestCollection"]);
            RC.Add(R);
            if(RC.Count > 10)
                RC.RemoveAt(0);
            System.Web.HttpContext.Current.Session["RequestCollection"] = RC;
        }

        else

        {...}

        {
            RequestCollection RC = new RequestCollection();
            RC.Add(R);
            System.Web.HttpContext.Current.Session["RequestCollection"] = RC;
        }

    }

    public static void AddRequest(string Comments)

    {...}

    {
        Request R = new Request();
        R.Time = DateTime.Now;
        R.Comments = Comments;
        
        if (System.Web.HttpContext.Current != null)

        {...}

        {
            R.Path = System.Web.HttpContext.Current.Request.Path.ToString();
            if (System.Web.HttpContext.Current.Request.UrlReferrer != null)

            {...}

            {
                R.Referrer = System.Web.HttpContext.Current.Request.UrlReferrer.ToString();
            }

            R.Form = System.Web.HttpContext.Current.Request.Form;
            R.QueryString = System.Web.HttpContext.Current.Request.QueryString;
            R.Cookies = System.Web.HttpContext.Current.Request.Cookies;
        }

        if (System.Web.HttpContext.Current.Session["RequestCollection"] != null)

        {...}

        {
            RequestCollection RC = ((RequestCollection)System.Web.HttpContext.Current.Session["RequestCollection"]);
            RC.Add(R);
            if (RC.Count > 10)
                RC.RemoveAt(0);
            System.Web.HttpContext.Current.Session["RequestCollection"] = RC;
        }

        else

        {...}

        {
            RequestCollection RC = new RequestCollection();
            RC.Add(R);
            System.Web.HttpContext.Current.Session["RequestCollection"] = RC;
        }

    }

    public static void AddRequest()

    {...}

    {
        Request R = new Request();
        R.Time = DateTime.Now;
        
        if (System.Web.HttpContext.Current.Session["RequestCollection"] == null)

        {...}

        {
            RequestCollection RC = ((RequestCollection)System.Web.HttpContext.Current.Session["RequestCollection"]);
            RC.Add(R);
            if (RC.Count > 10)
                RC.RemoveAt(0);
            System.Web.HttpContext.Current.Session["RequestCollection"] = RC;
        }

        else

        {...}

        {
            RequestCollection RC = new RequestCollection();
            RC.Add(R);
            System.Web.HttpContext.Current.Session["RequestCollection"] = RC;
        }

    }

    public SessionTracker()

    {...}

    {
    }

}

Outputting the Trace with the WebException Class

I know this is something I’ve posted about in the past but since moving to version 4 of the code and .Net 2.0 I was no longer getting the trace in my lovely error reports, after a little digging I’ve found a solution, in addition to the code that I posted earlier about enabling the trace using C#, the web.config needs to be set as follows:

<trace enabled="true" requestLimit="100" pageOutput="false" traceMode="SortByTime" localOnly="true" />

Storing the WebException code in App_Code Dir

If you use the WebException class in an ASP.Net 2.0 site, be careful you don’t do what we did and throw the site online uncompiled with a compilation error as it won’t get reported. Luckily I found this issue on a test site but it’s still worth noting.

Personally I wouldn’t put the error reporting code in the App_Code directory as this means you’ll end up needing to maintain a plethora of files throughout various projects. Instead compile a separate DLL and include that in your projects, then if like me you find a nice addition to the error reporting code you can easily update all sites to the latest version!

Setup a simple generic test page

Nothing fancy, just a button that throws an exception will do:

TestErrorPage.aspx

<%@ Page Language="C#" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<Script>

<script runat="server">
    protected void btnError_Click(object sender, EventArgs e)

    {...}

    {
        throw new ArgumentException("Test Error");
    }

</script>

<html xmlns="http://www.w3.org/1999/xhtml" >
<head runat="server">
    <title>Test Error Page</title>
</head>
<body>
    <form id="form1" runat="server">
    <div>
        <p><asp:Button runat="server" ID="btnError" Text="Throw Error" OnClick="btnError_Click" /></p>
    </div>
    </form>
</body>
</html>

Happy Error Reporting :) -I'm hoping this is the last time I need to blog about this code but what's the betting another post is around the corner ;)

How to use Phil's error reporting code

Useful Links:            Comments [2] 

Tags: ASP.Net | Web Development | WebDD | Error Reporting

 Friday, March 16, 2007

Payment on acounts

I recently released a mini-series of articles with a load of great business start-up advice (also available to download as a PDF) in which I talk about how the government is really going the extra mile for SMEs at the moment, helping them out in a load of different ways -really trying to encourage entrepreneurship within the United Kingdom. This was something I believed until I got my tax bill.

The deadline for your self-assessment tax return is fast approaching and the government kindly sends you an invoice for your last tax return, mine arrived just before Christmas and to my surprise it was 50% more than I submitted. Seeing as I was off to Cornwall I decided I would deal with it on my return.

I've never had an issue with paying tax, in fact I've always been proud of the fact that I've needed to pay tax, my logic is simple: if I need to pay tax, it means I'm earning! The more tax I pay, the more I've earned. See my logic?

In the past, through claiming back expenses etc my tax bills have always been relatively small and although I seem to recall something called “Payment on account”, but it was IIRC an optional payment so next year the amount you have to pay is smaller. This is a good idea. As a small business owner I do put money aside for my tax bill but that’s always been 25% of each invoice. So what’s my issue?

To me, payment on account seems like a half-arsed attempt by the government to help self-employed people out, while getting a couple more quid in the process, the issue however is I feel they’ve focused more on taking more money than helping out the self-employed people. The idea is simple: You submit your tax bill for the previous tax year, they take an additional payment (payment on account) which will go towards the next tax year, this payment is estimated on their data for you -in this case your tax bill. The first payment is due with your current tax bill, the second in July of the current tax year. Thus splitting the next year’s tax bill into two more manageable payments. -Bollocks does it. Please excuse my French but this is not at all thought out and this is why:

I would consider myself to be one of the more sensible self-employed people when it comes to saving for tax, admittedly the first couple of years I was in business I paid the tax bill with a project we had on at the time but now I do put money aside for it with every invoice (now 50% of every invoice inc VAT goes aside). So come tax return time I have a nice chunk of cash to pay for my tax bill (always more than it needs to be because of expenses etc). I then follow the governments recommendation and fill out my self-assessment tax return online in plenty of time -a word of warning, I live with a chartered accountant who helps me out with this, it’s not something I just do on my own ;). In January I have the invoice for the bill and I pay it out of my savings, anything left I can use as I wish. All good so far! As far as the government is concerned I have followed what they’re recommending and that should be it. Payment on account however throws this into turmoil. What the government IMHO neglects to tell you is that you’re going to have to pay 50% more than you’re expecting in the first year you go over their threshold.

What I don’t like about that is they’re encouraging a lot of self-employed people to complete their own return online (best to do this with an accountant really) and save for their bill but by not telling you about this additional payment they can in theory put someone out of business over night -and there’s nothing you can do about it. Take the average self-employed business owner with a turnover of £50,000. Assuming no expenses you should expect to walk away with around £38,300 (using rough maths). Cool, so you’re good and put £11,700 into savings in preparation and use the rest to pay the bills, buy a holiday, a car etc.

You think all’s dandy until at the end of the year you get a tax bill for £17,550 with a further payment of £5,850 being required in July. That leaves you with £26,600 remaining rather than the initial £38,300 you were expecting. Why? That’s simple -payment on account, the tax office say "Well, you earned £50,000 this year, so you’ll do that next year so we’ll take that money from you now, that way next year you’ll already have some money on account -helping you out. Don’t worry though, if your tax bill for next year is lower, we’ll refund the money." -there are so many issues to this statement but I’ll come back to those.

I've made a graph demonstrating the two differences. The light red segment is the amount you will need to pay in the July following your January tax payment, the dark red section must be paid in addition to your main tax bill.

Take my industry -the IT industry. It’s not unknown (or an infrequent occurrence) to have a large project (i.e. £100,000) which you can complete within a financial year. If you're already working in the sector and this £100,000 contract is your reason for going it alone this could be a serious issue for you. Ignoring how you take this money, by the end of a single fiscal year you take £100,000:

• Turnover:£100,000
• Tax Allowance:£5,000
• Taxable Income:£95,000
• Tax at 22%:£7,700
• Tax at 40%:£24,000
• Expected Tax Bill:£31,700
• Expected to you:£68,300

In the event you’ve taken £100,000 for a project you’re likely to spend a fair amount of that on things like credit cards, niceties after having gone without for a while etc, so you spend a fair whack, perhaps put a chunk on your mortgage, buy a house or invest a fair chunk. Say you get the payment a couple of months before the tax year ends so you take the time off and relax a little, basically using a fair amount of the money, but you’re ok as you’ve put £32,000 into savings in preparation for your tax bill. When it comes to filing time however you’re told that you owe them an additional £15,850 with your current tax bill followed by another payment of £15,850 in July. Starting to see where my issue lies?

If you didn’t spend anymore of that £68,300 than you absolutely had to and some how had the additional £31,700 available you’re fine, but what if you decided to treat someone special, or invest the money where it’s not readily accessible, what can you do? I called the tax office to talk it through with them as I didn’t want to pay this payment on account as it would mean that things would be a little tight until the end of a current system development. The representative had absolutely no concern or understanding for my situation, when I asked her if I could spread the payments a little I was told that interest would be charged on the money if I didn’t pay it and a fine would be incurred. Furthermore she told me that this payment on account was ok because it was tax on money we had already earned (the payment on account is in theory for the current fiscal year), I did think about pointing out that a business’ earnings are not the same as a worker’s salary as they are frequently sporadic and go through highs and lows -in the case of The Site Doctor, the majority of our year’s income comes in during the final fiscal quarter.

There were a couple of things I didn’t like about the representatives statement/government’s perceived understanding of the situation:

1. Great they are trying to help you out with your business -hopefully making the next tax year’s tax bill a smaller payment (or not at all if you think about the logic) but how does making it a forced payment without making it very well known about help?
2. It assumes that your business’ monthly turnover is the same as an employee in that your entire year’s earnings are the same (or similar) each month with no seasonal fluctuations. I know there are some more established businesses which do have a regular income but The Site Doctor certainly doesn’t. As already mentioned, The Site Doctor has the majority of the year’s earnings paid in the final fiscal quarter -after you have to pay your tax bill!
3. They say they’ll refund the money if your next bill is lower than the last so it’s ok. But taking the example of the £50,000 turnover above, that’s a years worth of interest on £11,700 you’ve just lost potential interest of £936 or £2,536 in the example of our £100,000 contract. Can you afford to loose out on that?
4. This can in theory put someone out of business, as it happened, I had to pay this bill mid contract when normally I wouldn’t have had any money for a fair while, to make things worse Stacey was having a short sabbatical. Luckily we had the money in savings but if we didn’t we would have without a doubt found it hard to pay.

The solution?

I don’t like bitching and moaning about things without having some form of solution and I can understand that the government wants to get this cash into the bank and after the first year or two it makes things better for them but for goodness sake make the payment optional, perhaps offer a monthly payment option without penalties or at least inform people about this so they can make provisions for it. Had I not found this out before this tax year I would be in serious trouble. Of course, having a tax specialist do your books should have highlighted this for you -and it goes to show that just because they’re an accountant, if they’re not a specialist they may not know about something that can break the bank (I’ve got no blame for Stacey before you wonder!).

Note: These figures are derived from my own experience so please take professional advice on the matter as for all I know, there may be a sliding scale (I would hope there is) otherwise the government is killing businesses left right and centre. I’d be interested to know if anyone else knew of/has experienced this issue.

Payment on acounts

Useful Links:            Comments [0] 

Tags: Business | Business Start-up Advice | The Site Doctor | WebDD

 Thursday, March 15, 2007

When customer service goes mad

I rate good/great customer service very highly when it comes to retaining clients and obtaining new clients so it always make me laugh at how some companies value their customer service and more to the point wonder how they’re still in business!

The other day I had a couple of conversations with Fasthosts that I simply had to share. A little background though –we’ve been with Fasthosts for a couple of years now, I think we originally signed up in 1999 through 4as1 and I know we were one of their first customers. Since then we’ve registered around 300 domains with their sister company UKReg without a problem.

More recently however I’ve been feeling less easy about their service. Firstly they started charging for things left right and centre –ok, they’ve got a business to run, next they started to overload the servers (ok it’s a shared server, I can live with that), to combat the overloaded servers they started moving domains –I’m guessing higher demand sites onto their own boxes. That was ok, except they gave you no advanced warning and seeing as a fair few of the sites in question would reference MS Access databases elsewhere would cause us no-end of update fun.

Then their support went down hill to the extent that we often had to wait for ages until the phone was answered (listening to “you are 17th in the queue”). At this point we had just opened an account with Rackspace so had the view to slowly move the domains over. That was until I got an email saying they would now be charging for a number of services that were historically free which cut into our margins –making them negative. That annoyed me somewhat so moving the sites away from Fasthosts was moved up my priority list.

When we first signed up with Fasthosts they were the dogs, cheap hosting with all the bells and whistles –we had a Windows account which meant –unlike others at the time we had free use of ASP and all sorts but now all they seem interested in is the Yankee dollar rather than customer retention.

The other day was the straw that broke the camels back in regards my tolerance of Fasthosts. I had two fairly simple questions and it wasn’t easy to get a straight response to either. The first related to a domain I was adding for a client, they don’t use it but have access to a personal control panel which allows them to manage their emails etc. The control panel although a little outdated does the job. The catch however is recently you need to pay for use of Fasthosts’ control panel or you can build your own through the API. I didn’t need half the stuff included in the “Standard Bundle” and only wanted Fasthosts’ personal control panel so thought it would be best to ask which package I needed:

Massively long delay before a response finally comes

Great! They’ve removed the control panels I highlighed (and only those ones!) but they were just the ones in the next month! In fairness, after a few more emails I did get the past invoices refunded as well as a fair few future ones taken off the account but it makes me very concerned as to who else is still paying this charge... I would never have questioned it as they did send a mailshot out saying they would now be charging for them etc. Most odd

Conversation 2 – SQL Server Express

As I’ve already mentioned, we’ve been with Fasthosts for around 8 years now (tha