Monday, July 28, 2008

Is Google using Analytics data to crawl additional pages?

I've been wondering for a while how Google has managed to find a couple of hidden pages. Although they were securely locked down we noticed a few rejected GoogleBot requests in the audit logs. We put this down to the users having a Google toolbar installed but today we got an error from the new Avant Garde hair salons site that's just gone into beta testing which got me thinking.

This particular link is hidden behind a form post and within a jQuery call (to track an action) so not something the GoogleBot has easy access to. I know they're getting more clever but not *that* clever! We started getting the errors shortly after adding the final Google Analytics code so the only conclusion I can come to is that they're not just registering the URLs for reporting purposes but they're also using them to crawl additional pages.

Does anyone know if they use the URLs tracked in Google Analytics to find new pages? All I can say is if this is the case, you better make sure your "secure" pages check the access permissions on a page level!

Is Google using Analytics data to crawl additional pages?

Useful Links:            Comments [0]  Trackback

Tags: SEO | The Site Doctor | Web Development

 Friday, July 25, 2008

Identify IIS Sites and Log File locations for WWW and FTP –the source

Exactly a year ago today I posted a little application that output the sites in IIS to a text file and as a few days ago Lars asked for the source, I thought it would be a nice thing to release it exactly a year later.

I didn't plan it that way, it just happened! Cool :)

Identify IIS Sites and Log File locations for WWW and FTP source

using System;
using System.DirectoryServices;
using System.IO;
using System.Collections;

namespace IISSites

{...}

{
    class Program

    {...}

    {
        static string fileToWrite = String.Empty;

        [STAThread]
        static void Main(string[] args)

        {...}

        {
            fileToWrite = String.Format("IISExport{0:dd-MM-yyyy}.txt", DateTime.Today);
            if (args != null && args.Length > 0)

            {...}

            {
                fileToWrite = args[0];
            }

            SortedList www = new SortedList();
            SortedList ftp = new SortedList();
            try

            {...}

            {
                const string FtpServerSchema = "IIsFtpServer"; // Case Sensitive
                const string WebServerSchema = "IIsWebServer"; // Case Sensitive
                string ServerName = "LocalHost";
                DirectoryEntry W3SVC = new DirectoryEntry("IIS://" + ServerName + "/w3svc", "Domain/UserCode", "Password");

                foreach (DirectoryEntry Site in W3SVC.Children)

                {...}

                {
                    if (Site.SchemaClassName == WebServerSchema)

                    {...}

                    {
                        string LogFilePath = System.IO.Path.Combine(
                            Site.Properties["LogFileDirectory"].Value.ToString(),
                            "W3SVC" + Site.Name);
                        www.Add(Site.Properties["ServerComment"].Value.ToString(), LogFilePath);
                    }

                }

                DirectoryEntry MSFTPSVC = new DirectoryEntry("IIS://" + ServerName + "/msftpsvc");
                foreach (DirectoryEntry Site in MSFTPSVC.Children)

                {...}

                {
                    if (Site.SchemaClassName == FtpServerSchema)

                    {...}

                    {
                        string LogFilePath = System.IO.Path.Combine(
                            Site.Properties["LogFileDirectory"].Value.ToString(),
                            "MSFTPSVC" + Site.Name);
                        ftp.Add(Site.Properties["ServerComment"].Value.ToString(), LogFilePath);
                    }

                }

                int MaxWidth = 0;
                foreach (string Site in www.Keys)

                {...}

                {
                    if (Site.Length > MaxWidth)
                        MaxWidth = Site.Length;
                }

                foreach (string Site in ftp.Keys)

                {...}

                {
                    if (Site.Length > MaxWidth)
                        MaxWidth = Site.Length;
                }

                OutputIt("Site Description".PadRight(MaxWidth) + "  Log File Directory");
                OutputIt("".PadRight(79, '='));
                OutputIt(String.Empty);
                OutputIt("WWW Sites");
                OutputIt("=========");
                foreach (string Site in www.Keys)

                {...}

                {
                    string output = Site.PadRight(MaxWidth) + "  " + www[Site];
                    Console.WriteLine(output);
                    OutputIt(output);
                }

                if (ftp.Keys.Count > 0)

                {...}

                {
                    OutputIt(String.Empty);
                    OutputIt("FTP Sites");
                    OutputIt("=========");
                    foreach (string Site in ftp.Keys)

                    {...}

                    {
                        string output = Site.PadRight(MaxWidth) + "  " + ftp[Site];
                        OutputIt(output);
                    }

                }

            }

            // Catch any errors
            catch (Exception e)

            {...}

            {
                Console.WriteLine("Error: " + e.ToString());
            }

            finally

            {...}

            {
                Console.WriteLine();
                Console.WriteLine("Press enter to close/exit...");
                //Console.Read();
            }

        }

        static void OutputIt(string lineToAdd)

        {...}

        {
            Console.WriteLine(lineToAdd);

            if (!String.IsNullOrEmpty(fileToWrite))

            {...}

            {
                StreamWriter SW;
                SW = File.AppendText(fileToWrite);
                SW.WriteLine(lineToAdd);
                SW.Close();
            }

            else

            {...}

            {
                Console.WriteLine("locationToOutput is Null or String.Empty please supply a value and try again.");
            }

        }

    }

}

Identify IIS Sites and Log File locations for WWW and FTP –the source

Useful Links:            Comments [1]  Trackback

Tags: C# | IIS | Server Management | The Site Doctor | Web Development

 Tuesday, July 15, 2008

How to: Convert Hexadecimal Strings

As it's my Birthday today I thought I'd post a silly ditty. I'm currently altering Protx's old ASP.Net library to accommodate their changes in regards 3D Secure and while reflecting some of the code came across an enum with their number representations as Hexadecimal strings. I needed to convert these to decimals so thought I'd share a quick and easy way to do it.

Open up Window's Calculator (Windows Key + R then type in calc) under the View menu select "Scientific". Press the F5 key to switch over to Hex entry. Type in the value after the 0x and hit F6

Simple, easy and will help you convert all those Hexadecimal strings (ones that look like this: 0x01 or 0x1a).

Right, time for a coffee :)

How to: Convert Hexadecimal Strings

Useful Links:            Comments [2]  Trackback

Tags: C# | Development

 Thursday, July 10, 2008

Setting the page title with ASP.Net

We've been setting up the page's title using ASP.Net for quite some time now, we tend to word it: Product Name | Category | Site Name as this IMO is the most comprehensive naming convention (though the pipe (|) gets converted to a space for the bookmarks).

When editing one of our sites today though I noticed that the title was resorting to Avant Garde hair salons -which was setup as the brand's name. Looking into it I found that if you set the <title> tag within the page or master page, ASP.Net doesn't override it from the codebehind so watch out!

For those of you who don't know how to set the title of your page from codebehind it's simple:

this.Page.Title = "Put your title here";

Setting the page title with ASP.Net

Useful Links:            Comments [0]  Trackback

Tags: ASP.Net | C#

 Wednesday, July 09, 2008

You can’t make this stuff up

This is one of the most bizarre stories I've heard in a while, you really can't make it up. http://news.bbc.co.uk/1/hi/england/norfolk/7496923.stm

You can’t make this stuff up

Useful Links:            Comments [0]  Trackback

Tags: Random

 Saturday, July 05, 2008

Deleting SVN directories with PowerShell

I've been re-working our new SVN structures recently as I'm now starting to understand how it works but one of the issues I had was trying to move the files/folders from a previous SVN directory.

PowerShell is great if you understand it (which I'm also learning) so I thought I would share this little script with you. It just loops through the files/folders and removes all those named _svn. I found this script from Wyatt Lyon Preul and he complained about the length of the script, but from what I can tell you can condense that down to:

I'm not that great with PowerShell yet but I hope that helps someone :)

Deleting SVN directories with PowerShell

Useful Links:            Comments [0]  Trackback

Tags: PowerShell | Server Maintenance | Server Management | The Site Doctor | Web Development

 Thursday, July 03, 2008

Market rates –can I have the same hourly rate for all clients?

This started out as a response to a comment and then I thought it might be better as a post in it's own right.

In his comment David Conlisk said:

First off Tim very well done on providing some excellent information on the site. I've just spent my first afternoon as my own boss reading your business start-up advice and it's been excellent (it's called research, not slacking off!)

One question I would ask you about this post is what about market rates? I am going from being a contractor on an hourly rate to being a limited company. I never considered working out a base rate like you've done, instead I spoke to as many people as possible in the marketplace to gauge what the rates are and I price accordingly. Of course this works fine for more corporate clients, but I doubt I could charge smaller companies similar rates. Let's hope I can make a good enough impression on my corporate clients to keep that kind of work coming in!

Keep up the good work,

David

Hi David,

Thanks for your kind words, I'm glad to hear you found it of use.

In regards market rates, it's one of the oldest debates in the book AFAIK and has a rather unhelpful answer of "You should charge what you feel comfortable charging". I'll try to improve on that a little as it's always hard but in essence it's true. Basically from experience I would keep it as simple as possible, have as few rates as possible for all clients, just make sure you feel you're worth the rate in your own mind.

Although you need to keep an eye on the "market rates", you'll find your rate will determine the type of client you work with. Being the cheapest on the market is not necessarily a good thing. One advantage of offering a freelance service to other development companies is that we get to see what happens when your prices are rock bottom -take it from me, more often than not, it's more hassle than it's worth. When you have someone going el-cheapo all the way you often find they're overly picky about every aspect and require a lot more management time (that's not to say those paying higher rates aren't, I guess you just notice it more).

As long as you're reasonable with your rates, clients who are willing to pay your rates, will use you (they may complain a little but it's unlikely) but at the end you'll both be happy with the work produced. As long as you believe in yourself -and your rates, this will be conveyed to your clients so if you know you're value for money you will be able to justify it to any client (corporate or otherwise). It's up to the client to decide whether you're value for money.

Believe it or not the service industry is not the only industry to set it's fees and then get them negotiated on -Stacey used to work in Debenhams a few years ago, for those of you who don't know what Debenhams is, it's a large department store in the UK, they sell items for a set fee, everyone knows this but regardless of this she still had people trying to negotiate on the fee. Be open to negotiation but don't be silly about it otherwise the client may always expect a discount of that level (so stick to no more than a 10% variation).

Don't worry about having clients not use you because of your rate, as long as you're around the market rate there will be a client for you. At the end of the day, you can't realistically expect to service every prospect that comes through your doors -sometimes you just have to say "sorry that's the price".

I'm not saying charge £1,000ph when the market rate is £10ph as that's just silly but I would say your base rate shouldn't be cheaper than the market rate or more than 3 times the market rate (unless your service really is that good and you're bogged down with work [I did have a link for here about an ?SEO company charging $1,000ph and still being too busy but I can't find it atm], in which case go for it!).

Tip: How do you find out market rates? That's simple, find a couple of companies who offer similar services, to a similar client base who are a similar size to you, call them up and just ask them what their daily rates are. Call 10 or so companies and you should have a few prices to compare :)

Another tip: Always ask for an rough idea of their budget -even if it's just a range, this will give you a good idea of they're realistic or not.

And one more: Don't forget your rates don't need to be fixed. If you find you're too busy, increase your rates a little, if you're too quiet (whereas everyone else is really busy) then you may need to look into how you market your business, your presentation skills and finally possibly reducing your rates.

A word of warning: I would avoid dropping your rate "for the nice client" as the majority of times you'll end up regretting it, either because it gets out of control and you get frustrated because "you're doing them a favour" whereas they feel they just negotiated your service rates down (and so should be getting the same level of service. Remember, it's business, you don't need to do anyone a favour, charge what you feel is fair for your time and you'll always enjoy your work :)

On the flip side of this, if you're lucky enough to get a large corporate, make sure your rate is their market rate as we've lost work for being too cheap (and in my eyes we were already overcharging for the workload).

It's easy to be busy and cheap, but being a busy fool is no way to live!

HTH

Tim

Market rates –can I have the same hourly rate for all clients?

Useful Links:            Comments [1]  Trackback

Tags: Business | Business Start-up Advice | Client | Expanding Your Business | General | Marketing | The Site Doctor

 Tuesday, July 01, 2008

Is Amazon back up to its old tricks?

It's gift time again (for me that is!) -yey! But when I was checking out on Amazon.co.uk earlier today I was a little puzzled by this...

On the product details page it said £4.45 shipping (correct me if I'm wrong)

But then when you check out it's suddenly £7.36. I was checked in by this stage so did Amazon think I was prepared to pay for Express Shipping? I tried to change it to default shipping (as they often upsell) but I couldn't.

.most odd.

Is Amazon back up to its old tricks?

Useful Links:            Comments [2]  Trackback

Tags: Business | General