Preferred Domain and URL Redirect

Last time I set up Custom Domain and HTTPS for my Azure web app, there remains an issue - my website can be accessed from both the root domain fanray.com and the www.fanray.com subdomain.  This is bad for SEO, we need to tell search engine which one we prefer, hence we have to decide on a Preferred Domain either www or non-www. The one you choose will be the one that will be used to index your site's pages and be used for your site in the search results.

So, www or non-www?

This is like a religious debate and there are numerous resources out there, to list a few

• http://www.yes-www.org/why-use-www/ (interestingly there used to be http://no-www.org, this domain currently is for sale)
• https://www.youtube.com/results?search_query=www+or+non-www
• https://webmasters.stackexchange.com/search?q=www+or+non-www

Luckily, Google does not care and you can just pick one and stick to it. I chose www because of its ability to restrict cookies and it’s more flexible with DNS.

Set up Preferred Domain on Google

To set up preferred domain, go to Google Search Console and add a website property for each of the URL variations that your site supports, including https, http, www, and non-www.  You will go through a verification process to prove you own the site; I chose to add a TXT record at my registrar.  And you will receive an email titled “Preferred domain changed for site …” for each property you set up.

You set the preferred domain by going to the Gear icon > Site Settings

One common question is why Google only gives you the option to set up the http version but not the https? I found this question How to set preferred domain with https in Google Webmaster Tools and according to one of its answers,

Google takes this automatically from your canonical link tag.

<link rel="canonical" href="https://example.com/">

So whenever the Google spider sees this line in your head section, Google automatically indexes the HTTPS version of your site.

If you go to any one of the post page on this blog and view source you will see for example, something like the following tag with https.

<link rel="canonical" href="https://www.fanray.com/post/2017/11/26/custom-domain-and-https-for-azure-web-app" />

Note the canonical link does not appear on the blog’s main page but in the individual post, I visited other sites like StackOverflow, Techcrunch and this seems to be a common practice.

URL Redirect on Azure

Now I told Google what my preferred domain is, I still need to make the redirect actually happen for requests going to the less preferred domain over to the preferred domain. Two of the most commonly asked redirects for websites are

• HTTP to HTTPS
• Non-www to www, or www to non-www

There are many solutions to get both done, you can even do domain forwarding from your registrars, however those are not reliable and thus not recommended.

In the last post I took care of HTTP to HTTPS redirect by turning on HTTPS Only inside Azure portal, this is the easiest way to achieve this in Azure, but there is also an Azure extension someone wrote that can do it.

For the preferred domain, if you decided to go from www to non-www, there is also an Azure extension that does it. But currently I didn’t see an extension that goes the other way around from non-www to www.

URL Redirect in ASP.NET Core

To do URL Rewrite in ASP.NET Core, the common way is to use the RewriteMiddleware class, it’s part of the ASP.NET Core BasicMiddleware project.

To use this middleware, wire it up inside your Startup.cs Configure() method, and typically have the regex matching rules in a separate config file.

app.UseRewriter(new RewriteOptions()
     .AddIISUrlRewrite(env.ContentRootFileProvider, "urlRewrite.config"));

For example to redirect http to https in the urlRewrite.config

<?xml version="1.0" encoding="utf-8"?>
< rewrite>
   <rules>
     <rule name="Redirect to https">
       <match url="(.*)" />
       <conditions>
         <add input="{HTTPS}" pattern="Off" />
         <add input="{HTTP_HOST}" negate="true" pattern="localhost" />
       </conditions>
       <action type="Redirect" url="https://{HTTP_HOST}/{R:1}" />
     </rule>
   </rules>
< /rewrite>

URL Redirect with HttpWwwRewriteMiddleware

Above I explained some of the options to do URL Rewrite on Azure and in ASP.NET Core; and the RewriteMiddleware is actually quite powerful and can do very complex URL redirect rules.

However I intended for Fanray to run anywhere .NET Core can run, not just on Azure; furthermore, I wanted the easiest configuration experience possible to get these two common scenarios done for users.  Therefore I’ve written a middleware called HttpWwwRewriteMiddleware that does only two things, redirect

• HTTP to HTTPS
• Non-www to www, or www to non-www

To use this middleware, in your Startup.cs Configure() method add this line of code

app.UseHttpWwwRewrite();

Then in appsettings.Production.json there are two settings

"AppSettings": {
   // The preferred domain to use: "auto" (default), "www" or "nonwww".
   // - "auto" will use whatever the url is given, will not do forward
   // - "www" will forward root domain to www subdomain, e.g. fanray.com -> www.fanray.com
   // - "nonwww" will forward www subdomain to root domain, e.g. www.fanray.com -> fanray.com
   "PreferredDomain": "www",  // Whether to use https: false (default) or true.
   // - false, will not forward http to https
   // - true, will forward http to https
   "UseHttps": true
 }

When you deploy to Azure, if you would like the non-www version, just simply update “PreferredDomain” to “nonwww”, all requests to www subdomain will then redirect to your root domain.

Summary

At the end of this post, I have deployed Fanray to Azure App Service, got my Custom Domain and HTTPS working, and now all requests can redirect to the Preferred Domain I chose.