Fanray v2.0-a4 released!

I released Fanray v2.0-a4, building on v2.0-a3 released from two weeks ago. Now you can drag and drop images directly in Composer and they will be uploaded and inserted in the editor.

dnd images
Drag and drop images to editor directly!

Currently you can only upload image files of type .jpg, .jpeg, .png and .gif, and the max file size allowed is 5MB. Error handling has been re-worked to support client side checking on these file constraints first and only sending valid files to the server side.  Server side validation remains of course.

Upload file error checking
Upload file error checking

All images uploaded are resized to different sizes, small, medium, large and original, the only exception is .gif files, only originals are saved and no resizes.  This is due to a remaining technical issue I yet to be able to solve with Magick.NET, when resizing gif files it takes a long time and the resized file size is larger than the original file. I put in a temporary fix till I figure out how to resize animated gif with Magick.NET or another library. 

Uploading files by clicking the UPLOAD button has been re-worked for Microsoft Edge.  Edge does not work with file input onchange event, so I implemented it with addEventListener instead.  I also added file input tag in the html markup instead of dynamically creating it in js.

Finally I improved on slug generation to make duplicates unique work better.  Posts and files uploaded are siloed with time, for a post you have year/month/day as part of its slug and for files you have year/month in its path.  So the chance of file name and post name collisions should be small.  But still it could happen, for example, you uploaded a file earlier this month and later you upload another file with the same name, what do you expect should happen?  Rest assured, I don't replace the existing file, instead I append a number to the file name, like my-file-2.png, to resolve the conflict. This number starts at 2, so if you upload another one you'll get my-file-3.png and so on.

Fanray v2.0-a3 released!

I released Fanray v2.0-a3, building on v2.0-a2 released from two weeks ago. Thanks to CKEditor5's latest release, now we can embed in our posts YouTube and Vimeo videos.

Simply copy a YouTube URL and paste it into your editor.  You will see a placeholder of the video show up in your editor.  This placeholder image is not playable directly inside the editor, however you can play the video when you preview the post.

What’s inside the editor is an oEmbed tag.  From the oEmbed's website

oEmbed is a format for allowing an embedded representation of a URL on third party sites. The simple API allows a website to display embedded content (such as photos or videos) when a user posts a link to that resource, without having to parse the resource directly.

To show the actual video, what I did was I parsed the oembed tag and swapped it out with the YouTube’s own embed code. And to keep it simple I wrote the parser on the server side, take a look at the OembedParser.cs for more details.

Secondly, YouTube has a few different URL formats, most commonly are the following three. 

  • youtu.be/ 
  • youtube.com/watch?v= 
  • youtube.com/embed/

I did run into one edge case on an URL with the v param that did not start in the normal order, like this

https://www.youtube.com/watch?time_continue=75&v=MNor4dYXa6U

In this one case the editor didn’t like it, but for vast majority of the scenarios you are covered.  

I wrote some unit tests for my OembedParser that shows you the parsing of the different URL formats, you can check out OembedParserTest.cs for more info. 

Also if you take a look at the CKEditor’s documentation, you will see that it supports a variety of media embeds like a tweet, google map etc., but for this release I only support YouTube and Vimeo videos and will support others down the road. 

For other things I did for this release check out the milestone.  Thank you!
 

Implementing the Mediator Pattern with MediatR

Thus far I have a single service in Fan.Blog that handles everything, it's called BlogService.  As my code grows this becomes less maintainable, with each new feature I want to add to the blog I have to pile on top of this class, not to mention it's less than ideal if there are others working on the same project and everyone modifies one file.  So I wanted to break this giant service class into smaller more focused services.

In a YouTube video I want to show you how I refactored my BlogService with the Mediator Pattern using the MediatR library, please check it out.

Fanray v2.0-a2 released!

Today I release Fanray v2.0-a2, building on v2.0-a1 released from last week.  This release provides the ability to preview a post (issue #208) and other fixes.

Preview post
Preview your post

The preview will show you exactly how your post will look before you publish it, and it is fast and convenient.  It happens right inside the composer, when you click on the Preview button a dialog pops up full screen to show you the post as if it's published.  The URL of the post is printed on top of the preview window.  To exit you can press Esc or hit Close button.

One thing I want to point out, by design I want each Fanray theme to provide a content.css.  This file is meant for the editor to consume, it includes things like the typography and the content width styles of your particular theme. It enables the users to see a similar styling to the final published post while they are still editing.

But the content.css does not make your content 100% resemble the eventual published post.  This is due to some of the limitations of the editor, for example it currently does not support block source code which is a known issue and will be addressed in the future.  In the meantime, I have come up with a shortcode for you to input block source code into your post.

The ability to preview a post before publish is a much needed feature for the increased usability of the blog composer.  It'd be even nicer if it could have a keyboard shortcut assigned to the preview to help user get into the preview quickly.  Also if you are writing a long post it'd be helpful if there is an option to sync the preview and the editor so you don't always start from the top and have to scroll down.  These nice-to-have additions I don't have time to pursue right now, if anyone in the community is interested please feel free to improve on it.

I hope you find this feature useful :)

Using Vuex

I implemented Vuex in my project this week, I'm new to Vuex and this is what I learned.  In a Vue app when you need communication between two components, you have the following choices

One recommendation I read from the Vue Forum when someone asked about Bus vs Vuex is 

My advice would be as follows: use vuex for everything as a default, and drop back to an event bus when vuex becomes a hurdle.

The basic idea is your Vue app is a tree of components and the tree could be N layers deep, and to pass data between any two components in the tree becomes a challenge.  Events when used here and there is OK but in a large app if you add a lot of them your code becomes cumbersome.  So what the app need is a global object store to contain shared data accessible to all components and that is what Vuex is.

Here I have an issue #252, it describes that when you insert some selected images from the gallery dialog into your post and then you re-open the gallery two things should happen 1. the previously selected images should be de-selected and 2. the Edit, Delete and Insert buttons which only show up when there are selected images should be hidden, neither is happening. Not only that both should happen when the dialog is closed in most cases.

Composer with gallery open
Composer with gallery open

There are 4 ways to close the dialog

  1. Click on Insert, this should insert and de-select the images and close it
  2. Click on Close, this should de-select images and close it
  3. Press "Esc" key on keyboard, this should de-select images and close it
  4. Click anywhere outside the gallery, this should close dialog but without de-selecting the images. I find this scenario often happens by mistake not by intention.

The gallery is the child component, it contains the toolbar of buttons and a list of images.  When you select images they are stored in an array named selectedImages on this child component.  The gallery is then contained inside a dialog component which is part of the parent composer. Whether the selectedImages is empty is what controls the visibility of the three buttons on the toolbar.  When I click on Insert I pass the selected images to a method of the parent which then writes out html to the editor.  

When I click on insert I actually could easily de-select the images and empty out selectedImages because the Insert button is part of the gallery component, same with clicking the Close button.  When I press Esc on my keyboard to close the dialog though it is not as simple because that is part of the parent.  In other words the parent needs to access selectedImages on the child. 

I feel this is a great example of when you could use Vuex.

let store = new Vuex.Store({ 
   strict: true, 
   state: { 
       selectedImages: [], 
   }, 
   mutations: { 
       setSelectedImages(state, newSelectedImages) { 
           state.selectedImages = newSelectedImages; 
       }, 
       addSelectedImage(state, image) { 
           state.selectedImages.push(image); 
       }, 
       removeSelectedImage(state, idx) { 
           state.selectedImages.splice(idx, 1); 
       }, 
   }, 
   actions: { 
       selectImage: function ({ commit }, image) { 
           commit('addSelectedImage', image); 
       }, 
       deselectImage: function ({ commit }, idx) { 
           commit('removeSelectedImage', idx); 
       }, 
       emptySelectedImages({ commit, state }) { 
           commit('setSelectedImages', []); 
       }, 
   }, 
});

Then the selectedImages array breaks loose and accessible globally, thus the parent can take a hold of it and empty it when necessary

this.$store.dispatch('emptySelectedImages'); 

Fanray v2.0-a1 released!

Fanray v2.0-a1 has been released!  Fanray v1.0 was a minimal viable blog, you could only post through a client like the Open Live Writer. Fanray v2 builds on v1 and introduces a brand new Admin Panel you can now easily manage your blog's posts, tags, images etc.  Here are a few highlights and tips to get you started and please check out the Wiki for more details.

Get Started

First to get up and running is very easy, just follow the Quick Start steps.  Once you launch the application you will see a revamped setup page to help create your blog.

Fanray-blog-setup
Fanray blog setup

Site Settings

Once you created your blog, I recommend complete the setup by going to the Settings page and enter your Disqus and Google Analytics information.

Admin-Settings
Admin Settings

The Composer

Bloggers probably spend most of their time writing posts. To help you be productive, I highly recommend spending a few minutes to get familiar with the Composer, particularly the Editor and experiment how to input different content efficiently.

Here is a simple challenge for you to get started, open up the Composer and try input the following

  • Paste some html content without styles
  • Add heading, list, bold text with Markdown syntax
  • Add link without clicking the toolbar, then try move your cursor outside the link to continue typing regular text
Composer
The Composer

Media Gallery

The Media Gallery gives you a grand view of all your blog images.  Here you can upload more images or edit image info etc. The uploaded images are resized and stored on either the file system or Azure Blob Storage, you can configure which storage in the appsettings.json.

Admin-Media-Gallery
Media Gallery

Categories and Tags

The Categories and Tags pages allow you to easily manage your blog's classifications.  For categories there is a default category out of box named Uncategorized, go rename it to something you write about. 

Categories
Categories

Deploy

When you are ready to run this app on Azure, please refer to Deploying to Azure.

Upgrade

If you are running v1.0, the upgrade is very simple.  You can just run v2.0, the EF Core Migrations will upgrade your database for you automatically.  If you prefer to know what changes are being made to your database or prefer to upgrade manually, a .sql script is provided for you.  The file is 1.0-2.0.sql located in the solution's sql folder.

Contribute

Fanray is in its early stages and requires support to move ahead. You can contribute in many ways - ideas, bugs, testing and docs etc.  Follow me @fanraymedia and let me know what you think.

Thank you and happy coding :)

Fix slow login response and verify email using regex

This week I got my first PR thanks to Flyznex, he helped fix issue #234 Login has a very slow response and along implemented a better way to verify if a string is a valid email. 

The issue came up when I found that sometime when I login it took a long time.  I'd put my email/username and password in and click on the login button and nothing would happen, and after a while like a few seconds then it let me in.

PasswordSignInAsync

As Flyznex pointed in the PR, the SignInManager<TUser> class in Asp.net Core Identity has two PasswordSignInAsync methods, one that takes in a string for userName while the other takes in a user object.  If you call the one that takes in the userName, the method will call UserManager.FindByNameAsync(userName); to look up the user first by its userName, hence you make an extra call to the database.

In my Login method I already looked the user up with its userName by calling my UserService method FindByEmailOrUsernameAsync(loginUser.UserName); so I could just pass the user object to PasswordSignInAsync.  My mistake was that I passed in user.UserName instead thus incurred the extra db call.

Verify if a string is valid email

On my login I allow user to use either either user name or email to login.  When a user is logging in I first check if what's inputted is a valid email or not.  If the string is not a valid email then the user is passing in his user name.

So checking if a string is email sounds easy but in reality it is very complex!  If you don't believe me just take a look at Phil Haack's post I Knew How To Validate An Email Address Until I Read The RFC.  Luckily there is a solution provided to us in the .NET documentation How to: Verify that Strings Are in Valid Email Format.  And when you take a look at that implementation it's like 60 lines of C# code just to do this verification, for good reasons.

Before this PR I was using System.Net.Mail.MailAddress class constructor to verify the validity of an email

// bad code, do not use! 
bool isEmail;              
try  
{  
    new MailAddress(emailOrUsername);  
    isEmail = true;  
}  
catch (FormatException)  
{  
    isEmail = false;  
} 

I originally thought good, less than 10 lines of code to get the same thing done.  But this implementation is actually not reliable.  Try pass in this string a@a as input which is not a valid email, it actually returns true!

Another thing to notice is that verifying email feels like a utility feature, as the name of the class in the .NET doc also suggests RegexUtilities.  For that normally we want to use a static method, but this implementation depends on a local field bool invalid = false; that is shared between the two methods inside the class.  But Flyznex was able to fix that by removing this local variable, as a result our implementation now is really simple to use, you can just call Util.IsValidEmail("email.to.verify@test.com").

Version scheme and Roadmap

It's been a year since v1.0 was released, to be able to release more often I need to put in more effort to issue management and planning.  As part of that I want to have a Roadmap that highlights roughly what's coming in the foreseeable future, but first I need a versioning scheme that could work for me.

I came across this Wikipedia page on Software Versioning and found that people use all kinds of schemes to version their releases, it can be anything really.

So I decided to adopt a semver major.minor.patch-(a|b|rc)[1-9] kinda scheme starting v2.0. To get releases to users faster, a release is planned every milestone. Milestones can give or take be weekly or bi-weekly (or a long time if I get stuck :) 

This table shows an example of this version scheme for the next full release cycle. 

Life CycleVersion ExampleDescription
alphav2.0-a1Active development stage. Each alpha release contains one or more new features or fixes. During alpha breaking changes like DB schema updates are common.
betav2.0-b1Planned features are complete. Each beta release contains fixes only, no breaking changes.
rcv2.0-rc1A rc is a beta that is potentially production ready. Each rc release contains fixes only, no breaking changes.
rtmv2.0An official release.
patchv2.0.1Bug fixes after an official release. A patch can go through alpha, beta and rc as well.
minorv2.1Next minor release. About 20 milestones away from v2.0.
majorv3.0Next major release. It should bring major value to the product, not time bound.

Based on that I came up with a Roadmap

  • v1.0 - Minimal Viable Product (Oct 2017)
  • v2.0 - Admin Panel (Dec 2018)
    • v2.0-a1 - initial v2 alpha
    • v2.0-a2 - feat: preview post in composer
    • v2.0-a3 - feat: dnd images in composer
    • v2.0-a4 - feat: social links
    • v2.0-b1 - fixes
    • v2.0-b2 - fixes
    • v2.0-rc - fixes
    • v2.0-rtm
  • v2.1 - Featured image and a second theme
  • v2.2 - Navigation
  • v2.3 - Pages
  • v2.4 - User roles

Finally I'm trying these out for now to see how it goes, as a result the time frame and planned features can change.

"Swap with preview" should be used when upgrade a production database

I wrote once previously about how I used EF Core migrations to upgrade production database. I have two slots production and staging and I swap them every time I push new code that don't involve database changes.  This week I had to make a data change to production database, it's a reminder of when to use the Swap with preview feature provided by Azure App Services.  

I recently decided to change versioning scheme a bit in hope to turn out releases more often, so the upcoming release originally planned as v1.1 has now become v2.0.  As a result, I opened up issue #235 EF Migration and SQL upgrade files got wrong version numbers to fix the out dated text in these files.

Migration changes
Migration code changes

In addition I need to update a record in the __EFMigrationsHistory table, specifically it’s the second row’s MigrationId needs to become "20180530163323_FanV2_0".

__EFMigrationsHistory table
__EFMigrationsHistory table needs data update

To update this record I can run a simple SQL update statement while the site still running, because EF only checks this table every time the application starts up to see if there are new migrations need to be applied.  The challenge is I don’t want a request hit the database before the new code gets there, because the existing code looks for the "20180530163323_FanV1_1" record and if it’s not there it’ll apply migration thus causing problems.

When using Swap with preview, these are the steps to take place:

1. First get staging database and server ready with new data and code (since it’s staging I can just stop the server and do whatever so that’s easy).

2. Update production database with the data change (the production site is still running).

3. Start the swap with “Swap with preview”, this will apply all production configurations to staging slot while still keep production running. It seems you can do this in either slot now, I somehow remember the option was only available in production slot previously.

4. Azure then restarts staging using these production configurations, now staging runs against production database which is OK since staging has the new code.

5. Finally do “Complete swap”, Azure moves the already warmed up staging slot into production.

In conclusion I could do this because it was only a data change.  Consider if there were schema changes then you may not be able to update the production database first while production service is still running, in that case I'd refer to my previous post.  

Structured Logging with Serilog and Seq

Fanray uses Serilog and practices Structured Logging.  One of the output sources is Seq.

Setup

The wiring of Serilog happens inside Program.cs

  public static IWebHostBuilder CreateWebHostBuilder(string[] args) => 
     WebHost.CreateDefaultBuilder(args) 
       .UseApplicationInsights() 
       .UseSerilog() 
       .UseStartup<Startup>(); 

And the configuration of it with Seq happens in appsettings.json and appsettings.Production.json. Below is the snippet from appsettings.json with MinimumLevel set to Deubg for local development.

  "Serilog": { 
   "Using": [ 
     "Serilog.Sinks.Console", 
     "Serilog.Sinks.Seq" 
   ], 
   "WriteTo": [ 
     { 
       "Name": "Console" 
     }, 
     { 
       "Name": "Seq", 
       "Args": { 
         "serverUrl": "http://localhost:5341" 
       } 
     } 
   ], 
   "MinimumLevel": { 
     "Default": "Debug", 
     "Override": { 
       "Microsoft": "Information", 
       "System": "Information" 
     } 
   } 
 } 

Structured Logging

Structured Logging enables you to dump entire objects as JSON into the logs and you can later search for them with their properties as search criteria.

For example I have this code in BlogService.cs, it logs a BlogPost object.  Notice the special syntax {@BlogPost}

_logger.LogDebug("Show {@BlogPost}", blogPost);

That outputs the following and notice it also outputs the nested objects inside BlogPost like Category, Tags etc.

{ 
   Type: "BlogPost", 
   CategoryTitle: "Technology", 
   Tags: […], 
   TagTitles: […], 
   Body: "<p>This is a test post.</p>", 
   BodyMark: null, 
   Category: {…}, 
   CategoryId: 2, 
   CommentCount: 0, 
   CommentStatus: "NoComments", 
   CreatedOn: "2018-09-22T02:23:40.0000000+00:00", 
   CreatedOnDisplay: "yesterday", 
   UpdatedOnDisplay: "09/21/2018", 
   Excerpt: "This is a test post.", 
   ParentId: null, 
   RootId: null, 
   Slug: "this-is-a-test", 
   Status: "Draft", 
   Title: "This is a test", 
   UpdatedOn: "2018-09-22T02:23:41.0002676+00:00", 
   User: {…}, 
   UserId: 1, 
   ViewCount: 0, 
   PostTags: […], 
   Id: 5, 
   _typeTag: "BlogPost" 
}

Seq

Seq is the tool that allows you to search through what you have logged taking advantage of Structured Logging.  And it is my recommended way to log during the development of Fanray.

To get started with Seq first download and install it from https://getseq.net/ and then to use it during development simply go to http://localhost:5341.

Search for Objects

To search for an object, say give me all BlogPost with a Body that contains the word "Welcome" case insensitive.

Seq filters BlogPost with Body that contains word Welcome
Seq filters BlogPost with Body that contains word Welcome

Here are docs on Seq's Filter Syntax and Cheet Sheet  https://docs.getseq.net/docs/query-syntax

Search for a Request

A request is common thing to search through the log, ASP.NET Core outputs detailed debug information during development. 

For example, when I publish a new post from Open Live Writer, I want to see what exactly happens during that request of publishing my post.  Since Asp.net Core provides these detailed info for each request thus you can search for it with its id "0HLH1ICD1KGGF:00000001".  The screen shot below shows the entirety of that request, the bottom rectangle marks the start of the request while the top rectangle marks the finish of it.  From this you are able to see things like all the SQLs EF Core executed, the MetaWeblogAPI newPost endpoint was called and the XML it received and other good stuff.

Seq shows a request from Open Live Writer
Seq shows a request from Open Live Writer
Older posts