Hosting Jekyll on Azure Web Sites

If you are reading this post, your DNS has successfully updated and you are viewing carpeaqua now hosted on Microsoft’s Azure Web Sites service. As I wrote in Hurdles, the process of publishing this site is a 7-step process using Jekyll. I am in the process of trying to simplify that process, and one of the first things was moving to a new host.

Why Azure? Well, for one I’m a fan of the platform. Glassboard uses it extensively for managing its API, web app, and data.

Second, it allows me to remove a step from my 7-step deployment story. Previously to publish this site, I would have to run a Rake command that would rsync my site’s content to the host. With Azure Web Sites, I’ve connected carpeaqua’s public GitHub repository to Azure and it automatically listens for a new commit to master. When it detects that new commit (it takes just a few seconds), it automatically updates and deploys the updated site.

Configuring Jekyll for Azure Web Sites

Azure Web Sites runs on Microsoft’s IIS web server rather than Apache. As an end-user you likely won’t notice, but if you’re using Jekyll or some other platform that is traditionally hosted on Apache or some other open-source web server, it takes a few extra steps to get everything configured.

Note: I won’t run you through the process of creating an Azure Web Site. There are plenty of tutorials on Microsoft’s site and The Google to do that.

Commit Your Published Site To Git

First you want to ensure that you’re saving a copy of your generated web site in your GitHub repository. Previously I wasn’t since I was just rsyncing from my local Mac to the web. I have my site broken down now by _source for the Markdown files and files I edit and website for the generated site.

You can see this on the repository.

Create A .deployment file

To tell IIS which page directory you want to server, you’ll need to create a .deployment file in the root of your GitHub repository. The contents are fairly basic. Here’s mine:

[config]
project = website

Create a web.config file

In your _source directory, you’ll want to create a web.config file. This is IIS’s equivalent to the .htaccess file on Apache. It has a different syntax that is XML based and takes some getting used to.

My web.config file does two things:

  1. Points to a custom 404 page at the root of my websites folder.
  2. Redirects all requests for www.carpeaqua.com to carpeaqua.com (no www).

Technically, you could consider this an optional step, but I think the no-www looks cleaner. I wouldn’t skip it personally.

Pushing Changes To Azure Web Sites

Now that everything is set up, you should be able to generate new posts and share them with your readers? How? After you finish writing just open Terminal and run git push origin master just like you would any GitHub repository.

Deployments

That’s it. That’s the list.

I’ve now gone from a 7-step publishing process to a 6-step publishing process. I’m not really changing the world here, but it is nice to not have to open 1Password to get my old host’s crazy long SFTP password to publish a new article. I can now spend those extra few seconds refreshing Twitter again.

Jared Sinclair on Glassboard

Jared Sinclair of Riposte and Unread fame is doing app design reviews each Friday. I happily offered up Glassboard and told him to be merciless in his critiques. He obliged.

Jared’s big points:

  1. The hamburger in a basement is pretty cluttered.
  2. Separation between comments and top-level messages are hard to differentiate.
  3. The app has no unique design language.

I pretty much agree with all of these, and have had them on my to-fix list since the acquisition went through. I’m knee deep in working on #NewGlassboard now. Hopefully we can fix a few of these when that ships.

Moonshots

tl;dr — Photos+ has a new home at Silver Pine Software. Elements has been discontinued due to lack of sales and so I can focus on Glassboard.

When I decided to take over Glassboard, I knew I was at the time biting off more than I could chew as a sole developer. Balancing a product that has three front-end clients, an API server, and tens of thousands of users while also working almost full-time as an iOS and Android contractor isn’t easy. Add on top of that, my existing suite of products and it’s a bit insane.

One of the things I said I would do when I decided to take on this adventure was to simplify and focus my work. I’ve always had a bit of ADD, which is why my suite of apps has ranged from text editors to GitHub clients to photo galleries.

In late 2013, none of these apps were necessarily setting the world on fire financially, but they had users that I wanted to try and do right by. I have spent the past several months working on finding new homes for each of them.

Committed

We announced that The Magical Panda acquired Committed a few months ago. Since the acquisition, Saul and friends have been hard at work squashing bugs and adding new features such as 2-factor authentication support for a more secure GitHub login experience.

Photos+

As of today, Silver Pine Software is the new home for Photos+, the pro’s photo browsing app I released back in December of last year. Photos+ is an app I always wanted personally. I had a pretty large roadmap for where I wanted to take Photos+ in terms of functionality, but being able to realize that feature set and work on Glassboard simultaneously is nearly impossible.

I handed Silver Pine my roadmap as a source of feedback on where I think they should take the product, but it’s totally theirs to do as they see fit. I am happy, however, that their 1.1 release includes support for browsing your Dropbox photo library: by far the biggest thing I wanted to add from day one.

Elements

I was unable to secure a buyer for Elements unfortunately, so it has been discontinued. Sales of Elements have not been more than lunch money and enough to cover paying my support minions for almost a year.

As much as I enjoyed working on it, the market for a paid Dropbox text editor has passed. It’s become too saturated and too mature to reasonably think you can make a decent living from it.

To the Elements fans still using the app, my apologies, but I’m a firm believer in not keeping an app around to rot over time. The good news is that there are dozens of great competitors out there you can choose from.

Focusing On Glassboard

This is it. I’m all in on Glassboard. I don’t have any other products to fall back on or bring in supplemental income.

Google is big on talking about taking moonshots with their self-driving cars, Glass, and terrifying life like robots. These are big ideas, but I wouldn’t say they are moonshots because the company is still almost entirely funded by their insane ad revenue. If self-driving cars or evil robots don’t pan out, they’re going to be fine.

I’m taking a moonshot. Glassboard is a giant undertaking and the biggest risk of my career. It’s a make or break decision that will likely shape the next decade of my career.

I’m working full-time right now to try and launch the first piece of my Glassboard overhaul this summer. At that point, I’ll start to have a pretty good idea whether this was a good idea, or whether I’m going to be heading to the local McDonalds looking for a job.

Wish me luck.

Refactoring in the Cloud - Notifications

One of the things I have been thinking about a lot recently is the architecture of Glassboard. Now that I have everything mapped out for where I want to take Glassboard’s direction going forward both financially and functionally, it’s the time to look at how to actually build that functionality into the existing code base.

The Current Glassboard Architecture

Glassboard’s backend is powered by Microsoft Azure. In terms of computing, there are three main components: two worker roles and one web role.

Azure defines a Web Role as a server that has a running copy of the IIS web server on it, so that users can connect to it. A worker role doesn’t run IIS, but is still a Windows Server instance that you can use to do things like background processing on it and offload some of the heavy activities from your main server role.

This is a rudimentary diagram of the Glassboard architecture as of April 2014.

Glassboard's Architecture

  • The Kitchen Sink Web Role: I currently deploy a single web role that autoscales between 2 and 4 instances based on user traffic. This single web role does a lot. It handles all user authentication and signups as well as managing and distributing the Glassboard JSON API (through WCF god help me). In addition, it also has a few web hooks connected to it, and the Glassboard web app which is a rudimentary ASP.net MVC application with a Backbone.js exterior. It’s a beast.
  • Attachment Processor Role: When you upload a photo or video to Glassboard, it’s thrown into this role to do some post-processing like rotating it to the proper orientation and compressing to save us some server side bandwidth. Right now it’s using ffmpeg, Handbrake and a bunch of scripts to accomplish this. It’s more brittle than Greg Oden’s knees. We’ll get to refactoring it eventually.
  • Notifications Role: The Notifications Role handles distributing all the push notifications and emails that are sent out through Glassboard. Currently it supports Apple’s Push Notification Service, Google Cloud Messaging, Google’s legacy Cloud to Device Messaging, and Microsoft’s legacy Windows Phone Notification Service. Email is distributed via Sendgrid.

Everything in the current Glassboard architecture is housed in a single Visual Studio solution with about 36 C# projects in it. It’s a lot of code to manage. Luckily it’s also well tested, which makes it easier to manage.

Current Notification Role Issues

There are a few technical and ideological issues I have with the current notification architecture.

To send messages between the web role and the attachment processor and notifications roles, I use Azure’s Storage Queue mechanism to pass messages. When you post a message to one of your boards, the web role generates a new XML payload in one of our storage queues that the Notification Role listens for to handle the processing.

Sometimes this queue and the processing gets extremely backed up. Usually the CPU usage spikes to around 90% or higher for long periods of times which can lead to message postings being delayed for several minutes (in some cases I’ve seen hours) at a time. This is no good, especially when building a messaging service.

I’m not sure what the cause is for the spiking CPU other than a lot of delicate C# threading issues which I’m guessing are running amok.

The other issue I have with the notification role is how much of the code for things push notification services is in-house built. At the time Glassboard was originally built a few years ago, there weren’t too many open source solutions for handling push in C# I’d imagine.

I’m fairly conservative when it comes to third-party dependencies, but for things like push notification services there’s not much sense in maintaining your own code library to connect to the service when there are a few good solutions out there used by hundreds (thousands?) of startups and businesses around the world. They’re well implemented, battle tested, and code that is maintained by the community.

Being on C#, that limits the amount of options I have for dealing with push providers. PushSharp seems to be the main provider out there.

The New Notifications Role

Given that I’m not a fan of the C# threading model currently implemented, the amount of code used to power something that’s relatively simple, and the homegrown nature of many of the push wrappers we’re using, I made the decision to rewrite this portion of Glassboard’s architecture using Node. Hang on. Put those Buzzword Bingo cards away. I’m not a fan of rewriting for the sake of rewriting either.

Ultimately, my decision to switch to Node is all about code management. I’m confident I can accomplish the goals of my notification roles with a lot less code using Node than C# thanks to it being a bit less verbose of a language and a variety of NPM packages I can take advantage of for handling APNS, SendGrid, and GCM. The less code I have to manage and maintain, the easier my job is as the sole developer of Glassboard.

I don’t believe I will have any performance degradation in switching from C# to Node for this portion of the code either, though it’s hard to tell before actually writing the code. If anything, I’m optimistic that performance will improve.

A Fork In The Road

There’s two different ways I can approach handling push notifications being distributed through my new worker role. The first is to use these npm packages like node-gcm and its similar ilk to distribute the messages myself like I have always done. The difference being mostly in the language being used (Javascript versus C#).

The other option is to take advantage of Azure’s Notification Hubs which will handle distributing the messages to devices for me. Notification Hub has some advantages that stand out to me:

  • It works with all the platforms I care about: APNS, GCM, and WNS.
  • It handles platforms I don’t care about, but might again someday: Kindle.
  • It does all the device feedback management for me so I don’t have to keep track of which devices I should be adding or removing (such as when a user installs or uninstalls Glassboard).
  • Tagging devices based on certain features seems appealing. For instance, I could tag all of a user’s devices by their UUID and then easily distribute a message to all their devices in one message.

This all sounds great. Why not just do this? Cost.

Notification Hubs are another cost layer on top of the existing charges I’m paying Microsoft for my cloud instances and storage. I’m still in ‘losing money’ mode with Glassboard, so I’m trying to be fairly conservative with my spending.

Glassboard currently sends around 150,000 emails a day according to Sendgrid, but I have no idea how many push messages it sends because it has never had a way to track that sort of information. If I opt into Notifications Hub, I’m likely going to be out at least $200 extra a month, but that could also sky rocket to an even higher amount.

Now you understand why I am giving so much important to data and analytics these days.

My gut says use Notification Hubs. My wallet says roll my own.

Service Bus vs Storage Queues

One last thing I’m still debating is whether to keep using Azure’s Storage Queues for managing the messaging between the different components in my cloud stack or to use the new-to-me Service Bus functionality, which looks to offer a few more features. The Notification Hub functionality Microsoft offers is powered by the Service Bus, which has me considering the switch.

There are plenty of decent articles out there comparing Service Bus vs Storage Queues, but from a more abstracted level that leaves me still questioning which way to go. The conservative developer in me says keep using what I’ve already got because no major Service Bus feature stands out to me as a must-have right now. The liberal developer in me sees Service Bus as the more modern architecture that will likely be able to grow with me as Glassboard (hopefully) expands in years to come.

Feedback certainly welcomed.

Writing The Code

Even though the core architecture of two worker roles and a single web role won’t change with this refactoring of Glassboard’s notification architecture, hopefully this has been at least a little eye opening in terms of how Glassboard is built, and how I envision it being built going forward.

The obvious next step is to write the code, which I started earlier this week by writing the unit and integration tests for the new Node code base.

Since I spend so much time in Visual Studio these days, I thought I’d give the Node tools that Microsoft released a try. Then I remembered that while I love what Microsoft is doing with the cloud and Azure, Windows is still Windows.

A More Responsive carpeaqua

Just a short note to point out that carpeaqua’s design has been refreshed to finally support a responsive layout that should render much nicer on your phones and tablets. It’s something I’ve known I needed to do for a while, but never seemed to get around to.

Thanks to Michael Tofias for being annoyed enough at this to agree to take on the challenge. He is a scholar and a gentleman.