Monthly Archives: December 2018

Spite is the mother of invention

December 28, 2018.NETRikard Ottosson

Premise

This is a tale about a blog on WordPress.com that had a loyal readership and regular, high quality content (so yeah, not writing about this blog). The owner wanted to use the odd plugin and advanced theme, and I was always bothered that WordPress was living off of, well I exaggerate wildly now, this person’s words by putting ads everywhere in addition to the massive annual fee.

Liberation

So with the lure of freedom on them yonder hills, we moved the blog off of WordPress.com onto a Windows VM on Azure (yeah, well… yeah…). Domain hosted on dnSimple, so the logistics of pointing the domain to Azure instead of WordPress and setting up verification TXT records and such was a doddle.

Hosted MySQL instance on Azure was easy enough, but the WordPress.COM theme we had been using was not available on WordPress.Org so we had to pick another one. Sadly we went with Customizr which really means vendor lock-in, as you do a bunch of customisations, hence the name, that are all out the window once you change themes.

Of course, there is no option but to run HTTPS today, and trying to pinch pennies we weren’t going to buy an EV cert from one of the remaining dodgy CAs out there, but iinstead we went with Let’s Encrypt using a tutorial posted by Scott Hanselman.

Selling out – but is anybody buying?

To make the big bucks we hooked the the site up to Google Analytics and ditto Adsense, and there were plugins to really automate that stuff. Yoast SEO beat out MonsterInsights on features for the analytics and integrates both with Search Console and Analytics. The killer feature for Yoast SEO is the customisable canonical URL which is useful if you reprint blog posts from another site and want to beg Google for mercy for the crime of duplicate content.

The actual ads, how do they work? Well by cunningly just clicking like an insane person (which really is the best way to learn), I managed to understand the concept of Auto Ads. This again is abstracted away by a plugin, in our case Advanced Ads. As the site owner didn’t want ads on all pages, we had to hack it by creating a plain text and code ad with the Auto Ad code from Google pasted in there and then the Advanced Ads thing deciding which pages to actually serve the ad code. The downside is a persistent nagging that you ‘shouldn’t display visible ads in headers’, but I guess that’s fine. They are just script tags, so there is nothing visible there..

Also, all the cool kids enter the Amazon Affiliate program, so we did that. They do have a minimum number of referrals you have to make as they don’t want to deal with tiny unprofitable sites, so I suspect we shall be unceremoniously booted out fairly soon, but the concept of having widgets where you choose your favourite books related to the subject of your blog and maybe in the long term share some revenue if people take you up on your recommendations seems fair. Shame that the widgets themselves are so immensely horribly broken and difficult to use. Allegedly, they are supposed to update when you make changes in the affiliate program site, but they really aren’t. I don’t get paid by Amazon so I shan’t debug their system, but it can really only be that the command that goes back to save settings isn’t picked up, or that they are unable to bust the cache and have old widgets served, but I strongly suspect it is the actual save that is broken, since the widget loses the data already in the wizard before you even enter the last page.

AMPed up

After a few hours I noticed that all the permalinks from the old site were broken on the new one, so I checked the Permalinks tab and it turned out there was a custom setting that I just set to default which made things work and there was much rejoicing. No audit log here so I can’t check, but if I made that change it must have been unintentional. My favourite hypothesis is that somehow the otherwise impressive WordPress XML-based import somehow failed to bring over the settings correctly.

As I rarely venture out into the front end I had not quite grasped what AMP is. I realised I was getting another load of 404s – his time for URL’s ending in /amp. I did a bit of googling and I realised I should probably get yet another plugin to handle this. Like with most WordPress plugins there are varying degrees of ambition and usually they want you to spend $200 in extras to get what you need, but although I brought the site off WordPress.com to deny them ad revenue for the site in question, I was under no illusion that I would be able to produce any such revenue to the owner as whatever $3 would be produced would definitely be eclipsed by the hosting cost.

By going with the default WordPress AMP plugin you can’t do ads, but it works – ish, by using the major competitor you get a functional site, but a completely different look compared to the non-AMP site, and we didn’t want that after all the effort we had already put in.

After reading some more, I realised that everybody was going off AMP anyway, for varying reasons, but that was all the peer pressure I needed, so I broke out the Azure debug console and edited web.config to put in a URL redirect from AMP URL to a normal one.

This was incredibly frustrating as first I forgot that .NET Regexes are different from normal regexes and also you have to not be stupid and use the correct match in the redirect expression ({R:0} is the whole source data, while {R:1} is the first match, which is what I needed).

<staticContent>
     <remove fileExtension=".woff2" />
     <mimeMap fileExtension=".woff2" mimeType="font/woff2" />
</staticContent>
<rewrite>
     <rules>
          <rule name="Disable AMP" stopProcessing="true">
              <match url="^(.)amp\/?\r?$" />
             <action type="Redirect"
                 url="https://<awesomesite>.com/{R:1}" 
                 redirectType="Found" />          
          </rule>          
          <rule name="Redirect to naked" stopProcessing="true">
              <match url="(.)" />
              <conditions>
                  <add input="{HTTP_HOST}" 
                    pattern="www.<awesomesite>.com" />
              </conditions>
              <action type="Redirect" 
                   url="https://<awesomesite>.com/{R:0}" 
              />
          </rule>
          <rule 
               name="WordPress: https://<awesomesite>.com"  
               patternSyntax="Wildcard">
             <match url="*"/>
             <conditions>
                 <add input="{REQUEST_FILENAME}" 
                      matchType="IsFile" negate="true"/>
                <add input="{REQUEST_FILENAME}"
                      matchType="IsDirectory" negate="true"/>
            </conditions>
            <action type="Rewrite" url="index.php"/>
         </rule>
     </rules>
 </rewrite>

So there are a couple of things here – first a mime type correction to make IIS server web fonts, thne a redirect for AMP sites, then a redirect from www.awesomesite.com to awesomesite.com for prettiness, and also to canonicalise it to avoid duplicate records in the offices of Google, which they do not like. WordPress itself will force https if necessary, so all we need to do in this config file is to curb the use of www.

Summary

The actions we took to move the blog were the following:

Set up the new site
1. Create site
2. Create blob storage
3. Create redis cache (I did this later, but you might as well)
4. Set up a database
Export existing data from old blog
Import data into new system
Choose a theme
Verify that old google links work on the new site (I didn’t do this fast enough)
Verify that any way you try and call the site is redirected to a canonical represenation. Use a hosts file if you haven’t redirected the DNS yet, which with hindsight should have been the way I did it.
Move the DNS to point to the new site
Add the LetsEncrypt support to the site by following the guide. No more certificate errors
Install plugins for analytics and ads.
Create a Google account
1. Register with Google Search Console
2. Register with Bing search console (for those two or three people that don’t know Google.
3. Register with Google Analytics
4. Register with Google AdSense

Conclusion

So this was very easy and horribly frustrating at once. DnSimple and provisioning resources was a doddle. Following the internet guide to set up Let’s Encrypt and HTTPS was super straightforward, but then WordPress plug-in management, PHP and Amazon widgets were shit shows to be honest. I mean I realise Amazon has a complex architecture and their systems are never 100% up or 100% down and so on, but a save button being completely broken doesn’t feel even slightly “up” from the point of view of the end user.

PHP is garbage and brittle and you are hard-pressed to build anything viable on top of it (but obviously some have succeeded). These plugin smiths aren’t Facebook though. They would correctly interject that I am running WordPress on the least suitable platform imaginable. That is true (it has to do with how the Azure VM instances are set up, on the fact that they run on Windows and most importantly NTFS which has performance characteristics that are completely unsuitable for Unix style applications and favour a small number of large files where EXT4 favours large amounts of small files), but if the Powers that Be really consider Windows and NTFS to be such tremendous deal-breakers, then they should simply not allow Microsoft to host WordPress on Windows at all. As it stands, it does WP no favours with 1 minute turnaround to save settings for a plugin and similar. Then again, I also live in the UK which notoriously has a Internet infrastructure dating back to the Victorian Era, so it’s hard to tell what’s actually the worst culprit, but the sidecar web app that hosts the debug console for the blog is a lot snappier than WordPress, and that is hosted in the same IIS intallation as the WordPress site, although not in the same app pool.

Structural Equality – or is it?

December 27, 2018.NET, F#Rikard Ottosson

I was recently presented with a conundrum. We had constrained data valid for the domain in a record type. Sadly this record type contained a reference datatype, so built-in structural equality broke down as the reference type never was equal in the way we thought would make sense.

This gave me the opportunity to learn how you override the implementation of Equals and GetHashCode in F# which I was previously unfamiliar with.

This is the finished implementation of the record type, or one like it, rather:

 [<CustomEquality>]
 [<CustomComparison>]
type Structure =
    { 
        Name: StructureName
        Status: StructureStatus
        Format: Regex
    }
    with 
        interface IComparable with
            member this.CompareTo { Name = name; Status = status; Format = format } =
        compare ( this.Name, this.Status, this.Format.ToString() ) (name, status, format.ToString())
        interface IComparable with
            member this.CompareTo obj =
                 match obj with 
                     | null                  -> 1
                     | :? Structure as other -> (this :> IComparable<_>).CompareTo other                
                     | _                     -> invalidArg "obj" "not a Structure"
    override this.Equals (o: obj) =         
        match o with              
            |  :? Structure as os ->                 
                (this.Name, this.Status, this.Format.ToString()) =
                    (os.Name, os.Status, os.Format.ToString())
            | _ -> false
    override this.GetHashCode() =
        (this.Name, this.Status, thus.Format.ToString()).GetHashCode()

So yeah, ujse of pattern matching to determine data types in the non-generic functions and extensive use the built-in structural equality in tuples.

Very nice. With thanks to TeaDrivenDev and Isaac Abraham on Twitter (and this StackOverflow response)

Bind? No.. Apply…? No… Map!?

December 17, 2018.NETF#Rikard Ottosson

After continuing my foray into the functional with Giraffe and .NET Core, I have struggled with the code not quite reading very well. Various data sources are pluggable, i.e. they can be replaced with mocks if you don’t want to run other supporting systems while debugging. This pluggability along with the way Giraffe handles some of its configuration means that – while we would like to compose partially applied functions with either mocked or actual services baked in that are then passed in to the relevant handlers, we instead have to pass a HttpContext around everywhere. If you followed that link you probably wondered why we did not just use the strategy outlined in that blog series. I can only say that we did not understand fish operators at this point.

So we turn to the googles, and as usual the first or second hit lands us on F# for fun and profit. Scott Wlaschin introduces us to the Reader Monad. We watch his talks, we read his blog posts. A week of enthusiastic coding goes by.

Essentially – the reader monad allows you to write and construct all the code and only at the end plug in the HttpContext as the whole thing is evaluated and returned to the caller.

You make a Reader<‘env, ‘data> and you create functions to wrap and unwrap things in and out of the Reader. Then you make functions that mind their own business but they return Reader<‘env, b: and you allow the reader to handle some railway oriented programming for you. Essentially you lay the track all through the code but on the last line in the handler you put the train on the tracks and set it off, only then realising whether or not it ended up running through the failure cases or went all through the success track.

Wrappers, essentially, like Option<‘a>, IEnumerable<‘a> or Reader as described above, are introduced as Elevated things. The proper word seems to be Monadic types, but that does not seem to be a good term, as there is no need for a wrapper type to be a monad, although they can be. After a few specific examples the general case is presented and it turns out you can do a bunch of stuff with these elevated types only using functions like bind, return, apply and map. There is a brief epiphany as we experience a flicker of understanding of the monad laws.

A monad is just a monoid in the category of endofunctors, what is the problem?

Now I cannot explain the chaining of elevated and non-elevated functions properly. I know what bind does, and I can use it correctly on the first try. Return is obvious. Map and apply though… it’s like being ten years old again and round-robining commands until the compiler is happy. As it stands all I can do is link to the posts we have read and the metaphors that have been presented to me, that I have half understood and then tried to reconcile with other conflicting metaphors.

Inevitably the golden success case of “first I need this, then I pass it to this other thing and then I flip it, kick it and reverse it and then I return it” is soon replaced by “well, first I need this one thing, that I need to just check against this other thing, but then use it again in this third thing which I convert to this fourth thing that I might return if this fifth thing is true” and the nice chaining goes out the window and it makes you sad. Every let b = … feels like a let-down (hence the name).

This let = thing is apparently called applicative style, where you compose values, while chaining is called monadic style. At least it has name.

So after a while when you have been writing things like

task {
    let! a = coolFunc b c
    return! a |> modifyInAnAwesomeWay
}

async {
   let! a = thingThatDoesDatabaseThings b 
   return! match a with 
      | Some data -> coolThing data
      | None -> unCoolError
}

You start wondering what the heck it is you are doing. What are these things? It turns out they are computational expressions. But how do I make them? You essentially create a ThingBuilder class with members for bind and return, and then create a let thing = new ThingBuilder(), after which you can write expressions like:

thing {
    let! unwrapped = funcThatGetsPassedToBind arg
    return unwrapped
}

To be a bit enterprisey I have come up with a draft of a brilliant thing. The F# Functional Maturity Model (FFMM):

Can get samples to compile even with modifications
Can get write small programs that do useful things in prod
Can write software systems that fulfill a business purpose
Can see how bad OO in F# looks and strive to create functional style code
Attempt to write a monad tutorial
See the beauty of computational expressions and want to use them everywhere
…
Stops being afraid and starts to love Kliesli composition (the fish operator)
Realise computational expressions are an antipattern and curse their existence
..

As you can see I have yet to ascend the ladder enough to know what all the levels are, but I am currently on 6 I believe, but given the hate people have for do in Haskell, it seems that there must be a level where you realise the computational expressions are of the devil and must be eliminated at all cost, but at the level where I am now, they do indeed seem like they cut away a lot of plumbing code where I otherwise map and bind to call various things. Also, I see the fish operator everywhere so clearly it must be awesome. I now at least understand what it does, but I can’t say it fits very well in the code I want to write. I’m sure that is an epiphany for a later date.

Disparate Opinions

Various tidbits