All posts by Rikard Ottosson

About Rikard Ottosson

.NET developer. Father. Couch potato. Old, cynical, willing to learn, but quick to judge. Likes old hard rock. Hope to buy more guitars.

IdS4 on .NET Core 3.1

C#, IdentityServer,

Sometimes I write literary content with substance and longstanding impact, and sometimes I just write stuff down that I might need to remember in the future.

Wen migrating a .NET Core 2.2 IdentityServer4 project to .NET Core 3.1 I had a number of struggles. The biggest one was that the IdentityServer threw the following error:

idsrv was not authenticated. Failure message: Unprotect ticket failed

After scouring the entire internet all I got was responses alluding to the fact that I needed to have the same data protection key on both instances of my website. The only problem with that was that I only had one single instance. Since I was also faffing about with changing DataProtection (unwisely, but hey, I like to live dangerously on my free time) – this was a very effective distraction that kept me debugging the wrong thing for ages.

After starting off from a blank .NET Core 3.1 template and adding all the crust from my old site I finally stumbled upon the difference.

In my earlier migration attempt I had mistakenly put:

            app.UseEndpoints(o =>
               {
                   o.MapControllers();
               });

Now, my main problem was that I had made multiple changes between testing, which is a cardinal sin I only allow myself in my free time, which is why it is so precious now. If you had only made this change and run the website you would notice that it isn’t running, probably. But since I changed a bunch of stuff at once I had a challenge figuring out what went wrong.

In the default template the writing was different and gave completely different results: 

      app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllerRoute(
                    name: "default",
                    pattern: "{controller=Home}/{action=Index}/{id?}");
            });

And lo, not only did the website load with the default IdS4 development mode homepage, the unprotect errors went away(!!).

The main lesson here is: Don’t be stupid, change one thing at a time – even if you think you know what you’re doing.

More WordPress

.NET

I previously dabbled with moving blogs off of WordPress.com onto Azure. This is a new story about moving on from there.

Lazy Loyalty

I have stuck with Azure for WP things for a number of years. I guess it’s like a gym membership, you set it up, is costs money, you think it is going to be useful, but you end up feeling it is a waste of a lot of money and the shame grows.

I had the unfortunate and for me unusual position of early adopter, so I had Windows based WP “Azure Web Apps” as they were called back in the day, long before MS ❤️ Linux and all that. The most weak sauce image available at that, backed by the most pathetic MySQL instance possible.

File System Philosophy

As no doubt even the least Windows hating computer guy will gleefully tell you, when the topic of Windows performance comes up, Microsoft bet on the wrong horse when designing NTFS, the file system of the future for the Windows NT Operating system in the late eighties. They thought the concept of an abundance of tiny files was a bygone era, the future was large media files, databases, documents. The file system is pretty good at caching large files fragmented over a physical disk.

Linus “Linux” Torvalds believed in the UNIX principle, believed that thousands of tiny script files would configure the future. In some ways he was right, and definitely in the case of file system performance. Ext4 is vastly superior to NTFS when it comes to storing/hosting websites because, you know it, websites are a shedload of tiny files.

Total Tragedy

So – underpowered cloud instance, wrong file system, cheapest DB… what was the end result? One word – abysmal. It would take 30 to 40 seconds to load a page in the admin interface. Completely unworkable.

So what to do? Well, the move had been arduous enough that I was tempted to just leave it, rather than move it again. I didn’t have another Christmas holiday I could spend on it (I had missed the “maintenance window” for hobby IT as it was already January at this point).

Scaling up?

Azure started offering a Linux based VM that came with WordPress preinstalled, i.e. a grown-up version of what I already had, with four times the virtual hardware as well, with a beefier MySQL instance becoming available as well. But at what cost you ask? Well- a significant one. It was that or all the Sky Sports you can possibly buy – so not worth it. Also, they had no upgrade path, so yeah, it would be like a complete move all over again, and like I wrote, just missed the maintenance window.

Scaling out?

So, instead I tinkered. Added a shedload of plugins. Azure Redis Cache, Azure Blob Storage & CDN and the WP Total Cache plug-in. Tried to smush images, but the admin interface was still too slow, so it was too difficult to remember what you had been doing when the page finally rendered. Now using all of these resources didn’t exactly make things cheaper, but it was more like a couple of movies off Amazon Prime than full on Sky Sports.

In a fit of desperation I even signed up for Cloudflare – which meant that the front office of the site, when cached, was the fastest thing in the universe, from anywhere, but Cloudflare couldn’t fix everything, so the error message of brokenness was very common, ruining the end-user experience.

Scaling down

Eventually after getting yet another Azure billing notification I just went at the Googles and searched for WP HTTPS NGINX or similar, and my filter bubble helped me find Bitnami. This will read like a transcript of a YouTube reaction video, but I don’t know anything about Bitnami, I didn’t want to know. It seems to be a company that knows about setting up Linux boxes. You can either download their stuff or buy AWS or Azure VMs through them, preloaded with their scripts and – shockingly – documentation on how to set it up.

I chose the smallest thing they had fitted with SSD on AWS, it came preconfigured with the correct firewall settings, a WordPress preloaded with useful plugins, and scripts to help you fetch free certificates from Let’s Encrypt and instructions on how to configure NGINX to use them.

Moving on up

The default – empty – WordPress was lightning fast, and the plug-in All in One Migration was most excellent. It has a sidekick that allows you to upload bigger files, you will need that, but overall, that  plugin made the migration a lot less awful compared to the default WordPress Import/Export tool. For the first site, I installed that same plug-in on the old site, downloaded all site data except spam comments, uploaded the File onto the new sites using the plugin and after some confusion when I had to log in with my credentials from the other site, and the browser became sceptical of this site now stating it was somebody it isn’t, but after authenticating and dismissing the security prompt the site was still acceptably fast. I changed DNS to point to this new machine and felt pretty good about myself having moved stuff across – even the media files were working, something that never happens with the normal import.

After the DNS had propagated I configured the lets encrypt stuff and got my padlock back. I started to see traffic and I noticed that there was a slight problem. The admin interface and the WP Rest API were completely broken due to mixed content warnings (crossed-over padlock!). The internet seems so baffled by my error, I presume it Just Works on the more common Apache Bitnami configurations, but that NGINX does some HTTPS termination before the web apps hear anything about the incoming requests, meaning the sites must be set to accept http internally.

By setting a parameter in WP-config.php forcing the admin interface to be loaded over HTTPS, the API started working again and there was much rejoicing.

Basically, a few hours in, the migration I had been fearing for such a long time had completed.

It remains to be seen what the total cost per unit of time will be, but the estimate looks good, at around a third of Azure prices – since one of the new images was made slightly more powerful and thus expensive.

…and the number of the counting shall be 3

.NET, C#

.NET Core 3.0 is here, allegedly the penultimate stop on the roadmap before the Singularity, when they finally bin .NET Framework and unify on top of Windows XP… er… .NET 5. The news are packed with stuff about WinForms, WPF and other legacy technologies, but I’m going to stick with the webby and consoley bits, where I’ve been mostly operating since I started using .NET Core back in 1.0 days.

Scope

As usual I will mostly just write down gotchas I have come across so that if I come across it again I will have a greater chance of not wasting so much time the second time around.

We are starting from a .NET Core 2.2 app that initially was .NET Core 2.1, so it may not have been fully upgraded in all respects between 2.1 and 2.2 if there were changes I couldn’t be bothered implementing.

Breaking changes

I followed an excellent guide to get started with references that need to leave your project file and other that need to come back in after they were exiled from the magic default Microsoft.AspNetCore.App assembly, as well as other breaking changes. It’s not that bad, and you really will enjoy the experience.

There has long been a trend among hipsters to forego the unstructured default folders in ASP.NET projects that buckets Controllers, Views and Models into separate folders, in favour of instead having two folders in the root, one called Features and another called Infrastructure. The Features folder would contain – you guessed it – each feature, with controllers, views, viewmodels and the data model grouped together. To make this work, there was the necessity of creating a new Convention for adding controllers with Feature, so that the view resolver would know where to look for the Views. Since AddMvc is now called AddControllersWithViews, I made that change hoping to make things look happy again. I noticed that the FeatureConvention had a squiggly. This is because the old interface IPageConvention that the FeatureConvention used to implement no longer existed. What to do? Well I looked around all over the internet and I found nothing, so finally I discovered the IControllerModelConvention interface, and by literally just replacing the name of the interface , everything just compiled, so the interfaces were identical.

services.AddControllersWithViews(options =>
            {
                var policy = new AuthorizationPolicyBuilder()
                    .RequireAuthenticatedUser()
                    .Build();
                options.Conventions.Add(new FeatureConvention());
                options.Filters.Add(new AuthorizeFilter(policy));
            })

Controller Actions

So with .NET Core 2.2 the ActionResult<T> type came into being, but now you are starting to see squigglies around IActionResult, saying the bell tolls for untyped responses. This is not such a big deal. Often this means you get to cut away vast swathes of boilerplate where you respond with Ok() or Json() around something, instead just returning what the handler created, replacing Task<IActionResult> with Task<ActionResult<SomeExcellentDto>>. Not only have you now achieved a lot of automagic swaggering where you otherwise would have had to write attributes manually to inform the consumer what the payload looks like, you have also eliminated the need for a class of tests just ensuring that the action methods return data of the right kind.

The thing to look out for is that if your handler returns an IEnumerable<T>, due to the way ActionResult<T> works, you need to cast the enumerable to an array or a list, because otherwise the type cannot be instantiated, i e instead of Task<ActionResult<IEnumerable<SomeExcellentDto>>> you need to go with Task<ActionResult<SomeExcellentDto[]>> or Task<ActionResult<List<SomeExcellentDto>>>

(Swash)buckle up, buttercup

The swaggering is a separate chapter – the latest versions of Swashbuckle are hard integrated with OpenAPI, so you have to replace any filters you may have created to format your swagger document since all the APIs are broken and replaced with similar ones from OpenAPI, and if you do any swaggering at all you have to get the latest prerelease of Swashbuckle to even be able to compile. Basically – if you are using Swashbuckle today – congratulations, you are about to start swearing.

IdentityModel crisis

The IdentityModel nuget package has been updated quite radically between versions, and if you were doing clever things in message handlers to keep track of or request tokens from the token endpoint when talking between services, you may need to update your code to do without the TokenClient that bereft of life has ceased to be.

The new method is to use extension methods on to HttpClient, and the canonical example is providing a typed HttpClient. created on demand by the HttpClientFactory, that in turn calls the extension methods. See documentation here. The token endpoint and the extension methods on HttpClient are covered in more detail here.

Show me the money

.NET

Payment gateways for small businesses. What do you look for in them? Personally I don’t care – I crave simple pleasures.

I would prefer if a checkout page looks like the rest of the site, but I need for it to be not-obviously-insecure and be compliant with current regs but also it must be less work to implement than the rest of the site was to create. If taking card payments was my core business, I would be in that business. I would prefer to sacrifice a significant chunk of revenue for this usability bonus.

The checklist

I would like a payment API

  1. To which I can connect with, ideally a .NET Core client, but pure HTTPS is fine.
  2. Where I can specify what my customer is buying
  3. I get to know who bought from me (email is all I need)
  4. Where I can indicate how much they will be charged (so that I can do discounts) – in fairness this is only sometimes missing
  5. That understands the concept of VAT and can just handle it for me. In the EU, VAT now in some cases has to be declared in the customer’s country. This is the type of faff a Stripe, Paypal et al should handle for me.
  6. Deal with 3D Secure automagically.
  7. Deal with PSD2/SCA automagically

Reality

Dodgy simile

Proper guitar amplifiers have a spectrum of volume*. Your exact volume knob indicators may vary, but the segments are universal.

Volume 1 – 4

practically silent,

4-4.8

Audible,

4.9

Decent volume, speakers are operating at reasonable dynamics, you can play. It’s just a bit quiet.

5.0 – 11.0

Massive noise complaints, police arrive.

Cards

From what I can tell, payment gateways operate similarly.

Level 1

You just need a button, and money might appear on your account. Never you mind who paid you for what.

Level 2

You can get to know who paid for what, but you’re SOL on VAT and have to do discounts manually like some schmuck. And webhooks. MOAR webhooks FTW.

Level 3

First you must create the Universe, then you must do 3DSecure manually and do three API calls to just begin to set up the first thing that might eventually become a card transaction.

Resolution

There is none that I can see. Am open to suggestions.

* I am aware attenuators solve this problem, but play along please.

Logging

.NET

I have had the misfortune of delving into logging a lot lately. To save time for next time I will write down the findings here.

My goals are simple. A couple of sites and APIs log into the same log aggregator, could be loggly, seq or graylog for instance. Given that I supply a correlation ID, I want to be able to tag all log entries related to one user as it travels through the system. This isn’t even on the bare minimum Charity Majors event logging, this is just glorified text but searchable with fields.

As of the date today, I want to be clear that for .NET, Serilog is best. Log4net is out since a long time ago, NLog tried but cannot explain how to do structured logging, so will have to be excused. Serilog has a more pleasant interface t and although I have struggled in the past to get the log context to enrich properly and had to resort to the Microsoft log abstraction combined with Serilog.AspNetCore to succeed and had problems getting the loggly sink working at all since docs skipped the need for the loggly-csharp nuget package. Still, it keeps winning, on old .NET Framework as well as .NET Core.

Setting up the Correlation ID has two parts. The first part is a piece of middleware in the request pipeline that wraps the call to the next stage in the pipeline in a using() statement. Here you extract the correlation ID from the caller or supply a suitable unique default for this call.

Then you create a message handler for setting a correlation ID on the outgoing HttpClient call. You can use the IHttpContextAccessor to get the incoming CorrelationId or the same default as earlier and map the message handler to any HttpClients you have defined in the projects.

Spite is the mother of invention

.NET

Premise

This is a tale about a blog on WordPress.com that had a loyal readership and regular, high quality content (so yeah, not writing about this blog). The owner wanted to use the odd plugin and advanced theme, and I was always bothered that WordPress was living off of, well I exaggerate wildly now, this person’s words by putting ads everywhere in addition to the massive annual fee.

Liberation

So with the lure of freedom on them yonder hills, we moved the blog off of WordPress.com onto a Windows VM on Azure (yeah, well… yeah…). Domain hosted on dnSimple, so the logistics of pointing the domain to Azure instead of WordPress and setting up verification TXT records and such was a doddle.

Hosted MySQL instance on Azure was easy enough, but the WordPress.COM theme we had been using was not available on WordPress.Org so we had to pick another one. Sadly we went with Customizr which really means vendor lock-in, as you do a bunch of customisations, hence the name, that are all out the window once you change themes.

Of course, there is no option but to run HTTPS today, and trying to pinch pennies we weren’t going to buy an EV cert from one of the remaining dodgy CAs out there, but iinstead we went with Let’s Encrypt using a tutorial posted by Scott Hanselman. 

Selling out – but is anybody buying?

To make the big bucks we hooked the the site up to Google Analytics and ditto Adsense, and there were plugins to really automate that stuff. Yoast SEO beat out MonsterInsights on features for the analytics and integrates both with Search Console and Analytics. The killer feature for Yoast SEO is the customisable canonical URL which is useful if you reprint blog posts from another site and want to beg Google for mercy for the crime of duplicate content.

The actual ads, how do they work? Well by cunningly just clicking like an insane person (which really is the best way to learn), I managed to understand the concept of Auto Ads. This again is abstracted away by a plugin, in our case Advanced Ads. As the site owner didn’t want ads on all pages, we had to hack it by creating a plain text and code ad with the Auto Ad code from Google pasted in there and then the Advanced Ads thing deciding which pages to actually serve the ad code. The downside is a persistent nagging that you ‘shouldn’t display visible ads in headers’, but I guess that’s fine. They are just script tags, so there is nothing visible there..

Also, all the cool kids enter the Amazon Affiliate program, so we did that. They do have a minimum number of referrals you have to make as they don’t want to deal with tiny unprofitable sites, so I suspect we shall be unceremoniously booted out fairly soon, but the concept of having widgets where you choose your favourite books related to the subject of your blog and maybe in the long term share some revenue if people take you up on your recommendations seems fair. Shame that the widgets themselves are so immensely horribly broken and difficult to use. Allegedly, they are supposed to update when you make changes in the affiliate program site, but they really aren’t. I don’t get paid by Amazon so I shan’t debug their system, but it can really only be that the command that goes back to save settings isn’t picked up, or that they are unable to bust the cache and have old widgets served, but I strongly suspect it is the actual save that is broken, since the widget loses the data already in the wizard before you even enter the last page.

AMPed up

After a few hours I noticed that all the permalinks from the old site were broken on the new one, so I checked the Permalinks tab and it turned out there was a custom setting that I just set to default which made things work and there was much rejoicing. No audit log here so I can’t check, but if I made that change it must have been unintentional. My favourite hypothesis is that somehow the otherwise impressive WordPress XML-based import somehow failed to bring over the settings correctly.

As I rarely venture out into the front end I had not quite grasped what AMP is. I realised I was getting another load of 404s – his time for URL’s ending in /amp. I did a bit of googling and I realised I should probably get yet another plugin to handle this. Like with most WordPress plugins there are varying degrees of ambition and usually they want you to spend $200 in extras to get what you need, but although I brought the site off WordPress.com to deny them ad revenue for the site in question, I was under no illusion that I would be able to produce any such revenue to the owner as whatever $3 would be produced would definitely be eclipsed by the hosting cost.

By going with the default WordPress AMP plugin you can’t do ads, but it works – ish, by using the major competitor you get a functional site, but a completely different look compared to the non-AMP site, and we didn’t want that after all the effort we had already put in.

After reading some more, I realised that everybody was going off AMP anyway, for varying reasons, but that was all the peer pressure I needed, so I broke out the Azure debug console and edited web.config to put in a URL redirect from AMP URL to a normal one.

This was incredibly frustrating as first I forgot that .NET Regexes are different from normal regexes and also you have to not be stupid and use the correct match in the redirect expression ({R:0} is the whole source data, while {R:1} is the first match, which is what I needed).

<staticContent>
     <remove fileExtension=".woff2" />
     <mimeMap fileExtension=".woff2" mimeType="font/woff2" />
</staticContent>
<rewrite>
     <rules>
          <rule name="Disable AMP" stopProcessing="true">
              <match url="^(.)amp\/?\r?$" />
             <action type="Redirect"
                 url="https://<awesomesite>.com/{R:1}" 
                 redirectType="Found" />          
          </rule>          
          <rule name="Redirect to naked" stopProcessing="true">
              <match url="(.)" />
              <conditions>
                  <add input="{HTTP_HOST}" 
                    pattern="www.<awesomesite>.com" />
              </conditions>
              <action type="Redirect" 
                   url="https://<awesomesite>.com/{R:0}" 
              />
          </rule>
          <rule 
               name="WordPress: https://<awesomesite>.com"  
               patternSyntax="Wildcard">
             <match url="*"/>
             <conditions>
                 <add input="{REQUEST_FILENAME}" 
                      matchType="IsFile" negate="true"/>
                <add input="{REQUEST_FILENAME}"
                      matchType="IsDirectory" negate="true"/>
            </conditions>
            <action type="Rewrite" url="index.php"/>
         </rule>
     </rules>
 </rewrite>

So there are a couple of things here – first a mime type correction to make IIS server web fonts, thne a redirect for AMP sites, then a redirect from www.awesomesite.com to awesomesite.com for prettiness, and also to canonicalise it to avoid duplicate records in the offices of Google, which they do not like. WordPress itself will force https if necessary, so all we need to do in this config file is to curb the use of www.

Summary

The actions we took to move the blog were the following:

  1. Set up the new site
    1. Create site
    2. Create blob storage
    3. Create redis cache (I did this later, but you might as well)
    4. Set up a database
  2. Export existing data from old blog
  3. Import data into new system
  4. Choose a theme
  5. Verify that old google links work on the new site (I didn’t do this fast enough)
  6. Verify that any way you try and call the site is redirected to a canonical represenation. Use a hosts file if you haven’t redirected the DNS yet, which with hindsight should have been the way I did it.
  7. Move the DNS to point to the new site
  8. Add the LetsEncrypt support to the site by following the guide. No more certificate errors
  9. Install plugins for analytics and ads.
  10. Create a Google account
    1. Register with Google Search Console
    2. Register with Bing search console (for those two or three people that don’t know Google.
    3. Register with Google Analytics
    4. Register with Google AdSense

Conclusion

So this was very easy and horribly frustrating at once. DnSimple and provisioning resources was a doddle. Following the internet guide to set up Let’s Encrypt and HTTPS was super straightforward, but then WordPress plug-in management, PHP and Amazon widgets were shit shows to be honest. I mean I realise Amazon has a complex architecture and their systems are never 100% up or 100% down and so on, but a save button being completely broken doesn’t feel even slightly “up” from the point of view of the end user.

PHP is garbage and brittle and you are hard-pressed to build anything viable on top of it (but obviously some have succeeded). These plugin smiths aren’t Facebook though. They would correctly interject that I am running WordPress on the least suitable platform imaginable. That is true (it has to do with how the Azure VM instances are set up, on the fact that they run on Windows and most importantly NTFS which has performance characteristics that are completely unsuitable for Unix style applications and favour a small number of large files where EXT4 favours large amounts of small files), but if the Powers that Be really consider Windows and NTFS to be such tremendous deal-breakers, then they should simply not allow Microsoft to host WordPress on Windows at all. As it stands, it does WP no favours with 1 minute turnaround to save settings for a plugin and similar. Then again, I also live in the UK which notoriously has a Internet infrastructure dating back to the Victorian Era, so it’s hard to tell what’s actually the worst culprit, but the sidecar web app that hosts the debug console for the blog is a lot snappier than WordPress, and that is hosted in the same IIS intallation as the WordPress site, although not in the same app pool.

Structural Equality – or is it?

.NET, F#

I was recently presented with a conundrum. We had constrained data valid for the domain in a record type. Sadly this record type contained a reference datatype, so built-in structural equality broke down as the reference type never was equal in the way we thought would make sense.

This gave me the opportunity to learn how you override the implementation of Equals and GetHashCode in F# which I was previously unfamiliar with.

This is the finished implementation of the record type, or one like it, rather:

 [<CustomEquality>]
 [<CustomComparison>]
type Structure =
    { 
        Name: StructureName
        Status: StructureStatus
        Format: Regex
    }
    with 
        interface IComparable with
            member this.CompareTo { Name = name; Status = status; Format = format } =
        compare ( this.Name, this.Status, this.Format.ToString() ) (name, status, format.ToString())
        interface IComparable with
            member this.CompareTo obj =
                 match obj with 
                     | null                  -> 1
                     | :? Structure as other -> (this :> IComparable<_>).CompareTo other                
                     | _                     -> invalidArg "obj" "not a Structure"
    override this.Equals (o: obj) =         
        match o with              
            |  :? Structure as os ->                 
                (this.Name, this.Status, this.Format.ToString()) =
                    (os.Name, os.Status, os.Format.ToString())
            | _ -> false
    override this.GetHashCode() =
        (this.Name, this.Status, thus.Format.ToString()).GetHashCode()

So yeah, ujse of pattern matching to determine data types in the non-generic functions and extensive use the built-in structural equality in tuples.

Very nice. With thanks to TeaDrivenDev and Isaac Abraham on Twitter (and this StackOverflow response)

Bind? No.. Apply…? No… Map!?

.NET

After continuing my foray into the functional with Giraffe and .NET Core, I have struggled with the code not quite reading very well. Various data sources are pluggable, i.e. they can be replaced with mocks if you don’t want to run other supporting systems while debugging. This pluggability along with the way Giraffe handles some of its configuration means that – while we would like to compose partially applied functions with either mocked or actual services baked in that are then passed in to the relevant handlers, we instead have to pass a HttpContext around everywhere. If you followed that link you probably wondered why we did not just use the strategy outlined in that blog series. I can only say that we did not understand fish operators at this point. 

So we turn to the googles, and as usual the first or second hit lands us on F# for fun and profit. Scott Wlaschin introduces us to the Reader Monad. We watch his talks, we read his blog posts. A week of enthusiastic coding goes by.

Essentially – the reader monad allows you to write and construct all the code and only at the end plug in the HttpContext as the whole thing is evaluated and returned to the caller.

You make a Reader<‘env, ‘data>  and you create functions to wrap and unwrap things in and out of the Reader. Then you make functions that mind their own business but they return Reader<‘env, b: and you allow the reader to handle some railway oriented programming for you. Essentially you lay the track all through the code but on the last line in the handler you put the train on the tracks and set it off, only then realising whether or not it ended up running through the failure cases or went all through the success track.

Wrappers, essentially, like Option<‘a>, IEnumerable<‘a> or Reader as described above, are introduced as Elevated things. The proper word seems to be Monadic types, but that does not seem to be a good term, as there is no need for a wrapper type to be a monad, although they can be. After a few specific examples the general case is presented and it turns out you can do a bunch of stuff with these elevated types only using functions like bind, return, apply and map. There is a brief epiphany as we experience a flicker of understanding of  the monad laws.

A monad is just a monoid in the category of endofunctors, what is the problem?

Now I cannot explain the chaining of elevated and non-elevated functions properly. I know what bind does, and I can use it correctly on the first try. Return is obvious. Map and apply though… it’s like being ten years old again and round-robining commands until the compiler is happy. As it stands all I can do is link to the posts we have read and the metaphors that have been presented to me, that I have half understood and then tried to reconcile with other conflicting metaphors.

Inevitably the golden success case of “first I need this, then I pass it to this other thing and then I flip it, kick it and reverse it and then I return it” is soon replaced by “well, first I need this one thing, that I need to just check against this other thing, but then use it again in this third thing which I convert to this fourth thing that I might return if this fifth thing is true” and the nice chaining goes out the window and it makes you sad.  Every let b =  … feels like a let-down (hence the name).

This let = thing is apparently called applicative style, where you compose values, while chaining is called monadic style. At least it has name. 

So after a while when you have been writing things like

task {
    let! a = coolFunc b c
    return! a |> modifyInAnAwesomeWay
}
async {
   let! a = thingThatDoesDatabaseThings b 
   return! match a with 
      | Some data -> coolThing data
      | None -> unCoolError
}

You start wondering what the heck it is you are doing. What are these things? It turns out they are computational expressions. But how do I make them?  You essentially create a ThingBuilder class with members for bind and return, and then create a let thing = new ThingBuilder(), after which you can write expressions like:

thing {
    let! unwrapped = funcThatGetsPassedToBind arg
    return unwrapped
}

To be a bit enterprisey I have come up with a draft of a brilliant thing. The F# Functional Maturity Model (FFMM):

  1. Can get samples to compile even with modifications
  2. Can get write small programs that do useful things in prod
  3. Can write software systems that fulfill a business purpose
  4. Can see how bad OO in F# looks and strive to create functional style code
  5. Attempt to write a monad tutorial
  6. See the beauty of computational expressions and want to use them everywhere
  7. …
  8. Stops being afraid and starts to love Kliesli composition (the fish operator)
  9. Realise computational expressions are an antipattern and curse their existence
  10. ..

 As you can see I have yet to ascend the ladder enough to know what all the levels are, but I am currently on 6 I believe, but given the hate people have for do in Haskell, it seems that there must be a level where you realise the computational expressions are of the devil and must be eliminated at all cost, but at the level where I am now, they do indeed seem like they cut away a lot of plumbing code where I otherwise map and bind to call various things. Also, I see the fish operator everywhere so clearly it must be awesome. I now at least understand what it does, but I can’t say it fits very well in the code I want to write. I’m sure that is an epiphany for a later date. 

Giraffe F# update

F#, Functional

In a previous post I have written about Giraffe and shown some workarounds for what I perceived as limitations considering my existing enterprise ecosystem.

Now that Giraffe has turned 3.0 loads of things have improved even from my perspective. Obviously they are now competing in performance benchmarks, so loads of optimisation work has happened behind the scenes. For me, though – the most interesting bit is to see how they dealt with my complaints.

Configuration, logging and service location is now available through extension methods on the context.

Response handlers now handle streams and chunked responses.

Content negotiation looks very clean.

We have of course welcomed these improvements and use them in all our F# APIs.

The only remaining complaint is that the Swagger support is still not usable enough for our purposes. The fact that we would have to configure an additional separate nuget source onto get the package prevents us from actually using it in anger although showing API actions was simple enough but I’m unclear as to how to document request and response payloads – but that only becomes worth exploring when the package is served from nuget.org

Contrarian style guide for C#

C#

Goal

I have decided to try and formulate what I want C# code to look like. Over the years I have accumulated a whole host of opinions that may or may not be complimentary or maybe contradict each other. The purpose of this post is to try and make sense of all of it and maybe crystallize what my style is – so that I can tell all of you how wrong you are and what you crazy kids should be doing.

Is this all my intellectual property? Hells to the no. I have read and heard a bunch of stuff and probably stolen ideas from most people, but the most important people I have listened to or read are Fred George and Greg Young. Sadly it didn’t take.

Object oriented programming

Loads of people hate it and cite typical C# code as the reason why. Yes. That code is horrible, but it isn’t OOP.

Why was Object Oriented programming invented? It was a way to make sense of large software systems. Rather than a very long list of functions that could – and did – call any other function perhaps modifying global state on the way, people were looking for a way to organise code so that changes were predictably difficult to make. This was achieved by introducing encapsulation. Private methods and fields could not be affected by external code and external code could not depend on the internal implementation of code in a class. If implemented correctly “change” means adding a new class and possibly deleting an old one.

Was this the only way people tried to solve this problem? No – functional programming was popularised, relying on pure functions and composition again offering a way to make changes in a predictable way.

I don’t mind you using C# to write in a functional style and by using composition, but if you’re creating classes – take encapsulation seriously. Encapsulation is The Thing with OOP. This means properties are evil. Think about it. Don’t put properties in the code.

But what about inheritance?

Yeah, when Borland C++ came out in 1992 the ads were full of Porche Targas that inherited from Carrera S but with the roof overridden to be missing (!) and similar. Oh – the code we would reuse. With inheritance.

Well. Barbara Liskov probably has some things to say about the Targa I suspect.

In short – don’t use inheritance unless the relationship between the classes is “is a” and the relation is highly unlikely to change.

Single responsibility principle

So you have heard about the single responsibility principle. When you see duplication you swoop in, add a new parameter and delete one of the functions. Boom!

Except what if those two functions weren’t doing the exact same thing? Their similarity was only fleeting. With the refactoring you have just made you have introduced a very hard coupling.

Code size

A method shouldn’t have more lines than three or four. Be liberal with the extract method refactoring. To “fit” in this metric, make guard clauses one line each. Put in one empty line after guard clauses and the meat of the class.

Why? What is this? Why are we counting lines now? Well code tends to attract more code as it ages and methods grow. Having a “magic number” that is The Limit helps you put in the time to refactor and thus battle the bloat before it happens.

Classes should not have more than two or three fields – as discussed earlier there will be no properties. If you feel yourself struggling trying to convince yourself to add another private field, do investigate whether or not there isn’t a new class in there waiting to break free.

Don’t be afraid to leverage the fact that private is a code level constraint, not an instance level one. I e, you can let instances of a class talk to each other and they will have access to the private fields of the sibling. This is one of the ways in which you can do actual work without properties.

Edges

Well with this whole Amish attitude to properties – how do you even deserialise json payloads or populate DTOs from the database? How do you write files?

By using the mediator pattern or writing adapters. At the edges you will find code getting less OO and more … procedural? I e anemic classes with only data, loads of members et c and verbose code that talks to very microsofty framework classes.

This is OK, and to some extent natural – but do not let it leak into the domain. Create domain and value objects for the actual processing without leaking any of the ugliness into the domain.

Avoid raw primitives

If you throw around a customer ID in your code – how do you represent it? As an int? How about an order ID? You know the drill – an ID cannot reasonably be added, subtracted, multiplied or divided and it probably can’t be <=0. And a customer ID and order ID should probably never be assignable to the same parameter.

Yes – take the time to do value objects. Resharper will generate equality and GetHashCode for you, so yes it is harder than in F# with more required boilerplate, but it is worth it.

Conditionals

Where the arbitrary constraint on method size really cuts into your Microsoft sample code style is in the area of conditionals. Do you even WinMain bruv? Nested if statements or case statements – what about them huh? Huh?!?

Yes. Get rid of them.

When making small classes an if statement is almost always too big. Lift the variability into its own class – a strategy.

Unit tests

There are two schools of unit testing. The London School and the Chicago School. The Chicago school relies on setup of instances, calling them and asserting the results. The London School relies on setting up a series of mocks, passing them into a class and then asserting that the mocks were called in the right order with the right values.

Of course – me being lazy – I prefer the Chicago school. With small classes you can just call the public methods and assert that the right results come out. If you feel the need to get inside the class to look at and validate internals – your class under test is too big or does too many things.

Creating a class

When you create a class, think about its purpose. “What’s its job?” as Fred George asks. Don’t create a class where the name ends in er or or. Find a concept that describes what the class understands.

It’s ok to spend 15 minutes drawing on paper how you see the next bit of coding to go before you write the first test and get going.

Disclaimer

It depends. Of course it all depends. But I’ve never noticed “omg these classes are too small and have no weird dependencies- I need to refactor” so I am fairly confident in these recommendations.

Reading list

The following books are useful to have around or to have read.

Some other books that are good for you as a developer: