Life is just people and things working together to make things difficult for you. Like on a rainy windy day where you can just lean into the wall of oncoming air and water and just push through.

Most of these things you cannot really do anything about, and there is no point to complaining about it, but then there are small wins, like going around the corner of a big building and it taking a few seconds for the wind to change direction and blast you in the face again. Those few seconds are golden.

Anyway – one of those breaks in the rain is that I’ve switched off comments on my blog. There are two people on average that read a post, and rarely do they want anything from me. A handful of posts have over the decades accumulated hundreds of views. Among humans my writing has the attention it deserves.

The bots though are big, unrelenting fans and have an insatiable appetite for communicating all kinds of offers through commenting on my posts (that they can’t have read according to the page statistics).

I pay for a service that is supposed to deal with my popularly in the bot scene. An inbox zero-as-a-service, basically. Well those guys were annoyed that I sent too much traffic. Again two (2) readers per day generates enough spam bots that I either have to get an even more ludicrously expensive anti spam tier, buy a higher tier blog hosting to be allowed to add a captcha, or lastly self-host with expenses of both money and time.

I don’t want to do any of those as they cost money, and if you have seen the rest of the blog you’ll see why I’d rather not be spending any money on it. So I’m shutting the comments. I know this may lead to reduced “engagement” but the thing is, people that reach this page know how to reach me, so nothing is really lost, except friction.

I get that brief respite from the rain that you get at a large building site where the hoarding and scaffolding are overbuilt into a luxurious chip board arcade with strip lights and trip hazard warning tape everywhere. You get in out of the direct rain, but big drops from 70m up the scaffolding hit you directly on your skull through a gap in the chip board instead. It’s a win, but you’re never allowed to be too elated.

Anyway, if you need me, you know where to find me.

The current state of affairs

In 2020 I learned the meaning of the English expression Busman’s Holiday, and it generally applies to software developers that write code on their free time, but especially so during a pandemic with abundant remote working. Putting that aside, I will make some predictions of what will be happening over the coming year.

Predictions

The Pestilence

Given how popular the Omicron strain has proven, my guess is that everybody will have had Covid, and the patience for government measures will have grown thin, especially given the attitudes with which they flout the rules within the government itself. If Labour takes power, of course this can all change and we can be heading for more lockdowns.

The Industry

Despite lockdowns and the inevitable destruction of the service industry (yes, for good reasons in managing the spread of the virus, but let us be honest with the consequences) the IT industry has fared reasonably well. As long as I have lived in this country there has been a general election every two-and-bit years, and we could be looking at one of those again, and in a run-up to that, Rishi Sunak will want to keep money pumped into the system, meaning IT people will most likely still do quite well for a bit longer.

The Great Resignation

From the discussions around recruitment before the above variant gained popularity, there seemed to be two main streams, people that want to work remote full time, and people who want to work in a hybrid capacity, where you do meetings and collaboration in the office, and focussed work remote – if not at home at least in a co-working space closer to your home. The crutch used by weak leaders to manage people – counting bums in seats – will probably need to be replaced by some kind of outcome-based measurement. Luckily that ought to align quite well with company targets. No company has a slide in an AGM saying “well revenue is down, profits are down but luckily we have 99.5% occupancy of our desks“, the goal is to make money, and with the right type of goals within an organisation you can have department and team goals that in some way works towards the overall business goals, but of course measuring the right thing is key, so – yes – it is harder than just counting empty desks.

My thinking is that if the pandemic calms down, we will se a subset of organisations that are unashamedly on-prem only, and those that look for work that is on-prem only will go there, but I suspect that it will be harder to hire for those positions.

The Continuous Delivery

People insist with this Agile malarkey, and even though “Scrum, but…” remains the dominant methodology, companies are starting to read Accelerate and realise that they need to move faster, so gradually obstacles are being dismantled. Management structures are tweaked, project management and budgeting is being replaced with product and portfolio management. Coordination exists in companies already. Organisations that are famously agile say they struggle to coordinate cross-cutting changes across an organisation, but in old enterprises, that coordination work is the thing they do well, because in their current day-to-day even the most trivial piece of work cuts across several teams and needs careful planning and second-guessing to be delivered safely. The big differentiator is to change the internal structure so that for 80% of changes, a single team can plan, test, construct and monitor features completely independently, while leaving some version of the existing structure to deal with the subset of changes where you still need to coordinate. If you achieve that, you are in a vastly better place than before.

The Hardware Shortage

Have you tried buying a graphics card? A car? Well, you may have noticed that there is a supply chain crisis in the world. US container ports are struggling now and what originally started with the double whammy of Chinese New Year and OG Covid shutting down electronics suppliers, got worse as there was a supply shock when the pessimistic demand prognoses turned out to have not accounted for stimulus checks inducing demand globally, and more recently there i are geopolitical issues when one of the main semiconductor suppliers globally, Taiwan Semiconductor Manufacturing Company (TSMC) is situated in region on the brink of war while at the same time Intel are struggling to produce any advanced processor nodes in their own fabs, even though they now are producing a competitive line of processors again.

My prediction is grim here, but let’s pretend like things will go well. I don’t think you should buy anytihng in 2022 if you can avoid it, which has been my advice from March 2020 onwards, that hasn’t changed.,

The Crypto Scams

Just like with drug trafficking and modern slavery, you can make a lot of money with cryptocurrencies and NFTs, and you can already see that the biggest profits are made when people are robbed of their coins.

As you dream up your practical use cases that will finally be the problem that crypto solves, just remember this: Like with all applications of cryptographic signing, the time it takes to encrypt or decrypt something is part of why it works, why it is secure. You will never have a world where these transactions are fast and secure. All exchanges for cryptocurrencies that trade fast circumvent a number of supposed features of a distributed ledger. There is no “it will be faster, eventually” unless you are prepared to sacrifice some of the key selling points.

Luckily China has decided that crypto currencies are inherently decadent and are clamping down on miners, and if western utilities start going after those that steal electricity with more zeal, we could start to see positive change.

Don’t forget that NFTs, Bitcoin and Eth singlehandedly is destroying the Paris Accord on climate change, You can heat a typical American home for six weeks on the energy required for one (1) bitcoin transaction. As computers become faster, this will gradually be worse as well.

Conclusion

As with any arbitrary point in time, the time immediately after will not be drastically different than the time immediately preceding it, so there will be much of the same next year, but I have still tried to make some statements that are specific enough that we can go back in a year to see what I got right and what I got wrong. Happy New Year!

Background

In recent times Microsoft have begun to performance test their web platforms. Whilst previous generations of their .NET framework and ASP.NET web platform had prioritised ease of development over performance quite dramatically, the latest generation ASP.NET Core performs quite well, on Linux no less.

After inventing the async/await model of abstracting away callback hell when writing asynchronous code, the New Microsoft, the Ones That Care About Performance realised that people will just allocate all the RAM in the universe if you let them, and that whilst engaging in the now very common practice of using ASP.NET Core to create web APIs that produce data as json payloads, users would mercilessly just serialise massive payloads of List<T> into one massive string that they would shove out onto the network, or have server endpoints that would accept arbitrarily large strings off of the Internet attempting to coerce into a List<T>, meaning ASP.NET Core services could be knocked offline by supplying a ludicrously large payload, and performance could be a bit erratic at times, depending in the size of data the user was requesting.

So what do they do? Well, a couple of things, but one of them is to introduce the concept of IAsyncEnumerable<T>, an asynchronous enumerable, that supports cancellation, clean exception handling and stable performance for handling variably sized payloads without suffering unpredictable performance impact.

The goal today is to successfully serve a payload in ASP.NET Core 6.0, and to deserialise it in a client application, also in .NET 6, serialising onto streams, deserialising off of streams, processing data without allocating massive payloads, also – beginning to receive data right away rather than to wait before the full payload has been buffered in its entirety in various services along the way before eventually reaching the end user.

Physics and leaky abstractions

Just to preface this – just like the async /await doesn’t fundamentally change physics, i e there is no getting away from the fact that you first kick off an operation and basically schedule code to be run when that operation has finished, leaving you to do other things. I.e. since your code will actually return to the caller directly after you’re scheduled the first async operation, the code has to return something in addition to the normal return value, it has to return a handle through which you can access the state of the function, and – once the operation has completed – the return value. This way the surrounding code has a chance to deal with the asynchrony but most of the time just pretend that the code is synchronous.

You see, if the human squishy brain cannot fathom mulithreading, don’t let me get started on asynchrony.

So with a normal asynchonous function that returns a scalar, the caller receives a System.Threading.Task that encalsulates the asynchonous state and eventually the return value. The async keyword lets you pretend it isn’t there and write synchronous code , as long as you put an await in before the asynchronous call is made.

Contagion

You’ll notice though, like with monads, that when you’ve started wrapping your return values in Task<T>, it’ll go all the way across to the other side of the application, i e if your database code is asynchronous, the repository or other database access layer topology you have will be asynchronous too, and then you turn around and then you find that it has spread all the way to your ASP.NET controller. On the plus side, the ASP.NET controller automagically injects a CancellationToken that you can send all the way down to the database and get automagic cancellation support for long running queries if people refresh their page, but that’s an aside.

The point here is the contagion. You can attempt to force things to be different with GetAwaiter().GetResult() to block a thread while it’s evaluating, but that is very dangerous performance-wise, better to just let it spread, except for in places where Microsoft have been lazy, such as in Validation and Configuration, cause clearly when it would mean work for them it’s “not necessary” but when it’s eons of work for us they are fine with it. Our time is free for them.

Anyway, I mean it makes sense that the abstraction must leak in some cases, and IAsyncEnumerable is no different. Any one return value would fly in the face of the whole streaming thing. So awaiting a task doesn’t really make sense. Instead it’s iterators all the way down. Everywhere. Each level yield returns to the next, all the way down the chain.

Dapper allegedly comes with support for IAsyncEnumerable, but at the time of writing there is zero documentation supporting that allegation.

You can simulate that by writing this bit of code:

    public static async IAsyncEnumerable<T> QueryIncrementally<T>(this SqlConnection conn, CommandDefinition commandDefinition, CommandBehavior behaviour = CommandBehavior.CloseConnection)
    {
        await using var reader = await conn.ExecuteReaderAsync(commandDefinition, behaviour);
        var rowParser = reader.GetRowParser<T>();

        while (await reader.ReadAsync())
        {
            yield return rowParser(reader);
        }
    }

From that you can then pass the payload up iterator style, yield returning all the way up, until you get to the controller where you can declare the controller to return IAsyncEnumerable and the framework will handle it correctly.

Obviously as you cross the network boundary you have a choice in how to proceed, do you want to receive the data incrementally as well, or do you want to wait for all of it to arrive?

Since you made such a fuss in the first API, we will assume you want the consuming side to be as much work.

    private static async Task<Stream> GetStream(HttpClient client, string endpoint)
    {
        var response = await client.GetAsync(endpoint, HttpCompletionOption.ResponseHeadersRead);
        var responseStream = await response.Content.ReadAsStreamAsync();
        return responseStream;
    }

    public static async IAsyncEnumerable<T> HandleGetIncremental<T>(this HttpClient client, string endpoint)
    {
        var stream = await GetStream(client, endpoint);
        var items = JsonSerializer.DeserializeAsyncEnumerable<T>(stream, CreateSerializerOptions());
        await foreach (var item in items)
            yield return item;
    }

And then, of course, you yield return all the way up to the next network boundary.

Is this ready for prime time? Well, in the sense that Jay Leno was ready for prime time when he ceded the Tonight Show to Conan O’Brien, but everybody would probably like some more pace and less awkwardness.
Apparently letting lambdas yield return is on its way, and hopefully that can make it easier to pipe an IAsyncEnumerable through one API to the next, easily adding some filter or transformation mid flight rather than the incessant await foreaching that is now necessary.

Working life, like any series of events, can be compared to other stories, such as those in cinema. Is your workday like Avengers Endgame when all your coworkers show up out of thin air and swarm to solve a difficult problem? Are you Malcolm Tucker of In the Thick of It helping your co-workers by providing astute observations and giving gentle constructive criticism? Or is it more like the middle of a 70s social realist movie when the alcoholic father / engineering manager promises you that although sure it’s bad now – mistakes have been made – but you know it’ll be great, we’ll stand up kubernetes and we’ll never deploy manually again?

Obviously – when they make a big budget movie out of the Phoenix Project, we can just look at that, but until then – what film are you living now? Which one would you like to be living?

You want to believe assurances of a bright future, but deep down, you know you’ve heard it before. Perhaps problems being indiscreetly alluded to in an early act comes back in the final act to cause a massive predictable calamity making you think your movie has a poorly crafted arc. Perhaps you give your social realism engineering manager the “you need to cut down, think of the kids” speech – but it’s met with denial. “Our network guys are diligent [to be fair -they probably are, regular John C Reillys the lot of them], it takes 2 minutes to make a configuration change – why would I take hours out of their day to write scripts to do things they complete in half an hour including the red tape we have imposed upon them? Do you realise how busy they are?”.

What if you want to switch franchises – so to speak? Get into a better movie? Let’s say your film is the social realism one, and after a few accidents in the workplace, the union is shutting the site, and the owners are threatening to move production overseas. Car factory, sounds Birmingham-based on the accents. Lovely soundtrack with early seventies Black Sabbath. Your character has to stop the mayhem on the factory floor so that the union will allow production to start before the owners scrap the factory for good, your budget is £0 but you happen to have massive rolls of black and yellow adhesive tape, some PPE and a loudhailer. Basically, you can turn your film around, you can do it – but you do have to literally start doing something.

I’m writing this to continue on a ball of yarn I’ve been unravelling in other posts. Basically I want to state that DevOps doesn’t need to mean shiny and new. Any type of automation that does the job is fine. You don’t have to change platforms , you can – and I personally mean should – start by automating the existing stuff and not by building a new feature complete platform. Take the first step! Stop dreaming about a service mesh and kubernetes. It won’t happen soon enough.

This next bit will be very marvel oriented, by the way, but feel free to translate this to your own cinematic universe. It’s like – you can manage to automate and ship software reliably, but you may not be ready to be Tony Stark. You are still be part of the MCU, but you won’t be an arms dealer billionaire or even an Australian Norse space God, or even a fighter pilot with accidental alien super powers and amnesia. The best you can hope for is to write PowerShell or bash. PowerShell and bash, perhaps. Cobble together some automation with whatever CLI you have laying around. Automate the simple things. Even if you are disrespected in the office like Agent Carter you can eventually save the day. The big first step is to figure out how all your hand crafted bespoke servers are really built and figure out how to build them from scratch with scripting. This is the painful, tedious first step that you have to take. How can I create my production environment using only scripting and free or affordable tools that my people already know how to use?

In too many companies the deployment automation is:

1. Download packaged tested software from archive
2. Disable monitoring to avoid scaring your on-call people
3. Divert network traffic from node
4. Decompress archive and copy files in place
5. Restart services
6. Re-enable traffic.
7. Repeat 2 -6 for all other sides in the load balancer.
8. Re-enable monitoring

This is not enough. There may be any number of unknown things that just live on your VMs without which things just wouldn’t work. Crucial OS settings that were made once that nobody remembers anymore. Such hidden things are the potentially big surprises that derail containerisation projects or cloud migrations. You need to Agent Carter the Whole Thing.

1. Define networking. You have some leeway here – use a wild card cert, generate a new short-lived cert- create a load balancer or just a rule for a central load balancer. This depends on what you have in your infrastructure and what tools you know how to use, but basically – if the starting state is nothingness, after the automation is run, there should be a way for the outside world to find your service and know if it is healthy. If things already exist, your scripting should only make expected changes to it and be able to run multiple times without accidentally causing mayhem. Make sure any WAF rules or similar to enable access to dependent services are also set up here. If you can’t reach a necessary service at all, this should be immediately obvious from tooling without even digging into logs.
2. Define virtual servers. If all you have is VMWare CLI, then create a VM based off of a suitable template. If you have some fancy cloud provider, use the highest abstraction level you can get away with. Azure Webapp, AWS ECS or Lambda. Stay away from raw VMs if you’re running in cloud, they are expensive.
3. Install your servers to their desired infrastructure state and patch level. Ideally you use Ansible or even Powershell Desired State Configuration. There are so many non-trendy options that you already probably have a few installed. Chef or Puppet works too, if you have guys that know that stuff already. Find out what people already know and pick the simplest technology. The specific technology you choose isn’t key here, the big idea is learning how to take empty metal and get your stuff working there without having to do any manual intervention whatsoever. All of the infrastructure must be code.
4. Now you’re at the point where the previous list is relevant. Of course depending on your choice of technology you may not need to repoint load balancers as some tools like chef and puppet support in-place upgrade. A central brain/source of truth will announce that new software exists, and you have to manage in place upgrades through ruby scripting if you’re unlucky, but it works . Either way, only here are we at what the previous CD solution thought was all of it.

You aren’t done until you can spawn a service as easily as your users shout “Another!”. You can get to this point with tools your guys already know. It may not as sexy is just flying straight through an enemy star destroyer using helm whilst your mechanical keyboard glows in addressable LED colours, but the point is your organisation most likely possesses the skill to do this already. You must take the first step.

Whichever cinematic universe your life’s film belongs to, should be proud at what you have achieved in the face of such adversity.

In which I add a custom prompt by making a hack in the PowerShell profile.

As I have mentioned in previous posts, I use Oh My Posh to set the theme in Powershell. While working with Pulumi to create deployment stacks, I thought I could use a way to see which stack is the current one, i.e. to effectively have the output of pulumi stack --show-name appear in the prompt automatically.

Back in the old world, the agnoster theme was the prettiest. In my terminal at least, it looked quite a lot worse after upgrading to Oh-My-Posh 3, so I did exactly what they say in the documentation, I used Get-PoshThemes to look at all of them, exported the one I liked best into a json file and went to work.

Command

The naïve implementation would be to add a new segment in the prompt, using the segment type seemed to be “command”, which does what it says on the tin, it allows you to call a command and display the output, like it works in Bash.

        {
          "type": "command",
          "style": "powerline",
          "foreground": "#000000",
          "background": "#ffff00",
          "properties": {
            "shell": "powershell",
            "command": "pulumi stack --show-name"
          }
        },  

They do warn you that there will be performance implications, and – yes- on my 16 core desktop it still takes forever to start a process in PowerShell, so that didn’t seem to be a workable way forward. The suggested approach is to “abuse environment variables”, so… let’s?

Environment variable

I have previously made hacks to set window titles in cmder to work around iffy built-in support for showing the path as the tab name. The idea was to replace the built-in “cd” alias with a PowerShell function that also does dodgy stuff on the side apart from changing directory. In this case I would test if a pulumi.yaml file exists in the new directory, and in that case set the variable PULUMI_STACK to the output of pulumi stack --show-name, or set the variable to empty.

# --- other stuff
function Change-Directory() {
    param(
        [string]
        $directory
    )
    Set-Location $directory
    $env:PULUMI_STACK = ""
    if (Test-Path "pulumi.yaml") {
        $env:PULUMI_STACK = & pulumi stack --show-name
    }
}
# --- other stuff
Set-Alias -name cd -Value Change-Directory -Option AllScope

I of course don’t want to globally change this variable, I explicitly only care about the current terminal session, so hence I’m not trying to update the registry or anything like that. To read this variable and show a prompt, we then modify the theme json file to leverage the envvar block and to contain the following:

{
    "type": "envvar",
    "style": "powerline",
    "foreground": "#000000",
    "background": "#ffff00",
    "properties": {
       "var_name": "PULUMI_STACK"
    }
},  

After this work, the prompt is much faster, beyond acceptable, maybe even pleasant.