If you write code to SQL Server then you might be interested in this: at the end of 2017 I wrote a tSQLt tdd training course which has helped over 300 people learn both tSQLt and how to apply TDD practices to their SQL Server T-SQL development, you can join the course at https://courses.agilesql.club. The course is free if you are happy to wait 10 weeks to complete it, with 1 lesson being made available per week - if you are in more of a hurry or you would like to help support the project you can purchase the course which makes it instantly available to you.
Writing infrastructure as code is pretty nice and I enjoy the declarative approach of defining what you want and letting the tooling take care of the how. I guess this is because of my background as a SQL developer.
What I don’t like so much is the development experience and if you don’t get this part right it can be a right pain.
What are the challenges?
The first real challenge is writing the correct json for each thing you want to create. For example, to create a virtual machine you need to know what the api version is that you should use and which properties are needed and which properties are required vs optional properties. Once you have that for a virtual machine you need to know the same details for a nic, nsg, storage account for vm diagnostics and any vm extensions you might want to include (and what properties they have).
Getting all this information together can be quite painful, there are four approaches really:
- 1. “Automation Script” in the portal
- 2. Viewing JSON schema docs
- 3. Examples
- 4. Visual Studio Deployment project
The first seems the easiest which is you deploy something then in the portal you hit the “Automation script” button which generates the arm templates and a script you can use to deploy it with.
There are two problems with this, firstly not everything is supported and so if you are deploying cutting-edge resources this won’t work for you. The second issue is the way the script generator names all the resources and creates parameters for everything called something like resource_name_name_parameter if you have a fair few resources then this gets really confusing, really quickly.
The generated script is also for everything in the resource group, even if you only try to generate it for a single resource. Finally, the generated script also includes properties like status which aren’t part of a deploy and just obscure the actual bits you need. If you do use the automation script option it must only really be used as a guide to get you started or to check something.
The second option is to look at the arm JSON schema’s and work out what properties are supported with api version, you can find them:
The third option using examples from either the Microsoft docs site or github is good but often samples do one very specific thing and you need to spend a while tweaking them. The samples are also all different and there is not a common way to name things so merging multiple together is often a pain.
The last option is the visual studio deployment project which has a wizard to add resources, this is often quite out of date and the fix suggested by Microsoft (see comments https://docs.microsoft.com/en-us/azure/azure-resource-manager/vs-azure-t…) is to just edit the json manually.
So, there really isn’t one ideal way to create arm templates the way I work is a combination of 1-3, I don’t really use visual studio because I have migrated to use vscode.
Once you have the resource versions and properties you then have two more problems, the first is that you end up with a big blob of json and navigating resources and doing things like copying and pasting and moving things about is a little risky. Visual studio has a “JSON Viewer” that is a tree viewer and lets you navigate different objects like parameters and resources but not the properties of resources so if you have a large resource you can still only navigate to the beginning of it.
If you use this sort of thing to name your objects you don’t really know if they are right until you deploy, these things can get really complicated. Both visual studio and vscode have add-ins that provide intelligence and red squiggles to show if these are right or wrong but won’t let you see if you have used the right parameter or got the final concatenation correct, the amount of times I have deployed something to see the name isn’t quite correct so I have had to redeploy is literally infuriating!
What is the solution?
The first thing is getting the correct resource versions and properties and to be honest I haven’t found a better way than using a combination of the approaches. I will look at bits already deployed and use the automation script to see what it can look like and also use existing samples and the azure docs such as:
If I get really stuck then I will use the template schemas.
For the tooling I was using visual studio but I was having a problem because I was using octopus deploy variables which would do a find and replace with parameters in the parameters file and this is fine for strings and numbers but when passing arrays the format I need in the parameters file means the file is invalid json and when I copied and pasted something into that file Visual Studio tried to help the invalid format and ended up messing it up so I would have to undo the changes and be very careful about changing that file.
Because of this problem with the parameters file I started using vscode and the Microsoft Azure arm extension which gave me some bugs like intelligence etc and I could edit the parameters file easily but it didn’t have the JSON outliner that Visual Studio had so I back to having a blob of JSON and editing it became a pain again.
This led me to write a similar JSON outliner for vs code but because I wanted to get a better idea of what the code would look like I also evaluate any properties which have values that are code values like:
Then the treeview also shows the code value and the evaluated like:
I also added the facility to run a template script like the one above to see what it would look like when deployed, typing in lots of if(equals((((((((( etc into a string and not knowing what it will actually do is scary, so this helps me.
To get this to work there were a couple of things I have to do, firstly we need a parameters file that we can find so I look for a file called parameters.json if you have a different file for your parameters or you have something like I have invalid JSON then you can create a dummy parameters file and point to it using the .vscode/armed.json file so when we evaluate an expression we have something we can use for the parameter value.
I also haven’t implemented everything, if you use “reference()” to do something like looking up a property of a deployed resource then I don’t evaluate it - I don’t want to make the api call as part of an extensions which would be very slow - it is something I am thinking about implementing using something similar to the parameters file but will see if I need it or not.
There are also a couple of things like the uniqueString, I can’t find the type of hash that is used so can’t guarantee that it will be the same and the gt() function on strings behaves the opposite way to the way I expect it to. If you call gt on a start ‘a’ is normally greater than ‘A’ but in arm it isn’t. I’ve implemented it the way I think it should be and using gt() with two strings is odd anyway so I wouldn’t use it.
So I use a mixture of ways to get the resources and my armed extension to navigate templates, I display resources and properties in the window so you can navigate either resources easier. I also use the evaluate tool to see whether my code not just compiles but looks correct.