Lightning Fast Updates

How fast do you deploy updates? If Microsoft released a run of the mill update today, how soon would you see it on your production systems?

I like to patch my systems quickly. Over time, I have gotten quicker and quicker at rolling them out. WSUS was a great addition to Windows Server. Not only do I auto approve the important updates, I also auto approve just about everything else.  I found myself blindly approving them twice a year anyway.  That tends to create a monster patch. The problem with monster patches is that people notice them. The login takes longer so you tell them its the patches. Then if something like a hard drive goes out, then people blame the patches.

I still hold of on service packs, new versions of IE, and feature packs. All for good reason. I have only been caught off guard a few times because of it. So I have everything on auto approve.

One issue that I did have for a while was patches showing up later in the week than I expected. I would be ready for patch Tuesday expecting that things would be patched Wednesday morning and they were not. It felt like most of our patches hit on Thursday instead. So I started to look into it.

Our machines patch at 3:00 am if they are powered on. I tried to get WSUS to update just before 3:00am so things would be ready to go when the computers went to update. Sounded good in theory but that is not how the client updates. I found out that the computer will check in with WSUS once every 24 hours by default. If WSUS was not pulling updates until 3:00 am, then everything was really updating 24 hours behind.

So know I knew why my updates felt a whole day behind. I increased how many times WSUS would pull updates from Microsoft to 3 and let it run for a long time. My WSUS server was checking for updates at 11:00 am, 7:00pm, and 3:00am central. This way I was catching any other updates that showed up at odd times. I would get a few more machines and servers updated a day ahead, but the bulk of them was still 24 hours behind.

There is a very subtle detail here that I overlooked for the longest time. I had no idea what time Microsoft actually released at on Tuesdays. I ran with this schedule for a very long time. Then one day I was really reading an important IE patch that everyone was rushing to load and I saw the expected release time. It was at 10:00 pst. Seeing this time reminded me to check my update schedule.

Sure enough, I had it in my mind that they were released in the evening. I could see my logs showing updates getting pulled at 7:00pm every Tuesday patch day. With 10:00pst being 12:00 central, it clicked with me where my issue was.  I moved that early sync to 1:00pm and everything started updating right on schedule. All my servers and workstations were updating as expected with a very clear schedule.

I also started scheduling wake on lan to power up our workstations and combined it with a check for updates event. So now all my computers are getting updated as fast as reasonably possible and I know exactly when to expect issues.