Enable-PrintHistory

I recently wrote a script that parses windows events to report on printing by user or by printer. To make that happen, it was important that a special log gets enabled. I was able to create a small script to do that for me.

$EventLog = Get-WinEvent -ListLog Microsoft-Windows-PrintService/Operational 
$EventLog | 
   %{$_.IsEnabled = $true; 
      $_.MaximumSizeInBytes=50MB;
      $_.LogMode = "AutoBackup"
      $_.SaveChanges()} 

One thing you will notice is that I call SaveChanges() after I see all the values. None of the settings will be saved if you don't do that. It is one of those details that could easily be missed if you were not looking for it.

I polished it up a bit as a CmdLet: Enable-PrintHistory

How do I track pages printed?

We had a 3rd party service managing our printers for a while. It didn't work out in the long run but they gave us these nice reports showing us how many pages each printer printed in the previous quarter. I found that information very valuable thought and kind of missed it.

After a little digging, I came up with a way to track that information without having to walk to every printer. To be honest, I had a lot of ideas but I eventually found a Windows event log that that gave me everything I needed. The event log is called Microsoft-Windows-PrintService/Operational. You first need to enable the log but it collects a lot of good details. 

$log = Get-WinEvent -FilterHashTable @{ "LogName"= "Microsoft-Windows-PrintService/Operational";"ID"="307"}

Once we pull all of the events, it they will be easy enough to parse with RegEx. Through this together quite quickly but it gets the job done. I have only tested this message format on Server 2012R2.

 *RegEx excluded from the post because I can't get it to render correctly in blogger without a lot of rework. See script at the end.

Once I parse out the values I need, I package it back into an object. From there you can do whatever you need to do with it.

$log | ?{$_.message -match $MessageRegEx} | 
    %{ New-Object PSObject -property @{"Document"=$Matches.Document;
       "UserName"=$Matches.Username;
       "IP"=$Matches.IP;
       "ComputerName"=$Matches.Computer;
       "Pages"=$Matches.Pages;
       "TimeStamp"= $_.TimeCreated;
       "Printer" = $Matches.Printer;
       "PrintHost" = $_.MachineName
    }}

Then write that out to a CSV file when you are done. If you take a look at the values that I can parse out of it, I get a lot more information that I expected. Not only can you get page counts per printer, you can track printed page counts back to individual users. I pull this dataset into Excel and transform it into a pivot table for easy reporting.

After I clean this up a bit, here is my resulting CmdLet: Get-PrintHistory

Why Powershell Desired State Configuration?

I am a huge fan of Powershell. I use it where I can to manage my infrastructure and workstations. Powershell remoting is such a wonderful thing. I start to think about my infrastructure as nodes or objects in code. As someone that shifted into IT from development, this becomes a very powerful tool. 

I craft and test my commands on test workstations. I then take those commands and order all of my nodes in AD to go take care of it. I see a job spin off for every one at almost the same time. And I know they are doing exactly what I am telling them to do. It just gives you that feeling of power. It is even better when you can see and hear it happening.

Our largest dental clinic has 50 chars in it. Each cubicle has dividers between them, 2 people could stand up in opposite corners of this large room and they could see each other. There is a central control station that faculty use where I love to issue remote commands. When I send something like Wake On Lan to all the clinics, I can hear every machine in there click on at almost the same time. I can hear the fans spin up and I can see the monitor glow cast a blue shine across the room. There is nothing like seeing your code interact with the real world like that.

This is the perfect fix it now type of remote scripting. This is great for when I want to refresh group policy for flush DNS on every node. I target my Active Directory nodes and say go take care of this. But not every script is a fix it now type of script.

Because I blindly target nodes when I do this, some slip through the cracks. Maybe WOL was miss configured, bios has a low battery warning, the mobile user is out of the office, or it is a spare machine sitting on a shelf. I am left with several machines for whatever reason missed the script. I don't care so much when I flush dns on everything but what about enabling bitlocker? That just has to get taken care of. So I have to decide how to handle this going forward. GPO, SCCM, scheduled task on some random box, or I run this script again when I think about it. 

Personally, I make that call case by case and use whatever I think is best for the change. I have to consider newly deployed machines, returning machines, and how likely those settings will drift. And this is where Desired State Config starts to enter the picture. We have to remember it is just another tool like GPO or SCCM, but it does have a lot to offer.

DSC lets you declare how a system should look, and it will take care of it for you. You once again get to say "make it so" and the system takes action. Not only does this set configurations on nodes, it makes sure they stay configured. With a pull server in place, they never miss a command. You set it and forget it. 

I am just getting started with DSC and I love the idea. Still not sure it is the best match for my current environment, but I think it is important we take notice of it. I felt Powershell was overlooked by a lot of people for the longest time. Step back and see how important Powershell has become already. And when they Jeffery Snover says the reason they created Powershell was so that they could create Desired State Config, you know this is something you should pay attention to.

I don't think we will see this for workstation management yet, it is only a matter of time before we see tools like SCCM switch over to use DSC.

My Notes from TechED 2014

I just got back from TechED 2014. One thing I really love about TechED is that all the sessions are presented by the people that work very closely with the products they are talking about. Often time you can gain a lot of insight just by the way they talk about things. Every sessions has 1 or 2 real gems.

As I review my notes, here are some of the key takeaways for myself. Things I didn't realize before or things I need to pay more attention to going forward. These are my notes and they are a little cryptic at times.

Azure now offers Redis cache
SQL 3014 offers %3000 performance increase for some workloads
API management is new Azure feature
Antimalware now part of azure
Office 365 encryption with each file having its own key
Site recovery now support replica directly to Azure
Managed apps are on IOS/Android
Azure Remote App
Cortana runs in the cloud is and is context aware
Xamarin offers .Net support for IOS/Android with Visual Studio 2013
HDInsight is Microsoft's implementation of Hadoop
PowerBI.com is like PowerView but in Azure with a powerful natural language search
Oslo and the Office Graph
One source of the truth
vNext Intune managed apps gives more control over data. Can block copy paste to other apps
Win-B310 TWC: Bullet proofing your network security - Download PPT and check blog for online sessions
defaultpassword.com
exploit-db
Security Compliance Manager: http://technet.microsoft.com/en-us/library/cc677002.aspx
    Can compare you settings vs best practice. Can also import GPO and compare it.
Microsoft Attack Surface Analyzer: http://www.microsoft.com/en-us/download/details.aspx?id=24487
    Run it before and after you install an application and it shows you how it changes the system attack surface
ITIL V3 DCIM-B224 - Watch this session again
   Start IT Steering Committee
   What service do we offer and how do we measure it
Desired State Config
  Scale * Complexity =  Exceeds Skill
New-CimSession - is this the replacement for New-PSSession, Investigate more
When hacked, do a memory dump before you shut it down. Then do a disk dump.
   Lot of important stuff in ram, like possible encryption keys or memory resident tools
Whatever runs is in memory and the rest of the OS is just a bunch of files
The prefetch can identify recent programs that were ran
This is turned off by default on SSD drives, turn it back on
X2works pf64.exe can analyse prefetch files
RPD History shows recent connections
Bitmap cache can show pieces of the last sessions screen
mstsc /public
Recommended to centeralize logs
Nvidia K2 or K5000 for VDI
Use change logon /drain
Local accounts have the same name, the fact that they have the same SID is not important
If 2 accounts have the same password, they have the same hash.
You can deny access to computer from network with group policy
New local group for local accounts that can be used in GPO to deny access
NT AUTHORITY\Local account
Run LSASS as protected process, it requires dlls to be digitally signed
mstsc /restrictedadmin
You can create silos in Active Directory
Monad Manifesto http://www.jsnover.com/Docs/MonadManifesto.pdf
DSC will be the primary administrative interface for Microsoft
released in October and Stack Overflow is using it in production
You can push or pull, use pull over https
DSC uses WSMAN
VMM can inject .mof files already
Must import non more modules in config files
connect.microsoft.com - where to provide feedback
Don't let existing habits dictate how you use this
DSC for Linux
Become a pull admin - Things are changing fast enough that you have to pull the info yourself to keep up
Focus on servers
One version of the truth
Virus Total integration with sysinternals
Uploading suspicious files can tip off attackers that you are on to them
clip command takes standard input to the clip board
AccessChk new options -h file/print shares -l detailed list -f filter out things
procmon can export to XML
there are scripts on blog to identify changed items and then copy those files/reg keys to a folder
if you run xcopy from a 32 bit cmd, then it will put things into the 32 bit locations automatically
You can use scheduled tasks to bypass UAC prompts
PsExec now encrypts on the wire
PsPing can to UDP testing, timed tests, and created firewall exceptions
darkreading.com
winflip for windows 8 embedded thin clinets
Storage spaces limit to 80 disks per pool and 4 pools per SOFS cluster
disk x columns = write speed
When trying to max iops, consider latency
Performance paper in PPT from Wed Tiered Storage Spaces Best Practices
use MPIO, use SMB multichannel
4xSSD + 20x7.2rpm drives = 700 VDI users
Force dedup after deploy then run tiering
There is a script to validate all the drive you need to run before you set up spaces
Monitor health, latency, tier usage
Server Performance Monitor (SPA) tool can configure alerts
xJEA
Get-PSSSessionConfiguration shows several default configurations
uses local account. 127 character password that changes daily
Get-Command will show you what you can run
DevOps - If you cannot measure it, you cannot improve it
Campfire, look at it
Automated SCCM Deployment that uses a variable.xml file for all the settings
   will download everything, take care of dependencies, and fire up VMs if needed
jp64 from TZworks can read the NTFS Journal changes
in IIS, application pool settings can dump passwords in plain text
Same issue with services
Sysinternals Process Manager uses purple for packed files
Search strings for HTTP
Change the default refresh rate when hunting malware
Add verified signature column
Add VirusTotal column
Autoruns can scan offline systems
ProcMon - filter category write, then proc tree view to include subtree
Azure Express Route - 1-2 ms delay, with 10GB pipe

That is more of a mess then I expected. I may revisit this list later and clean it up with more details as I work with these things.

Fixing print drivers missing dependent files

I was having an issue with print drivers that felt corrupt. My quick way to fix a broken machine was to reset the spooler and delete the driver quickly in print management before it was locked by the spooler. Some times it took a few tries but it worked.

I continued to investigate the issue and it lead me to a set of registry keys that defined the print drivers along with the dependent files. My broken computers had a null key for the dependent files. I was able to write a script to identify the broken drivers.

#Quick check for bad printer driver
$keys = Get-ChildItem 'HKLM:\system\CurrentControlSet\Control\Print\Environments\Windows NT x86\Drivers\Version-3\'
    $keys | ?{$_.Name -notmatch "Microsoft enhanced Point and Print compatibility driver|Amyuni Document|Microsoft Office|OneNote|Foxit Reader"} |
        ?{ !((Get-ItemProperty -Path ("Registry::" + $_.name))."Dependent Files")}

I figured out that I could delete the driver key from this location and the spooler would download the correct driver from the print spooler after it was restarted. Delete key and restart the service. I ended up with this script:

#Quick fix bad printer driver
$keys = Get-ChildItem 'HKLM:\system\CurrentControlSet\Control\Print\Environments\Windows NT x86\Drivers\Version-3\'
$keys | ?{$_.Name -notmatch "Microsoft enhanced Point and Print compatibility driver|Amyuni Document|Microsoft Office|OneNote|Foxit Reader"} |
    ?{ !((Get-ItemProperty -Path ("Registry::" + $_.name))."Dependent Files")} |
    %{ ($_ | Remove-Item); Restart-Service spooler -force}
 

The nice thing about an identify and repair script like this is that I can run this remotely on all the computers that we have. It turns out that 10% of our computers had this issue.

I investigated several of these before I fixed them. I found issues from printing silently failing, to printing artifacts, to printers ignoring settings (like print dual sided), to advanced settings that were just missing. I think this issue is accounting for a lot more service calls that I initially gave it credit for.

I now have a script that I can run building wide to fix this problem. While that is good, I still don't have a fix that prevents the problem.

There are several threads talking about this issue and I think Microsoft has a patch in the works.
http://social.technet.microsoft.com/Forums/windowsserver/en-US/08bc6b4b-0190-40b0-be16-8b82249148e4/print-driver-being-modifyied?forum=winserverprint
http://social.technet.microsoft.com/Forums/windowsserver/en-US/e2acb625-027d-47a9-b4a7-1616e270bcbc/print-drivers-on-windows-7-clients-missing-dependent-files?forum=winserverprint
http://forum.support.xerox.com/t5/Printing/Secure-Print-5330-7545-5745-Basic-Mode/m-p/13445#M2980
http://support.microsoft.com/kb/2864755

My printer drivers are corrupt

I hate dealing with printers. I know a lot of system administrators feel the same way. We recently started to see an increase in printer related calls. Some of them turned out to be a real pain to fix. Almost as if the driver was corrupted. It had to be a system issue because our users are not administrators. We either push the printer out or they add it from the print server. Not much for them to mess up.

We had one printer related issue that took the team too long to fix, so I found myself in the middle of it. Not just fixing the problem but also calming down the customer. Although we mostly had her fixed, someone else on the same printer had just broken. The issue they had was the advanced options to their Xerox printer were not available. They could not change paper trays or print dual-sided. The types of issues that IT often does not see as a big deal but it is very important to the customer that it works.

I walked over to a third office where everything was working. I checked the driver and it was the same as the others. I then printed a test page from all three of them. Not only was the driver the same, it was even the same version number. But something very important jumped out to me. The list of dependent files only printed out from the computer that was working correctly. So it looked like they were missing all the additional printer driver files. 

I delete the printer and reconnected again for good measure and the test print was the same. I pulled up the printer management tool and tried to delete the driver, but it said it was in use. Access denied. Disconnected the printer and rebooted for good measure. It was still a problem. I finally got it removed by restarting the spooler and deleting the driver as fast as I could. It took me 5 attempts, but I got it. Connected to the printer and everything was working correctly. Did the same on the first broken user and it fixed them too.

Now I have a sure fire fix to the problem. It looks like IT witchcraft though. Stop spooler, start spooler, and delete driver. Repeat as needed. Sometimes it works the first time but most of the time it takes a few attempts. The user is usually sitting there with me.

Over the next few days, I fix a few more this same way. I want to dig into it deeper but printer drivers only break when someone need to print. The one day I had 3 break all within 10 minutes of each other near the end of the day. It was getting clear that we need to understand the issue better.

That very next day, one of the earlier users broke again. This time I made a copy of the spooler folder before the fix and after the fix. It turned out all the missing files were still there. Checksums were the same too. There goes the idea that the drivers are corrupt. For some reason Windows has the driver files but does not know it.

Edit: I was able to write a script to help with this issue: http://kevinmarquette.blogspot.com/2014/04/fixing-print-drivers-missing-dependent.html

Over the last 10 years

Interim Director of Information Systems and Learning Resources 02/2013 to present
Developer/Systems Analyst II 07/2006 to 02/2013
Workstation specialist 2003 to 07/2006

Proposed and implemented a 4-5 year equipment refresh policy. Architected and implemented a virtual desktop solution to reduce expenses. Implemented Server 2012R2 deduplication and storage spaces to extend the life of existing storage equipment and to acquire new storage at reasonable prices. Repurposed existing hardware for our backup plan to include full workstation backups for the entire college. Backup and disaster recovery systems were revised for better reliability and security.

Serve on the Leadership Council (2013-Current), Clinic Implementation Committee (2005-Current), Joint Privacy and Information Security Workgroup (2013-Current), End-User Device Security Team (2010-Current). Recipient of the Chancellor Council Silver Award of Excellence in 2012 for my role in the electronic dental record implementation.

Before serving as the Director, I filled many roles within Information Systems. As a Systems Administrator I architected, deployed, and administered Active Directory, extensive Group Policy Objects, DNS, DHCP, SQL Server clusters and availability groups, Hyper-V virtualization clusters, terminal servers, clinic system applications, fileservers and related share structures. Acquired and deployed into production entry level enterprise SAN for SQL and Hyper-V.

Owned the mission critical applications and systems. Served as the only database administrator managing vendor and internal databases. Handled system upgrades from Server 2003, to 2008R2, to 2012/2012R2 across our infrastructure. Executed the SQL cluster upgrades and hardware refreshes to our clinic systems from SQL Server 2000, to 2005, to 2008, to 2012. Implemented SQL Reporting Services (SSRS) and laid the foundation for a future data warehouse implementation.

Performed the initial implementation of Active Directory as a disaster recovery solution in 2004. Managed the migration of our users, computers, group policies, servers, and custom applications to another university domain in 2011. Assisted with the analysis, implementation, and ongoing support for a replacement clinic system in 2005 and for the digital record modules of that system in 2012. Implemented secure prescription printing solution. Performed the server side management and clinical integration of a MiPacs digital radiography system.

As a Developer, I work mostly with PowerShell, SQL, .Net, and aspx. My implementation of SSRS for reporting was to complement our clinic system. Worked closely with the business manager to define and implement the financial reports they required. Significant effort was also put into implementing grading, insurance, and research reports for the various areas of the college.

Developed other applications that interacted with the clinic system to increase user productivity. Many of my add-ons were used by the vendor at other sites or the features were integrated directly into the product. Some of those enhancements include a calendar style schedule for the clinicians, inventory management for the central dispensary, chart management for the records clerks, and a streamlined patient check in dashboard used by reception.

Wrote the design documents and was the project manager for a biomonitoring application we outsourced to Dell. Developed a DRM video player to protect human cadaver videos. Wrote an electronic insurance integration component for an internal clinic application. Built CD/DVD splash screens. Wrote USB activity auditing processes and various other odds and ends as solutions to issues the college was facing.

As a Workstation Specialist, I migrated us from XP to Windows 7 starting in fall of 2009. I assisted in setting up the imaging server and helped craft our master images. I am a strong advocate for least privileges so I spearheaded the effort to remove administrator rights from our users. I was able to leverage my developer experience to resolve issues applications had with either Windows 7 or restricted access. I now direct my own team on our Windows 8 and Virtual Desktop deployments.