Thursday, April 19, 2018

Confession: Handwarming

Confession: Sometimes I turn on Kerbal Space Program on my laptop just so the keyboard will get hot and warm up my hands...

Tuesday, February 20, 2018

Tank 432 is a terrible movie.

Netflix is pretty good about predicting movies I'll like, and it offered me "Tank 432" on my day off yesterday.  This is a terrible movie; save yourself.  I wish I could have the time spent watching it back for something else.

Thursday, January 4, 2018

Changing the battery on a 2011 Chevy Equinox

My dearest tried to start our SUV the other day and it was dead.  The OEM battery has been a faithful trooper, but it died.

So there is a problem with this.  The manufacturer decided to play hide-and-go-seek with the battery.

Here's what it looks like under the hood.

That big red cap is not it.  That's a place to connect jumper cables.

The battery is under the ECU, wrapped in a metal battery box so you can't see it.  It's a pretty good hiding spot.

Changing it is pretty easy.

On the right (non-connector) side of the ECU there is a plastic clip that holds the black plastic cover on.  Unclip it and remove the cover.

With that out of the way there is a bolt on the left side of the ECU that holds it in place.  Remove this bolt with a 10mm socket.  You do not have to disconnect the electrical connectors.


With that removed, slide the ECU right and up to disengage it from its mounting hooks and you can swing it free.

The battery is held down by a metal strap with a visible bolt, and a second bolt that's buried under the front plastic fairing.  You can undo the one visible bolt and bend the strap up and out of the way, or you can use a plastic clip remover tool like this one to remove the plastic cover and get to the other bolt.

This is the plastic clip remover set I want.   I know I've spent more than $12 replacing broken clips from using the wrong tool for this. :/

I'm going to have to get that

Back to the task at hand.  With the strap removed you can finally get to the battery.  Prudence suggests disconnecting the negative battery lead first, so you don't cause a shower of sparks if you accidentally put your wrench between the positive lead and the chassis.  I used a 10mm wrench to loosen this, a socket would not fit.  The negative terminal had a funny little angle-block clamp that I've not seen before.

Installation is the reverse of removal.  Connect the positive lead first, then the negative.  It's probably worth putting some vaseline or battery terminal protector spray on them to prevent corrosion given how difficult it is to get to this.

I hope this helps.  You'd think stuff like "Where is the Battery?" would be answered in the Owner's Manual, but it isn't.  It's remarkable that a book that thick can be so useless.

Tuesday, December 26, 2017


Today we gave away Dusty the keyboard warmer. 

He was one of four kittens that a stray had in our Garage; Dusty, Reeses, Patches, and Aota.

Goodbye friend.

Sunday, November 5, 2017

... and the ads are gone.

... and the ads are gone.

I've pulled adsense, and thank you for your putting up with that.  I may still occasionally drop in an affiliate link to something on Amazon, but it won't be on every page like it was before.  I usually surf with an adblocker, and didn't realize how bad it was.

Thanks again,

Monday, October 30, 2017

Removing Adsense

About a dozen years ago I signed up for AdSense on this blog.  Here is an odd thing about AdSense; They have a $100 payment threshold.  As of today I'm $0.80 cents away from that magical threshold for the first time.  As soon as I get that check I'll be removing the ads.  One more month should do it.  Thank you for putting up with them for so long.

I will occasionally post links to stuff on Amazon that is relevant, but I'm done with intrusive advertising after this. 

Thursday, October 12, 2017

Fixed: ACK/RST intermittently connecting to RPC/Kerberos/LDAP ports on a Server 2012 R2 Domain Controller

I worked an interesting case this week.  My customer had widespread domain controller outages on their Windows Server 2012R2 DCs.  A network trace showed that connections to the domain controllers were intermittently being refused with an ACK/RST (acknowledgement/reset) in response to the initial SYN (TCP Hello) packet.

It only happened under load.
It wasn't a firewall.
It wasn't the AV software.
It wasn't a laundry list of other things.

Here is the odd bit.  In a netsh trace on the DC, we could see the SYN packet was getting to the DC, but it wasn't getting to the RPC endpoint mapper, lsass, or netlogon.


That took the weekend and Monday to figure out.  What we discovered was there was a TDI filter driver in the networking stack that was, for cause unknown, slowing down network connections to the DC.

TDI "Transport Driver Interface" is an older Windows technology that lets software drivers hook between the TCP/IP driver stack and applications.

Further digging led us to find this KB article from Trend Micro that indicated a problem with the VmWare NSX Network Introspection Driver (vnetflt.sys) was causing the trouble.  This driver is installed with the VmWare Tools Package on the DCs, and it was several years out of date.

The solution from the KB worked.  We disabled that driver via the registry and the problem stopped immediately after a reboot.

Specifically, in the registry:
Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\vnetflt\
Key: Start
Value: 4


It's interesting to note that I haven't seen or heard of any other similar issues with this.  I assume it was an interaction between the ancient VmWare tools on the box and some of their security software or a Software Update, but I'm not sure.  They wanted to do that root cause analysis in their lab independently.  I asked them to cc me on the root cause, and I'll update this blog post if they do.

Lessons Learned:
  • You really should keep your VmWare tools updated.  
    • Like all drivers, a fire and forget mentality can seriously bite you.
  • You can convert a netsh network trace ETL file to a big text file with this command:
    • netsh trace convert input="input.etl" output="output.log" 
  • You can capture a netsh network trace ETL and add in extra ETL providers, like the TCPIP driver messages, with this command: 
    • netsh trace start capture=yes overwrite=yes maxsize=2048  tracefile=c:\output.etl scenario=netconnection provider={EB004A05-9B1A-11D4-9123-0050047759BC} keywords=0xffffffffffffffff level=0xff provider="Microsoft-Windows-TCPIP" keywords=0xffffffffffffffff level=0xff
  • The Potbelly Sandwich Shop in Normal, Illinois makes a fantastic grilled cheese sandwich.