Troubleshooting

I’ve been testing Microsoft’s Direct Access at work.

In my opinion it’s a true VPN killer, if you only need to deal with Windows Technology in your workplace.

Direct Access uses certificate/IPSEC based encryption/authentication to authorise a remote user and allow them to access a corporate network. The key thing here though is that the remote user doesn’t interact with any software to authenticate. The user’s authenticated without any action on their part – all they need to do is insert their username and password at the login screen much like you would if you were logging onto a terminal in the office.

No more fumbling around with a VPN dongle – if you have a broadband connection you can login and access network resources at the office from home.

I have noticed some problems using Direct Access with a Vodafone 3G/HSDPA connection though so I’d like to share what I did to get the service to work because it can be a pain in the butt to trouble shoot and the advanced diagnostic logs can be hard to decipher.

Firstly make sure you’re using the latest version of your Vodafone Mobile Broadband software (formerly called Vodafone Mobile Connect). I use version 10.2.302. Type “Vodafone Mobile Broadband Software” into Google and your first hit should take you to a page with the latest version. It’s important you get the latest version installed – I initially tried Vodafone Mobile Connect 9.4.6.20539 and couldn’t get Direct Access working no matter what.

The second part of the puzzle is 6to4. I had to completely disable this virtual adapter to force Direct Access to make a connection via Teredo. My suspicion (and this only seems to be a problem with mobile broadband connections) is that Direct Access doesn’t auto-configure the 6to4 adapter address properly and as a result Direct Access doesn’t fail over to any other connection type if it can’t communicate with the 6to4 adapter.

So start a CMD prompt with “Run as administrator” – as below. It’s important you start the CMD prompt with “Run as administrator” as a domain user even with local admin privileges will not be sufficient.

Once you have the CMD prompt up use the following command to disable 6to4:

netsh interface 6to4 set state disabled

And Direct Access should miraculously connect

The following command will take you back to where you started:

netsh interface 6to4 set state enabled

Have fun – I hope this helps you out of any Direct Access problems you may be having

I had problems with a virus recently making posts to my Facebook profile of all things. When I first noticed the problem I started going through in my head what I did in the past couple of days to expose my laptop to malware. The cause was simply down to being lax updating my AVG antivirus.  Always, Always, Always keep your anti-virus up to date

So I got home after my work day and set to work killing this virus. First came MalwareBytes which detected two malware and one trojan.

It was then time for an AVG update. After one definition update I ran the updater a second time and more definitions were installed (I’d been very lax)

All was well until I restarted the laptop. Which is when my problems started. The laptop wouldn’t boot back into the OS.

I could detect a bluescreen but couldn’t see what the error message was.

If this happens to you boot into F8 (Safe Mode) then choose the option to “Disable automatic restart on system failure” – this will give you a chance to see what’s causing the blue screen.

In my case it was an Unmountable_Boot_Volume.

I can only put this down to the virus corrupting my MBR when Malware Bytes removed it. Nasty…

So I set about booting into my WinXP setup cd to access the recovery console and run the fixmbr cmd to repair my hard disks Master Boot Record. This had no effect however. Fixboot was a non starter also.

It was at this point that I began to think I’d need something more powerful. My second thought was Microsoft must have a fix for this?

CHKDSK – chkdsk really? The program that runs when your PC restarts after it’s power plug caught your foot? Yes – That chkdsk….

http://support.microsoft.com/kb/555302

chkdsk driveletter: /f

In my case chkdsk f: /f

Damn if it didn’t work! Here’s what I got back….

I’ll never underestimate chkdsk again…

The following scenario highlights the importance of having regular server status reporting in place for unexpected events.

For a couple of months at irregular intervals we had users complain that some mails (not all) they’d been expecting from external users were arriving days later or were not arriving at all.

I’d been examining the headers of delayed e-mails when they did arrive and had found that external mail servers delivering e-mails to ours were taking a number of hours or days to do so. Most of the delayed e-mails “seemed” to originate from Eircom (an ISP based in Ireland) so I chalked the problem down to an issue on Eircoms side. Our Security team agreed. This was an oversight on my part however which I’ll explain later…

As I examined more closely I was able to pull more delayed e-mails from external users whose e-mail were not routed through Eircom so logically this meant the problem wasn’t specifically with Eircom.

As a test I had some of the users that reported delays e-mail my employers e-mail address whilst CCing in my personal gmail address. Eventually the issue reproduced itself.

Mails arrived in my Gmail Inbox within a couple of minutes whilst e-mail destined for my employers e-mail address arrived a number of hours later. In the end I was able to reproduce the issue with the help of 4 seperate senders.

Convincing the Security team was a problem however as everything looked fine on our end. They had a hard time accepting the issue was on our side even though I could reproduce the problem with four seperate senders. Four seperate external mail servers behaving in the same way at the same time? and the problem is not on our end?

They needed more evidence…

So the next port of call was Eircom – They were able to provide the following logs. IP addresses and e-mail addresses removed:

2010-07-13 17:22:19.744009500 starting delivery 2146375: msg 3317173 to remote [email protected]
2010-07-13 17:28:59.581941500 starting delivery 2147602: msg 3317173 to remote [email protected]
2010-07-13 17:49:00.077173500 starting delivery 2150674: msg 3317173 to remote [email protected]
2010-07-13 18:22:19.009197500 starting delivery 2158998: msg 3317173 to remote [email protected]
2010-07-13 19:08:59.078851500 starting delivery 2171856: msg 3317173 to remote [email protected]
2010-07-13 20:08:59.270208500 starting delivery 2188502: msg 3317173 to remote [email protected]
2010-07-13 21:22:20.122931500 starting delivery 2203875: msg 3317173 to remote [email protected]
2010-07-13 22:48:59.138197500 starting delivery 2224308: msg 3317173 to remote [email protected]
2010-07-14 00:28:59.064738500 starting delivery 2244997: msg 3317173 to remote [email protected]
2010-07-14 02:22:19.239298500 starting delivery 2261581: msg 3317173 to remote [email protected]
2010-07-14 04:28:59.989719500 starting delivery 2279211: msg 3317173 to remote [email protected]
2010-07-14 06:48:59.004682500 starting delivery 2299239: msg 3317173 to remote [email protected]
2010-07-14 09:22:19.021063500 starting delivery 2342483: msg 3317173 to remote [email protected]
2010-07-14 12:08:59.033052500 starting delivery 2428140: msg 3317173 to remote [email protected]
2010-07-13 17:22:19.744009500 starting delivery 2146375: msg 3317173 to remote [email protected]

2010-07-13 17:24:09.861329500 delivery 2146375: deferral: Connected_to_<IPAddress>_but_sender_was_rejected./Remote_host_said:_451_#4.1.8_Domain

_of_sender_address_<3rdparty@theircompany.ie>_does_not_resolve/

From the above you can see that Eircoms gateway had attempted to deliver the mail in question numerous times over two days but was being rejected because the senders e-mail address could not be resolved.

So I asked the Security team to investigate if any work had been carried out on our DNS infrastructure (or any failures) on the 13^th that would prevent the Ironport performing a DNS lookup.

They found that one of the DNS servers which our Ironport server was actively using for sender verification was restarting intermittently causing our Ironport to drop connection attempts when the external mail server attempted delivery. 

They didn’t have status reporting in place for the DNS servers to report any unexpected events.

Moral of the story – don’t rely on e-mail headers to judge mail delivery attempts (they only indicate successful connections) and make sure you have status reporting in place.