Thursday, December 11, 2008

The Story of an Insider - Part 2. The Sys Admins Story

This is my second part to a fictitious story about a theft of Intellectual Property by an insider and the detection and investigation of the incident.  The intro for this story can be found here and part one here.

The Sys Admins Story

So I get a call that Mark has locked himself out of his PC, well I guess when I implement password restrictions there's a price to pay. He says he never put the wrong password in though, yeah right, that's what they all say. I'll just check it out in the logs anyway.

Strange, that's not a design computer he's logging on from. Oh here he comes now, I'll ask him.

Man, I can't believe that, I just got my ass chewed from his boss for the last 10 minutes because HE forgot his password. Well I've got stuff to do, maybe I'll take it up with him later when he's in a better mood.

The next day......

Well something is definitely going on. Either this is a wind up or someone is trying to hack my network. I've had nearly every manager kicking my ass today about the lock out policy, all within the last few hours, I've got a stack of reports to write and patches to test and now my boss wants me in his office. This is just great! I'll quickly raise the support ticket and get this on the system so I don't get an ass kicking for that too.

Well, I've been told in no uncertain terms that either the account restrictions go or I do. I tried to explain that this is more likely an attack of some sort because I have changed nothing on the network. No new software, no new policy settings, nothing. But it's no good, he wants the lockout policy gone immediately. It's a total knee-jerk reaction and he would not listen to reason.

Oh god, I hope it's not a virus or something, I better check the logs and see what's going on. We'll I found nothing in the AV logs, but there is definitely something wrong. In my Server event logs I can see loads of lockouts today.

I used EventCombMT.exe to get all the lockout events and export them to a file.

Eventcomb will rip through the eventlogs on the server and extract into text files just the event's I'm interested in. From the results I can see that accounts are being locked out, but from the IP of the OWA box.

It must be that its infected with a virus. There was only that PC in buying that got an account locked out too, I bet that's got the same virus.

Well from the logs on the OWA box I can see all the lockouts are coming from something trying to get in using valid accounts with a wrong password.

It seems it's always an iPhone that is locking out the accounts, I'll just grep through the other logs with PowerShell to see if this has happened before.

gc *.log | select-string "iphone"

It hasn't happened before but in the older logs I can see that an iPhone has only been used once before, and it was Pete in buying that used it. Interesting!

I still don't have too much to go on but doing something has to be better than doing nothing right! Maybe the PC in buying does have something to do with it.

Again the AV checks out okay and is up to date, the firewall has pretty tight egress filtering and those logs are pretty clean. If there was something bad on the network I would expect it to show up on the firewall or the proxy logs and there's nothing.

I checked the PC is up to date on security patches. Let me run a script on the PC and the OWA box to see what's running, there must be something wrong. I have a batch file I created to respond to incidents such as this.

REM Usage: filename.bat ip-address/hostname outputfile
@echo off
set header1= **** Voliatile Data Gathering Script ****

type get-vol.bat > %2
REM Start Date & Time

time /T >> %2

date /T >> %2

REM System

psexec \\%1 systeminfo >> %2

REM Processes

psexec \\%1 tasklist >> %2

psexec \\%1 tasklist /svc >> %2

REM Networking

psexec \\%1 ipconfig /all >> %2

psexec \\%1 arp -a >> %2

psexec \\%1 netstat -anob >> %2

psexec \\%1 nbtstat -s >> %2

REM Finish Date & Time

time /T >> %2 date /T >> %2

The script isn't perfect because it's using the executables on the PC which could have been compromised, but it'll do for now until I can get round there and run something locally.

remote-info.bat gt-buy-01 c:\results-gt-buy-01.txt

The script runs commands that gets me details of the running processes, network connections, logged on users and all that sort of stuff. It's pretty good and I can really do with knowing what's happening on the PC right now.

Well the script ran fine, no weird processes, but there were some network connections to the file server in Design. Looking at my results file I can see that the NBTSTAT -s command shows some pretty big data transfers from the File-Server to the buying PC.

That's pretty weird, buying doesn't access files on that server they have there own server. I also saw that Mark from design had logged onto that PC. That's strange too, why would the Design Manager log onto a PC in buying to access his files? I'll update the Support Ticket and give him a call.

Well Mark from Design knew nothing about logging into a PC in buying. He said he has never logged onto any PC but his own and he is sure he has never given out his password.

I went back to my logs and I saw that my little HoneyPot has been tripped as well.

It's just a folder I set up that has object access logging on it. All the guys in design leave it alone and no one has any reason to go to it. That is no one goes to it if they now what they are looking for. Obviously someone doesn't know what they are looking for. Now I'm getting somewhere!

Now I know there is definitely something going on, and it looks as though someone has been accessing files that they shouldn't be.

I quickly check the profiles on the buying PC and sure enough it backs up what all the other tools have told me, Marks login has definitely been used on the buying PC.

I've informed my manager of what I have found, he has emailed me giving me permission to examine the buying PC whilst he has a chat with Pete in his office.

My goal at this point is to find out if an incident has occurred and then discover how this incident happened.

Coming Next.............. Sys Admin plays CSI.


Ken Pryor said...

Another excellent story. I really enjoy reading these. Can't wait for the next installment!

James said...

Really enjoying this story, cant wait for the next installment.

markus said...

I really enjoy reading these stories, they show real-life examples and help to take notice of things like that.

Great job

Greetings from germany

LonerVamp said...

Awesome post! That first part about the knee-jerk reaction is a scary (and too common) approach to security. Too much work, too little time, and decision-makers who just want to lessen the immediate pain. The sysadmin's follow-up work is why I strongly advocate not working your admins/security dudes 100%, but rather give them free time to investigate things like this!

SynJunkie said...

I'm really glad you guys have enjoyed the story and taken the time to comment with you support.