Your basic ITPro blog... What's going on at work, what I'm interested in.

Thursday, August 28, 2008

New Love for One-Liners

Hey Scripting Guy! answered a question about 'svchost' and what services this process is actually hosting. I've seen svchost listed in Task Manager, but never really put any thought into what it was doing there.

In any case, the Scripting Guys posted a script, about ten lines long, that reports the services running in each svchost PID. Check it out at the link above.

Now, if you have been reading along, you know that I am trying to learn to think "pipeline" when it comes to Powershell. To that end, I thought it would be fun to try and come up with a one-liner to output the same information. Here's what I came up with.

gps -n svchost | %{$_.Id} | %{gwmi Win32_Service -f "processID = $_"} | group processid | %{$_.group} | select processID, Name, state, startMode

It's not a short one-liner, but it returns essentially the same information as the script. In case the line is too long:

gps -n svchost | %{$_.Id} | %{gwmi Win32_Service -f "processID = $_"} | 
group processid | %{$_.group} | select processID, Name, state, startMode

Wednesday, August 27, 2008

Programming vs. Scripting

The famed 'Powershell Guy' visited again... and again I am blown away. I posted about working on text parsing and included my script to solve the problem. My buddy did basically the same task in Perl in just a couple lines of code. I thought that was cool. I couldn't even imagine a way of doing it in PoSH with that little amount of code.

But then, I'm not MoW... He drops the following in the comments:

gc iptest.txt |? {$_ -match 'REMOTE_ADDR: (\S+)'} |% {$matches[1]} | group | sort count

Very, very cool! (BTW, the second one-liner you posted errors out with a 'Cannot index into a null array' message. Odd.)

This prompted a great discussion (after a few moments marveling over the one-liner) about the differences between the one-liner and the script. I think in scripts... assign values to variables; compute against those variables, assigning the results in more variables; if-then's and loops; etc. It is never my first inclination to start with a one-liner. I realize that this is in large part to my lack of background at the command-line (and newness to Powershell and scripting in general).

I want to be able to do this. The pipeline (the heart of the one-liner) is very powerful. I need to learn how to use this tool. There is a time for a structured, formal 'solution' and there is a time when you just 'need it'. MoW's one-liner answers the question and gives me what I need. For the one-off request, this is the way to go. If this turns out to be analysis that I am doing on a regular basis, a script will be more appropriate; formatting output, reusable, etc.

Episodes like this make me more frustrated and more excited about my job than ever!   :-) I love this stuff!

Tuesday, August 26, 2008

Perl, PoSH, Processing Logs

Or, maybe a better title would be, "Beets, Bears, Battlestar Galactica". In either case...

We had a very interesting item come up today that, among other things, introduced me to Perl, gave me a renewed appreciation for one of my co-workers, and made me think about firewall management.

We are running Arena here. Recently, we got word from them that our system was throwing a bunch of strange exceptions. When Arena has an exception, it reports it back to 'the Mothership'. Anyway, after some digging, we determined that the exceptions were attempted SQL injection attacks.

This is pretty scary, but it seems that our system knows enough about these attacks to throw the exception rather than attempt to process them. That's the good news. So, our next thought was, "How can we identify the attackers and can we cut them off"

Manually looking through the logs, we found an entry in each record that read: REMOTE_ADDR: xxx.xxx.xxx.xxx

We had found our attacker... or one of them. It turns out that, over the past few days, we had over 1800 such attacks! We needed a way to get a list of these attacker IP addresses. I thought this would be a great exercise for me, especially since I haven't done much in the way of text parsing/processing (in PowerShell or anywhere else). So, I got a copy of the offending log entries (a 6MB text file) and started trying to figure out how to find what I needed using Powershell. Mind you, I am new to text processing and regex, so this was a bit of an adventure!

After some research and a few iterations (and input from my aforementioned co-worker), I came up with this:

$content = Get-Content u:\scripts\SQLHackAttempts.txt
[regex]$regex = 'REMOTE_ADDR: (\S+)'
$allHits = @{}
$hits = $regex.Matches($content)
foreach ($hit in $hits) {
    $ip = $hit.Groups[1].Value
    if ($allHits.ContainsKey($ip)) {
        $allHits.Set_Item($ip, $($allHits.get_Item($ip) + 1))
    }
    else {
        $allHits.Add($ip, 1)
    }
}
$file = 'u:\scripts\ParsingOutput.txt'
$allHits.GetEnumerator( ) | Sort Value -Descending | Tee-Object $file
$footer = @"
--------------------------------------
Total records in log: $($content.count)
Total Unique IPs: $($allHits.Count)
"@

Out-File -FilePath $file -InputObject $footer -Append
'--------------------------------------'
"Total records in log: $($content.count)"
"Total Unique IPs: $($allHits.Count)"

This goes through the log entries, finds the proper IP address, and reports back (to console and to a file) each unique IP address and how many times it showed up. The tail end of the report looks like:

image

Yes, 704 unique attack IP addresses used in 1869 individual attacks! Wow.

This was a great exercise, and a lot of fun... until, that is, I saw Nick's Perl script that did pretty much the exact same thing. Here's his code:

-------------------------------
I run it like this:

perl getIPs.pl < input.txt > out.txt

Here is the code:

while(<>)
{
    $hash{$1}++ if ( /REMOTE_ADDR: (\S+) / );
}
foreach $ip ( sort {$hash{$b} <=> $hash{$a}} (keys(%hash)) )
{
    print "$hash{$ip}\t$ip\n";
}
--------------------------------

I have never worked with Perl before, so this took a bit of explanation. But I thought it was pretty cool! I knew Perl was really good at text processing, but this still blew me away.

Now, after using either method, we had our list of attackers. The question now is, what do we do with this? I don't think a 700+ line ACL on my firewall is that great of an idea. But, I want some way to deny these addresses access to our systems.

Thoughts? Suggestions? Improvements on my PoSH code?

Wednesday, August 20, 2008

Rabbit Hole... Or Dominoes...

Hey there... I saw this little PoSH script on one of the blogs I track... (Minor tweaks made by me)

-------------------

$Printers = Get-WmiObject `
    -Class Win32_PerfFormattedData_Spooler_PrintQueue `
    -ComputerName 'ServerName' `
    -Filter 'Name <> "_Total"'
foreach ($Printer in $Printers) {
#    if($Printer.Jobs -ge 10) {
        Write-Host $Printer.Name $Printer.Jobs
#    }
}

-------------------

It uses WMI to poll the print queues on a computer.

This got me looking at What's Up Gold. I knew you could set up WMI monitors in WUG, but have never done so. So, I created some monitors in WUG that monitor these same WMI items.

I am curious to see how these look/work over a weekend as we run our check-in system through its paces.

Cool, though, how one technology makes you think of how you might use another technology differently. I had never really looked in to using WMI monitors in WUG. But, as I have been learning more about PoSH, I have been increasingly exposed to WMI. Fun!

Tuesday, August 12, 2008

Which Bits Are Going Where?

I got to be part of an impromptu exercise this morning that proved very interesting and informative. Nick and I took a few minutes to look at the traffic coming and going within our Cisco 5510. This box has three primary interfaces; INSIDE, OUTSIDE, and DMZ. We were curious as to how much traffic was coming/going on each of these interfaces. With the help of What's Up Gold, we were able to look over historical numbers for these interfaces and do some interesting  math.

Here's the obligatory graphic (please forgive my lack of artistic ability):

20080812diagram

The transmit and receive numbers are 30 day averages. We had to make some assumptions, one being that just about all of the RECEIVE traffic on the OUTSIDE interface was bound for the INSIDE interface, becoming TRANSMIT traffic there.

As you probably can guess, the INSIDE interface services our LAN, including our mail server. The OUTSIDE interface goes to our ISP. The DMZ interface primarily services our web server.

I know these numbers (and resulting analysis) is available in the syslogs from the 5510. I have tried using Splunk, but just have not had much success getting valuable reports/data out of it. Anyone know of any other good tools?

Monday, August 4, 2008

More to Learn

I had a small task today... Replace one drive letter with another in a bunch of text files. Specifically, I was moving my PowerShell scripts to a new drive and I wanted to change any references to the old location in my scripts to point to the new location.

First of all, this showed me how annoying it is to hard-code paths in scripts! There's gotta be a better way. (Hint: This is a solicitation for comments!)

Second, I ran into an oddity that I think is a result of some RegEx-related reality.

Here is the script that I ended up using to search all of my .PS1 files and replace the test string "c:\scripts" with "u:\scripts".

$files = Get-ChildItem -Recurse "U:\Copy of scripts\*\*.ps1"
foreach ($file in $files) {
    $content = (Get-Content $file) -replace "c:\\scripts", "u:\scripts"
    Set-Content $file $content
}
The oddity is the double-backslashes in the quoted text after the -replace. After reading a bit out RegEx (still and very new topic to me) it looks like the '\' is used as an escape character. So, to actually search for that character, you need to double them up.

Additional Info

My photo
email: support (AT) mangrumtech (DOT) com
mobile: 480-270-4332