A customer opened an issue with us, saying that his server emailed him a crash report. It did, and it was included in the incoming ticket. It was quite detailed and elegant in a way, well designed and prepared... and we had no earthly idea what it meant. The supervisor didn't know. The advanced support people didn't know. For awhile, the internet didn't – until I'd searched enough random lines in the report to come up with some hits on abrtd, and later it made sense due to the customer running CentOS. We though at first that it was a system crash from some 'abort daemon' but no, the name was an acronym: Automatic Bug Reporting Tool (daemon), a Red Hat/CentOS thing. The web hits further suggested that this tool watched for application crashes, not system crashes. We figured that uptime results would confirm that the box hadn't gone down, but the customer spent another two or three tickets finding the server password...
When the password was provided in the third ticket (I'd been the only one picking these up since I'd gotten the first one and was perversely curious), I got in and uptime confirmed that the box had been up for over 50 days. Then, I checked around the logs for something to correspond with the cryptic reports from abrtd. No luck. Fortunately, the net hits mentioned cli commands for this daemon, one of which was “list”. This was the jackpot, or at least the only promising lead. It gave six incidents (little report notices of about 5 lines each) with a timestamp and the app concerned in the fail. I recognized 5 out of six as normal for a CentOS box with cPanel, but #6 was odd: something or other ending in .py, and sitting in somebody's home directory.
Even a novice would suspect something amiss. I sprung into action and nicely suggested that perhaps he'd paid a developer to write this and might indicate to us what the script did, if anything. Also, the tool would probably flag as a problem some application activity that it hadn't been told about.
Success. The customer speculated that it was a local tool and that he'd look into it. Ticket retired.
The moral of the story is that these modern operating systems are so extensively appointed that you can be blindsided by features that no one knew about – even in a roomful of professionals. I am continually amazed at what I learn just by either answering a ticket, answering the phone or just standing around.
Wednesday, August 12, 2015
Monday, July 20, 2015
I Apologize In Advance For This Post (I swear the SecurityOnion test is coming!)
One should not have to think about this one, or write about it. I see server operations "professionals" almost five days a week that can't be bothered to lift a finger about security until after it's too late... but I've just seen something that's worse, if you can imagine that.
It appears that another major social media site has been hacked, but this one (which I won't dignify with a mention) is one devoted to the furthering of cheating. According to media reports it bills itself as “the world’s leading married dating service for discreet encounters”. In a world in which the internet is becoming more of a party line every day, the idea of establishing a central point for illicit activity would seem not to be a very good idea. But some evidently avail themselves of this.
Now, the guys I was mentioning before (Admins who slack off on security) are sometimes people who are intimidated by such simplicities as Windows Update. But simply being nontechnical doesn't mean being stupid. Often the uninitiated hire somebody else to handle that stuff. Fine and dandy. Some of them put it off until a problem impels them to hire outside help. Fine and dandy. That's one level of the problem. That's not as dumb as cheating on a spouse through a maybe-insecure site that publicly advertises its purpose as enabling illicit hookups.
This has implications for the security and server operations industries. How do you protect against a user base that pushes back frontiers in stupidity? This creates an internal threat that is dynamic (apparently getting worse by the year). Talk about a cost of doing business. This is the end of the post. I have no idea how to fix stupid (as the comedian Ron White says). I guess that making sure to do backups is the only answer, since by my logic, sooner or later (depending on how many bad hires one makes), you're gonna be rebuilding something.
It appears that another major social media site has been hacked, but this one (which I won't dignify with a mention) is one devoted to the furthering of cheating. According to media reports it bills itself as “the world’s leading married dating service for discreet encounters”. In a world in which the internet is becoming more of a party line every day, the idea of establishing a central point for illicit activity would seem not to be a very good idea. But some evidently avail themselves of this.
Now, the guys I was mentioning before (Admins who slack off on security) are sometimes people who are intimidated by such simplicities as Windows Update. But simply being nontechnical doesn't mean being stupid. Often the uninitiated hire somebody else to handle that stuff. Fine and dandy. Some of them put it off until a problem impels them to hire outside help. Fine and dandy. That's one level of the problem. That's not as dumb as cheating on a spouse through a maybe-insecure site that publicly advertises its purpose as enabling illicit hookups.
This has implications for the security and server operations industries. How do you protect against a user base that pushes back frontiers in stupidity? This creates an internal threat that is dynamic (apparently getting worse by the year). Talk about a cost of doing business. This is the end of the post. I have no idea how to fix stupid (as the comedian Ron White says). I guess that making sure to do backups is the only answer, since by my logic, sooner or later (depending on how many bad hires one makes), you're gonna be rebuilding something.
Monday, July 6, 2015
Two Items Actually Connected With Linux and/or Security And Technology!
First, let's get the Windows matter out of the way: I noticed something on one of my Win7 boxes that will be relevant to the security updates Redmond just pushed out. I'd seen McAfee VirusScan interfere with the installation of several KB items (updates) before (to fix it you just set the agent to STOP, do all the download/install/config stuff then set it back with START), but just experienced the same problem where McAfee wasn't on board. I'm trying out Bitdefender and realized that it has a comprehensive status control panel, on which it checks for, not only its own definition files but for things like Adobe updates and Windows Updates. In this case, when I checked for updates through Bitdefender's panel rather than through the ordinary OS updater (WU panel under Security in W7), it worked ok. Just sayin'.
Second, here's the page for a nice security-based distro some of you already know about:
Security Onion
It's based on the easy-to-use Ubuntu and includes tools like Snort, Snorby, Network Miner and Bro. I'll do a more in-depth analysis on it as soon as I kick the tires.
Second, here's the page for a nice security-based distro some of you already know about:
Security Onion
It's based on the easy-to-use Ubuntu and includes tools like Snort, Snorby, Network Miner and Bro. I'll do a more in-depth analysis on it as soon as I kick the tires.
Monday, June 22, 2015
3 Short Topics
Hmm. Several things technologic have cropped up with me lately. First, a hardware problem: I have several computers in an outbuilding, stored in less than optimal, rather humid conditions (there's A/C out there but not often running). At least one computer is 18 years old. Trying to start it up yesterday, I got a indication that the HDD could not be located (although a mouse cursor did appear successfully). I restarted and just got a blank screen. Three restarts later - same thing. How to fix it? What worked was to leave the power connected and come back in 30 min. Evidently, either something on the mobo "charged" up, or perhaps caps or resistors warmed up to a temperature they liked - or that something downstream from them liked. Startup went fine.
Another thing - a news story. Yet another break-in at some big organization compromised a bunch of information, this time lots of personal info per account, like medical data for thousands of users. What to do if that happened to you? I haven't read this in any industry publication but I'd think that it would be good to look at all data exposed, then separate the pile into these categories: what cannot be changed, what can be changed, and what is less consequential. Your balance in the office coffee fund is probably a member of the third category (although perhaps it could be used to track your activity) but the first category would include stuff like SSNs, DOBs, military discharges, traffic tickets and other matters of public record. The second category interests me. Logins as well as passwords, and even vlan's, IPs and so forth are things that don't have to be what they are. Though sometimes complex to change, they probably should be changed periodically.
And a third thing... on jobs in the tech area, there's a headline I saw recently that claimed that the amount of time that a job went unfilled went from 25 to 27 days on some average or other. Don't have much way of checking the validity of the claim, but assuming it's true, I wonder why it happens. Maybe there are fewer babies being born, maybe fewer immigrants are those of technical proficiency, maybe students are dumber, maybe teaching standards are lower... but what if it's something else? What if economic pressures on businesses and government organizations are such that fewer people have to do more work? And therefore must know more than in previous times? This would mean a greater reliance on guru-level workers that have never existed in vast amounts. Maybe. Just maybe.
Another thing - a news story. Yet another break-in at some big organization compromised a bunch of information, this time lots of personal info per account, like medical data for thousands of users. What to do if that happened to you? I haven't read this in any industry publication but I'd think that it would be good to look at all data exposed, then separate the pile into these categories: what cannot be changed, what can be changed, and what is less consequential. Your balance in the office coffee fund is probably a member of the third category (although perhaps it could be used to track your activity) but the first category would include stuff like SSNs, DOBs, military discharges, traffic tickets and other matters of public record. The second category interests me. Logins as well as passwords, and even vlan's, IPs and so forth are things that don't have to be what they are. Though sometimes complex to change, they probably should be changed periodically.
And a third thing... on jobs in the tech area, there's a headline I saw recently that claimed that the amount of time that a job went unfilled went from 25 to 27 days on some average or other. Don't have much way of checking the validity of the claim, but assuming it's true, I wonder why it happens. Maybe there are fewer babies being born, maybe fewer immigrants are those of technical proficiency, maybe students are dumber, maybe teaching standards are lower... but what if it's something else? What if economic pressures on businesses and government organizations are such that fewer people have to do more work? And therefore must know more than in previous times? This would mean a greater reliance on guru-level workers that have never existed in vast amounts. Maybe. Just maybe.
Friday, April 17, 2015
Be Your Own Network Administrator!
Whether it's the stories about the low-security Internet Of Things (IOT), or the stories about vulnerabilities aboard airliners running in-flight WIFI from the cockpit, it's not difficult to imagine problems from convenience arrangements that happen via connections that are always on, and that foster connections between stuff that doesn't go through people or devices that we monitor things on (but directly). That was a jawbreaker of a sentence. Lemme put it another way:
The lazy layman that turns on everything that's conveniently wireless is easy to see as heading for trouble if Ivan Ivanovich can get to his "Tax-O-Matic" files via the toaster. But just how bad could it get? Consider that the Information Technology field is chock full of security pros either hobbled by bad budgets and policies, or mediocre to bad in their job performance. Even banks are getting robbed remotely. Considering how badly the PROS are doing at safely running networks, what chance does Joe Sixpack have?
Well, there's probably no hope for the lazy ones since they don't lock their cars, either. Those who give these matters a thought at all, however, have some hope but a sobering realization: we must now be our own Network Administrators - that is, if we insist upon doing things over a network. This means two further things. First, for matters that need a network, one must know basic safe practices about routers, antivirus scanners, backups and such. Second, for matters that only optionally need a network, one should consider a non-network alternative.
Networks make lots of sense concerning computers, gaming platforms, streaming services and the legit sharing of various connections and resources. On the other hand, you can work a To-Do list in non-electronic forms quite easily. Printing out a hard copy of an item can often be avoided, but not always (certain concert tickets are in hand instantly that way, making will-call unnecessary). And as to online purchase alternatives, ever hear of cash paid in person? Sometimes big discounts are offered for that.
I'm no Luddite, longing for a return to the days of Fred Flintstone. But crappy security may make non-networked alternatives increasingly more attractive. After all, Usenet was a technology that served us well for awhile but died in an ocean of spam. Many say that that's now happening to email. If the internet becomes even more of a party line than it already is, it could become a vulnerability as such, rather than a tool.
EDIT: It's a bit untoward to end this post there... for two reasons. First, I think if the 'net becomes more dangerous than useful, it will become so only for the average and casual user rather than the pro. If one uses complex passwords, encryption (VPNs and such) and general common sense, then one should be able to do business. The casual will get bitten. Second, for the casual user, things could get worse as the hardware moves away from separate and recognizable computing devices (like CPUs, drives, desktop computers themselves, hardware firewalls, switches, hubs, routers...) and closer to the smart phone and nothing else. Most of us know someone who has already abandoned the desktop, laptop and home internet connection in favor of just the phone, never mind needing to beg access to somebody else's printer or flash drive when that's needed. Issues of routing, security and such will be even harder for the novice to conceptualize when only seeing colored buttons on a phone... but perhaps I've been too harsh on the learned, who will be likely to recognize annoyances.
The lazy layman that turns on everything that's conveniently wireless is easy to see as heading for trouble if Ivan Ivanovich can get to his "Tax-O-Matic" files via the toaster. But just how bad could it get? Consider that the Information Technology field is chock full of security pros either hobbled by bad budgets and policies, or mediocre to bad in their job performance. Even banks are getting robbed remotely. Considering how badly the PROS are doing at safely running networks, what chance does Joe Sixpack have?
Well, there's probably no hope for the lazy ones since they don't lock their cars, either. Those who give these matters a thought at all, however, have some hope but a sobering realization: we must now be our own Network Administrators - that is, if we insist upon doing things over a network. This means two further things. First, for matters that need a network, one must know basic safe practices about routers, antivirus scanners, backups and such. Second, for matters that only optionally need a network, one should consider a non-network alternative.
Networks make lots of sense concerning computers, gaming platforms, streaming services and the legit sharing of various connections and resources. On the other hand, you can work a To-Do list in non-electronic forms quite easily. Printing out a hard copy of an item can often be avoided, but not always (certain concert tickets are in hand instantly that way, making will-call unnecessary). And as to online purchase alternatives, ever hear of cash paid in person? Sometimes big discounts are offered for that.
I'm no Luddite, longing for a return to the days of Fred Flintstone. But crappy security may make non-networked alternatives increasingly more attractive. After all, Usenet was a technology that served us well for awhile but died in an ocean of spam. Many say that that's now happening to email. If the internet becomes even more of a party line than it already is, it could become a vulnerability as such, rather than a tool.
EDIT: It's a bit untoward to end this post there... for two reasons. First, I think if the 'net becomes more dangerous than useful, it will become so only for the average and casual user rather than the pro. If one uses complex passwords, encryption (VPNs and such) and general common sense, then one should be able to do business. The casual will get bitten. Second, for the casual user, things could get worse as the hardware moves away from separate and recognizable computing devices (like CPUs, drives, desktop computers themselves, hardware firewalls, switches, hubs, routers...) and closer to the smart phone and nothing else. Most of us know someone who has already abandoned the desktop, laptop and home internet connection in favor of just the phone, never mind needing to beg access to somebody else's printer or flash drive when that's needed. Issues of routing, security and such will be even harder for the novice to conceptualize when only seeing colored buttons on a phone... but perhaps I've been too harsh on the learned, who will be likely to recognize annoyances.
Wednesday, April 15, 2015
Top Ten Server Rental Mistakes [this week anyway]
1. "Drives can be resized on the fly without a reload."
--- probably a misinterpretation encouraged by programs like Partition Magic, which just make use of unassigned space rather than actually moving a partition. Even some virtual server offerings are in specific sizes only, so as to mimic physical servers (as opposed to other storage options which might be by the meg or gig).
2. "The “cloud” is a backup." --- nope. And I have no idea why some believe this. Wishful thinking?
3. "The “cloud” is something new." --- nope. A network's a network by any other meme.
4. "Any form of storage/CPU should be as fast as I need/do what I want." --- nope. You pays for what you gets. And don't get the cheapest storage and then want HA performance or try to run PBX/telephony or video.
5. "The hosting company is my administrator." --- nope. If you aren't specifically paying an administrator, nobody is your administrator.
6. "Reboots shouldn't be necessary." --- depends. Some systems rely on it more than others. And if all you're doing is reacting to preset email alerts and not bothering to log in, who are you to know?
7. "Reloads or backups shouldn't be necessary." --- nope. You or your administrator had better save all data and configurations or at some point it'll all be over but the crying.
8. "Security just gets in the way." --- and that's what it's supposed to do.
9. "I'm in a hurry/don't write anything down/get on with it." --- nope. I'm gonna document and cover my ass, and surprise! We're recording this conversation.
10. "This is easy; you shouldn't have to research it. You should be able to do it right now." --- Dear Reader: I'm not making this one up; I've actually heard it - a logic hole big enough to drive the Hindenburg through.
Sunday, March 8, 2015
"The least important part of the computer is the operating system."
The titular quote above, opening this post, is something an old friend long ago said to me, a few years before he got hired by the storied Bell Labs (then known as Lucent, later Alcatel/Lucent). His point was that an OS is just something to enable human interaction, plus something that provides a context for additional applications post-setup. The real work is done by CPU, memory and other hardware. Even within the OS, the human part could be considered secondary, since the kernel does the core duties of file system management, memory management and job stack scheduling (to use antique terms). And if the kernel, needed programs and related libraries are working fine, who needs GUIs or even terminals? Put the whole thing on a card and have a remove/replace guy power down, switch for new part, power up and rock on, dude. And if you must, virtualize the whole thing, script that replacement process and you have the same freedom from interactivity (with three additional problems if you're in the cloud: hypervisor vulnerabilities, API vulnerabilities and access at the cloud provider by staff members whom you don't control).
But I digress.
I understand that personal computers are tools that allow humans to get stuff done, so I'm not telling you not to "startx", so to speak (to use a graphical user interface). After all, I'm now posting on a board related to a User's Group that is dedicated to a particular OS, which is FOR human interactivity. I'm merely advocating a concern for computing above and beyond the which-OS-is-better debate. I certainly have an opinion as to what's better or worse, but in point of fact, I use any and every system even remotely popular. That's due to curiosity as well as for professional reasons. The OS choice, while not unimportant, should be made with an eye toward ultimate use. What do you need to do? Why do you need to network from A to B? Is security a concern - or not? Is the computer for a narrow technical use or a general-use entertainment device? Is location or compatibility a concern? Is a computer or network even necessary? The OS question is only one of many questions on the way to the success of some ultimate use, project, activity or work.
But I digress.
I understand that personal computers are tools that allow humans to get stuff done, so I'm not telling you not to "startx", so to speak (to use a graphical user interface). After all, I'm now posting on a board related to a User's Group that is dedicated to a particular OS, which is FOR human interactivity. I'm merely advocating a concern for computing above and beyond the which-OS-is-better debate. I certainly have an opinion as to what's better or worse, but in point of fact, I use any and every system even remotely popular. That's due to curiosity as well as for professional reasons. The OS choice, while not unimportant, should be made with an eye toward ultimate use. What do you need to do? Why do you need to network from A to B? Is security a concern - or not? Is the computer for a narrow technical use or a general-use entertainment device? Is location or compatibility a concern? Is a computer or network even necessary? The OS question is only one of many questions on the way to the success of some ultimate use, project, activity or work.
Subscribe to:
Posts (Atom)