This is a perfect example of what not to do: the guy has ignored the lessons of denyhosts (etc.) and came up with his own regular expression that doesn't do the right thing.
Works out of the box to prevent brute force attacks against ssh, and can also be configured for other services, like web authentication, POP, IMAP, etc.
Just a friendly tip: Try to verify whether Fail2ban actually works and keep monitoring your log files (Logwatch). I once found some dumb ass trying to brute force his way into my Postfix server, unfortunately Fail2ban didn't catch it and I only noticed when Logwatch showed me the authentication failures (several hours and thousands of attempts later).
Just for the record, Pastebin is not completely secure. Even private pastes can still be accessed if you know the URL - they aren't password protected like your account may be.
But like you said, if you posted the key alone anonymously, it will be hard to guess what lock it fits.
> Just for the record, Pastebin is not completely secure.
Note that it already happens that people put their private keys into pastebin, by accident or probably by sheer incompetence.
> if you posted the key alone anonymously, it will be hard to guess what lock it fits.
As an attacker, I would simply collect as many of those "published" private keys as I can get. Then, when attacking a bunch of systems, I'd simply try one key after another on each system.
The important difference to the "real world" is that an attacker can try lots of "locks" at once (i.e. can connect to multiple target systems at once).
And this means you can add their public key as well, meaning you can control (i.e. revoke) their access easily and don't have to give them any root passwords.
Maybe a play off of In case of emergency, break glass with hammer and copy private key: a one-time-only retrieval of a private key at some URL, so when your buddy retrieves it, it triggers an email that goes out to you.
I recently began trusting KeyPass with my credentials.
It uses two mechanisms to protect my details:
- a secret key file (backed up on an usb stick and on dropbox). If you steal my keypass db you need this file as well
- a ridiculously long sentence that I can never forget for various reasons as a passphrase. It's plain text, so probably everything in there is in a dictionary, but - good luck trying to figure this out.
With this set up I'm relaxing now. I know that I ~cannot~ lose access or forget my passwords :)
Me too. But if it has a password on it, you'll be protecting your password with another password. If it's just an obfuscated, unprotected URL, then you're going to have to store the obfuscated URL somewhere safe.
I think this only works if you want to give credentials once to someone while still knowing when those credentials are being given, unfortunately :(
Privnote (https://privnote.com/) can do exactly that. For added bonus, the message is stored encrypted on their servers and decrypted with a key in the URL's fragment.
"We store the notes encrypted in a way that only the sender and receiver know how to decrypt, which means we cannot read the notes sent through Privnote."
This is a pretty pointless idea. If someone has root on your machine for even 500ms that's already too late to trust any of its state without a complete reinstall (and even this is not enough for an advanced attacker).
Inevitably, if you have such a buddy, all they can really say is "yup, I'm not the only user here", and you're back at square one.
A better investment of the time would be systematizing your configurations, so in this situation you can just rebuild a new machine and kill the old one (ideally, after figuring out how the password was compromised in the first place).
> This is a pretty pointless idea. If someone has root on your machine for even 500ms that's already too late to trust any of its state without a complete reinstall (and even this is not enough for an advanced attacker).
As the article says, he just wanted it shut down. Fixing it can come later, but stopping it from sitting on your network sending out millions of spam e-mails is still useful.
Presumably the point is not "have someone that can fix your machine if it actually gets rooted", but "have someone you can call in case of an emergency who is competent".
Seems like for a business, this would be a great service to have available from the likes of Linode, which pays people to be there in any event, rather than your 'buddy', who might go on vacation when you do.
Servers are with Gandi - which has a hosting API. So... SMS gateway via some shared hosting I have had for ages. This calls a script which figures out which server I mean and uses the API to shut it down.
I did used to use Twilio until they dropped international text messaging etc.
A shared hosting account which has a free (5 credits a month) SMS gateway included - I think it is reselling txtlocal.co.uk (who look quite good, but I haven't gotten round to trying them out directly yet).
It was kinda thrown together in a few minutes after Twilio flaked :)
I have a similar system, but instead of a plain passcode, I have a (simple, and probably not very secure) hashing method that I can do in my head.
My message looks like this: <machine> <hashed passcode> <number>
The passcode is "hashed" with the number. When the server receives the message, it adds the number to a deny-list. That way, the number is only good for one use.
Also, the number has to conform to a certain pattern. I might change that to a pre-generated list of valid numbers, but for the moment, I can work out a valid number with pen and paper.
> But it made me realize that I need to tighten up my SSH buddy plans for the next time.
I'm going to ask a really audacious question: Why?
His systems all worked as expected. Injection attacks were thwarted and login attempts failed. He received (as it turns out, erroneous) notification about the login attempts and knows he needs to do something about it for the future.
Why is that something the sharing of credentials? Why is it that people still allow for remote root login? Why do people still allow user SSH access via password?
There's a better way, and it leaves you a damn sight better prepared for intrusion attempts than receiving SMS messages that, as he so perfectly demonstrated, were not actionable. Limit logons to PKI-only. Live happier. Sleep easier.
Though the main problem if someone breaks into your system, the first person you'd suspect is your SSH buddy, because there is a possibility of them using a public machine to log in. Who would want to carry the burden of being your first suspect?
Many messages here like "why not do X?" or "why in the world are you doing Y?" All good suggestions, but sometimes X or Y may not be good enough, or you may have implemented them incorrectly (like the author's errant regex), or the completely unexpected Z might happen; no one expects Z.
I don't see anything wrong with having a human backup. Heck, what if you die and things need to be wound down?
i don't get this. is it about a personal site? if so, how paranoid can you be? or is it about a professional site? if so, who not try calling a colleague?
At the risk of talking about the actual topic at hand instead of diving off into minutia about putting SSH on your phone, I'm not sure what this gets you, from a paranoia point of view. Assuming the idea is that you want 100% coverage of your server, this might marginally increase your odds, but not really all that much. Times when you aren't available to administer your server are likely to correlate highly with the times when your buddy isn't available. And what if you'd never gotten the SMS in the first place because you were entirely out of service?
One person simply can't cover a server 100%, adding a "buddy" doesn't help much. Either you need a full-on netops operation, or you need to be able to deal with not having 100%.
Isn't the real problem that he only has 1 sys-admin for his servers (as far as I can tell)? I mean he can have as many SSH Buddies as he wants but if the alert comes in when he is asleep or drunk or whatever it's pointless.
Maybe the alert goes to multiple buddies who have the tech skills to handle it ... maybe he could be their SSH Buddy in return? In other words, rather than a SSH-Buddy, have a pool of server sysadmin friends for emergencies.
ps. If the original poster is reading this, I would have posted this on your blog but I didn't have other account passwords to hand ... if your moderating all comments anyway, why not allow anonymous comments?
I can't believe no one has suggested this yet: this is the PERFECT argument for two-factor authentication. He's even already sending texts to himself.
For every new connector, send some sort of code to your phone, and require it to be inputted back into the computer before any access is given whatsoever. Summarily disregard every single connection and command given without this code re-input.
You're even most of the way there: if you're sending messages to yourself, you might as well implement this behavior with it.
It's expensive, but have data-roaming enabled on your phone. You'll only be using it for emergency server-maintenance after all - and surely you can bill it to your employer in those emergencies anyway.
I don't think it really needs to be secure for this purpose. It's easy enough to fake the number you're sending from, and probably (I've no idea) easier enough to listen in on SMS messages sent (assuming it is for the sake of worst-case).
Have the server set up to require a.) sent from correct number and b.) correct password following the "shutdown" command.
This isn't a system set up to be regularly used, it's about killing the server until you can get to a computer, when you're under attack. So have the server only accept the password once then disable the system - once he's back and made sure the server is fine, that code can be changed, so it doesn't matter who was listening in on the original message, after it was sent that code stops being useful.
I've had an "ssh buddy" for years, but for a variety of things such as asking for a "download on how to hotwire a motorcycle".
I have an agreement with many friends to be an operator so they can look something up on the web, or anything else requiring some efficient technical prowess.
We just call up each other and say "operator", and if you're near a computer, you help out with whatever it is.
Ultimately this turned out to be a false alarm. Although the machine was under attack (on many levels: there was activity hitting the packet filter, trying all sorts of injection at the Apache level and having a go at SSH) the actual alert (based on looking in auth.log) was a false alarm based on a bad regexp.
Any particular reason you didn't want to use Snort or Bro?
Alternative Solution, with iptables and perl below.
You send the magic string to any tcp port and it'll instantly kill all SSH logins and disable the root shell. Send the string again and the process is reversed. Caveat: if your syslog contains trigger entries more than a year old this will blow up.
I also recommend you disable root logins and password authentication, but if you insist on enabling them, this may work for you. Modify as necessary.
iptables -t raw -A PREROUTING -p tcp -m string --algo bm --string "_-()ThisIsAReallyLongAndComplicatedRandomStringToMatchOn()-_" --from 0 --to lengthofthelongstring -j LOG --log-prefix "29CharacterMaxTriggerString "
#!/usr/bin/perl
# sentinel - disable root based on a syslog trigger
# Copyright (C) 2011 Peter Willis <peterwwillis@yahoo.com>
use strict;
use POSIX qw(mktime);
my $LOG = "/var/log/syslog";
my $LOCKED = 0;
my $TRIGGER = "29CharacterMaxTriggerString";
my %M = ( "jan"=>0, "feb"=>1, "mar"=>2, "apr"=>3, "may"=>4, "jun"=>5, "jul"=>6, "aug"=>7, "sep"=>8, "oct"=>9, "nov"=>10, "dec"=>11 );
for ( ;; ) {
sleep(1);
open(F, "<$LOG") || die "Error: $!";
for ( ;; ) {
sleep(1);
while ( <F> ) {
select(undef, undef, undef, 0.001);
#print STDERR "Reading \"$_\"\n";
if ( /^(\w+) (\d+) (\d+):(\d+):(\d+) \w+ kernel: $TRIGGER / ) {
my $time = time();
my $stamp = mktime($5, $4, $3, $2, $M{lc $1}, (localtime($time))[5]);
if ( $stamp > $^T ) {
#print STDERR "Found Trigger\n";
trigger();
} else {
#print STDERR "Error: found trigger but timestamp $stamp is before script begin time $^T\n";
}
}
}
}
close(F);
}
sub trigger {
if ( $LOCKED ) {
system("/usr/bin/chsh -s /bin/bash root");
$LOCKED=0;
} else {
my @procs = map { @_=split(/\s+/,$_); $_[1] } grep(/^root\s+.*sshd:/, `ps -aux 2>/dev/null`);
#print STDERR "Killing processes: @procs\n";
kill(15, @procs);
kill(9, @procs);
system("/usr/bin/chsh -s /bin/false root");
$LOCKED=1;
}
}
Wouldn't a OTP scheme like OPIE be able to do this decently? My own computers do not permit root login via ssh, but still it'd be more or less a trivial matter to set up a local account for your "SSH buddy", and then over the phone give him the next OTP in sequence for the root account or so for a successful su - -c halt.
Isn't there a service where you can SMS something like shutdown, restart or other commands for the box to respond to, that would help in that instance where net access is poor
I found that password scheme pretty OCD and not very secure considering once you have that paper, you have a pretty good chance of very quickly brute-forcing your way in.
If having your password on a note is good enough for Schneier ( http://www.schneier.com/blog/archives/2005/06/write_down_you... ), it's good enough for me. For pretty much the same reason. If you're going to even use a password, it should be a good one. And if it's a good one, you aren't very likely to be able to memorize it.
With upper and lower case and "1337" speak thrown in between and a few other characters replaced, it should not be that hard to come up with a decent enough password that you can even remember.
And why not a password manager instead of paper? Would be even better if it was one that reads your fingerprint or so.
If you read it properly you'll see that he wasn't notified because of the intrusion attempts, but because he screwed up some code which was scanning the log files. It's not like every time someone port scans him it sends an SMS.
Yes, but the issue here is not root login, it's the prospect of something going wrong with your server(s) and having a trusted buddy around whom you can call on for assistance. It's actually pretty reasonable. My servers always crash when I'm literally somewhere in the jungle in Asia and can't possibly get to an online connection for at least a day. It's great if you can just call someone who takes care of the most pressing problems if/when they happen.
I can't quite decide whether I really failed to bring my point across or if you're just pulling my leg here. Okay, I'll be more specific. Things happen to servers when you're not around. It helps to have someone you can call who can fix small things when they go wrong. You can't meaningfully automate every single thing that might happen. It might not even be a technical malfunction, maybe something unexpected needs to be done and it requires a person to do it...
Well on the surface nothing, however login credentials can be stolen easily. Focus is on "easily". Sure keys maybe hijacked but the problem becomes non trivial and I am all for making burglars jump through a monkey bar before they even get to the door.
(Not that I'm arguing your point. Just taking all steps that you can think of to prevent something, doesn't mean you'll actually prevent it. Being prepared for a worst case is always a good idea.)
Ideally you should not have to circumvent the default PermitRootLogin settings; it is there for a reason. And no need to give out root passwords, if you are so security conscious.
Our polo-necked messiah has given us iPhones that can even run an SSH client so how sort-of pointless to talk about "ssh buddies" and showing iPhone screenshots in the very same article? Since you were getting IDS alarms on your iPhone anyway, why didn't you SSH in and shutdown the machine yourself?
And on the root login bit: you would only need a sort-of dummy account for you logging in or for your "ssh buddy", if you love that idea. And all they need to be able to run is "sudo shutdown".
I was waiting for the article to mention the fact that you can SSH from any capable smart phone. Then he/others are saying using SMS to send commands from his phone is a good idea. [facepalm]
Get a mobile ssh client, set it and your sshd up with RSA keys, setup port forwarding on your home router on a non-standard port, disable Root login and move along.
You may have a cell signal but no (or prohibitively expensive, in the case of traveling abroad) data access.
Then again, if one of my servers was being hacked, I think I'd be ok spending $50 in data roaming to shut it down.
Although, there's the case of a really slow data. On T-Mobile I'll occasionally get an EDGE signal too useless to even establish an ssh session, let alone be interactive enough to use ssh. A text message can often be much more reliable and easy to use. Not sure about the security implications related to SMS remote control, though.
With passwords you are handing out what is needed to log in. With the ssh public key scheme, you only prove that you have the private key, but don't hand it out to the server.
I don't feel safe carrying a private key around with me on my mobile. It could be copied without my knowledge and brute forced in privacy. Whereas with password authentication, I can impose limits on the number of tries and also on the time between the tries. Am I wrong in thinking this?
This is why I have a completely separate private key that I generated for my phone, using a different password that's unrelated to the passwords I use for all my other machines' private keys, and I use password protection on the phone itself to thwart casual intrusion. If I lose my phone, it's trivial to remove the phone's public key from my servers' authorized_keys files, at which point the phone can no longer be used to log in.
Doesn't seem like front page news. Use key based auth and disable root login. You can also get an ssh client on your iphone if you don't mind keeping your private keys on it, just make sure to use a long password and be prepared to change all your server keys if you lose your phone. I also ban every IP that even tries to login as root.
Code re-use please!