Neat little project to needle some of the botters a bit (though I assume they'll all evolve to recognize this fairly quickly). I'd echo tyingq's comment below though:
Any tarpit has the potential to piss someone off. I'd run it on a sacrificial server with no obvious way to tie back to who is running it.
Yeah, just speculating but at least from what I've seen in the past if you successfully tarpit some script kiddie and they notice the IP their scan got stuck on there is some potential to move from "one of a billion random lowest common denominator bulk scan targets" to "paid attention to specifically some minimal amount", which is a genuinely different scenario. Even if all that amounts to is a relatively low volume revenge DDOS pointed at you for a bit it's still more of a disruption then if the auto scan had just moved on without seeing anything of note in the first place. This looks like fun on some systems, but on anything real I'm inclined to just stick to cutting down on log spam via single packet auth or a port knocker or the like. The old outrun-the-hiker-not-the-bear aphorism fits a lot of cases, just something to keep in mind before implementing something like this if you aren't directly experimenting with more active reactions.
Conversely as a research project I'm now actually curious what sort of extra attention even something like this could attract. Maybe these days everyone would just adapt and move on instead and the above is all obsolete?
Realistically if you do a massive scan you have to handle all kinds of behaviors and this would be a debugging point basically ("oh my script is stuck, let's add a timeout").
The thing is most people will not see the trap as malicious but as a "misconfiguration" on their end or the other person's end (network or ssh in this case).
I see your point but then I never heard of honeypots getting some kind of "revenge" for instance.
Good advice, being ready for a revenge DDOS, but I suspect most of these SSH hunters are busy business people who have a quota of zombie hosts to fill for whatever mining or spamming they're up to. DDOSing doesn't pay like those things do.
Exactly.. figuring out why one host out of 500000 you probed got hung up is generally not a wise use of a bot operators time. Now if enough of these tar pits start showing up you can be sure they will quick patch a time limit to defeat stalls like this.
Even sophisticated/organized hackers have free time, and if someone "messed with them" and caused them to need to push a hotfix at 6pm on a Friday because their cluster got stuck on your tarpit, you can bet they'll give the tarpit operator some of their attention. Woe to you if they happen to find something that's not patched.
Turn it around. How would hackers adapt if nearly every host responded on ssh the same way? Maybe it’d be behavior worthy of being in openssh or elsewhere. Don’t make it cheap to cheat. This might not be the holy grail, but it’s a good avenue.
Seems to me like they'd just implement a timeout on waiting for the identification string. I don't know the SSH protocol that well, but from the article it sounds like it's easy to bypass this tarpit if you don't strictly follow the SSH standard. I doubt a timeout would have much impact when connecting to legitimate hosts since they probably respond quickly with their identification, so the fix is easy.
The 141 bot never stays longer than 5 seconds. I'm guessing they are using 'expect' and waiting 5 seconds for a particular string.
The only thing missing is setting some glibc hardening options
RELRO STACK CANARY NX PIE RPATH RUNPATH FORTIFY Fortified Fortifiable FILE
Partial RELRO No canary found NX enabled No PIE No RPATH No RUNPATH No 0 8 ./endlessh
RELRO STACK CANARY NX PIE RPATH RUNPATH FORTIFY Fortified Fortifiable FILE
Full RELRO Canary found NX enabled PIE enabled No RPATH No RUNPATH Yes 3 4 ./endlessh
If you cannot trust that 725 line program without any mention of recv() nor read() will never call recv() or read() on network socket then you have much larger reliability (not security, because whether something that is broken to the point of being unusable is secure or not is completely irrelevant) issue in your system that is not going to go away by applying random hardening options to said program.
If it opens a socket, it gets hardening. It's just a best practice I follow. Code can change. Subtle changes can lead to gaps, so I wrap that rascal regardless.
I've been running this for a couple days now. In summary, about 85% of the bots give up in 5 to 30 seconds. The remainder of them will remain stuck anywhere from 700 to 9000 seconds. It is at least interesting to see how bots have evolved.
The thought crossed my mind, but I am lazy and the proper change would be to write a configure script that checks for the availability of the various hardening options.
This is the Internet equivalent of keeping phone scammers on the line by not just refusing to hang up, but actively interacting with them until they hang up (search YouTube for "tech support scammer trolling" for some entertainment...)
That said, I'm a little confused even after reading the linked RFC paragraph; it claims "The primary use of this feature is to allow TCP-wrappers to display an error message before disconnecting." but I'm not familiar with that term and there are no other occurrences of the word "wrapper" in that document.
Firefox and Chrome will spin on that server for hours before giving up.
For a web browser, this behaviour makes a lot of sense: timeouts are usually based on "I've waited for X and haven't received anything", not "this connection is like collecting water from a dripping tap". As long as there is a slow but steady stream of content coming in, you wouldn't want to abort until the user gives up or an internal limit on the data collected is hit (which is probably what happened with the author's tests --- a quick search doesn't yield the maximum size of the response headers browsers will accept, but servers usually have a configurable limit of a few KB at most for the request headers alone.)
I opened up port 22 once so that it connected (without crypto) and issued the phrase,
YOU ARE STANDING AT THE END OF A ROAD BEFORE A SMALL BRICK BUILDING.
AROUND YOU IS A FOREST. A SMALL STREAM FLOWS OUT OF THE BUILDING AND
DOWN A GULLY.
>
and if they typed in something it would continue to play the game. After hundreds of thousands of automated SSH worm probes over several months, an actual human did connect and play the game for a few minutes! But even this triumph was short lived, they did not even play it to the end to see if something interesting might be unlocked (it would have been an invitation to contact my email address).
I love the fake arp responses, that's a brilliant way so you can run this on any machine (doesn't have to be a router) and not worry about managing new hosts (no black/whitelist to manage of which IPs are unused).
I'm definitely not going to try this because it sounds like an easy way to get in trouble, but would this work on a host like DO/linode/vultr, etc? Again, this seems really easy to detect
Yes be super careful with Labrea on VPS providers and dedicated server providers. They tend to get upset if you are trying to pick up IP's. In most cases it won't even work on KVM hosts.
I've used Labrea on unused address space to make ARIN/RIPE happy back in the day and that was harmless, as entire /16's and /17's were unused. They used to nag about nothing in the /16's being pingable, so Labrea made it all 100% pingable
Whether you own them or not is subject to a lot of debate and not clear cut. If you owned them, why would you need to fool ARIN into believing they are being utilised?
The agreement you make to get an allocation requires a certain level of utilization depending on the size of the block to prevent squatting and exhaustion of resources. This is a good thing, and if it was actually respected we wouldn't be in a mad rush to migrate to IPv6.
> IPv6 is intended to replace IPv4. In December 1998, IPv6 became a Draft Standard for the IETF, who subsequently ratified it as an Internet Standard on 14 July 2017
it only took 20 years to standardize after all. we're seriously rushing here!
Oh I was thinking home network or some small business. I'm not yet used to this cloud thing, still hosting my email and website at home. For a network that is not yours, like at a VPS provider, I would totally not try this. They would not be amused.
If you don't want to waste resources and run a separate service, you can do a very similar thing with just iptables. Look for the TARPIT target.
> Captures and holds incoming TCP connections using no local per-connection resources. Connections are accepted, but immediately switched to the persist state (0 byte window), in which the remote side stops sending data and asks to continue every 60-240 seconds. Attempts to close the connection are ignored, forcing the remote side to time out the connection in 12-24 minutes.
I'm guessing this may not work, since OP's app sends every 10 seconds and seems to imply that there is some timeout associated with receiving data from the server, but not necessarily a timeout associated with receiving the correct 'SSH-' string needed to move on.
Wouldn’t cycling over the lines of an ASCII middle finger be more efficient and equally as useful as generating random strings? The middle finger would fit in a few cache lines at most.
Just need a protocol that allows for server initiated key renegotiation with PBKDF. Then you trick the client into repeatedly calculating some new key.
Neat little tool! Since switching to Teleport I haven't had OpenSSH listening publicly for a while, just reverse proxies. Still, I'll prob spin up a small server for this to observe the patterns and IP locations.
Also, don't think that posting this blog article on March 22 didn't go unnoticed :)
SSH brute-force attacks still pose a significant risk. See our related paper on SSH auditing driven by attacker's attempts here https://pmcao.github.io/caudit/
Perhaps this could be useful if you ran it on a vast number of ports. Just running it on the one doesn't seem to make an attacker's life much more difficult.
Just remember to open those same ports in the INPUT rules. If you get DDoS, then also create NOTRACK target rules in the raw table to avoid hitting the conntrack table.
No, but you can minimize the impact or effectiveness of the attack. This is a long topic however. Probably better suited to a blog post or a youtube video.
Running it on just 22 should be pretty effective on it's own. Large scale port scanning is semi-difficult, as you'll get banned really fast, even if you're just scanning an uncommon port. To pull it off, you'll either need your host to be okay with the abuse complaints (that are usually automated) or have a botnet/proxy list.
However even then there's 65535 different ports (not to mention almost 4.3 billion ipv4 addresses), and you'll likely can firewalled off pretty fast. So instead of scanning 65535 ports, scanning just 22 usually works anyway, as few people change their ssh ports (and those that do are more likely to have other security measures in place)
Assuming you run a legit SSH server on port A and an SSH tarpit on port B. What hinders an attacker to connect to port A an B at the same time? What is the advantage having an attacker connected for weeks on port B?
They will usually try port 22 for SSH, only. So, if tarpit is on port 22 and real SSH is on another - mission accomplished: you're making them (at least one of their threads) wait.
I can recommend setting up a server and do extensive logging. Then watch hackers as they attempt to break in. I had a server hacked last week and today i read the logs to figure out how they did. From first discovery, trying to figure out what system they're on, to gaining a shell, and figure out how to execute stuff, gaining network access. It was better then watching an actual movie. And a humbling experience. But also entertaining, watching the struggle against my hardening and even stumbled on a bug, feeling his/her frustration. And the wow feeling when they finally got a shell.
very interesting, and useful (including newbies like me who do not appreciate the complexities and dangers of having internet-accessible IP address).
I wonder if security-oriented OSs like openBSD could offer something like this out of the box. So that an admin can just say enable_tarpits='yes', and it would automatically enable tarpits like this for all the currently installed packages (with some default ports).
OpenBSD ships spamd, a greylister which also tarpits.
PF has the max-src-conn-rate state tracking option which is commonly use to blackhole scanners:
pass in on egress proto tcp to port ssh keep state \
(max-src-conn-rate 10/60, overload <scanners>)
block quick from <scanners>
Rather than blocking you could instead redirect new connections to a tar pit by replacing the block rule with something like
pass in on egress proto tcp from <scanners> to port ssh \
divert-to 127.0.0.1 port 2222
where your SSH tar pit service listens on 127.0.0.1:2222. This is basically the reverse of how spamd works, which diverts all inbound SMTP connections not in the <nospamd> table to spamd. When spamd's greylisting rules are satisfied it adds the sender's IP to <nospamd>.
IPv6 poses a problem for selective tar pitting, though. Realistically you need to tar pit at least /64 subnets, but even then it's not difficult to get your hands on a /48. But if you lumped /48s together you'd have a huge false positive problem. Also, AFAIU PF's state tracking capability can't track subnets, anyhow.
This will not allow any machine to connect on the port it's listening on.
The idea is that you set Endlessh on your server's port 22 (standard SSH port), then configure "actual" SSH to listen on a different (randomly selected) port. You connect to that port to get stuff done. Bots that troll for connections on port 22 will get stuck on port 22.
The idea is that you run your real SSH server somewhere else, like on a different port. If you scroll down a little (like, skim over the article) you see some example code that is basically the entire server. It's just a few lines of code, there is no real SSH implementation in this thing. And that's the whole idea: it can't really be hacked and it thwarts attackers.
I could see a few ways to use this. For me, what makes sense for my public sftp servers is to put this on a higher port, then use ipset lists to dynamically add bots to a DNAT rule and send them to this instead of the proper sshd.
i.e. you get 2 chances to authenticate correctly, then I put you in this hamster wheel for a day. Hamster wheels and intermittent fasting are all the rage these days.
I was thinking of something similar, but in reverse -- turn this into part tarpit, part honeypot.
I was thinking I could take the IP address of anyone who hangs out in the tarpit for longer than a minute or so and automatically add it to my firewall's blacklist.
I suspect that bots that randomly scan IP addresses for vulnerable hosts don't bother trying anything besides 22. I've been running my sshd on port 622 for a long time and I never get fishy connections (while I'd get at least one a week on port 22).
On the other hand a dedicated attacker who really wants to pwn your server will just scan all the ports and figure out what is listening and where. At this point you're better off implementing some form of port knocking if that's a cause for concern.
That being said, running sshd on port 2222 is probably not a good idea because it's not a privileged port.
He's trying to crash a script-kiddy's box with fuzzing. It's a fun thought, though I'm not sure it's practical as you'd only get a few tries. Fuzzing is typically an approach of throwing it the wall and seeing what sticks; if your deep learning found something that crashed most ssh clients that would just be a zero-day.
I'm not sure I like this. How secure is it? Anything you put on port 22 is going to be attacked relentlessly. I think I'd rather have OpenSSH handling those attacks than some tarpit program that may or may not have been subjected to the audits and the trial-by-fire that OpenSSH has.
It's as safe as it can get. None of the versions he showed in the post actually read any data from the attacker. It only accepts the connection, then writes random data to it every 10 seconds forever.
I'm not sure I catch your drift. It seems like the point is that it doesn't actually implement the SSH protocol- it just exploits the fact that a real SSH server is perfectly within its rights to send any data it wants to before a version string. (And on top of that, it looks like it doesn't even read anything from the socket?)
Is there something I'm missing here (probably related to poll(2)) that could cause this to be insecure?
A program like this has vastly less attack surface than a full SSH protocol implementation. In particular, it doesn't support the part of the SSH protocol that allows executing commands on a remote host. So the only way this will be less secure than OpenSSH is if there's some memory unsafety or a leak of some kind.
almost all of the SSH protocol isn't exposed until you authenticate, and in the vast majority of setups that means you get a shell to execute commands with anyways, so while still important, the key problems are with pre-auth security.
You can always tie it to the organization who is running it, by the simple fact of looking at the IP address that is "acting slow".
Years ago when I was playing network admin for a company, I set up a tarpit system which ran for a few years. In that time we never encountered one pissed off person in person (or on the Internet).
Why worry about pissing off the kind of person who scans for open SSH ports for no apparent legitimate reason? Are you concerned that the Chinese are going to try even harder to hack your machine, like they're not already doing everything they can to break in?
On my home server, there are only two ports open to the internet -- one for my SSH server, and one for my VPN server. Both of those are running on nonstandard ports.
In order to DDOS me, an attacker has to find those ports, which would require a comprehensive scan (increasing the chances that they'll get blocked by my ISP or detected by my antiscan monitor) or a huge amount of luck. And even then, they'll just be able to DDOS my SSH server and VPN. That wouldn't be awesome, but wouldn't exactly be disastrous.
You can change your ports but it won't prevent "Eve" from scanning for those as you mention yourself. I think this belongs to the realm of "security through obscurity" tactics and won't be safe in the long run.
I can prevent being blocked by your ISP by adjusting the timings etc. of my scripts.
I think it's far more important to take a look at the supported key exchange algorithms etc.
You are correct. This falls into the cost/benefit calculation of security. A DDOS attack on my home network would be annoying, but not devastating, so I'm willing to accept a little vulnerability to them in exchange for other benefits.
> I can prevent being blocked by your ISP by adjusting the timings etc. of my scripts.
But that won't let you escape detection by my system's own defenses. Well, you could if you slowed down the scan enough -- but "enough" is pretty darned slow.
Any tarpit has the potential to piss someone off. I'd run it on a sacrificial server with no obvious way to tie back to who is running it.
Yeah, just speculating but at least from what I've seen in the past if you successfully tarpit some script kiddie and they notice the IP their scan got stuck on there is some potential to move from "one of a billion random lowest common denominator bulk scan targets" to "paid attention to specifically some minimal amount", which is a genuinely different scenario. Even if all that amounts to is a relatively low volume revenge DDOS pointed at you for a bit it's still more of a disruption then if the auto scan had just moved on without seeing anything of note in the first place. This looks like fun on some systems, but on anything real I'm inclined to just stick to cutting down on log spam via single packet auth or a port knocker or the like. The old outrun-the-hiker-not-the-bear aphorism fits a lot of cases, just something to keep in mind before implementing something like this if you aren't directly experimenting with more active reactions.
Conversely as a research project I'm now actually curious what sort of extra attention even something like this could attract. Maybe these days everyone would just adapt and move on instead and the above is all obsolete?