Real-life tests are THE best thing to send job candidates. It scales well (you don't have to spend personal hours on them) and you get real information.
This applies even to sysadmins. We have a favourite: set up a VM with a slightly-broken application in a slightly-broken Apache and Tomcat, and get them to ssh in and document the process of fixing it. Even people who aren't a full bottle on Tomcat will give useful information, because we get an insight into their thought processes. I recommend this to all.
(I note we've just done a round of interviews where we get a nice-looking CV and conduct a technical grilling. Hideous waste of time for everyone involved. All CVs should be regarded, on the balance of probabilities, as works of fiction. Do a remote self-paced test like this. You won't regret it.)
> Real-life tests are THE best thing to send job candidates
I agree, but only if you're allowed to use references/google/etc and given a reasonable amount of time to accomplish it. I've had a "real-life" test where I wasn't allowed to verify or look up information, or where I'm giving a very short time to execute, and I've always thought those were absurd.
The idea that you would ever have to do something like this in a bomb-diffusing type of scenario with no internet and a ticking clock, would only test if you have already solved the problem before, not whether you could investigate a new problem.
>I agree, but only if you're allowed to use references/google/etc and given a reasonable amount of time to accomplish it. I've had a "real-life" test where I wasn't allowed to verify or look up information, or where I'm giving a very short time to execute, and I've always thought those were absurd.
Employers that do this are ridiculous. As are educators, too. What is this, preparation for when coders are kidnapped by terrorists and forced to recite how to tar/untar stuff on command? (relevant xkcd, of course)
In my experience, they're usually startups or smaller companies that are trying to "copy google" in their hiring practices, but don't fully think out how they're doing it. It's not a good sign.
Well Google does it too. Their 'gang bang' interviews are notorious for weeding out people that don't perform well under short term pressure like that. "just write an algorithm for a stack on this whiteboard with a big O complexity analysis and memory usage pattern in 15 minutes while we all stare at you. What? You don't work well like this? Well this is how elite Googlers work all of the time so GTFO"
To be fair, a lot of people are capable of becoming that type of employee after gaining comfort at the company, and after building relationships with their co-workers. To expect someone to do that in front of strangers while under pressure doesn't necessarily correlate with a programmer who can evolve into such a person.
I've been in two interviews where they asked me to code without googling or looking at the reference.
Personally, I think it's as absurd as writing shell scripts on a computer with no `man`, or without being able to refresh your memory with `<command> --help`. Good luck with that!
I was in an interview where they asked me to configure specific firewall rules on a notepad with no access to a terminal, so I couldn't check man iptables for stuff like how to match certain sources/destinations hosts/IPs. And I was not too familiar with the opts (was it --dest, --dst, or -d? and how do calculate the network mask for this range of IPs?). It was also for a junior position so I thought they were taking a piss at me and simply told them so.
It's gets a bit complicated when your job candidates may find your test interesting enough that they post a complete walk-through of the solution online.
The candidate was kind enough to avoid posting the name of the company (to hopefully hide the walk-through from Google queries for "crack me hiring test company name", but an HN discussion might then arise around the walk-through, in which the company would be named, and well, there goes your test.
It's hard to accurately run "take home" tests when hiring because of just this.
If you're not changing your tests on every interview batch you're doing it wrong.
You're asking a potential candidate to spend hours cracking your program but you can't be bothered to change it for each batch?
I expect any company applying this interview technique to at least put some effort into it and even then it pays off since a batch could contain hundreds of candidates and you only have to build the program once.
This ^ "You only get out of it, what you put into it." goes both ways - employers should expect to extend some effort if they expect their candidates to extend some effort.
I strongly suspected that sometimes goals of the person responsible for hiring are not quite aligned with the company they work for.
Example:
Company's goal: Hire an exceptional candidate.
Lazy or overworked hiring manager's goal: Reduce the size of the stack of 1000's of resume's I have to go through using any means possible that will make it appear like he/she actually did the work to find the candidate.
"Lemme see, use keyword search to ditch the ones without 'Github' on their resume'. Now ditch the ones who don't have a college degree. Still some resume's left? OK, how about tossing the ones without a CS degree?"
Run a filter with enough conditions and you can get that pile down to nothing pretty fast. If that is still not enough filtering, ask the candidates to do something very time consuming. This a great way to weed out people who are very busy or exceptional at what they do and don't need to jump through your hoops.
The problem is that this makes it difficult to compare evaluations. If you want to have as objective an evaluation of candidates as possible, you need to subject them to exactly the same process.
I'm surprised they didn't make him sign an NDA for it...
For any person who claims to have years of Unix experience, they can easily do anything they want using man pages. Though I agree with you on the general premise though. One must be allowed to look up information they need to solve a problem.
>>I've had a "real-life" test where I wasn't allowed to verify or look up information, or where I'm giving a very short time to execute, and I've always thought those were absurd.
You certainly haven't worked at major IT services firms here in India. I spent early part of working years (around 2006) coding by purely reading books and discussing things with folks on internal mailing list. Because only managers and levels above were allowed internet access.
>>The idea that you would ever have to do something like this in a bomb-diffusing type of scenario with no internet and a ticking clock, would only test if you have already solved the problem before, not whether you could investigate a new problem.
This is unfortunately true even for coding interviews. Most algorithm questions are impossible to answer unless you have rote memorized the algorithms and their implementations.
If you have to give white-board coding interviews, the goal should be to have a question that's complex, potentially elegant and straightforward.
By complex, I mean the question should be difficult enough that you can't easily hold every parameter, case and step in your head: I want to see how you deal with situations that you have to mentally walk through and issues that you might lose track of.
By potentially elegant, I mean both brute-force and optimal solutions should be able to fit on a small whiteboard, without requiring more than 2-4 variable declarations and no more than a nested for loop (or some sort of recursion). The optimal solution shouldn't require putting together pieces that they aren't really expected to know/use, i.e. no more complex than arrays, maps, strings and ints, and any other "data structures" the candidate ought to know, given their background. For instance, if they claim to be a crypto expert, they should know how to use a black box that gives them private and public keys.
By straightforward I mean, you don't want the candidate to have to have an "aha" moment. If you're asking for them to write a max-flow variant, you'd better be explicit that you don't mind a really brute-force solution, or most of the interview will be spent trying to remember complex algorithms. Is the algorithm one that you can understand by walking through test cases/figuring out what steps you took in your mind to go from input to output? If it takes more than that, you're going to get a few false positives, i.e. "Wow she was great! No one's answered this so fast!" (because she just spent 20 hours doing this on a homework), and many false negatives.
Yeah, but B5Geek's is a pretty entry-level thing. I've encountered whiteboard tests where you have to recreate an algorithm that someone spent their PhD thesis creating. It seems like a good way to find "geniuses", but it just isn't practical.
In a lot of ways, it's a matter of identity crisis for software developers. We're all, industry included, not quite sure whether we're mechanics, carpenters, architects or scientists.
I've taken a whiteboard test that asked me to come up with an algorithm that was difficult for me. I got the job even though my solution was horribly inefficient (and I knew it), and the interviewer later told me that the point wasn't to see what algorithm I knew - it was to see if I acted like an ass when I didn't know stuff.
Not sure that's a good test either. When tasked to figure out something as part of a job, by a manager one knows and trusts, who takes feedback of "I don't know how to do this but I will investigate it", a person may act one way, whereas being tasked with someone's PhD thesis by a stranger on an interview may lead to a very different response.
I know I am far less tolerant of bullshit interviews than I once was, and would have far fewer qualms vocalizing what I'm thinking - "I don't know. If I saw this in real life I'd check Google, because it reads exactly like the sort of contrived problem used in interviews and thus is well documented, and likely solved quickly by those who have memorized it. I am not one of those. I'll take a stab at it now if you just want to see how I reason through it, but if coming up with an efficient, correct implementation is part of your criteria we can probably save both of us the trouble" - is that considered acting like an ass?
True. Testing for sysadmins is easier: you're after a way of thinking. So a competence test ("can you do what you claimed?") with a freeform "keep notes on your thinking along the way" is quite informative. You still need to interview, of course.
I confess I don't know how I'd apply this to developers. They pass fizzbuzz, OK - what do you do next?
> I confess I don't know how I'd apply this to developers. They pass fizzbuzz, OK - what do you do next?
I know what I would do, just have a 10 minute conversation with them about technology. I can easily figure out just where a person is in regards to their development as a programmer just by listening to them talk about what they're paying attention to in the tech landscape. If they don't blink when I mention Hacker News, I know they're on the right track. Do they know what Go is, or Haskell? Do they have an opinion about Node.js?
These criteria are intended to gauge motivation and enthusiasm in the absence of an impressive track record. If they're capable but not highly motivated, then that's fine so long as you're only hiring them to do stuff inside their comfort zone. We have a Microsoft guy just like that. If they're motivated but not capable, then with coaching they'll be able to do anything. The best way to discover someone's motivation is to get a gander at what they're thinking about when they aren't working.
If you're looking for someone who is motivated and capable, you can eventually find someone like that, but you'll have a hell of a time keeping them. Developers have so many options these days that you need to have at least 3 of: competitive salary, decent perks, fun work environment, a glamorous field; in order to hold on to them for longer than a year.
Most companies are lucky to manage one, if they're very lucky two. Since salary is often the easiest thing to manipulate and honestly the most important to the employee, err on the side of paying too much. It's expensive but not having competent developers is far more expensive.
Not infallible - I could talk tech near-perfectly (I am a native speaker of BS), but I am most definitely not a coder pretty much at all - but a good start.
Also, using HN and its fads as a measure of motivation sounds a little elitist. There are really awesome and motivated coders out there who couldn't care less about politics, mainstream tech media news articles, gadgets, Apple-does-this, Google-does-that, cool new language or JS toy of the week... these coders are likely focused on their code instead.
Programming is a wide field, there's much more to it than the front page discussions on HN can cover.
Of course, if you're looking for a Web developer to make "cool new stuff" for the latest fadgets..
You make light of web development, but that's driving the enormous demand for programmers in recent years and all of our salaries. I bumped my pay $20K when I took this job and I'm now asking for $30K more after a year. Most fields calcify after a few years and salaries level off as the managers do their thing. But making "cool new stuff" and the "latest fadgets" is still really hard. It might be the fashion industry, but business is booming.
Speaking as an iOS developer who's had a hand in hiring, it's the same as the "slightly broken Apache and Tomcat" scenario someone posted for sysadmins. We ship people a slightly broken app with a few common-yet-googleable problems, then ask them to build a tiny thing on top of that and ship it back to us.
The ability to solve common problems acts as a good proxy for experience, If they can't google how to fix a retain cycle or a broken constraint, we know they probably don't know what they're talking about.
The ability to build a good, small system on top of the broken app acts as a good proxy for development talent. If they can't easily set up a dead-simple delegate to do an asynchronous web call, they can't hack as a mobile developer.
Most importantly, if they don't care enough about the job to take two hours to whip up a super-simple app, then we don't have to waste our time interviewing them.
For iOS developers I found that a good test was to download and parse some XML or JSON API of a web service and display the result in a UITableView. . It' surprising how many people can't accomplish this.
For developers that have the academics, but not the experience, I've always said that the industry needs to adopt the apprenticeship programs that other trades have. Where you do a mixture of work for the company, and trade schooling run by the industry to teach you the specific skills you need.
For example, an IBEW apprentice will do a lot of gruntwork on the job site. Pulling wire, for instance. But they'll get exposed to how things are done, to trade industry expectations, and what a project looks like from start to end. Part of the week, they spend in a classroom learning the specifics of their trade. Not just the high level theory, but the nitty gritty, based on real world applications and problems. This apprenticeship lasts from 2-5 years depending.
I don't see why this model wouldn't work really well with programming.
Apprenticeship requires a constant stream of mentors. I'm not sure there is an ample supply of professional programmers who would be interested in taking on the responsibility of mentoring in addition to their full-time jobs.
It really depends what those professionals are used to. I did an Apprenticeship as Software Developer in Switzerland. That is, 1 year of school followed by 3 years alternating between the company and the school. So once you start at the company you already have a viable amount of knowledge in programming, databases, networking, etc and supposed to be able to work on your own.
You don't get constant mentoring at the company, but can ask one of the developers or fellow apprentices when you're stuck with one of the tasks you get assigned. Or for example, any time a developer had a small task that he felt, I couldn't do by myself, or it was too time critical he just asked me to sit beside him and watch, listen and take notes while he did the work.
How it works is different in any company but the outcome should be the same: real world experience on real projects with real clients and their time/budget constraints.
That means, once your finished and a professional yourself, it
isn't a question of interest but a completely normal thing to help out the apprentices. Because that's how you got to the point where you are.
We may laugh, but starting in technical support gives you an idea of how your users are using your product, if your documentation is accurate and helpful, what the pain points for people not in the department you are destined for are, and gives you an idea of how the product works.
It also quickly weeds out the people who can't effectively communicate before they get to the programming team.
We've been rotating the sysadmins through customer service. Not taking calls (our CS calls tend to be long and detailed, and a n00b isn't going to help), but being there for the CS people. It's been marvellously instructive. It also cheers the CS people that someone cares. Now we need to get the devs doing the same.
When did block release training or work experience start getting called co-op given that in various areas of the world a coop co-op means a cooperative like Mondragon.
I have only seen it (co-op) used in the last few years.
Algorithm tests only measure the number of man hours the candidates spend on career cup daily. Or how good they are rote memorizing things.
There are many ways to check how good a candidate is at doing their actual work. For instance you can take a candidate to a code review and see what kind of inputs they contribute to the discussion. This alone will give you good deal of idea on how well they understand code and quality. There are other ways, like say asking them to write unit test cases for a class- That would give a lot of idea on how they think with regards to breaking and fixing things.
One more way I have after doing basic checks is pair programming, or picking a totally new problem and working with the candidate to see how they think, how they work and how good their communication skills are.
There are many other ways too. Recently I've been given coding assignments for very practical applications, though those turn out to be a little difficult and time consuming. And at times the expectations are very high. Like asking the candidate to complete a whole application in very short times. It can be hectic with your current job, and a family. But generally those are a good indication of how good the candidate will do at their actual work.
A nice simple test that I like to give as a first-wave elimination:
Have a computer setup and running (all properly configured). Pull the network cable out of the jack a little bit (so it looks like it's plugged in but isn't).
Ask the person being interviewed to show me an IP being used by microsoft or google. (so ping/dig/nslookup/etc)
Let the person know that (a) the computer is in working condition (i.e. no drivers are missing) (b) the network works (i.e cables are good, switch is good, DHCP is enabled, etc.)
(c) tell them that this is a test to determine their troubleshooting skills
It is always disappointing to see how few ever open a term/cmd window (depending on the OS). 90% of participants just try to open a web-browser and type in "what is the IP address of google"
>Let the person know that (a) the computer is in working condition (i.e. no drivers are missing) (b) the network works (i.e cables are good, switch is good, DHCP is enabled, etc.)
So you lie to them? I don't understand what the point of this test is. My first inclination is to open up a term and ping google but I would be pretty annoyed that your "first-wave" test involved actively lying to a candidate.
If this is a user-facing tech-support position, then it is a hard job requirement that the candidate be able to handle being lied to by the user. I don't care what they say, they didn't check the cable, they didn't reboot, and they most certainly didn't not do anything that might have caused the problem.
>If this is a user-facing tech-support position, then it is a hard job requirement that the candidate be able to handle being lied to by the user.
Yes, I agree. Are we supposed to also assume that our bosses and interviewers lie to us, too? At no point does he say that he is emulating a user that needs support help.
Well if not lying, you should always entertain the possibility they are mistaken. The best confirm and reaffirm constantly. You know what they say about assumptions.
To be fair the first thing you do is look for relevant buttons to click on if there are any, the next thing is to google the problem, the thing after that is check it's all plugged in
All he said was that everything was in working order. He didn't say that the cable was fully seated into the receptacle. Your first inclination would challenge your assumption and you would likely follow the stack down to the wire. At that point you'll note that the link lights are off and reseat the cable.
This means - to me - that the network... well, works. And it clearly does not. If this was a Windows machine I'd probably wonder what the hell was going on with the Ethernet (!) sign in the bottom right of the tray and if it was a decent version of Ubuntu (11.x or earlier) then you'd also notice it quickly.
But still. He's pretty obviously leading the person astray by giving him the solid impression that the machine is in good working order. (which it is not) If you mean to say that he didn't CLEARLY specify that the "cable was fully seated..." I don't know what to tell you. That's a "gotcha" question. Not a good diagnostic one. Unless this is for a $12/hour CSR position. In which case, OK fine.
Every day, I have QA, tech support, my boss, and other developers tell me something that is not actually true, or simply omits any of the relevant information necessary to fix the problem. This is something you live with on a daily basis in a technical job involving complex systems that not everyone understands the entirety of.
If you can't solve a problem with only some information, and some of it incorrect, then how are you supposed to be able to help?
You are looking at this like it is a single-variable problem. This "test" should be easily solvable by a candidate. It is also a very bad one if it involves lying to the candidate. These two factors are not mutually exclusive.
No statement in that list is a lie; the computer is in working condition, the network itself works, the cable works, the switch it is connected to is working, DHCP on the computer is enabled. The cable being unplugged doesn't break any of those.
I don't know why this is downvoted. This is Hacker News, a hacker always uses the least amount of effort (the least number of lines of code for example) to achieve his goal.
Windows key + R / "cmd" / Return / "nslookup google.com" then when that fails, "ping 8.8.8.8" and when that fails, "Ipconfig /ALL", then ask "Do you use Dhcp or are ip's assigned?" as I reboot the computer and check the physical connections at the same time.
Doesn't giving someone a non-working computer that you tell them is working make you look like you haven't bothered to test it? As a candidate that'd be a warning that I'd be working with lazy idiots.
You should always remember that hiring is as much about persuading the best person to say yes to a job offer as it is about weeding out the bad people.
The question was "show me an IP being used by microsoft or google"
If you are asserting that google is not using the IP of it's public facing DNS service, this company is full of horrible tech people, so nevermind: I don't want the job.
I love this approach. I am good at what I do, and appreciate the opportunity to prove it directly. Spending a couple hours solving a technical problem or writing an essay is much easier for me, not to mention sending a more reliable signal, than trying to figure out what networks / buzzwords / whatnot various people in an opaque organization are going to want to see. Not to mention, it communicates to me that the people doing the hiring know what they want and how to get it, which is encouraging.
I believe there is a happy synergy here, by the way. A candidate who welcomes this type of test -- who sees a direct work sample as the easy and efficient way to prove themselves to you -- is probably someone who knows their work is good.
It's an effective strategy from the employer's side, but what about the applicant who is asked to spend hours on some test for every job he/she applies for? Are you paying these applicants for the time they spend jumping through these hoops?
And if your "technical grilling" fails to identify strong candidates, you're obviously asking the wrong questions or, at the very least, not asking the right ones.
As a candidate, I'd much rather they ask; I find that companies that don't ask for code are almost by definition unlikely to ask the right questions. If you must screen candidates for skillset, the less you have to rely on a proxy for that skillset the better off you are. As an employer, I find code samples irreplaceable, and I'm comfortable screening out candidates who decline to take the challenge out of principle or limited time.
I would rather spend hours on some test than be judged on a whiteboard problem, or any other problem which has nothing to do with programming on an actual computer that has an internet connection.
For the most part I don't complain if asked for an hour (or less) worth of work. I mean, I probably spend more time researching the company, working on the cover letter, replying to communications, and scheduling follow-ups. I'd much rather spend an hour at home with resources I know, than an hour in a room with a whiteboard.
The technical grilling was the thing that the people with apparently-strong CVs weren't passing.
Your objection is a fair one, but having come into the company through the process I describe I am fond of how it let me (a) show that I could actually do the stuff I claimed to be able to (b) show my thought processes and how I would deal with having received a box in this condition. So I was quite pleased to do it as a candidate. Certainly less of a waste of time than a clashing interview.
Its not hoop jumping to ask a candidate to directly demonstrate how they would go about doing the actual job they're applying for. If you were interviewing a cook would you hire them having never tasted their work? I doubt it.
The way I do this is the real coding test is only administered to final candidates. That's a bit kinder since only a few people with a serious shot at the position end up having to commit the extra time and since we observe the whole process (and discuss) it doesn't waste tons of our time either. Whiteboard and discussion can reveal a lot but IMO there's nothing so revealing as actually watching someone work.
As an aside I would much rather be interviewed this way than at a whiteboard. At least for me there's a certain amount of my programming skill that's nonverbal and flows easier for me at a keyboard.
> If you were interviewing a cook would you hire them having never tasted their work? I doubt it.
A cook is someone who can be hired with very little experience. A chef is someone who is trained professionally. And plenty of chefs are hired without a "taste test" based on their academic and employment history.
They're hired on a provisional basis, true, but they are paid while they prove themselves.
I was paid for a days work, when after the initial interview to test that I knew some really basic programming (think Fizz-buzz, but slightly more interesting), they brought me in to solve a real problem in their app. And I did, and they hired me. It was quite brilliant, actually, and they paid me for the days work at a decent rate. I was impressed.
The point is that you'll only get applicants up to a certain level. Once an applicant has enough skill and knows it, and also values his time, he'll take one look at your 14 hour challenge and laugh.
So in the end you're only hurting yourself because you'll mostly get naive and easily manipulated people who are fine for the lower ranks but absolute poison once they move up. That's if they don't wise up and leave first, costing you time and effort to find and train replacements and scramble to rediscover and distribute the lost knowledge they left with.
But then again, such a long duration challenge is already very telling of the company culture. If you can't respect other peoples' time, then your company is already on shaky footing.
he'll take one look at your 14 hour challenge and laugh
These things shouldn't be designed to take 14 hours. They should be designed to take one, maybe two or three . . . if you know what you're doing.
One of the other posters took so long because he didn't know what he was doing. Well, perhaps that is an uncharitable way to say it, but you know what I mean. :)
The test I describe should take an hour or two - it's a test of "can you actually do these things your CV claims you can?" Which is nontrivial, but I enjoyed taking it (fix a broken thing!) and feel fine about giving it to others.
>"can you actually do these things your CV claims you can?"
Yeah but how many questions must you ask before you are certain? Otherwise you're rolling the dice that the candidate has simply seen that problem before but has terrible habits otherwise.
Reduce the time required and you risk more false positives. Increase the time required, and you risk the skilled ignoring you.
But the process described in the post above is clearly about proficiency, so a challenge is not supposed to last 14 hours. ;)
And I admit you have a point. The thing is, people really have well-looking CVs and they can't code BFS in Python. Probably balancing of 'technical grilling' is a hard to master skill in and of itself.
Seems like this sort of test should be very effective at finding people who are already reasonably good at the job you're hiring for, but gives you little to no information about how quickly they learn, their willingness to try new things, etc.
Given how fast things change, even if you imagine the person staying for only a year or three, learning abilities are nontrivial.
Totally. YMMV. That's why you look for stuff on the CV like willingness to experiment, play with OSes at home, "describe your home network", etc. This doesn't replace the interview, it just verifies claimed competence and helps show thinking ability.
"Describe your home network" is an interview question now? Jeez, guess I shouldn't leave my job, I'll never get another one.
I am going to go ahead and challenge the assumption that your willingness and eagerness to learn on the job is not really correlated with how you spend your time outside your working hours.
I think it's an excellent question. Not because a candidate needs to have something special about their network (although if they do, that's a plus and a chance for them to talk about their interests) but because if they can't describe something they own, why should anyone think they will have the communications skills to describe something they're going to be working on?
I mean as extra points that help the candidate look interesting. It's like the "other interests" section on a CV: you should treat it as an opportunity to market yourself and add a hook to catch the interviewer's interest.
As an interviewer I've done something like this when needing to hire for a very straightforward job with very little time and an effectively endless supply of candidates.
This addressed the problem of our time crunch, but I doubt it helped the selection process overall.
I think this sort of testing is subject to all the same issues as most any other method of interviewing, but limited to a much smaller sample size.
Basically, it's arguably good for the interviewers and a coin flip for everyone else.
As an interviewee, I haven't encountered a test like this, but I expect my response would almost surely involve questioning the architecture decisions that led to such a problem in the first place.
In this case, as a sysadmin I was pleased to be able to show my chops in a realistic test. And our previous hire was also quite pleased to get a real test. But of course, there is potential to abuse it.
In this case, being able to Google how to fix a slightly-misconfigured Apache and Tomcat counts as passing the test with flying colours. Becoming a Google-Certified [whatever] Engineer at the drop of a hat is something we regard as a serious positive in a sysadmin. Open book tests are never easy.
Someone still calling sysadmins instead of devops in HackerNews is a rare thing. :)
I know many so called sysadmins who could not pass this test in a reasonable time. Sometimes people without even basic script-fu or networking knowledge are considered sysadmins. And this drives me crazy.
Yeah. I'm thinking of Joel Spolsky's 199/200 who are just the same bozos cycling through interviews until someone hires them by accident ...
"devops" is fighting words in our team. Damn, if I could get our devs who'd like "devops" on their CV to give a hoot about the "-ops" half of that buzzword ... when a dev shows awareness of ops issues with their shiny new thing, I make a point of mentioning their name positively to the dev manager. "X knows their stuff. Give them more good stuff."
:) When devs are in the loop it always comes to "working in localhost" debate. I like to say developers are working or living in their isolated bubble. And whenever their code comes out of that bubble and reach the real world ops are on fire.
Always the ops guys are in fault.
I think every developer should be given sysadmin 101 and 102 lessons.
I'd just like to note here that the sysadmins know damn well when a dev tries to blame them for something the dev did ... and we NEVER. FORGET. So don't do that.
I'm not sure how to educate devs to think ops. Perhaps have their phone ring at 3am when stuff breaks? I think that would close the feedback loop nicely.
(Let me note again: I LOVE the devs who think operational issues, who think end-user issues - what customers are actually like - who think "developer of the whole thing from go to whoa" and not just "coder on my PC." As I'd hope we all do here.)
As both a developer and admin, it's a little hard for me to not think across the aisle. However, I can think of developers that basically refused to think outside their solo - they tend to fit the type of person that really doesn't like to go too far outside the bounds of their workday tasks. Not saying that everyone is this way, but you wouldn't ask John Carmack to try to think about continuous integration and deployment stuff while he's stuck reading physics books and reading a bunch of papers, would you? Yes, good devs tend to have the curiosity and, more importantly, the concern to think beyond just their immediate day to day concerns of hyper focus and in-depth knowledge.
Sometimes you just need to hire more people or ask the rest of the team to accommodate someone that's that one random oddball. A management book I read mentioned Phil Knight talking about how the Bulls had room for one Dennis Rodman and only one, and made it clear that nobody else can pull the stunts he does without disrupting the harmony.
There's a huge difference between a dev that does something a bit out of ignorance and one out of indignance.
Some companies have developers spend some time in SRE (I believe Google practices this sometimes) so they can gain some insight, but it may not be the best idea for a lot of orgs. It's part of why most orgs that do some form of devops well tend to remove a lot of concerns off the table by using stuff like AWS. Meanwhile, silozation helps people maintain some sanity and focus in larger orgs where there's so much BS work on top of your technical duties.
Sometimes the culture is short-sighted and people are at odds with goals though. I've been penalized by managers and peers for not paying enough attention to my dev duties (which were pretty meh) when I was busy helping support and sales help bring in and retain $2 million in accounts that they later named me on calls as their informal engineering MVP.
But really, being aware of what other people care about in their job is a contentious issue that I mostly think boils down to personality and general ideas of teamwork.
But presumably that happens some time down the road long after you have made your decision. Either that, or said secrets are published and heard about by an audience which are tuned to hear about interesting things quickly, who may also be what you're after.
I remember a website providing failing VMs for this purpose. I'm not even sure it wasn't just a suggestion .. but if someone knows such a site, feel free to leave the url.
Given this, what is the best first step? Best we can tell, he was running root in a VM that was running as an ordinary user. What more can be done, aside from running it on a throwaway machine? Running on a cloud instance, perhaps?
Afraid you're woefully incorrect. The root user is no more contained inside a default docker image than they are inside a chroot, which is, not really at all. It isn't difficult to break out of a container if you have uid 0 aka root. This will change when the userns stuff lands, but it hasn't yet, so that doesn't count yet.
Well, so far I have only heard a lot of FUD and nothing concrete.
http://opensource.com/business/14/7/docker-security-selinux seems to have some information. But to refuse the points there, on my system:
a) docker containers do not have a /dev/mem
b) /sys/fs is mounted read-only
c) /dev/sd* is not visible
And so on. What's the thing with cgroups? Can a container delete the host's cgroups? Can it overwrite it's own cgroup constraints?
Can you or someone tell me if there is something I can try out on my own to understand actual issues? I run my containers unprivileged (which is the default) if it matters.
Again, all I am asking is for some material to educate myself on what actually the flaws are. All I have seen are vague articles claiming it's insecure.
But nothing is stopping you from creating a binary and running it as root that perhaps mounts those filesystems, or sends ioctls / badness to the kernel.
I'm just shooting from the hip here and might be wrong, but I'm almost certain devices aren't namespaced. If you were to mknod a block device with the right major / minor numbers as the host's sda/sdb/etc, what would stop you from mounting the host's root filesystem and making changes?
Granted the "work around" is to drop the privileges of the containers as much as possible, use sVirt (via SELinux), drop as many capabilities as possible, don't run untrusted containers, etc. But this isn't a theoretical security problem, it is a real big attack surface.
Sorry, I didn't imply what Shykes said is FUD. I meant most of the articles out there about docker security have little to no information other than "breakout is possible".
That quote from Shykes could be paraphrased "it's safest to assume breakout by uid0 is possible." I understand the academic/hacker interest in identifying the exact avenues for breakout, and why vague articles are disappointing. My point is, it's important everyone understand it's not to be considered secure, even if we can't/don't name the vulnerabilities off the top of our heads.
This is Intro to Systems homework at UChicago (the course is heavily based on CMU's equivalent.) You're given a personalized binary that asks for a series of passwords to complete each level. If you get a password wrong, it phones home to a server run by the professor and decrements your grade.
The point is to teach you to reason about assembly using GDB. You can pretty trivially set a breakpoint at the phoning-home routine so that you never actually lose any points; then it's just a question of thinking and reading hard enough before the deadline arrives.
Levels range from very simple string comparison, to arithmetic, to pretty weird tricks.
It was about the most memorable homework assignment I've ever done.
You get a customized binary tied in a server-side database to your campus single-sign-on identity. Everyone gets different passwords, and the passwords are validated server-side to make sure you didn't just GOTO the success reporting routine.
There could easily have been a secret embedded in each bomb, reported on defusal or detonation, to prevent people from detonating each other. The project was graded on an absolute rather than relative scale, so there was little incentive to spend time on something other than defusing your own bomb.
(The project was called "the binary bomb." We all had a great time telling our friends that we were up all night at the library defusing a bomb.)
Do you know if they did anything to mitigate replay attacks? If not, you could potentially capture someone's first call-hope and replay it until they fail.
Presumably no one would want to, but people still like messing with one another for fun. Also, if you discover a vulnerability like that, you could plausibly claim that someone exploited it to bomb your grade, particularly if you used it against a bunch of people.
The binary is tailored, in the beginning it does some identity check. In CMU, as I remembered, you have to login to the school's cluster using your own account to actually diffusing the bomb.
That program was not nearly as devious as the one discussed in the blog post. It made no efforts to stop us from using gdb, and the password encoding was never as reflective as xor-ing with the program bytecode.
I don't know where you find candidates that can even approach this level of skill or desire to solve puzzles. Most people I interview struggle with a few lines C program coding.
You want experts? Try paying them like they're in the top 1% of their field and their work is worth at least 10 times if not 100 times the average salary to many employers. Make an effort to actually, you know, compete to get them.
The market for programmers is really inefficient for pay. Google has exploited that to the max and succeeded hiring a lot of very talented engineers at rates you pay for average tax accountants and attourneys. Just like the way they exploited Free software licensing by expempting themselves from the publish part by distributing access to the software not the software itself.
That's their two best tricks right there.
Where they crossed the line massively is when they engaged in a criminal conspiracy with Apple et al to act in a cartel keep those wages down. This in addition to all the other legal ways they try pushing the market down when it really should be a sellers market. Andrew Moreton paid less than a football player of similar standing? How about 2 orders of magnitude less.
Same experience here. We have an offering for normal java development with high performance aspects - tons of applications. Our monitoring/devops position? Few and far between.
Hehe. I was in DevOps and it was intellectually challenging but not really an exciting gig. I find more excitement in creating new tools for the public consumer over locking down servers, building scripts, setting up automation testing and doing configuration management. I do recognize that all of that is critical to being able to scale your operation but I'd rather have someone else do it than myself. That being said, I would still do it, if no one else knows what they're really doing--I just wouldn't apply for a job with DevOps in the title.
iknorite. Job post for pool boy at Hugh Hefner's mansion - tons of applications. Stall cleaner at adult video arcade - just a trickle.
You're comparing a fun job to a job where at best nobody ever says anything to you and at worst, you're taking the blame for the whole company's servers being down.
You could help them find you. I see many people like you lament about not being able to find people with certain skills, but their lamentations provide no information about who they are hiring for or how to contact them. :)
Yeah that's why I was always thrilled with (cr|h)ackers. Back in the time when I was young actually this word ( hacker ) meant something completely different than what it is now. I think people like this have a talent in another direction, compared to programmers.
Something like : "As much as I am good in making programs I am not good at all in reverse engineering them."
It's an infosec job, so this kind of thing is actually quite appropriate. Though of course, perhaps not so much depending on the exact area, e.g. for network security it would not be as relevant compared to application security.
Speaking for the security community as someone who's graduating this year, everyone I know who works on CTFs regularly has an astounding level of enthusiasm. It's really a fantastic group of people to be working with.
My experience with this (as a C and C++ developer who's worked with embedded systems before) is those kinds of job postings tend to require an engineering degree, and most electrical engineers I've talked to want to do hardware not firmware (and didn't like their programming classes).
You could try either relaxing the degree requirements, or reaching a wider audience -- I generally don't see firmware jobs advertised on HN and StackOverflow and those are basically the only two places I ever pay attention to job ads.
Depending on your existing volume of applicants you might also ask them to attach a semi-trivial C assignment to their application. Yes, this is accessible to anyone who can google, but you might weed out the disengaged who don't care enough to read the instructions that way.
Hmm, as andrewljohnson said, I expect that the posting impacts who applies, but I would expect that to disproportionately turn up applicants who can write C.
Another dynamic at play is the fact that those who don't have useful skills apply more often as they continue to be out of work, that doubtless accounts for some of it.
Some examples of coding questions I've asked in the past: reverse a string in place. Ones count of an integer. I'm testing if they understand pointers and bit manipulations for example. These should be answerable in 5 minutes max in 5 lines of C code. The incorrect answers candidates generate surprise me.
In my interviewing experience (looking at the time for something not dissimilar), lots of people think that 'knows C' and 'embedded systems' are the same as "I took a single module involving C at university, and by the end of it, my things were only randomly segfaulting maybe 70% of the time."
Or they've never touched either C or an embedded system in their life, but somehow it ends up on their CV and they tell you about how passionate for learning they are. Which may well be true, but it raises questions about your honesty and/or capacity for introspection. Also, bare-metal embedded really isn't the place to learn C[1], given that most failure modes are "It doesn't work. And maybe something caught just fire."
I have had great luck hiring game devs and cross training into embedded.
"Oh I have to stream data into my processing routines with not nearly enough bandwidth? No problem."
I've also had smiles of delight when devs got 100% raw access to every little bit of hardware. Discovering DMA controllers is /fun/ for them. We came super close to getting a decompression routine running completely by our DMA controller!
Most people in this area are now older. I'm approaching 40 and I was recently at a tech demo and everyone around me seemed to be older than me. I remember taking Assembly in college and our professor explicitly telling us that we would not need to know any of it, but it was a good foundation to build on. As far as I know, of the people in that class I'm the only one that ended up actually writing a ton of assembly after joining a semiconductor company. Most of the embedded people were from EE, but didn't actually have as much of a programming background. I had approx. 2 college courses dealing with hardware in undergrad and that was not enough.
So, it seems you need to either accept a recent CS grad who seems adept at programming and can learn, or pony up the 200k+/yr to poach someone if you want to hire quickly. Most people I know that have done embedded work tend not to move around a bunch and aren't motivated by small amounts of money to move jobs, so you'd have to offer a large incentive to find the good people.
Really nice overview of the process. I was hoping to get into debugging and breaking code, but my career took a wild turn away from that part of the job. It's still something I would like to learn, so I'm reading as much about it as I can.
I'm going to take this way off topic here, but it's a curiosity of mine. Please don't take this as an insult; it seems to be very common and as a language learner myself I'm just wondering where it comes from.
At first , it looks...
analyze.Lets...
debugger.Therefore , there...
mode.In my opinion , Intel...
So , those lines will basically scan the memory , if there is a 0xCC , it will crash your program and such ...
Specifically in these examples, I'm seeing a missing space between a period and the next word, as well as a space before a comma. As English is one of my native languages, I'm not sure how people go about learning English or what resources are available to anyone learning English.
I've noticed this with a lot of English as a second language speakers, and it doesn't seem to matter what their original language is. In this case, Spanish, but I've seen native Russian and Japanese speakers with the same thing. Can anyone tell me why this is?
In Japanese, "、" (読点 tōten) is similar to the English comma; it is used to separately multiple clauses in a sentence. To my understanding, there really aren't grammatical rules for using it like there are in English. In addition, Japanese has no spaces which might lead to some confusion on the how punctuation and spaces are laid out in English. I think the most notable thing from Japanese -> English speakers is the lack of article usage (words like a, an, the) because they simply don't exist in Japanese. But I digress, the main take away is that other languages may not have as strict rules on some of the grammatical structures that are similar to those we use in English.
Not to mention the fact that in Japanese, Chinese, and other Asian languages, most punctuation is full-width, with spacing built into the character itself. Compare the English characters (?!.,:) with the Chinese characters （？！。，：）, and you can see how a lot of Asian language learners might just forget to hit the space bar.
I know that's what the Catalan government says, but is it actually the case? Wikipedia says
"As an official language, Spanish is understood almost universally in Barcelona. In addition, 95% of the population understand Catalonia's own native Catalan language, while 74.6% can speak it, 75% can read it, and 47.1% can write it"
A writing literacy of less than 50% sounds pretty poor.
As far as I know, punctuation association rules are the same in all indo-european languages. Most of them bind to the left and have a space to the right. Emdash, and ... have two spaces, and endash and apostrophe have no spaces.
But people struggle with this. I distinctly remembering practicing this in class when prepping for matura (Slovenian version of SATs) so I guess it isn't inherently obvious to everyone.
> Emdash, and ... have two spaces, and endash and apostrophe have no spaces.
In English, Em-dashes have a couple of different uses, but in the most common use they are usually set closed (no spaces on either end), though some styles set them with a space on either end. En-dashes have one use (similar to that of em-dashes, which is used varies by house style) where they are usually set open, and a number where they are usually set closed.
Given that the rules for these marks aren't particularly consistent, or even fixed, in English, I think its safe to say they aren't consistent for the same piece of punctuation across all uses in all Indo-European languages.
FWIW, Turkish is not an Indo-European language. It's an Altaic language [1] like Mongolian, Japanese, Korean, etc. Though, it looks like one because Ataturk (founder of Turkey) converted the Turkish alphabet from Arabic script to Latin script when the new republic was found.
> Modern style guides recommend no space before them and one space after.
And that's really interesting about French. Personally I think a colon surrounded by spaces looks funny, but then again, I've never learned a language where that wasn't the correct way.
edit: my French girlfriend says French puts spaces on both sides of all punctuation except commas and periods. Like this !
Funny story, I'm a native English speaker but I've been studying French for the past few years. I now constantly and subconsciously add spaces before punctuation like French grammar.
To be precise, the space before punctuation is of course non-breaking, and it is a narrow no-break space (i.e., thinner than a usual space) in careful typography (e.g., LaTeX with "\usepackage[francais]{babel}" does it).
(Another fun pedantic remark: French typography rules require that you apply the rules of a foreign language whenever you quote text in that language, so that the English rules should be respected for English quotations within a French text.)
And I agree with akx -- being French, I've seen my share of French people writing in English with the French rules, and I always thought that it could be used as an indicator by a careful reader.
(As an example, there used to be an ads campaign in the Parisian subway some years ago which advertised "I speak Wall Street English !" or something or the kind. As this was for an English class, I always found it funny that they couldn't get their English typography right.)
As an additional anec-datapoint here, I've noticed it's a relatively common habit amongst native english speakers who have some degree of dyslexia. I've asked about it once or twice, and the most consistent answer is something like 'I don't even notice I'm doing it; when I finish a word I insert a space, then think about the punctuation and add it on...'
This is not to suggest that this structure indicates dyslexia of course, the rest of this thread has plenty of other good reasons for non-native english speakers and otherwise.
Ah, I thought it was something more than poor instruction. I participate in some foreign language pen-pal services and I see this a lot, more often than any other English grammar mistake.
My guess would be that he is a coder and has a habit of spacing either side of a comma for his own readability preference. A dot between words would be like method calls in many languages.
The late Katja Kladnik once sent me a diskful of 'crackme' virii. I tried to deadlist one of them; it infected me when I did, and dared me to try a less obvious approach.
Mangled symbol table => buffer overflow in debugger => arbitrary code. Sneaky.
Using a disassembler to get the "source" (deadlist) assembly code rather than stepping through with a debugger.
The portions of assembly code in the article are deadlists
Those labs (specifically the binary bomb, buffer bomb, and malloc) were probably the most influential projects I ever did in terms of my career development. Great fun.
Haha, if that's the one I remember you could completely trivialize that lab by fiddling with the program counter in gdb (skipping the line that sets off the bomb). Don't know if they were able to fix that in later versions of the lab. Good times, thanks for reminding me.
When I did that, it seemed like it was intentional. You could only get to the portions that signal success to the server by cracking parts of the bomb which was what we were graded on. The "don't let it explode" part was really the hey here's how you can use gdb to stop your program before it does anything serious.
Yeah, I thought that's how everybody did it; when my class was assigned the project, we lost half a percentage point off our final grade for each time the bomb exploded. We very quickly learned to do things like "b explode_bomb".
So if I recall, you were supposed to have to do a lot of investigative work to bypass stuff. There were several different "levels" that got progressively harder to crack. But there was nothing that stopped you from moving the program counter directly to the code that signals success. Which is what we did. Every level, exactly the same: fire it up, move program counter, send the success notice, finished!
That was the point. When I did them, the TA encouraged us to do that. I'm pretty sure they had other measures in place that would have made it hard to fake an authentic success signal to the server. Furthermore, if you sent a corrupted one, you'd lose all credit for the assignment.
I loved this - a lot of the students really struggled but I already had background in reverse engineering, it was right down my alley. I absolutely loved this and for a long time, it's what I wanted to do professionally.
That lab was the most fun I've ever had with a homework assignment (too bad my own school didn't offer it...). I got it off the CMU website as well.
A couple of years ago I did a (mostly) full walkthrough of the Binary Bomb, and it still gets a lot of hits during the semester. If anyone is curious: http://www.vedranb.com/post/19338235616/phase-1
This is similar to about level 2 or 3, out of 5, of some cracks I've done for security CTF (capture the flag) competitions. Almost every competitive team/individual will solve this pretty easily, and I haven't even been in the elite competitions where it probably gets much harder. So, I would expect there are harder cracks or exploits, if the company wants to hire the best.
By the way, if this sort of thing interests anyone, look into security CTF competitions. You'll probably have to read some tutorials to learn, but CTF's will give the opportunity to apply what you've learned.
Hopefully not, I did a few similar crack-mes after reading just a couple of tutorials for dummies on the matter. On the Hack x Crack (awesome Spanish magazine from the early 2000s) forums, these were just the level 1 :)
Does this guy have any idea how hard it is to come up with good interview questions? By posting the question and solution online (complete with an md5sum and everything!), he has ruined the question, and his employer will now need to spend a significant amount of time coming up with another way to evaluate candidates.
A quick experiment shows me that you can call ptrace(PTRACE_TRACEME,..)
on OSX multiple times without it failing (the constant is actually PT_TRACE_ME on darwin). I wonder if that's the same for all BSDs ?
Interesting and educational writeup though, and just the thing to get me tinkering myself!
Yes microcorruption was a lot of fun and the last part of this post reminds me very much of the last level.
The nice thing about microcorruption was how you didn't need much more than familiarity with assembly and C (and language of choice in the end game). This interview challenge on the other hand... I wouldn't even know where to start.
I also tried to crack exactly this program a while ago. The company (I believe it is MilSoft, one of the most reputable software companies in Turkey) sent this challenge to university students to hire a part-time CS student. Nevertheless, this was the first time I've ever attempted to crack something and while I had little to no idea what was going on, it was a very thrilling experience. I think I went on 14 hours without taking a break.
I began by trying to run the program in GDB, got SIGSEGV'd. Afterwards I inspected the faulty address and tried to avoid it by changing its value, instead it crashed at somewhere else. After trying this hopeless catch-and-run for several hours, I decided I needed a better disassembly tool and went on to IDA Pro.
This particular program contains a trick that intrigued me very much, and it is the reason why I was getting SIGSEGV'd at different locations when altering the program code.
The main payload of this program is simply XOR-encrypted by some key. The whole thing begins by decrypting the payload and then begins its execution as normal. The gist is, the particular key that encrypted the main payload is the decryption code itself (for the unacquainted, assembly code is also just a byte stream). Here, this exact part:
As far as I can remember, the key was a bit more than that, but I'm sure it was including this part.
At the end of every iteration (of something involving this loop which I can't precisely recall now) the program checks whether it is running under debug mode (essentially makes a PTRACE call and reads its output, the OP also talks about it) If this is the case, it makes a jump to random address, so even if you are just neatly watching the program run under debug mode, you weren't going to achieve anything.
The next thing that occured to me is to manipulate how PTRACE returns its value, but I thought it would involve some kernel code fiddling and running the program under the modified kernel, which is WAY beyond my ability for now. I didn't know how to do it, but later by some very stupid trick I managed to pass this decryption part and the program made a jump to something like "__glibc_start". I needed to save the altered program and run it under gdb again (I don't remember why), but I was using the trial version of IDA Pro which prohibits me of such a thing. After making a few more desperate attempts I gave up.
But this "using the code as the key".. I think spending 14 hours to see this done was well worth it.
At the end of every iteration (of something involving this loop which I can't precisely recall now) the program checks whether it is running under debug mode (essentially makes a PTRACE call and reads its output, the OP also talks about it) If this is the case, it makes a jump to random address, so even if you are just neatly watching the program run under debug mode, you weren't going to achieve anything.
Could you change the jmp into a nop? That should let you attach in debug mode.
Could you change the jmp into a nop, then xor every nth byte of the program with jmp xor nop, where n = whatever offset the jmp was at in the key? The result should be a valid decryption.
Nah, if instruction "foo" is xor'd with "jmp", then xoring foo with "jmp xor nop" will remove the jmp and add the nop. Then the nop is removed during decryption, because "foo xor nop" xor nop = foo.
The reason this is clean and convenient is because nop is a single byte (0x0f or 0x90), which lets you replace any instruction of any size with nops. But if you had to transform jmp into an instruction of a different size, things could get hairy if the byte sizes don't line up. But you could still replace several instructions with other code.
Dude?? Did you wait until you go to Spain to post this?
Still, it is very fun to read this post and comments here.
Actually, I prepared these two crackmes in order to arrange a small competition among universities at Turkey. But, they became very good interview questions also.
It's really good to read these responses. Cracking ability is really rare in CS student community in Turkey. Our intention was increase awareness. Reading these comments showed me it was a really good step.
If so, they should say what company they are, because being associated with a clever puzzle like this is great for recruiting (even if it's not being used anymore). Unless they have their own reasons for remaining quiet (government? :)).
If not, they should probably take it down, as having the solutions posted would ruin the evaluative value of what must have taken a very long time to make.
Crackme's like this are like a handshake in the reverse engineering world. This is a basic problem and standard for what I'd expect as an interview first step.
Back when I was in high school, I had a Palm IIIxe. This was the days before app markets and nearly everybody who made PalmOS apps tried to sell them as shareware with a price of $20-50 -- well beyond what I could afford as a broke high school student.
Fortunately, I had learned Z80 assembly programming my TI-83, which had led me to dabble in 68k assembly when I bought a TI-89. I never mastered 68k the way I did Z80, but I knew enough to find the routines that ran the registration key check when the OK button was pressed, and by trial and error, I'd invert conditional jumps until I found the one that would turn a failed registration attempt into a success. Then I'd hex edit the binary to make the switch. Worked like a charm about 80% of the time!
what's a good, simple intro to the basics of this kind of cracking for someone who is an experienced programmer, knows some C, etc, but has little system level or assembly experience?
I would really suggest purchasing a good book about it. From my point of view, I would go for "Hacking: The Art of Exploitation, 2nd Edition" from Jon Erickson which goes in depth about how to crack programs using gdb and other tools. It's really a wonderful book if you want to learn more about the world of cracking in general, and it doesn't require much prior security experience.
I've done some of this book. It's good, but you're better off knowing the basics before trying to really learn from it I think. I only knew a little C when I started, so I only got about 4 or 5 chapters after much googling before I drowned.
Learn some ASM and get familiar with gdb before attempting
It's definetely out there if you have only done a little C. It doesn't require any previous security knowledge though as I stated. But like you realized, it does require you to know your way around low-level programming.
This is totally nostalgic of the "binary bomb" assignment in CS 107 @ Stanford. You have to run the program from Stanford's network. There are 6 levels and each level has a password you have to enter. If you enter the wrong password, the course server is notified, and a point is deducted from your grade. The correct way to solve each level is to disassemble the program and figure out what it's doing.
We did that in our second year and it was awesome. I don't know if it was the same binary but it was well made. The first password was very easy (just a constant string) and it got progressively harder from there. At the end we had to step through some recursive algorithm and you learned to read assembly almost as well as normal code. And the server for submitting the passwords didn't filter the username so you could enter arbitrary HTML into the high-score...
Should have sent them a password locked program called *DoorHasBeenCracked". The only thing it does is post passwords to an http server that you control. There is a good chance they try their own password on it. New school phishing attack. /s
Ignoring the /s tag, it sounds apt for a hollywood movie or a sci-fi show, but in reality it will most likely be run on an unnetworked VM. But funny nonetheless :P
The post started very well but with the first screen shot, my mind started tingling: What the heck a security engineer is doing in a root shell? An unknown binary sent via an email is run in a root shell. There is also no mention of email source tracking.
Hey you are a security engineer you know about weakness of smtp right?
Even if this is a virtual machine, I would really reconsider employment of him or sit down and do a serious talking about this blog post if I were the employer.
I could not continue reading the post before ranting about it.
Is it really that probable that an email was sent to him from the company he is applying to and someone spoofed that to send him malware?
I agree with the root point only because the company could have easily done something like `echo "I just rm -rf'ed your / because you ran me as root"` as part of the test.
Why would you consider it still an issue if it was a VM he used only for this purpose?
Well, email is the primary threat distribution medium right now. So If a security engineer do not show scepticism about an unsigned, unencrypted email from an unverified source, I get picky.
Looking from probability perspective, yes you are right, this is a low threat vector.
Also I consider working root in a vm an issue because security 101 lesson 1 is "avoid privileged accounts as much as possible". Why not work in a unprivileged shell account and use sudo whenever needed? VMs are not bullet proof and they can leak memory, can make host machine unstable or even crash it. There are hardly any poc's out there but VM's may be exploited to switch context to host machine. Aside from these low probable threats, while working with unknown originated binaries, losing your whole work is a big probability. In this case it would not matter if you are inside a vm or not. VM's can be recovered but lost time can not be.
"Security 101" isn't all that relevant when you actually understand the threat vectors. If you can't create a clean, isolated, snapshot'd VM for this sort of playing, you have no business applying for this sort of job.
Old school virii that could burn your display card comes to my mind. Working with root is always a threat. A livecd is consience but still does not make it fully secure.
I told I am being picky. Most of my concerns may be handled but their are not shown to be.
Most pentesting distros run as root by default, check out Kali Linux which is the industry standard. It makes it a lot easier to run all the strange applications we use. As long as you keep it in a VM you'll be pretty solid.
I love the way the way m00dy dissected the problem. About 2 months ago, I was watching some advanced LLDB videos from Apple and they went into a lot of the tricks detailed in this post for setting breakpoints and debugging a program. That being said, some of the knowledge about halting commands and configuring gdb to ignore debug mode are just some things only a pro would know.
So the only way for programs to get data from the outside world is to poll with system calls? I always thought that programs defined a "holding area" that the kernel would write into when it had data - the program still might poll, but it's polling (potentially very small, perhaps a single register) local data rather than making a system call.
Largely, yes. Which is why strace is so useful as a debugging tool. It's also why you want to do buffered IO (read/write in kB at a time) to minimise call overhead.
There is mmap[1] (controlled, unsurprisingly, by syscalls), which will give you a memory region that may be updated transparently for you by the kernel, but in general, it's all syscalls.
The other nice thing about the syscall interface is that it gives the kernel an opportunity to suspend the calling process and schedule something else if it can't fulfill their request right now (hence, blocking IO), so you're not actually polling. Of course, a preeemptive scheduler (like everything actually is) can step in and suspend you anyway, but the syscall is a nice place to do it cooperatively.
I have a pretty cool crackme that I programmed and I wanted to offer it as a puzzle to some candidates, but without the proper reverse engineering tools, I think most candidates would really struggle -- especially if you're looking for just general developers.
I've always believed a test a a fair way of hiring. It takes "the good ole boy", and the whole "my friend is brilliant" out of the equation. Personally, I've never liked, actually despised, the whole networking thing.
This applies even to sysadmins. We have a favourite: set up a VM with a slightly-broken application in a slightly-broken Apache and Tomcat, and get them to ssh in and document the process of fixing it. Even people who aren't a full bottle on Tomcat will give useful information, because we get an insight into their thought processes. I recommend this to all.
(I note we've just done a round of interviews where we get a nice-looking CV and conduct a technical grilling. Hideous waste of time for everyone involved. All CVs should be regarded, on the balance of probabilities, as works of fiction. Do a remote self-paced test like this. You won't regret it.)