If you wanted to handle this scenario with the serverless AWS stack, my recommendation would be to push records to Dynamo with TTLs and then when they pop have a Lambda push them onto the queue. Would cost almost nothing to do this. If you had 10 million requests a month your Lambda cost would be ~$150 to run this (depending on duration, but just pushing to a queue should be quick). Dynamo would be another ~$50 to run, depending how big your tasks are.
Granted now you need 3 services instead of 1. I personally don't find the maintenance cost particularly high for this architecture, but it does depend on what your team is comfortable with.
I've explored this space pretty thoroughly, including the Dynamo approach you've described. Dynamo does not have a strict guarantee on when items get deleted:
TTL typically deletes expired items within a few days. Depending on the size and activity level of a table, the actual delete operation of an expired item can vary. Because TTL is meant to be a background process, the nature of the capacity used to expire and delete items via TTL is variable (but free of charge). [0]
Because of that limitation, I would not use that approach. Instead I would do Scheduled Lambdas to check for items every 15 minutes in a Serverless Aurora and then add them to SQS with delays.
I've had my eye on this problem for a few years and keep thinking that a simple SaaS that does one-shot scheduled actions would probably be a worthy side project. Not enough to build a company around, but maintenance would be low and there's probably some pricing that would attract enough customers to be sustainable.
The young people I know hate going into the office, it's all the older folks that want to go in. I've been almost entirely remote since I've graduated and haven't missed being in person at all. I've developed my skills more being left alone and having time to read and learn on my own than I ever have being forced to sit in an office and struggle to be productive. I'm absolutely not established, but I've had no issues convincing others of work of my impact. You simply need to take writing seriously and learn how to communicate your work.
Companies are just learning they have terrible onboarding processes and documentation. Turns out those things are important and being in-person is an expensive and poor quality bandaid.
Yeah, the only young people who I know want to go into the office are interns and fresh-grads. There is a lot of in person upskilling that happens in those years, but after that, learning is effectively self-guided.
There are 2 kinds of jobs in tech. Demanding jobs and chill jobs.
Demanding jobs involve effectively resigning your life to the company for 5 days of the week. The hours saved on commute and ability to actually get chores done between at-home downtime allows people to put a lot more hours from home than from an office.
Chill jobs on the other hand, lead to a terrible commute vs hours worked ratio. You're in the office for 40 hours of the week, but spend 5 hours commuting, 5 hours chilling during lunch break, and another 5 hours waiting in line to select one of 10 fancy coffees. That's effectively 30 hours of work, 10 hours of in-office loitering and 5 hours of out of office mind-numbing commute. Make those same people work 'real' 40 hours from home and you'd see a 30% productivity increase just from hours saved. Remote employees take meetings during lunch, coffee is instantaneous and there is no commute.
People complain about how much better in-person meetings are. But, have we even tried to make remote work palatable. Slack and Zoom are terrible tools for remote work. Give every employee a 2000$ digital whiteboard. Make them stand and give their presentations using better cameras. Use gather.town to make collisions feel more natural. Guess what? with remote work, all your meetings & interactions can be captured. You can send out meetings summaries & key screenshots for every interaction without ever lifting your finger. But nope, remote work is relegated to having the same in-person secondary tools (zoom, slack) without leveraging new primary tools that work well for remote work.
I find personally that my math ability is set to approximately 2-3 levels "below" the highest level math I completed, and I've seen hold for others. I have an applied math bachelors so I've taken analysis, dynamical systems, and other high level math classes, but I find that the stuff that I actually remember at a level to pass undergrad exams is up to linear algebra or maybe a little more advanced. Of course if I were to relearn it'd be much faster, but years of being a software engineer have caused me to forget all that stuff.
I was literally about to write this same comment. I did physics so I finished some group theory and complex analysis, but 20 years later all that stuff is gone because I never applied it in other classes. Only the stuff that kept coming up, like calc 1/2 and first order diffeq, really stuck.
With that said, does anyone know of a good method to relearn math efficiently? I found it to be really hard to self-learn any math topic, most books repeat everything from the basics at the beginning like what a set is, and then suddenly turn into ultra-advanced with “the proof is trivial” all around.
I think this experience is typical of self teaching from a math textbook. It's extremely difficult to find a good book that leaves no gaps while simultaneously explaining everything that might be difficult to understand. The key thing when encountering this for me is to expand my horizons and begin looking for videos or other supplementary materials. A teacher would show you the proof, or at least help you along the way, the internet can be used for this as well, up to a point.
While it's frustrating and time consuming, self teaching a difficult subject is just like that unless you're a god amongst men (and few of us are). Sometimes you'll want to fight through the strange unproven thing by thinking hard for a couple weeks about it while googling intermittently to find key steps. Sometimes you'll have to give up on it and keep moving. If it's foundational then fighting through can be highly beneficial, but a lot more things are presented as foundational than actually are.
I'd recommend finding good books by searching relentlessly on reddit and other forums for opinions, dedicating the time necessary to self teach something difficult (it can take upwards of a year to get through a smaller textbook if you have other things in your life going on), and if you really want it then fight for it. Give it everything you have, really let the problem consume your thoughts because eventually you'll wake up at 3am and know exactly what to do. And finally, move on if you don't want to do that. Try just keeping moving. Review from time to time but don't let a hard first couple chapters prevent you from ever learning the concepts. Or you could find something you want to know and work backwards through every term that's used until you're at a concept and then attempt to apply it to the larger idea.
In general, self teaching math is extremely difficult, and only really works if you're willing to dedicate the time to fight through ideas.
This is a great comment. To add on to a point that really aligns with my experiences:
> "Review from time to time but don't let a hard first couple chapters prevent you from ever learning the concepts."
This is a very good approach, and I wish I started doing this earlier. Even in my university math courses, the professors sometimes skipped ahead to have students focus on a few later chapters before coming back, or told the class to skip several pages in the book. I also found that working on later exercises in a textbook would sometimes help me better understand concepts introduced in earlier chapters.
Lastly—though this may not be completely relevant to studying mathematics—I've explicitly been taught in various language courses (explicitly for audio courses and implicitly for in-person university courses) that it's okay to move ahead if I know at least 80% of the material. The percentage may be higher for studying math topics, but especially for someone self-learning out of interest or for a specific application, it's much more preferable to move forward and revisit earlier exercises as needed, instead of quit the book. If you find yourself getting lost in later chapters, there is no problem with revisiting earlier chapters. You'd also likely be no worse off (possibly even better) than many undergraduates studying the textbook for a course for the first time.
The most important thing is just to not quit the habit of consistent study. Perfectionism in understanding is a pitfall for self-directed studies, which consistency in studying beats every time.
> it's okay to move ahead if I know at least 80% of the material.
One of the worst feelings is moving on in a textbook and realizing you are indeed totally lost.
Here’s my own list of necessary but not sufficient conditions for deep learning:
- Motivation. Easy to overlook; hard to get if you don’t already have it.
- Frequent experience of “I have no idea how to solve this,” followed by hours or days of playing with the problem, followed by a eureka moment. You can’t be sure you’ve learned the thing unless you’ve constructed the solution yourself. Builds confidence too.
- Seeing the same material in different contexts or presented in different ways. It’s like looking at an object from different angles.
And for bonus points:
- Teach the concept to a curious friend. Their questions will lead you to deeper understanding.
I wish I could agree. Every aspect of pytest that I like (fixtures, plugins, etc.) is marred by a boneheaded move to make them autoloaded by default. Loading should be explicit/opt-in. It's dumv that if you pip install something that adds a new pytest plugin it'll automatically modify your runtime behavior. Some of the plugins are great (shoutout to pytest-socket), but it has the same issue flake8 has with the plugin ecosystem where the quality varies wildly.
I might be in the minority but I also find python's mock implementation awful. Mockito in Java is way better where you explicitly state which calls you expect and if you get an unexpected call it errors out.
Is this really a positive thing? It creates in-groups and decreases diversity. Sure, it makes hiring easier but this is exactly how you end up with old boys clubs.
One funny incident I had on this topic was when I was TAing calculus one student proudly proclaimed that there's no gravity on Mars while doing velocity problems.
This didn't stop them from using the gravitational constant provided in the assignment or from getting the correct answer even, but it was funny regardless.
Regarding your last two points, you're absolutely spot on. I don't have much in terms of data, but I want to at least offer my story so that people understand how fundamentally broken the system is and how it discourages you from doing anything other than leaving.
I internally transferred at Amazon and had many internal offers to choose from - so I went with a team that had great tech survey results and where the manager seemed like an honest guy that I could trust. In < 2 years I had 6 different managers at Amazon, so I felt like I could read managers pretty well and knew what I was looking for. I joined the team and was ecstatic, up until the manager announced he was leaving 2 weeks later. I had to report to his manager who I had interviewed with and could tell I didn't work with. He had newly been hired by Amazon just a few months earlier, and he was exactly the type of micromanaging bureaucrat that Amazon gives too much power. He also clearly didn't want to manage ICs and they weren't clear to him during the interview process that he was going to manage ICs rather than managers.
Things went about as poorly as you can expect. The new manager had no technical knowledge, poor leadership, and blatantly favored the people he was managing before he inherited my team. But I'm not sure if I can convey how bad he was. He would have engineers manually updating excel spreadsheets for his monthly business reviews as part of their oncall work (which was literally his job). He would gaslight engineers into telling them they'd be promoted, and then hand them a PIP. He would lie at status meetings and has numerous documented performance issues and complaints in his connections and directly to his manager and director. His performance was so dismal that our Senior Manager made his team meet together and write down ideas (anonymously) for the manager to stop being so bad. He would blatantly not pay attention in meetings and ask us to repeat whatever we went over in the previous day's meetings (which he attended himself) in standup. He would never back up his engineers on any push back, even when we would get paged to do things outside of our SLA windows and completely let our stakeholders run us over with requests and expectations.
No list about all of his horrible practices could be complete, but I want to emphasize that he was so disliked that he caused 9 people to leave his team in 10 months. I don't want to say I was the best engineer, but I got positive feedback from all of the senior and principal engineers I worked with and I thought I was on a steady path to being promoted. But it turns out this manager realized he could weaponize performance ratings, and gave me and the other L4s that he inherited the lowest performance rating possible while giving his original reports promos and the highest rating possible. This wasn't an absolute shock to me and I already intended to leave Amazon because I couldn't stand him any longer, but this set my team's senior engineer off. He was absolutely livid and escalated the issue to the senior manager that this was a disaster and completely ridiculous. The senior manager met with me personally, told me it was a problem and that he'd make it right. He and the director decided to have my team report to a new (remote) manager, but for me it was too late and I decided to leave amazon.
But what happened to the manager? Well, I wish I could say HR stepped in and fired him for weaponizing ratings, or his bosses PIPed him for being awful, but actually none of those things happened! He caused ~60-70% attrition, had documented performance issues, and abused tools in a documented way and still manages at Amazon! A few months ago he internally transfered and as far as I know his new team doesn't have him on a PIP yet.
What is the takeaway for anyone reading? Just remember that just because you interview with a manager doesn't mean they'll be your manager forever, or even really at all. And it's okay if you underperform at Amazon, as long as your title is SDM. The last thing I wanna say is that I really liked the Senior Manager as a person, but he was too nice to the shitty manager. PIPing is inherently a bad system, but if it can't even catch someone who is so blatantly underperforming for a year straight then it doesn't even do what it says on the box.
Also no matter what don't join Amazon Go Boston. Just as a heads up.
Granted now you need 3 services instead of 1. I personally don't find the maintenance cost particularly high for this architecture, but it does depend on what your team is comfortable with.