This sounds logical enough but I think I remember reading that larger embeddings were only better for certain classes of words, because increasing the dimension size (in those cases) introduced noise. Will try to find the paper — it’s escaping me at the moment.
That's all true but matters significantly less "at scale". In the days of lean models, you needed to verify that your input parameters were functionally independent variables, meaning they couldn't correlate with other input parameters. When every document is transformed into a billions-long vector -- even if you took the noninsignificant amount of time it would take to compute a correlation matrix -- the heavy associations between a few features don't mean much, especially when you can just add more data. Plus, people misusing or repurposing words can introduce some interesting twists to features you'd assume 1:1 on paper.
Does the paper reach some conclusion on an optimal embedding size?
I was thinking about that the other day. It's interesting from a linguistics perspective. I wonder if each dimension in an optimal-size vector could be given a human-comprehensible label.
When I lived in Kampong Speu, Cambodia, they’d burn plastic trash as a means of disposing of it. It was the only time in my life I ever experienced black snow.
I can’t go into too much detail here, but I interrupted a severe depression by fasting for two straight weeks. It’s hard to say why it worked — my friends assume that the microbiome is implicated because of the dietary change. Plausible, but hardly a proof. This gives me at least one direction to look in.
As an alternate theory, I’ve always thought that deliberate, controlled suffering gives you a mood boost. Or rather, the relief after the suffering has ended. The theory is that it’s not good for us emotionally to be comfortable continuously. Maybe fasting is an example of that?
That’s personality-specific. My Dad is like that and doesn’t understand why nobody else in the family gets off on self-imposed austerity. (He’s a great guy by the way — we all have our quirks.) He loves to say it’s a lack of discipline, but it’s not— I quit smoking after a decade-and-a-half of heavy smoking, lost 20% of my body weight on an 800 calorie per day ketogenic diet, worked many very difficult jobs, and did a number of other things that have required sustained discipline, as have others in the family. He doesn’t understand that the difference is that he enjoys self-imposed misery and the feelings of superiority he gets from it, which not only leads to a lack of balance in his life, sometimes strains his closest relationships.
> My Dad is like that and doesn’t understand why nobody else in the family gets off on self-imposed austerity.
My gut feeling: men of a certain generation seem to have confused the skill of seeming aloof-- like when meeting a stranger or thrust in a new situation-- for being emotionally unaffected in general. One sign I associate with this would be talking only in the past tense about having felt certain emotions, but you never really witness the person feeling or expressing those emotions (outside of anger/frustration). Alternatively, the person may never really engage in discussions about certain negative feelings, unless it's to offer low-effort problem-solving advice to others.
This is difficult because there are obviously times when a small, seemingly insignificant problem can trigger an outsized emotion. It's natural and helpful to be able to sit with that negative emotion-- to feel it, express it and talk about it as an emotion you are experiencing-- to be able to eventually come to terms and get to know better whatever it is that's driving it. It's scary to do, but most people have some techniques for doing it.
If you have few or no tools to do that, it must generate an immense amount of stress. Hence wacky alternative stress-relieving techniques that are more physical and less emotional in nature. (Plus the projection of "lack of discipline" makes me wonder if he'd feel shame from sitting with a negative emotion.)
I’ve seen that in people though I don’t think it really describes him, specifically. He just really enjoys a lot of things in life the way some people really love getting the most punishing workouts possible, and would probably do it even if there were no health benefits.
I don't find it difficult to talk about negative feelings, as long as no one is listening. Talking to other people is mostly useless these days, they're all busy hiding from themselves.
A good technique to help w this is "affective labeling" (sorry, citation needed, I think I got it from one of Anne Laure Le Cunff's typically awesome newsletters/posts). Set a 5m timer, start writing words that describe your emotions and don't stop writing for even 2s till the 5m are up.
As an aside, I miss the slashdot ability to optionally assign a property to a vote. It would be neat to have some sort of indicator in addition to the greying out of downvoted or flagged comments to show specific types of upvotes. This has obviously proven to be the better system overall, but it has its pluses. Too bad it also had a bunch of swastikas and 4-Chan-esque trolls.
I'm not sure I agree this has proven to be the better system.
Slashdot had, for many years, fairly insightful commentary with that system. The fact that it has since become less insightful is, I think, less a reflection on the voting system in comments and more on people moving from the site wholesale for various other reasons.
I don’t mean the more effective voting system — I mean the more effective system overall. I don’t really think that’s controversial. As I said, I like the Slashdot voting system
> One sign I associate with this would be talking only in the past tense about having felt certain emotions, but you never really witness the person feeling or expressing those emotions (outside of anger/frustration). Alternatively, the person may never really engage in discussions about certain negative feelings, unless it's to offer low-effort problem-solving advice to others.
That's really insightful! Thank you. Saving your words in a scratch note for later
That's a lot of psychoanalyzing of people you don't know.
If it helps give you some perspective, I've been through a decade of therapy and years of group therapy, and I haven't found that my interest in "discipline" lessened or became less relevant as I've come to grips with emotional issues and learned to relate to people in different ways. There hasn't been much interaction between the two, except that when I'm doing well at one I tend to do better at the other, for the obvious reasons.
An alternative explanation is that voluntary suffering is a comforting reminder that you can do it when you need to, as well as an effort to maintain that ability. My observation is that people who engage in "gratuitous" suffering are people who were raised by parents who didn't pass on the basic skills of discipline (among which are important emotional skills) and had to learn it from scratch later in life. Since they didn't grow up with it, they don't take it for granted, and they exhibit the "zeal of the convert." Or, if they never achieve it, the persistent lack keeps it at the front of their minds.
The parent poster seems to have quite a lot of discipline but doesn't feel any need to remind himself that he has it. Maybe that's because he learned it as a child from his father?
My father had a hard and restricted life growing up, and he wanted me to grow up in a more free and easy way. He wanted me to feel free to goof off and enjoy myself. I certainly learned that, but I also grew up with a lot of anxiety about my inconsistent ability to apply myself to things I cared about. Sometimes I did, and thrived, but very often I disappointed myself. I was especially poor at boring things and things where consistency mattered, such as taking care of my health, but often, even at the things I was good at, I failed to put in effort at a crucial time and screwed up something that was important to me. My easygoing father wasn't going to tell me I "lacked discipline," but my frequent underperformance and self-failure told me loud and clear.
So I spent years figuring it out and getting better at it. Like anything that doesn't come "naturally" (i.e., wasn't learned in childhood) it has always felt like an artificial bolted-on part of myself that could fall off at any moment. So I consciously tend to it. I don't show more discipline in my life than other people, but I certainly show more interest in it, and I do some things that probably seem weird to other people. Other people probably think it's a meaningless focus that has no impact on my life, since I'm not conspicuously different in my habits and accomplishments. But not being conspicuously different is the accomplishment!
A lot of people can relate to the effort required to maintain a healthy weight after growing up overweight. People who take a healthy weight for granted might think that an above-average amount of attention and effort should yield an above-average physique, or what's the point? They might feel amused or condescending towards their friend who thinks and talks so much about what they eat but looks very average. They might get annoyed that their friend declines an extra round of drinks or skips dessert. What's the point? It's not like they're maintaining a fabulous physique. They look like anyone else, even slightly below average. Clearly their obsession with eating doesn't accomplish anything other than annoying the people around them. But it does accomplish something. It lets them be closer to normal.
“Enjoys misery” is an oxymoron I think. He enjoys joy, which he gets from doing difficult things. I’m just taking your dad’s side on this because I suspect I share some of his tendencies :)
Austerity is not the same a misery. In some ways I live an austere life but I am happy. Misery comes from poverty not self imposed austerity; the first is often imposed by the outside world whereas your father's and my austerity are, as you say, self imposed and under our own control.
You’re making arbitrary definitions for those words. Austerity can absolutely lead to misery and some of the happiest times in my life have been some of the most impoverished.
Does he not understand the situation even when you lay it so bare in front of him? People tend to get off the course sometimes on their own, but smart mature ones prefer honest correcting feedbacks (to certain extent), if it means ie better more stable relationships with closest ones
He’s pretty hard-headed, but so are his middle-aged sons. These days we’re all grown-up enough to know when to sort of shake our heads and change the subject. My mom has her own way of setting boundaries which seems to work for her but unless she wants to talk about something specific, it’s normally nothing I need to even think about. She feels comfortable enough confiding in me though. To be clear he’s a 100% stand-up guy, doesn’t have an anger problem, and even though he can be frustrating to deal with sometimes, there is never a question about him doing what’s best for his family, even if that means swallowing something he’s annoyed by if it’s necessary.
> People who self-harm, writes Arnold, have “learned that, while the pain peaks with self-injury, it then comes down the other side. The physical pain lessens – as does the emotional pain.”
People don't self-harm because it hurts. They self-harm because it _feels good_. Pain is "pleasure", for lack of a better phrase.
There's also the not so constructive tendency to punish the body for what the brain comes up with and the feelings that come with it. It's really the only part of yourself you can harm.
Most of them self harm because the removal the pain feels good - and it takes with it some emotional pain.
Though there probably are some masochists that truly enjoy the pain, but that's a different condition than the more common I-hate-myself-so-I-hurt-myself.
Not quite. The pain causes an endorphin release. It's also addictive (not technically by the DSM sense, but those who do will often call it as such) and has been compared to heroin.
This interview about dopamine and homeostasis in the body would support the idea that resetting that system through suffering would likely be a valid approach:
When I learned about the hedonic treadmill I thought back to a number of athletic programmers I’ve known who always seemed to be a lot more put together than the rest of us. I always figured the relationship went the other way. That having your shit together meant more time for activities. It may be the other way around.
As a person who did 21km run in fasted state (didn't eat for 36 hours before) and then swam in a 7 degrees pond with ice, I can definitely say: NO.
I tried for 8 hours straight to code just a little bit, but was unable to. I guess that is burnout, or, in general, understanding that I am not interested in
all this computer stuff anymore after doing it for 15 years.
Interesting hypothesis. Mental acuity does seem higher while fasting. Is that a dopamine specific effect, or could there be a handful of hormones impacting mental state?
Anecdotally, I've observed I feel better with a certain amount of difficulty in my life and felt particularly bad when there was nothing -wrong- but I still felt depressed. If you don't have any problems you can attribute the bad feelings to, then you naturally start to consider the possibility that the problem is you and you're broken in a way that means you're going to feel bad regardless of how much your life circumstances improve. Which is a particularly despair-inducing thought to have.
Last year I cracked my hip joint and ended up in the hospital for a couple of weeks, doing physio to regain my ability to walk. I certainly don't want to repeat that experience but I was surprised that I felt less depressed during it, because there was clarity in what I immediately needed to do and I was focused on just getting through it, not existential angst.
I'm no stranger to discomfort. I enjoy it. Like, my bed is a piece of plywood with a blanket on it.
The increase in energy and clarity of my thinking, the tingly feeling in my body, and the improved sleep quality -- just to give you a few examples -- are hard to attribute to the mechanism you're talking about. To add to that, I regularly fast for five days at a time and I have not yet experienced the same kind of mental difference. This time I pushed it to two weeks because my energy levels continued to fluctuate upward until the 12th or 13th day.
You do feel a lot of relief that the boredom of fasting is over. But that effect lasts maybe a day or two or three. Certainly not months.
> Like, my bed is a piece of plywood with a blanket on it.
Other than the potential for mold accumulation (you definitely want some air to circulate), this is not actually a bad bed base for sleep quality. Depending on your size, it's likely better for you than some very expensive plush mattresses.
Plywood has give to it that a hardwood wouldn't, so what you're sleeping on isn't entirely rigid anyway.
But drill some holes in it :)
I fast somewhat frequently, and I don't really associate it with pain or discomfort, but I'm also doing it for 24-48 hours.
Extremely long fasts (like your two weeks) should come with a disclaimer, they're not for everyone.
Certainly, the highs feel better if you're coming from a low; but I tend to prefer my highs coming from mids. I will say that, although not a low, the sweet relief of getting through a stressful situation/project is great.
Anhedonia is sometimes used to refer to 'flat affect', which I guess could be taken as 'mid', but usually it is used to describe a lack of joy/desire that is quite common in major depressive episodes.
I believe this is a well-understood feature of the dopamine system? The sensitivity of the receptors is like a balance scale, and will correct to one side if the other is flooded.
Give yourself big dopamine rushes and the scale balances by reducing your sensitivity to dopamine and (crudely put) causes feelings of discontentment, and you’ll need more dopamine released to feel like normal. Alternatively, push down on the pain side of the scale by doing some controlled suffering — fasting, cold plunge, intense exercise — and the receptors become more sensitive, and you feel better.
I’m sure this is too simplistic of a model but it makes sense to me in terms of lived experience.
A two week fast is incredibly difficult, accomplishing it, taking part in etc is very meaningful. Consistently throughout time and history a sense of purpose has been remarked as key to human happiness.
The obligatory “To love is to suffer. To avoid suffering one must not love. But then one suffers from not loving. Therefore, to love is to suffer; not to love is to suffer; to suffer is to suffer. To be happy is to love. To be happy, then, is to suffer, but suffering makes one unhappy. Therefore, to be happy one must love or love to suffer or suffer from too much happiness.”
Behaviour therapy would say that it worked because you took some action to improve your life and counter the depression. (Even cognitive therapy says that small, meaningful actions can have a dramatic impact on one's depression). The motivation and action of fasting, and the positive and noticeable / observable physical effects of fasting changed your focus from your depression, which alleviated your depressive symptoms. Once depression alleviates, you start thinking more clearly (make less cognitive distortions in your thinking) which further leads to a 'normal' mental state of well-being (in an otherwise mentally healthy individual).
Yes. A long fast can reduce overgrowth of harmful bacteria that survives on a continuous supply of food. This paper mentions Morganella morganii, which is an example of harmful bacteria that thrives in a environment full of undigested proteins and carbohydrates.
If your depression remained abated after resuming a healthy diet, I'd say the microbiome hypothesis is intriguing. Otherwise I'd there are way too many complicating factors at work during a fast to dwell on that.
Also, you could toy around with different fermented foods while making no other changes.
I completely cured my severe anxiety and minor depression by eliminating wheat and sugar from my diet. I come from a family where everyone has digestive issues and some type of illness that is tentatively linked to the microbiome.
For how long, is the question though? Depression and anxiety are like the "common cold" of mental health suffering, and everyone endures it a few times in their life.
That may be true for severe cases. But if you consider the checklists used to determine depression, it offers the results on a scale. For example, the Becks depression Inventory (BDI - II) suggests that the depression maybe either "minimal, mild, moderate or severe". Similarly the Burns Depression Checklist (BDC) tests identifies depression as either "mild depression, moderate depression, severe depression or extreme depression". Anxiety is also measured in a similar scale.
The "mild or moderate" segment of depression, or anxiety, is what is said to be a more common short-term occurrence amongst all of us. Most of us manage to often deal with this kind of "situational" depression / anxiety due to some stressful event (losing a job, death, breakup etc.), on our own. But long-term and / or severe / extreme depression is what often requires an outside intervention.
This has me very curious! How did you decide on eliminating wheat and sugar? Or did you cycle through various eliminations until you found a combination that worked? And how did you know when to keep going/stop?
I was doing a big job for some hippies, and they fed me lunch every day. And even though I thought it was bullshit, I was polite and ate their gluten free, sugar free, organic, free range, etc, etc food. My entire life I hated eating, but I quickly realized that I really enjoyed eating with them, and I felt really good afterwards.
And since then I've done a lot of experimenting and reading other peoples thoughts, and reached the conclusion that simple carbohydrates are fucking terrible for you, for multiple reasons. The biggest is that it causes an insulin spike. If you're then physically active, you'll burn off the sugar you just ate, but your insulin levels stay high, so you'll never start burning your body fat. You just get hungry and eat more sugar.
There is also a known phenomenon of "starvation euphoria." I don't know at what point of caloric deficit that kicks in, but it probably varies dramatically.
I assume it was a water fast. Did you supplement with anything? Did you have a doctor's supervision during the 2 weeks? How did you reintroduce food? I've always been fascinated by long fasts, but I'm scared to try them.
I had some electrolytes at first but ditched them on the third day. After that it was just plain water. If I had to do it again, I would take potassium and table salt throughout.
I did have friends and my girlfriend with me for a lot of it.
Yup, potassium - or part of a banana. I've read that most deaths attributed to crash diets are caused by cardiac problems caused by potassium deficiency.
Asking for medical advice is a waste of your doctor's time? I'd rather do that than trust a bunch of random posts from people I don't know on a technology website.
Good for you! This is how it all starts. By adding foods back slowly you may start to find what foods are a trigger for gut inflammation and depression.
There is one medical diet, called the FODMAP diet you might want to look into. It helps you find what foods might be causing some bacterial in your gut to become imbalanced.
I am not at all saying it is all you microbiome, foods may affect us in so many more way than only in the gut. It may be that not eating has lowered oxidative stress in your body allowing your mitochondria to recover and are now able to provide the energy you need.
I think there are other plausible answers, and I'm generally skeptical of people who believe there are diet cure-alls, especially for depression. I'm not a psychiatrist, but I do have a bipolar 2 diagnosis. I only found out it wasnt a unipolar depression because I had a hypomanic episode. I can only describe a hypomanic episode as a wave of constant energy, and I've also heard it described as: 'being on meth for a week straight'. It gave me the energy to forego food for multiple days because I didn't feel the pain of hunger pangs. I also didn't sleep for a few days, not because I wanted to, but because I just had too much energy.
I'm not a doctor, but hopefully this is an example of other places to look that aren't diet related.
I certainly agree there's a connection from the gut to the rest of our body (including mood) that we still don't understand... But I'm curious if you've done anything to rule out psychosomatic ? I saw a brief mention that you think there is a link to childhood experiences, which means you do agree there might be a mental connection... Have you tried a blind experiment at all? I could see it being relatively possible if you had someone prepare something liquid, steps were taken to conceal the flavor, and perhaps using a straw to limit how much you can notice.
I did have some experience with dog food, where I'd feed the dog and after a few weeks noticed joint pain. I'd not paid attention, but there were beets in the dog food. That was kinda blind until I noticed symptoms. I realized, I was licking the leftovers on the spoon before I washed it by hand. Not sure this counts as blind testing.
Sadly it feels impossible to find a doctor in germany that knows what to do about it. The last success was with something like oxalobacter. For a few months now I could eat at least some salad, but seemingly that effect weakens over time.
So I think I have found a probiotic that works, but it doesn't stick well. This article mentions genetic traits being connected to the biome. So I think I'll have to have genetic testing eventually, but I'm way paranoid to hand over my genes to some random lab.
Interesting, I had a bout of depression that I believe started when I did a monthlong low carb low calorie diet (with doctors and nutritionist). Gut health is a huge factor.
Low carb and low calorie for a month sounds rough in general. Maybe the depression could be attributed to exhaustion and a suppressed metabolism as well.
Very individualized. I have drastically better gut health, skin health, energy when on a low carb diet for an extended duration.
Maintaining that diet for an extended duration, however, is always the challenge when others in the household do not follow it. Logically, I should stop being a baby and just power on with it. Or at least determine what in my "normal" western diet negatively impacts gut health/skin health and overall energy.
Having dealt with strange neurological states, i somehow got sensitive enough to perceive what could be changes in the gut microbiome (depending on diet, probiotics intake, antibiotics) these would have similar effect as change in serotonine based therapies, albeit short lived.
I wonder if people who are questioning if fasting 2 weeks is the main cause of interrupting your depression did ever fasted a week.
Would you consider that you "cured" the depression?
Yea. Many of the comments hurt me to read. I'm trying to remind myself that they come from a place of curiosity and wanting to rule out confounding variables. But they do make a lot of assumptions about me and things I know and/or do.
To answer your question: I hesitate to use the word "cure", because 3-4 months on, I'm starting to wonder if certain things have begun to slide back, or if it's just been a challenging week. Time will tell.
A datapoint that I did not mention in my first post was that there was a negative, persistent, digestive side-effect that paralleled the positive mental changes. As that has faded, so have some of the noticeable mental changes. I'm still doing better, but maybe with less energy than a month or so after the fast. I hope not, but maybe this is something I'll have to repeat every now and then.
Negative persistent digestive state could be related to the new digestive context you created after a 2 week fast, which is a huge event in a life.
How much, how often, what and when you eat should be adapted, following your feedback rather than theories. Alimentation is specific to each.
Good luck dude, you can fully heal!
Just a thought from a random stranger but you might want to consider the possibility of celiac disease. It takes a couple months for antibodies to build up so the time scale you mentioned here stood out to me. Celiac disease has huge impacts on mental clarity and energy.
Have you done any experimentation with pre/pro-biotics? Just wondering if your fasting adjusted your gut biome, but now as you've resumed whatever diet that biome is being modified based on the fuels available to it.
Yeah might be worth regularly eating foods like kimchee or kefir. Or high quality yogurt. It seems to help with my psoriasis which is related to inflammation.
I'm pretty sure vegetable sources are way more potent than yoghurt, fermenting your own veggies is not that complicated. Nor is making your own yoghurt. The thing is, it's alive, so shelf time matters a lot.
I wonder if shorter fasts between the long ones would make a difference?
Ignore the hurtful comments. A good portion of people don't have much empathy for others, combined with the anonymous nature of this forum, makes people say hurtful things.
Thanks, I appreciate that. And by the way, I had the same thought as you. I’ve been doing monthly 5-day fasts just to see what would happen. So far not much to report, but I’ll wait until 6 months have passed to draw conclusions.
At the most extreme end of the other side, Buddhist monks, in addition to not eating meat (or, often, aliums like onions and garlic), also don't generally believe in dinner—they have to eat all their solid food for the day before noon, so you could view this as fasting for half the day every day.
(There are some caveats... At least for the Tibetan monks I knew, morning prayer is early at like 5am and comes with a sort of pita bread and tea, bedtime is closer to 9pm, and during these 9 hours there will be more tea. With a little googling I am able to confirm that some of the "pita" (pao balep) is consumed at the lunch tea, and I think this is after the lunch meal, so it might be 1pm. I think there's none at the evening tea that you'd have around sundown? Also in terms of calories the Tibetan tea is “bulletproof,” consisting of a very long steep for the leaves to extract maximum bitter flavors, that get mixed with a bunch of yak butter and salt. So liquid calories are very much a thing for them.)
Would also want to know this, since one probably needs a lot of muscle to go through this 14 day períod, but those muscles would come from exercising, which also help reducing depression.
You're welcome. I read a lot about fasting and I have a lot of experience with it. It's helped me, personally, heal many things (but not all of them -- it's good to have low expectations.) In any case, it's a mind-expanding experience to go without food for a while, regardless of the outcome.
Please Google Thomas Seyfried and William Makis. The latter is controversial—if he’s not completely fraudulent and making things up, he might be onto something. He’s sharing emails from numerous cancer patients who claim to have healed using these two particular molecules.
Exercise can supposedly help outcomes for some types of cancer—-I wonder whether the mechanism is similar to that of fasting. The supposed mechanism AIUI is that exercise makes less glucose available to the tumor. Podcast with more info here: https://overcast.fm/+6j6rLbfGM
I'm one of those stubborn types that will refuse all medical treatments, lots of fasting is my current go to strategy if some tumor gets out of control.
steve jobs and many others tried things like that. if it makes you feel good then sure, but that alone will kill you faster when a tumor(s) gets out of control.
Spoiler; every newspaper and media source is biased. They have to be. Starting with what stories they run and don't run and ending with the specific word choice they make inside the articles.
Critical thinking is the only thing you can use to spot bias. Compare and contrast different stories. Then make up your own mind.
Media literacy and virtual l critical reasoning should be the bedrock of modern education, and it is not.
You're supposed to use a combination of your life experience and best judgment, which is as it always was. Extreme trust and extreme distrust are both irresponsible.
Your responsibility is to remember that institutions are made up of flawed people just like yourself, and work towards improving them. You don't let someone manipulate your emotions to turn them into abstract enemies, tear them down, and replace them with nothing.
I guess I'm cool coming out and saying it. Out of curiosity I bought some helminths online and infected myself. I was inspired to do it after reading a book called An Epidemic of Absence.
I can't say a whole lot -- positive or negative -- came from it, but it was easy to do and inexpensive. So even though the article says you shouldn't try it at home, I did. I consider it safe. I'm not a doctor, I just read a lot of PubMed, so take that for what it's worth.
Is the beta amyloid theory wrong? I read a review paper on this and the science was unclear. I think the point was that all AD patients had a pretty heavy amyloid burden, but not the converse. Feel free to send me some reading.
https://pubmed.ncbi.nlm.nih.gov/39981248/
https://pmc.ncbi.nlm.nih.gov/articles/PMC11303312/