Yudkowsky texts represent my mental image of a vector continuously scanning a latent space in some general direction. Changes just pile on and on until you come from concept A to concept B without ever making a logical step, but there’s nothing to criticise cause every step was a seemingly random nuance. Start at some rare values in most dimensions, crank up the temperature and you get yourself Yudkowsky.

> our coherent extrapolated volition is "our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted (…) The appeal to an objective through contingent human nature (perhaps expressed, for mathematical purposes, in the form of a utility function or other decision-theoretic formalism), as providing the ultimate criterion of "Friendliness", is an answer to the meta-ethical problem of defining an objective morality; extrapolated volition is intended to be what humanity objectively would want, all things considered, but it can only be defined relative to the psychological and cognitive qualities of present-day, unextrapolated humanity.

I doubt that a guy who seriously produces this can say something meaningful at all.