So tired of defending against this same, old, completely wrong intuition from people especially those saying "do the science" to justify their ignorance instead of looking themselves since the science has already been done and it's coming up on a full century old.
From this one paper alone, humans can perceive information from a single frame at 2000 Hz.
Humans can read numbers and reproduce them immediately a 5 digit number is displayed for 1 frame at 400 fps. This is a single exposure, it is not a looping thing with persistence of vision or anything like that. 7 digit numbers required the framerate to be 333 fps. Another student produced 9 digit number from a single frame at 300 fps. These were the average results. The record results were a correct reproduction of a 7 digit number from a single viewing of a single frame at 2000 Hz. This was the limit within 2% accuracy of the tachistoscopic equipment in question. From the progression of the students chasing records, no slowing of their progression had ever been in sight. The later papers from this author involve considerable engineering difficulty to construct an even faster tachistocope and are limited by 1930s-1940s technology.
This research led the US Navy in WW2 to adopt tachistotopic training methods for aircraft recognition replacing the WEFT paradigm (which had approximately a 0% success rate) to a 1 frame at 75 fps paradigm which led to 95% of cadets reaching 80% accuracy on recognition, and 100% of cadets reaching 62.5% accuracy after just 50 sessions.
Yes, humans can see 2000 fps.
Yes, humans can see well beyond 2000 fps in later work from this researcher.
Yes, humans can detect flicker well above 1000 fps in daily life at the periphery of vision with cone cells as cone cells can fire from a single photon of light and our edge detection circuits operate at a far higher frequency than our luminance and flicker-fusion circuits. Here's flicker being discriminated from steady light at an average of 2 kHz for 40 degree saccades, and an upper limit above 5 kHz during 20 degree saccades, which would be much more typical for eyes on a computer monitor.
There is no known upper limit to the frequency of human vision that is detectable. As far as I know, all studies (such as this one I link) have always been able to measure up to the reliable detection limit of their equipment, never up to a human limit.
At this point , due to non-deterministic nature and hallucination context engineering is pretty much magic. But here are our findings.
1 - LLM Tends to pick up and understand contexts that comes at top 7-12 lines.Mostly first 1k token is best understood by llms ( tested on Claude and several opensource models ) so - most important contexts like parsing rules need to be placed there.
2 - Need to keep context short . Whatever context limit they claim is not true . They may have long context window of 1 mil tokens but only up to avg 10k token have good accuracy and recall capabilities , the rest is just bunk , just ignore them. Write the prompt and try compressing/summerizing it without losing key information manually or use of LLM.
3 - If you build agent-to-agent orchestration , don't build agents with long context and multiple tools, break them down to several agents with different set of tools and then put a planning agent which solely does handover.
4 - If all else fails , write agent handover logic in code - as it always should.
From building 5+ agent to agent orchestration project on different industries using autogen + Claude - that is the result.
Wealth of Nations (read past pg 50, unlike most current economists)
Das kapital, as a critique to Smith's writing.
Communist manifesto, to understand the point of the laborer, and not capital.
Read about worker cooperatives and democracy in the workplace, including Mondragon corp in Spain.
(One of the largest problems we have with any economic system is that none can properly model infinites. The cost of creating new is expensive be it art or science. But cost of copying is effectively 0. I can highlight the problem, but I have no good solution. But OpenAI's response is 'let us ignore copyright law' which wrongs creators.)
This is tech. This is how it has always been. From Archemedes to DaVinci to Edison to Ford, technologists are always captured to serve the interests of those in power. Most modern technologists don't want to believe this. They grew up building an Internet that had a bit of countercultural flair to it and undermined a few subsets of entrenched elites (mass media, taxi cartels, etc.), so they convinced themselves that they could control society under their wise hands. Except the same thing that always happened happened: the powers that be are now treating tech the way tech treats everyone else.
But who should pay? The model developers? Training models is a cost center. And what about open source AI, should we legislate it out of existence?
How about the AI providers? they operate on thin margins, and make just cents a million tokens. If one provider is too expensive, users quickly switch.
Maybe the users? Users derive the lion share of benefits from AI. But those benefits are hard to quantize.
Maybe a blanket tax? That would simplify things, but would put all creatives on a quantitative rather than qualitative criteria.
I think generative AI is the worst copyright infringement tool ever devised. It's slow, expensive and imprecise. On the other hand copying is fast, free and perfect. I think nobody can, for science, regurgitate a full book with AI, it won't have fidelity to the original.
The real enemy of any artist is the long tail of works, sometimes spanning decades, that they have to compete against. So it's other authors. That is why we are in an attention economy, and have seen the internet enshittified.
The most creative part of internet ignores copyright royalties. From open source, to wikipedia, open scientific publication and even social networks, if everyone demanded royalties none of them would be possible.
After a quick test this looks incredibly good and fast. I'll use it as a terminal for the next weeks to see how it goes, but I have good feelings. Thank you so much for writing it.
EDIT: WOOOW, for me this is going to be a game changer. I was just working at Redis stuff outputting a ton of debugging info and results, and normally the terminal was the bottleneck, and here instead it printed half million of results in the blink of an eye. And then I could go back in the history without any performance degradation. I love this: for development of systems it makes a big difference.
Might be hard to spot if you’re American in the US, but LLMs feel very American even outside of region and language. Concretely, I have to constantly ask for recipes to be changed to metric. Less concretely, the undertones and mannerisms of politeness, positivity and “excitedness” comes across as very American to me, probably even within the rest of the Anglosphere. How would I describe it? Maybe similar to how you’d feel about a mix of Ned Flanders, Ted Lasso and some Valley girl stereotype – im sure it’s a bit off putting also for many Americans.
I guess it’s training data but also heavily RLHF. I doubt that the trainers are aware of their own cultural biases and values, and they may not care. And why should they? In either case, from a thousand yard perspective, it’s probably an effective vector for spreading “American values”, if you will.
I recently ported this to Metal for Apple Silicon computers. If you're interested in learning GPU programming on an M series Mac, I think this is a very accessible option. Thanks to Sasha for making this!
As someone who has designed and taught those courses, my experience (admittedly only one persons) is that you pick what will work with the least hassle - because you'll have plenty of hassle elsewhere and probably no real time to deal with any of it without making more.
This is a frustrating article because it never explains why e is the natural logarithm base. To me, the easiest way to understand it is via continuous compound interest:
* If you invest $1 at 100% interest for 1 year, you get $2 at the end
* Compounded 2 times in a year, you get 100/2 = 50% interest every 1/2 year, which amounts to $2.25
* Compounded 4 times in a year, you get 100/4 = 25% interest every 1/4 year, which amounts to $2.44
* Compounded n times in a year, you get 100/n percent interest every 1/n year, which amounts to (1+1/n)^n dollars
* So continuous compound interest is the limit as n approaches infinity, which amounts to $2.71828 at the end of the year
(This is a great problem to give to pre-calc students to see if they can figure out the calculation for themselves.)
That may be how it was arrived at historically, but it is not the best way to explain it.
e arises when you ask the question: is there a function that is its own derivative? And it turns out the answer is yes. It is this infinite series:
1 + x + x^2/2 + x^3/6 + ... x^n/n! ...
which you can easily verify is its own derivative simply by differentiating it term-by-term. When you evaluate this function at x=1, the result is e. In general, when you evaluate this function at any real value of x, the result is e^x.
But the Euler equation e^iπ = -1 has nothing to do with exponentiating e, it's just a notational convention that is defined to be the series above. When you evaluate that series at x=iπ, the result is -1. In general, the value of the series for any x=iy is cos(y) + i*sin(y).
This problem is traditionally solved by attempting to teach the aliens how to decode the pattern inside the pattern itself. For example a rendering might look like
x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
x _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ x
x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
x _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ x
x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
x _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ x
x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
x _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ x
x _ _ x _ x _ x x x _ x _ x _ _ x _ x _ x x x _ x _ x _ _ x
x _ _ x _ x _ x _ _ _ x _ x _ _ x _ x _ x _ x _ x _ x _ _ x
x _ _ x x x _ x x _ _ x x x _ _ x x x _ x _ x _ x _ x _ _ x
x _ _ x _ x _ x _ _ _ _ x _ _ _ _ x _ _ x _ x _ x _ x _ _ x
x _ _ x _ x _ x x x _ _ x _ _ _ _ x _ _ x x x _ x x x _ _ x
x _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ x
x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
x _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ x
x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
x _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ x
x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
x _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ x
x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
The idea is that the aliens can figure out first that these opening and closing sections have periodicity 30, and because they are so repetitive they do not contain the actual data bits, but then there is a central section which does contain complexity.
So then they will hopefully alight upon the idea of looking at the code two-dimensionally, and adjust their browser width until the lines all line up, at which point they get the custom message.
I debated saying something like this too because I've had it backfire, but in a way I didn't expect.
My daughter was excited to go to a coding camp with me, until several people (not all at the same time) each stopped to praise her for being a girl, and me for bringing my daughter. Everyone truly had the best of intentions, but the unfortunate impact was it gave her a bunch of attention that she didn't want, and only served to raise her awareness of how few girls there were. It made her feel more like a misfit and an alien. Afer she realized that it wasn't "normal for a girl to do coding," she didn't want to go anymore. That was a hard and heartbreaking lesson for me. I don't know if it's even applicable outside of my circle, but figured I'd mention it in case it's helpful to others who are trying to create a warm/welcome/inclusive environment. Since then I basically just try to "act normal." It's a hard problem.
From the discussion of Brad Myers' classic 1990 paper (originally published by the ACM CHI conference in 1986, then updated in 1990 in the Journal of Visual Languages & Computing), "Taxonomies of Visual Programming and Program Visualization" (where Brad dropped by to answer questions):
>Brad Myers' paper answers the age-old argument about whether or not spreadsheets are visual programming languages!
>Google sheets (and other google docs) can be programmed in "serverless" JavaScript that runs in the cloud somewhere. It's hellishly slow making sheets API calls, though. Feels like some kind of remote procedure call. (Slower than driving Excel via OLE Automation even, and that's saying something!) Then it times out on a wall clock (not cpu time) limit, and breaks if you take too long.
>A CS grad student friend of mine was in a programming language class, and the instructor was lecturing about visual programming languages, and claimed that there weren't any widely used visual programming languages. (This was in the late 80's, but some people are still under the same impression.)
>He raised his hand and pointed out that spreadsheets qualified as visual programming languages, and were pretty darn common.
>They're quite visual and popular because of their 2D spatial nature, relative and absolute 2D addressing modes, declarative functions and constraints, visual presentation of live directly manipulatable data, fonts, text attributes, background and foreground colors, lines, patterns, etc. Some even support procedural scripting languages whose statements are written in columns of cells.
>Maybe "real programmers" would have accepted spreadsheets more readily had Lotus named their product "Lotus 012"? (But then normal people would have hated it!)
Excerpt from "Taxonomies of Visual Programming and Program Visualization", by Brad A Myers, 1990/3/1, Journal of Visual Languages & Computing, Volume 1, Issue 1, pages 97-123:
Spreadsheets, such as those in VisiCalc or Lotus 1-2-3, were designed to help nonprogrammers manage finances. Spreadsheets incorporate programming features and can be made to do general purpose calculations [71] and therefore qualify as a very-high level Visual Programming Language. Some of the reasons that spreadsheets are so popular are (from [43] and [1]):
1. the graphics on the screen use familiar, concrete, and visible representation which directly maps to the user's natural model of the data,
2. they are nonmodal and interpretive and therefore provide immediate feedback,
3. they supply aggregate and high-level operations,
4. they avoid the notion of variables (all data is visible),
5. the inner world of computation is suppressed,
6. each cell typically has a single value throughout the computation,
7. they are nondeclarative and typeless,
8. consistency is automatically maintained, and
9. the order of evaluation (flow of control) is entirely derived from the declared cell dependencies.
The first point differentiates spreadsheets from many other Visual Programming Languages including flowcharts which are graphical representations derived from textual (linear) languages. With spreadsheets, the original representation in graphical and there is no natural textual language.
Action Graphics [41] uses ideas from spreadsheets to try to make it easier to program graphical animations. The 'Forms' system [43] uses a more conventional spreadsheet format, but adds sub-sheets (to provide procedural abstraction) which can have an unbounded size (to handle arbitrary parameters).
A different style of system is SIL-ICON [49], which allows the user to construct 'iconic sentences' consisting of graphics arranged in a meaningful two-dimensional fashion, as shown in Figure 5. The SIL-ICON interpreter then parses the picture to determine what it means. The interpreter itself is generated from a description of the legal pictures, in the same way that conventional compilers can be generated from BNF descriptions of the grammar.
10. Conclusions
Visual Programming and Program Visualization are interesting areas that show promise for improving the programming process, especially for non-programmers, but more work needs to be done. The success of spreadsheets demonstrates that if we find the appropriate paradigms, graphical techniques can revolutionize the way people interact with computers.
>By the way, something I always meant to ask you, Brad:
How does "C32" fit into your acronym theme of gemstones and rocks? Is it a teeny tiny 4x4x2 carbon atom block of diamond? How many carats would that be?
>Brad Myers wrote several articles in that book about his work on PERIDOT and GARNET, and he also developed C32:
>C32: CMU's Clever and Compelling Contribution to Computer Science in CommonLisp which is Customizable and Characterized by a Complete Coverage of Code and Contains a Cornucopia of Creative Constructs, because it Can Create Complex, Correct Constraints that are Constructed Clearly and Concretely, and Communicated using Columns of Cells, that are Constantly Calculated so they Change Continuously, and Cancel Confusion
>Spreadsheet-like system that allows constraints on objects to be specified by demonstration. Intelligent cut and paste. Implemented using Garnet.
If you really want to start from the beginning, get “Build Your Own Metal Working Shop From Scrap” by David Gingery.
Quoting the description from Amazon (link below)…
“[It] is a progressive series of seven projects. Beginning with a simple charcoal fired foundry, you produce the castings for building the machine tools to equip your shop. Initially the castings are finished by simple hand methods, but it is not long before the developing machines are doing much of the work to produce their own parts.”
That professional responsibility falls entirely on the developer[s]. Not on an inanimate object.
To make a non-tech analogy. This is functionally equivalent to locking up books in safes, so people can't abuse the knowledge they contain. Sure: if an incompetent person used a structural engineering handbook to build a bridge, and did it incorrectly and it collapsed, that would be a bad thing. But most people agree that's not the fault of the book, or negligence on the part of the library for failing to restrict access to the book.
You are proposing an interesting thought experiment or hypothetical (“to what extent was the design team expecting a 10x longer lifetime?”). That’s worth pondering.
Furthermore, you are answering it with a “they must have known!”
I just don’t see how you can know this. (For context, I work at JPL and interact regularly with some of the Perseverance team, but nobody from the helicopter team, which was very small.)
Some context. The helicopter was originally quite controversial because it had no clear science rationale (mission science requirements did not imply a need for a helicopter). My understanding is that it was put on as a tech demo and (paraphrasing) cool thing. The lab director at the time was a proponent.
Of course it was a huge success.
Because of its precarious status, it was not designed to fly a lot. Planning flights takes a lot of resources, coordination with the rover takes resources, etc. Bandwidth, time, and people. My point is that the “plan” was kept small partly because these ancillary costs had to be kept down. This would hold even if the hardware could last forever. The tech demo must not get in the way of the prime science mission, which had already been planned.
The use of COTS components is another factor. The effects of radiation on microprocessors (in the Mars-surface or deep space environment) are not really well known, as a practical matter. There is just not much engineering experience with how bad it can get. So the longevity of the Snapdragon was (AFAIK) highly uncertain, not to mention whatever other COTS components were on it.
Expression of regret. "I am truly sorry." - Check.
Explanation of what went wrong. "I must have directed angry words at you, who clearly had nothing to do with how the day had gone." - This is weaselly. "I must have...", not "I did".
Acknowledgment of responsibility. " I clearly forget that I am the oldest and most experienced—and I should act this way in a post-mortem conversation." - Weaselly. Whether he was oldest and most experienced or not, no one should behave this way. And rather than taking the responsibility, it's chalked up to forgetfulness.
Declaration of repentance. - Nothing here. No commitment to change. No stated intention for better emotional management.
Offer of repair. - "Finally, as you surely recall, I encouraged you throughout 2019, in person and in email, to join the Racket leadership;" In the absence of any kind of effort or commitment to change, this would not be received as a positive offer. But rather an extension of exposure to an abusive individual. Not much of a repair.
Here's the thing. I've always been kind of cold on OpeanAI claiming to be "Open" when it was clearly a for profit thing and I was concerned about the increasing move to the commercialization of AI that Sam was taking.
But I am much more concerned to be honest those who feel they need to control the development of AI to ensure it is "aligns with their principles", after all principles can change, and to quote Lewis "Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies. The robber baron's cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience."
What we really need is another Stallman, his idea was first and foremost always freedom, allowing each individual agency to decide their own fate. Every other avenue will always result in men in suits in far away rooms dictating to the rest of the world what their vision of society should be.
Someone probably already suggested this, but I haven’t seen it yet, so I’ll throw a wild speculation into the mix:
I saw a comment (that I can’t find now) wondering if Sam might have been fired for copyright reasons. Pretty much all the big corpuses that are used in LLM training contain copyrighted material, but that’s not a surprise and I really don’t think they’d kick him out over that. But what if he had a team of people deliberately adding a ton of copyrighted material - books, movies, etc - to the training data for ChatGPT? It feels like it might fit the shape of the situation.
The topic of libraries comes up frequently when Lisp is mentioned.
If a certain task or problem arises that isn't directly related to or
considered one of the main problems that you're trying to solve with
your program, you would consider the use of a library to accomplish
that task. If there is such a library available, go ahead and use
it. If there isn't, Lisp programmers can spend a short amount of time
implementing only what they need without generalizing the task.
This isn't quite re-inventing the wheel since you have a unique set of
constraints, and it's certainly not as onerous a task as writing a
generalized library. There are those (Edi Weitz for example) who do
tend to implement complete libraries, and the Lisp community is of
course grateful for that.
Libraries and frameworks can be useful, but they also have the effect
of levelling the playing field. Lisp empowers the programmer to
`discover' a natural solution to their problem in a unique way.
> all so that the same binary runs on multiple operating systems, which isn’t actually very useful.
I like to mention my use case when this comes up: my log file viewer (https://lnav.org) uploads an agent to remote hosts in order to tail log files on that host[1]. While lnav itself is not built using cosmo, the agent is. So, it works on multiple OSs without having to compile and include multiple versions of the agent.
But I had a son 14 months ago.
There was absolutely no way I was going to miss any of a critical part in my baby’s life in order to be in an office at 2am managing a bad deployment.
Maybe I gave up my chance at PPU or RSU riches. But I know I chose a different kind of wealth that can never be replaced.