Immediate mode UIs come up every now and then, especially in games, and for relatively simple things they're a neat idea (imagine not retaining any extra view state!) But - and there's a big "but" coming up - it does not scale to complex and stateful UIs.
The problem is that, for games that have complex needs - think simulation and strategy games, not racing or arcade - complex UIs are needed to express complex relationships between underlying data and the relevant UX flows. But how do you do that in IMGUI? This information about UI state, layout, etc. is not in the UI anymore, but it has to live somewhere - so you end up having to build your own layout and UI management stuff, and basically duplicate what a RMGUI library would offer, but now with less generality and potentially more bugs.
For the probably best known example, Unity used to have an IMGUI [1] but they've actively moved away from it. It's still there in vestigial form, the API is not removed, but game developers don't really use it.
I mean, React is essentially a hack to get an immediate mode API on top of a retain mode platform (DOM), because the immediate mode API scales better to handle complex state.
I'd push that yes, you do have to maintain your state externally in an immediate mode UI, but when you have a lot of "state" that's really just a function of other state, that's exactly what you want.
The interesting thing about React is actually that it encapsulates a lot of state retention inside an immediate-style API. state/setState (and now state hooks) are the unsung heroes of the ecosystem, because they let you focus on only the business-relevant global state, making it natural for a dropdown menu to handle its own open/closed state and insulate its user from that complexity. And many intermediate mode libraries gloss over this need, so you have to track much more state than you’d like.
It’s a fascinating hybrid - rather than “BYO State” where the library does nothing but layout and draw, or “BYO hierarchy” where you need to retain instantiated components, React is “hierarchy tracking included” - all the benefits of persistent components, but tracked by the renderer rather than the caller. It’s an evolutionary step that I think desktop programming would do well to adopt and continue to iterate on.
> For the probably best known example, Unity used to have an IMGUI [1] but they've actively moved away from it. It's still there in vestigial form, the API is not removed, but game developers don't really use it.
It's true that the newer retained mode GUI has supplanted the older IMGUI for "game UI" (e.g. what users interact with when playing an actual game built with Unity). On the other hand, the Unity editor itself is still built entirely with their immediate-mode GUI library, and I haven't heard of any plan to move away from that. Building "editor-like" tools (including custom user tools) is still significantly easier to accomplish using immediate-mode libraries.
In Unity you're still forced to use the IMGUI for making editor tools. In fact their new xml layout prototype still uses the IMGUI to draw an instantiated layout! What a mess...
Honestly -- given the amount of time I've spent adapting my own data model classes and structures to the straightjacket of various GUI or web toolkits -- not having the toolkit enforce its normative lifestyle assumptions on data structure feels like a feature to me.
I'd rather manage the state and relationships myself.
>given the amount of time I've spent adapting my own data model classes and structures to the straightjacket of various GUI
This is why every tells you to use MVC or some variant that separates out the UI from the data. IMGUI lets you bend the UI to the data instead but the real lesson is to decouple the view and the model.
I'm very familiar with MVC -- I've been in the industry for 20+ years -- but so far disgusted with the way pretty much every GUI framework expects you to handle this. In reality I may already have a data model, but often have to wrap _another_ model around my model, just to satisfy the GUI framework's constraints on what it thinks a model needs to inherit from, etc. Recently ran into this with Qt. Very frustrating.
The trick is to just accept that the UI model, the business logic model and the denormalized db model all have different goals and often times should be decoupled and adapted.
Which is why it seems almost every project with a complicated UI turns into a bloated pile of semi-duplicated boiler plate 'model' classes with little business logic. Clearly, MVC as the industry understands it, has a problem.
Which is why the original Smalltalk people -- who invented MVC -- mostly switched to Self's Morphic framework in their later projects (Squeak, etc.)
> But how do you do that in IMGUI? This information about UI state, layout, etc. is not in the UI anymore, but it has to live somewhere - so you end up having to build your own layout and UI management stuff
Immediate mode is about the interface to rendering the UI, not necessarily about the implementation (although some people conflate these). So your UI elements can be stateful and retain their own state, but they can be released once they are no longer rendered in the current frame.
Well with retained mode one still has to handle a bunch of application state, and handle keeping that state synchronised with the view. Immediate mode typically forces you to manage the view state too but it’s typically easier to keep it synchronised with the application state that way.
I think the following is an interesting example here: your application state is some type, Config which would correspond to some boring components, e.g. text boxes and radio buttons, but the interface you want to present looks like:
[X] default settings
Or
[ ] default settings
...
Now the type of data you want by the end of the form looks like (eg Haskell)
Maybe Config
Where Nothing means “do the default thing”.
But the state of the view has to look more like:
Bool * Config
Because you want to remember what options the user had changed in case they change their mind a few times about using the default state. And now maybe you want a state like
(mutable) Bool * Config * (mutable) Config
Because the default state is not static and you want to make fields bold if they are not default. And so now you still have to do the complicated conversion from view to application state (but at least you don’t have to go back).
I suppose this is like the very early web dev style of having no direct state and directly reading values from controls, but at least in this world one has control over how the view state is managed and represented so it can be easier to avoid getting into invalid states.
Immediate mode GUI's don't let you apply useful object oriented programming techniques like subclassing and prefabs (persistence) to implement and configure custom components.
Properly object oriented retained mode GUIs let you subclass components to implement custom views and event handling, and develop and publish reusable prefabs and libraries of pre-configured, stylized, specialized component.
> don't let you apply useful object oriented programming techniques [to get] reusable libraries of specialized component.
But immediate mode GUI lets you compose very well. It's a bottoms up technique of programming, where instead of following a framework established by the library author, you are given low level building blocks. You will find it hard to mix and match two retained mode GUI libraries unless they all follow a common set of standard apis (e.g., java swing). But with immediate mode GUI, it's actually easy to mix and match (as the immedate mode gui has no need to know what framework you're working under nor need to make any assumptions about threading or api).
I don't know what you mean about easily composing immediate mode API's, or agree that different immediate mode APIs don't need to know about each other. They depend on a lot of behind-the-scenes implicit hidden state (practically a hidden shadow-DOM-like retained mode), so I suspect in many cases they'd walk all over each other (especially text editors dealing with input focus and keyboard navigation). Unity3D has two immediate mode APIs, one for the editor and one for runtime, and they sure don't mix.
I've implemented various adaptor wrappers for composing different retained mode API's myself, including wrapping The NeWS Toolkit OPEN LOOK widgets in HyperLook widgets, which I used for the first Unix port of SimCity to HyperLook/NeWS. (The OPEN LOOK pin-up menus and pie menus and buttons and sliders are implemented in TNT, but I wrapped them for HyperLook (which has its own persistence system and class hierarchy), so you can dynamically create and edit them at runtime with property sheets and script editors, ala HyperCard.)
For X11/NeWS at Sun, we even made an ICCCM X11 window manager that wrapped X11 windows in NeWS tabbed window frames with pie menus! (The X11 window manager was written in object oriented "retained mode" PostScript code, subclassing standard and custom NeWS window and menu widgets, like tabbed window frames and pie menus. While PostScript has an immediate mode drawing API, it was driven by extensible "retained mode" widget instances and classes represented by PostScript dictionaries, pushed on the dict stack.)
More recently (and more successfully), I've wrapped HTML components and floated them over OpenLaszlo/Flash (not so recently) and Unity3D (quite recently) retained mode user interface layouts in the web browser.
For example, I'm embedding an ACE code editor in a Unity3D retained mode user interface running in WebGL, that falls back to a regular TextMeshPro input field on other builds. There's no way you could do that practically with an immediate mode API.
WebGLAce.jslib (JavaScript library Unity extension, compiled by Unity to WebAssembly, converts between C#/WebAssembly and JavaScript data types and thunks messages back and forth between the browser and the Unity3D app): https://github.com/SimHacker/UnityJS/blob/master/Libraries/W...
Notice how WebGLAce_TMP_InputField simply subclasses the standard TextMeshPro TMP_InputField, and adds some more hidden state like the editorID and editorConfigScript of the corresponding ACE code editor running in the web browser.
All that is hidden from the code using it, so it doesn't need to come up with another place to store that editorID and editorConfigScript, or even know if it's getting an Ace editor under WebGL or falling back to TextMeshPro on other builds. That's what I mean by leveraging OOP to subclass existing components and hide internal state, which you can't do in immediate mode.
(My plan for non-WebGL builds that aren't already running inside a web browser, is to create an embedded web browser just to run the Ace code editor, and the C# WebGLAce_TMP_InputField API will remain the same, and it will just work transparently on different platforms. I want to support live coding Unity apps in JavaScript, but there just isn't anything for Unity that approaches ACE for editing code, so it's worth going through all the trouble to make an adaptor.)
How would you write a functional extension for Unity3D's old immediate mode GUI that let you embed an ACE code editor in a Unity WebGL app? What would the API and state management even look like? How could you make it platform independent?
And even if you solved all of those problems with an immediate mode API, by its very nature it still wouldn't enable you to build GUIs in the interface editor or store them in prefabs, and you'd still have to recompile you app (which can take half an hour with Unity) every time you wanted to tweak the user interface (which is why I like programming Unity apps in JavaScript as much as possible).
Of course you can take it too far, trying to please everyone by combining the worst of all possible systems and then giving it a ridiculous sounding name (see "MoOLIT"), but we're discussing whether it's possible and practical, not whether it's desirable. And even if it's not desirable, it ends up being necessary.
Embedding object oriented "retained mode" widgets with different APIs inside of each other is old hat and common for backwards compatibility. Whenever you write a new GUI toolkit, embedding widgets from the last toolkit is one of the first things you do (including recursively embedding widgets from the toolkit-before-last).
Concrete example: Microsoft lets you embed old fashioned OLE controls in Windows Forms / Presentation Foundation applications, which might be implemented in MFC themselves. And MFC is all about embedding old Win32 widgets implemented in C, and newer OLE components implemented in Visual Basic or whatever, in venerable C++ MFC user interfaces. Say what you want about how much Win32/MFC/OLE/WF/WPF sucks, and I'll wholeheartedly agree, but if you're running Windows, you probably have widgets on your screen using several different retained mode APIs embedded more than two levels deep right now.
You can always encapsulate your custom IMGUI component in a function (or even a class, if you really want!) You should look at how custom components are made in IMGUI (for example, plotting variables: https://github.com/ocornut/imgui/wiki/plot_var_example)
I've used Unity3D's immediate mode a lot, both the editor API and the runtime API (and I've also implemented and used a lot of different retained mode GUIs), and I'm much happier with the newer "retained mode" GUI in Unity3D.
The problem with immediate mode that is you have to come up with somewhere to store and retrieve any values or state required on a case-by-case basis (including the edited value itself, and other view state like scrollbar state for text editors, etc), and that tightly couples your component function with whatever's using it, so they're not very reusable or simple. With OOP, the object has its own place to store that stuff, which is cleanly decoupled from whatever's using the component.
Then there's the issue of event handlers. Some user interface components just aren't so simple that they only have one true/false return value like buttons. Text editors can notify on value change, end edit, select, deselect, etc. And Unity's retained mode GUI supports persistent event handlers that let designers hook up events with methods of objects with parameters, without writing or recompiling any code.
And there's also a lot more to layout that you can easily express with an immediate mode GUI. Sometimes you have to measure a bunch of things and align them in various ways, like grids or flex layouts, and adapt to the screen size and other constraints (i.e. responsive design), and that's really hard to do with immediate mode, while retain mode has a rich set of layout components you can use and extend. And there's nothing like XCode's layout constraints for immediate mode.
A perfect example is TextMesh Pro's text editor. It has so many different (and useful) features and parameters and callbacks and ways to configure it, and it keeps so much state in order to redisplay efficiently with the fewest number of draw calls and reformatting, that it would be extremely impractical and inefficient and unwieldy for it to use an immediate mode API. Especially with C# positional arguments instead of Pythonic or Lisp-like optional unordered keyword arguments.
When you encapsulate your immediate mode layout in a class, or write an engine that interprets (i.e. JSON) data to drive the immediate mode API, or use reflection in your wrapper functions to dispatch event callbacks and bind your widget data to your model data, you're just rolling your own informally-specified, bug-ridden, slow implementation of half of retained mode. (See Greenspun's tenth rule.)
Here's a demo of Unity3D's new retained mode UIElements user interface for the editor, that lets you wrap old immediate mode IMGUI code in IMGUIContainer wrappers!
Typically we build GUIs using event driven callbacks. This causes problems in managing life times, dynamic layouts, threading issues etc. The idea behind immediate mode GUI is that you get a frame and in each frame you create/update layout, check for user actions - just like you do in render frame in game loop. So there are no call backs and you don't have to worry about issues arising from event-driven code.
Does this pretty-much rule out (without a huge amount of effort) nonmodal GUIs? It seems without callbacks, you would need to roll your own window management, Z-ordering of overlapping UI elements, state transition validation (e.g. clicking on the parent dialogue’s Cancel button while a child window is still open), ...
There's no reason that that can't all be done as part of the underlying IMGUI implementation; what you're working with above that is essentially a DSL that renders out to a workable set of controls. Layout management and the actual rendered graphics / animations are handled by the code you don't have to see.
If anything, this is actually a lot like Web 1.0 programming: server side rendering, often using templates that end up looking a lot like a DSL. Each page view is rendered based on those templates against data fetched in advance of the output.
You can get pretty deep with that kind of thing, though of course the page refresh time was what killed this otherwise fairly nice style of writing an application. Users expected actions like sorting a column to be as fast in the browser as it was in a Windows 95-era file explorer or Excel, and it simply wasn't happening.
(Imagine if first-class client-side-interactive UI components like a sortable table, tabbed panel, etc. like what XUL provides had been in HTML in the 90s. We wouldn't need half the Javascript bullshit people have to use now to replicate that functionality half-assedly...)
Note to authors: please define your terms when you first use them, or provide links to the definitions.
Of course I know what a GUI is. I've been building them for decades, and building tools that other developers have used to build GUIs. (I was one of the authors of the "visual" part of Visual Basic.)
But I have never before seen the terms "immediate mode GUI" or "retained mode GUI". And I'm pretty sure there are many developers like me who will have no idea what you're talking about. It takes only a moment to add a quick definition or a link the first time you mention a specialized term, and your readers will definitely appreciate it. Thanks!
(Edited to remove fluff and make the essential point more clear.)
Do you know who his intended audience is? Just because his article got posted here doesn't mean he was writing it for the general HN audience.
Most likely he's writing it for an audience that knows very well the difference between immediate/retained modes. I mean, he's writing it on a blog that's all about graphics programming, game engines, etc. So it would be a bad move for him to define terms common to that field in every article.
I remember an article on the woodworking technique of riving that made it to HN front page, the comments took the author to task for implying riving was invented by Vikings because the article was written for Viking enthusiasts on a Viking enthusiast website.
I don't know, he actually spends half the article explaining the difference between the two. It's just not clear he's going to do that until you get that far. So I think it's strange to suggest that his audience already knows the difference.
>But I have never before seen the terms "immediate mode GUI" or "retained mode GUI". And I'm pretty sure there are many developers like me who will have no idea what you're talking about.
Well, developers who write their own GUI toolkits (what the article is about), probably know retained vs immediate mode. Those are two very basic distinctions when it comes to GUIs.
Retained mode is the average GUI toolkit you probably know (Swing, GTK, Cocoa, etc). Immediate mode is where the GUI is draw directly frame by frame by the application.
In retained mode you deal with buttons, windows, panels, checkbox objects, textboxes, etc, arranged in a tree-like structure.
In immediate mode you deal with painting commands (of course you usually will abstract them to a higher level too). You want a button? You draw a square and the label in it, and depending on the user input (e.g. clicking at the moment or not), you might draw it depressed.
> Retained mode is the average GUI toolkit you probably know (Swing, GTK, Cocoa, etc)
Not quite. Certainly Cocoa, and probably the others as well, are mixed mode.
An NSView has a drawRect:: method that draws that particular view in "immediate mode", with drawing commands of the underlying procedural graphics API (essentially Quartz/CoreGraphics).
If you want pure immediate mode, you create a single NSView and just draw everything.
If you want pure retained mode, you exclusively use predefined widgets, which hide the "immediate mode" drawing that they do.
>An NSView has a drawRect:: method that draws that particular view in "immediate mode", with drawing commands of the underlying procedural graphics API (essentially Quartz/CoreGraphics).
Yes, but that's the case for almost all retained mode GUI toolkits (they provide something akin to that), and it's used as a last resort (when you need a custom look or to implement your own widget, or complicated UI for something like plotting or a piano keyboard, etc).
Regular use of the APIs in Cocoa (and what all books advice) is retained mode style, using pre-made widgets with the Interface Builder and so on. Not "drawRect".
Usage appears to have shifted that way. However, that wasn't the case not so long ago and the fact that a facility is no longer being used (as much) does not mean it's not there. If you need it, it's right there to use as you wish.
Just because you tend not to open the roof doesn't mean your convertible is no longer a convertible, as long as you still can open the roof.
And yes, I did mention that other toolkits probably work the same way.
>Usage appears to have shifted that way. However, that wasn't the case not so long ago and the fact that a facility is no longer being used (as much) does not mean it's not there. If you need it, it's right there to use as you wish.
I recall having discussed this with you here again.
I'd say it was the case for as long as I'm on IT (so, as long as mid-nineties or so). Heck, even back to Delphi and co.
>Just because you tend not to open the roof doesn't mean your convertible is no longer a convertible, as long as you still can open the roof.
Well, if 99% of the car has been designed for one use (A), but has the option (that few use) to do B, doesn't mean the car's a B.
So, this is more like "just because the car has a lot of storage space in the trunk, doesn't mean it's a truck".
> case for as long as I'm on IT (so, as long as mid-nineties or so)
Late 80s for me. What you saw depends very much on which part of "IT" you were in.
When doing forms automation, maybe you only ever saw the widgets. And that's OK. However, forms/widgets are not all of IT and all of UI work. So when doing graphics, DTP, CAD/CAM, document processing etc.: not so much.
>if 99% of the car has been designed for one use
But it hasn't. Not 99% and not one use. These frameworks are flexible and were explicitly designed to be flexible. drawRect:: and handling mouse events is a fundamental feature of the view, and widely used. The fact that you have convenience widget libraries that encapsulate its use (they ALL use it), doesn't mean it's not there.
Just because you use your truck like a car doesn't mean it's not still a truck.
>So when doing graphics, DTP, CAD/CAM, document processing etc.: not so much.
Yeah, haven't done graphics, DTP, CAD/CAM, etc, and I guess immediate mode or mixed mode is more prevalent there.
But I'd say the vast majority of GUIs (whether enterprise or desktop/business apps etc) are forms and regular widgets based. If they need some custom part (e.g. a plot or a custom drawn control) they implement it by drawing it, but still use it in retained mode, in the classic widget tree, embedded in some window, layout widget, etc.
Immediate mode is almost non-existent in enterprise GUI work for example -- except niches like CAD apps for factories or some special case (a custom widget drawn but embedded in an otherwise retained mode UI).
>These frameworks are flexible and were explicitly designed to be flexible.
Yes, they have escape hatches to draw your own widgets when needed.
But their main idea wasn't "immediate mode", but "we provide these ready-made widgets", and they're judged by how many and good those are, not by the ability to drop in drawRect or such.
In the same way, that OpenGL and DirectX is 99% immediate mode, regular GUI toolkits is 99% retained mode (on the API level programmers are meant to use, and mainly use).
Almost any book on GUI programming on Cocoa, GTK, QT, etc I've read, primarily focuses on the available widgets and retained mode use, and any "drawRect" equivalent is an aside about what you can achieve if needed.
No, as I've written, their main idea was to provide mixed mode. On top of which you can provide a widget set. Something like GKS or Phigs was true retained mode graphics, and toolkits like AppKit were reactions to the limitations of those types of mechanisms.
There is a reason the Mac had QuickDraw and NeXT had DPS and not GKS or Phigs. Well, many reasons, but one was that the limitations of pure retained mode graphics systems were well known.
> they have escape hatches to draw your own widgets when needed.
Yes and no. While you can view them as escape hatches if you're primarily doing widget-only programming, they aren't really escape hatches as the entire system is based on / built on them. Just like in Objective-C, objc_msgSend() is not the escape hatch, it is both the basis for all messaging and it enables the escape hatch, which is -forwardInvocation:. Similarly -drawRect:: and responder handling: they are the basis on which everything else is built. Since they are exposed, that means you can build your own views/widgets as peers to the ones provided.
> Almost any book on GUI programming on Cocoa ...primarily focuses on the available widgets
The books I learned AppKit with focused on creating a custom view, drawing with DPS and pswrap, handling mouse events and the responder chain.
What's more - things like drawRect, QPainter etc. are hardly comparable to what a real immediate mode GUI toolkit offers - you'd need to basically reimplement something like Dear ImGUI on top of Cocoa or Qt in order to get something similar.
I didn’t know also, but typing immediate mode.. basically autofills and shows the resulting definition on an iPhone without evening pressing enter. At this point I’m not sure which is more tiring, the lack of an intro/definition, or the expected top comment about the omission. I think, as professionals we can be less hard on each other, especially since we are all too aware of the difficulty in taking time out to write a blog/article.
That's a fair comment, and you are definitely right about the difficulty of finding time to write a blog post!
Please understand that I was not intending to be hard on the author or criticize them. I am offering some writing advice that I hope could be useful for anyone writing technical articles like this. Of course that may just be arrogance on my part. ;-)
What a rude comment. It's not a specialized term. It's a common term. Immediate mode GUI as a concept is well known. Tens of thousands of programmers use Immediate Mode GUIs. There are several well known Immediate Mode GUI libraries used by tens of thousands of projects.
When I don't know a term I google it. I don't assume the entire world revolves around me and that it's the author job to make their article more verbose because everyone reading it might not know every term.
Here's are galleries of some from one of the more popular libraries
I don't think it's rude at all. I did Google "immediate mode GUI" after reading the first paragraph, because I also didn't know what it meant. The first three results were a Wikipedia snippet which didn't help my understanding at all, a Gist that begins with "Before you continue, if you don't know what IMGUI is don't bother reading this", and then third result was this HN comments section.
I would argue that "immediate mode GUI" is an extremely specialised term.
Specialised enough that I didn't come across it during my years of work on Qt. Work that included reading all the manuals, style guides and UI books we could get hold of.
> Specialised enough that I didn't come across it during my years of work on Qt.
Of course you didn't, Qt is retained.
It's a bit like imperative programming vs functional programming. Before functional programming gained any significant traction, imperative programming was just "programming".
Actually immediate mode came first, retained came later for perf reasons (redrawing the entire screen from scratch every frame is slow, even though immediate mode is easier). The original context was however graphics (OpenGL retained vs immediate modes) rather than UI (widget toolkits started out as retained, like OGL retained scene graphs, likewise canvases have always been immediate).
Of course I didn't? I filled a wall with the books I bought and didn't come across the term. What did I do wrong, what books could I have bought but didn't?
How many of those books weren't about the mainstream GUI toolkits (Qt, GTK, MFC, Cocoa, Swing…)? Has even one of those books described something that resemble "let's redraw everything every frame"? If so, how that technique was called?
> what books could I have bought but didn't?
Possibly none at all. Immediate mode anything is mostly a game dev thing, and what happens in game dev tends to stay in game dev. They don't seem to document their practices as much as other fields do.
Hence my suspicion that the authors of your books didn't even know about IMGUI. They just knew about "the" way to do GUI, which happens to involved retained state.
You're letting the tail wag the dog. You're assuming that people know terms that are used in "possibly none [no books] at all", or put differently, that the set of terms used in books for software developers is uncorrelated with the set of terms software developers may be assumed to know.
BTW, the code in Qt that draws rotated/sheared text/pixmaps was influenced in part by a book or chapter by Michael Abrash, as well as by articles in either Graphics Gems or conference papers (memory's fallible I'm afraid).
I did not find it rude and I agree that a short explanation would have been nice. I had heard the terms Immediate Mode GUI and Retained Mode GUI before but I couldn't remember the details. A short reminder would have been nice. Just a sentence.
p.s. I did get a chuckle from your deleted comment that suggested I was living in a cave. After the heat wave we've had here the last couple of days, I wish I were living in a nice cool cave! ;-)
I knew the term and I didn't find the comment particularly rude, even if I don't agree with it fully.
I agree that the docs could have been improved by providing some sort of definition. But I think something as simple as a link to wikipedia could have been enough.
I also agree that the intended target audience will most likely already be aware of immediate vs retained. Adding the aforementioned link would not have hurt said audience anyway.
He wasn't rude, he politely asked people writing articles to try to be more descriptive. However, in my decades in the industry, I've never heard of "Immediate mode GUI" or "retained mode". Not once in 30+ years. I can't be that common.
Agreed. Better to bore people with brief review than alienate a wider audience. I've had people say to me "I know that" angrily when I'm mid-explanation of a concept, but it's better to be thorough. This habit comes in handy in forums or in a room with more than two people. In both scenarios, lurkers/eavesdroppers then get background info and don't walk away misinformed.
It's where it first came to prominence. The earliest D3D api was immediate mode (meaning, bare metal, talking directly via an api to the hardware). Retained mode is about retaining statefulness and some better abstractions to the underlying needs of the hardware (eg automatic texture resampling, etc).
Considering it's an NVidia guy talking: he's been around for a while.
A minor detail, but the very first version of Direct3D was a retained mode API.
In 1995 Microsoft acquired a company called RenderMorphics and turned their Reality Lab [1] API into "Direct3D". The first two versions to ship from Microsoft had both retained and immediate mode APIs, but they dropped the RenderMorphics-based API due to lack of adoption.
Interesting, thanks! That would explain why I and probably many other readers, who have built plenty of GUIs but have not worked specifically with APIs like DirectX, would not be familiar with the term.
The terms are in common use since the 80s at least (no, Microsoft did not invent these terms with D3D). You can prove this in google scholar.
In fact for 3D rendering, retained mode was touted as a the new way forward, there was even a standard predating OpenGL called PHIGS, a retained mode system.. OpenGL was the worse is better (also PHIGS had many limitations basically limiting its use to CAD) backlash promoting its immediate mode in the early 90s.
Retained Mode Direct3D is the first and only place I've really heard that term used. After D3D stopped offering Retained Mode I haven't heard anyone use it except for a couple of articles about "immediate mode GUIs". Last time I saw one of those I had to look it up.
Yes, of course that is true. It doesn't change the fact that it is considered a common courtesy, and a sign of quality writing, to define any unusual terms like this when you first mention them.
It's the same principle as writing readable, self explanatory code. You write your article once, but you hope that many thousands of of people, with all different levels of experience, will read it. If you can take a few minutes to save thousands of readers a minute or two each, that is a big win.
If you really want to do it right, you can link to those Wikipedia articles when you introduce the terms. After all, your reader may be on a mobile device on the bus, where it is easier to tap a link than to copy and paste or retype the term to find out what it means.
I agree that it's important for people to be independent about locating and ingesting information. I also think that it's good to include a quick definition, particularly for stuff that's really specific (indeed - while looking for a definition I found a blog post that starts with "If you don't already know what Immediate Mode GUIs are then please stop now").
I did a Google search but there weren't any clear definitions on the first page or so.
I did search for the Wikipedia article and here's what I got:
"Immediate Mode GUI is a GUI where the event processing is directly controlled by the user. The key is the aspect of retained (RMGUI) vs non-retained UI (IMGUI). In the latter, the user code is holding on its own natural data, in the former it is stored in the GUI system, often hidden from the user code. The immediate mode GUI widget toolkits are more direct and thus less complex, thus they are a good choice for those who want a simple but easily changeable and extendable GUI toolkit, they are usually generic, open source and cross-platform. The original Immediate Mode GUI toolkit was imgui[1] which is based on OpenGL. The idea was popularized by Casey Muratori. There are others such as nuklear[2] with backends for several graphics libraries, kiss_sdl[3] for SDL (Simple DirectMedia Layer) or dear imgui[4] written in C++ (here the jvm port[5])."
First, the quality of this writing isn't great, to the point where it makes it harder to read.
I think that the above definition makes sense if you already know what Immediate Mode is, but it was pretty opaque on my first read (partly because I misunderstood "user" in the first sentence to mean "the customer sitting in front of the computer and using your IMGUI program", instead of "other code that uses the IMGUI").
The definition now makes sense to me after reading other posts here explaining that this was a Direct3D thing.
So yeah - I can see the arguments that we're not the target audience (and I understand that the person who posted this to HN probably isn't the author), but I think that Stratoscope's point that articles posted to the general HN audience would benefit from short, clear definitions (or links to definitions) in the articles is spot-on.
Also, props to Stratoscope for their calm, constructive replies in the face of a lot of disagreement! If we all did this (and I include myself here for sure) the Internet would be a better place.
immediate mode and retained mode are technical jargons sure, but this article is written with a technical audience in mind. It's not like the author has invented the term, nor are the terms used here esoteric at all. Defining it just adds noise tbh.
If an author thinks that defining something inline will add clutter, the simple solution is to link the term to an article that defines it.
We may be a technical audience here on HN, but I wager that the vast majority of HN readers have never heard these terms before, just as I never had.
Based on what kyberias said in a sibling comment, it sounds like these terms will be familiar to those who have written DirectX and perhaps other GPU code, but for the rest of us they are esoteric terms.
It is so easy to add a link when you first mention a term like this. Why not do that?
The article goes on to explain the difference between immediate mode and retained mode in detail after the first example. A link would just distract the reader (the reader would then leave the site to learn about the terms, then come back, only to have them explained again).
Why are you assuming that HN is the intended audience? Just because some random person posted the article here does not mean you are the intended audience of the author. It seems quite presumptious to criticise on that basis.
At the risk of repeating what I mentioned in one or two other comments, I am not criticizing the author, and I am not assuming that HN is the intended audience. (And please don't put words in my mouth like that.)
I am offering writing advice that can benefit anyone writing on a technical topic.
You may have an idea of who your intended audience is, but when you put something out on the internet, you don't know where it will end up and who will read it.
It only takes a few minutes to link terms to their definitions or to write a very short definition.
Your expected audience may not need it, but your unexpected audience will definitely appreciate it.
So if his article were posted on a non-technical forum should he also define what a gui is, just in case? Would it not get extremely tedious defining the basics in every article?
I think it is perfectly OK to have an intended audience and ignore an unintended audience if catering for the unintended audience dumbs your writing down for your intended audience.
It is good writing advice to not clutter your writing with needless explanations.
You have a good point that writing out all your definitions may be needless clutter. It was always a good idea in print, but now we have the web and hyperlinks.
So wouldn't linking to definitions or references make both of us happy? If you're in the target audience you can ignore them, and if you're someone more ignorant, like me, you can click the link to learn more. Seems like a good thing all around.
Technical terms are easy enough to Google. I didn't know the difference either, but that would be my first thing to try before complaining on HN.
At the company I work for, so many things I need to know are internal and so impossible to Google, and even asking somebody usually yields a less than thorough answer. I have to build a picture of how the whole thing works in my head over time. Then I become the resource everybody who is in the position I was before hounds.
Having to Google a term because the author didn't think to include a link is roughly a 0.01 on the frustration meter compared to my current job.
I get the desire for easy and flat interlinking of information. But we need to look at the people who do the work to make that happen as bearers of gifts, not sources of misery. Be the change you want to see in the world, don't just demand others do it for you. Google is your friend.
adding a link is very different to adding a definition. The web is made of links, and linking to the wikipedia article (or other sources) is not only recommended, but encouraged.
Immediate mode and retained mode are not technical jargon in user interface APIs, however, which I the technical audience of this article. I don't think it's reasonable to assume the audience is familiar with jargon from an unrelated field, even if that jargon is in common use in that field.
> I was one of the authors of the "visual" part of Visual Basic.
You must have understood the difference between the VB shape control and the VB drawing commands then, if you wrote these things?
The shape controls were retained and the drawing commands immediate. You could move the shape controls around, but not the drawn shapes after they've been drawn.
The double buffer option on forms (can’t remember the name) was a workaround for a limitation of the immediate drawing commands, where they disappeared when the window was covered and revealed.
Ah, my friend, are you seriously quizzing me on code I wrote 30 years ago and haven't revisited since?
But you ask an honest question, so I will give you an honest answer: I have no clue!
We didn't have any terminology like immediate or retained. What we did have was the early Windows API with update regions and WM_PAINT. There was no desktop compositing, so if a window was partially or fully covered and then uncovered, the newly-uncovered area would be invalidated and the window proc would receive a WM_PAINT message to notify it to repaint that area. Could that be related to what you're referring to?
I should explain that our team didn't write all of the "visual" part of VB. We built a product called Ruby (no relation to the programming language) that was originally going to be a user-customizable desktop for Windows 3.0, replacing ProgMan and FileMan. After we turned it over to Microsoft, they decided to combine Ruby with Basic to make a full programming environment, and of course they made many changes and additions along the way.
There are many fun stories I could tell from that era, like the true origin of the term "fire an event", but I'm afraid none of them will involve immediate or retained mode GUIs... :-)
> Graphics APIs can be divided into retained-mode APIs and immediate-mode APIs. Direct2D is an immediate-mode API. Windows Presentation Foundation (WPF) is an example of a retained-mode API.
No, we did not use that documentation. There was no such thing as Win32 at the time, and certainly no Direct2D or WPF. How could we have used Win32 docs? We were just 16-bit Windows programmers, we weren't time travelers!
> What we did have was the early Windows API with update regions and WM_PAINT. There was no desktop compositing, so if a window was partially or fully covered and then uncovered, the newly-uncovered area would be invalidated and the window proc would receive a WM_PAINT message to notify it to repaint that area. Could that be related to what you're referring to?
Ye that's it.
So if you drew a shape onto a normal window, it went into the screen buffer, and would be erased if another window covered that part of the screen. If you wanted the shape to be visible again, you had to draw it again when the WM_PAINT message came. VB didn't remember that you used a drawing command and didn't keep the original details of the shape stored anywhere. That's immediate.
If you used one of the VB drawing controls to draw the same shape, then when the window was revealed after being covered, the shape control, an object in memory, was still there in order for the shape to be automatically redrawn by the VB runtime on the WM_PAINT message itself.
A half-way was to make the shape drawing commands write to a separate GDI (the Win32 graphics API) surface, which was then in a sort of retained mode itself, redrawn to the screen whenever the window was revealed. That was the double-buffer option on forms.
> There was no such thing as Win32 at the time
Well you know what I mean. GDI was part of the 16 bit API as well, possibly called WinG back then I think? Obviously they've updated the documentation as they've gone. But Windows included both retained and immediate components back then.
This is awesome, thank you for the conversation. So I guess we were doing immediate mode GUI and just didn't know what it would eventually be called?
We didn't use WinG [1] - it came along years later, and I think only on Win32, not Win16? We did use GDI, of course, it was there from the very beginning, but there wasn't anything resembling retained mode back then. Just WM_PAINT and update regions.
We actually didn't even worry much about having controls automatically redraw themselves. For the most part, we were just wrapping standard Windows controls or combinations of them, and we let those controls respond to their own WM_PAINT messages as usual.
Of course MS undoubtedly made many improvements after we finished our part of the project. Fun times.
> So I guess we were doing immediate mode GUI and just didn't know what it would eventually be called?
On second thought, that's not right either. Immediate mode is like Unity's OnGUI where you draw everything on every frame.
On Win16, you only got a WM_PAINT when needed, and it came with an update region of whatever had been invalidated since your last WM_PAINT. Many apps would optimize by just repainting the update region.
The Mac worked the same way. What would we call this way of doing things, "paint mode"? We never thought of it as a "mode" at the time because there wasn't any other mode.
For me this concept of expose/damage/paint event which bubbles through the window hierarchy is what defines the retained mode.
In theory in immediate mode widget library you can have global state that defines which part of screen has to be repainted if any, but there is not that much of performance benefit because you still end up doing most of the processing for each frame and when there is any damaged region you still in effect redraw everything and clip it to the region.
From the hip, I'd say one big difference is the API for modals is blocking (waiting for user input), vs callback. That's pretty cool.
I don't yet know how non-modal is handled. Edit: Ah, it looks like return values are also wired up directly, vs the indirection of an event model or listeners. Clever.
--
At this time, I don't think comparisons (in this thread) to OpenGL's immediate ("interactive") vs retained ("batched") modes are relevant.
If you read beyond the first paragraph, you'd learn what the author means by those terms.
The article starts with an example, and then goes on to explain the difference between immediate mode GUI and retained mode GUI is. It is well written and easy to understand article.
What you are asking for is a TLDR at the top of the article. Sure, that would be nice, but on the other hand I don't see anything wrong with writing a blog post that requires reading more than 50 words to fully understand the topic.
> If you read beyond the first paragraph, you'd learn what the author means by those terms.
Ha, you got me there! (I know the HN guidelines recommend to not suggest that someone hasn't read the article, but in this case I'm fair game.)
When I first skimmed the article, I only had a few minutes between tasks at home, so I didn't catch all the details. But since my initial comment stirred up some controversy, I figured I'd better take a closer read. (Well, still kind of halfway skimming since it is getting rather late here.)
In the next to the last paragraph was the clue that made it connect for me: "The OnGUI of Unity Engine actually uses IMGUI mode."
Aha! So this "immediate mode GUI" business is like OnGUI? Where you draw everything from scratch on every frame, instead of the engine remembering stuff for you?
That's when it all made sense, since I had done a bunch of Unity work 4-5 years ago and some of it involved OnGUI().
60654's comment led me to the current OnGUI documentation [1], where they even call it "Immediate Mode GUI (IMGUI)".
I'm pretty sure they didn't use the IMGUI terminology when I was working in Unity, it was just OnGui() with no other special name. Otherwise I would have remembered "immediate mode".
But it's possible that my memory is not as good as I hope it is!
Please use this comment on __projects__. Which fail to include a proper readme or exaplainer instead of a blog article geared towards gfx/game programming
Some also argue that executing UI logic of every widget is wasteful. This might indeed be a concern when implemented poorly
For a game, which is already quite busy even when "idle" (and there are probably other things going on in each frame which take far more CPU and GPU to render), it won't be much of a concern; but with an application where most of the time is spent waiting for user input, spinning the CPU to re-render the GUI every frame is very energy-inefficient.
I don't believe this is a proven. The amount of code being run to cache objects, create and destroy objects, try not to create and destroy objects, marshal data into and out of GUI objects, manage and respond to events, etc. is easily 10x that of an ImGUI, maybe more. Many GUIs now a days are moved almost constantly. Pretty much all phone apps where the user scrolls through a feed of data. A static GUI doesn't need to be re-rendered in an ImGUI, only if something changes. Sure in some ideal test case the Retained mode GUI wins but I suspect it's far less clear what happens in the real world.
Electron is not exactly an epitome of a resource efficient framework either. If I was to compare efficiency, I’d choose a native baseline, such as the X window system or WinAPI. Granted, these APIs are low level and not as rich in features. Typically, the user calls some sort of “mark as dirty” function to mark a region of the window to be updated on the next refresh cycle, thus not wasting resources on portions of the screen that were not touched.
Electron (and nearly all retained mode GUIs) will execute 10x more code trying not to execute the rendering code. Just skipping all that checking and jumping to the rendering code is arguably a win in many many cases.
To put some empirical data on it, game devs use ImGUIs because they keep up with perf where as that 10x overhead of retained mode guis creating gui objects, deleting gui objects, marshaling data into and out of gui objects, adding and removing event handlers and callbacks, and then trying to write a bunch more code to optimize all of that never leads to a GUI that is performant enough for the GUIs needed for game dev.
I think if you check actual apps, especially phone apps, you'll find it's not so clear which one wins. Phone apps almost always move in relation to the users fingers meaning the screen is being re-rendered. Usually new GUI elements are moving on and off the top/bottom of the screen. The overhead of creating/deleting/managing those objects is arguably greater than just re-rendering. The screen is already going to be re-rendered. At best you're getting a few cached textures from your retained mode gui all at the expensive of managing GUI objects.
For desktop apps it really depends on the app. It's got nothing to do with the complexity of the UI and everything to do with what the app does. A text editor might be a win for a retained mode GUI. Photoshop/Maya it's less clear. For example Unity is ImGUI and has very complex UI. Much of the UI has to respond in realtime to the state of the user's document that is changing constantly.
> Electron (and nearly all retained mode GUIs) will execute 10x more code trying not to execute the rendering code. Just skipping all that checking and jumping to the rendering code is arguably a win in many many cases.
How do you figure? Rendering a single Truetype letter will have dozens of conditional branches, use complex floating point math, and touch tens of kilobytes of cache. And there are many thousands of them in the GUI. Even if you have rendered cache, there is still lots of memory pressure copying data back and forth.
Compared to that, retained mode GUIs should be much more efficient. Even 50 checks are nothing compared to ability to void rendering one line.
Note this is all predicated on screen being mostly unchanging. For example, as I am typing this comment, everything is stationary except for a single line where I am typing.
> The screen is already going to be re-rendered.
This is the key! As you said, if you have to re-render whole screen anyway, then retained mode GUIs will have to maintain draw state and re-render every element every time anyway -- a strictly worse performance.
But most of the regular apps, like editors and chat clients and web browsers, do not re-render whole screen every time. They only do one thing at a time.
For an empirical evidence, look at WinAPI design: this is a retained mode GUI, with lots of effort dedicated to figuring out invalidated rectangles and which controls need to be rendered. This was the only way to make a GUI which is performant enough.
And yet the WinAPI is not actually preformant for types of apps that an ImGUI excels at
There's best case for Retained mode GUIs and best case for ImGUIs. ImGUIs excel when lots is changing. In a scrolling mobile app the entire page is being re-rendered constantly as the user moves the page up and down. GUI widgets get created and deleted, data gets marshalled in and out, various algorithms are applied to try to minimize re-creating stuff, all code that doesn't need to be executed in ImGUI mode code.
There are tons of cases where ImGUIs win in perf. In fact the reason they are so popular with for game dev is exactly because they vastly out perform retained mode guis. There's a reason there are 100s of books and 1000s of blog posts and articles trying to get retained mode guis to run smoothly. They almost always fail without lots and lots of specialized client side code to try to minimize GUI widget object creation/deletion, reuse objects, minimize state changes, etc, all that code disappears in an ImGUI and they run at 60fps where as all the retained mode guis struggle to get even 20fps consistently as they hiccup and sputter.
Phone apps almost always move in relation to the users fingers meaning the screen is being re-rendered.
I'm not really familiar with the mobile platforms, but given their reliance on battery power, I hope(!) that scrolling is done mostly in hardware and actual re-rendering happens only for areas beyond the scrolling boundary. Of course you will need to re-render when the actual content changes, but not with just scrolling.
To put some empirical data on it, game devs use ImGUIs because they keep up with perf where as that 10x overhead of retained mode guis creating gui objects, deleting gui objects, marshaling data into and out of gui objects, adding and removing event handlers and callbacks, and then trying to write a bunch more code to optimize all of that never leads to a GUI that is performant enough for the GUIs needed for game dev.
That sounds more like an implementation/application problem --- overuse of OOP or an application where too much is constantly changing --- rather than an inherent inefficiency of retained mode. The Windows native GUI, since Win16 in Windows 1.0, was basically designed for retained mode. Immediate mode is more common in games because it's more suited to that application.
IMGUI always brings a couple of things to mind for me, which don't seem to be discussed often:
Smalltalk/Self GUIs:
AFAIK, the idea (or at least terminology) of MVC comes from the Smalltalk80 GUI. MVC seems to be intrinsically retained-mode: the "view objects" (buttons, forms, etc.) are separate to the "model objects".
Self was inspired by Smalltalk, and its Morphic UI seems to be immediate-mode: the UI objects often are the model objects. This idea is more explicit in the "Naked Objects" approach. Squeak Smalltalk uses Morphic, although its descendents (e.g. Pharo and Newspeak) seem to have replaced it (I'm not sure of their paradigm though)
Denotative, Continuous-Time Programming (DCTP):
This was originally called "Functional Reactive Programming" ( http://conal.net/papers/frp.html ), but that term is now mainly applied to discrete, event-driven systems like Elm and React, which can make some articles/papers a bit confusing to read ;)
In this approach, anything which varies over time (e.g. user input, display widgets, application state, etc.) becomes a function taking the current time as a parameter, called a "behaviour". These behaviours can be composed together in various ways to create an application's overall behaviour, still parameterised by time. The "main loop" just samples this behaviour at an appropriate rate, e.g. a fixed frame rate, or varying based on system load, how much activity is happening, etc.
This is very much IMGUI, but has some nice advantages over the fixed ticking of a traditional main loop. For example, different components of a behaviour can be sampled at different rates, e.g. rapid sampling for smooth animations, coarse sampling for expensive calculations. Interpolation behaviours can remove jerkiness (e.g. a smooth animation based on an expensive simulation).
> Squeak Smalltalk uses Morphic, although its descendents (e.g. Pharo and Newspeak) seem to have replaced it (I'm not sure of their paradigm though)
Pharo is still using Morphic, though there is a new graphics layer called Bloc (vector and constraints based) that has been under development to replace it.
Immediate mode GUIs are an extension of the simple display update loop: (1) update display, (2) check input, (3) goto (1). Immediate mode GUIs add a context parameter to the update function. In the draw context, the update function draws to the screen. In the input context, the update function doesn't draw; instead it checks if mouse clicks or keystrokes should do something.
For example, part of the update function might look like this:
if (Button("Do That Thing"))
doThatThing();
In the draw context, the call to Button will draw a button on the screen and return False. In the input context, the call to Button will check if a mouse click fell within the area it would have drawn. If so, it will return True and the callback will run.
An immediate mode GUI is basically the opposite of a typical Model-View-Controller GUI because it deliberately mixes the Model, View, and Controller together into one function. Just one function draws the GUI based on whatever state it's in and whatever the model is. The same function gets reused in a different context to update the GUI state or handle callbacks.
I'd argue an ImGUI does't mix anything. It's just a library. If you want MVC it's up to you to implement your MVC and use your ImGUI as a library. Retained mode GUIs don't magically make your code MVC and ImGUIs don't magically make your code not MVC. That's entirely up to you.
Also ImGUIs are not multiple pass. That's in implementation detail. Unity's ImGUI is multi-pass. One of the most popular ImGUIs (Dear ImGUI) is single pass
I've taken part in some WPF application development and I think there are some benefits in immediate-mode rendering of the UI. Particularly when the UI changes faster than a monitor refresh rate or there's too much data complexity and you want the data update and redraw processes to be independent.
For my first example, updating controls and overall layout in a WPF application feels much more lightweight than in WinForms because the renderer just shows the result at the next monitor refresh instead of trying to update the layout in realtime or using hacks that disable redraw until you do everything.
For the second example, I've used SciChart to display trading data and it again was much more responsive because it replaced a control-based rendering pipeline with just an immediate-mode canvas tgat would render the selected data window once per monitor refresh rate.
And I think overall once you hit the performance barrier where you can't just fill a panel with controls and let them render and instead have to make a custom draw routine, the immediate-mode rendering adds this extra smoothness to the UI and gives you a bit more performance headroom.
I find it ironic that Unity is used as an example... the immediate mode UI was/is considered a joke.
Terrible performance, doesn’t scale to UIs more complex than a form (... like the ones many games tend to have)
It got to the point where there was a website dedicated to track when we’d get the current non-immediate UI, and Unity actually hired the developer of the leading non-immediate mode UI library in the asset store to work on their own offering.
I think you're mis-understanding why Unity switched. They only switched their in game GUI system. The entire app of Unity itself uses an ImGUI system and it is massively more complex than any Game UI and runs just fine.
The reasons they switched the in game system are many but they have nothing to do with performance.
The old system would stack up draw calls like nobody's bussiness, doubling drawcalls if you used the built in layout tools. OnGUI being called period would be expensive, even when you had properly implemented your rendering logic, so you’d have to disable the entire game object when you didn’t want to draw.
The docs for mobile literally used to say not to use the OnGUI stuff during gameplay.
While I find the idea appealing, I haven't used them because I haven't figured out how to make immediate GUIs accessible to people who are visually impaired. How do you make them work with screen readers? You're drawing text onto a canvas but you don't retain a copy of the text for the screen reader.
Good question. I don't know. When you use the native GUI you get all of that for free. You don't have to do anything special for accessibility. So I always thought that screen readers somehow traversed the window/control hierarchy to find the control that has the focus and ask it for any text that should be read.
Maybe you're right and you can send the text to be read directly to the screen reader. Not sure how well that works when your GUI updates every 20ms. You don't want to keep resetting the reader.
I was thinking maybe resend the stuff only when the text changes, so you'd have to have a cache of some sort, hidden in the library.
Anyway, I believe Qt is screen reader compatible, so it must access the relevant API somehow. I'd be surprised if it was the exclusivity of the OS vendor.
I'm going to date myself here, but this reminds me in some ways of Microsoft's BASIC 7.0 (circa 1989).
One of the new features of 7.0 was a library for producing text mode GUIs with dialog boxes and the like. Because the language didn't support anything resembling a callback, everything had to be written in what was essentially "immediate mode". Lots of control flow that would normally be down in the library stayed up in user code. The net result aligned well with the description in this article: "...much shorter but more intensive, every thing is packed in a single function."
This approach also presented lots of opportunities for errors caused by failing to correctly code something that would be default behavior in a normal library.
Never seen anything else quite like it, aside from this. (Which, quite frankly, I'm happy about.)
Immediate vs deferred, is just another paradigm that goes back and forth in trends like keyboard input/buffering, deffered/one pass lighting, monoliths vs modules, State vs events, flat vs 3D
Are there any popular implementations of immediate mode GUI for the web? It might be interesting to have one that could then "eject" into a stateful UI for when things start to get more complicated.
This is if you expect to build an immediate mode with the same flat rendering model.
imgui's model is "you draw onto pixels". You could create another mental model which is "you draw onto a tree." so instead of saying "drawText(x, y, text)" you would do something like "drawText(domLocation, text)"
Granted, this model gets a bit messier, and _some_ might say defeats the purpose of immediate mode. But I think you could still get conciseness benefits on this.
Sneathlane is a little different because it outputs to canvas and has a rather complex Widget type to handle focus and to allow different outputs to work together, but the Monad instance has the same behavior of waiting for widget completion. Imperative leans more on the DOM, it has its own VDOM implementation where yours looks to use React's but otherwise the libraries look very similar.
I'm a web developer, I've worked with everything from FRP to React, I think this approach is the best. I hope Concur will help to popularize it! I'm writing this on my commute but will be looking into this further today, to see if there's anything in Imperative that might enrich Concur or vice versa.
How does Flutter sit in the spectrum from Immediate mode to Retained mode? I presume it might provide an interesting erspective. On the one hand widgets are constructed on the fly, on the other it uses an OOP language to do so. So I suspect this maps to statefulness and statelessness?
Flutter is implemented in layers. The RenderObject layer looks very much like a traditional retained mode UI. It's possible to write Flutter entirely at this level, nobody forces you to use the Widget layer above it. The Widget layer is basically React-style, where the API looks fairly similar to immediate mode, but it uses caching and diff computation to reflect everything into the retained RenderObjects below it.
Again IMGUI. Several years ago there was a chain of videos and blog posts by different authors. People were drawing two buttons and saying "see how easy it is".
In this post the author seems to go further than two buttons, but I'm skeptical. I wish there was a video showing how it works in dynamics - inputs clicked, texts entered, scrollbars scrolled, dialogs dragged around. Or source code available to build and try locally.
From the code in the post I suppose the majority of UI is dead - can not be interacted with.
I think IMGUI can only be convenient for very simple cases.
In real frameworks the lowest level is immediate mode (see WM_PAINT in WinAPI), but real UI needs quickly evolve them into the traditional RMGUI.
BTW, this thinking is my source of doubts about React.
I think you have it exactly backwards. Most ImGUI use cases are extremely complex UIs and that's where they excel.
One of the most complex apps out there, Unity, is entirely ImGUI based for the app itself and often has 1000s of controls visible. Going though the screenshots of typical ImGUI usage most of them are on the complex side.
Immediate mode UIs come up every now and then, especially in games, and for relatively simple things they're a neat idea (imagine not retaining any extra view state!) But - and there's a big "but" coming up - it does not scale to complex and stateful UIs.
The problem is that, for games that have complex needs - think simulation and strategy games, not racing or arcade - complex UIs are needed to express complex relationships between underlying data and the relevant UX flows. But how do you do that in IMGUI? This information about UI state, layout, etc. is not in the UI anymore, but it has to live somewhere - so you end up having to build your own layout and UI management stuff, and basically duplicate what a RMGUI library would offer, but now with less generality and potentially more bugs.
For the probably best known example, Unity used to have an IMGUI [1] but they've actively moved away from it. It's still there in vestigial form, the API is not removed, but game developers don't really use it.
[1] https://docs.unity3d.com/Manual/GUIScriptingGuide.html
PS. And there's also the lack of visual layout tools, which is directly related to IMGUI's reliance on code to produce layout.