The vega-lite wrapper is very slim too, basically just converts your slithery_snake keys into humpyCamels and corrals between different data sources, so if the docs on the elixir page seem sparse its because most of the "how do I build a graph like x" is just on the vega docs. You can easily dump it to json to share too, etc.
This concept sounds quite a bit like the usage of .mdx files in OCaml. Those are markdown files which can load and execute OCaml code in them.
They are useful for unit testing code like libraries, providing usage examples for end-users (as well as the text format markdown usually supports) and testing it at the same time. Definitely a great idea for documentation, and one which I should use more of.
Apart from running code inside a "markdown" file, livebook can do much more. You have Smart cells to show charts, run sql queries against a db, run Neural Network tasks such as Image-To-Text generation using Bumblebee[1], etc. It is collaborative as well.
The entire product site is a thing of richly informative beauty.
---
My only question was whether the generated 'docs' snippets would add value over just reading the story in your DASL. Any markdown site generator (such as the chosen Material for MKDocs) can just embed the ```yaml anyway. But then I realized what was generating e.g. …
The ability to execute sample code during documentation generation seems invaluable. Instead of being subject to rot, documentation turns into an executable test suite.
I've been working on something like this for Kotlin using a compiler plugin that allows code to access the source text of lambdas, functions and classes being executed. You write code that spits out markdown and captures its own source into code blocks.
You can write inline examples in your docstrings and they’re automatically turned into unit tests. So again you get that documentation as an executable test suite not subject to rot, but at a per-function level. Same kind of idea but different levels of the documentation stack.
- Elixir's scripting makes it super easy to write a one-liner that tests a function, that'd be a lot more verbose in many other languages
- You need the docs tool and unit tests tools to 'speak the same language' to make it work, and for whatever reason Elixir devs don't seem as intent on reinventing tools like those as some other languages
That's fantastic. We use Kotlin to build Taxi (taxilang.org) and orbitalhq.com. this would be an awesome addition to our docs.
However, do I understand it correctly that this allows me to output verified, working code in my test suite, but its not a tool that I can embed in my docs to create runnable code samples? (which is what the OP is showing).
super valuable either way, just trying to validate understanding.
That sounds about right. The approach I'm working on won't allow you to embed runnable code in your docs. Instead, it requires you to write your documentation in Kotlin e.g. using a Markdown DSL where source code from the codebase can be included in the output. I'm working on a separate project to programmatically generate file trees and Docusaurus doc sites in code.
I'm not aware of anything exactly like Livebook in the Kotlin ecosystem but you might be interested in kotlinx-knit which takers a more incremental approach to executable documentation:
I would suggest to anyone to download Livebook and test it out, even if you don't know Elixir or Erlang. You can do so much with it. I even use it for things outside of programming due to it being pretty much a markdown notebook that you can run code inside of.
You actually don't. You can use smart cells which lets you do a lot without actually coding. You can transform data, build charts, run AI model and more all without writing code.
You might have to Google how to read a file in Elixir, but you don't actually need to know Elixir.
In the future, you won't even need to know that to use it.
If you're interested in this hybrid of docs and live coding, I'm building something similar for the Typescript / Javascript / React ecosystem @ https://www.typecell.org (open source). See https://twitter.com/YousefED/status/1677342430545301504 for a preview of the next version which will be more "wysiwyg" than the current notebook environment
Today, in a very round-about way, one could use a livebook as if it were a blog post: download-install Livebook on your machive, download the blog post livemd file, run it locally.
If you go through the steps, you'll notice that the livemd isn't stylized as a blog post. This is where the next opportunity lies: creating beautiful blog posts as livebooks, and without the need to install and run a livebook server locally.
Is there a way to render the livebook as read only until they try a change? I tried to open the link and it failed. I could download it and run it but I just want to see it as you intended.
One small note: It looks like the 'Mix standalone' runtime option was removed late last year. Instead, you can use `Mix.install/2` pointing to your project directory:
While Livebook will mostly have an appeal for people already working with erlang/elixir, it does have a few features that are pretty nice.
- it’s collaborative (think Google docs for code) when several people are working on the same instance of a livebook;
- it’s easy to extend with so-called smart cells (which are essentially pieces of gui you can inject in your document https://news.livebook.dev/v0.6-automate-and-learn-with-smart... ). Smart cells are available for various tasks (db connection/ interaction, data frame exploration, ML tasks, maps), and building your own is relatively easy;
thanks for the detailed answer of the differences. The interactive (I assume using CRDT's) is a good feature, jupyter server does not have this and is at a time a bit of a hassle.
Also deploying it directly as an app is pretty sweet.
Livebook actually uses state reducers as described here[1].
Except for cell content. There it uses Operational Transformation.
The source code[2] is really instructional and easy to understand. I highly recommend to look into it :)
This is about live "documentation". Docusaurus doesn't have programmable cells linked to live processes or systems. Docusaurus is fantastic, but it's static. Livebooks are like next-generation REPLs with slick UI, security, reproducibility, etc.
I'm not sure to understand what you mean by "Docusaurus is static".
Docusaurus builds static pages and allows you to plug JS/React code anywhere in your docs, so it's quite interactive and can run anything that can run in a browser, including REPLs.
I read the Fly article but still don't really understand what Livebook is about.
They say they use Livebook themselves, but the examples linked to only display a regular non-interactive doc to me.
Do you have any production url showing me an experience that is possible in Livebook, and impossible in Docusaurus?
Sure, here's a simple demonstration of what livebook can do that docusaurus cannot, at least without maintainer expertise ;)
Imagine the following.
<here's a code block containing project metadata>
we click a play button to start an elixir runtime based on that metadata code block
<here's a "component" that securely connects to an elixir system running in a kubernetes pod in production, exposing an elixir REPL to interact with it>
with this component, we then issue commands to inspect state, manage elixir "processes" in the runtime, etc
> we click a play button to start an elixir runtime based on that metadata code block
Can you show a concrete example? IE a real production url running this?
What is "metadata"?
In Docusaurus you can have a live playground evaluating on your browser, or you can embed any embeddable playground if it requires a server integration.
> with this component, we then issue commands to inspect state, manage elixir "processes" in the runtime, etc
Another example would be useful.
So this is just an embedded widget to interact with something remote? Why can't this be built as a React component that you can add to any Docusaurus page?
---
It looks to me that you don't need maintainer knowledge to build that, and React knowledge is enough.
Not to be rude.
The live playground probably can’t run elixir/erlang code. And it most certainly cannot connect to a remote running node instance.
It’s unfortunate, that the angle here is documentation since, while Livebooks are great for testing and documentation ( https://blog.appsignal.com/2022/05/24/livebook-for-elixir-ju... ), they also provide a great environment for experimenting with elixir code.
I’m sure docausorus is great, I’m not sure why you would think Livebooks would be a competitor in the same space.
As to you asking for in production links, I’d encourage you to install it and test it for yourself.
The code blocks need to be run in sequence, from top-down, as one builds on the next. The "metadata" at the very top often describes the project and its dependencies.
After reading this article, I'm still not sure to understand what Livebook is.
Can someone show me a real production url of what is possible to achieve in Livebook and impossible/difficult to achieve with other tools?
I'm the Docusaurus maintainer, and making your docs interactive, and giving the ability to run the documented project inside its doc does not feel like something new.
One simple thing I need thats stopping me from jumping into Livebook fully: Salesforce API integration. Right now, I can't manage to make the auth work there (since oAuth needs some http redirect dance).
Connecting to some proper SQL database is a breeze, but if I cannot access some silo'ed data that is absilute critical, its not that useful after all :-/
I slightly complain a single time for the last few years, and jose directly chimes in with kind words immediately. You rock, honestly!
But on the topic: I think its a simple problem ultimately, the SF Api (https://developer.salesforce.com/docs/atlas.en-us.api_rest.m...) needs OAuth, which needs a redirect at some point, where I have no idea how to capture it from a livebook instance (easy in a phx app).
This is a reference lib in JS ( https://jsforce.github.io/ ) and there seems to be no maintained alternative on hex.pm unfortunately. So, from scratch it seems, somehow.
Additional insight: many big companies do use salesforce, and its internal tooling to produce reports/insights is rather limited, but there is a SQL-like language to do complicated stuff, but there is no way to execute a handwritten query in SF itself, except via API calls from outside. This... would be a perfect fit fro livebook! query SF + other databases to generate a report ad-hoc thats not easily possible right now without adding some heavyweight data warehouse connection!
I wonder if we could proxy OAuth flows to notebooks. For example, a notebook could initiate an OAuth2 flow, Livebook handles it, and once it gets back, we notify the notebook which may approve or deny the request. The biggest concern though is that we need to have a static callback URL, so you would need to run Livebook internally at a fixed address?
If this is something you would like to work on, please reach out, I think it could open up interesting possibilities!
update: this specific issue is something I could work around differently: defining a user that has its own credentials within SF who can be used in the auth flow did the trick without any redirect needs!
Jupyter outputs to JSON. Livebook outputs to markdown, so it's more naturally renderable outside of the tool itself.
There are still some rough edges -- it would be nice if livebook could cache prerendered content. for example, kino graphs only chuck a big fat JSON into a html comment, it would be nice if it also popped out an svg or encoded png with data uri.
xcode swift playground are supposed to be the same kind of ideas. However, it's really one of the worst maintained piece of software in the whole toolsuite. Really a shame..
hopefully this will get us closer to a orgmode-ish confluence that can run any language, with CI docs test checks, and something more human than jiraql or whatever
In the same "standing on giant's shoulders" stance, you can use Explorer (see example LiveBook at https://github.com/elixir-explorer/explorer/blob/main/notebo...), which leverages Polars (https://www.pola.rs), a very fast DataFrame library and now a company (https://www.pola.rs/posts/company-announcement/) with 4M$ seed.