I would say functions are important, but absolutely pale in comparison to modeling[1].
If you are unable to describe, on paper, what your problem domain actually is, then you have no business opening up an IDE and typing out a single line of code.
I will take that further. If, with your domain model, you are unable to craft a query that projects a needed fact from an instance of the model, the you should probably start over. Dimensionality and normalization are a big deal with your model. You have to be really careful about nesting collections of things within other things if you want to allow for higher-order logic to project required views. This is something we struggled with for a really long time. Every rewrite conversation began something like "well customer and account need each other in equal measure...". And it took us god knows how many iterations to figure out the relationship should not be an explicit property either way.
Put shortly, Modeling is the core of it all. Start with simple top-level collections of things. E.g. instead of modeling Customer.Transactions[], model Customers, Transactions and CustomerTransactions. This fundamental principle of keeping your model dimensions separated by way of relational types can help to un-fuck the most problematic of domains. These will start to look a lot like EF POCOs for managing SQL tables...
At the end of the data, data dominates, and SQL is the king of managing data. If you embrace these realities, you might be encouraged to embed SQL a lot deeper into your applications. I spent a long time writing a conditional expression parser by hand. Feels like a really bad use of time in retrospect, but I did learn some things. Now I am looking at using SQLite to do all of this heavy-lifting for me. All I have to do is define a schema matching my domain model, insert data per instance of the model, and run SQL against it to evaluate conditionals and produce complex string output as desired.
What if code is needed to explore the problem domain? There is utility in discovery, especially for analysts/data scientists who tend to write a surprising amount of code.
I think you’re both right. If you frame the original comment as “don’t write final production code without thorough modeling” it works both ways. If you want to counter with “well our non-final code always goes to production anyway!” You have a cultural problem that needs addressing.
To do that, you need to be willing to delete code you worked hard on. Lots of people aren't good at that.
And some think asking your boss if it's okay is a good idea, (spoilers: they'll say no). That is just a way to pass blame for a decision you can't stomach.
You model a certain experimental ("discovery") thing, then you implement it then you analyses the result then you change the model etc.
And sure in many cases in practice people might not skip the modeling but don't properly write it down in any later one usable way. Especially during initial experimental discovery phases. Which isn't good. But understandable and can be fully ok. Honestly especially for boring simple web API's this is pretty common. It's just important to know when to stop ;=)
The thing about modelling is that it often works on a higher abstraction level then the programming languages provide and that a bunch of (often performance related) thinks are (preferably) not represented in modelling.
This IMHO makes most tools to generate code from models just painful to use.
But I still agree that you have no business programming something which you can't somewhat model in a higher abstraction level.
> SQL is the king of managing data
Hm, not so much IMHO. SQL Is terrible bad at it in some contexts because it's inherently made for a 2d table projection which is (more or less) only joined to larger 2d table projections of data which often in it's nature is neither 2d nor maps well to 2d representations. And while you can extend SQL to support that or work around it with e.g. recursive queries it's not very nice to do at all.
There's a fascinating book "Data Model Patterns: A Metadata Map" by David C. Hay that's pretty much a Pattern Language or catalog for data models. You can just implement the subset of Hay's patterns that make sense for your application.
Relying primarily on relational modeling reminds me of Out of the Tar Pit[0]. The well known paper suggests a combination of functional programming and relational data modeling.
I definitely agree with this. Some of the fasted implementations I have ever done were after spending a bit of time modelling the problem solution. Basic data flows, classes (using verb/noun parsing of the requirements doc) and system architecture were all decided before I wrote any code.
The implementation itself just flowed, allowing me to focus on smaller details that can't be modelled (e.g. error handling). By copying the design of classes and function names, I didn't have to backtrack and redo anything, I didn't have to think about names of things - which were pre decided and so consistent throughout the codebase and my code dovetailed nicely with parts that other people implemented.
wiring pure functions decouples processing from data. I find this encourages developing well defined models naturally as well as making the model much more agile and malleable
I definitely agree, I've realized that most programming languages store data in a hierarchy (structs within structs) which you then "query" in a very static way with the "." operator. Normalizing data and storing it relationally seems way more flexible for a lot of use cases which is why most databases are relational and not hierarchical.
However, I've tried to figure out how to actually store and query data relationally in a language like C++ and haven't been able to figure out a good way. I don't want to use SQLite because of performance, this needs to be close to real time (like a game or something similar). I just want a way to store data relationally in memory in C++. I'm still trying to figure out the right approach here. C++ is very static which makes it difficult. I've been able to figure it out in Javascript though.
>"If you are unable to describe, on paper, what your problem domain actually is, then you have no business opening up an IDE and typing out a single line of code."
I agree with this part assuming that graphics and video along with the words are allowed.
>"I will take that further. If, with your domain model, you are unable to craft a query that projects a needed fact from an instance of the model, the you should probably start over."
This is very narrow minded approach that will only work for a very limited set of possible domains. Simplest example to the contrary: the domain is a creation of efficient way of solving some math related problem. What query?
I think you may underestimate the potential scope of a domain model and the capabilities of SQL. It is certainly math. That is actually the incredible part. That its all just math underneath 20 different joins which express a very complex and meaningful view of the domain facts.
I challenge anyone to present a problem domain which cannot be meaningfully represented in terms of tables in a database. I would prefer if this were bounded by the set of problems you would use any software development strategy upon, but I welcome a more difficult problem as well.
>"I think you may underestimate the potential scope of a domain model and the capabilities of SQL."
I am being practical. There are many languages that are Turing complete but ill suited for particular domain. It's been proven that SQL is Turing complete as well. However should you propose using SQL to write implementation of say FFT you are not likely to find much of a sympathy.
I would not use SQL to implement the actual algorithm, but I would certainly consider using it to hold all of the data around such an operation as required. For instance, tables like Samples, Spectrograms, etc.
Hierarchies are possible with an RDBMS, but I'd suggest it's the wrong tool to model them with, unless they're static of course. Really any graph that can't be encoded in the table relationships themselves.
I have trouble understanding this kind of talk. What's a problem domain, what's its dimension and normalization, and what's the high order logic all about? Can we use plain words people from our grandfather generation can recognize?
Higher-Order Logic = combining basic functional building blocks in order to compose more complex functionality. SQL enables direct, declarative access to the whole space of higher-order functions. E.g. You want the list of widgets made 3 quarters ago but scoped to one factory line, and only when a certain rotation of employees was on the factory floor? You got it. That's like 10-15 lines of SQL.
A higher order function is a function that takes another function as a parameter or returns a function as its result. Famous higher order functions include "map" and "filter", for example. Javascript, for example, uses higher order functions all the time. See [1].
The term "higher order logic" typically means program logic that uses higher order functions. An object-oriented programming style is inherently higher order because objects typically contain functions and are passed to methods.
The term "higher order logic" can also mean a system of logic that allows statements about logical statements. [2]
If you are unable to describe, on paper, what your problem domain actually is, then you have no business opening up an IDE and typing out a single line of code.
I will take that further. If, with your domain model, you are unable to craft a query that projects a needed fact from an instance of the model, the you should probably start over. Dimensionality and normalization are a big deal with your model. You have to be really careful about nesting collections of things within other things if you want to allow for higher-order logic to project required views. This is something we struggled with for a really long time. Every rewrite conversation began something like "well customer and account need each other in equal measure...". And it took us god knows how many iterations to figure out the relationship should not be an explicit property either way.
Put shortly, Modeling is the core of it all. Start with simple top-level collections of things. E.g. instead of modeling Customer.Transactions[], model Customers, Transactions and CustomerTransactions. This fundamental principle of keeping your model dimensions separated by way of relational types can help to un-fuck the most problematic of domains. These will start to look a lot like EF POCOs for managing SQL tables...
At the end of the data, data dominates, and SQL is the king of managing data. If you embrace these realities, you might be encouraged to embed SQL a lot deeper into your applications. I spent a long time writing a conditional expression parser by hand. Feels like a really bad use of time in retrospect, but I did learn some things. Now I am looking at using SQLite to do all of this heavy-lifting for me. All I have to do is define a schema matching my domain model, insert data per instance of the model, and run SQL against it to evaluate conditionals and produce complex string output as desired.
[1] https://users.ece.utexas.edu/~adnan/pike.html