Exactly right. This is not an argument to not use Django.
I think a lot of us have come to the same conclusion as this author: the abstractions frameworks provide are often too high level. Very quickly the needs of your project cannot be served by their generalizations. Especially in a language like Python, it's often easier just to roll your own systems.
However, starting off with a framework is a great way to get off the ground quickly. If you were to write everything from scratch from the beginning, you might burn out before you even produce anything usable.
Personally, I think CherryPy is the way to go. You're probably not going to need to rewrite HTTP [1], so that's an abstraction layer with which I'm comfortable. Everything else will need to be easily customizable for your project.
[1] Unless the threaded model doesn't work for you, like it doesn't for our chat backend.
From what I read, it didn't sound like he had much trouble. And, honestly, the various "swappable components" frameworks are basically in the same boat as far as I know: the ability to mix and match decreases dramatically once you've written application code, because their emphasis is on the up-front choices, not on swapping things later on.
It certainly wasn't easy. Although the vast majority of the difficulty can be attributed to porting application code, some difficulty can be attributed to Django. I would have preferred a framework which makes it easier to swap out components, but it is unclear how much it would have really mattered except for in terms of psychology.
Well, this is something I've been quietly agitating about for a while, because I have a feeling that your case is one where "swappability" doesn't help much -- once you've written your application code around a particular component, switching to another component is going to involve rewriting. How much rewriting may vary, but AFAIK there's nothing that can automagically port code, say, between popular Python ORMs.
Agreed. I eventually moved from django to pylons going exactly through this process- except that instead of rewriting pieces I was able to just plug them in with pylons. I believe the reddit guys had a similar experience.
It's bizarre... I started writing a project in django 2 weeks ago and had to think about some of those elements in the first week. But I'm still running "clean" django!
Here are my solutions for anyone interested:
* URL Routing: They mention only breadcrumbs functionality here. I created breadcrumbs by template inheritance instead. There are url routers split by application and by modules. Template inheritance tree resembles the url routing levels, so that each template has: "{% block breadcrumbs %}{{ block.super }} · current-level-text{% endblock %}" Simple, but works so far :)
* Authorization: My views are decorated with @require_auth() if they require authorisation, @require_auth('group name') if they require special privileges and @require_auth(redir_to='...') if user should be redirected to a special place after authorisation. (by default they come back to the same place) Simple and works.
* Authentication: I simply didn't use the standard user classes. I used my own model. I don't consider it dropping Django - it's more like not using standard helpers.
* Templating: So far I'm happy with the whole logic in the view. Not sure how complex their application is... I'd really like to see some samples that required programming in the template...
* ORM and Admin UI: Can't comment on this one. I'm writing for google app engine and simply using their object store. It feels more comfortable than RDB so far. But sometimes I use GQL if it's faster.
* Form Validation and Generation: I've created a couple of dynamic forms extended with dynamic fields by jquery, as well as some forms with fields and validation generated on the fly (from user input, or saved data)... I can't find anything that's not possible with django forms. Sometimes I customise forms as needed, sometimes the autogenerated ones are almost perfect. Again - not using helpers all the time is not "dropping django".
I'm a bit disappointed that they didn't provide more examples and their solutions... maybe we could learn something from that.
It doesn't even sound like he made an effort at learning django, let alone python. . .look, django has a bit of a high learning curve behind it (I'm still at it myself) but when it comes down to its a web-framework, a web-framework for people who like programming in python.
I'll admit I felt the same way at first, but I was honest with myself enough to stop toiling with django and set aside the time to learn how to program in python. Once I started pin-pointing the obvious advantages of writing application-level programs in python I started to grasp the underlying advantages of django's design and philosophy.
That's the impression I got as well. He didn't seem to "get it." I can't see complaining about Django being poorly decoupled, only to then go on to describe how little of it you're still using. How tightly-coupled can it be if you're only using small pieces?
Also, it seems like a lot of the complaints are about contributed apps like authentication and the admin. It seems to me that he never really tried to learn the admin, or was using a very old version. And you don't have to use authentication at all, you can completely write your own, or fork the one that comes with it and make it work for you.
But the main thing that bugs me is that he obviously doesn't get opensource either. If you think you have a better way, get on the mailing list and suggest it, create a ticket, or implement your changes and submit a patch. Writing your own stuff, not releasing it, and complaining on a blog doesn't help anyone.
Well, with auth I think there's an easy confusion to fall into, because we conflate a couple things:
1. The default backend stores information through the User class, persisted to the auth_user table.
2. All backends are required to return instances of User when asked to find the user who matches a given set of credentials.
So in the first case, we use the User class as a persistent object store. In the second case we use it as a consistent API implemented by records coming out of auth backends.
Where the confusion comes in is people assuming that, since you have to return an instance of User out of your custom backend, you must also have to persist that instance to the auth_user table. Which isn't correct; you're free to just chuck information into an instance on the fly, and Django will neither know nor care (third-party code may make bad assumptions about being able to save that object, but third-party code should be introspecting to find out where the object came from before trying that).
Any rate, this is on my list of things to document more completely, and hopefully that'll help.
While I agree with the premise of this comment, I'm not sure it necessarily applies to this specific situation.
Web frameworks do their best to provide useful, general abstractions for writing web applications. Sometimes you need different behavior on a lower level to which the framework provides no interface. In other words, your particular application breaks the general mold in some way. It doesn't necessarily make sense to change the framework for each of these application-specific needs. Writing those portions yourself and not contributing them to the project seems like a sensible way to go.
If the framework is really trying to work for the entire universe of web applications, it will quickly become too bloated to be useful for getting simple, common apps off the ground. As a framework maintainer, you have to strike a balance between the set of applications that your framework can accommodate and the usability of the framework.
That said, the author's tone does seem to suggest that Django is doing a poor job as a web framework because it doesn't fit his needs. That's an unfair criticism. Django fits a lot of people's needs. It's not supposed to be all things to all people.
The admin is bar far the biggest offender of tight coupling: primarily to authentication and the ORM. Being "contributed" isn't an excuse. Everything is "contributed". It's open source!
I haven't got time to actively participate in Django's development. Especially because I'm not particularly interested in the details discussed on the mailing list and I've already found components I'd prefer to work with (and, by extension, contribute to). I've got a product to build. I took some time to write down my thoughts and the key contributors are welcome to read my rant, ask questions, and learn from it.
Open source software is not a one-way street. If you had spent as much effort on this gripe post as participating in the community (or even reading the docs!) then you would have had the answers to your questions rather than having to roll your own framework.
But now he has something better than django that he can use on his next project. It works well for the kinds of projects he builds, and does what he expects. He wins.
There's no reason to contribute it back to the community, because it's specifically targeted to his needs (and therefore wouldn't be considered better by anybody outside his team). Spending even an hour polishing it up for somebody else's consumption would be a distraction.
But if he would contribute back, then -everyone- would have something better than the (current) django. That is how it works. I agree with grandparent poster that more people should take the time to contribute and/or make their changes available.
That said, at least he documented things and presented his issues cogently so that the dev team can see where issues are, and possibly address them in the future.
Not likely. I'm a big advocate of rolling your own framework that meets your (and only your) needs.
The thing we use at Expat is incredibly fast and turns out really good code. It's the reason we can move so fast, and it's a competitive advantage. But I wouldn't release it to the world.
It just has too many optimizations for the way we like to do things. And there's still the matter of the order-of-magnitude gap in level of effort between something you use in house and something you can release as a product.
Right now, this guy has something stripped down to the bare essentials to do what his shop does. Ideally, he's shed all the useless garbage that naturally accumulates in any commercial framework the size of django. And as a result, the final product probably isn't of any use to anybody outside his team.
I know Python quite well; I have done several significant projects in it over the last few years. I have also read every page of the Django book and the vast majority of the official documentation. I became quite familiar with the internals of it, as I have spent many hours studying the source code before setting off to completely customize things.
What did I say which indicates a lack of experience?
I didn't feel this way in the beginning. At first, I quite liked Django. It was a huge time saver. I don't even really dislike Django now, our project has simply out grown it.
I'm no big fan of Django, but it sounds to me like you started using a well-established framework and whenever something was painful you chose to roll your own. Maybe a better approach would have been to engage with the Django community and help make these things less painful. That way your experience would benefit the next users, and you'd benefit from the collective wisdom and shared resources of a big and serious community. All other things being equal, rolling your own is rarely the right solution.
"We wouldn't ever hire a designer who couldn't pass a code review for some trivial template logic."
I've worked with four design firms now, three of them excellent, and none of them could have been expected to understand templating code. If you're asking your designers to work with a template language, you're doing it wrong.
This has nothing to do with whether you put logic in your templates or not. The designer's deliverable isn't a working site; it's HTML/CSS/PNGs, and it's a dev's job to pull that into the framework.
Those two opposing viewpoints on a designer's role probably have a lot to do with the type of application you are designing. If I was doing the hiring, I would expect more template language understanding for a web application than from marketing or traditional web site.
I think it's fair for him to say he wouldn't ever hire a designer like that but I think it's because he's only built sites where it makes sense to optimize for that.
Why on earth would you want to narrow your options? There are dozens of template languages. Why would you want to spend one cent extra, or sacrifice one extra cent of quality, to find a designer who was not only skilled in information design, graphic design, typography, and interaction design, but also Django (or Erb or PHP or Clearsilver) literate?
Django templates are significantly different than all the other ones you mentioned.
It was architected from the beginning to be friendly to non-programmers -- the idea was that someone writing an article for the Lawrence Journal-World (or a designer) could easily use custom templating in a constrained templating language that developers can augment with custom tags and filters.
It surprisingly works incredibly well for that use case. The templating language strictly enforces view-separation -- the template author can't do anything the developer didn't intentionally let them do.
It can be ridiculously frustrating if you're expecting it to be a normal language -- for instance __getitem__, __getattr__, and __call__ all use the same "obj.property: arg" syntax. It calls hasattr at every call site to figure this out, and you can't use () or [] even if you wanted to. Fuck, you can't pass multiple arguments to a function!
I've heard the refrain about keeping logic out of templates so many times, and yet professionaly I've never come across a situation where totally programming-clueless designers were actively editing templates. It always falls into one of two categories:
* the designer doesn't go anywhere near the real app and does all their work as mockups of HTML / CSS or even Illustrator / Photoshop. Developers do the work to integrate templates, designers at most fiddle with the CSS or bare HTML in the template and never touch any logic.
* the designer is reasonably competent to deal with the primitive programming in templates and is involved in the full lifecycle of product development
The bottom line is that if they are too leary of programming to cope with simple logic structures in a template then they are altogether incapable of working productively as part of the development lifecycle and should just hand their stuff to people who have those skills.
If you're going to have external designers: I agree, their deliverables include ZERO logic.
However, zero logic is even less than Django templates. If it is the devs jobs to retrofit the HTML/CSS/PNGs with templating logic, it is an even bigger argument for choosing a developer friendly template language instead of a designer friendly one.
Django is not without it's problems, but some of this isn't correct (upon a quick skim). For example, the Django admin does support filtering and sorting, you just need to indicate this in the admin class.
You can set sorting order and filtering options in your admin class, but the UI doesn't provide clickable column headers or anything like a search box.
EDIT: Apparently I was mistaken, as the replies point out. I read the Django book, but I do not recall this chapter. My apologies.
The admin provides both of these features -- specify the columns you want to be clickable in the `list_display` attribute on your `ModelAdmin` class. Similarly, specify a `search_fields` attribute to add a search box for the columns you want to search.
Why read the documentation or look for examples when you can write a blog post screed and post it on HN?
By default, any field on the model that's displayed as a column in the list of objects is clickable to sort. You can also display things based on other attributes of the model (e.g., calling a method), and if you like you can specify that they're tied to particular fields to get clickable sorting for those as well.
I was going through the django book and here is something that could help you out:
search_fields = ('title',)
"Finally, the search_fields option creates a field that allows text searches. It allows searches by the title field (so you could type Django to show all books with “Django” in the title)."
I've found the same thing with django. It does some things well (the admin interface, for example.) Other things it does in ways that seem very unintuitive and unpythonic.
For example:
- includes a test framework, but tests on a fresh project don't pass
- no relative paths in settings.py make it hard to work with other people (sure, since settings.py is python code you can write code to get around it, but I don't like that approach)
- you can't extend the User class (except by using deprecated code); the workaround is to create a one-to-one table but that's ugly.
- templates have their own control structures rather than using the ones pythonists already know (django's ifequals vs. python's if, for example)
If you're fed up with django, try Pylons. It is very loosely coupled, to the point where the recommended ORM and template system are separate projects (SQLAlchemy and Mako).
"We are now using SqlAlchemy via schema reflection; no declarative layer." (emphasis mine)
What next? Will web development come full circle to using raw SQL queries like Phil Greenspun suggested years ago? (Not that there is anything wrong with it)
I've worked with a number of different ORMs, including two on the project I'm on right now, one custom one, and a handful of open-source ones on Ruby and Python and the like, and every single time, the story's the same.
At the start of the project, when queries are few and datasets are small, all of the above worked great. But the further you get into the project, and the more complex things you are trying to do with larger and larger datasets, the ORM ends up blowing up in your face. For example, LINQ to SQL--which I want to emphasize I'm picking on only because I'm using it righ tnow--happens to silently and transparently convert from IQueryable (meaning that your LINQ will execute in the DB) to IEnumerable (which means you'll suck in all data required for filtering past that point), which means that you never have a good idea whether your query is going to be optimized or not unless you check the SQL being generated. When you have to do this a few dozen times, you start wishing you'd just written SQL in the first case.
The situation gets worse when you start getting into join-heavy queries. I've never seen an ORM that actually can cover everything you need to make an app performant. It doesn't handle full joins, or compound indices, or distinguishing between clustered and nonclustered indices, or calculated columns, or some other thing that, if you were in raw SQL, would easily solve your problem, so at that point, you start introducing raw SQL. The older your app gets, the more instances of this you get, until you'd have been better off simply using SQL the entire time anyway.
I love the idea of an ORM in theory, but I'm pretty solidly convinced these days that if you want objects, you should us an OODBMS, and otherwise, you should just embrace the relational model for what it is, and not pretend you're working with objects. For most people, that'll mean writing SQL and getting back query sets.
I haven't used Linq to SQL except for to play with it for a few minutes. It seemed like a very capable ORM, but I sensed that the IQueriable to IEnumerable conversion may cause some headachs.
Linq to Objects, however, is freaking amazing. C#'s functional and querying features are like Python's generator expressions on crack, speed, steroids, and acid. Absolutely love them.
LINQ to Objects is C#'s (very good) take on Python's list comprehensions, and is awesome. However, that's just providing a declarative, functional way to query/iterate objects--not an ORM. LINQ to XML is likewise also okay, since XML's tree-based model happens to map extremely well to objects in a way that a relational model cannot do.
LINQ is great. LINQ to SQL is actually very good, so far as that goes; it just happens to be trying to do something that I'm not convinced can be done well.
Web.py does something like that. Not raw SQL but almost:
result = db.select('table', what='id, name',
where='id>3', order='id', limit=5)
Is better than cursors, but for medium to large web applications is too much work.
I used to be against ORMs ("Another layer? How inefficient"), but now I see them like compilers: They're easier to work with and, most of the time, SQLAlchemy does the job better than me (lazy-loading, etc.).
My bet would be schema-free databases, such as MongoDB.
Schema synchronization is just a major headache, no matter from which end you approach it (declarative -> evolution or Raw DDL -> reflection).
Once your number of databases exceeds 1 you are in trouble either way.
The implicit assumption that a schema definition would ideally be set in stone once and then never changed is a legacy carried over from the very early RDBMS days. And it's one of the biggest albatrosses to date.
I've had similar experiences... too often Django makes it difficult to do something really simple. If you are not building a newspaper-like site that fits exactly to the Django model, it gets annoying.
But it does support a lot of stuff out of the box, so it's still my favorite python web framework, and I have probably set up about a dozen different Django projects.
One problem I often run into is that the Django libraries expect the whole application to be run by something that uses manage.py, which sets particular os.environ settings. So if you make code dependent on a Django library, often it cannot be run any more by non Django code. The eventual result is that things like cron jobs and unit testing are second-class citizens.
Another problem is unicode. Current Django assumes you will deal with all strings as valid unicode, which is actually different from pre-1.0 Django. There is no way to store an arbitrary sequence of bytes without a unicode encoding. This makes perfect sense for a newspaper website, but is problematic for many other applications.
Another problem is imports in general. Django does a lot of work in __init__.py files which causes problems integrating it with other systems that do a lot of things at run time.
The article gave some examples that I've run into myself: using email addresses instead of usernames for authentication, and adding additional fields to the User class. I also dislike the template language. Why invent your own language and control structures when everyone using it knows python? Mako (http://www.makotemplates.org/) does a good job of not reinventing the wheel.
"The Django template system recognizes that templates are most often written by designers, not programmers, and therefore should not assume Python knowledge."
I figured that had to do with it, but it doesn't really explain why they don't just use an equivalently powerful subset of python. Better yet, allow full python like Mako does, and let the designers decide what subset of that to use in templates.
I copied the chained actions model from Catalyst, and the routing is automatically generated. I also have template tags defined that construct these URLs for me (not uploaded in the repository).
I structure my views in a structure like ...
views/root.py
views/articles.py
views/users.py
views.root contains a "begin" method, that will be chained to all other requests. If views.articles needs users to be authenticated, I also add a "begin" view to views.articles (where I do the authorization required). So as an example, a chain could look like this ...
This is really far from the way things are done in Django, but it seems more intuitive to me.
What I find cool about Django is that the dispatching is really simple to understand, and it can be extended (and components aren't hard to replace in general).
I don't quite understand the URL complaint. Each app can have its own urls file that you include in the main urls file. Isn't that a tree-like structure?
Also views can be served by callable classes, so an init function can populate the context (or a chain of them with class inheritance) and then the appropriate code can be run to serve the view.
It's a tree of lists of trees. I rather a tree, than tree-like :-)
Some of our apps have deeply nested paths (typically a no-no, but actually quite appropriate for our problem domain), so it is impractical to create a module and call "include" for each tree node.
As for callable classes: someone has to instantiate them. That's no different than our custom routing and authentication. I just choose function composition over class inheritence.
I'm trying to keep my work, personal, and startup lives seperate. My project is no secret, but one of my recent blog posts was trafficed by some co-workers. I don't need my manager wondering if/when I'll be quiting.
There is a new version on the way with a huge host of new features and some significant cleanup. The version that is live now is more of a proof of concept.
* start by using it as it is intended. Save lots of time by being up and running quickly.
* find a flaw.
* realize that the workaround is more work than rewriting a piece of the framework.
* stop using that part of the framework, but continue to use the rest of the code that you still haven't had to write.