I'm at the very early planning (read: daydreaming ;) stage for a tiling project of my own, (where my "constraints" are "has to be nerdy-awesome"). Like find a specific conway's game of life still life, or perhaps use the newly discovered aperiodic "hat" tileing.
Any chance you would be willing to dump your notes somewhere I can see?
I used a pretty straightforward formulation, with a binary decision variable for each possible tile placement/orientation, and constraints to ensure that the pattern is possible to build: uses correct number of tiles of each type, no overlap ("if there's a 2x2 tile at (x,y), there is no 2x2 tile at (x,y+1)").
Another approach to the constraints could be to define the set of covered subsquares for each placement, and have a single constraint that says that each subsquare must be covered exactly once.
It was a pretty small patio (about 2 by 3 meters), with 17 square tiles and 12 rectangle tiles, and this formulation worked well enough to find all solutions to that in reasonable time. However, you get a lot of minor variations on the same general pattern. On this small scale just quickly scrolling these variations was manageable, but with a larger project you'd have to do something more intelligent.
* a certain aspect ratio,
* rotational symmetry,
* no spots where four corners touch,
and a few other ad-hoc constraints that I can't remember exactly, to exclude certain sub-patterns.