Check out the work that Local Projects (NYC) did for the Cooper Hewitt Museum. One component of their installation involves museum visitors standing in front of a screen and a computer searching for art that matches their pose.
Look up Plink Art. It was created by a couple of guys at my uni, and eventually acquired by Google a few years ago. They may have a white paper or two kicking around outlining their methodology.
Conceptually, if you’re trying to classify artwork by a category like artist or period, the problem is just ordinary image classification and won’t require anything unique so I’d recommend just reading the current SOA papers for that task.
The foundation for many of the current SOA papers is He, et. al:
can't think of any off-hand, but a good start would be building a palette histogram for each of the images, I would expect that visual art will have color palettes that are very distinct from photographs.
Detecting copies of known images? Detecting styles? Detecting artists? Detecting whether a picture contains art at all?