Ooh! I have a mockingbird in my yard that I can try this on! I counted 20 different songs the other night, and it was probably a lot more, I just stopped being able to distinguish. They can have a repertoire of over 200! I was actually wondering if something like this project existed, or maybe just an app really, but then hadn't thought about it since (I'm just an insomniac, not a bird watcher). Very cool.
If you don't know about the North American mockingbird, they fly up to a high perch at night and claim a yard sized area of territory by reciting all the songs it knows, as loud as it can. Over and over again, until it is exhausted. The little psychos. Thus impressing all the lady mockingbirds in the area. Apparently they do a little dance as well, but given it's usually the middle of the night when this happens, I haven't seen it.
They used to drive me nuts until I read up on them. Now I try to count how many songs they know.
Merlin is the most widely-used audio recognition app. It's amazing, try it out. BirdNET is another project (also affiliated with Cornell) that works in a similar way. This project is about locally installing BirdNET.
For me, the app version is able to detect mockingbirds (and the similar mimicking Brown Thrasher) as themselves most of the time. I'm honestly impressed it can do it at all for their songs on small samples.
BirdNET and Pl@ntNet have to be my favorite uses of AI to date. A true use of technology for the greater good.
I've actually been thinking about automated bird recognition for a while now, I live underneath a flight corridor for migrating common cranes (near Berlin). I'd love to be able to one day track their migration across the continent in real time using data from crowdsourced base stations.
I wonder if migrating birds could be identified in flight using optical/radar/audio. If anybody else has had similar ideas I'd really love to chat on this topic.
The iNaturalist app is also quite amazing for identification of all kinds of organisms by image. The also have the "Seek" app which identifies species even offline.
I use both apps, they’re pretty amazing. I typically use Seek first and if it can’t identify the image it lets you upload it to iNaturalist seamlessly for manual identification. I believe then those identifications are then used to further train the model.
The Seek app is one of those things that you can show people and they think it is magic, even in this day and age. I got my mom and my grandma using it!
I've been toying around with photogrammetrically matching in-flight bird silhouettes in order to have a plantnetlike bird classifier.
Very much in the data acquisition stage but I'm seeing moderate success in clustering images by bird shape so far. In between other photogrammetry work I'm starting to think about how to use colour information and reading up on how to select and train a ML model for this problem.
I don't know if you've ever photographed birds, but I suspect this would be difficult because shots that are detailed enough to make out enough detail would be tight in, and shots that covered a large patch of sky would not be detailed enough.
Just speculation though, I'm sure with enough resources these are surmountable issues... I'm just not sure what quantity of resources that is.
Yeh this was my initial concern with optical silhouette classification. The altitude and targets are quite small (and rapidly moving). I've thought about using optical flow analysis to track wingbeat frequency etc as I feel this might be a little simpler to acquire.
On wingbeat frequency I'm very curious to play around with mmWave doppler radar (there's some RF on chip stuff around), emitting a relatively isotropic signal might allow for a pretty wide field of view and I imagine you'd get a pretty reasonable wingbeat signal.
It would be very cool to have network of these listening to calls while birds are migrating at night. With a large enough network you'd get an amazing way to visualise large scale movements which are generally completely invisible. Wiring up the eastern seaboard around Massachusetts, southern Spain, the vast steppes around the Caspian Sea or pretty much anywhere else would be incredible.
BirdNET is ace, looking foward to building it into our app...Birda[0] (Strava for birdwatching). We are hiring too if any nature loving engineers want to work in conservation - dom@birda.org (need to be able to get to London once a fortnight)
I love hackernews because I come here to procrastinate and end in rabbit holes (was looking at getting a raspberry pi for this project), and now I found about this app!
I use strava all the time so that's a great description
Cool. Bought it and much to my surprise, it worked. Surprised since I live in a city, construction going on, motorcycles passing and still: it detected my beloved house sparrows :)
Will give it a shot later tonight when the noise pollution dies down and the birds get going.
Oh, wow! I'm currently working on a sort of BirdNET for pollinators, and am super excited by the possibility of gamification for amping up participation in these citizen science projects. Strava, Pokemon Go, etc. provide an excellent model that folks in conservation can draw on, & this sort of large-scale data is just radically empowering for sample-size constrained ecologists. I'll be looking for work in a few months, but really don't want to abandon the conservation space-- I'll be in touch!
Look for my CV in your inbox in the next day or two. (Based in Ireland and would rather avoid the emissions of flying, maybe a ferry option could work out though)
Birdwatching is so rewarding, I can really recommend it. It feels Real (TM) to go out and watch what happens in nature, live, with your own eyes. You get out, visit new places, practise your senses in ways you don't do otherwise, and are rewarded with new experiences.
Is this really capable of live 24/7 local bird sound identification? That sounds like it could have a huuuuuuge effect on all kinds of things world-wide. Imagine real-time tracking of whether bird sounds are getting fewer and fewer, or tracking real time changes in bird song around the world due to climate change, or just specific species disappearing due to environmental factors. Or being able to push back on new development that harms local bird populations by measuring a reduction in song. The possibilities are endless!
Awesome! I've been eyeing up a Vizycam (https://vizycam.com) to do bird identification but via visual recognition. Sounds like this would be a great addition and would pair nicely with the Vizycam, both run on Pis, too!
This looks terrific - I live next to a forest and have so many birds here, I was thinking of building such a tool myself. I'm definitely going to set some time aside in the next week or so to try this out.
I have a few acres of grazing land that I've been letting go fallow and reforesting for a few years. I've wanted to track the arrival of different species (when will the swallows get here?) etc. and do bird counts over the years, but have always been too lazy to wire this up. This will be very useful!
A wonderful and colourful book on the subject of rewilding is: Wilding by Isabella Tree [1]. It's made me really want to do what you, she, and her husband, have done, though unfortunately my dank city flat lacks the sufficient acreage.
I've read about this person in the Guardian I think. Will have to check out the book.
For what it's worth, I had a dank city flat, but moved to the place with acreage after getting tired of paying tons of rent to live near bars I could never go to after having a kid.
I'm working on a similar project for pollinators. We've got a bumblebee identification webapp up (https://beemachine.ai/), and I'm working on a tool to automatically monitor pollination activity. Have been working on running quantized versions of my models on a Jetson Nano, and I would love to build something analogous to BirdNET-Pi if I can figure out a satisfactory camera situation for the Pi. There's a lot of exciting conservation potential in this space, but a lot of research is constrained by the expense & difficulty of getting decent sample sizes. Decentralized data collection (citizen science!) is the answer.
If there's any others working on similar projects, please reach out, I'd love to chat!
I have Cornell's Merlin app installed on my phone. It will recognize bird vocalizations and seems to do it rather well. Ambient noise has to be low, and if there are too many birds it doesn't work well either.
Weird caveat, but I wonder about the legality of this in various jurisdictions. In my own state in the US, it is generally illegal to record audio without consent. There are various interpretations of this, and probably putting a sign up saying "audio recording in progress" is enough, but it's worth considering.
To be clear, even private, indoor security cameras are not allowed to record audio in my state, though few people realize this. Security camera websites and software do nothing to make this obvious.
Those laws are usually about eavesdropping, and most states (e.g. my state of Illinois) have carveouts for a "reasonable expectation of privacy" (though I found something saying Montana doesn't?). Recording people inside an Illinois hotel room, felony eavesdropping. Recording audio in the hotel lobby, irrelevant. I think you'd have an extremely uphill battle that says a mic outside was "knowingly and intentionally" used to eavesdrop. You would then also need to demonstrate one of:
1. it was done purposefully to eavesdrop (no, it was meant to record birds)
2. it was done to record/transmit/overhear the conversation
3. even if inadvertent, someone used/disclosed anything in the private conversation heard
If all the audio is getting processed to be maybe this or that bird, and the raw feed isn't stored, let alone rehosted, I don't think there's an issue.
Even then, you'd still need someone to bring an action. No one is going to bring an action over a private bird monitor.
It might be nice though if the "live audio" detected human voices and auto-muted them. If you're making a live audio feed available you should post a notice for sure.
Yup. I myself get along fine with my neighbors and wouldn't be concerned, per se. I simply think folks should be aware. There's plenty of lawsuits in my state regarding "illegal wiretapping" and not everyone has good relationships with their neighbors.
I wonder if this would be a way to passively search for calls of the possibly extinct ivory-billed woodpecker; even with a small number of audio recordings in existence.
Does anyone have experience with how this deals with background noise? It appears to be analyzing audio spectra, but can it detect a birdsong against the background of a busy road or a railway nearby? I've seen and heard so many birds near where I live (red-tailed hawks, northern flickers, european starlings, magpies, finches, ...), but audio recordings of them have always been hampered by the sounds of human activity.
The original paper [1] me tions data augmentation in the training dataset such as background noise addition, so it seems to be part of the initial design.
As always with non-stationary noisy signal, any estimator will reach its limit to a certain point.
This is so awesome. I tried to write something like this in 2007 (I called it "Tweeter") but the state of embedded hardware for makers back then was a bit more challenging so it was all PC based. I tried simple spectrogram correlation by sliding a window and comparing low-precision spectra, but I don't know enough about DSP so it never worked. Glad to see it is finally a reality.
Cool. I live next to a park in Somerset, UK. The birdsong in the morning is currently near deafening. The summer house at the bottom of our garden happens to have mains power, a PoE switch and an AP. It sounds like it needs a second Pi (the other runs a TV)
Cool. I can get behind a data gathering tool like this project. My problem with Merlin and Merlin-style apps is they attempt to do everything related to identification which shortchanges users because they don't teach fundamental skills.
I don't think I agree, I was just on holiday in a new country and am not a birder, but with birdnet and Merlin I learned to identify birds by song quite quickly. If not for the apps how would you even find out what bird you hear if you never see it, as happens often in dense forest?
A really specific scenario that might illustrate my point! App or no app, I would accept at the outset it may not be possible, or even advisable, to attempt positive identifications (by ear) of unfamiliar species in a country I've never visited, unless I were in the company of an experienced local friend or guide. Otherwise, I would concentrate on getting good quality field recordings to share when I got home. Same thing goes for birding at home..Merlin and other apps simply cannot approximate that kind of experience building, at this time.
This is so great! I feel like most ML ("AI") products are just linear regressions with fancy packaging, but this actually looks super cool! I want to set this up in my summer house ASAP. Thanks for sharing.
I wonder in which environment this is used. Does it handle all the noises of a typical human-populated environment well enough? Or do people only successfully use it in very remote areas?
there's usually a simple apt-get behind that, but this is not the case. clearly they went through a we-dont-need-apt-just-for-this phase and things got out of hand. there's like a dozen scripts to handle installation of scripts, crons, services and to handle their lifecycle.
No, I'm sorry, I seriously don't understand how that applies in this case. How is "trusting the content you get from `apt-get`/`dpkg`/`pacman`" safer than curling-to-bash? In both cases, you are downloading and installing content from a trusted source (assuming you are curling via https to avoid MITMs). In the curl-to-bash case, you have the _ability_ to at least inspect the code first before you install it (though, realistically, how many folks would really have the dedication and the ability to do so?), whereas with package managers you're installing "blind". Doesn't that make curling-to-bash the _safer_ option?
I clearly must be missing something, here, because everyone decries "curling-to-bash" as a bad security practice, but I haven't seen an explanation of _why_. I tried doing some research for myself - this page[1] suggests that package managers have built-in checksumming and signature verification, but then goes on to propose an attack vector where the load balancer is compromised, so (presumably?) those same checksums/signatures could also be intercepted and faked. This gist[2] has a lot of people speaking against it, but from following those links, the only downside I could see was here[3], pointing out that the shell will keep executing if a connection closes mid-stream - which, yes, is certainly a bad thing and a good argument to use package managers, but not what I would call a security concern (unless you don't trust the entity on the other end of the stream - and, again, I'm taking that as a prerequisite both for `curl | bash` and for using package managers). This[4] demonstrates that a server can detect `curl | bash` and render differently - but, again, if you trust the entity at the other end, that's not an issue.
This is a genuine question, not a fight - you clearly know more about this than me, so I'm asking for your perspective, not challenging you. I recognize that there are non-security flaws to `curl | bash` (handling updates, handling unreliable connections), but I don't see how it's a security concern. How would a malicious attacker exploit someone doing a `curl | bash` from a trusted location over https?
Would that even work (assuming you mean the bird songs)?
I'd imagine bird songs are mostly equivalent to non-verbal singing (e.g. like "la la la") combined with very general information like "I'm here, healthy and single". I really doubt there's anything in bird songs that would constitute language in the sense of verbal expression of information (not to be confused with warning calls, which could as well be specific to certain predators e.g. different calls for "snake", "raptor", or "cat").
As an aside, there was interesting piece with the Birds Aren't Real creator recently, where he steps out of character and talks about his accidental satire fits into the wider field of conspiracy theories: https://www.theguardian.com/us-news/2022/apr/14/the-lunacy-i...
Clearly this software is actually for tracking the varrious surveillance drones that are disguised as "birds", it's just easier to distribute if they play along with the ruse. Checkout the logo of the software-- they're clearly saying that birds are mechanical. :P
If you don't know about the North American mockingbird, they fly up to a high perch at night and claim a yard sized area of territory by reciting all the songs it knows, as loud as it can. Over and over again, until it is exhausted. The little psychos. Thus impressing all the lady mockingbirds in the area. Apparently they do a little dance as well, but given it's usually the middle of the night when this happens, I haven't seen it.
They used to drive me nuts until I read up on them. Now I try to count how many songs they know.