Cool, man! I've wanted a standalone player to do this ever since WinAmp used to support this 20 years ago, but seemingly the capability was lost to time when everyone decided they'd rather watch recorded video. The composer Stephen Malinowski's YouTube channel is one of my favorites because all of the videos are visualizations of music he's generated with custom midi-reading software he made, i.e.: https://www.youtube.com/watch?v=ffYKCNY6kUk. He first wrote this software in 1985 a decade after getting the idea from listening to Bach on LSD.
It's just too bad this is a browser extension because I really don't want more of those. I get that the economics probably don't make sense to do any more, given the effort to make a standalone desktop app would be a lot and a website people could upload tracks to in order to get a visualization would force you to pay hosting costs.
Also a fan of Malinowski's animated music videos. My immediate thought was that it would be cool to see this extension sort of mimic his piano roll animations by having it appear vertically on the left. Less pixel space though, so I don't know...
Would also be cool if the dots on the pitch line that represent accidentals were perhaps a slightly darker shade so it'd be easier to see the typical piano pattern in them. (Like ⦾⦿⦾⦿⦾⦾⦿⦾⦿⦾⦿⦾ kinda thing.)
Anyway, to original poster: very cool extension, thanks for sharing! :)
Actually, you can use mpv player and install visualizer.lua on mpv-scripts repository. The engine of YouTube Musical spectrum (showcqt-js) is a javascript port of ffmpeg showcqt filter.
That Stephen Malinowski link was great. I don’t know that song by name, but I could unmistakably tell what song it was just by looking at the visualization (even though I left the video muted)!
When I read notes, I express through interpretation as a spectrum of emotion. So this spectral visualization is pretty much how I visually interpret musical notes, especially when playing by ear.
> As someone who has been learning how to sing for the past year, I have tried many spectrum tools and have come across one that is really useful for singing with my own voice but isn't that great for multi-frequency visualization and therefore isn't that good for learning other songs but THIS extension really fills a void for learning other songs!!
Awesome to see projects like this! I have music-to-visual synesthesia when smoking pot and making something that shows how music looks to me is a project that I’d like get to sometime also. My problem is that I don’t really care to get high often enough to get it done. It would look quite different than yours or Stephen Malinowski’s visualizations.
Does anyone know any stand alone feature rich audio visualizer for Windows? I am surprised that Spotify never included any visualizer for the desktop client. I miss the old Winamp days.
There are apps like Plane9 (very similar to Milkdrop from Winamp) and the commercial Luminant Music. I'd say FL Studio's integrated plugin ZGameEditor Visualizer is probably the top alternative considering the sheer number of presets and templates as well as configurability. They're all listed at AlternativeTo.
There is a standalone visualizer that is my go to, projectm. It's a spiritual successor to the visualizer from winamp days and is compatible with the old milkdrop presets.
I haven't been able to find a proper way to set up mesh size on windows but if you're running it on linux or android, those options are available too.
The visualization engine is showcqt-js (https://github.com/mfcc64/showcqt-js). It is a javascript port of ffmpeg showcqt (Show Constant Q Transform). Probably, more correct name is Variable Q Transform.
I've implemented the Constant Q transform before, and it does work well for western music. Are you using an autocorrelation to boost the fundamental? It can work well, particularly when there's just one note being played
It's a cool project. I'm just curious, because I haven't bumped into anyone else who's dabbled with the CQT before.
So in what way is it variable Q? I thought the innovation of CQT over a regular FFT is that the bins represent each note step with the same 'Q', making notes more distinguishable? (whereas a FFT has more 'Q' in the high frequencies)
I think having an autocorrelation option on the visualization could be cool, as it could reduce the spikes from the overtones (and also show a missing fundamental). But I think you'd need a different convolution for each instrument.
linear STFT's window length in time domain = k (constant)
CQT's window length in time domain = k/f
showcqt-js window length in time domain = a * b / (a / c + b * f / (1 - c)) + a * b / (b * f / c + a / (1 - c))
whera a = 384, b = 0.33, c = 0.17
I also apply asymmetric window to reduce latency before doing VQT.
> I think having an autocorrelation option on the visualization could be cool, as it could reduce the spikes from the overtones (and also show a missing fundamental). But I think you'd need a different convolution for each instrument.
Research needed. Doing autocorrelation on simple single instrument monophonic audio is probably easy. But doing it on complex multi instrument audio isn't easy. Probably, it needs some sort of machine learning.
> So this is some sort of compromise to increase speed?
Partially. The main purpose is to increase time domain accuracy on low frequency. If you want to do experiment on window length (in time domain), use ffmpeg showcqt filter (https://ffmpeg.org/ffmpeg-filters.html#showcqt). showcqt-js window length is hardcoded to tc=0.33 attack=0.033 and tlength='st(0,0.17); 384tc / (384 / ld(0) + tcf /(1-ld(0))) + 384tc / (tcf / ld(0) + 384 /(1-ld(0)))' (https://github.com/mfcc64/mpv-scripts/blob/a0cd87eeb974a4602...).
> Have you thought about implementing in WASM?
It is implemented in WASM.
> Agreed, but for a visualisation, it could just be a parameter just for the user to mess with.
If you want to do experiment on it, showcqt.js exposes intermediate color data cqt.color[]. It can be modified arbitrarily, including autocorrelation. If some day I do experiment on it and find that the result is satisfying, maybe I'll include it in YouTube Musical Spectrum.
> The main purpose is to increase time domain accuracy on low frequency
I thought so. The main issue with regular CQT is latency on low frequencies due to the excessive window required. It makes it unsuitable for real- time applications, as you know. Thanks for the insights
It's just too bad this is a browser extension because I really don't want more of those. I get that the economics probably don't make sense to do any more, given the effort to make a standalone desktop app would be a lot and a website people could upload tracks to in order to get a visualization would force you to pay hosting costs.