Hey everyone, Josh here the creator. As a developer, I’m always looking out for an emerging market trend on which to bootstrap my own SaaS product. This is since market awareness and timing are often so critical, on top of execution skill of course!
So I built Trennd. Under the hood it continually monitors the web for interesting keywords & topics, classifies them using Google Trends data and packages everything as a neat web app where other people can contribute too.
The app itself is built with the Next.js React framework along with Express, Bootstrap and MongoDB. Next.js was new to me, but made sense since it comes with so much out of the box, including server side rendering.
Let me know if you have any questions, and any feedback is much appreciated!
> I have scripts that are crawling the web in places like twitter, reddit and facebook to generate keyword ideas.
So the trends that it detects is biased towards the corners of the web that I am monitoring. Which skew heavily tech!
Of course - I have scripts constantly monitoring the web for potentially interesting keywords. I then check google trends data for that topic/keyword and analyze the gradient of the line of best fit over different timeframes. If the gradient is higher than a certain (arbitrary) amount, then I class it as a trend!
Pretty neat, I see the most granular time scale currently supported is a 3-month window -- are there any plans to increase granularity towards a (near) real-time system? I get your crawler might be rate-limited at that scale and you would likely have to rethink the entire system architecture, but I think real-time data would open up a whole new dimension of possibilities for detecting/predicting trends perhaps even before public awareness catches up. I'd imagine there would be parties interested in paying for that kind of capability if that's the direction you're planning on taking this idea
I'd love to go as granular as possible, at least 30 days. Real-time would be ideal, but as you note - Google Trends rate limiting soon becomes a hurdle especially when you potentially multiply by different geographies.
Awesome I enjoyed making some trend charts, and the results are fun (i.e I added DevOps, Erlang, Elixir). I think there is a risk it could conceal too many errors and biases for it to be useful for trend analysis. Using services like Twitter and Reddit might introduce biases into your data skewing trends. The challenge is using data and not understanding the intent (is it positive, negative or neutral/sarcastic potentially negative/positive). Google Adwords is good at this. I applaud your effort; I hope you keep going but its a tough nut to crack. I use markets like Google Adwords Tools to do this at the moment, have you looked at it? I think more info around time, intent and volume.
Hi, it would be nice if the keywords had short explanations. I had to find out what an affinity designer and an enneagram was. Bubble tea was also quite new.
Hey Josh, very interesting concept. Thanks for building it!
I tried to use the "technology" category, but the results are not what I expected. Perhaps it would be more useful if I could narrow it down to more specific things, e.g. programming languages, or databases.
Yes - City would be great for people doing stuff around activities, events and local places. But Country would be more powerful for e-commerce I suspect (my usecase).
This definately has a market, but i'm not sure SaaS is it. More typical eCommerce products have a "flavor of the day" like effect, where an article is written in some niche community, and everyone in that community rushes out to buy the thing. Then a few months later, the craze is over. If you're Amazon, you can spot these trends easily (because they have everyone's sales data from the marketplace). If you're a small eCommerce company, unless you just happen to have that product on hand, and someone just happened to have found you, you may never even realize the trend existed. Being alerted that there is a sudden demand for a product, might give you a headstart on marketing and might help you negotiate a deal with a supplier before the supplier realizes what they have.
I'm a moderator of one of the bigger Facebook advertising groups that exist. This caught my eye instantly as the e-commerce and affiliate crowd are going to be all over it. They love being able to catch trends relatively early. This product is ready made built for flavor of the moment marketers.
When I was doing affiliate marketing a decade ago, what I did manually was watch informercials and jot down the names of all those products. Then manually run searches in Google Trends to see which ones were spiking, and create content for those products (ie reviews, guides, etc). Here's an example of a product that really took off after they did an informercial in 2006: https://trends.google.com/trends/explore?date=all&q=almighty... . I created a page for this product and was ranked #1 instantly. Because nobody else was writing about it.
I stopped doing affiliate marketing so don't know what people do now, but just throwing you some ideas.
For sure. So shorter term. They're going to want to move faster than 3 months. Maybe like a past 30 day trend if possible? I would say also if there's a way you could dig into Alibaba and see if there's a bunch of new items of the same type getting listed recently that would be extremely interesting for them as well.
That's an interesting angle regarding Alibaba. The hard part might be automatically grouping similar items, but definitely can see the huge value in this.
not sure i agree so much because its not showing things that are just starting to trend, which is where the real money is at... being in front of the ball rather then behind/beside it. best i can tell this service just shows you the search volume history from about 1100 select terms that anyone can already lookup using googles trend tools.
id be much more interested to see terms each day whose search volume has increased 10x or something... that would be very useful for people trying to get in front of trends.
The "flavor of the day" comment really caught my attention, been thinking of a startup between those lines. I think with people having hundreds of decisions to make everyday from a million choices, it would help to answer those questions for the users without them having to spend time deciding.
I mean its great to have review sites, but what should I have for lunch today? what is trending just give me the answer I am happy to pay $5 a month you decide for me. What programming language is trending?
Will look into it at some point, happy to have a chat with anyone who's interested my email is in my profile
Appreciate your insight - really trying to figure out who I should be building this for, so I know what features and direction to focus on. I was thinking eCommerce entrepreneurs could be it, but then a lot of other people potentially find it valuable too. I want to see who keeps coming back to Trennd in the future, the power users, and ask them why/what they're using it for!
Thanks Josh, I sent you an email I think you answered most of the stuff here though just came back to the comments now. Would still be great to have a chat sometime
Very cool! As a front-end dev it's interesting (but I guess also not surprising) to see React and Vue with very steep curves.
I wanted to try adding Svelte, so I signed up by email, but that doesn't seem to be working. Then I tried Sign In With Twitter but, wow, it requires an awful lot of permissions! Among them... Follow new people, Update your profile, and Post Tweets for you. I'd be a lot happier signing in with Twitter if it were limited to read-only abilities.
The key value-add, in my mind, from that product, besides the trend, is the editorial content they include in the email. Of course, that's just me as a layperson.
I'd probably dupe their pricing model and approach: i.e., a limited number of weekly/monthly trends, subscribers get more.
Definitely - the glimpse newsletter looks good and is the same concept of surfacing new Google trends. I didn't like being limited to just a few trends per month everyone else sees via newsletter though.
I was looking for an interactive app with 1000 trends I could dig through. But as you say, then the editorial content/insight for each trend is key value-add. That's part of the reason I've built in for users to be able to add their own trends, insights and comments. If we can crowdsource it, then to some extent we don't have to trade-off quantity and quality.
And/or sell the service where you BYO keyword list and this ranks them. Keyword lists can themselves be scraped from forums and websites in the marked the client is interested. The scraping could also be an add on service.
This would lock in nicely with the keyword research that ahrefs, moz, etc. offer. And other various colour hat seo software.
Selling the data as an API is definitely a potential. I was originally thinking to allow CSV downloads of the data. Would an API be more useful for your client's use case?
You seem to have done most of the hard work, collecting the data. I'm not sure they'd care what format the data was in. Providing it could be expanded to cover the topics their clients need.
It's interesting to see that topics like 'fentanyl', 'opiod' have done up.
Also 'criminal defenses', 'wrongful death claim' have gone up a lot as well - Perhaps they are related to spikes in searches for plastic surgery topics.
Also it's weird why almost all the topics on the front page are related to software development. It's like there is some kind of conspiracy to turn the world's population into software developers.
Nice idea. For very short time spans, I noticed some "trends" seem to be seasonal trends ("aperol", "lawn mower"...). It might be interesting to compare with the trend for the same search the year before at the same period, to be able to identify those cases (for intance, I cannot see from the graphs whether Aperol is more or less trendy this summer compared to last summer)
The examples on the front page looks surprisingly good, which makes me suspect they were hand-picked. Kind of beats the purpose of the app if I'm just shown trends the author and me already know about, since we seem to inhabit the same subcultures.
When building a system that surfaces interesting trends, you have to be careful about curating which historic examples you show. People will craft their understanding of predictive power based on those examples... but if the examples aren't representative (i.e. randomly sampled) of the algorithm, then you're selling people rainbows and unicorns.
Many years ago I built a service on appengine to hit the auto complete api endpoint for every single and 2 letter prefix, as well as things like "how do I" or "what is the".
It ran for a few months before blowing up the free quota, but it was pretty cool while it lasted.. Seeing certain things bubble up and then disappear. Like right now 'lion king' is the first suggestion for 'l', but I doubt it was a month ago.
A peak every April and a dip every December?
This seems like the result of some smoothing algorithm needing ~4 months of data and resetting every year.
People search for more craft beer before Christmas (because they're looking for a unique gift) and less for AirBnB (because they get together with their families for the holidays).
Seasonal patterns in data about human behavior are the norm rather than the exception. That includes search trends. There are also monthly, weekly and daily cycles, as well as some that have not quite yearly period, e.g. Ramadan.
I'd also like to see downward trends, not just growing trend. From the last page or two, I see some downward trend, like "blog", "facebook", and "gmail". I don't know if it means the decline of popularity, but it is definitely interesting to see the declining graph.
Brilliant stuff. Fascinating to see how seasonal some of the searches are – "rosé" and "Aperol" spikes in July and December (southern/northern hemisphere summers?), "carbs" and "low-carbohydrate diet" peaks in January (when people want to change their habits?).
This is pretty brilliant. I'd hook up something to identify trends with companies that can potentially be invested in. Could be useful from a publicly traded stand point (Good ole stock picking) or even VC investing if the product was rapidly growing and not yet heavily invested in.
Is there some open source version of something like this? I would be interested in monitoring some non-tech topics and it would be useful to have something with most of the basic features in place to do so, even if it's a library
I guess the most logical conclusion is that every time they make a sacrifice under the full moon during the equinox, they get rewarded with more interest in Gmail.
This is awesome. I'd love to have the ability to view topics based on category. IE. see the biggest trends in entertainment or the biggest trends in sports players, or the biggest trends in startup brands.
Very cool, but this is essentially not "discovery". There is a heavy skew toward tech - but niche tech (Notion for example) specifically. It would be cool to get a broader, less bias perspective.
If possible, I'd like ti integrate with the real monthly search numbers for each keyword, I can't get it anywhere, only see a trend score, could u explain a bit more on this?
That's very neat and interesting; I guess the two questions are
- how do you generate keywords to search for ?
- how do you filter out noise (or perhaps you don't filter it) ?
Yeah they're normalized relative to a scale of 100, as it's Google trends data. In the future I want to include actual search volume numbers as a line on the chart too though. Just need to figure out where to get that data first!
This could have massive potential with music industry A&R's looking to capitalize on viral songs. That's where virtually all recording contract money is going nowadays.
Looks good, but I miss units on the axis. On mobile, it's missing from the start page. The units for the y-axis on the detail view is missing (as far as I can see).
Right now spike/potential/hyper classifications are my subjective opinion.. which I hate because that varies depending on the day I'm having and my specific biases.
Figuring out how to classify them mathematically is next on the to-do list. For example a spike would have extreme outliers in the middle and hyper would be over a certain exponent.
So I built Trennd. Under the hood it continually monitors the web for interesting keywords & topics, classifies them using Google Trends data and packages everything as a neat web app where other people can contribute too.
The app itself is built with the Next.js React framework along with Express, Bootstrap and MongoDB. Next.js was new to me, but made sense since it comes with so much out of the box, including server side rendering.
Let me know if you have any questions, and any feedback is much appreciated!