I played around with a machine learning demo and used a banana, apple and an orange for learning via webcam, and used speech synthesis to make it speak out load. After the accuracy was good I point the cam on my wife and it said: - 100% certainty a banana
hah! keep in mind the model will always make a prediction with one of the labels it is trained with for any image it is shown. You can add a "none" label and add images of things that are not banana, orange, apple, to learn the important features of a picture that make it a banana. if you are using your webcam, you can collect images of you, your office, backgrounds, etc
Do you ever think that it's a fundamental limitation of these systems that they aren't good at knowing what they don't know? Like they always give an answer, and their failure modes are so different to ours that that it can be hard for non-experts to interpret the outputs.
In some of the less harmless applications of computer vision and machine learning, sometimes it will have very severe consequences for real people that a computer says yes or no when it really doesn't have the information to say either or. Some people are afraid of what will happen to society when these systems become as accurate as humans - I am honestly more worried about what will happen if they don't.
Who's going around designing a system that will have severe consequences without mitigating the problem of misclassification? There are techniques available, such as autoencoders, sensor fusion, ensembles, using multiple images, training on "neither" examples, asking for human confirmation, etc. It might never be perfect but neither are humans. We see monsters in the dark, the virgin Mary on toast, a face on Mars, optical illusions, get our attention distracted (magic tricks), act maliciously, fall asleep, etc.
The weird failure modes thing already happened with lossy image compression. Characters in non-OCRed text go replaced with different ones by photocopiers, and people saw spaceships in space probe photos of the sun. We'll get used to the odd banana riding a motorbike and realize what's up.
How is the failure mode different from a human’s? The human mind comes with an answer no matter the situation it’s presented with, and that may be a stupid answer.
There is thankfully no (known) input that makes the human mind fail. There are known inputs for some animals though (like chickens).
Though humans do often confabulate, they can also say, "I don't know", "what is that", "wait a minute", "there's something fishy about this", "huh?". Sometimes they can invent a new label or phrase on the spot that captures component attributes.
That would be a response, wouldn't it? If you give an AI "a way out", which is essentially the same, it will take a way out when it thinks it the smartest thing to do.
The trick is to lower the punishment for taking the way out. It's not free, but saying a car is a dog gets you -1, where as seeing a car and saying I don't know, only gets you a -.1 punishment (or even a .1 reward, vs a 1 reward for a correct answer).
Pretty inconvenient, with chickens apparently this sometimes happens by accident. They don't get up, ever. They lie there until they get attacked or just die.
Fascinating... That must be a "drop dead" self-defense mechanism triggered by what looks to it like its beak drawing a line in the dirt due to it being dragged away by a predator.
I've been wanting for quite some time to build a device with a camera that could recognize my cat on the counter and turn on a servo that would release a jet of compressed air. It looks like I could actually use this for that.
I’ve been debating doing this too. I’m not too worried about the cat detection but haven’t got a clue how to programmatically release compressed air? Wouldn’t it be as easy just to play the sound of compressed air through a speaker? My dumb cat wouldn’t be able to tell the difference.
Use high quality relays purchased from Digikey or another distributor (should be $10-30 in single quantities for AC mains) and put thermal fuses on the mains side of the relay (I generally put several, each going to different parts that are at risk of shorting). Slather everything but the heatsinks (add some if necessary) with fire retardant epoxy, carefully pushing the thermal fuse into the epoxy until it makes contact and pushes out most of the epoxy under the fuse.
Burning down the house isn’t even an issue for me. USB relay throws 24VDC from a UL-approved power supply that opens the valve. The noise from the air compressor OTOH....
Add a random delay 0-60 sec delay for the jet air. This would create a fear and whenever she would sit there will think about jet of air coming any moment of time and now you can remove the machine after 1 month when your cat is trained.
Yup. I also seem to remember studies showing that if the reward/punishment is given randomly, it strengthens the Pavlovian response.
I plan (if I ever do this) to program a decay over time, starting at 100% chance/zero seconds, and moving to lower chance and higher random time interval.
Exactly. Minus the job and kids, I'd love to have the time to learn all the tools I need for all the dozens of projects in my head. But if this makes it easier, it greatly increases the chance of the project actually happening. (And heck, I still have to work out how I'll actually send the signal to the servo...)
Check out the oss project [autofocus](https://github.com/uptake/autofocus) used by the Chicago zoo to operate camera traps. Could be useful for you too!
omg please make this project haha I love this - we'll make it a project highlight on our twitter @lobe_ai! And be a hit on our subreddit https://reddit.com/r/lobe
I wonder if they could do something with Proton to make a Linux desktop version. Since Valve made Proton for games I imagine that graphics cards support for things like CUDA might work just fine still.
All the Cuda stuff runs fine in Linux already, no wine required. (But possibly a weekend of tricksy driver setup... For which I blame nvidia, not anyone in the Linux side.)
I really like how the website is done. Visually and content-wise. It transports the message pretty well into my brain.
Concise, not overloaded, good font sizes and looks good on mobile and desktop.
Thank you very much Aldipower! I am one of the designers from the Lobe team here speaking. This is what we strived for when we designed the website, and to be honest, when we designed the product, too, so I am really happy it came across this way.
In fact, our process for every feature we work on is the same—we start thinking about it with the way users are going to learn about it in mind, that allows us to simplify the way we talk about it and massage the messaging as much as possible, so when we have to talk about it externally, it's so tested that it just comes natural to us, and hopefully to the world.
Maybe this is just on Firefox Mobile, but for me there is an image of some berries and no text shown for 0.5 seconds, which then gets replaced by a black rectangle, and then text appears that is borderline impossible to read over the black rectangle. The first thing I see is basically broken to the point of uselessness.
Edit: yep, on Chrome Mobile I actually see an animation and stuff seems to work. On Firefox it's borked.
Thanks for the feedback! We will be adding "Object Detection" next, which is identifying objects within an image and adding a bounding box around it. We are also exploring multi-label classification to classify an image as "dog" and "cat".
For Image Classification, that is a good approach is predicting if an image has both a cat and dog is important.
Seems to be marketed as 'machine learning', but upon closer look is only for machine learning on images. Anyone know of something similar for analysis of other kinds of data; particularly interested to analyze records (like spreadsheet data)?
Under project templates - coming soon they say it will work with tabular data.
I expect, as they allude to, it will begin with simple classification tasks in order to stick with the clean user experience they've built. But I'm super eager to see what they propose in this area.
We are working on adding more project templates in the future, and Lobe is designed upon the idea that machine learning should be made easy—no matter the problem type you are facing.
I'm the founder, but you can use sysrev.com to review pdfs, json, text, etc. and assign labels / do annotations. You can see https://blog.sysrev.com/simple-ner/ for how to build something like a gene named entity recognizer in text. We have mechanical turk like compensation tools too, but you'll need to ping me (tom@insilica.co) for access.
There are other options for this too, I think spacy.io has an annotation app.
Yep we are starting with image classification for this initial beta launch, but plan to expand to more data types and problem types in future releases! The vision is to make a tool usable by anyone to build custom machine learning
This seems pretty cool -- but one issue to me is that (similar to the chasm that exists in low-code app building once the magic doesn't suit you) that if I already have the skills to create a mobile app that integrates tensorflow, I probably also have the skills to train my models. It would be cool if feature-extraction (image pre-processing and first network layer(s)) could run on the front-end, and the rest of the network/search on the back-end, similar to how distributed speech recognition works. Then I could use a canned lib on the device that integrates w/the camera, and get my results via a websocket. (Of course, I could still run everything on the client still as well.)
What's that old joke? Something like in the 1980s a Media Lab teacher gives the class a computer vision assignment where they're supposed to be able to tell whether or not an image contains a bird, and 40 years later they're still working on it? Lobe.ai reminds me of trying to identify plants with Google Goggles 10(ish?) years ago. It didn't work very well then, and then Google killed Goggles. Side note: none of the "click on the leaf feature" web-based plant identifiers gave a satisfactory answer either.
Google Lens (on the stock Android camera app) works pretty great for identifying plants to me. I'd say it's right about 95% of the time (in New England) if I can see a flower, and about 75% of the time for a leaf.
I was going to say, Lens works incredibly well. Working with a farm box of random vegetables that we couldn't identify, Lens had a 95% accuracy and instantly, too.
I've used Google Lens a little bit, to see if it could be used for object identification.
It always seemes assume that I'm trying to buy something and tries to find products me that are somehow visually related to what it's looking at, or else to the content of some lettering that it's able to detect.
Unsure. That maybe sounds like the Google Lens standalone app from a couple years ago. The Google Lens mode with the stock camera app never shows me anything to buy. ....actually, I just pointed it my rollerball pen, and it identified the pen correctly and then showed me a link to the pen on Alibaba. But I think that was just the image it was matching. It never does that for natural objects.
That explains it; it is the Google Lens app I used. Or does it?
However, if you go to the Google Store page for it, it does tout this feature:
"IDENTIFY PLANTS & ANIMALS Find out what that plant is in your friend's apartment, or what kind of dog you saw in the park."
I have the current version, which was updated on August 13, 2020.
I will give it another spin, and also look for the lens mode in the camera app. (My phone is from Google, so the camera is the Google one; if any Android camera app has a Google Lens mode, it should be that one.)
"Working" can be interpreted differently. 75% accuracy (?) is not that great for predicting a specific class (depending on distribution on course).
If you need high accuracy, let's say for e.g. estimating eco system performance based on specific plant distribution, 75% is very low (especially if you want to feed it into another predictor) compared to a professional field biologist.
Seek (the iOS) app works great -- I use it all the time on walks to identify plants (and even bugs and birds, if I can get close enough and they'll sit still). I think it's based on iNaturalist, which has a large community of professional botanists etc. collecting and classifying image data.
I think your comment is behind the state of the art. I bought a house with a beautiful garden in the backyard. I didn't know what any of this stuff was or how to take care of it. The app my wife got (not sure what it is) got all of them correct except 1, with a single photo. That's for at least 50 different plant species.
We're really excited to bring machine learning to more people and make it more accessible. What we are most excited about is empowering people to create custom machine learning models for specific and personal use-cases and solve problems in new and unique ways
I recall that in the 18th century people tried to build a computer, so... What's your point? Computer vision didn't work back than, and it does work now, but not for everything. If you don't understand it you should go read about it, not posting pointless replies
There are a number of new apps that take a crack at it. I've been using PlantSnap, and it seems to do OK. It doesn't always get it, and I wouldn't stake my life on it, but it's been good for assuaging idle curiosity.
I remember a previous version from about 2 years ago... I think Lobe.ai was web-based, at the time? And you could drag and drop various blocks around to do image recognition and analysis? I probably have some of the details wrong but the demos were very impressive.
While I never got approved for that beta (probably rightly so, I'm just some random person with no actual connection to ML or AI), I was excited to see what their work led to. Congrats on releasing this latest iteration and acquisition!
Thank you! And you are right, the general idea of Lobe 1.0 is in your message! For Lobe 2.0 we switched a bit as you can see, and the good news is, you can go and download the app at lobe.ai without the need to wait any longer!
The reasoning behind the change and the why we abstracted some of those details you are mentioning was to actually make it even more accessible for people to be able to build machine learning models. We think that this is a paradigm that should be used by everyone, and that’s why stripping down the onion of complexity was really important for us when we started with this project.
Thank you so much! We deeply explored how to make it easy for anyone to get started with machine learning. We looked at where people were spending the most time getting started and iterating on creating a custom machine learning model. This is why we expanded Lobe to focus on 3 fundamental steps:
1. Collecting & labeling images
2. Training your model and evaluating the results
3. Playing with your model and seeing how well its performing
Why does this exist? I don't mean what do you use it for, the many uses are obvious, but why has a company (Microsoft) made it and released it for free.
Reading the license I assume it may change at some future version to require money to use it, and that a new version will install and then say please pay us to continue using? Or probably just this product is no longer available? Note these are not things I am thinking will happen but rather my theoretical assumptions to try to answer the question of why has Microsoft, a for profit company, made this closed source, free tool that I think might be pretty useful for a lot of people.
Our driving force is to make this technology as accessible as possible to as many people as possible. We believe that machine learning will be a huge new way that people interact with computers going forward to better their lives.
Lobe will always let you train custom machine learning for free on your computer. We hope this becomes a vibrant ecosystem, and the business model around the edges can come later for value-add services.
>Lobe will always let you train custom machine learning for free on your computer.
ah ok, fine, just the legalese was making me wonder. And of course that we are in a capitalist system so not sure I follow the value for Microsoft in this scenario, but I guess you find making machine learning more available to apps somehow drives value.
You agree to receive these automatic updates without any additional notice. Updates may not include or support all existing software features, services, or peripheral devices.
This is the part that causes me to assume it will stop working at some point in the future? And when would that be:
a) Term.
The term of this agreement will continue until the commercial release of the software. We also may not release a commercial version.
But honestly unsure if I am just paranoid. Or even if paranoia is the right term for my feeling about it, it's something Microsoft is letting me use and at some point it won't be usable anymore - such is life - might be the more reasonable response to it.
I must have missed it, but Lobe is owned by Microsoft. The product looks clean and well suited for CV 101 applications. Looks like a no-code meets AI solution. Anyone using it beyond research / personal project implementations?
One thing I thought of when I saw the demo video, that is probably on the team's radar:
There would be a lot of cool ways to improve the model by giving feedback, either showing training images where the model is uncertain, or some more advanced explanations for classifications flagged as incorrect, in order to guide the user to gather the training data that can improve it.
And possibly providing a summary of where it knows it works well.
There are a lot of benefits there, both for improving models people are building but also to help users understand why their model is performing as it does.
Thanks for your suggestions here. We are always looking at ways to improve Lobe, and the feedback loop of how to improve your model is one of the most important ones for us.
The app is beautifully done. I'm really impressed by how well it works given the knobs available.
However I tried to train it to recognize some images of characters from an anime (so a little different than facial recognition), and I managed to break the model: achieving 64% error with significant number of examples per class. I think one downside is Lobe doesn't expose how potentially overconfident the model is. I would love the ability to take the existing model and test it on a new image that I can import into the app.
EDIT: I would love to see the following in a future version:
1. What are the percentages associated with each image per class. I see that an image was misclassified, but did it at least include my desired class in its top 5 predicted classes?
2. Test the model on unlabeled inputs directly in the app to see how well the model might generalize. I would like to see a "Test" tab on the left once training is complete.
3. View other metrics of model goodness like F-1 score and training details like CV partitions in the app somehow.
Hey there! Thank you so much for the feedback, we are planning improvements like this.
Here's a few tips for now:
1. You can view by "Test Images" on the Train tab (view options). So you can see how well your model is performing on your test images (a random 20% split from all of your images).
2. You can test your model on the Play tab, by dragging in new images your model has not seen, to see how well it is performing. You can also tell Lobe if it was correct or not and iteratively improve your model.
We believe there are several advantages of Lobe over tools like Google AutoML :) Lobe is making the entire process of creating custom machine learning accessible, from creating your dataset to training and playing with your model, to integrating it into apps:
* Easy to use - no coding, cloud configuration or machine learning experience required.
* Free & private - train for free on your own computer without uploading your data to the cloud. No accounts required.
* Ship anywhere - available for both Mac and Windows. Export your model and ship it on any platform you choose.
AutoML requires paid accounts with high friction setup and is focused on just training a model on your data. You would have to pay and retrain your model manually every time you want to make an iteration. Lobe gives fluidity with iterating and providing feedback to your model through Play.
Thanks for the details! It's good to know this is an option for projects that require a custom image recognition solution. I have a feeling my company's clients will appreciate being able to train their data privately on their own computer.
Teachable Machine is a good way to start! Lobe tries to give you more ways to, not only continuously work on your model by adding more images, changing the number of labels, and even re-label a bunch of your images while training a custom machine learning model in the background—but also, it gives you the ability to analyze your results in real time, and test your model while giving it feedback, so the loop to make your model better is continuously happening. I love that about Lobe.
One of the things it shows on the main page says "train an apps to count reps" while a lady is doing physical exercises.
This is next to ridiculous. I don't need an app or any assistance in counting my reps. I can do that myself. That's easy.
What I really dream of an app for is app to tell my mistakes in technique/posture for every particular exercise. I don't even mind putting a funny costume or some motion sensors on to make its job easier.
Hey qwerty456127! We’ve actually had users build models like this in the past due to some respiratory tracking they need to get done in order to do their exercises properly, so they can’t concentrate on counting and therefore, an automated system proved useful for them.
On the other comment, yes! The app you are describing sounds really interesting, and it is something that could be build using image classification, you just need the right images and camera setup, though!
I see. Thank you for explaining. The case you describe didn't come into my mind. Nevertheless I doubt counting is important. Some reps less, some reps more, I believe the feeling you always feel when you've done enough. Or you can set a timer - it's not the number that matters, it's the time you spend under the load anyway.
On the other - the way you do an exercise, small details in your posture and the sequence of changes in it - that's what decides if what you do is going to make you more fit/strong, have no effect or just cause pure harm. It's extremely important (at least, in the beginning) to have somebody qualified to watch how you do it and correct you. Many people prefer to train alone though so they need such an app.
No catch! We are first and foremost trying to make this technology accessible to as many people as possible, and we want to grow an ecosystem around it. Business models around that can come later.
Wondering this too. It seems to compete with some Azure AI services so my feeling is that either there's a gotcha, or it won't last long. Hopefully I'm wrong.
No gotcha! We're trying to fill a sweet spot for customers looking for a simple and quick way to get started with machine learning using their PCs or Macs without requiring any need for the cloud.
This is really neat, and to me this is revealing of the true power of machine learning. That you can program a computer not by telling it what to do and how to do it, but what you want it to do and it'll learn to imitate and replicate the behavior even on new inputs, very similar to how you teach people tasks.
That’s exactly the whole thesis behind Lobe, back to the conception phase of the project, this was the main principle we tried to infuse in our tool—how can we make the process of building software more humane—and machine learning is one of the tools to achieve that—with Lobe, we wanted to solve the aspect of building the machine learning, as that wasn’t humane either.
I see MobileNet and ResNet embedded as application resources. Because of the speed of the training, I suspect they are doing feature transfer learning.
While the UI is quite nice (thanks to Mike Matas, I am sure), I don't see a strong advantage to using this on MacOs, when CreateML is available. CreateML doesn't have the simple interface of Lobe, but the UI is quite accessible and gives you access to additional classifies, like sound, text and tabular data. If you need ever more power, you can use TuriCreate if you want to stay in the Apple ecosystem.
The simplicity of the UI is a feature, but also a disadvantage when you start having more than a handful of labels and training images. I totally see how Lobe could be a nice intro into the world of labelling and classification.
The model training time using Lobe is comparable to similar transfer learning tasks using other machine learning frameworks. I did a comparison between Lobe and Turi Create using transfer learning with ResNet and the time were similar. Training using a complete convnet would take much long, unless the Lobe team has made some serious advances.
I'm on the Label page of the app and it's asking for five images, but I don't have any... could you please give a few example sets of images, e.g. drinking/not drinking, holding up # of fingers, etc, so I don't have to create the images myself?
Hey there! I'd recommend using your webcam to quickly add images of things on your desk or in your house :) to get a feel for the magic of machine learning. Press and hold on the shutter button to take a burst of images and move the object around to get variety. Then use the Play tab to try it out and see live results from the model!
This begs the question: why not ship it as an app itself targeted to normal users and let them custom fit it to their needs (unless this is already the case and I am missing something thinking it's targeted to engineers to build their apps with)?
this link here promises a react based web-app sample, but then it says "You need to get you setup so you can build, launch, and play with your app. These instructions are written for macOS, the only system you can develop iOS apps on." It then proceeds to provide instructions only for mac os. How come? Why does one need mac os to run a web application?
Is there any recommendations for any other robust plant/insect/microbe identification ML solutions? I usually post in respective subreddits for identification, would think of a ML solution but never acted on it.
I realise this is a Microsoft one, but roughly how many of these are there now? And are they all just productised frontends for the Python libraries like I think they are?
The impressive part is the ux more than the particular choice of algorithms and frameworks. As you alluded, those are fairly commodity but executing so that an end user can understand and be engaged enough to get where they want is a big accomplishment.
I will be very curious to see how this makes it into production applications. It demos extremely well, but that doesn’t necessarily translate into something that’s production ready.
i am really looking for an ai service which is able to detect signatures and threads out of email messages and extract the „real new message“ part - does anyone know some tool?
I don't think AI is needed here -- I wrote an Outlook plugin about 15 years ago that used fuzzy diff between messages in the thread that extracted only the new information added by each person and presented it as a message digest or as a labeled people digraph with msg-bubbles. For the life of me I don't understand why this isn't built in to all email clients -- the way gmail especially quotes replied emails, I need to spend time hunting up and down the message trying to recreate the timeline -- even getting the most recent bit seems harder than it should be.
Do you use Outlook? -- If there is interest, I can try and resurrect it. Although it's not as necessary as it once was -- not as many "Re: re: FW: re: fw: hello!" messages now that people use Slack and Teams, etc.
yes i am using outlook, would be awesome to see some approach here.
I thought of AI as there are so many different ways of mail thread formartings these days.
Hey Markus from Lobe here :) all images and labels stay private to your computer, we don't ever see any of it. We only collect some generic app usage data for telemetry if you opt-in to sharing analytics after installing Lobe.
We do ping online to check if an app update is available! It checks the current app version and our hosted app version to see if we should show a notification that an update is available and a link to our website download.
We do not send any app analytics when it is turned off.
You can read our privacy policy, but more than that, you could use any sort of network traffic visualizer to see that we are not lying we are not in the business of selling data, we are in the business of making machine learning accessible to everyone.
Where is the app-specific privacy policy? the link the footer links to some general "Microsoft privacy policy", which covers all kinds of things and if it has anything specific to this app, it's impossible to find.
Not yet! We are starting in beta with image classification. We are working on adding more project templates in the future, and Lobe is designed upon the idea that machine learning should be made easy — no matter the problem type you are facing.
Hmm seems the title of this post was changed and I cannot edit it anymore, can someone change it to 'for training' or 'to train'. In its current form it's incorrect and sounds like some sort of locomotive AI.
