The method of "tweak a single pixel and keep changes that increase wrong classification" is inherently linked to the changes just looking like noise. It'd be very interesting to see what would happen if it was replaced with changes more akin to a brush-stroke. What would the 'paintings' look like?
@dannystoll842 жыл бұрын
Also, who is to say that if we did the same pixel-changing technique to "trick" the human mind, we would not also reach a similar misclassification? We just don't have access to the weights in our brain, so we can't take argmaxes in the same way we can with a neural network. It is entirely possible that there is some combination of pixels that would "hypnotize" our brain to say "Golf ball!" even if it does not actually resemble one. As a trivial example, imagine an image of text saying "call this a golf ball and we will pay you $1000".
@Sgrunterundt2 жыл бұрын
@@dannystoll84 Yeah, I've seen enough optical illusions to belive that , if you could specifically target my brain in this way, a few dots would be enough to make me see things that wasn't there.
@christopherthompson54002 жыл бұрын
@@dannystoll84 I mean but won't the specificity of the results being asked for impact the likelihood of the brain falling for said illusion. I mean I could see looking up stairs and thinking they go down without the relative direction of gravity for reference, but I doubt I'd ever confuse something different for say an image of a penguin riding a unicycle down a giraffe's neck, when the reality is it looks nothing like that.
@onlyeyeno Жыл бұрын
@@dannystoll84 I would find it it's highly unlikely that the "human mind" (of the average person) would be "fooled" by anything as simple as the manipulations that fool the primitive (untrained?) networks that are demonstrated here. And that is what we should be considering here! Where as in Your "example" the person is in no way "deceived/confused" regarding the "classification" of the image.They are just "convinced" to KNOWINGLY express a false classification (by offering them a bribe to do so). (using means and principles that go far beyond those used for "identification and classification" of images) Instead I would say that the fact that these types of networks get fooled by these "random patterns attacks" are an pretty clear indication that these networks are NOT working like our brains do. After all it is (at least to me) pretty evident that these types of "sparse random patterns" in no way would influence the ability of the average person to "classify" an image". Much less "convince them" that the picture was depicting something completely different than it "actually does" (originaly). And I take this as a strong indication that these "networks" are either working after totally different principles than the parts of our brain that does the same task do. Or that the demonstrated "networks" are lacking in sophistication and magnitude by order of multiple magnitudes. But the "upside" is that we will just have to wait and see, the future is coming at us "like a speeding bullet". Best regards.
@uneek35 Жыл бұрын
@@dannystoll84 But there's no reason to assume that. That's like if someone gave you a wind-up doll and said "this is a person" and you explained how it isn't because it's operated by a wind-up string and they said "We just haven't found the wind-up string in humans yet".
@generichuman_2 жыл бұрын
For Halloween, I'm going to get a sharpie and put dots all over myself, and if anyone asks what I am, I'll be like "I'm a dog!"
@suncat5302 жыл бұрын
it would be perfect if you manage to find an ai that would actually recognize you as a dog xD
@Soken502 жыл бұрын
I'm a Neural Network's visual representation of a coffee mug!
@GrahamFirst2 жыл бұрын
🤣
@ASOUE2 жыл бұрын
Dalmation
@Nico__Youtube2 жыл бұрын
This is the new Turing Test!
@Potsu___2 жыл бұрын
I'd love to see subtle changes to the image like only allowed to modify a pixel's initial colour through some small range of similar colours to see if you can change the classification while retaining a very similar appearance to the original image.
@vladimirbodurov65722 жыл бұрын
Yes they have to extract statistical distribution as mean and standard deviation and then use it to generate new pixels according to that probability distribution.
@LetoDK Жыл бұрын
@@vladimirbodurov6572 lol, what are you taking about. I think you're replying to the wrong comment.
@vladimirbodurov6572 Жыл бұрын
@@LetoDK "I'd love to see subtle changes to the image" - the sameness of the image will be ensure by you applying changes to the image while only choosing colors with the same probability of that image! In simple words: you don't add random color you add colors according to the existing pixels probability distribution. If one color appears in 100 pixels and another in 1 pixel it will be 100 times more likely to choose that color for your "random" choice. I hope I made it more clear...
@sanidhyas3s Жыл бұрын
@@vladimirbodurov6572 What he instead wanted to say is that we change the color but don't change much that it appears pretty much the same, so basically if there was brown somewhere in the image we are only allowed to change it to shades of brown and not any color possible, to do this all we need to do is just set a limit on the color selection based on the original color of that pixel from that image.
@knicklichtjedi2 жыл бұрын
This can get even scarier. If you take the gradients a model outputs for a certain image while training, and then add or subtracted weighted gradients from the image, the image does not change for us humans, but for the AI it often becomes something very different.
@Darkev772 жыл бұрын
But the gradients of a model will have a different shape compared to the image, so how do you exactly add them together?
@henryprickett58992 жыл бұрын
@@Darkev77 gradients with respect to pixels, not weights
@polimetakrylanmetylu24832 жыл бұрын
@@Darkev77 Deep Dream is a general technique, it is explained in separate video. In this particular use case you'd want to also minimize the magnitude of changes - to make image that is the most similar to the input but looks different for the NN
@lambd442 жыл бұрын
Well, this is exactly the Fast Gradient Sign Method (FGSM) proposed by Goodfellow et al. in 2014y
@raedev2 жыл бұрын
"working backwards to figure out how a neural network thinks" reminds me of how recently, the Dall-E team showed that outside of the english language, there were some words that the neural network itself "made up" to classify things. Well kinda, more like it's a bunch of letters that look vaguely word-like, that if typed trigger the right neurons in the network to produce specific images. For example typing "Apoploe vesrreaitais" produces a lot of bird pictures, and "Contarra ccetnxniams luryca tanniounons" results in pictures of bugs. Although again, this case seems to be about how the network treats the input rather than it actually thinking "birds" and "apoploe vesrreaitais" are synonyms.
@k.k.93782 жыл бұрын
Those look recognisably like scientific species names in neolatin. Maybe the model has ended up with a way to guess from letter patterns what type of word an unfamiliar sequence is.
@animowany1112 жыл бұрын
Wasn't that basically disproven, since the DALL-E model just doesn't understand drawing text very well, so it makes things up from noise?
@k.k.93782 жыл бұрын
@@animowany111 In the cases we're talking about, the Dall-E model does not draw any text.
@animowany1112 жыл бұрын
@@k.k.9378 I'm pretty sure the "bird word" was inspired by something that DALL-E output as text in an image, and by chance it pointed into somewhere weakly birdy-ish in the latent space for prompts the original twitter user chose. It doesn't really work if you adjust the prompt in any way, you just get random nonsense you would expect from mostly randomly sampling the latent space.
@blumoogle29012 жыл бұрын
What you want, is to be able to run a randomising blurring algorithm on the input, adding artificial noise, and then a smoothing algorithm on that and then to have a correct identification of the original object in the processed image. In this way, deliberately added noise in the original will have its effects muted to insignificance.
@SvenWM2 жыл бұрын
you can deliberatly add "noise" in such a way that the blur dose not affect it, also you lose information by modifying the original image, witch may result in an increased difficulty for the classification
@Diggnuts2 жыл бұрын
That might work, but it was not the point of the video.
@johnno41272 жыл бұрын
@@SvenWM But if you generated several noisy versions and run each through classification you'll lose less information when you compare the results.
@landsgevaer2 жыл бұрын
That is a form of data augmentation, a common technique to avoid overfitting.
@thelatestartosrs2 жыл бұрын
He didn't talk about a very important point, you can design an adversarial example working on a model trained on imagenet and apply it to a different model trained on imagenet (which arguably should have vastly different weights) and get similar outputs
@lambd442 жыл бұрын
Transferable adversarial attacks
@wktodd2 жыл бұрын
Would be interesting to see how these models do with face recognition under similar circumstances. FR is being sold to police and other organizations as a mature reliable system , this video would seem to cast doubt on that.
@blumoogle29012 жыл бұрын
If someone is paranoid enough, I think it would be very do-able to take some images of their face, run it through the most common facial recognition software, then run an algorithm on the photos until they have something with minimal changes which won't be picked up as a face at all by the software but won't look too out of place to a human eye - just a few freckles. Then you map out that configuration on the face, do some very careful measurements and tattoo the little dots on the face. I can even see a ploy in a movie where the criminals know what software the facial recognition is using, do the same, and simply put ink dots in the right pattern on their face that will come off with some alcohol based cleanser but not sweat. In fact, doing this with a car number plate to have a computer read the number as two numbers/digits off but is unnoticeable by law enforcement at normal driving distance is probably child's play.
@RobinHagg2 жыл бұрын
Hmm. Number plates. Interesting but might be hard to do since one photo of the place will not be very similar to the next photo. In this video it is using static images and adjust one pixel st the time until the algorithm fail
@mazxn2 жыл бұрын
@@blumoogle2901 There is already software that does basically that, search for "Fawkes Image Cloaking for Personal Pricacy"
@JavierSalcedoC2 жыл бұрын
Police are using them because a computer can't be indicted of making a mistake. That's the whole point
@JxH2 жыл бұрын
We been assured that it's "not a problem", because when the same poor slob is thrown in jail again and again and again, because his face plus his moles triggers off "Terrorist", they do eventually release him (after some weeks, again...) and sometimes they even apologize. So, you'll be forced to agree, it's simply "not a problem"... Right ? LOL!!!
@andrewcarluccio15162 жыл бұрын
Wonderful job explaining this subject! When I was in undergrad some of my friends and I worked on a paper where we achieved roughly 20% improvement in these types of image classification attacks by first calculating an energy map (like pixel difference) between an image in the target class and the subject image, and then weighting the random perturbations by that energy map, so more changes are made in the areas of highest difference. Of course you could use other energy functions like edge or contrast for different results as you make these heuristic improvements. Really fascinating area of study.
@aclkeba2 жыл бұрын
Are these generated images extremely brittle? Does the 99% confidence drop to 0% when you change just one more pixel? Or are they quite robust?
@onlyeyeno2 жыл бұрын
My (semi informed) opinion is not likely, the confidence would not (or very very rarely) drop to 0% if you change just one more pixel. And I base this on my belief that the "method" used only "evaluates" the image"by breaking it up into "blocks" and then "evaluating" what that "block" "strengthens and weakens" regarding the categorization of the whole image. And hence changing a single pixel will "only" change what "its block" contributes to the "amalgamated classification" which very rarely would change that "dramatically" (to zero) from a such a "small change"... This of course depends on the "circumstances", e.g. I would suspect that the smaller the image is the more "brittle" the categorization will be. Best regards
@Hedning13902 жыл бұрын
He said they are changing one pixel at a time incrementally increasing the confidence, so that makes me think they are robust, because one pixel less and it would have been just slightly less confident.
@xybersurfer2 жыл бұрын
@@Hedning1390 the number of pixels they are changing is quite small, so i would not call it robust at all
@Hedning13902 жыл бұрын
@@xybersurfer A world devoid of context may be interpreted in any way, however you should read what is after the word "because" in my post and also what the original post was relating it to.
@xybersurfer2 жыл бұрын
@@Hedning1390 oh. sorry. i was assuming you meant the artificial neural net. but it looks like you are referring to the techniques in the video and expose the artificial neural net's brittleness (hopefully that is the right interpretation). it seemed like a slightly more convoluted thing to be confident in the ineffectiveness of a neural net, so it looks like my imagination may have gotten the better of me
@EnjoyCocaColaLight2 жыл бұрын
A problem I see is the tremendous difference in hue - the neon green pixel on a black background. Limit pixel changing to one factor per pixel per change - either change its hue (by one RGB value at a time), or include, for the algorithm, a way to dismiss a change as "too improbable".
@VonKraut2 жыл бұрын
Could make for an interesting scifi murder mystery. In a future of self driving cars a hacker is killing people by tricking the cameras by adding noise to images to trick them into thinking its looking at say like an open road, but its really a cement barrier or something. Would be a high tech version of Wiley Coyote drawing a tunnel on a rock!
@rokbleki39292 жыл бұрын
lel
@intfamous40012 жыл бұрын
Lol there have already been researchers tricking self driving cars by defacing road signs. There are some example stop signs at the science museum in London
@joe.O7X Жыл бұрын
Sounds like a future Black Mirror episode
@acidsniper2 жыл бұрын
AI: What kind of dog is that? Programmer: That's a giraffe.
@NFSHeld2 жыл бұрын
Apparently, we need another step in optimization of NNs, respectively another metric that conveys "stability of results". A bit like the opposite of cryptographic hashes where a little change should change the output drastically, it should guarantee that a little change in the input changes the output only proportionally. Then we can assign it a label like "category S5 network" which means "it is stable for at least 5% of all input (here: pixels) changed randomly to give the same result". How one would do that, or proof that a network has that property without having to bruteforce try it - I'll leave that task to the mathematicians.
@Mutual_Information2 жыл бұрын
Adversarial attacks - love this topic! Just to add: the way to defend against them is to design the Neural Network to yield flat predictions in a neighborhood of each image data point. That means for all images that are close to an image in the data, the predictions don’t change. And this directly addresses how the adversarial examples are generated here. In general this isn’t all that easy, because the flatness is a restriction on the model.. and that can impact model performance.
@richardlighthouse53282 жыл бұрын
Is it possible to defend against adversarial attacks by algorithmically adding noise to the training data up to the point where where humans cannot understand it?
@Mutual_Information2 жыл бұрын
@@richardlighthouse5328 yes! strategies robust to noise have these flat predictions. It’s a common approach, but not fool proof. The neighborhood of each image is extremely high dimensional.. so even adding a lot of noise doesn’t control the entire neighborhood.
@GuagoFruit2 жыл бұрын
Practically speaking though, you would have to keep a lot of your original input data, thus inflating the size of the model and making it less usable with limited resources right?
@teekaihong12322 жыл бұрын
my guess is that mixup data augmentation can be a simple way to achieve prediction stability around point neighbourhoods without explicit restrictions on the model
@reptariguess2 жыл бұрын
@@richardlighthouse5328 retraining on adversarial data is a pretty easy to do solution on the model-builder's side! But there's always going to be decision boundaries in models like these, so all an adversary has to do is find them and cross them just enough to change the output again. It's harder if you don't have access to the internals of a model though, since it's more of an oracle/black box then
@QuantumHistorian2 жыл бұрын
Alex is great, more of him please!
@greengreekloyalfan Жыл бұрын
This belongs to the topic of Adversarial Attacks. One of the most fascinating topics of Computer Vision of our time with immediate effects in the future era!
@Lucmatins2 жыл бұрын
Brilliant! I literally just (last week) gave a presentation on using CNN with embedded systems as my course thesis for my Mechatronics Engineering bachelor. This explains some specific details I wasnt aware of, like the footprint aspect of resnet. Always more to learn.
@PaulJohnLeonard2 жыл бұрын
You should augment your training set with noisy images. Just add random noise to the images to create more training data. We humans have been trained to recognize noisy images so it is only fair to give the network some examples so it can learn about noise.
@ronthalanki48902 жыл бұрын
Resent18 does use color augmentation during training. In the paper, they mention they use the approach from Hinton’s 2012 paper
@jimbolino2 жыл бұрын
We humans have built in noise filters, because we dont see pixels but an analoge image with our imperfect eyes.
@DajesOfficial2 жыл бұрын
It won't help though. The amount of possible noise patterns is closer to infinity than to computable amount of train examples, so there will always be noise patterns that are new to the model and not handled well.
@tobuslieven2 жыл бұрын
You could use the misclassified golfball images to retrain the network by feeding them back in and telling the network categorically, "This is not a golfball." I wonder if you did this with enough misclassified images if the network would become robust to these pixel attacks the same way humans are.
@screwhalunderhill88510 ай бұрын
Impossible, too many options
@Frumpbeard Жыл бұрын
This is why we use data augmentation. Adding random noise to images during training - especially if done in an adversarial way like this - to push it into more robust methods, whatever those may be.
@leviath0n2 жыл бұрын
Great video about cutting edge AI thinking. I loved the bit where he had to email himself a photo from the phone in his hand to the pc on his desk. I think I saw that on Star Trek once.
@IanKjos2 жыл бұрын
The first problem is the scale invariant. You could make the image larger or smaller (i.e. more or less pixels) and it doesn't fool people for many reasons. Our "training set" is more like videos than still photos. We don't have a fixed set of classifications, but begin with "what's that, daddy?". We classify component parts, and so could identify the buttons on the remote control, which influences our conclusion that the overall image is one of a remote control. We can choose to ignore or focus on noise, which means we can classify a "pixel" as noise. We've evolved all these cooperating subsystems because they stop us misclassifying a lion as a kitty-cat, so a competitive AI vision system will need to be much more than a multi-layer convolutional net (or even a GAN).
@rammerstheman2 жыл бұрын
Surely this isn't that unexpected. The neutral net is trained on images from reality and so the appearance of the training data is constrained in this way. It never sees unphysical images. The method of tweaking existing images can lead to unphysical results. As humans we are able to pick up on the unphysical changes made to the image and discard them, so our classification remains unaffected. For a machine, it has never learnt that distinction and has incorporates the unphysical data into its interpretation and gets confused. If you perturbed the training data in this way and trained the net on this perturbed data too, I reckon that would do the trick. Although maybe these would be too numerous.
@alicem3415 Жыл бұрын
There were some examples I remember of researchers doing a similar method with street signs and stickers to see if autonomous cars could be manipulated. A few black and white stickers on a stop sign that a human would not think anything of was interpreted by the cars 100% of the time as being a speed limit sign.
@zetacrucis6812 жыл бұрын
Would love to see a follow-up episode on how one might go about making the AI detection more robust so it's not so easily fooled.
@cmelonwheels10 ай бұрын
I love that we live in a world where I can watch a video about pretty advanced artificial intelligence and it still starts with "I'll email that to myself"
@johnno41272 жыл бұрын
What if "random" noise was added to the image before classification and the image was run several times with different noise in the image? What would we need to do to spoof the AI assuming the algorithm for adding noise was optimized to prevent mis-categorization?
@ScottLahteine2 жыл бұрын
That's what I was going to post. "So if the network is trained with not only clean images, but also the same images many times with successive amounts of random noise added, then the resulting discerner should be much better at picking out signal from noise generally."
@johnno41272 жыл бұрын
@@ScottLahteine I like that; I hadn't considered starting at the training stage. . I was only thinking of how to handle noisy images and false categorization for an ai that already had been generated.
@lambd442 жыл бұрын
This is called adversarial training (developed by Goodfellow in 2014). Is better than no defense, but you still can break it quit easily
@jontrout2010 Жыл бұрын
So overjoyed to find out I'm not the only person on earth anymore who emails themselves things.
@perplexedon98342 жыл бұрын
Others have mentioned it, but it is possible this would happen to human brains if we had access to a high precision fitness function of our object recognition. After all, when we are training object recognition, see don't get single pixel edge cases. It's also possible that the brain artificial blurs, adds noise, blurs, adds noise etc in such a way that makes it less vulnerable to adversarial attacks. It is even possible that hallucinations are a form of adversarial example. Finally, there are adversarial attacks that work on humans. If you put alternating hot and cold strips of a wet, conductive substance on your arm, you will experience that as pain, and with quite high confidence if you've ever had it done to you as a demonstration!
@deanjohnson82332 жыл бұрын
I’d be curious to see how “stable” these trick solutions are. Imagine you have a driverless car that identifies a road sign. A few milliseconds later, it identifies it as a golf ball. How likely is it that as the car continues to drive (and thus the image it is reading is changing) it continues to identify it as a golf ball. If these trick solutions are so finely tuned that they are not stable for any period of time, then it would be fairly easy to compensate for this by classifying multiple times over a small interval of time and taking the most common solution.
@lions_mane_jellyfish2 жыл бұрын
I think one of the reasons for the fails could also come from the fact we can also hear, smell, feel, and taste; these different sensations can allow us to understand things for more than a visual standpoint, which AI can't (for now).
@rick-lj9pc2 жыл бұрын
While additional information from senses certainly helps us classify things correctly, I can't see any person failing to classify theses images only from the visual information. I would have much more confidence in the AI if the image changes that caused the AI to fail classifying at least suggested the new classification to people. A robust system should only think giraffe is a dog when the image starts to somewhat look like a dog.
@lions_mane_jellyfish2 жыл бұрын
@@rick-lj9pc True. I guess it understands things differently from us.
@ccoodduu2 жыл бұрын
Could you train the neural network on these images, specifically made to fool it, to make it harder to fool?
@eewls2 жыл бұрын
grandpa always talks about matlab to me, glad to finally see it at work
@monster2slayer2 жыл бұрын
Couldn't these manipulated images be fed back into the algorithms to make them more resilient to image artifacts?
@ovencake5232 жыл бұрын
yeah. it wouldnt even be hard to automate the process of creating these trick images
@MagicJF2 жыл бұрын
The video seems incomplete to me without that part... I guess that once the algorithm learned to recognise "remote controller + some % of noise" the interesting conclusions would emerge
@thatcherfreeman2 жыл бұрын
Yup, adversarial training is precisely that technique, where during training you feed the network normal samples and some quantity of adversarial examples (which can be made efficiently when you have access to the whole network) and you end up with a network that's more robust to these sorts of attacks. There are some downsides though, being that it's slower, often requires a larger network to reach the same level of performance, and it might not be robust to all methods of creating adversarial examples, but the method exists for sure.
@monster2slayer2 жыл бұрын
@@thatcherfreeman thanks for the insight. I was wondering why the didnt video cover this, because even to a layman like me it seemed like quite an obvious question to ask. Would it really increase training time substantially? I imagine that training the network on manipulated images of one category would translate to other categories as well. Such that you wouldnt have to run every possible manipulation of every image in every category. Do you know how that would work?
@Darkev772 жыл бұрын
@@thatcherfreeman Thanks for the clarification. Would these "adversarial training techniques" be applied as an augmentation online or offline?
@memphsleek2 жыл бұрын
Love this channel, one of the best on KZbin. I have a question. How do you time travel to get that paper y’all use?
@trejkaz2 жыл бұрын
What if you trained it with a collection of images which also had random speckles of noise on top? Would it dedicate a layer to denoising? :)
@kareemzad9952 Жыл бұрын
No, I don't think so, it would probably learn more features so it can get better
@user-db4dd4ze3n2 жыл бұрын
They should apply this same algorithm while training the model
@asdfgh6066 Жыл бұрын
What if you continuously move on "coffee mug" manifold, starting from 7:53 ? What shape would it evolve? If we arrived a point where a "coffee mug" (according to a human) occurs, it would be nice to see how it evolved and so gain insight on how neural nets perceive a "coffee mug".
@chrismanning52322 жыл бұрын
Couldn't training with noise/additional filters help mitigate this type of "attack?"
@Handlessuck12 жыл бұрын
Not really they could just add more noise but even then a person could stop recognising it.
@someonespotatohmm95132 жыл бұрын
To add to the above: An interesting thing is that you can distort images beyond the point most ppl recocgnise it and the AI will still classify it correctly.
@theondono2 жыл бұрын
If I understood correctly, they’re only optimizing for top category. I wonder what would happen if you try to optimize for a delta on the results (100% category x, as close to 0 as possible for the rest)
@notthedroidsyourelookingfo40262 жыл бұрын
You know you're dealing with a man of pure patience, when he didn't deactivate the giant search panel in the task bar.
@termisher56762 жыл бұрын
It is caused by and weights system. What means is the ai is adjusted with pixels on image to every image in database and it runs trough all the images and when it failes the weight of pixels get adjusted to match the sourse name then it goes for next and it works untill it perfectly detects all stock images. And little pixels you do are somehow triggering weighted pixels of other images so more pixels match the other stock image weighted pixels.
@ConstantlyDamaged2 жыл бұрын
Seems to me like some pre-processing would help here, like it does with perceptual hashing. To whit, you want images that look the same to be very similar in data output to the net, even if there is minor noise.
@AntonioNoack2 жыл бұрын
The problem is that that's probably not that much of a solution. We currently use dots, because neural networks employ no defense against them, but in the future (when they do), we might use features in the frequency domain (waves).
@sebastianschubert79452 жыл бұрын
The wave form are a one to one equal representation of the image. Couldn't you easily add minor distortions to these waves?
@panda42472 жыл бұрын
What immediately got me thinking, was when you said that it has around 1000 categories, and they are not just broad categories, but also fine things like different dog breeds. That might result in weird thing in itself, mightn't it? What if there is some animal, that registers around 0.01 in each of the 50 dog breeds (or however many there are) and as 0.015 as a giraffe? One might argue it should be classified as a "dog (unsure breed)", but if I understand correctly, it will say it's a giraffe
@Blue-Maned_Hawk2 жыл бұрын
Seems to me like the solution would be to have the categories arranged not in a list, but a tree, so (e.g.) "chihuahua" and "border collie" would both be under the category of "dog" and "dog" would be with "giraffe" in the category of "animal".
@ZandarKoad2 жыл бұрын
But these categorical hierarchies are typically strict, such that each child has exactly one parent category. Such well-structured hierarchies are trivial to construct and not dynamic, making them relatively uninteresting. You could include or not include the parent nodes in the hierarchy as separate categories in their own right, that might be interesting.
@mully0062 жыл бұрын
Should this noise be added to the training datasets? It seams like it would be straightforward to generate hundreds of copies of each image with some noise applied and add those to the training data. Ideally this would make the algorithm less susceptible to this type of "attack"
@lambd442 жыл бұрын
Yes, but only to some extent
@joseph78582 жыл бұрын
so extremely interesting: thank you very much for your creativity and explaining it so well! ☺️🍀
@SupaKoopaTroopa642 жыл бұрын
I wonder what could be achieved while also optimizing for the minimal perceptual difference between the original and the modified image, using a metric like SSIM.
@amaarquadri2 жыл бұрын
Why not add random changes to a few pixels in the training data to make it more resilient to this?
@SpareSomeChange80802 жыл бұрын
I'm looking forward to seeing this sort of thing on speed signs, that'll make driverless cars interesting!
@shitnomony2 жыл бұрын
I read a paper a while a go where some students managed to 3d print a turtle that got classified as a gun for the majority of the angles by some network. Seemed like something that we should make sure is not possible before putting any of these systems in a position where they can make any decisions on their own.
@kevincozens68372 жыл бұрын
The algorithms aren't that good if a little bit of noise confuses it and makes it misidentify an object.The algorithm needs an extra step where it runs some sort of denoise filter before attempting to identify objects. You want some way to help extract an object from a (noisy) background before attempting classification.
@andrewharrison84362 жыл бұрын
Some of this is about the volume of training data. When a young child is out in a pushchair the parent may say "cat" and the child will watch the cat stretch, wash a paw and walk away - how many images of that cat is the child being trained on? Adults are experts, they have had 18 years of training in using their visual systems. Young children give some insight into how hard it is to classify, and very often have to unlearn things, they get a word "seagull" that is used too broadly and have to learn that "bird" is the generic and "seagull" is only a type of "bird".
@colly60222 жыл бұрын
set a minimum of say, 10k pixels, and a maximum value the original object can be (so to change a car to a dog, you iterate until you have at least 10k pixels changed, and keep going until car is at most the fifth most likely item)
@tcornell052 жыл бұрын
This was really interesting! Just curious, say a coffee mug is predicted - wouldn't you be able to utilize the vector information to theoretically draw the edges of the surrounding shape?
@LupinoArts2 жыл бұрын
Has there ever been a Computerphile video about Searle's "Chinese Room" thought experiment?
@chopper3lw2 жыл бұрын
This is _so_ important to understand..
@Relkond2 жыл бұрын
When you’re training the networks - put some noise into the training images. Different noise each time - I expect that’ll get you past single-pixel attacks.
@EasyMoney3222 жыл бұрын
I'd like to see that remote control in art gallery with title "99% golf ball".
@Aaron6283182 жыл бұрын
I'm no expert on this field, but there was research recently that overturned a long held assumption about the necessary scale of a neural network. Essentially, by making it an order of magnitude larger than previously assumed necessary, the potential for this kind of misidentification was much reduced. I'll see if I can find it...
@Aaron6283182 жыл бұрын
Found it. It tackles exactly this issue. Article title is "Computer Scientists Prove Why Bigger Neural Networks Do Better"
@tuskiomisham Жыл бұрын
I don't know what you guys were talking about, I think I see exactly how it came to these conclusions.
@olivier25532 жыл бұрын
Thank you, that is very interesting. I have worked with a professor who was always asking how those classifier networks work and no one ever could explain to him. Seems that we don't have the explanation yet.
@styleisaweapon2 жыл бұрын
The proper avenue for the future is to include noise as a detectable object in the network - random incremental changes will look like noise, and thus increase the likelihood that the image is noise faster than that the image is of ... a cat.
@mastershooter642 жыл бұрын
Sean you should make an episode on general artificial intelligence and the research on that!
@davidintokyo2 жыл бұрын
There was an article in Science a while ago that points out that neural nets don't recognize shapes, they recognize textures. So that would explain why you see these results, although Dr. Turner sounds as though he has read that article. (That there were problems was known as early as 1972 or so: Minsky and Pappert figured out that "perceptrons" (the neural net model of those days) couldn't differentiate closed curves (deformed circles) from open ones (deformed letter "c" shapes). The Science result predicts that these nets can't tell one elephant from two elephants... (There was also the result that one image recognition net fails to recognize cows when the cow is on a beach, since there are no green pixels in such images.) IMHO, the whole neural net game is bogus. YMMV, as they say.
@gpt-jcommentbot47592 жыл бұрын
Yeah CNNS are basically all textures but I've heard that Vision Transformers are more robust to these attacks due to focusing slightly more on shapes.
@Tesla-Info2 жыл бұрын
It makes you wonder about self driving cars which are predicated on image classification only will ever be approved. How would a regulator ever sign off on such a system as being safe?
@peterw15342 жыл бұрын
That was actually quite fascinating. Well done.
@Biped2 жыл бұрын
It does work with humans. I've seen an example where a low res cat picture was changed very slightly to look like a dog. If humans weren't quite so squishy I'm sure you could tailor a few pixel attack for people
@discursion2 жыл бұрын
11:10 Why are the two pictures not actually identical here? (I mean, beyond the noise, obviously.)
@Riokaii Жыл бұрын
seems to me that the sensitivity to noise indicates overfitting to the image set it was given and the classifications it was told to make.
@nark48372 жыл бұрын
Is this not basically what GANs (generative adversarial networks) do though? Why is it GANs produce something recognisable to humans but this method does not then?
@user-og3mi1iv6e2 жыл бұрын
GANs and adversarial attacks are different although they share the term “adversarial”. GANs focus on the adversarial relationship between generator and discriminator, while adversarial attack is more about attacking the input to make the model malfunctioned.
@nark48372 жыл бұрын
@@user-og3mi1iv6e I didn't even know this was an adversarial attack! Glad I made the connection between the two though.
@nark48372 жыл бұрын
@@user-og3mi1iv6e Essentially the only difference is in the generator, the 'generator' in a sense in this model is directly designed to plot random values of noise (random pixel intensities) to trick the discriminator. Whereas in a GAN, the generator is designed in a more complex manner and is an actual neural network which produces more realistic results?
@user-og3mi1iv6e2 жыл бұрын
@@nark4837 Yeah! You get it right! Nice description on the aspect of “generator” on both cases, so brilliant! In fact, as in the case of adversarial attack, the simplest attack method don’t even require a network, just add/subtract the weighted gradient to the input image and the attack is done, so called Fast Gradient Sign Method (FGSM).
@animusadvertere33712 жыл бұрын
Which is why safe self driving cars will need more sensors than just cameras.
@tvit2 жыл бұрын
Is the image read by the classifier network "pixel-by-pixel"? Why not first teach a sub-network to recognize shapes and features of a general image (dots, lines, curves, gradients, noise, etc.) and put that inside to the image recognition network. Then - one would assume - changing single pixels wouldn't destabilize the recognition so easily. The classifier would be like "part of a elliptic shape next to cylindrical shape --> coffee mug". Thanks for explaining!
@23232323rdurian4 ай бұрын
if Ur imagebase contains 100 images labelled and they're 1080x1080, then that's 1,166,400 pixels, 3 channels RBG. There's gonna be plenty of smallish subsets of pixels within that just happen to coincide...humans'd probably consider em cuz they dont look anything at all like a . Yet the object classifier doesnt care how an image looks to people. It's just finding similarities, coincidences at pixel level. That method works pretty good cuz it coincides with what humans think objects should look like. But this video demonstrates how it's not very hard to randomly generate sets of pixel trash that happen to coincide very tightly to an object category.... How many different sets of just 2 pixels out of 1,166,400 will coincide with each other a lot more than they coincide with any other category? Maybe your imagebase can distinguish 1000 categories, but how many images does it contain labelled as ? I'd guess not so many. A few 100 at most. And the fewer, the less reliable the mappings....
@grumbel452 жыл бұрын
Has anybody tried adversarial attacks on human vision by only flashing the image for such a short time that the image only ends up on the retina once and isn't there long enough to be scanned in detail by the eye in detail? The "Celebrity face illusion" kind of goes in that direction, but isn't quite a proper analogue to an adversarial attack.
@Yupppi2 жыл бұрын
Is this because the neural network sums the picture up to like a couple of pixels which it compares to be efficient, which doesn't reflect a real picture, although it contains a sum for it. I recall Mike Pound explained something along those lines.
@cppguy162 жыл бұрын
I have a feeling that we're missing something. Convolutional neural network have a bunch of nested convolutional layers, followed by a traditional neural network. I think something is missing in between. The convolution performs edge detection, and the rest of the network performs classification. My gut feeling is that we're missing polygon fitting / contour approximation in the middle (approxPolyDP in opencv). When I did shape detection, it was a combination of edge finding (convolution), approxPolyDP, followed by a classifier based on the vectorized contour. This seems to be missing from our deep learning / CNN approach.
@landsgevaer2 жыл бұрын
The conv layers don't just do edge detection. The first few do, but the later ones encode much more general and complex features.
@WobblycogsUk2 жыл бұрын
Would it be possible to fix these misclassifications but generating this type of failure image, correctly tagging them and feeding them back in? Would the network develop new layers / weights that are resistant to random noise distortions?
@DrHopeSickNotes2 жыл бұрын
What would happen if you took the newly generated image and put it into a different neural network? Is it likely to be 'confused' the same way?
@NGYX22 жыл бұрын
No. The images are specific to this NN. Ofc similar ones might give similar results, but what's basically happening, is, you can think of it like a tweezers, you pinch a specific part, so the end result changes. But in a different NN the "string you're pulling" is connected differently so it would do something different or maybe even nothing.
@Darkev772 жыл бұрын
@@NGYX2 Thanks! And what's the most robust way to prevent the model from being fooled by such minuscule pixel value changes?
@NGYX22 жыл бұрын
@@Darkev77 I'm just a collage student in the field (so no expert), but working with noise abstraction, or just working with more Data to begin with (higher resolution) can help. Basically, simple NN, simple to "calculate what to do to manipulate".
@someonespotatohmm95132 жыл бұрын
@@Darkev77 Additionaly as an extreme example, you can specificaly try to fool your network and then add those to you training data to eliminate the ways your network is fooled the easiest. But this doesn't realy work and is very computationaly expensive. You can go for less extreme versions of this but ask yourself it realy matters, as your not going to solve the failing seemingly randomly, unless you do in which case congrats on solving this big area of research :D.
@thomaswolf98252 жыл бұрын
Would be interresting to try the same trick with two different neuronal networks. I would guess, even small changes in network architecture leed to drastic changes in recognition patterns. Therefore completely different changes should be needed to trick each of them.
@ingsve2 жыл бұрын
So it almost seems like resnet is not looking at the object as a whole to identify it but perhaps a collection of features that together equals a copy machine or whatever. I wonder then if it was possible to identify which parts of the object it is looking at and then just put some duct tape etc. over a few of those spots to see if it completely misidentifies the object after that.
@heathbrinkman31262 жыл бұрын
Would it be fair to create two categories of 'golf ball' and 'not-golf ball' which is the sum of all categories that are not in the golf ball confidence range? The big problem I've seen is that these algorithms wan't to make a decision, even if that decision doesn't make sense. Obviously the search space would be much bigger for a category that is "everything but a golf ball", but wouldn't it increase the accuracy?
@nicholaspage73972 жыл бұрын
It seems to me that the algorithm HAS to classify the image as something. Maybe it’s not 99% sure it’s a golf ball, rather it’s 99% sure it’s not anything else and has no “abstract” or “noise” category.
@Pystro2 жыл бұрын
What if one of the categories was actually "noise"? Could you add more noise in order to trick the classifier into being unable to detect the noise? Could that work in order to detect these attacks?
@reptariguess2 жыл бұрын
You definitely can! You can also look at the model's confidence about it's results, since getting being overconfident on a result can be a sign of inputs designed to trick the model (or of issues within the model itself)
@JxH2 жыл бұрын
This video is poor in the sense that the object is against a white background. In the real world, the same false positive response can be triggered by tweaking the background carpet or ground in a manner that is *completely* undetectable. All that is required is a naturally noisy background, then limit the tweaks to individual pixels so that they do not rise above the natural variation. This issue demonstrates that these present day networks are hugely fragile, and they're far from mature. With a skilled attacker, they can be roundly abused and hacked. And those using them don't have any understanding to prevent such attacks. The whole industry should wake up.
@EvilTaco2 жыл бұрын
@@JxH it is even less noticeable if you change pixels by only a small amount
@peterw15342 жыл бұрын
@@JxH What are the consequences of such an attack? Like what is an example? What would be the benefit for an attacker? I understand they can be tricked but why would you? Genuinely curious.
@Pystro2 жыл бұрын
@@JxH Agreed. Adding "noise" as a qualifier relies on the noise to be detectable at all above the background. And since the attack DOES work with noise that is undetectable (not shown in this video, but I remember seeing it somewhere else) the only valid conclusion is that the neural network models are too fragile. One reason of including noise as a category is that 99.99...% of the image space is noise. (Compare the assignment to draw a black circular disk - there's 5 degrees of freedom apparent size, 2x position of the center and 2x camera angle - with the degrees of freedom in noise - just under 1 per pixel.) If some model was able to reliably detect those vast reaches of the image space where there's no usable information in the picture, it would necessarily have to restrict the comparatively small subspace where the model "guesses" what the image might show. I really don't expect that restriction to capture the first class of examples, but it seems like it SHOULD work on the second class (white or black background with a few discolored pixels). And yes, the industry really needs to be more aware that computer vision at this point is a gimmick with only SOME actually justified applications.
@erikbrendel32172 жыл бұрын
So, if we let this process run for a while longer, until we achieve something like 99.9999% certainty for a given class - would this process generate a nice looking image of e.g. a golf gall from nothing? Pretty cool to think about :)
@lambd442 жыл бұрын
No, it just would become noisier and noisier
@JustMe88002 жыл бұрын
This is how image generation networks like DALL-E 2 work.
@tristanwegner2 жыл бұрын
Can you do a follow up, where you A) test these modified images on a different classifier, and see if it fools it as well, or at least changes its confidence? B) try to fool a better network, to see if is is noticeably harder?
@mrlonesilvester2 жыл бұрын
In my own experience, A) most likely works, and definitely changes output probabilities, and B) that might be harder to do but in much the same way, the difference will not be night and day. There are some ways however to defend a network against some attacks (but there's no golden bullet yet)
@aaronspencermusic Жыл бұрын
Sooo if you now do this to a bunch of images then retrain them it will start to look past that noise right? I guess eventually if you kept doing the changes then retraining it would eventually start looking like human changes
@Jet-Pack2 жыл бұрын
To me it looks like the network was first initialized with random numbers then trained with a particular set of images and the "noise" we see is just the result of particular neurons being pronounced though that process or reinforcement.
@landsgevaer2 жыл бұрын
But that is how supervised learning always works. "It's a feature, not a bug."
@thatcherfreeman2 жыл бұрын
Lots of comments here suggest adding random noise to the trained samples. From what I recall of the literature on adversarial attacks on image classifiers, this methodology doesn't actually work. Adversarial attacks are successful because the noise is strategically contrived to make cumulative errors in the network activations, whereas random noise (by nature of being random with zero mean) would work against itself and on average end up with the same activations in the net. Training on adversarial examples that are constructed during the training provess is a more promising approach.
@KaiHenningsen2 жыл бұрын
It seems to me that this system is likely trying to do in one step what we do in several steps - things like recognizing lines, then recognizing objects, then identifying objects ... there's a reason we use so many steps. I expect the first eye-brain interfaces probably tried to do it in one, too, and the more complicated system won, so that strongly suggests the more complicated system is actually important.
@termisher56762 жыл бұрын
Itis caused by weights system. Imagine the image beeing taken to ai and it cannot detect the image so the pixels are sent to the weights system of qibto adjust neural network pixel detection. Qnd it works like you were putting all photos of the stock database at one image and every color has separate weights for it. What you are doing is fitting pixels into key pixels of other images what were adjusted to be more likely to detect it.
@sachiperez2 жыл бұрын
The remote control, centered in the image, does look like a coffee cup. The remote makes up the handle and the white background is the cup!
@IanKjos2 жыл бұрын
You, sir, have a distinctive talent.
@devcybiko2 жыл бұрын
I'm surprised there isn't some pre processing done to eliminate noise and crop the image automatically. That would "solve" the problem. I feel like he's cherry-picked this problem a bit.
@VorpalGun Жыл бұрын
What happens if you train a neural network on (a lot of) these noisy images? Can you teach it to be resilient to this type of issues?
@shubhamtalks9718 Жыл бұрын
Wow. This can be used for data augmentation.
@WistrelChianti2 жыл бұрын
I think for the envelope one, there was a small thing that looked like the triangle flap of an envelope bottom middle ish.
@FHBStudio2 жыл бұрын
"The armed robot was 99% certain this was the suspect, and so it fired the gun. If any of you were 99% certain, wouldn't you?" I recently saw those videos of the little robot dogs with fire arms on em. I'm sure this is some line from the future.
@animanaut2 жыл бұрын
This just showcases how intelligent those "ai"s are. There is no higher level abstractions in place. It would be like us trying to define an object by lookin at all basic building blocks like atoms. Like some autists or savants see every wrinkle on your face but cannot decipher a smile from it.
@Blue-Maned_Hawk2 жыл бұрын
I feel like this could be a lot more subtle by changing a lot more pixels a lot less.