152 - How to visualize convolutional filter outputs in your deep learning model?

[@fadilyassin4597]

thank you I urge any one who want to understand machine learning to see all the videos because Mr seerni is covering the process in a way that it is very essential to any one who wants to understand ML just look at the toics and see what I mean

[@MaralSheikhzadeh]

I am learning so much from you in the two days I found your channel! Thank you:)

[@PUBUDUCG]

This is very valuable when model debugging is necessary . . . . !

[@rhezapaleva7169]

Thank you very much, this is really help me to visualize the process of the cnn layer

[@DigitalSreeni]

Glad it helped!

[@Ahmetkumas]

Thank you it's a very useful video for who wants to dig into more. Can you make more and deeper videos for different problems on semantic segmentation on U-net or etc

[@DigitalSreeni]

Sure thing!

[@ausialfrai3335]

Thank you Sreeni for the great video, it came out at the right time :)

[@DigitalSreeni]

My pleasure 😊

[@RizwanAli-jy9ub]

every video is a gem

[@DigitalSreeni]

Thanks :)

[@VivekKumar-zw1wx]

Sir there are 128 filter in the 3rd convolution layer then why it is showing the output for only 64 filters...

[@juanodonnell]

You say that by stating the conv_layer_index (1,3,6) command you will be working with the 6th-ish first layers, but the model that gets called is a totally different one, actually the 3 conv2d counting from the end, thats why the shape is different than the prev examples. It is a confusing example.

[@qusayhamad7243]

Thank you Sir it was too helpful and clear

[@seaniam]

I never thought about simply cutting the model and predicting on that......seems obvious now. thanks

[@DigitalSreeni]

You bet!

[@VivekKumar-zw1wx]

@Sean their is 128 filters in 3rd convolution layer then why it is showing the output only for 64 filters

[@seaniam]

​@@VivekKumar-zw1wx as Sreeni chose to only show 64 images in the plot lines 77 + 78 -> number of rows and columns are defined (8,8) line 79 -> n_filters = cols * rows # this is used to determine number of loops (8*8 = 64) so n_filters determines how many images to pull from filters - if you want more change that same idea on line 116 -> range(1,columns*rows +1)

[@ahmedalaa4699]

Thank you very much for your amazing tutorial

[@iangleeson3338]

Thanks. What happens if I’m just predicting something like age? How can I apply this?

[@spider853]

it's the same process as described in this video, you'll probably get filters that extract the wrinkless and so on

[@azaralizadeh3492]

Amazing, amazing video. thank you so much

[@channelforstream6196]

Thanks was looking for just the thing

[@samarafroz9852]

Amazing tutorial sir I'm learning so much from you. Can you please make tutorial on Capsule network for image analysis.

[@DigitalSreeni]

Capsule networks are very new and they only work reliably on simple datasets such as MNIST. It is evolving fast so let us see if they turn into something useful. Thanks for the recommendation though!!

[@samarafroz9852]

Welcome sir

[@armankarimipy]

Thank you it's a veryyyyyyyy useful video for me!

[@omrahulpandey]

Sir, how do we do this with the UNet from your earlier video.....I used this code...but it ain't working with this integration..... at the console it shouts..." File "F:\UNet1.py", line 137, in filters, biases = model.layers[1].get_weights() ValueError: not enough values to unpack (expected 2, got 0)" I placed this code just after the model.summary() line......Let me paste the entire code here for your convenience...(leaving the resizing part) #Build the model inputs = tf.keras.layers.Input((IMG_HEIGHT, IMG_WIDTH, IMG_CHANNELS)) s = tf.keras.layers.Lambda(lambda x: x / 255)(inputs) #Contraction path c1 = tf.keras.layers.Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(s) c1 = tf.keras.layers.Dropout(0.1)(c1) c1 = tf.keras.layers.Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c1) p1 = tf.keras.layers.MaxPooling2D((2, 2))(c1) c2 = tf.keras.layers.Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(p1) c2 = tf.keras.layers.Dropout(0.1)(c2) c2 = tf.keras.layers.Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c2) p2 = tf.keras.layers.MaxPooling2D((2, 2))(c2) c3 = tf.keras.layers.Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(p2) c3 = tf.keras.layers.Dropout(0.2)(c3) c3 = tf.keras.layers.Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c3) p3 = tf.keras.layers.MaxPooling2D((2, 2))(c3) c4 = tf.keras.layers.Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(p3) c4 = tf.keras.layers.Dropout(0.2)(c4) c4 = tf.keras.layers.Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c4) p4 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(c4) c5 = tf.keras.layers.Conv2D(256, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(p4) c5 = tf.keras.layers.Dropout(0.3)(c5) c5 = tf.keras.layers.Conv2D(256, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c5) #Expansive path u6 = tf.keras.layers.Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same')(c5) u6 = tf.keras.layers.concatenate([u6, c4]) c6 = tf.keras.layers.Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(u6) c6 = tf.keras.layers.Dropout(0.2)(c6) c6 = tf.keras.layers.Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c6) u7 = tf.keras.layers.Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same')(c6) u7 = tf.keras.layers.concatenate([u7, c3]) c7 = tf.keras.layers.Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(u7) c7 = tf.keras.layers.Dropout(0.2)(c7) c7 = tf.keras.layers.Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c7) u8 = tf.keras.layers.Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(c7) u8 = tf.keras.layers.concatenate([u8, c2]) c8 = tf.keras.layers.Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(u8) c8 = tf.keras.layers.Dropout(0.1)(c8) c8 = tf.keras.layers.Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c8) u9 = tf.keras.layers.Conv2DTranspose(16, (2, 2), strides=(2, 2), padding='same')(c8) u9 = tf.keras.layers.concatenate([u9, c1], axis=3) c9 = tf.keras.layers.Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(u9) c9 = tf.keras.layers.Dropout(0.1)(c9) c9 = tf.keras.layers.Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c9) outputs = tf.keras.layers.Conv2D(1, (1, 1), activation='sigmoid')(c9) model = tf.keras.Model(inputs=[inputs], outputs=[outputs]) model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy',tf.keras.metrics.MeanIoU(num_classes=2)]) model.summary() ############################################################################ #Understand the filters in the model #Let us pick the first hidden layer as the layer of interest. layer = model.layers #Conv layers at 1, 3, 6, 8, 11, 13, 15 filters, biases = model.layers[1].get_weights() print(layer[1].name, filters.shape) # plot filters fig1=plt.figure(figsize=(8, 12)) columns = 8 rows = 8 n_filters = columns * rows for i in range(1, n_filters +1): f = filters[:, :, :, i-1] fig1 =plt.subplot(rows, columns, i) fig1.set_xticks([]) #Turn off axis fig1.set_yticks([]) plt.imshow(f[:, :, 0], cmap='gray') #Show only the filters from 0th channel (R) #ix += 1 plt.show() #### Now plot filter outputs #Define a new truncated model to only include the conv layers of interest #conv_layer_index = [1, 3, 6, 8, 11, 13, 15] conv_layer_index = [1, 3, 6] #TO define a shorter model outputs = [model.layers[i].output for i in conv_layer_index] model_short = model(inputs=model.inputs, outputs=outputs) print(model_short.summary()) #Input shape to the model is 224 x 224. SO resize input image to this shape. from keras.preprocessing.image import load_img, img_to_array img = load_img('monalisa.jpg', target_size=(224, 224)) #VGG user 224 as input # convert the image to an array img = img_to_array(img) # expand dimensions to match the shape of model input img = np.expand_dims(img, axis=0) # Generate feature output by predicting on the input image feature_output = model_short.predict(img) columns = 8 rows = 8 for ftr in feature_output: #pos = 1 fig=plt.figure(figsize=(12, 12)) for i in range(1, columns*rows +1): fig =plt.subplot(rows, columns, i) fig.set_xticks([]) #Turn off axis fig.set_yticks([]) plt.imshow(ftr[0, :, :, i-1], cmap='gray') #pos += 1 plt.show() ############################################################################ #Modelcheckpoint checkpointer = tf.keras.callbacks.ModelCheckpoint('model_for_nuclei.h5', verbose=1, save_best_only=True) callbacks = [ tf.keras.callbacks.EarlyStopping(patience=2, monitor='val_loss'), tf.keras.callbacks.TensorBoard(log_dir='logs',histogram_freq=1)] results = model.fit(X_train, Y_train, validation_split=0.1, batch_size=16, epochs=25, callbacks=callbacks) #################################### idx = random.randint(0, len(X_train)) preds_train = model.predict(X_train[:int(X_train.shape[0]*0.9)], verbose=1) preds_val = model.predict(X_train[int(X_train.shape[0]*0.9):], verbose=1) preds_test = model.predict(X_test, verbose=1) preds_train_t = (preds_train > 0.5).astype(np.uint8) preds_val_t = (preds_val > 0.5).astype(np.uint8) preds_test_t = (preds_test > 0.5).astype(np.uint8) ###################################################### # Perform a sanity check on some random training samples ix = random.randint(0, len(preds_train_t)) imshow(X_train[ix]) plt.show() imshow(np.squeeze(Y_train[ix])) plt.show() imshow(np.squeeze(preds_train_t[ix])) plt.show() # Perform a sanity check on some random validation samples ix = random.randint(0, len(preds_val_t)) imshow(X_train[int(X_train.shape[0]*0.9):][ix]) plt.show() imshow(np.squeeze(Y_train[int(Y_train.shape[0]*0.9):][ix])) plt.show() imshow(np.squeeze(preds_val_t[ix])) plt.show() Sir can you please tell me where is the problem...

[@DigitalSreeni]

U-net is not a straight forward constitutional layers, it has down scaling and up scaling path where outputs are concatenated. So you cannot easily visualize outputs as images. You may try visualizing them during the down scaling path.

[@abderrahmaneherbadji5478]

Hi Sreeni Many Thanks for this video

[@DigitalSreeni]

My pleasure 😊

[@nirajgautam403]

how to print all filter

[@kemmounramzy6232]

Thank you it's a very useful video

[@nirajgautam403]

i have 5*5 filter but it print 3*3 filter

[@raflyhidayat9687]

what name this file in github bro?

[@guardrepresenter5099]

Sir, I copied this code from your GitHub page. But I give an error as follows. tensorflow.python.framework.errors_impl.InvalidArgumentError: Negative dimension size caused by subtracting 2 from 1 for '{{node max_pooling2d_2/MaxPool}} = MaxPool[T=DT_FLOAT, data_format="NHWC", explicit_paddings=[], ksize=[1, 2, 2, 1], padding="VALID", strides=[1, 2, 2, 1]](Placeholder)' with input shapes: [?,1,1,256]. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:/Users/Bilen-Intel/PycharmProjects/TenSorFlow/ConvulationLayersVision.py", line 36, in model.add(MaxPooling2D((2, 2), strides=(2, 2))) File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\training\tracking\base.py", line 517, in _method_wrapper result = method(self, *args, **kwargs) File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\engine\sequential.py", line 223, in add output_tensor = layer(self.outputs[0]) File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 951, in __call__ return self._functional_construction_call(inputs, args, kwargs, File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1090, in _functional_construction_call outputs = self._keras_tensor_symbolic_call( File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 822, in _keras_tensor_symbolic_call return self._infer_output_signature(inputs, args, kwargs, input_masks) File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 863, in _infer_output_signature outputs = call_fn(inputs, *args, **kwargs) File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\layers\pooling.py", line 295, in call outputs = self.pool_function( File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper return target(*args, **kwargs) File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\ops n_ops.py", line 4606, in max_pool return gen_nn_ops.max_pool( File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\ops\gen_nn_ops.py", line 5326, in max_pool _, _, _op, _outputs = _op_def_library._apply_op_helper( File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 748, in _apply_op_helper op = g._create_op_internal(op_type_name, inputs, dtypes=None, File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\framework\func_graph.py", line 590, in _create_op_internal return super(FuncGraph, self)._create_op_internal( # pylint: disable=protected-access File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\framework\ops.py", line 3528, in _create_op_internal ret = Operation( File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\framework\ops.py", line 2015, in __init__ self._c_op = _create_c_op(self._graph, node_def, inputs, File "C:\Users\Bilen-Intel\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\framework\ops.py", line 1856, in _create_c_op raise ValueError(str(e)) ValueError: Negative dimension size caused by subtracting 2 from 1 for '{{node max_pooling2d_2/MaxPool}} = MaxPool[T=DT_FLOAT, data_format="NHWC", explicit_paddings=[], ksize=[1, 2, 2, 1], padding="VALID", strides=[1, 2, 2, 1]](Placeholder)' with input shapes: [?,1,1,256]. Process finished with exit code 1

[@christopherhimmel9942]

I am also getting this error. Any solution?

[@hachimbani2674]

Hello, First of all thank you for your videos they helped me so well to improve in this field. I am trying to visualize Receptive fields for a given unit in the pretrained AlexNet model in pytorch. To do that I am using the occlusion method used in the ICLR'15 paper "Object Detectors Emerge in Deep Scene CNNs". So basically I need to get top K images with the highest activation for the given unit from a dataset. I tried to do that by comparing feature maps but it doesn't give good results. My question is, if we give an image to the model, how can we get the activation of each unit ? Thank you again for your help.

[@alikasim7658]

I am facing the same issue in my work, in fact, i am trying to extract feature maps from each conv layer and do some computations on them before feeding them to the next conv layer ( this must done on fly during training not after model trained). I fed up searching how to do that extraction coz KerasTensor (which is the type of conv layer) is not changeable to numpy array. @DigitalSreeni, could you please help in this issue

[@rezadarooei248]

is it possible to use this in unet?

[@DigitalSreeni]

Yes.

[@wesuli]

Thank you for the helpful video ..

[@brunospfc8511]

05/07/22 journey continues

[@callmebiz]

what IDE is this?

[@DigitalSreeni]

Spyder. Comes with Anaconda.

[@shashireddy7371]

Thank you Sreeni.

[@DigitalSreeni]

You are welcome.

[@salilmarathponmadom7255]

Nice Tutorial

[@coemgeincraobhach236]

Thanks so much for this!

[@junaidiqbal5018]

Nicely done

[@DigitalSreeni]

Thank you! Cheers!

[@geogob]

thank you for your videos

[@DigitalSreeni]

My pleasure!

[@duoyizhang4665]

Thank you so much!

[@DigitalSreeni]

You're welcome!

[@omkarpatil9234]

Sir , if you think about it, your voice is just like Jordan Peterson

[@ManishKumar-bb5ql]

u r a jam👌

[@82shadowheart]

Thanks! )

[@caiyu538]

thumb it up

[@qusayhamad7243]

Thank you Sir it was too helpful and clear

[@qusayhamad7243]

Thank you Sir it was too helpful and clear

[@qusayhamad7243]

Thank you Sir it was too helpful and clear

[@qusayhamad7243]

Thank you Sir it was too helpful and clear