you're right, in 30:30 I said "the maximum along the column dimension", actually it takes the maximum along the row dimension.

I'm so glad that you liked the video. Thanks for suggesting these two papers. I'll definitely look into those. The thing is I'm recording two videos on an entirely different topic. It might take me a while before getting back to vision transformers.

Thanks for your suggestion. I'll add it to my to do list. The thing is I'm recording two videos on an entirely different topic.

Usually, an image classification model is used at the beginning of object detection pipeline, and it’s called backbone. Most object detection pipelines use ResNet as backbone. I assume that you want to replace ResNet with Swin, just like what they did in the paper (section 4.2). If that’s the case, your best option is to use MMDetection library. They already included Swin as a backbone on their github: github.com/open-mmlab/mmdetection/blob/cfd5d3a985b0249de009b67d04f37263e11cdf3d/mmdet/models/backbones/swin.py

The settings I used in the video were simple just to have a taste of this transformer. In my opinion, a proper experiment would be replicating the results in the paper on ImageNet-1K dataset (the ones in table 1). This way we can judge the model, and then look for improvement.

@@mashaan14 im sorry. which table? (Also, big thanks your videos and replies have been a big help. however, any chance I can ask somewhere more convenient than yt comments?)

Table 1 on page 6 of swin transformer paper. You can text me on twitter or linkedin, whichever is convenient to you. twitter.com/mashaan_14 linkedin.com/in/mashaan

@@mashaan14 Okay thanks

The defaults were patch size=4 and window size=7. You can find all default settings in: github.com/microsoft/Swin-Transformer/blob/main/config.py You’re right, there are no positional encoding in this model. The model uses (Relative position bias) and they said, “We observe significant improvements over counterparts without this bias term or that use absolute position embedding”. They added the (Relative position bias) inside WindowAttention class: github.com/microsoft/Swin-Transformer/blob/f82860bfb5225915aca09c3227159ee9e1df874d/models/swin_transformer.py#L101

here you go: github.com/mashaan14/KZbin-channel/blob/main/notebooks/2024_08_19_swin_transformer.ipynb

@@mashaan14 thankssss

Thank you, that's great to hear. I'm currently working on a Swin transformer visualization video. Hopefully, I'll post it in a week or so.

Glad it was helpful!

Thanks for your question. It made me go and dig a little bit. I read a couple of papers and I found out that GCN can’t be used alone to find influential nodes. However, it can be used as an introductory step to find influential nodes. For example, in a paper named: “Finding Critical Users in Social Communities via Graph Convolutions” the authors used GCN before graph attention to learn “criticalness” over nodes. By criticalness, they mean influential nodes. Another paper called: “SocialGCN: An Efficient Graph Convolutional Network based Model for Social Recommendation” arxiv.org/pdf/1811.02815 They built a recommender system based on GCN embeddings not the feature matrix. They stated that “the proposed SocialGCN model is flexible when the user and item attributes are not available”, which makes sense! because if a node does not have features, GCN will assign embeddings similar to its neighbors.

Thanks for your suggestion. I skimmed though the code on github. I think the main contribution is class Ensemble(), which you can find in (models.py). That class concatenates the outputs from GCN and FFNN. I’ll try to fit the paper in one of my upcoming videos.

That's the beauty of coding, it can be done in different ways..

I loved your explanation, yeah it totally makes sense. But it was far simpler than that, I was training on two different feature matrices. If you notice in the jraph part, I passed this command: nodes=jnp.eye(data_Cora.x.shape[0]) I was training jraph on the identity matrix while training pyg on the feature matrix. I know it’s crazy how jraph got so close with only the identity matrix. Anyways, I couldn’t fix the notebook in the video because it was written in haiku. So I took it down and write a new one with JAX/Flax: github.com/mashaan14/KZbin-channel/blob/main/notebooks/2024_03_21_jraph_GCN.ipynb I’d love if you can take a look at the new code.

Glad it was helpful!

Thanks, I just checked VMamba on github. Sure I'll add it to my todo list. The thing is I'm recording a series on graph neural networks. Once I'm done with that, I'll get back to vision transformers.

actually I'm not aware of any tool that can convert pytorch to jax.

@@mashaan14 can you give me some idea about what can I change to run PyTorch to jax

@@oneplus383 Instead of torch tensors you have to use jax.numpy arrays. For each pytorch layer, check its equivalent in jax documentation. Start with something small and build your way up. Here's how to code a simple NN in jax kzbin.info/www/bejne/fX-vgJRqp86sqZo

Thansk mashaan brother

You live in United States?

happy to help..

I guess you want maps similar to the ones in this paper: Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate, 2014 If that's the case, I'm sorry I'm not familiar with that topic.

thanks Vivek.. great to reconnect with you. It’s been a long time!!

Glad you liked it

I’m installing manim into google colab using the command: !pip install manim You can check the code I used to make this video: github.com/mashaan14/manim/blob/main/manim_visualizeSoftmax.ipynb

@@mashaan14 Thank you. Last time, I don't know why but when i used its not working properly. Let me check again.

Thanks for bringing it up. You're right. It should be square brackets like this: n_samples=[int(self.size*0.6), int(self.size*0.2)]

A Tupel is needed -> round brackets, then all works fine.

I just checked it on scikit-learn . It does need a tuple as an input. That’s strange!! when I run it on colab, it didn’t throw an error. Anyways, I’ll fix it on github.

وعليكم السلام، To be honest I just heard about Jraph library from you. I’m trying to learn Jax, but I haven’t made any tutorials using Jax yet. Anyway, to learn Jraph, I suggest starting with something that you already know the outcome, for example a graph with 10 nodes. I’ll make sure to put Jraph on my todo list ✅.

@@mashaan14 brother I appreciate your reply. Can you suggest me steps or things to study to get into the Graph neural network in short terms.

@@oneplus383I think Stanford CS224W is good way to start: kzbin.info/www/bejne/gHKlkKOin5elmKM

thank you 🙏

thank you 🙏

You can start playing with GPUs in google colab. But they have limitations on overusing GPUs.

I think you need to define some similarities between image pixels. For example, color and position similarities. Those similarities will serve as graph edges. Here’s a good reference on modeling image data as graphs, it summarizes a decade long of research: Arbelaez, P., Maire, M., Fowlkes, C., & Malik, J. (2010). Contour detection and hierarchical image segmentation. IEEE transactions on pattern analysis and machine intelligence, 33(5), 898-916.

Thank you..helpful video. Can i use GCN for some biological dataset? How it give results,pls suggest and help

@@darpan2648 GCN can be used wherever you got a feature matrix X and an adjacency matrix A. For example, one of the most used datasets in GCN is Cora dataset, which contains a set of documents. The words inside the documents represent the feature matrix X. The citation links between the documents represent the adjacency matrix A. So, before running GCN on the biological dataset, you need to identify what are the feature and adjacency matrices.

So glad it helped you. Thanks for your suggestion, I saw several papers using GCN for spatial temporal data. Hopefully, I can write a tutorial on one of these.

Glad it was helpful! Actually, I haven’t come across a GCN for large scale applications. But there’s an interesting algorithmic change to GCN. The authors claim it runs faster with the same accuracy. paper: proceedings.mlr.press/v97/wu19e/wu19e.pdf code: github.com/Tiiiger/SGC