You Can't Setup Cluster in Azure Or Can You? | Shared Disks

[@matans844]

Thank you for all your help and support!

[@AzureAcademy]

Thank you!

[@echo3347]

Thank you for creating this video. I appreciate the level of detail while also not wasting time. Great help!

[@AzureAcademy]

Anytime! Let me know what other topics you are interested in so I can make more videos ☺️

[@echo3347]

@@AzureAcademy One thing we are running into I was hoping you'd have some insight on. It seems when one node is offline (we turn a server off) the second node disappears. The cluster only functions when both machines are running. Is this expected behavior in a 2 node cluster?

[@AzureAcademy]

That would not be expected behavior. Sounds like the cluster is misconfigured, or the cluster IP is directly tied to that one node. When you powered off that one node...did you successfully fail the cluster over before that?

[@echo3347]

@@AzureAcademy A little clarification of the behavior. It seems if both nodes are taken offline at the same time, both nodes have to be online for the cluster to function again. IE if both are offline and one comes up, the share fails until both nodes are online. Once they are online, either node can go down without affecting the share. Was just curious if this was normal behavior. Appreciate it!

[@AzureAcademy]

Just to be clear...I want to make sure you have tested this correctly. You have 2 nodes. A and B A works fine holding the cluster roll B works fine holding the cluster roll A works fine holding the cluster roll even if B reboots B works fine holding the cluster roll even if A reboots A works if B is shutdown and doesn’t come back B works if A is shutdown and doesn’t come back If all these things are NOT true Then I believe you’re cluster is miss misconfigured in some way. ALL cluster dependencies should be on 1 VM at a time so node A or B can be the only VM running so the cluster can have a node reboot and failover cleanly

[@matans844]

Thanks! This is great and very valuable for me! I have a few questions, as I want to create a similar setup, just with Linux VMs: 1. Is this setup still relevant (or maybe there is a service in Azure today that encapsulates the steps done here)? 2. Can I add Linux VMs to the cluster (or, alternatively, to have just Linux VMs in the cluster)? 3. If the answer to 2 is yes, then how? ChatGPT is telling me to use cluster-aware file system like OCFS2 or GFS2 and then a matching cluster management system on each VM. What do you think? I think this suggestion does not lead to a robust or efficient solution, as it fails to mention components you used here: - availability sets, - quorum (with regular storage account). - server manager. By the way, I’ve never seen the server manager before. Is this part of Azure? Is it also applicable to using Linux VMs?

[@AzureAcademy]

Thanks very much for your support! 1. Yes this is still relevant for any shared disk scenario 2. Yes you can have a Linux cluster, but you can’t mix windows and Linux in the same cluster 3. How to setup a Linux cluster depends on the distro you are using. Most often your use something called pacemaker Chat GPT is right You need a common file system for the cluster But as you point out ChatGPT isn’t thinking of the cluster in all the Azure components It is only thinking about the Linux cluster So a better prompt would be How to build a highly available Ubuntu Linux file server cluster on Azure keeping in mind Azure high availability principles. Not sure what you mean about server manager…? It is the windows server manager since 2012. Of are you referring to something else?

[@matans844]

​@@AzureAcademy Thank you! I searched for a video that describes what you did here and I was very excited to find it! First, let me tell you what I am trying to achieve: I need a cloud architecture in which I can write to a shared disk from one Ubuntu VM, and then, possibly a few minutes later, access if from another Ubuntu VM, but without restarting the second VM or detaching and reattaching the shared data disk from it. Wtih this in mind, I searched for existing implementations, and I found your video. Then I wanted to understand how to translate your setup from a Windows cluster to a Linux cluster, and so I asked about all the pieces you had in order to find the Linux counterpart (this includes asking about the Windows Server). Thank you very much for answering! Let me address some points you wrote: First, I asked about the Windows Server to understand what was its Linux cluster mirror component. I think Pacemaker does this. Now, following your video, and working with ChatGPT about my target Linux cluster setup with a cluster-aware file system for the chosen data disk, I played around a bit with Azure and premium storage disks (premium v2). I managed to achieve my target behavior using an implementation based on link [1]. Nevertheless, I found there’s an issue with missing kernel modules for Oracle’s OSFS2 file system in the image I used from Ubuntu (see link [2]). I managed to fix it using extra modules and have it added at startup, but I fear it affects performance. During crude read/write tests ChatGPT gave me for my setup, I found that I reach the best speeds with the following components in Azure: - VM size: Standard L8as v3 (8 vcpus, 64 GiB memory). - Working outside proximity set (and thus also outside proximity placement group). - Using premium v2 shared disk, with LRS redundancy, with 10000 max IOPS and 1200 max throughput, and 4096 MB (4 GB). This was tested again ZRS premium v2 storage, and LRS and ZRS premium storage, performance tier P80. I believe my system caveats are: 1. I don’t think I am using the best kernel for my chosen file system, as adding external additions to kernel modules seem like a shortcut, and not a solution. 2. I don’t know which file system to choose. Regarding caveat 1, I need to find a better image. Regarding caveat 2, the possibilities I considered with ChatGPT are: 1. NFS 2. OSFS2 3. GFS2 4. GlusterFS 5. ceph Do you have any recommendation? Also, do you provide counseling? I've written a long post here (and my first comment was also long). I would be happy to pay you for your help. Links: [1] serverfault.com/questions/1132989/what-is-the-simplest-way-to-set-up-shared-clustered-read-write-filesystem-across [2] serverfault.com/questions/950775/ocfs2-on-ubuntu-18-04-missing-kernal-modules

[@AzureAcademy]

Cool

[@zimcanit6647]

Glad to have found your channel!

[@AzureAcademy]

Me too! What kind of stuff are you interested in learning?

[@zimcanit6647]

@@AzureAcademy I'd be keen on seeing: some CI/CD with azure devops, arm template deployments, azure powershell, architecting end to end projects from high level to low level designs and even playlists dedicated to certifications such as AZ-400 or AZ-305. Thanks again for your wonderful channel!

[@AzureAcademy]

I have multiple playlist on the channel for most of those things. If you had to pick the AZ-305 or the AZ-400 which would you want to see a series on first?

[@zimcanit6647]

@@AzureAcademy AZ-400! 🥳

[@AzureAcademy]

Thanks for letting me know...Stay Tuned!

[@Raghava_Akula]

At 08:08 , You haven't shown us how to join VMs to the domain , Could you please suggest a video for this. I have created a virtual network and a subnet and selected this existing subnet in the same virtual network

[@AzureAcademy]

Many options manual domain join Scripted join Azure VM join domain extension Etc The key with the network is to have your vnet DNS configured to use custom Then set the DCs IPs as your DNS SERVERS

[@TinyAarna]

Thanks for sharing, it is very informative and helpful.. Can you please also help on taking backup of shared disk and cluster servers..

[@AzureAcademy]

Shared disks are supported in Azure backup like all other managed disks As for backing up a cluster Azure Backup does not backup clusters directly And you might not need to If your cluster data is in the shared disk or a file share

[@madetivasu8182]

This is a great content. 👍 1 observation with using shared disks is they are not supported for backups in case of ultra sharedisk and not supporting Azure site recovery in case of Premium shared disk. Can you please do a video on utilizing the local storage disks of VMS to Suppory cluster by using StorageSpacesDirect to resolve above issue and s2d is supported for HA in case 1 of the node is down?

[@AzureAcademy]

Thanks for the suggestion...I will look into it

[@mikeplowden1099]

Dean... DNN.... It's doing my head in! I cannot ping or connect or see the shares in the File Server role node from outside the Owner node, I am using File Server for gen use and not SOFS. Is it possible to use DNN in Azure? I think this is not working because I set the FS role with Powershell and gave it a static address but that address is not taken in Azure Network Interfaces... in short - I cannot make my cluster work if I use File Server for gen use but I does work fine for SOFS

[@AzureAcademy]

Yes you can use the DNN in Azure…I showed it in the video. In Azure VMs you don’t set static addresses on inside windows. You make the address lease forever on the Azure network cards…

[@mikeplowden1099]

@@AzureAcademy I figured it out that the DNN feature does not help when creating a Gen Purpose FS but works great for SOFS. For Gen Purpose FS we need a Azure Load Balancer created with the FCS Role IP address. But still your Video opened up so many doors and windows! Thanks Dean, Really you are saving jobs out there!!! :)

[@AzureAcademy]

awesome...thanks for sharing!

[@satyaganakanapalli3331]

Its a pretty good Video, How to attach shared disk to two VM's which those Vm's available in Zone 1 and Zone 2?

[@AzureAcademy]

The VMs can span availability zones, however the proximity placement groups purpose is to keep them as close together as possible…so what you might do is have 4 VMs 2 in zone 1 in the same PPG and 2 in zone 2 in their own PPG

[@P88DAL]

Another great video, Dean. Thanks again.

[@AzureAcademy]

Thanks! Let me know what other videos you want me to make!

[@ExpertInvisible]

Great video. Thank you for sharing this with us. Can you make video on Azure Virtual WAN?

[@AzureAcademy]

Virtual WAN is already in the works...stay tuned!

[@srmvel]

I deployed 2 SQL virtual machines and did setup fail over cluster from windows. I see, the cluster ip is not reachable from any SQL nodes/outside.Is this expected behavior from Azure?

[@AzureAcademy]

if I understand correctly, is the Cluster IP is: 1. From the Azure virtual network 2. From a private virtual IP between the VMs in the cluster 3. An Azure public IP address

[@AlinneaSilva]

Thank you for this great video. This is still the best way to use shared disks in azure? I'm trying to configure a shared disk in a pool of AVD machines I'm aware that I can use fslogix for user profiles but I need to share some files with all the machines with the best performance and reasonable cost

[@AzureAcademy]

Most of the time AVD VMs use windows 10 or 11. Shared disk is only supported on windows Server OS learn.microsoft.com/en-us/azure/virtual-machines/disks-shared#windows FSLogix does not allow multiple users to access a profile disk simultaneously. It is connected to a single user. Generally what you want is done with a network share or Azure Files share

[@AlinneaSilva]

@@AzureAcademyok, thank you!

[@AzureAcademy]

anytime!

[@gilmarmaia89]

Thanks, this video really helped me.

[@AzureAcademy]

Glad to hear it Gilmar! Please share The Azure Academy with others!

[@shilpaamberkar2494]

can you please share how to implement the process without joining the domain...

[@AzureAcademy]

As far as I know clustering requires a domain. You have to creat the cluster computer object and have all the shared rights etc

[@deepakshelke6224]

Thanks Dean, very informative.

[@AzureAcademy]

Very welcome

[@a807440]

Does this allow for Azure Backup to properly backup the disk behind the shared cluster? Currently Azure Backup documentation states that Clustered Shared Volumes are not supported and I was hoping to work out a solution between either Azure File Shares or the shared disks but it doesn't appear shared disks would be supported. Your take? Thank you for the video, really helps tie together the existing documentation with a real world application.

[@AzureAcademy]

Hey a807440, YES Azure Backup can be used to backup the Azure Files share and the SQL instance as well as the VMs in the cluster. so it depends on HOW you want to protect or restore the data. Happy to help, and let me know what other things you want to see in an upcoming video!

[@lcehndjust5855]

Hi Dean and thanks for you channel. What it's with oracle cluster, it's possible on azure? Oracle documentation to create cluster say: VIP based failover which means Active/Passive cluster configuration. Database is using Local Data Guard. Note: VIP (Virtual IP & virtual hostname) can be created by any of the clusterware software only. But on azure all ip are virtual. What i need to create a cluster on azure?

[@AzureAcademy]

I have not used Oracle clusters, but my understanding is that it requires multicast to communicate with the cluster nodes, which Azure doesn't support...is that correct? Otherwise, YES all IPs in Azure are virtual, but I think you may be referring to a cluster virtual IP, this MIGHT be able to be simulated with the Azure Load Balancer

[@bruce6944]

Great stuff Dean!! But what about Storage Spaces Direct? You can create a SOFS without shared disk using S2D. Any recommendations on which approach is better?

[@AzureAcademy]

Hey Bruce, as you pointed out S2D does not need shared disk and SOFS and traditional clustering does. as to which is better I think it depends on the requirements. They can both do the job of being a file server...in fact to make things worse...the next video I am working on is Azure Premium Files as the storage for a SQL Failover Cluster Instance...the benefit being storage performance. You CAN do that with S2D but the setup of S2D to tune it and get performance optimal IMHO is more difficult.

[@sarann4258]

@@AzureAcademy Also if I am not wrong S2D requires windows datacenter edition which adds additional cost?

[@AzureAcademy]

Not sure if S2D requires a datacenter license...I don't think it does...but the default images for Server 2019 that are in Azure are DataCenter edition...I don't remember offhand if you can select a standard image, nor do I remember seeing a different price model for a server version.

[@jintojacob]

How does it differ from azure file sync server?

[@AzureAcademy]

Completely different and unrelated. Shared disk allows multiple servers in Azure to connect to the same physical disk at once Azure Files Sync is a replication service That holds the data in Azure files And replicates it through an agent multiple servers so they can function as a local file cache

[@jatinchugh2586]

Hi Dean, just my thoughts. For a multi region, single node in each region, we would not need proximity groups, do we ?.Planning to use storage account as cloud witness and shared disks as you've described in the video.

[@AzureAcademy]

Correct PPG keeps the VMs as close as possible to lower latency. If you do a stretch cluster across regions the PPG won’t help and I don’t think you can even put the VMs in the group.

[@jatinchugh2586]

@@AzureAcademy Thanks Dean, appreciate.

[@AzureAcademy]

any time!

[@sidzhang]

Hi Dean, Does this feature need to be registered for preview or something? When I provisioned that using your Github link, an error indicates that "This subscription is not registered to use Microsoft.Compute/SharedDisksForPremium feature." Thanks

[@AzureAcademy]

that was not my experience @Ceng I was able to just use it? Were you able to register the feature and test it?

[@sidzhang]

@@AzureAcademy I guess we need to register for access if we want to use Premium SSD, for Ultra SSD we don't need to register. I guess your subscription had already registered that before. Reference: docs.microsoft.com/en-us/azure/virtual-machines/disks-shared-enable?tabs=azure-cli aka.ms/AzureSharedDiskGASignUp

[@AzureAcademy]

looks like it must have been...but if I did it, it was long before I made this video...or I would have included it 😉

[@TimMullett]

How do you handle adding a generic service to these types of failover cluster? An IP-address is created as part of configuring a role, and by default this is assigned by DHCP - but this doesn't work in Azure and previously had to be configured manually; is this still the case?

[@AzureAcademy]

Right in Azure DHCP is handled by Azure. You should be able to leave a ip config dynamic and it will just get an address. If however you are creating virtual adapters...it depends...how those adapters are setup.

[@TimMullett]

@@AzureAcademy So I looked into this a bit more; if you are adding a Generic Service role then you still need to front the cluster with a load balancer and configure the cluster to respond to a TCP health probe: docs.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/failover-cluster-instance-vnn-azure-load-balancer-configure

[@AzureAcademy]

Thanks for sharing!

[@teaearlgrayh0t]

Hi Dean. Azure does not allow virtual IP addresses, don’t you need to setup a LB with the same IP, as your virtual file server?

[@AzureAcademy]

well...all IP addresses in Azure ARE virtual 🤦‍♂️ in a traditional cluster you would not be able to use a virtual IP in Azure, you needed a load balancer because there was no other way to hand off the IP from 1 VM in the cluster to the other one. - YES the cluster would know what happened...but you need people to get to the cluster, and that is what the Load Balancer did for you...

[@mikeplowden1099]

@@AzureAcademy This is my issue, using DNN and only the owner node knows of the IP, the other node can't even ping it... what is the workaround please?

[@AzureAcademy]

did you setup the DNN Listener? 👉 docs.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/availability-group-distributed-network-name-dnn-listener-configure?view=azuresql

[@LelandVelasco]

Is it possible not to use iLB and for SQL Availability Group in Azure? I noticed this sofs is not required to use iLB

[@AzureAcademy]

I am not 100% sure on always on availability groups but a normal Failover cluster on server 2019 can use Distributed Network Name (DNN) instead of a load balancer in fact I am working on this for my next video on Azure Premium Files...stay tuned

[@LelandVelasco]

@@AzureAcademy looking forward to see this. Good luck Dean!

[@AzureAcademy]

let me know what you think of it...or join the Premiere and chat with me

[@waynehoggett]

Is it true that you can only use a DNN with SQL? Can you use it with a General Purpose File Server, and if you do, will it be supported by Microsoft?

[@AzureAcademy]

s far as I know the DNN is a feature of windows cluster, not SQL. So you should be able to use it with any other clustered service

[@mikeplowden1099]

Hi Wayne... did you ever got DNN to work with FS For gen use? I am having the same issue when I cannot connect to the VIP IP to see the shares...

[@AzureAcademy]

are you doing a SQL cluster...the traditional way or an Always On Availability Group?

[@stefanminehan]

Me: Spends a few hours looking into clustering and the need for dual NIC's and heartbeat config/best practices This video: Lol, no need fam.

[@AzureAcademy]

LOL happy to help make Azure easier for ya 👍👍

[@sasidharan8723]

Hi Dean, When configuring cloud witness we are getting Error code : 0x80131500; Native error code : 1.. WinRM cannot complete the operation. Verify that the computer name is valid.. Can you help ?

[@sasidharan8723]

I am able to add Fileshare from same storage account but not able to configure cloud Witnesses disk

[@AzureAcademy]

Make sure that all he computers for the cluster can be found in DNS

[@AzureAcademy]

ah...you can't use the fileshare as a cloud witness...use Blob storage for the witness

[@sasidharan8723]

Ya got it thanks, we are working on the case, will update you

[@AzureAcademy]

nice! 👍👍

[@zzaidi_]

where can i copy the full text for template deployment ?

[@AzureAcademy]

the deployment of the shared disks is on my GitHub github.com/DeanCefola/Clustering

[@zzaidi_]

@@AzureAcademy too bad for me This subscription is not registered to use Microsoft.Compute/SharedDisksForPremium feature

[@zzaidi_]

Edit Template - under resources - network, virtual machine & availability set these 2 i cant find the script

[@AzureAcademy]

try this - sign up for shared disks microsoft.qualtrics.com/jfe/form/SV_cMCj3LBlv47M6xL

[@AzureAcademy]

what do you mean can't find the script?

[@alphabanks]

Distrubted Network Names are great until you need to deploy a multi site Always On.

[@AzureAcademy]

LOL sounds like you have a story to tell...?

[@abi123123]

Could have made it a much faster video , so as to make sure i dont understand it

[@AzureAcademy]

Well I could record it at 2x speed if you think it would help 😀.

[@AparnaBL]

@@AzureAcademy 😀😂

[@AzureAcademy]

🤦‍♂️😁👍🤷‍♂️