Airflow tutorial | Install Airflow | Write and run your first DAG | Apache airflow on Windows Docker

How to Install Airflow on Windows
Airflow local setup
Create first airflow dag
Installing Apache Airflow on Windows A Step-by-Step Guide
install apache airflow on windows with docker
install airflow
airflow scheduler
airflow windows
airflow on windows
How to run Airflow on Windows
airflow interview questions
airflow interview questions and answers
apache airflow windows
install airflow windows
airflow installation on windows
install apache airflow on windows
airflow install windows
airflow docker windows
airflow for windows
apache airflow installation on windows
install airflow in windows
install airflow locally windows
How to set up Airflow on Windows
install airflow on laptop
install airflow on pc 2023
install airflow on computer
install airflow on windows 11
install airflow on windows 10
install airflow on windows with docker
install airflow
airflow docker
docker windows
apache airflow
airflow in windows
airflow windows installation
install airflow docker windows
windows airflow
pip install airflow
apache airflow github
install airflow on windows without docker
airflow on windows 10
run airflow on windows with docker
install airflow docker
How to develop airflow dags on windows
Airflow hello world dag
install airflow on windows without wsl
install airflow on windows without ubuntu
install airflow on windows server 2016
can we install airflow on windows
can i install airflow on windows
pip install airflow on windows
How to install Airflow in Windows with Docker
How to install Airflow using pip
How can I install Airflow
install airflow python
install airflow locally
install airflow using docker
pip install airflow providers
pip install airflow postgres
How to Install Apache Airflow on Windows without Docker
Installation of Airflow Airflow Documentation
Install Apache Airflow on windows 10 without docker
How to Install Apache airflow in windows machine using
Apache Airflow Installation on Windows 10
How to Install Apache Airflow
Download Docker Desktop for Windows:
docs.docker.co...
Download Visual Studio Code:
code.visualstu...
Any issues during wsl --update, then please refer below video:
• How to install Windows...
dockerfile, docker-compose.yml and welcome_dag.py codes:
sleek-data.blo...

[@shafiulamin4128]

Cannot thank you enough. Everything working perfectly. I suppose my Airflow journey has just started .

[@SleekData]

Dear Friend, Glad you like them! To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, it would be so grateful! Thank you for your encouragement.

[@jonathan.matheus]

thanks dude, I spent 2 days trying to find how to do it step by step without any complicated context going on, appreciated

[@SleekData]

Dear Friend, Glad you like them! To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, it would be so grateful! Thank you for your encouragement.

[@szymonolejniczak7342]

I got here from some course on udemy as i wasnt able to set-up docker with their videos. Your alternate way worked perfectly! thank you very much!

[@SleekData]

Dear Friend, Glad it helped! To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@trunksssz]

after many tutorial i have watched, only this works on me:) thank youu

[@SleekData]

Dear Friend, Glad you like them! To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@NandaAcademies]

Fantastic tutorial! No one can explain better than this! Of course, I am yet to install docker and airflow, but anyway it should work.

[@SleekData]

Dear Friend - Thank you for your kind words! We're delighted to hear that you're enjoying our content. To stay updated with our future videos, consider subscribing and sharing with your friends. Your support means a lot to us. Thanks again for your encouragement!

[@mygmail-ti4js]

This is really a great Guideline. Thank you for sharing.

[@SleekData]

Dear Friend - Thank you your kind feedback! To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. Your support is valuable to us. If you ever need technical support, please feel free to leave a comment on any of our videos, and we'll be sure to respond promptly or create a video addressing the issue. If you could share our video/channel with your friends, we would be so grateful!

[@channuangadi7504]

to the point explanation thank you so much

[@SleekData]

Dear Friend - Thank you your kind feedback! To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Once again thank you for your encouragement.

[@kumaran3534]

Helpful!😊

[@SleekData]

Dear Friend, Glad you like them! To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, it would be so grateful! Thank you for your encouragement.

[@sfcsarah]

really looking forward to your video on setting up Postgres

[@SleekData]

Dear Friend - Thank you for your interest! I understand that you're eager to get started, so I'd like to provide you with a high-level overview of the steps required to set up PostgreSQL as your metadata database, even before we create the video tutorial: sudo apt install libpq-dev sudo apt-get install postgresql sudo service postgresql start sudo -u postgres psql CREATE USER airflow PASSWORD 'airflow'; CREATE DATABASE airflow; GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO airflow; Exit cd to airflow home vi airflow.cfg executor = LocalExecutor sql_alchemy_conn = postgresql+psycopg2://airflow:airflow@localhost/airflow We appreciate your support and would love it if you could like and subscribe to our channel to stay updated with our future content. If you could share our video/channel with your friends, we would be so grateful! Once again, thank you for your encouragement.

[@sfcsarah]

@@SleekData Thank you so much! I'll try it out :)

[@valentinosimeoni6993]

@@SleekData great tutorial! the only one thats worked for me . now im trying to connect postgre with your instructions but isnt working. do i have to change something on the docker-compose.yaml file ?

[@vinodtengli6882]

Thank you so much for sharing very useful video

[@SleekData]

Dear Friend - My pleasure 😊 To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@shafiulamin4128]

Nice one mate. Very useful

[@SleekData]

Dear Friend, Glad you like them! To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@user-qy6gk5fs8k]

Take a close look at the airflow while watching the tutorial video. Thank you, in progress. I cannot log in with the user name, password, and information in the log in the visul studio code.I previously installed airflow for testing, and can only log in with airflow/airflow information.

[@SleekData]

Dude - Pls check the video i've shown a file from where you can copy password.

[@pushankarmakar1783]

theres a problem. at 5:44 in my computer its showing ✘ sleek-airflow Error 3.9s Error response from daemon: pull access denied for sleek-airflow, repository does not exist or may require 'docker login': denied: requested access to the resource is denied The terminal process "C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -Command docker compose -f "docker-compose.yml" up -d --build" terminated with exit code: 1. Can you please help me with this Edited: nevermind. it was a silly error. im keeping this in case someone makes the same mistake as me. i made the image with the name 'airflowstuff' but used the code given in the description which had the name as 'sleek-airflow' hence it wasnt able o find the image

[@SleekData]

Dear Friend - Did you complete the build step? during that step you are supposed to give a name for the image, I've given sleek-airflow as the name. So i have used sleek-airflow:latest in docker compose, what name did you give? once after sucessfull build it should locally create sleek-airflow:latest for you...if this is not available locally then it tries in the remote repo and gives you this error. Please watch the video once, you will get it. Good Luck, please like and subscribe.

[@franklynchiemekam1682]

Thank you, made the same mistake as well and your correction saved me a lot ❤

[@aruk1040]

Thank you so much 😭💘

[@SleekData]

Dear Friend, Glad you like them! To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@anhhung4737]

sorry but my Docker doesn't show any logs and localhost 8080 is not responding. Can you help T.T

[@sureshm8687]

Hi I am facing an issue with localhost 8080 is not responding

[@SleekData]

Dude - pls try http vs https as well as localhost vs 127.0.0.1 Also make sure the port 80 / 8080 is forwarded in your docker config files as i did. Good Luck, appreciate if you could like and sub.

[@Templestarz]

@@SleekData I have followed the same path as you have shared in your video. The composer file has the port "8080:8080" still it is not working. can you advise what we need to check more

[@ThànhNguyễnViệt-f8u]

Hi Sleek, Im trying to open the localhost but it doesn't response. When View Details, it says "SyntaxWarning: invalid escape sequence '\d'"

[@SleekData]

Dear Friend Please double-check the Docker Compose file. You should expose port 8080 from a container to the host. You can refer to the previous video in the playlist. Try all possibilities: 127.0.0.1:8080 127.0.0.1:8080 127.0.0.1:8080 localhost:8080 localhost:8080 Also, please open the command prompt and type ipconfig and hit enter. You will get your IPv4 Address. For example, if it is 192.168.56.1, then please try all possibilities with that IP address: 192.168.56.1:8080 192.168.56.1:8080 192.168.56.1:8080

[@jeffs4810]

Really enjoying this tutorial, but running into an issue on Windows 11. I can get the container to start if I remove the mounted volume from the YML file, but when the volume is included the container fails after a short time with errors that appear to be related to accessing the mounted folder. I can see that the 'airflow' folder is getting created. The error messages in the container log look something like this: FileNotFoundError: [Errno 2] No such file or directory: '/opt/airflow/logs/scheduler/2024-01-11' ... FileNotFoundError: [Errno 2] No such file or directory: '/opt/airflow/logs/scheduler' ... PermissionError: [Errno 13] Permission denied: '/opt/airflow/logs'

[@SleekData]

Friend - pls double check docker files you should switch to correct user ids as i have done.

[@azizkanch5257]

Firstly, thanks for the wonderful content. I have followed the steps precisely. However, while trying to do compose up, I am getting this error failed to create network airflow-workspace_default: Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: read-only file system. Please help me. Thank you!

[@SleekData]

It indicates Docker is encountering a permission issue while trying to create a network for your Airflow workspace.

[@kartikeyapande5878]

localhost:8080 page isnt opening and says "This page isn’t working localhost didn’t send any data. ERR_EMPTY_RESPONSE"

[@SleekData]

Dude - In you main PC, open command prompt and issue ipconfig command and see if you can locate the ip address of your docker (See if you have anything like - Ethernet adapter vEthernet (WSL (Hyper-V firewall))) Now use that IP address instead local host, for example you fond the IP address as 192.168.16.1 then try this... 192.168.16.1:8080/ also pls try http and https both. Good luck, to stay updated with our future content, I kindly request you to consider subscribing and liking our videos.

[@kartikeyapande5878]

This worked. Thanks a lot. 👍🏻 And yup, have subscribed you already.

[@alaaeldinmanse5228]

I appreciate what you did , and hope you keep going , but there is something i have , the .yml file have more lines than you have in the video , that what i have got from apache airflow website , and i found there is more than on container have been created , Do you have an Explanation for this ? and i can't set my username and password

[@SleekData]

Dear Friend - Setup in this video is intended for development and testing purposes. But the yml file may be more suitable for higher environments: airflow.apache.org/docs/apache-airflow/2.7.3/docker-compose.yaml More over the password generally displayed in the log when you start for the very first time. other wise look at the folder path that i have shown in the video for the location of password file. To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@mathandnatureman]

Hi , I am getting error while generating the build (4:02) like "ERROR: failed to solve: process "/bin/bash -o pipefail -o errexit -o nounset -o nolog -c apt-get update && apt-get -y install git && apt-get clean" did not complete successfully: exit code: 100" could you please help me out. Thanks.

[@SleekData]

pls try to install git and retry.

[@a05n0pavithra7]

I have the bootstrap servers, using the bootstrap servers I wanted to consume messages from a particular topic in those bootstrap servers and using the in-built operator which is ConsumeFromTopicOperator. And the version of Apache Airflow I downloaded is 2.9.1 Can you please help me with this?

[@SleekData]

Dude - Pls try something like this: from datetime import datetime, timedelta from airflow import DAG from airflow.providers.apache.kafka.operators.kafka import ConsumeFromTopicOperator default_args = { 'owner': 'airflow', 'depends_on_past': False, 'email_on_failure': False, 'email_on_retry': False, 'retries': 1, 'retry_delay': timedelta(minutes=5) } def process_batch(batch): # Process batch of messages here print("Processing Kafka batch:", batch) # Example: for message in batch: do_something(message) with DAG( dag_id='kafka_consumer_dag', default_args=default_args, schedule_interval=None, # No scheduling interval as the task executes indefinitely start_date=datetime(2024, 6, 14), catchup=False ) as dag: consume_task = ConsumeFromTopicOperator( task_id='consume_from_kafka_topic', kafka_topic='your_topic_name', bootstrap_servers='your_bootstrap_servers', group_id='airflow_group_id', apply_function_batch=process_batch, # Function to apply to batches of messages commit_cadence='end_of_operator', # Commit offsets at the end of the operator max_messages=None, # Read to the end of the topic max_batch_size=1000, # Maximum number of messages to read in each batch poll_timeout=60 # Timeout for polling messages ) Good luck, to stay updated with our future content, I kindly request you to consider subscribing and liking our videos.

[@sridharsriii3125]

In yml file image container was showing error(404) page not found. Any other image link is there ("sleek-airflow Error pull access denied for sleek-airflow, repository does not exist or may request 3.4s Error response from daemon: pull access denied for sleek-airflow, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

[@SleekData]

Brother - you have not followed the steps, you have to first build the image locally (sleek-airflow is local image not from repository) since this is not available in your local its trying to download from repository and giving you this error, pls follow the steps side-by-side and step-by-step. Good luck.

[@bishop9168]

The docker file ran successfully but the docker-composer.yml file gives a 'version' obsolete message, any assistance will be appreciated.

[@SleekData]

dude - please check below in your system docker-compose --version and accordingly try to Update the Compose File Version: version: 'x.y'

[@madtitan44]

Hello , I am getting cannot use relative path to connect to sqllite. Please use absolute path

[@SleekData]

Dude - Pls doublecheck file mounts in docker file, pls maintain the folder hierarchies just like mine.

[@testapps-tm6uv]

Hi there, when I clicked the 8080:8080 link - it leads me to a blank page instead of Airflow admin. I did every step you did and restart the computer as well.

[@SleekData]

Dear Friend Please double-check the Docker Compose file. You should expose port 8080 from a container to the host. You can refer to the previous video in the playlist. Try all possibilities: 127.0.0.1:8080 127.0.0.1:8080 127.0.0.1:8080 localhost:8080 localhost:8080 Also, please open the command prompt and type ipconfig and hit enter. You will get your IPv4 Address. For example, if it is 192.168.56.1, then please try all possibilities with that IP address: 192.168.56.1:8080 192.168.56.1:8080 192.168.56.1:8080 Good Luck, please like and subscribe.

[@QuangHuy-is7jo]

@@SleekData thank you

[@mamadoubabadjanballo1977]

Hi, please i need your help. I can not lunch airflow. this is the error i am getting ( sleek-airflow Error 3.3s Error response from daemon: pull access denied for sleek-airflow, repository does not exist or may require 'docker login': denied: requested access to the resource is denied * The terminal process "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -Command docker compose -f "docker-compose.yml" up -d --build" terminated with exit code: 1. * Terminal will be reused by tasks, press any key to close it.

[@SleekData]

Brother - Looks like the docker image build step was not done or not successful. Due that sleek-airflow is not available locally as a result it tries to find in the public docker remote repository. Please retry the video step by step, it should work. Good Luck, to stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful!

[@user-df1lk5fz3u]

THANK FOR THE COURSE I tried everything went well until I waited for the dag to appear on the web interface I saw nothing and then if I restarted the yaml the password no longer worked PLEASE SOME HELP FOR ME

[@SleekData]

Dear Friend - pls refer the video at 6:10, I have told 2 ways to get your password. To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@ronin959]

i have a problem - "(root) Additional property coversion is not allowed" (5:42)

[@SleekData]

Dude - May be you are running airlow with root, pls check docker and docker compose files... first try to exactly follow my files and then once every thing is up and running then do your changes on top.

[@IuryLeal-zj8ec]

It didnt workout when i cliked on the link to open the webairflow interface. what can i do?

[@SleekData]

Dude - Pls revisit the video and see if you've opened the port in the docker file. also try with http vs https and also local host vs 127.0.0.1. Good luck, t o stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your comment.

[@shawnschreier5978]

I had set this up perfectly last week and was able to set up some DAGs. However, now I get an error saying localhost did not send any data. Any guidance or help?

[@SleekData]

Dude - Pls try http vs https and localhost vs 127.0.0.1 Good Luck.

[@sanikabhosale6513]

@@SleekData I am also facing the same error tried all the above solutions by yet stuck please help me out

[@MitchMcCall-bd4ul]

the docker image you have is out of date, i am refused access and i cannot find the docker image file on dockerhub

[@kian69topgun1]

came across the same error myself.. did you find any other image to run ?

[@MitchMcCall-bd4ul]

@kian69topgun1 no, my advice is dont run airflkw off of docker and find a tutorial that doesnt force you to use docker because its ultimately unnecessary just use a virtual environment in ubuntu

[@SleekData]

Dear Friend - Did you complete the build step? during that step you are supposed to give a name for the image, I've given sleek-airflow as the name. So i have used sleek-airflow:latest in docker compose, what name did you give? once after sucessfull build it should locally create sleek-airflow:latest for you...if this is not available locally then it tries in the remote repo and gives you this error. Please watch the video once, you will get it. Good Luck, please like and subscribe.

[@basavarajpn4801]

Am getting error when running in visual code for welcome_dag.py saying that "No Module named Airflow", Please help me with this

[@SleekData]

Dude - You can not run in vs code, you need develop in vs code and the execution has to be done in side docker container using airflow UI.

[@SouhaCherif-vp2vw]

hey i want to use sudo command and it keeps asking me to write the airflow password, but i cant seem to find it anywhere

[@SleekData]

Dude - Please play the video side by side may be on another device/phone and check, it should work. Good luck.

[@HoangQuocat-xp5eu]

I do exactly what you do but when i run the final code this happens "airflow.exceptions.AirflowConfigException: Cannot use relative path: `sqlite:///C:\Users\Admin/airflow/airflow.db` to connect to sqlite. Please use absolute path such as `sqlite:////tmp/airflow.db`".

[@SleekData]

Dude - could you try this in airflow.cfg? sql_alchemy_conn = sqlite:////C:/Users/Admin/airflow/airflow.db if it did not work, pls erase all and start from scratch, step by step and side by side.

[@StsGamin]

Hy there is a problem in launching the UI as its not connecting. I tried to delete and rebuild the image, tried https still its not connecting. What to do ??

[@SleekData]

Dude - did you try opening the port in docker compose file? ports: - "8080:8080" also pls check if there are any errors in the docker log?

[@deepaksinghchauhan489]

@@SleekData Hi bro I also face the same issue, my 8080 port is in use for some other thing, So I enter 9000:9000 and then I ran, the container was built and working properly but when I clicked on for UI . It shows "This page Isn't working" and also, the login interface is not coming in my browser for airflow login. Although password is showing in my airflow folder. please help me in resolving this issue.

[@ananddarji86]

Same issue i am facing @sleekdata

[@ahmedtr4305]

Could I schedual execute python script located in my local machine?

[@SleekData]

Dude - To execute a Python script in Apache Airflow, you typically use the PythonOperator provided by Airflow. Please search around that you will get an idea. Good luck., to stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your engagement.

[@user-wm1hz8jm6z]

I have a problem in the parte of cmd could you tell me how i can do that

[@SleekData]

Dear friend I could you understand your problem, pls add a new command elaborating the issue. To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@hemav9981]

while running dockerfile,it is showing error.It's showing "error during connect".Please help me.

[@SleekData]

Dear Friend - this is error is not sufficient to provide some suggestions. May be please restart the PC and retry. To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@abinashkumar1533]

these three tasks has been executed successfully , But where these tasks is excuted I do not know . How I executed this task I do not know. Just I know I have executed these tasks. A little knowlege is dangerous thing.

[@SleekData]

Hi Dude - This is playlist, please watch full playlist you would get better understanding, it is just around 1 hour. kzbin.info/aero/PLc2EZr8W2QIAI0cS1nZGNxoLzppb7XbqM&si=iAK3qM2qcZ69LgSi

[@Wolololo_Wololo]

Hi, When I compose up, it shows the DAG Import Errors at the UI I used some module in the DAG and I installed the module locally. How to install the module into docker container? Broken DAG: [/opt/airflow/dags/podcast-summary.py] Traceback (most recent call last): File "", line 219, in _call_with_frames_removed File "/opt/airflow/dags/podcast-summary.py", line 7, in import xmltodict ModuleNotFoundError: No module named 'xmltodict'

[@SleekData]

Hi Dude - pls watch the next video, i have also given the link below, it talks about similar error and 2 ways to address it. kzbin.info/www/bejne/haa1motnjbaar7s Good luck, pls like and subscribe to stay up to date.

[@rajjoshi5111]

hello buddy...thank you for the video...but i am having problem with the localhost:8080 page....it isnt opening and says "This page isn’t working localhost didn’t send any data. ERR_EMPTY_RESPONSE"

[@SleekData]

dude - pls try changing between http https? Good Luck, to stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@rajjoshi5111]

@SleekData thanks but the issue was resolved...I just refreshed the file on docker and it worked

[@kartikeyapande5878]

@@rajjoshi5111 I'm getting same error. Please help to resolve. @SleekData

[@rajjoshi5111]

@@kartikeyapande5878 all I did was refresh the file on docker.... And I don't remember doing anything else

[@A_AMARI]

hi sir, can you please check on this , when i click on the compse up it shows me this error " sleek-airflow Error 2.8s Error response from daemon: pull access denied for sleek-airflow, repository does not exist or may require 'docker login': denied: requested access to the resource is denied" can you help me please

[@SleekData]

Dude - sleek-airflow is build locally in the previous step, pls check it since its not available locally it is searching in the remote repo. Good luck, To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@A_AMARI]

thanks a lot@@SleekData

[@hurstilthymy4943]

I am getting a Import "airflow.operators.python_operator" How can I fix this?

[@YukeshAS]

instead of python_operator use python, hopefully that will fix it!

[@SleekData]

pls try this - from airflow.operators.python_operator import PythonOperator

[@muhammadfaridansutariya4411]

can i implement this tutorial on production sir?

[@SleekData]

Dear Friend - This setup is intended for development and testing purposes, employing SQLite as the metadata database and the Sequential Executor. However, it is not recommended for production environments. For production use, it is advisable to use PostgreSQL metadata database and a more robust executor. Alternatively, in a cloud environment, consider utilizing managed Apache Airflow instances like AWS Managed Workflows for Apache Airflow, Google Cloud Composer, or the Airflow integration within Azure Data Factory. Also refer below yaml file to get fair idea about production kind of deployments. airflow.apache.org/docs/apache-airflow/2.7.3/docker-compose.yaml To stay updated with our future content, I kindly request you to consider subscribing and liking our videos. If you could share our video/channel with your friends, we would be so grateful! Thank you for your encouragement.

[@muhammadfaridansutariya4411]

can u give me reference tutorial deployment airflow on production sir?@@SleekData

[@SleekData]

Bro - Are you not keen into managed Apache Airflow instances like AWS Managed Workflows for Apache Airflow, Google Cloud Composer, or the Airflow integration within Azure Data Factory? You can bring up scalable Airflow in mater of 5 to 10 minutes with these services.. and they are mostly pay as you go cloud model. Please note if you want to implement manually there are many things you need to consider: Executors (Workers) Metadata database Not sure what you have decided for these? Note: Please add new comment every time, because i do not get notifications on existing comments, so i some time miss to reply..

[@abubakarsiddiquem9358]

Are these steps same for macOs?

[@SleekData]

Dude - Once you install docker and vs code for mac, the remaining steps are almost same. Good Luck.

[@kxt66]

Thank you very much for the video. I followed the instructions, but I encountered a problem. When I commend “Compose Up” for the docker-compose.yml file I get the following message : “ ✔ Network airflow_default Created 0.3s - Container airflow-sleek-airflow-1 Starting 0.5s Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:8080 -> 0.0.0.0:0: listen tcp 0.0.0.0:8080: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted. ” This is the first problem. The second problem is, when I change port to "8081:8081" in docker-compose.yml, the container is being created, bot when I try to open it in web browser I get the message : The connection was reset. I admit, that I am not very familiar with ports configuration issue. How can I fix the problems described above ? Thanks a lot for the help 😊

[@SleekData]

Dude - Restart and retry if still issue persists. try this in cmd and find the PID that is currently using 8080 netstat -ano | findstr LISTENING then go to task manager and process tab, right click on any column heading and enable PID, with this you can identify what is locking your 8080. Good luck, to stay updated with our future content, I kindly request you to consider subscribing and liking our videos.

[@kxt66]

@@SleekData Thank you very much for your help. It works parfectly and I will follow the next videos :) Subscribed of course :)

[@sultanchissonobie8376]

what version of python do you use?

[@SleekData]

Dear Friend - Currently the apache/airflow:latest and apache/airflow:2.5.2 images are Python 3.8 images. Any time you can reach out to this page to get latest information: airflow.apache.org/docs/apache-airflow/stable/installation/supported-versions.html Good Luck, we appreciate your support and would love it if you could like and subscribe to our channel to stay updated with our future content. Also if you could share our video/channel with your friends, we would be so grateful!

[@mohsinrajpoot3780]

where is content for dockerfile

[@SleekData]

Dear friend - Here you go, you can also find the link in description: sleek-data.blogspot.com/2023/09/how-to-install-airflow-on-windows.html Good luck, pls don’t forget to hit like and subscribe.

[@krishnavelamala7853]

getting this error while executing the dag: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.quotable.io', port=443): Max retries exceeded with url: /random (Caused by NameResolutionError(": Failed to resolve 'api.quotable.io' ([Errno -2] Name or service not known)"))

[@HoangQuocat-xp5eu]

same error. Have you fixed it yet?

[@SleekData]

looks strange, may please start from scratch, step by step and side by side.