Mastering Grafana Loki: Complete Guide to Installation, Configuration, and Integration | Part 1

[@cyrilgonzales8691]

Good day Sir,, i am trying to use loki and promtail with my grafana. But we are using windows environment, do you have any guide for loki and promtail on windows?

[@fagobg]

Is there an easy way to add host name label in the promtail. Can you set a dynamic label? Something like hostname:{HOST}. Where HOST is an environment variable. When looking at logs in Grafana you would want to know the host they came from.

[@KodeKloud]

Yes, in Promtail, you can use dynamic labels like hostname:${HOST} in the configuration to include the host's name in logs. This helps identify log sources when viewing them in Grafana.

[@Shiv_30]

I have a issue in this scenario, grafana not show all pods log, Give solution for that ...plz..

[@eylulucar8046]

./loki-linux-amd64 -config.file=loki-local-config.yaml I can only access loki from Grafana when this command is entered. when the command is killed Unable to fetch labels from Loki (Failed to call resource), please check the server logs for more details I get an error. Is this what it should be or am I doing something wrong?

[@AshfaqueAhmed-b8d]

Great explanation brother.Respect.

[@KodeKloud]

Glad it was helpful!

[@rupakrath637]

how can configure promtail to send multiple containers logs which are running on the same machine to loki with different labels

[@KodeKloud]

To configure Promtail to send logs from multiple containers on the same machine to Loki with different labels, create a Promtail configuration file and define separate scrape jobs for each container. Customize the labels and paths for each job to distinguish between the containers, and then start Promtail with this configuration. It will send logs to Loki with the specified labels for differentiation.

[@tokar86a]

Great explanation and very easy to understand and follow along. Hope to see more like this.

[@KodeKloud]

Appreciate your gratitude! We strive to make complex topics accessible. If there's a specific area you'd like us to cover, let us know. We're here to help!

[@jayamurugans2441]

after the promtail start command, am getting the output same as here,but when i check the grafana console i cant see my log file paths or job names over there, i can only see the existing files and log paths of other servers, do i need to refresh or do anything on the grafana or loki server, please advice

[@KodeKloud]

"If you're not seeing your log file paths or job names in Grafana after starting Promtail, try these steps: Check Promtail's Configuration: Ensure it's correctly set up to tail the desired log files. Verify Log File Access: Promtail should have permission to read these files. Look at Promtail Logs: Check for any errors or warnings. Confirm Loki Receives Logs: Use Loki's UI or API to verify. Check Grafana's Data Source Setup: Ensure it's correctly connected to Loki. Refresh Grafana: Sometimes, refreshing the dashboard or creating a new panel helps. Review Network Connectivity: If Promtail and Loki are on different servers. Examine Grafana Queries: Make sure they are correctly formulated. Restart Services: Restarting Promtail and/or Grafana can resolve issues. Consult Documentation: For any missed configuration steps."

[@MultiGaurav009]

That was really awesome demo on Loki, thanks Kodekloud team.

[@KodeKloud]

Thank you for the appreciation. You can check our videos related to DevOps and subscribe to our channel to stay updated with all the latest tools.

[@MultiGaurav009]

@@KodeKloud I am already a fan of Kodekloud and have taken yearly subscription 😊

[@neeraj_gk]

can we take logs from S3, if yes please do a tutorial or reply a article on it

[@KodeKloud]

Hey, please refer to community.grafana.com/t/it-is-possible-to-read-aws-s3-logs-by-loki/55579

[@rajasekhar194]

In my current role we want build the open source like loki, minor, tempo, is it possible do same videos for mimir and tempo with s3 bucket as datastore in k8s and best practice for production level

[@KodeKloud]

Hey, thank you for your thoughts, we might consider doing it in the future.

[@abhinavorugant5798]

Great explanation and very easy to understand!!👏

[@delmo2009]

Nice, when you post part 2 ?

[@KodeKloud]

Hi, We will be posting second part next week. Do subscribe to our channel and stay tuned!👍

[@jnelly3426]

Fantastic! Very well put together

[@KodeKloud]

Glad you think so!

[@teddybear4466]

Thanks for the video. I am using Grafana cloud, and I want to collect logs from the Airflow. Now as you said, I am only getting the labels, not the actual logs. So if I use promtail, would I get all the logs which are present in the Airflow? Also, if you could explain a bit more what do you mean by “basic logs” 4:18

[@vlauciani]

Thanks for the video I have a simple question regarding Promtail; I am using regexp to set labels. Are all the lines in the log that don't meet the regexp still sent to Loki? If "yes", is there any way to not do that and just send to Loki only the lines that pass the regexp? In other words I’d like to send to Loki only a subset of log lines.

[@sarvagyamaithani2093]

Can't we install/configure Grafana/Loki in the application server ?

[@independence_1947]

Thanks for the video. How do we set up to view Loki’s logs using systemd (rpa)

[@riquelinhares]

Good and straight!

[@KodeKloud]

Hi, we appreciate the kind comment! enjoy!

[@ramashish123]

That's was really clean and clear explanation ❤

[@KodeKloud]

Thanks a ton!

[@weilin4872]

good 😬

[@mohamedhaneef6379]

Great ❤

[@KodeKloud]

Thanks for watching our video. Cheers!

[@gursharanvicky]

Hi @KodeKloud and all.... I want your opinion about logging solution. Should we consider Grafana+Loki over ELK stack. I am getting huge cost for the monitoring and logging with AWS. Want to suggest particularly logging solution to my organisation. If you can provide your inputs on this please. Thanks in advance.

[@KodeKloud]

The choice between Grafana + Loki and the ELK Stack should be based on your specific logging requirements, operational capabilities, and preferences. Each toolset has its strengths and is suited to different scenarios Grafana + Loki: Good for simpler operations, integrates well with Grafana and Prometheus, more cost-effective, and easier to use. Ideal if you already operate within this ecosystem and want streamlined log management. ELK Stack: Offers powerful search capabilities and is well-suited for complex log analysis. Requires more resources and can be more expensive to run, but it's the go-to choice if full-text search and extensive data processing are necessary.

[@DieterFrueh-cp1go]

How to get syslog into loki?

[@Dioman172]

Guide from web is disgusting. Thanks fro your guide!

[@KodeKloud]

Glad it was helpful!

[@nsohche-t3k]

This was amazing and very helpful but i have a question .. is grafana loki better option in general than splunk ?

[@KodeKloud]

"Grafana Loki is cost-effective, simpler, and integrates well with Grafana. It's a good choice for straightforward log management, especially in environments already using Grafana. Splunk offers more comprehensive log management and advanced analytics features. It's better suited for complex log analysis and large-scale enterprise use, but it's typically more expensive. Choose Loki for a simpler, more budget-friendly solution, and Splunk for advanced features and large-scale operations."

[@tolearn1517]

Is grafana Loki open source?

[@ramashish123]

Yes Grafana has both open source and paid versions

[@KodeKloud]

Yes, Loki is an open-source log aggregation tool inspired by Prometheus architecture.

[@tolearn1517]

@@KodeKloud thanks a lot for your response and the video was just amazing and it helps me a lot

[@user-kg2qn8se4h]

How did you switch to different nodes in one machine? This thing should also be cleared.

[@KodeKloud]

We utilized Vagrant to create virtual machines specifically for demonstration purposes. To access these VMs, we executed the vagrant ssh command. For convenience, we opened multiple terminal windows, each running a separate vagrant ssh session, allowing us to switch between different VM environments as required.

[@AjayKumar-lm4yr]

How to store Grafana Loki logs in Azure Blob Storage

[@KodeKloud]

" To store Grafana Loki logs in Azure Blob Storage, first create an Azure Blob Storage account and configure the necessary access permissions. Then, install and configure Loki, specifying Azure Blob Storage as the backend in your Loki configuration file. Make sure to provide the Azure Blob Storage URL, container name, and the appropriate credentials. Restart the Loki service to apply the changes and verify that logs are now being stored in Azure Blob Storage. Remember to consult documentation for both Loki and Azure for specific configuration details to suit your environment and setup. You can refer more to grafana.com/docs/loki/latest/storage/

[@asifadam93]

Nice video bro, keep it up ☺

[@MichaelaMarkou-v4x]

Great!

[@KodeKloud]

Hi, we appreciate the kind comment! enjoy!

[@ThangMinh-ul3cs]

how to configuration loki delete log automaticaly ?

[@KodeKloud]

Hi, please refer to grafana.com/docs/loki/latest/operations/storage/logs-deletion/#:~:text=Configuration%20Enable%20log%20entry%20deletion,very%20careful%20when%20enabling%20retention

[@sagarhm2237]

How to get response time and alerting to response time of the apps

[@KodeKloud]

To monitor response time and set up alerts for applications with Grafana Loki, first configure your application logs to include response time data. Use Loki to ingest these logs and create queries to extract response time information. Set up alerting rules in Grafana based on these queries, defining conditions for when response times exceed specific thresholds. Configure notification channels to receive alerts, whether through email, Slack, or other platforms. Regularly test and adjust your alerting thresholds to ensure you're notified when response times become critical. This approach allows for proactive response to performance issues and helps ensure your applications meet their service level objectives.

[@ingazaitseva794]

Thank you so match for this demo. It was very hopefull for me

[@KodeKloud]

Glad it was helpful!

[@ingazaitseva794]

நீங்கள் தமிழ் பேசுகிறீர்களா தம்பி)?

[@KodeKloud]

Thanks for watching our video. Cheers!

[@madhand6045]

Hi I have one scenario working on im using loki and grafana agent here i just want to monitor log files under a directory, in this case the directory has more than 1000 folders and inside that folders more than 10000 log files are there and here the issue is folders are getting created everyday so I have used * for folders and * for log files also but I reaaly don't need older folders the folders are in YYYYMMDD format is there anyway to avoid these older folders being added because it cause an error while starting

[@Shiv_30]

I have a issue in this scenario grafana not show all pod log, plz... Me

[@madhand6045]

@@Shiv_30 both are different bro anyway I have done this using scripts

[@rogiestevan]

any chace of having a full loki course?

[@KodeKloud]

Hi, The second part is releasing next week. Do subscribe to our channel and stay tuned for future updates.

[@Levityy]

What are the specs for the servers? Ram, cpu etc?

[@KodeKloud]

Hey, please refer to community.grafana.com/t/hardware-requirements-for-running-grafana-loki/54790/3

[@dragonball41876]

Very good video, Thank you for making it!

[@KodeKloud]

Glad you liked it!

[@prateekgoyal3353]

amazing!

[@Vnehadalassss]

This is awesome, can you also make video on installing loki with helm chart ?

[@KodeKloud]

"Hey, thank you for watching our video. We will definitely look into your suggestions. Do subscribe and stay tuned for updates on our channel. Cheers :) "

[@laduveYT]

@@KodeKloud Hey i would also be really interested on this topic , even using minikube aand grafana /loki

[@emiljaquinjoy5811]

Great video!!! is there any way to store the logs in S3 bucket instead of local file system

[@therealslimaddy]

I think you need to rewatch the video

[@emiljaquinjoy5811]

@@therealslimaddy Yes I saw that part I tried it but I am unable to ship logs to S3 bucket

[@KodeKloud]

Hi, thanks for watching our vieeo. You can please refer to bluelight.co/blog/how-to-install-grafana-loki

[@YordisPrieto]

Low key, really cool! You see what I did there! HA HA! Dad Zoomer 👴 joke!

[@KodeKloud]

Hope you enjoyed our video! We have a ton more videos like this on our channel. We hope you will join our community!