Thanks!

Processors are optional. If they aren't specified, nothing happens meaning the data is not altered in any way. What you send in is what comes out again.

Prometheus can be understood as two different things. 1) The Prometheus data syntax. By this I mean the syntax of the lines of code that describe the metrics + the fact that metrics usually live at a standard endpoint. 2) The Prometheus "product". By this I mean the software that retrieves (or accepts) metrics, stores them, visualises them etc. Here I'm explaining that the OTEL collector (instead of Prometheus) can scrape the metrics (which are formatted like 1) and send them to a backend which isn't prometheus (2). My point about multiple receivers: Prometheus (the product) only deals with metrics. It doesn't do metrics or traces. But the OTEL collector process all of those signals. So... If your backend is capable, you can send your metrics, logs and traces to the same place.

Correct, the collector would be deployed on each VM (or at least in a location capable of retrieving the logs if the logs were on a network share (for example). One thing to mention is that you could use the OpenTelemetry collector builder (look for the “everything you need to know to get started with the collector” video on my channel) and custom build a collector with ONLY the filelog receiver, whichever processors you want (eg batch) and the exporter you need (eg. OTLP). The resulting binary would be tiny and thus a much easier “sell” to your boss to deploy in each VM

Thank you 🙂

Awesome, thank you!

Thank you!

Yes, Prometheus could scrape the collector or you could use the prometheusremotewrite exporter. TBH though, these days I'd be expecting any backend to be OpenTelemetry Protocol OTLP compliant and thus the OTLP(http) exporter(s) should "just work". I have no experience with VictoriaMetrics so can't comment

The collector and fluentd (sidetone, I'd recommend fluentbit over fluentd) are both the middleware. OpenTelemetry is not a backend (storage) for Observability data (logs, metrics, traces etc.) so you'll still need a backend system (ELK, Dynatrace, Datadog, Opensearch etc.). Would it be helpful if I made a short explainer viideo on "what is OpenTelemetry?"

⁠​⁠​⁠​⁠​⁠​⁠​⁠​⁠@@agardnerityes I guess that’ll be nice but I don’t know if a developer would find it useful or know what’s the use case I guess this is a very deep Devops thing which I am not I just wanted something to track logs in a nice way since I’m using docker and can’t keep running dog logs command I think I’ve been seeing this fluentbit thing as well but they aren’t a lot of resources like fluentd I didn’t even know this term flying around called observability but I guess it’s part of a broader name given to do certain things Thank you very much for your response also can I know which is the most simplest solution for my use case I just want to have a way of viewing my logs and nothing to heavy or resource intensive I’m currently dabbling with this elk thing and it seems packed with a lot of stuffs I don’t need and understand

Glad it helped!

I’m happy you found this useful. Let me know if you find any cool projects that I should review!

So glad you found it useful.

Sorry, I missed that. What collaboration?

I don’t add (and don’t recommend adding) large binaries to repos as it bloats the repo unnecessarily. There are many collector distributions available to choose from (that’s my next video about how you choose). But to get you going for now: download the appropriate binary from here: github.com/open-telemetry/opentelemetry-collector-releases/latest extract it and run it

That command is referencing an existing container ID because I had Jaeger running previously. Just replace 255 with jaegertracing/all-in-one:1.60 www.jaegertracing.io/docs/1.60/getting-started/

@@agardnerit Thank you so much. How can I start the collector up? Your code in github doesn't have otelcol file - I would appreciate your help.

You need to run Jaeger AND a seperate standalone Otel collector. It’s the standalone collector that you need to configure (via the YAML file) to listen on 6004. This is because jaeger already runs a collector on 4317 / 4318 so you can’t use those ports for your standalone.

Thanks. You're right actually. I missed that. I'll make a quick video explaining the different options soon. But in short, see this page: opentelemetry.io/docs/collector/installation/ There are many different distributions of the collector, the OTEL project produces a few that supposedly are optimised for particular use cases. Some vendors produce their own and you can even build your own. In these videos I use the contributors distribution. I downloaded the binary for my OS from here: github.com/open-telemetry/opentelemetry-collector-releases/releases/latest

@@agardnerit Thank you for your quick response. I will let you know how it goes, to provide some background, I was able to follow your kzbin.info/www/bejne/a3jHiJyuqNSjm8ksi=GIkjN1HisC-gSE6y video that let me visualize the traces. I am assuming this video will take it a step further. Am I not right?

Thanks for the suggestion, I'll put this topic on my backlog! The timing of your request is interesting as the Jaeger project is looking to move more towards the OTEL Collector. IMO using the OTEL collector is the future proof decision.

At 03:47 you can see I've download the otelcol-contrib collector (the OpenTelemetry contrib distribution) and am running it as a standalone binary. There are many ways to run the collector so I didn't want to go into each one. You're right though, for future videos I'll call this out more clearly. opentelemetry.io/docs/collector/installation/

Thank you. Please do suggest content you’d like to see and if I know enough about it, I’ll do it

Thank you so much. If you want specific content covered, let me know and if I know it, I’ll do an episode on it :) We’re all learning all of this together. Good luck!

Love the comment. It's one of the reasons I make this content. To provoke thought and discussion - so keep them coming :) I've always felt a bit uneasy with the projects use of that terminology. While technically, yes, spans are structured and traces are collections of spans. I feel like this oversimplifies and may lead new users (the intended audience for this video) to equate spans == logs. That would be a mistake because then they think: "oh well, I'll just use logs as I already do". You're not wrong though, technically. At the end of the day, Observability for me is the means to provide the ability to do your job (fix / prevent the issue, generate a report for the boss etc.). Whether that takes the form of a log line, or a span, or a metric, ultimately who cares (beyond those of us who live and breath Observability).

Thanks for the sub!

Glad you liked it!

Glad it was helpful!

I tried to make that clear in the video. Obviously I didn’t do a good job! Jaeger all-in-one image comes with its own collector so 4317 was already in use. It was also to demonstrate that even though 4317 is the standard, you don’t HAVE to use that

Great idea!

Yup! That's the very next video in the series. Coming soon. Part 3: Sending spans to Jaeger >> kzbin.info/www/bejne/qp_QnWmQmZprndU

Glad it helped