Splunk Commands: "rex" vs "regex" vs "erex" command detailed explanation(Part I)

This is the first part of the videos where I have discussed about regular expression related commands "rex" , "regex", "erex" in details.
PCRE Cheatsheet link : www.debuggex.c...
regex engine : regex101.com/
you can download the data and queries I have used from the below repo :
github.com/sid...

[@shalomsuresh]

Excellent work. Precise points and its added bonus that I got to know some of the splunk command syntax errors as well with you ;)

[@amys8471]

@@splunk_ml I'm trying to extract timestamp from string by using regex but no luck. Ex: 2020-03-27:34R8:34:6537Z Trying to ignore R and Z. Can u pls help me on this

[@paraskumar7123]

could you please help me, how to replicate dashboard from uat to prod, instead copy paste xml bcoz it is not working.

[@splunk_ml]

Ideally it should work until you have different indexes and knowledge objects setup in prod. What error you are getting?

[@devd6800]

very informative video..learnt alot...thanks!

[@poqbum]

You are the master, I've learned so much from your videos. Thank you for sharing 😍

[@dhakshanav956]

Hi, I have one doubt if one rex command matching two or more fields means how to get the all the fields with different column?

[@Mike-og4wc]

Thanks a lot for the video! That was a perfect explanation!

[@harikauddarraju2168]

Hi, thanks for the information Can you suggest the regex command for the random string "i. 0.6.8.b.5.4.d.d.b.9.e.1.c.5.c.3.4.6.3.0.a.4.8.2.0.0.0.0.1.0.0.2.ip6.arpa" . There will be random characters from i.0.6.8.b.5.4.d.d.b.9.e.1.c.5.c.3.4.6.3.0.a.4.8.2.0.0.0.0.1.0.0.2. we are trying to exclude the events that contains the above string. We have tried a lot but unable to figure out. It would be very grateful if you provide some inputs on this.

[@splunk_ml]

Question: 1.Will the length of the random string will be same always? 2. Will the format of the random string will be same? like X.Y.Z...

[@rajenderprasad1193]

Thank you, I have a question. I have an API.. Splunk event has URI : /api/abc/a1bc/v1/abcdefghApi/abcdef/v1/{prasad}.. so every call I get different name as parameter.. i am unable create a dashboard as it is taking as a single call everytime.. how can I group and see the stats for it..

[@splunk_ml]

you can do something like below, | rex "(?api\/abc\/a1bc\/v1\/abcdefghApi\/abcdef\/v1)" | stats count by api_base_url

[@suzilkhoja3384]

Thanks a lot, Sir for the video. Can you please help me resolve the below issue? I have an event: Session Type: SSL, Duration: 7h:18m:21s, Bytes xmt: 408659006, Byts rcv: 162000348, Reason: User Requested Is it possible to fetch all the data from above fields? Thanks in advance?

[@splunk_ml]

If your event has fixed format you can use the below regex to extract the data Session\s*Type:\s*(?\w+),\s*Duration:\s*(?.+),\s*Bytes\s*xmt:\s*(?\d+),\s*Byts\s*rcv:\s*(?\d+),\s*Reason:\s*(?\w+)

[@shaktijoshi2007]

Great Work, I researched many articles and videos but couldn't get the clarity. I now know something about Splunk commands and syntax and semantics.

[@rajivranjan9614]

HI Sid...i have a doubt if i need to extract the paricular term from a given field will this rex command will work..or it will work only with the _raw file.....suppose i have a field email id- and if we have to extract a new field from this with a company name can we do it..

[@splunk_ml]

Hi Rajiv, Yes it will work for a particular field as well. Thats why the "field" parameter. You need to specify the field name there on which you want to perform the rex command.

[@НиколайВолошин-я1й]

the best explanation thank you

[@joaomarcoscarvalho10]

thanks. Helped me a lot

[@bhavyashah1775]

Please cover a video related to spath. Would like to know what is spath input, path and output commands? Also, if you can create a video on lookups that would be much appreciated. Thank you.

[@splunk_ml]

Hi Bhavya, I will be covering all of the splunk commands for sure. Regarding lookup I already created for kv store and external lookup. Please have a look at my splunk development playlist. Sid

[@venky_1544]

hi Sidd i was using the regex |rex (?From:\s+)\s(?To:\s+) i have not used field = _raw what is the context of using _raw

[@splunk_ml]

Hi Prasad, By default rex apply the regular expression on _raw field. So field=_raw is optional, however if you need to apply the regex on another field then field= is required.

[@adarshmahi8019]

_raw carries the data indexed (per event) . So if we are extracting anything from a particular field , we can use fieldName but if that's not the case we should use _raw

[@kushagrajain6285]

We are using mode=sed to filter out events after they are indexed.. Can we do any such thing at the time of indexing also to filter out events that are getting indexed???

[@splunk_ml]

Yep we can do that using props.conf file...I have a plan to create a new video for that...please stay tuned...

[@KK-mp8yo]

Do you teach splunk?

[@snehalchikkodi7528]

Hi sir, unable to open www.debuggex.com/cheatsheet/regex/pcre this link from USA.please suggest any other site where i can access this type of detail information of rex .

[@splunk_ml]

Can you try the below link? www.rexegg.com/regex-quickstart.html

[@snehalchikkodi7528]

@@splunk_ml thanku sir.

[@ashok39197]

Nice content.

[@taruchitgoyal3735]

Hello Sir, Thank you for the tutorial. Can you please share meaning of ?P in rex?

[@splunk_ml]

its just use to create a named group. See this video, kzbin.info/www/bejne/rGLLe6SAadmMic0

[@taruchitgoyal3735]

@@splunk_ml Thank you sir

[@AnkitGupta-je8oj]

I am having issue with fields having multiple double quotes eg:"My name is "Ankit"" I want to preserve double quotes inside(could be any number) what would be the best way possible for that

[@splunk_ml]

Hi Ankit, Can you please elaborate your question. Are you trying to extract "Ankit" from that string or want to use that field for some other purpose. An example will be good. Sid

[@AnkitGupta-je8oj]

@@splunk_ml so I am trying to fetch data from postgresql using dbConnect app, and there are a few fields which have double quotes within text. For eg: I have a column Column name - DESCRIPTION Value - My name is "Ankit" After importing data into Splunk index, and auto extracting fields the value is shown as: Value - My name is Search query used - index = * | table * How should this be handled so that same search returns the full value? I have narrowed it down to tweaking props.conf file settings but not able to figure this out.

[@splunk_ml]

Please have a look at the below link. Its discussed the same issue you are facing. answers.splunk.com/answers/658833/escaping-the-double-quotes-when-ingesting-data.html

[@KevinAli66]

great tutorial

[@OPEJ1]

Thanks a lot!

[@kushagrajain6285]

does regex commands affect performance of queries??

[@splunk_ml]

It could based on the regex code u have written...

[@kushagrajain6285]

I have multiple server names in single event as an output of command and its in multiple lines. So when i am trying to run the field extraction for getting all server names it only matches one server name per event that too first occurrence where as in regex101 it matches all the server names . Is there a solution to that??

[@splunk_ml]

Can you send me an example preferably through email? It will be easier for me to understand your problem.

[@splunk_ml]

There is an input called max_match for rex. Can you try with that. I have given an example below. | makeresults | eval sid = "acdcswpinf5800 1 acdcswpinf5801 1 acdcswpinf5802 1 acdcswpinf5803 1 acdcswpinf5804 1" | rex field=sid max_match=20 "(?P(acdcswpinf580\d 1){1,})"

[@kushagrajain6285]

@@splunk_ml it worked !! thanks

[@fancywrong6405]

Thank you for this video! The results shof up, but my new field is not created. The command I use is ... | rex field=_raw "flashtime:\s+(?..*)+\s", did I make a mistake?

[@splunk_ml]

May be your regex is not correct. You can test your regex in rex101. Com.