The Proper Way to Do Web Searches (Searx)

[@LukeSmithxyz]

List of searx instances: searx.space Or start your own: asciimoo.github.io/searx/

[@ivanguerra1260]

Man, thanks for put this. Do know any free software that tell you who are tracking you in real time and where your data goes ?

[@olelund6821]

What do you think about YaCy?

[@daverok1113]

Looks a bit complex to install for the average Joe like myself

[@samahuja1156]

@@daverok1113 Lightbeam Maybe

[@daverok1113]

@@samahuja1156 ?

[@scarab5Q]

This is just bloat, high IQ people just sacrifice a lamb and consult the oracles

[@MrEnvy999]

High IQ people consult Chainlink oracles

[@beron_the_colossus]

Does the lamb need to be black, because they are almost extinct. Can't really find them anymore

[@balajaca]

My oracle uses HolyC

[@holymegadave]

@@balajaca that's divine intellect haha

[@bjarnestronstrup9122]

Just talk to Satan, he has all the answers.

[@mopme9475]

Why use a search engine? just manually search things by typing random links

[@DavidJBurbridge]

Same. You should eliminate DNS too. Just memorize IP addresses. I used to grep addresses from a text file but then I realized that is bloat that I could do without.

[@arrtemfly]

a little "for" script wouldn't hurt i guess. 100% suckless!

[@censoredterminalautism4073]

I remember doing that BACK IN MUH DAY, as a kid, in the 90s, before I even knew about search engines. Remember AltaVista? Do you remember running Netscape on Windows 98? They don't make them like that no more... *sip*

[@xtdycxtfuv9353]

eze dude just make a html file with links to them.

[@dolorsitametblue]

www.freebooks.com/search?q=connective+finger+tendon www.sans.org/ www.learningweb.com/lessons/C-S-T/articulars/index.php en.wikipedia.org/wiki/Ligament_joint www.nlm.nih.gov/medlineplus/ency/article/0012496.html www.aoe.org/SITES/citation_index.html

[@goulash31]

Searx is awesome for finding obscure content. I've been able to find product documentation that wasn't indexed by any of the major engines. Saves a ton of time at work if we don't have any copies of used equipment on hand.

[@monad_tcp]

the major engines have a serious problem of amnesia, you can't find anything past 4 years, and I think its on purpsoe.

[@starllama2149]

This actually would convince me to try it out

[@MiroslavKosanovic]

DDG just announced they will be using their own judgement to show search results so I will be switching to this. Thanks!

[@JH-jk3nm]

same

[@redgeoblaze3752]

Yeah, I was hoping they'd backtrack after the backlash, but now they're doubling down, and excluding independent news outlets. Searx is pretty good so far.

[@MiroslavKosanovic]

@@redgeoblaze3752 when you utilize all that it has to offer it's even better

[@tardwrangler]

Same

[@arcticcircle9178]

@@redgeoblaze3752 What independent news outlets are you referring to?

[@whathd579]

>find a searx instace i like >set it as the default search engine. >get blocked for searching too much >mfw i have to find another (good) one

[@LaZZeYT]

Start your own, that's what I've spent the last few days doing.

[@mauc3102]

wait, is this a thing? Do you get blocked from instances for searching too much? Lol

[@whathd579]

@@mauc3102 yes, a have been blocked from searx.me and some other instances in the past for searching too much in a short period of time, it's only a temp ban though, but still it's very annoying.

[@salvatoreshiggerino6810]

​@@LaZZeYT That would defeat the whole purpose, all your searches would appear to Google et al. to come from the same place, negating any privacy benefit. What they should have done was connect the instances together, and made sure any user's search happened on a random different node.

[@salvatoreshiggerino6810]

@@mauc3102 They have to, because practically all search engines are rate limited and will ban the Searx instance if they get too many queries. It's obviously not in Google's interest to provide free search service to aggregators without anything in return. Which is why I can't really trust Startpage not to slip the exact same data to them under the table that Google would have received if I used them directly.

[@mega-flare]

I think pinterest should be blocked on every search engine

[@ursochurrasqueira]

based

[@donartmc7129]

pinterest is horrible, based comment

[@TimeTravelingFetus]

Agreed. The image search results are infested with that shit.

[@arrecifior]

Pinterest is horrible until you are actually using it. Which is pretty unfortunate

[@urbanumbra6170]

@@arrecifior yeah I use it sometimes just bc I like finding cool images but the search results on search engines are so annoying, they’re not even there half the time

[@GhostofTradition]

Bet it can't find my dad

[@holymegadave]

He went to buy some cigarettes haha

[@tropicaljupiter]

He’s in my basement. He misses you every day, but I just can’t let him out because he’ll snitch. Sorry m8.

[@socialelitist4256]

Had to fiddle around with settings for 15+ min just to figure out how to not take a million years to do a search. This is extremely based. I give this a default browser out of 10.

[@finnk1289]

Search engines are our most used tool - yet I've never seen one as fully-featured as this. I've used Ecosia for a long time to plant trees with ad revenue, but maybe it's time for a change.

[@charlieruppe7030]

Running your own instance would be nice since I've had problems with reliability on lots of instances. The problem with running your own instance is that you won't have a lot of other users using your instance, anonymizing your search results.

[@MostWanted17O1]

I am so grateful you showed me this that I'm even gonna put a comment for the algorithm

[@jimbarino2]

You should feel proud, Luke - looks like you have been noticed by Google and are in the process of being shadowbanned. I went to check to see if you had any new videos and found that I had been mysteriously unsubscribed from your channel... At first I thought it had been deleted, but I found it again with a search.

[@FreeMan-wz3hj]

Nice, open source and decentralized is the way to go. Just like invidious and peertube.

[@victorprokop2240]

>google images >goi

[@driden1987]

Thanks for this video, haven’t heard of this before!

[@fghsgh]

What they should really do, is just having a public list of crawlers (you can become a crawler by running the FOSS) and it (your browser, not some centralized server of course) should just ask each of these crawlers (if it grows and there is too many of them, the number of queried crawlers should be limited to a configurable value) if it has any results on your search query. Each crawler could crawl a different part of the web, with some overlap of course, and give you the results. The javascript in your browser would then sort it by relevance. The difference between this idea and what you're demonstrating here (as far as I understand), is that you don't choose which crawler you want to listen to. It listens to all of them. Otherwise, I doubt the quality of the search results would be any good. This would of course be slower but searches are fast enough anyway and it could show results in real time as it received them from the crawlers.

[@timh.6872]

This is a great case for some kind of IPv6 multicast. You ping out a request to a multicast address that propogates to the crawlers, they then respond with partial results that are collated locally. The only concern I have with that is it makes it very easy to snoop on requests, since the return address is your IP. You'd need a Searx-like proxy to enforce the privacy constraints we want in a search engine.

[@fghsgh]

@@timh.6872 In that case you can also just use Tor or something instead of a server specifically designed for web search.

[@abhishek.rathore]

@@fghsgh Tor hits the speed significantly. It wouldn't feel instantaneous anymore. But that's a small price to pay for privacy.

[@michealhall7776]

"Not because its slow because I'm in a cabin in the middle of the woods" - Unabomber

[@Supertimegamingify]

This is so good, I can see a lot of these features being very useful.

[@playitnextgames8613]

It seems strange when Luke Smith tells you the 'proper' way to do things.

[@xthebumpx]

Prescriptivist

[@davidr2421]

Podcast should really be called, "Or Something Like That"

[@中野梓-i4b]

"but whatever"

[@musicamonarchy3062]

he can't get ride of his zoomerism

[@Enthalpiste]

Searx is under GNU AGPLv3 license. So you're definitely running GNU/Linux!

[@mattk6343]

I prefer to call it GNU + Linux.

[@alessandroceloria]

Allow me to interject for a moment...

[@killertigergaming6762]

@@alessandroceloria noooooo

[@xNiqueish]

>talk about trust >use brave

[@xNiqueish]

and ofc reflink in the description

[@niclo3611]

Yeah, nobody said a single word in comments about Luke using Brave but hey we can shit talk about other shady-software we don't have ref links for

[@timh.6872]

Good web browsers are hard. I don't know if we have a proper privacy-respecting FOSS browser that can be inspected right now.

[@abubakrakram6208]

@@timh.6872 And yet, 90% of what most of us do is in the browser.

[@womenkiller71]

Elo socko

[@SubhadipGhosh]

Here's a fun fact: I watch Luke Smith videos just to read the comments afterwards. Some of them are so funny that they really make my day sometimes.

[@seronymus]

Same

[@JuriDerCoole]

How is this more privacy friendly? I still have to trust the provider of the searx instance. They could run a compromised searx version, don't they?

[@trinity_null]

Exactly, idk why noone is talking about this

[@yoshikhurazi1769]

True but remember - you can self host this as opposed to duckduckgo or startpage

[@JuriDerCoole]

@@yoshikhurazi1769 That's right, that's the only possibility to search in a relatively private manner

[@yoshikhurazi1769]

I mean this is all about risk mitigation at the end of the day - not risk elimination. Information theory pretty much gaurantees that there is no way to interact with the internet without leaving some kind of bit leak even if it might be for practical purposes, improbable to trace. Until we have the quantum internet - and even then - the idea of 100% private browsing is a fantasy.

[@noahadams7961]

There should be a system that randomly selects from the faster searx instances each time you perform a search

[@theoreticalphysics3644]

Didn't know unown made a search engine.

[@charliecarpenter2840]

Thank you so much! I have a search engine again, fed up with advertising directories masquerading. I wouldn't mind ads if the functionality was primary or even extant.

[@leviticus8930]

Is this available in Latin and Hillbilly languages?

[@acrxsls1766]

That's redundant.

[@rogerrabbid3702]

This seems pretty useful especially now that duckduckgo results do not seems pertinent anymore if I search for technical problems or songs. Thanks

[@woj95]

Good timing, especially because there are rumors that A`pple wants to buy DDG.

[@arcade5765]

o shit fr

[@alecstewart212]

Son of a fucking bitch

[@recarsion]

god please no

[@abubakrakram6208]

Have they actually? I haven't heard of any _rumors,_ though I know a lot of Apple users _suggest_ that they do.

[@TheILOVEPIE]

Link?

[@freezejr2000]

10:10 "forced meme sites, that's what I meme"

[@TimeTravelingFetus]

I'd never notice if he didn't correct himself

[@holymegadave]

I imagine luke friends having to show their TLS Certificates to enter his domains haha

[@solidsnake7256]

DDG might still be better for privacy. With Searx you either have to trust some random guy with your queries or directly connect to the google/bing/yahoo botnet with your host.

[@LukeSmithxyz]

There is quite literally no oversight of DDG whatsoever and they admit it. They openly run a proprietary SASS. Trusting just some random guy is much better than trusting DDG CEO Gabriel Weinberg who had a career in selling metadata and such before DDG.

[@ctrackthezer0]

Startpage was acquired by an advertising company so it is probably compromised

[@oscarrzga4615]

That company has been buying some other companies like private internet access VPN provider and founding other search engines.

[@HarashiKalou]

thanks for the heads up, time to ditch them

[@HarashiKalou]

@Mialisus Everyone is accepted, except gingers and vegans.

[@MerseYattle]

@@HarashiKalou surprisingly efficient and correct outline haha

[@popereptilian4054]

You typed “search” instead of “searx” in the link in the description. Thanks for the suggestion btw, i’ll be checking this out

[@jxsl13]

vim like keys DISABLED?!?!?!?!

[@AZEMBadlen]

Your videos are basically about the edge of software and an internet

[@Epsilonsama]

With DDG starting to glow I'm looking more into this.

[@tylersingleton9284]

I just want an old see through Macintosh with original Google with 100 pages to look through.

[@Vermino]

This aged like fine wine about DuckDuckGone.

[@minhajsixbyte]

Finally a real chad way to search

[@hardbrocklife]

Based creator showing me how to used based search engine. What a based day to be alive.

[@silvernode]

I set my instance up a year ago and it was pretty simple, just some python that runs a server and a web server with reverse proxy. Then it's just a git pull to update and restart the service. Did you ever set up your own instance yet Luke? Thanks for covering this project and bringing it to light.

[@user-oj3gb8nh2q]

Any comment on Andreas Kling's latest video titled "Why I use the Bing search engine" ?

[@bruderdasisteinschwerermangel]

HM that seems neat, I'll open an instance that only I use on a VPS that has my name attached to it. You know, for privacy!

[@duckmeat4674]

searx.github.io/searx/user/own-instance.html

[@duckmeat4674]

That's not how it works...

[@aha8969]

4:20 "because he Speechures, the.. speachures -.- ... the FEatures..." awesome hahaha

[@obsidiansiriusblackheart]

Thanks Ted this is great

[@i_sometimes_leave_comments]

Regarding the trust issue... You said that having to trust the search engines like duckduckgo means you have to just trust them. The same thing would go for searx. I haven't looked into it much myself, but you're basically trusting the people hosting those searx instances to no track your traffic. If you're using searx because you don't trust the servers hosting it, switching to searx wouldn't make a difference in the trust aspect. Even if it's open-source, the people hosting those instances can still make their own modifications not shown on the official sources. If you're paranoid, you can host your own searx instance, in which case you're trusting yourself. I do find the meta functions interesting and useful though. Being more customizable is a plus in my book.

[@BurgerKingHarkinian]

You put plugs in your books?

[@i_sometimes_leave_comments]

@@BurgerKingHarkinian *plus though I don't even read paper books nowadays

[@qwertyasdfgzxcvbiop2114]

i found tis video by random clicks random links > super optimized searches :)

[@gnul]

Is there legal stuff to consider when hosting it publicly in EU/Germany apart from the mandatory imprint and privacy policy?

[@tardwrangler]

Right off the bat: Disable flickr for images

[@kjlw99]

I am surprised you missed the vim-like(inspired) bindings plugin.

[@anatolystepanovichdyatlov1747]

Implying todays internet isnt bloat.

[@didntaskyou7884]

These searc hengines are amazing

[@santiagolerin]

Your Internet is faster than mine... I'm not in a cabin in the middle of the woods☺️

[@tmbawebdesign]

searx is cool & the future of search, but if you want to block websites, the most effect way is using a firewall, not a search engine.

[@lionelbriand7595]

Thanks. You always showcase good tools :D

[@stellarorbit1341]

This is amazing. I never knew about this.

[@Carl-hs420a]

it's odd, you can search up any guide for any game and there's always a brand new article out about it that's mere weeks old, it's creepy af

[@georgep.2352]

why creepy?

[@axipont8383]

How does this compare to YaCy?

[@gintokisakata7490]

I would use yacy more, but ist a shame that this stuff is Java … Also more participants are needed I think. The serch results are still not good enough.

[@slevinkelevra5540]

ohh it's like the one disk called the internet . they gave away free at walmart

[@mujina93]

I also would love to have blocking features! But let's generalize a bit: I would like to be able to be able to provide feedback on search results (something like a "Like/Disklike" button that I can click near every result), so that future searches will take that into account. For example, it happens to me that every once in a while I do a query that I already did in the past. Let's say the first time I dug in the results and found that the 4th result was the one with "the answer". Now, I would like to be able to "like" that result so that the next time it appears on top. Or whenever I search for other things, that particular website gets promoted. With a "Dislike" button I might also be able to tell my search engine "please stop showing me shallow results from Medium, or please never show again this shitty tech website full of ads and with no content as a result of my searches". This seems something that could be easily implemented on a frontend/client for search engines. Does anyone know about something like what I described? But let's generalize even further: it would be really nice to have a "upvote layer" on top of search engines results. For example, when I like/dislike a result on my client, I might have the possibility of deciding to share my preferences (possibly anonymized), and then this data gets processed somewhere (maybe some in some decentralized, non-fingerprinting way) and is used to show every search engine user things like the average vote/quality of a url, or the top queries for which a certain url gets the most votes, or even more sophisticated "reactions" than simple thumbs up/down, like "this site is shady" or "the content is very long" or "contains advanced topics". However, a naive recommendation layer would be more harmful than useful, because you might get strong biases on results from "unknown masses of people". For example. I might personally prefer to have scientific papers or some cool guy's blog posts or youtube videos appear on top rather than often-very-basic Medium articles, but masses of beginners might prefer to find the latter. Therefore, what I would like to have is a "personal selection filter" on top of the upvote system. That is, I would like: 1. that my up/downvotes have the absolute priority over the crowd's opinions 1. to be able to select people whose opinions I care about (in this case they would need to have a "public profile", i.e. they would need to be users that express part or all of their opinions in a non anonymized way), so that their feedback (and maybe the feedback of people they care about) on searches has a much higher weight over the shallow crowd's aggregated feedback. If anybody knows about anything like this, or about work being done in this direction, please let me know!

[@mujina93]

Found this: github.com/jivesearch/jivesearch , a search engine that doesn't track you and that has an upvote system. But it's not mature and it's currently on hold (the main developer accepts PRs, if you want to help).

[@tanyaanyaaa]

Я знаю, что тебе нужно, но не скажу.

[@oo--7714]

Bro that is terrible, and upvote function would be crappy

[@astolfo3403]

DuckDuckGo ruined forever, this guy gets it.

[@donihalim3239]

Hi Luke! Any plan on making video(s) about Regular Expression?

I do think it is safer to use Duckduckgo (especially through Tor) than using a random public Searx instance. Duckduckgo's whole image (and privacy policy!) is based on the fact that they are not storing anything related to you, so it would be quite outrageous if someone would discover that they are logging your search requests, for example. It is possible, but not likely. Meanwhile, if you are using Searx (which is not hosted by you), you are ultimately trusting some random guy on the internet, it's like downloading a handy free utility software on Windows (not very wise). Other example is that it's similar to using a free VPN (duh). I would say that using Tor is the best. I do not fully comprehend how it works, but it seems to eliminate every likely scenario which works towards accessing your data. So, Tor + Duckduckgo can't really go wrong.

[@LukeSmithxyz]

It can't be "outrageous" because it's literally impossible to ever find out what DuckduckGo is running on their server, which is admitted proprietary software anyway. There are many companies: like Telegram, 100% of VPN companies, etc. that are openly advertised as for "privacy" despite being shams.

@@LukeSmithxyz It's true that it is not possible for *us* to find out, but the company has a lot of employees, who should not find out about the privacy policy violating features of DuckDuckGo. Assuming that a lot of their developers are privacy-centric (that is a strong motivation for working at DuckDuckGo) and if any of them found out their secret, it would be a big media-hit and a lot of pocket money (privacy policies have a lot of strength in courts), it is reasonable to think that they wouldn't do such thing. Yes, it is possible that they will leak something, but much less likely than Searx, where that random hoster can easily put a console.log or whatever to their instance's code since nobody's watching it besides them. The difference between DuckDuckGo and Telegram is again, the privacy policy. Telegram has a lot of loophole in their policy ("Telegram is a cloud service. We will process your data to deliver your cloud chat history, including messages, media and files, to any devices of your choosing without a need for you to use third-party backups or cloud storage."), while "DuckDuckGo takes the approach to not collect any personal information."

[@RaziOrSomething]

I would use Searx but the UI is not my favorite, so I'll use Startpage.

[@jan-fransojklise6877]

Remind me of corpernic (or copernicus?) Circa 2003-2004. I stop using cause google got so good. This look interesting.

[@ShayanBanerji]

Luke Smith on the role !

[@nativechatter999]

Wait, is Luke actually in the middle of the woods?

[@trinity_null]

Yes

[@courier7]

No he lives in Florida. There is no back country in Florida. You can only truly live in the woods in the central USA and northern Canada

[@dougtilaran3496]

YUP ! No Algorerythmz. No waterways for his yacht

[@idcwstk]

By the mouse movement I'm pretty sure he is using a trackball.

[@Jango1989]

Thanks for the PSA!

[@tdtrecordsmusic]

one day we are going to have to make an open source search engine... I would do it if I knew how...

[@Miniek0]

There are open source engines.

[@franny231123DMT]

thanks bro, I been looking for this advice

[@arrtemfly]

nice profile pic dude

[@A_Box]

I for one stick with Bing and Google. Why? Because I noticed that over time it shows you stuff relevant to other stuff you searched. Bad for narrative (news, events, etc) but great for specific and technical topics.

[@AugustusBohn0]

you see tracking and filter bubbles as a feature instead of an anti-feature?

[@Crabbadabba]

@@AugustusBohn0 For most of the uses in daily life, perhaps, yeah.

[@Mx25a]

Thank you, impressive and good. I've found nice stuff using it right now.

[@TheRealFaceyNeck]

Brave has Dark Mode?! Awwww yisssss Looks like I found what web browser I'm gonna switch to once Waterfox Classic inevitably bites the dust. 🤬 [EDIT] Yes I'm aware Searx dark theme was shown. However, it's also true that Brave has a dark mode. [/EDIT]

[@gplastic]

Can't believe I missed this

[@Raccoonov]

great vid, thanks Luke

[@satyabenson]

nope, Brave also has nice built in features like tor windows, web torrents, ipfs support, etc

[@lukas-vaelinalsorna1834]

To be able to save the set preferences on the searchx site I guess you'll have to enable cookie saving (do not delete the cookie when closing the Browser) right? (or is this even with cookie saving not possible?)

[@justanobody2347]

Luke wat abt MetaGer?

[@TheGrmany69]

I've used a more on hand approach to searching before by using the options of "similar" site on Google and it helped me discover different ways to handle searches: including listing sites which sole purposes are to set similarities between pages rather than exact matches; but ultimately I came to the realization that Google crafts or modify results so you fall into desired funnels, those numbers you see at the bottom of a page are usually fake pages after the first 5 to 10 pages, full of spam or copycat sites from there on. This seems like a viable option, thanks for the info.

[@tux_the_astronaut]

This is a good search engine but not my default do to some bugs that will happen like sometimes when I search for something it will just keep giving errors saying page can’t be displayed it seems though to happen the most on Firefox and works better on Vivaldi

[@lokzarts]

You’re basically giving all your search information to this searx thing as a proxy. But again you’re facing the same problem of not able to trust them unless you host your own service

[@xGOKOPx]

Kinda offtopic but have you heard about Brave hijacking links you visit to insert its affiliate links?

[@neobscura]

Duckduckgo: !aur firefox (pretty cool)

[@generalmichaelconstantine4598]

Hey, Luke. Does SearX have search commands like intext inurl and others? I find myself using those pretty often.

[@MozkarLozada]

Did you find out?

[@CigEconomy]

@@MozkarLozada He died before he found out :(

[@Njinx_]

@@MozkarLozada It does accept search commands

[@hoshiya4522]

0:21 Think Mark, think

[@johnwalthall4937]

I don’t know if you have time to read/respond to commends but, do you use TOR and why or why not?

[@ViboraUnleashed]

What about self hosted whoogle search? :)

[@UCm0i6w5lBlRthCtZEoj99tg]

that chair tho

[@ianpan0102]

I'd like to use a search engine other than Google or Bing, but these two 'giants' just feel so fast and others don't even come close...

[@seronymus]

It's better now

[@victornetto4024]

I'm a little confused about how searx works. For now, i'm running a docker container with searx, and I should be searching safely. But, isn't my container searching on web with my ip, like if i was with any other searching engine?

[@DaemonJax]

If you're running your own searx instance, then... yeah, you lose ALL privacy benefits if you're the only person using it -- no plausible deniability. You run an instance for OTHER people to use.

[@perplexity000]

Would there be a way for you to output those search engines and hotkeys from brave? They look very useful

[@thorinbane]

Useful search engine thanks

[@Vermino]

9:20 - based feature... God what I would do to be able to remove/block websites from SERPs

[@BrK0511]

Try it now, it's noooo good.

[@sarundayo]

So the TRIVAGO of web searches then 😏

[@meomweow]

looks like the pokemon unown

[@GoldSoulTheory]

How do you feel about the Brave browser injecting their ref codes into serveral urls?

[@LukeSmithxyz]

How do I feel about them openly using referral links for their intended purpose?

[@taylorspinks8162]

Is response time dependent on searx to the location or your time to searx to the location?