Search engines neutral? Is this post sponsored by Google?

Show some examples so we can enrich the discussion.

Why not just have an AI that uses the search engine? Why does it have to replace it?

Search engines are also designed and maintained by humans. They absolutely can filter the results to fit the agenda of creators. There were many examples of that with the Google vs China situation.

There isn't some great fairness ethos guiding the works of search engine creators. They can manipulate the results just as much as AI creators.

Because fundamentally a search engine can provide contradictory and self-excluding answers as is naturally gathered from people while AI will reduce that to a single yes or no. And that's the actual problem.

It's more about the theoretical side of it. Because fundamentally a search engine can provide contradictory and self-excluding answers as is naturally gathered from people while AI will reduce that to a single yes or no. And that's the actual problem.

Search engines are also designed and maintained by humans. They absolutely can filter the results to fit the agenda of creators. There were many examples of that with the Google vs China situation.

There isn't some great fairness ethos guiding the works of search engine creators. They can manipulate the results just as much as AI creators. It's all just some algorythms. All human creations are subject to the biases of their makers.

You seem to argue that AI creators are more likely to introduce bias to their algorithms.

Not talking about correctness here but rather about reducing user autonomy by taking away choice.

You could ask the AI to give you 10 answers instead of just 1 right?

That's not going to give you the opinion of 10 different people though. Just 10 answers from the same source.

Let's put that into context. Would you like to replace reddit with AI?

“Give me links to 10 most relevant websites that support or contradict what you just told me”. How’s that?

Again its not you choosing them arbitrary.

Users are lazy and will happily trade autonomy for automation.

That's another topic which doesn't contradict with the core statement. We are trying to figure out if having something decide for you what to think is acceptable.

Type the same thing into Google and Yandex and tell me there is no bias

The fact that you can compare it precisely confirms the freedom of choice that we are talking about.

You're getting it wrong.

AI is an optimization tool. It won't be used as a search engine, but it might solve optimization problems within the search itself. For example, it could be used to search more potent nodes and alleviate the load.

AI is not a black box. We know what's inside, and we build it with specific functions and layers in mind. The people in charge of building a search engine are likely in touch with the technology.

Datasets don't necessarily need to be large. "Large" is all relative, it depends on what you actually study. Images and videos are heavy. It is filtered and augmented if the size is not sufficient.

Every search engine, by design, is biased. You just accept the bias from the current one because that's all you've ever known. You don't accept another bias because you won't read how it was built. I seriously doubt you have spent much time on the search index to find out its biases.

AI could be useful in order to classify search nodes, but a search engine, by design, needs to be quick. If there is an AI, it must be sufficiently fast, and thus sufficiently small, to run within milliseconds, multiple times for each search.

An AI may not be fit for this kind of problem. AI requires the problem to be established within a (usually) twice differentiable space because of backpropagation. A tree search problem is typically non-differentiable in nature, and derivative-based methods yield poor results. You can throw money at a dataset problem. You can't throw money at mathematics.

I work in SEO and you can pretend all you want but 30% of searches are going to be obsoleted.

Things that require pictures, videos and actual up to date products are not.

Let's check ahrefs and see.

Let's use the keyword "Why" so let's see what people search for.

• why is russia invading ukraine
• why is the sky blue
• why were chainsaws invented
• 13 reasons why
• why women kill
• why is my poop green
• why are flags at half mast today
• why does russia want ukraine
• why does russia want to invade ukraine
• why are gas prices so high
• who paid the largest criminal fine in history and why
• why were graham crackers invented
• why am i so tired
• why do dogs eat grass
• why does my stomach hurt
• why do cats purr
• why is my eye twitching
• why do cats knead
• why do dogs lick you
• why is my poop black
• why can't i sleep
• why do dogs eat poop
• why is my period late
• why is there a formula shortage

90% of these can be answered by an AI to a very good degree and this is just the highest volume ones. Theres plenty of low volume keywords nobody searches for and some that don't even have answers. And the AI produces the answers.

So you'll see a proportional drop in traffic to search engines, that in term is going to lead to a proportional decrease in revenue from ads, that in terms is going to lead to a decrease in profits from google, that in term is going to lead lead to news articles calling google "obsolete", that in turn is going to make the ad agencies not trust that their money is well spent on googles ads and the circle turns.

It will stop when the market has adjusted itself to the new reality that all simple easy to answer questions can be answered by the AI.

What will survive is news, reviewers with up to date product reviews, eCommerce sites, blogs and stuff that have their own content and aren't made by content farms and so on...

Search engine results contain no bias?? Someone is very confused…

[deleted]

You still have the choice though.

The question in discussion is freedom of choice for the user. There is no freedom in receiving a single answer to anything.

The question in discussion is freedom of choice for the user. There is no freedom in receiving a single answer to anything.

Whole business model of most of currently existing search engines is based on presenting you results which will point you into direction of their business partners, buy stuff they produce, etc. That's business model of Google, Bing, and many more.

Compare search results of those two mentioned above. Then add duckduckgo and see

There you go - choice. Which doesn't exist with AI.

I have a strong feeling that there really isn't a choice and there is no free search engine nowadays. It's not viable for anybody to do that for free.

Can you host a website and get it indexed? Is that not free choice still compared to AI which cites only official approved sources?

Having a single source of truth has always been called dictatorship and censorship.

Read about SEO and answer yourself if you can index your website high in the ranks so easily. Most of search engines results ranked high are paid, so therefore are "officially approved sources".

Regardless of which, the term AI in this context is highly overused.

The choice is there. I am not saying search engines are the best. All I am saying is the concept of user crowdsourced content will always be more real than an algorithm choosing from approved sources only.

I guess I’m going to agree with this stance because I’ve been using both chatgpt and of course, google. Google has been and continues to be a wonderful “office assistant”. Google Search differentiates from ChatGPT by providing us with the resource links we are asking about.

I’ll continue to use google to pull up Academic Research papers, but I will say one thing: Ive been using google less than I was before (not an insane amount but less nonetheless) chatgpt showed up. Google is still my preferred method of learning material because I enjoy pictures while I’m learning, and that’s what you’ll often find inside links (tables to visualize the data etc). Additionally, as noted, I do enjoy google uses user search results to help provide accurate responses. ChatGPT can 100% give falsified information lol.

I would really enjoy a smash up between chatgpt and google. Something that summarizes what we are asking, textually, while also providing resource links.

It depends on what the AI optimization function is. The AI has been trained on a large corpus of data (scraped from the internet I guess?). Search engine has also been trained on a large selection of data from the internet. So what’s the difference ultimately? The search engine ranking algorithm also has biases when you go deep into it. There is an infinite variety of queries you can come up with and a rather limited training set of data behind it. Google themselves said that even after all these years 15% of queries are completely new queries. So for those new queries it’s likely you’re just getting some random results which could very well be all wrong. Same issue with Reddit … if you ask some common questions you get great answers. But ask something nontrivial and all you get are random and misinformed answers. The Reddit ranking function is also inherently very susceptible to bias because it depends on people to upvote or downvote without knowledge of whether the people voting actually know anything about the topic.

The real problem starts when we have an extraordinary situation where people have no idea what's going on and the only answer you get is from wikipedia and officials. That's when we're in big trouble when it comes to free society.

>The AI has been trained on a large corpus of data (scraped from the internet I guess?). Search engine has also been trained on a large selection of data from the internet. So what’s the difference ultimately?

One is an index where ultimately you decide what to read while the other one is a crystal ball with no alternative options.

This is laughably off the mark. Search engines are indeed aggregates of sites and info, but they have all sorts of human bias and curating.

Just look at google images.

>Search engines are a neutral collection of hyper links

Bahaahahhahahahhahahahhahahahhahahhahaaaa

breath

aaahahhahahahhaahhahaaaaaa

So is any personal opinion but you have multiples of them and this is how you decide.

AI can also be crowdsourced and managed in the same manner as search engine. Problem is bias is inherent to both of them. Webcrawler which indexes webpages for search engines with weighted algorithm giving your search engine results, and "AI" crawling web to find those resources for you (replacing natural language into query for weighted algorithm) is basically the same.

Yeah I'm sure a lot of people are thinking about freedom when they google any of the questions or answers they need. Espeically stuff like "Why is the sky blue"

Some answers are not dependent of freedoms, emotions or anything. They're simple truths.

Why is the sky blue? Light refraction. There no freedoms to this.

Even if the index puts an article on page 100 I can still read it. With AI I have no choice.

Because those ad budgets can only go to one or the other at a time.

Type in White Family into Google Pictures. See what pops up.

The do the same in DuckDuckGo or Yandex.

There is no secondary answer to Why is the sky blue. That's what you're not getting.

But there are secondary answers to many questions.

That's exactly the choice you don't have with AI.

And an AI bot can give those secondary answers just as well.

With crowdsourced open "AI" you would have too, given enough drilling from your side.

What you are trying to imply is that in search engine you see the results, whereas in "AI" you don't have presented where it found that information. So those should be presented in "AI" and the problem will be solved.

The point is, you already don't have that choice, there are pages which are indexed so low, you not gonna find them.

Yes officially approved and stamped right. The single source of truth. The dream of any dictatorship.

That's still choice and I use it.

Why do you think there will be one centralised AI and not one centralised search engine?

It's not about AI vs search engines it's about indexed user content vs officially approved single source of truth.

Search engines hide many pages

We are comparing the concepts not the behavior of providers. Indexed content choice vs an algorithm choosing the source for you.

You think you’re getting the full set of possible search results from which you then freely choose … but that’s not how it works. They have a ranker behind the scenes that decides what you get the see.

We are comparing the concepts not the misbehavior of providers.

And you can have this choice with "AI", just write one with open and transparent dataset.

Yes conceptually the search engine is better in that way. But we don’t live in a conceptual world. We are very much exposed to the misbehavior of providers so we cannot ignore them in the evaluation.

Of course, I agree with that and new search engines without caveats are built such as - https://ipfs-search.com/#/ which is not centralized. If the concept is alive new providers will come.

This doesn't defeat the problem with something making a decision on my behalf.

AI does not have to give a single answer to anything. This is a design choice. Artistic AIs usually give multiple answers because this is relevant in this field. Chat-GPT gives a single answer because multiple answers was deemed not relevant.

This has basically already happened. Pure indexed user content does not exist. Search engines already use algorithms as well as manual overrides to control what information you receive.

Some small groups of people will try to preserve the truth but they will probably be subjected to increasing persecution until they are snuffed out. Not looking good for humanity in the long-term

Spot on. The content still exists - https://ipfs-search.com/#/ and it's not totally doomed as long as we value it.

Yet it makes the choices on my behalf which is the core problem.

Chatbot with set of answer generators won't do it on your behalf.

How to explain it better. I don't need answer generators. I want raw content I can analyze myself.

Then use search engine and believe in misconception of those results being unbiased and possible to find

I don't need to believe anything else but the concept. Everything else is fixable and new providers are solving this.

You just need to ask it to give you multiple answers and it will

I don't need answers I need raw information to analyze myself.

Then ask for a hundred and pick 4 you like the most. The whole point of AI is to lessen the tedium. If you want multiple results, you ask for them. The search engine also gives you a set of links it deems most relevant to your inquiry. It pre chooses the things you pick from based on its programming.

That's what any search engine can do as well. Why do you think search engines aren't censored?

Putting it this way I can see more clearly the similarity between both and the fact that maybe search engines are already using AI.

They proved that two people doing the same serch were given different results based on location and what was perceived as the political views of the searcher

"AI, please give me raw information from 4 sources about that topic I am curious about."

Why do you think AI is going to censor the output, but Google won't censor the links it shows?

Because they give me a list and not an answer but as per the other convo they are closer than I think it seems.

[removed]

Because a search engine is supposed to return all results while AI will aim for an answer first and foremost.

It's not that AI is bad and search engines good. Both are subject to same biases. And in the future AI is going to get heavily monetised and probably turn just as shit as Google with ads and useless content everywhere.

Not putting labels. All that the topic is saying is that AI and search index are not the same thing.

Except AI is going to be exactly like the current landscape of search engines. There going to be one dominants ones and a ton of competitors that can never catch up.

Off-topic since we are not discussing search engines but rather the concept of search indexing vs AI as an assitant.

you can still use google

But if the answer is to return a list of information, how does this not fit the bill? I've used ChatGPT as a search engine for like a week now and while it isn't always great on the first response, it almost always gives me a synopsis of Google's top results and I can easily pivot on scope with a follow up.

Can you elaborate further on your experience? Does it produce the same results? Where does it get the index from?

If I do a search with perplexity.ai and then google perpelexity gives me an answer and references not search results.

[removed]

OP is trying to make a point, but doesn’t know how to properly convey that point. First his title references “search index” but then switches to “search engines” and further tries to dumb the process down to “hyperlink crawling”. These are specific terms with different uses. That’s why everyone is jumping down his throat instantly.

OP is also taking A.I. composition at its current face value and not looking even a few years ahead. Even at its current state, A.I. can be hyper focused to generate vastly different results. OP just doesn’t understand how to use it properly.

Example, if you wanted to bake cookies in 1980, you would generally open a cookbook (maybe you had 2 or 3 and maybe a box of index cards). You would be presented with the same recipe and create the same cookies, because you limited your recipes to only your kitchen. With the internet, now you have access to millions of cookie recipes. OP is stuck thinking that A.I. is only using 1 recipe. Just as with most internet searching, you need to ask properly or else be given canned results.

“Give me chocolate chip cookie recipe from Betty Crocker 1985, substitute sugar for Truvia”. Bam…A.I. just showed you the tip of the iceberg. Google will show you separate results for the individual keywords and make you go down the rabbit hole. A.I. is basically asking someone else to Google search a bunch of stuff for you and compile a proper response…in seconds.

Anything else about OP Mr. OP?

All this has been basically answered by the attached perplexity description. No need for cookies, cookbooks and OPs.

>2. Search engines are a neutral collection of hyper links

You want it to, but it isn't.

[removed]

You can pay to have your links shown above other ones, the introduction of biases in the result of search engines is actually their business model.

Oh no. You have to search for particular keywords or phrases, even just in your mind, to find the search results you want to find. With an AI like chatgpt, you can stop lying to yourself and find what you want, by prompting what you want to see. After all, knowledge is purely what you want to believe, not what everyone HAS to believe.

Well, my experience is admittedly limited as I've only used it where I work since the beginning of last week. I have only double checked Google's top results a couple of times to see the content of the results vs what I received from ChatGPT but it was largely the same. Granted, these instances were code requests where I was asking for methods to do a certain thing using different namespaces.

Whenever I have asked ChatGPT for suggestions or a list of ideas, it always obliges and gives me a lengthy, bulleted list. It is unable to provide links but it seems to bypass that need by extrapolating the information from top results.

It certainly can't replace Google now due to dataset limitations but since I've started using ChatGPT, my feelings have remained the same regarding it's use case. It feels and behaves like a more intuitive search engine and I could absolutely see it replacing search engines in the future. There isn't really a need for a collection of URLs when you can skip that step and immediately get the content you are looking for without the browsing.

Could my opinion change with further use? For sure. I am really only using it for mundane coding at the moment.

Yeah, the criteria that Google uses to evaluate its search results are documented publicly and in exhausting detail right here:

https://static.googleusercontent.com/media/guidelines.raterhub.com/en//searchqualityevaluatorguidelines.pdf

If you have a week to spare, you can read through and decide which principles you agree with and which you don't like. And you can call them either "biases" or "principles" depending on your mood.

Either way, I do not think there is yet anything remotely this comprehensive (and public) guiding the behavior of language models.

What answer? What even is the question? Your entire post was a poorly thought-out and worded declaration. You made a conclusion and tried to support it with other wonky conclusions, but with no meat.

yeah I kind of agree, you need to be able to see where you’re getting the information from, unfortunately I think just taking the ai’s answer at face value is easier for most people so it won’t matter

[removed]

I mean, by OP's logic, you should never ask anyone for any objective facts. Every human being on the planet has some kind of bias. intentional or not.

AI is ever-developing. At this stage, the risk of consequential bias is indeed pretty high. Over time, though, I expect this to diminish dramatically to the level of inconsequence.

>AI is trained on large datasets and injected with the bias of its creators

Search engines are a neutral collection of hyper links

I'm sorry but you can't be more wrong on these two.

I get what you are saying. Search results you can pour over multiple returns and compare and contrast to draw conclusions. An ai question answer engine gives you one result and you have to take it upon yourself to verify. I guess a parallel to look at is social media: TikTok/YouTube shorts are the most popular content delivery format atm and it is essentially deciding for you what to consume. So I’d say the majority of (young, probably) people are definitely willing to allow a machine to do their thinking for them

[removed]

I think after a little while, ai should be integrated into searches rather than replace it.

Let's say I type a question in on Google. Everything appears to be the same, but with an additional link to AI's answer to your question.

For one, ai is expensive in processing power. So it only processes it when you click the link. This would be more efficient than it it having to process an answer for every search.

For 2, people are using chatgpt regardless. It will just be easier and cleaner having it built right into a search engine.

You can very simply word your question with ai to have it provide references, multiple choices based on any parameter you choose and to provide counter arguments. It’s far more effective in my opinion than a search engine. Ask a search engine about a polarized topic and it may not deliver biased results (but sometimes does!) but the sites are primarily biased content in polarized topics.

This morning I googled 'is Athena Ishtar' and the first hit was for Ancient Origens an anti-science, for profit, borderline racist, lie factory, does that count? Are you saying Google picked that because it was the best non-biased resource? Definitely wasn't a useful reference and came in above all the anthropology and anyone associated with science or facts.

So we can only ever have 1 AI company ever? What kind of logic is that?

Your initial argument is that AI systems won't replace search engines. Of which you are wildly wrong.

You then (laughably) assumed that search engines don't introduce bias. A completely absurd statement.

Now you seem to be arguing that there's multiple search engines companies but won't be multiple AI companies.

So you get tired from constantly picking up and moving the goal posts?

You can't just inject bias into an algorithm. They've just blocked certain types of questions and give a generic response largely manually written.

Google rarely gives me what I'm searching for these days.

Chatgpt is overall a bit better for specific questions, but not websites.