CS 591, Guest Lecture

Jared Saia University of New Mexico

Class Outline

• The Abundance Problem • Link Analysis

• Clever Algorithm

• Clever Algorithm Analysis • Google Algorithm

The Two Types of Search Queries

Two types of search engine queries

• Specific Queries: E.g. “Does Netscape support the JDK 1.1 codesigning API?”

• General Queries: E.g. “Find Information about Java”

The Abundance Problem

For General Queries, we have

• The Abundance Problem: The number of relevant pages is far too large for a human to digest.

More specifically:

• There are now hundreds of millions of pages on the web • For most search queries, the number of pages that contain

the query words is on the order of thousands

• A person typically wants to look at less than five pages

What we want

One way to handle the abundance problem:

• Filter out from the huge set of relevant pages, those few pages which are most “authoritative” or definitive”

• To do this, we need a notion of what makes a page “authoritative”

An Example Problem

• If we search for all pages containing the word “Microsoft” we get back over a million pages.

• There is nothing about content of the page www.microsoft.com that makes it stand out.

– This page doesn’t use the term “microsoft” most frequently.

– This page doesn’t use the term “microsoft” most prominently.

– There is nothing completely unique in the textual content

of the page that identifies it as being authoritative.

• What is it that makes the page www.microsoft.com most authoritative?

Link Analysis

• Link Structure is one external measure of authority of a page • If page x links to page y, x is conferring authority on page y • There are some potential pitfalls here

– Links don’t always confer authority (e.g. they could be for navigation or to ads)

– There is a balance between popularity and relevance

The Web as a Graph

• Let each page in the web, be a node of a graph

• Let there be an edge from node x to node y iff there is a link from page x to page y

• This formulation allows us to bring lots of great tools of graph theory to bear on the problem of link analysis.

Popularity and Relevance

• Consider the simple heuristic for returning authoritative pages: of all pages containing the search term, return those which have the greatest number of in-links.

• This heuristic fails

– Many authoritative pages for a search term do not contain that term.

– Examples: “seach engines”, “automobile manufactures”

– Many pages which are not authoritative for a given term would be incorrectly identified.

– Examples: www.yahoo.com and www.microsoft.com would be considered authoritative for any search term these pages contained.

The Clever Approach

• Key observation: Certain pages point mainly to sites that are authoritative.

• These pages are called good Hubs

• Examples:

– www.yahoo.com

– “Jared’s Big Page Of Links To Cool Sites”

• Question: But then how do we define what makes a page a good Hub?

The Clever Approach

• We use a mutually-recursive definition of the notion of Authoritative and Hub-like

• Good Hub pages point to many good Authoritative pages • Good Authoritative pages are pointed to by many good Hub

pages

The Clever Algorithm

High Level Idea:

• Clever collects all pages containing the search term, or pages that are neighbors of pages containing the search term.

• It then iteratively propogates “authority” scores and “hub” scores

– All pages start with equal authority and hub scores. In each step:

– The pages with hub weight confer authority weight to the pages they point to.

– The pages with authority weight confer hub weight to the pages that point to them

The Clever Algorithm

Algorithm CLEVER: Input: Graph G (G has n nodes and adjacency matrix A) Constants: ϵ > 0, t(= 10)

1. w_{h}^{0} ← length n all 1’s vector, w_{a}^{0} ← 0, i ← 0

2. Repeat (a) i ← i + 1

(b) w_{a}^{i} ← A^{T} w_{h}^{i−1}, w_{h}^{i} ← Aw_{a}^{i−1}

(c) w_{h}^{i} ← w_{h}^{i}/|w_{h}^{i}|, w_{a}^{i} ← w_{a}^{i}/|w_{a}^{i}|

3. Until |w_{h}^{i} − w_{h}^{i−1}| ≤ ϵ and |w_{a}^{i} − w_{a}^{i−1}| ≤ ϵ

4. (Auth. nodes, Hub nodes) ← top t weight nodes in w_{h}^{i},w_{a}^{i}

What does this algorithm compute?

• What exactly does this algorithm compute?

• If E is the set of edges in the link graph, it returns the vectors w_{a} and w_{h} such that |w_{a}| = |w_{h}| = 1 and ∑_{(s,t)∈E} w_{h}[s] ∗ w_{a}[t] is maximized

• This is the same as returning the vectors w_{a} and w_{h} such that |w_{a}| = |w_{h}| = 1 and w_{h}^{T} Aw_{a} is maximized.

Singular Value Decomposition(SVD)

A = \overbrace{[ u_{1} ⋯ u_{m} ]}^{U:m×m} \overbrace{[ \begin{matrix}σ_{1} \\ & . & . & . \\ & & & & σ_{n} \end{matrix} ]}^{D:m×n} \overbrace{[ v_{1} ; ⋮ ; v_{n} ]}^{V^{T}:n×n}

If A is m × n , n ≤ m with rank r then A = UDV^{T} where U, V orthonormal, D diagonal with σ_{1}, … σ_{r} positive and nonincreasing and σ_{j} = 0 ∀j > r.

{u_{i}} called left singular vectors, {v_{i}} called right singular vectors, {σ_{i}} called singular values.

SVD is unique up to multiplicities in singular values.

Two Crucial Facts

Let u_{i} be the i-th column of U and U_{k} be the k × n matrix consisting of the first k columns of U. Then:

Fact 1 For any matrix A, max_{|x|=1,|y|=1} x^{T} Ay = σ_{1}(A) and maxarg_{|x|=1,|y|=1} x^{T} Ay = (u_{1}, v_{1}).

Fact 2 The matrix A_{k} = U_{k}U_{k}^{T} A minimizes |A − B| over all matrices B that have rank ≤ k. |A − A_{k}| = σ_{k+1} (A).

Link Analysis (Kleinberg, 1997 )

Input : Link graph with adjacency matrix A(= UDV^{T}) for pages relevant to a query. Output : Vectors w_{a}, w_{h} giving “authority” and “hub” scores for all pages in the query set. If E is the set of edges in the link graph, gives |w_{a}| = |w_{h}| = 1 maximizing ∑_{(s,t)∈E} w_{h}[s] ∗ w_{a}[t] (=w_{h}^{T} Aw_{a}).

Algorithm : w_{h} ← u_{1}, w_{a} ← v_{1} (this is same as limiting vectors of a weight updating algorithm).

Example Weight Assignment

Link Graph a b  c  hub  authority (0)a  a′(.89) (.89)b  b′(0) (.45)c  c′(.45)

Google

• Google does something similar to Clever but somewhat simpler.

• Google assigns pages “authorities” based on the first singular vector of a different adjacency matrix.

• This singular vector is related to the stationary probabilities of a random walk on the web.

• Google computes the authority scores offline so it is much faster (but possibly less accurate)