jcdl 2007 post 1: Keynote, Daniel Russell, from Google

[deborah]

So after several posts on my scholarship, here's a series of posts on the conference sessions from JCDL 2007. I'm not liveblogging the conference because there's no wireless in the session areas (on the one hand, WTF? No wireless in the session rooms at a conference co-sponsored by IEEE and the ACM? On the other hand, it probably makes for better framed responses from me not to be liveblogging).

Keynote, Daniel Russell, from Google

(clearly I am a completely non-standard google user. I use the cached link constantly, I go to google about 40 times an hour, I look below the fold and on "next page", I use "site:" and doublequotes ...)

Summary: New kind of literacy, fourth R: Research.

Sound bites:
First of all, there were some interesting sound bites that aren't part of what I took away the core argument

• On "Google" being verbed
  
  
  • "You only get to be a verb by doing something right," he says, though I'd disagree; of course you can also get to be a verb by doing something very, very wrong. Or, like Joss Whedon, just by being the person who is around when a community decides a concept needs a word ("jossed") .
  
  
  • He's heard "let me google this on Yahoo"


• His job is to figure out what searchers really want. (I would argue that in the last year or two, he's being overthinking his job)


• A lot of things people ask about are not in the traditional library space (and I would argue that a lot of the things people ask about are not on the open web)


• The word "first" is always in the top 50-most popular queries, worldwide. How odd.


• How does always-available search change conversations in the real world? (Certainly this is my life. "Augmented conversations," he calls them, and though I don't do it with Google Mobile, my socializing is often framed by searching. Though I use OED as often as Google, because it's more fun. Except for how wonderful it was to find the Great Square Dance Conspiracy)


• We have changed expectations about knowledge. For example, we don't need full references anymore (though I'd counter that with now we reference everything instead of bs-ing so much; instead we just cite sources online that may be full of bs.)


• We know it's not true that "you don't need to know something; you just need to know how to look it up",
  
  
  
  • But the cost of discovery and access is very low, and
  
  
  • does this change our expectations of others?
  
  
  • (note my thoughts the changing level of acceptable inaccuracy in the information-soaked world; for example, a piece of fiction which references the inaccurate name of a First Nations tribe and clearly false tribal traditions would probably get a much harsher response in the post-Wikipedia era)
  
  


• Constant change in search technique for skilled searchers

How does Google figure out what users want?

• Google gets a weak info signal from users on aggregate (on average in north america, less than a query a day). Wow, that's low -- I use Google as a bookmarking system so I can't even imagine a usage that low.


• Most results are adjusted by machine learning


• Remember the attention economy.
  
  
  • So just because a user reads aloud "news results for [query]" doesn't mean she knows she's reading news; it's not a UI problem, it's an attention problem.
  
  
  • One thing he didn't point out from his transcript is that his user was easily distracted by irrelevant shiny. (eg. "Are Filipinos ready for gay movies?")
  
  


• "I'm trying to get every Google user to become a literate user of the Google interface"
  
  
  • This counters the library conventional wisdom of 2007 (with which it is no secret I disagree) -- that Google is 'intuitive' and we shouldn't teach users how to use UIs, we should hand them things they know how to use. Since even Google doesn't find its users to be literate users of its interface, and they want to change the users, not the interface.
  
  
  • Google does not want "emulating without understanding"
  
  
  • Google also has some major ui problems themselves (eg "recent articles" instead of "sort by date" in google scholar), though I think Russell thinks that's an untrained user problem. Which proves my point above.
  
  


• Google is trying to do good ui (eg. sort by date) with broken metadata

How Google learns from how we search, multi-perspective approach. They use the "3M point of view":

• Micro: millisecond (eyetracking studies)
  
  
  • lab studies are flawed because people do unrealistic searches and accept inadequate search results, so good only for eye-tracking
  
  


• Meso: minutes to days (field studies)
  
  
  • vital to study them where they do their work, not in a lab
  
  
  • discovery of emotional reactions (eg advanced search pages are scary)
  
  
  • studies before every ui change
  
  


• Macro: millions of observations over days to months (log analysis)

Dicovereries about users they've made:

• Users will go to sites they expect will not work because it's familiar (eg ignoring FTD in favor of 1-800-flowers, depite knowledge that the latter is pricey)


• Users will ignore data that answers their query (eg $19.95 for "under $20") because they expect the string they typed


• later results help users refine undertanding of earlier reults (eg "the old man's glasses were filled with sherry" -- eye map of the semantic reparse)


• longer queries lead to more careful reading of results; shorter queries are often navigational


• Some users have different searching behaviours
  
  
  • Users can multitask while searching
  
  
  • Users can mine into their data to harvest a series of results
  
  
  • Any one user with these behaviours will idosyncratically do it consistently
  
  


• many users now know not to go to google for certain things and instead go to zappos for shoes, expedia or travelocity for airline tickets
  
  
  • though that focus can prevent users from seeing improvements, changes in tool available
  
  
  • google as a company is troubled by this, but it seems reasonable to me
  
  


• users strongly believe that result listing order matters
  
  
  • users go to froogle or image or search if that appears first on the onebox on the page
  
  

users need deep knowledge to be good users

• critical reading of results page (search strategy, pure engine technique)


• information mapping, what's out there (eg. reverse dictionary)


• domain knowledge (eg. medical knowledge, plumbing knowledge) (nb from deborah: this is why I love MeSH -- it helps users with no domain knowledge refine)


• what's likely to be at a given site

NOTE: This is exactly what librarians want users to do (see my note on information literacy above). But we mistakenly believe that Google frees them from that need to think or be literate of the page or critical readers. This was not Russell's message -- he was speaking to an audience of computer scientists, not librarians.

Mental models:

• how do users think about what a search engine does?
  
  
  • keyword? full-text indexing? partial-text indexing? link anchors?
  
  
  • people either don't understand the question or have answers that can't cope with exceptions (eg googlebombing)
  
  
  • by and large, people have weak mental models
  
  
  • weak mental model don't matter. Users don't need to know how it works to get adequate results
  
  
  • (I would argue this; understanding the googlebomb effect -- or the blog effect, etc, not to mention the amount of closed web out there -- makes people much more critical readers of results)


• what does one need to know how to search?
  
  
  • relevance? keyword term frequency? layered index? spiders?
  
  
  • a lot of users do have an intuitive knowledge of searching (eg "I wouldn't use 'grandma words' to talk about good search terms)
  
  
  • do users have to understand what kind of content is indexed? to understand that, say, there's porn in them thar hills? Or that there's products, scholarly hits, news, images, ads?
  
  
  • results predicatablity: do users have to understand subtle results differences caused by, say, throwing in quoted strings? (nb: does Google understand clearly non-deterministic bugs in their ui, cf. Language Log discoveries?) (eg. power switch icongraphy problem)
  
  
  • we all become Cargo Cultists. We are users of magic
  
  

His final slide sez:

For the digital library community
	To educate our users in broadly effective models of
		research
		content
		organization
			...and how to evolve...

See my note on information literacy!