tom gruber, tomgruber.org

Progress in the user experience on the web, if we look at what the user has to do and what the rest of the system has to do:

breadcrumbs (just links, user does everything)

-then-> portals (user picks yahoo, yahoo does more work) -then-> search (user queries, search engine does work) -then-> room service (agents)

examples:

'sandy' is an email reminder assistant

'farecast' for airfare. suggests alternate cheaper flights, trends. Looked cool from the screenshot and description.

Tom remarked at the end that finally, intelligence and computation will be able to be what we compete with, instead of just having "brand bullies" :)

And, "each time AI does a job well, it always disappears"

twine, nova

Remember when you started using delicious? it took 5 mins to learn most of the functionality, but then several days to notice that this is really worthwhile and it's going to help a lot. I expect a similar, but stronger, effect from twine. You learn the mechanics of checking information in, then after doing it for a while you notice which of your former laborious tasks have melted away. I also have high hopes for systems connected to twine. It's like a more polished version of piggybank. And they're going to add in recommendations, which may bring the 'smarts' closer to what magitti or calo is doing.

check Nova's blog for slideshow

semweb says, put metadata in the data so new software can reuse the past work (naturally!)

seems very close to that friendlist thing from that other blogger i read, i forget the exact name

builds a 'semantic interest profile' about you. picks people/places/organizations/topics you're interested in

create a 'twine' (like squidoo lens, page about a topic). The twines had surprising urls: like http://twine.com/twine/my-house, right at the global level. Are the urls different depending on who's logged in? Or does Nova's own stuff just go to the top? :)

A bookmarklet opens a transparent frame right on top of an external page you want to tag. From there, it's like delicious, but gathers a bit more data automatically.

When he used the bookmarklet on an amazon page, twine pulled some more fields from the page about the book

on the marked pages, twine finds words and topics and makes the links

edit-in-place UI to fix the fields of the data it found; add more fields. like freebase

they do some auto-summary of text from a wikipedia page

query is like newegg power search (or most semweb stuff for that matter), pick a type, add your filters

email in your own items to your 'recent items' list, just like a ticketing system would accept new tickets. URLs in the mail get crawled and those sites show up in your items too. (calo had a more turbocharged version of this, where they'd go hunting for info about everything and build big profiles about users and stuff)

goal of twine is organization. is this automating my tasks? the users will reveal what is valuable to automate.

PARC magitti

Finally, some novel UI work on a phone-based UI. It looked really nice-- low on sparkles and icons, high on usability. The app itself (recommendations and guides for your leisure time) seems good, and it was amazing to see a Japanese paper-printing company looking for ways to get into new media. Feels like the only stories I hear in the USA are about old companies putting their effort in keeping their old businesses going (e.g. big oil). Anyway, there was some cool personal activity prediction stuff like where they look at your messages and your past trends to guess what you want to do -right now-. I hope to get into exactly that kind of thing on my home automation project.

the name = magic + (something) + digital grafitti

19-25 year olds have 2x as much free time as other youth (japan, at least)

important for them to know what everyone else is thinking

predicts what to do, e.g. 'eat' (when it thinks you're hungry based on time, place, your emails, your explicit queries). Nice.

it reads emails only to guess what kind of activity you're currently doing. 11% of the test email dataset had information related to leisure activities (which is all magitti cares about). That seems low to me. Maybe that's all the ones they were able to correctly process (or maybe there's something I'm not estimating right about the emails of 20-somethings in Japan)

look at your past behavior to learn your patterns of eat/see/shop/... They can make plots based on day-of-week and time. This is what I want for my home automation.

ppl want to use the phone UI with one hand. 6 big buttons surrounding the content

pie menu on the phone. 4 quadrants only, sometimes more narrow ones for the border buttons. They looked really usable.

see yelp-style ratings on businesses, takes your star rating as you look at the page. collaborative recommendation stuff

the action buttons were arranged like this:

'M' [camera capture] [settings] (some content here) [any [eat]] [your location] 'clock'

hit the lower-left one to change your activity from 'any' to something else. Even if you dont say anything, they still list good ideas from their best matches of your activity, place, reviews, etc

you can force the activity ('shopping for clothes') and it refilters.

From the QA session: "what does the next 10 years of AI look like?"

answer: "busy"

yahoo

The phone-photo-tag part of this demo gave the most feeling of "you are looking into the future of technology" of all the presentations tonight. The UI was not elaborate. Mainly, it's that your phone camera is helping you tag your photos in real time (like delicious, except it knows your position and millions of past flickr tags too) and it's readily presenting you with other photos of interest. Everyone using this would essentially be running their own little version of justin.tv (photos, not video). The heavily-assisted tagging helps you organize your photos, and therefore organize your memories. Valuable! The speaker mentioned an example of looking up where you last had dinner with that friend. Since it was so easy at the time, you would have taken a photo and tagged it with the friend and the restaurant. Problem solved.

flickr photo locations plus tags shows popular tags on the map. 'tagmaps' from yahoo research berkeley. pretty cool to zoom in and out. using 4M photos, last year's data

upcoming version has 30M photos. Sometimes, these tags annotate world maps better than the pros do.

autotag your vacation photos by using the place of the photo

see the 'fireeagle' project for how web apps can know your location

i dont have live notes about the best demos, since I had to change seats to see the screen. The phone app that shows various feeds of pics included "wallet" (the photos you often show people), "my wife" (the photos she's taking now), "any flickr photos tagged with 'happy' near this location".

when reviewing all the tags on flickr, they consider the time too so as to figure out which things that are actually events ('bluegrass festival') and not places ('the mission'). This is like a topic I got into at a semweb meetup once: with just the tags on delicious, could you produce the names of all the states and their capitols? (I think yes)

CALO

The calo express part of the demo was pretty nice. It's a much smarter desktop search that would easily beat whatever you're using now. Especially what I'm using now, which is nothing (and I've tried a few OSS projects a little). Things took a turn for the industrial-strength-awesome when it got into the meeting planning and recording features, mainly for the amount of tech they're throwing at the problem. The AI testing stuff was also amazing, and it helped connect the project back to real life: if they don't make a certain amount of progress in their AI evalutions, they don't get funded for the next year.

This is a big research project that covers CPOF (recently in a Wired article) and has some kind of cross funding and sharing with many other projects, including twine.

cognitive assistant that learns and organizes

SRI, darpa

includes Command Post of the Future

builds 'relational model' of user's world. not sure if it's rdf

guesses what emails are about, what tasks they go with. you give feedback

'meeting understanding'. remote people are in everyone's headsets. CALO writes transcript, action items, Q/A pairs.

when he comes to a mtg, calo knows what all the people have been doing

has some kind of chat bot for scheduling a meeting (and other tasks, apparently). you use limited natural language

AI uses 'probable beliefs', revises them as new facts come in. 'probabilistic consistency engine' can update knowledge with new facts.

each year, they test the system (like an SAT test) and it has to improve. questions like "what to do when tom can't make a meeting: A. reschedule; B. tell tom; ...". They compare the baseline untrained CALO to an instance with 16 users for 2 weeks, and note whether calo does better at the test after that learning.

they have a full self-contained office environment, and a lite version (used by DARPA). lite one has almost no interface

the lite version does: google desktop search PLUS nlp (!). calo found someone's home page, pulled number and address and job title. Noted the person's publications and web pages to see what the person does.

followup query: "people with expertise in learning" then ".. that work at SRI" to narrow it down

A query for "slides about iris" finds individual slides in past presentations. then you search for similar slides to a near-match. Apparently the normal desktop searches look for keywords and stuff in a whole .ppt, which is obviously not as useful.

make a new presentation just based on title. digs up all relevant slides

'preppak' for a meeting. finds all documents that are required or recommended for the meeting

in the meeting, you can watch the transcript, which knows the person since everyone wears a mic. Testing within the government

calo is a personal assistant, doesn't share much with groups. some things (e.g. meeting schedule) are shared. you dont reveal all your meeting time prefs, but the calos negotiate it