Archive for July, 2009

SIGIR ’09

Tuesday, July 28th, 2009

Placing Flickr Photos on a Map. They place photos on a map based only on the tags of those photos. They exploit both info from nearby locations and spatial ambiguity

When More Is Less: The Paradox of Choice in Search Engine Use. They show that increasing recall works counter to user satisfaction, if it implies a choice from a more extensive set of result items. They call this phenomenon the paradox of choice. For example, having to choose from six results yielded both higher satisfaction and greater confidence than when there were 24 items to choose from

Telling Experts from Spammers: Expertise Ranking in Folksonomies. They presented a method in which power early-adopters  score highly. I call power early-adopters those who promptly tag items that happen to then become popular in the future.

Good Abandonment in Mobile and PC Internet Search. ” Investigation of when search abandonment is good (when the answer is right in the results list – no need to open page). Good abandonments are much more likely to occur on mobile device as opposed to PC; varies by locale (looked at US, Japan, China) and by category of query. “Our study has three key findings: First, queries potentially indicating good abandonment make up a significant portion of all abandoned queries. Second, the good abandonment rate from mobile search is significantly higher than that from PC search, across all locales tested. Third, classified by type of information need, the major classes of good abandonment vary dramatically by both locale and modality.”

Page Hunt: Improving Search Engines Using Human Computation Games. Microsoft Game Helps Make Search Better
Called Page Hunt, the game presents players with web pages and asks them to guess the queries that would produce the page within its first five results. Players score 100 points if the page is no.1 on the list, 90 points if it’s no.2, and so on. Bonuses are also awarded for avoiding frequently-used queries.

danah boyd’s gave a GREAT talk titled ‘The Searchable Nature of Acts in Networked Publics‘. In it, she debunked 3 myths about social networks:
1. There is only one type of social network. NO! There are 3 types of net
1) sociological network  (created from sociological study)
2) articulated network (created from listing friends)
3) behavioral network (created from interaction patterns)
those nets are very different but we have a tendency to assume they’re the same thing!!!

[Student Project Idea] Test whether the 3 types of social networks are related to each other and, if so, how!

2. Social ties are all equal. NO. The context of those ties and how strong they are are two important aspects, for example. (we have been discussing why context matters)
3. Content is King. In the tweet ‘i’m having for breakfast…’, the content isn’t important at all – it’s all about the awareness of sharing an experience.
danah then argued that social network sites are a type of networked public with four properties that are not typically present in face-to-face public life: persistence (what you say online it stays online), replicability (content can be duplicated (and can be taken of out-of-context – often u can’t replicate context)), searchability ( the potential visibility of content is great), and invisible audiences (we can only imagine the audience).  This networked public creates a new sense of what is public and what is private. For example, young people care deeply about their privacy, but their notion of privacy is very different from that of audults. finally,  danah introduced few stats on twitter (5% of accounts are protected, 22% include http://, 36% mention @user, 5% contain #hashtag, RT 3% are retweets, & spam accounts are proliferating) and highlighted some interesting research points for the future: 1)  how to make sense of content for such small bits of text; and 2) how social search can exploit analysis of the  network of twitters,  of context, and of tie strength.

IREVAL ’09: Workshop on the Future of IR Evaluation

Monday, July 27th, 2009

I recently attended the SIGIR ’09 IREVAL workshop on the future of IR evaluation, where I presented a poster on evaluating collaborative filtering over time. The workshop began with invited talks from (IR-research superstars) Stephen Robertson, Susain Dumais, Chris Buckley (videolectures), and Georges Dupret, giving talks that drew on years of research experience. The workshop participants then broke into groups to discuss different proposals related to IR-evaluation, and the workshop closed with a group discussion about each proposal. As can be expected, this workshop brought up many more questions than it answered. Below I’ve transcribed some notes that I took during the day:

(more…)

Facebook Advertising With Your Pictures (+ How to Opt Out)

Sunday, July 26th, 2009

A recent post here discussed emerging technologies that can be used for advertising on the go- and the threat that they pose to individual privacy. It seems a similar case is now found in online social network sites; places where users volunteer personal information as they interact with their friends. As Daniele mentioned on twitter, a recent TechCrunch article reports on how Facebook now wants to move user information from the private to the public domain (in order to compete with Twitter?)

One of the small steps in doing so involves using your photos to advertise products to your friends:

Facebook occasionally pairs advertisements with relevant social actions from a user’s friends to create Facebook Ads. Facebook Ads make advertisements more interesting and more tailored to you and your friends. These respect all privacy rules. You may opt out of appearing in your friends’ Facebook Ads [..].

Interestingly, until I saw some facebook status updates like this one below, I had no idea:

ATTENTION! FACEBOOK has agreed to let 3rd party advertisers use YOUR posted pictures WITHOUT YOUR PERMISSION! To prevent this: Click on SETTINGS up at the top where you see the Log out link, select PRIVACY,then select NEWS FEEDS & WALL next select the tab that reads FACE BOOK ADS, there is a drop down box, select NO ONE. Then, SAVE your changes. ( RE-POST to let your friends know!)

I’m curious to see what kind of photos will appear, and if facebook measures any change in click-through rates with this feature. However, one of the points this seems to make is that a central aspect of privacy is not only giving users control over the flow of their information, but telling them where it may flow in the first place.

Recommender Systems @ SIGIR 2009

Friday, July 24th, 2009

There were two sessions on recommender systems at this year’s ACM SIGIR (held in Boston). Overall, it was a good conference- organised well, run smoothly. It became very quickly apparent to me (a first-timer to SIGIR) that this is a tight community of researchers; there were many hugs at the opening drinks. Here is a quick summary of the recommender system papers and a couple other noteworthy papers/events.

(more…)

Advertising on the go – privacy, privacy, privacy

Sunday, July 19th, 2009

New technologies make it possible to run “web-like targeting in the real world. For example, a system developed by Singapore’s research agency lets advertising screens detect the genders of passers-by: it will soon be able to tell how old they are, too. IBM has worked on systems that can scan a crowd and estimate numbers, demographics, and where people are looking. And now that facial recognition has become a consumer technology it wouldn’t be difficult to install a series of ad screens that tracks individuals as they move through a subway system or mall, greeting them at each turn with a particular message or character. It’s hard to see how opt-out could be efficiently implemented for billboards on the street – at least without a new wave of devices that can communicate their owners’ preferences to nearby advertising systems.” More here.

nearest subway app

Thursday, July 16th, 2009

The AcrossAir Nearest Subway App for the iPhone 3GS overlays nearest subway stops on the live image in front of you. You can hold the iPhone flat to see arrows pointing you to various subway stops or hold it up to see signs and distances for nearby stations. … They’re working on versions for London and Barcelona (more here)

“Nearest Tube” got Juniper’s first Gold Award for Mobile Apps. Meanwhile, the Silver Award was given to Tapulous, the developer behind the highly successful ‘Tap Tap Revenge’ and its recently published sequel, ‘Tap Tap Revenge 2’.

A Principal Component Analysis of 39 Scientific Impact Measures

Wednesday, July 15th, 2009

Paper by Johan Bollen, Herbert Van de Sompel, Aric Hagberg, and Ryan Chute

Summary:

Traditionally, the impact of scientific publications has been expressed in terms of citation counts (e.g., Journal Impact Factor – JIF). Today, new impact measures has been proposed based on social network analysis (e.g., eigenvector centrality) and usage log data (e.g. usage impact factor) to capture scientific impact in the digital era. However, among the plethora of new measures, which is most suitable for measuring scientific impact?

The authors performed a principal component analysis on the rankings produced by 39 different measures of scientific impact. They find that scientific impact is a multi-dimensional construct that cannot be adequately measured by any single indicator, although some are more suitable than others.

From the results, they draw four conclusions that have significant implications on the development of scientific assessment.

  1. The set of usage measures is more strongly correlated than the set of citation measures, indicating a greater reliability of usage measures calculated from the same usage log data than between citation measures calculated from the same citation data.
  2. Usage-based measures are stronger indicators of scientific Prestige than many presently available citation measures. Impact factor and journal rank turn out to be strong indicators of scientific Popularity.
  3. Usage impact measures turn out to be closer to a “consensus ranking” of journals than some common citation measures.
  4. Contrary to common belief that JIF is the “golden standard”, usage-based measures such as Usage Closeness centrality may be better “consensus” measures than JIF.

Using Data Mining and Recommender Systems to Scale up the Requirements Process

Monday, July 13th, 2009

Paper by Jane Cleland-Huang and Bamshad Mobasher

Summary:

Ultra-large scale (ULS) software projects involve hundreds and thousands of stakeholders. The requirements may not be fully knowable upfront and emerge over time as stakeholders interact with the system. As a result, the requirements process needs to scale up to the large number of stakeholders and be conducted in increments to respond quickly to changing needs.

Existing requirements engineering methods are not designed to scale for ULS projects:

  • Waterfall and iterative approaches assume requirements are knowable upfront and are elicited during the early phases of the project
  • Agile processes are suitable for small scaled projects
  • Stakeholder identification methods only identify a subset of stakeholders

This position paper makes two proposals:

  • Using data-mining techniques (i.e., unsupervised-clustering) to identify themes from stakeholders’ statements of needs
  • Using recommender systems to facilitate broad stakeholder participation in the requirements elicitation and prioritisation process

Early evaluations show promise in the proposals:

  • Cluster algorithms (e.g, bisective, K-means) generated reasonably cohesive requirements clusters. However, a significant number contained requirements that were loosely coupled. The probabilistic Latent Semantic Analysis (LSA) method was used and early results showed improvement in cluster quality.
  • Their prototype recommender systems generated discussion forums that were more cohesive than ad-hoc ones created by users, and were able to recommend a significant number of relevant forums to stakeholders.

The proposed open elicitation framework introduces some challenges:

  • unsupervised group collaborations
  • decentralised prioritisation of requirements
  • malicious stakeholders manipulating the system for their personal gains

IJCAI ’09 Workshop on Intelligent Techniques for Web Personalization & Recommender Systems

Sunday, July 12th, 2009

Yesterday, I attended the 7th IJCAI Workshop on Intelligent Techniques for Web Personalization & Recommender Systems, in Pasadena. The IJCAI-2009 technical program will start on Tuesday. Here’s a summary of the sessions during the day: (more…)

crowdsourcing goes mobile – the extraordinaries

Wednesday, July 8th, 2009

We’ve discussed the potential of the Mechanical Turk for social research. Now,  here is a new crowdsourcing service on mobiles – short video (below), description, download the iPhone application. They deliver skills-based volunteer tasks to people whenever and wherever they are available by mobile phone.

Also, yesterday Aaron Shaw “presented upon his research into the potential Amazon’s Mechanical Turk holds for social science and the culture that surrounds it.” (here). “In Shaw’s view, there needs to be

a more serious examination of the question. Experimental evidence of research suggest subpopulations of people who would respond differently. Some people will be motivated by doing good, others don’t care, want the .05. We need better ways to test. It’s situation-specific.

Last point: The use of Mechanical Turk for enterprise search.

Deconstructing “the Twitter revolution”

Tuesday, July 7th, 2009

Hamid Tehrani of Global Voices gives a sober assessment of the role of Twitter in the Iranian election protests. One of the issues he raises is the temptation to relay breaking news without verifying it. The open source Ushahidi project, which was initially developed to aggregate and map reports of violence following the Kenyan elections in 2007/8, has proposed crowdsourced filtering to deal with this problem. However, the question remains, how can the people aggregating and filtering first-hand reports determine what’s true? Does citizen journalism still require a layer of professional editors, experts and fact-checkers, or can all these functions be shared among the crowd?

Workshop on Complex Networks in Information & Knowledge Management (CNIKM)

Thursday, July 2nd, 2009

Dell Zhang of Birkbeck  sent us a CFP for this workshop in conjunction with ACM CIKM-2009, Hong Kong, November 6, 2009.  Paper submission: July 20th!

Enhancing Mobile Recommender Systems with Activity Inference

Thursday, July 2nd, 2009

Daniele had briefly blogged here about this interesting paper, by Kurt Partridge and Bob Price, for which I will give a longer review. Some of the techniques used in this paper could be useful for further research and even its limitations are interesting subject of analysis.

Given that today’s Mobile Leisure Guide Systems need a big amount of user interaction (for configuration/preferences), this paper proposes to integrate current sensor data, models built from historical sensor data, and user studies, into a framework able to infer user high level activities, in order to improve recommendations and decrease the amount of user tasks.

Authors claim to address the problem of lack of situational user preferences by interpreting multidimensional contextual data using a categorical variable that represents high-level user activities like “EAT”, “SHOP”, “SEE”, “DO”, “READ”. A prediction is of course a probability distribution over the possible activity types.

Recommendations are provided through a thin client supported by a backend server. The following techniques are employed to produce a prediction:

  • Static prior models
    • PopulationPriorModel: based on time of the day, day of the week, and current weather, according to typical activities studies from the Japan Statistics Bureau.
    • PlaceTimeModel: based on time and location, using hand-constructed data collected from a user study.
    • UserCalendarModel: provides a likely activity based on the user’s appointment calendar.
  • Learning models
    • LearnedVisitModel: tries predicting the user’s intended activities from time of day, learning from observations of their contextual data history. A Bayesian network is employed to calculate the activity probability given location and time.
    • LearnedInteractionModel: constructs a model of the user’s typical activities at specific times, by looking for patterns in the user’s interaction with his/her mobile device.

Activity inferences are made by combining the predictions from all the five models, using geometric combination of the probability distributions.

A query context module is fed to the activity prediction module to provide prediction data of the context in which the user may be interested. For example, the user could be at work when searching for a restaurant, but his/her actual context could be the area downtown in which he/she plans to go for dinner.

Authors carried out a user study, evaluating the capability of each model to provide accurate predictions.  Eleven participants carried the device for two days, and were rewarded with cash discounts for leisure activities they engaged in while using the device. The Query Context Prediction module was not enabled because of the short duration. Results show high accuracy (62% for baseline=”always predict EAT”, 77% for PlaceTimeModel).

Some good and problematic issues with this paper

  • the prediction techniques used are interesting and could be applied to other domains; moreover I think it’s useful to combine data from user studies and learning techniques as user profiling helps developers (and apps) to understand users in general - before applying this knowledge to a specific user
  • the sample size makes the user study flawed: 11 participants carrying devices for 2 days approaches statistical insignificance; weekdays/weekends is the first issues that bumps into my mind, just to mention one
  • offering cash discounts for leisure activities is presumably not the correct form of reward for this kind of study as it makes users more willing to engage in activities that require spending money over the free ones (e.g. EAT vs. SEE)
  • authors admit they have mostly restaurants in their RS base, which I think is not taken in enough account when claiming high accuracy. Given that the baseline predictor has a 62% accuracy predicting always EAT, a deeper analysis would have made the paper more scientific
  • one of the most interesting contribution of the paper is the definition of the query context module, which is unfortunately not employed in the study for usability reasons related to the its duration. Again, a better defined user study would have solved this problem. I question whether it’s worth carrying out user studies when resources are so limited that the statitistical significance becomes objectable. However, there is some attempt to discuss expected context vs. actual context which is potentially very interesting: e.g., a user wants to SHOP but the shops are closed, so he/she EATs. It would be interesting to discuss how a RS should react to such situations
  • user-interaction issues: the goal of the presented system is to reduce user tasks on the mobile; yet, this is needed to tune the system and address its mistakes; yet, one of the predictors uses exactly user’s interaction with the mobile as a parameter. It looks like there is some confusion considering the role of user interaction in this kind of systems (imho, I think that a HCI approach could improve RS usability and, consequently, accuracy)
  • the systems is not well suited to multi-purpose trips (e.g. one EATs whilst DOing, or alternatively SHOPs and EATs) and in this case predictions are mostly incorrect.

Free isn’t the future of business

Wednesday, July 1st, 2009

Few months ago, I wrote a post on “Why Free Isn’t the Future of Business” and concluded that:

Google is the only company making money from ads, and the remaining web 2.0 companies are struggling to find viable business models, and they are not making any profit because they are pursing Starbucks’ business model (full article, excerpt)

Along similar lines, on the New Yorker this week, Malcol Gladwell critically reviewed Chris Aderson’s book Free:

There are four strands of argument here: a technological claim (digital infrastructure is effectively Free), a psychological claim (consumers love Free), a procedural claim (Free means never having to make a judgment), and a commercial claim (the market created by the technological Free and the psychological Free can make you a lot of money). The only problem is that in the middle of laying out what he sees as the new business model of the digital age Anderson is forced to admit that one of his main case studies, YouTube, “has so far failed to make any money for Google.”