hypersearching, as discovered in class, is a powerful tool in attaining a fast and accurate result when searching for a query term. Although it is not implemented globally yet, it would prove to be a very powerful searching mechanism. While surfing the web for my own entertainment sakes, I have found several aspects of Korean web blogs that could be utilized as a tool for hypersearching.

 In America, the primary form of online communication between a social group is instant messaging and forums. In Korea, a collection of forum boards and picture BBS’s called “Cafe” is the norm. The cafe exists for a certain motive (such as fan clubs, campaign for a certain action, etc) and the members communicate inside the collective chamber. Web blogs in Korea is like the individual replica of the cafe. There are numerous blog interfaces: blogin (http://www.blogin.com/), Egloos (http://www.egloos.com/), Tistory (http://www.tistory.com/), Cyworld (http://www.cyworld.nate.com/), and et cetera. Individuals join one of these services and create their own little Cafe filled with their interests. Although two blogs may be provided by the same provider, it could be very different according to the user’s style.

 Since the majority of the “netizens” in Korea use a blog, search engines naturally search blogs automatically. The most used search engine, Naver (http://www.naver.com/), searches Web pages, Knowledge search database (a trademarked Q&A service), Cafe&blog, dictionary, image, movie clips, music, and news when instructed to search a term. most unprofessional information or personal opinions cannot be found on web pages (since the majority of the people don’t have knowledge of web design), but they could be found on blogs where it is relatively easy to post information.

 But why blogs relate to hypersearching is because of the function “trackback”. Let’s suppose that I was reading Jack’s blog and I found very good information about what I’m interested in. Let’s say… pizza. Then, I could refer to Jack’s blog in my post and leave a hyperlink. I’d say in the post “Yeah so I was reading Jack’s blog (link) …” blah blah. This isn’t just a one-direction link; in Jack’s blog, it will show that I have linked to his blog. Therefore if many people link to Jack’s blog (Jack is a good authority), there is no special mechanism required to calculate the scores. And since my blog shows the pages that I’ve linked to, the visitors can see my hub score too. And blogs in korea are categorized very specifically, so there is less chance of a high authority score with a vast array of topics.

 It is interesting to see that in Google, the primary function is to search web pages. In Naver, searching web pages is the least used function. Implementing these web blog strategies in hypersearching mechanisms would bring more light into the information searching studies.

BigChampagne Is Watching You

BigChampagne

In class we recently finished discussing the idea of an information cascade. One application for information cascades that came up was trend-setting. The idea is to jump start a trend (fashion, music, etc.) by targeting and attracting key individuals who have a lot of sway in the industry. An example could be a fashion designer giving a celebrity a piece of clothing to wear at a highly televised awards show. The hope being that people will see the item on someone who they like and decide that they too like the item. This phenomenon is well documented, a company develops a product and wants to generate buzz to sell it.

In reality this is not how most products are sold. Major corporations such as Coca-Cola and Pepsi do not sell directly to the ultimate consumers, they sell first to retailers such as supermarkets and convenience stores. How then does the retailer decide which product to sell when all the products are working as hard as possible to generate a trend for themself? This problem takes on extra significance in the radio industry as new music comes out weekly and often a city will have multiple pop stations all vying to gain listeners from the same fan pool. If a station could gain extra information about consumers listening patterns, it would be a major competitive advantage.

In October 2003, Wired Magazine ran an article on BigChampagne, a data gathering company that monitors P2P networks such as Kazaa, limewire, etc. across this country. BigChampagne would track downloads on these networks and locate them geographically using segments of the users IP addresses. They were looking for discrepancies in which song downloads for a given group jumped greatly from week to week or situations in which for a given song, downloads greatly outpaced radio air time. The resultant data sets were then sold to radio groups such as Clearchannel for as much as $40,000 per month.

BigChampagne’s data sets represented a great improvement over previous methods of data acquisition which included cold calling potential listeners or simply copying playlists from similar stations in other cities. It is widely rumored that most if not all major record labels subscribe to BigChampagne’s services but record companies refuse to comment for fear that it will damage their cases in ongoing lawsuits against filesharing networks.

We have been talking a lot about network effects where the decisions of one’s peers (fellow decision makers) positively influence one’s choices. Since I am a cynical and jaded college student, this prompted me to investigate more subversive network effects where the decisions of others negatively influence one’s decision. The closest effect I could find after briefly researching the web, is called a “snob effect,” where people seek out goods and actions that will be unique: e.g. buying art and expensive cars, etc. However, this isn’t exactly what I had in mind; I wanted to find a way to characterize the rebellious tendency to undermine or at resist the trends of the mainstream. However, I couldn’t find any information-theoretic analyses of these sort of “counter-culture” network effects, where pluralities of agents resist the trends of the dominant culture. I was surprised at this result since this sort of effect intuitively seems to me an integral part of the ever-important 18-to-24 year old’s decision-making process. I was however able to find some examples where small but powerful media outlets have triggered some serious disturbance in the mainstream cascades.

The first of these examples I found in the New York Times on March 30: Howard Stern Tries to Kill ‘American Idol’ With Kindness for a Weak Link.

The article explains that Mr. Stern has been encouraging his listeners to vote for (objectively) the worst candidate on American Idol, in conjunction with the website votefortheworst.com. Apparently, this has triggered an information cascade of his own that has apparently had some success: the site picked Taylor Hicks as the objectively worst candidate out of five remaining, and he went on to win the competition. That the site had this much of an affect on the competition is just speculation, but the Times was bold enough to suggest the possibility. In any case, this is not the first time that Stern has tried to rally subversive-minded people to underminde mainstream media: he himself moved to satellite radio in order to avoid FCC radio regulation and corporate media giants such as Clear Channel, and much of his fan base moved with him to the new format–despite the fact that the satellite radio networks have been otherwise unsuccessful.

Another, possibly less intentional, subversive force that has emerged as an influential media source is the music review site, Pitchfork. Both Slate and Wired have run articles about “The Pitchfork Effect,” here and here, respectively. The site often gives virtually unknown bands favorable reviews while harshly criticizing lackluster efforts from more successful acts. While the sites’ traffic is not very large, these reviews have a considerable effect on the sales of the bands being reviewed. For example, the Arcade Fire received an extremely favorable review from the site, and their status skyrocketed almost instantly to what is now a nationally-recognizable name. Similar phenomena have happened with bands like Tapes ‘n Tapes and Broken Social Scene. Conversely, some major label artists have been panned by the site–however it does not seem that this affects the sales negatively, likely because the readers of the site would have never bought these types of albums anyway. However, some emerging artists have suffered significantly from unfavorable Pitchfork reviews: some radio stations and record stores will not purchase lesser-known albums that are not given good reviews from the site. The site has drawn accusations of ruining bands’ careers in this way, but it seems that the radio stations and record stores probably would not have even heard of many of these albums were it not for the review.

These two examples I think are both small-scale reactions to large-scale network effects, and are popular in niche communities who want to reject the mainstream. Significantly, both parties have become authorities in their respective markets, and have created information cascades of their own–inspired by the larger information cascades of media giants.

Asch Conformity Experiments

Social Conformity was briefly mentioned in class on its connection to information cascades. A famous experiment was done by Solomon Asch in 1951. Participants were faced with this situation. They signed up for a visual experiment and were seated in a room with seven other participants. Unknown to the participant, the seven others are confederates (associates with the experimenter) and everything is scripted so that the participant is the only real subject. The experimenter starts by placing two cards before each person. One card contains one vertical line, while the other card contains three vertical lines of different length. One by one, each participant is asked out loud which line in the second card is the same length as the first card. This is repeated over and over several times. The real subject is placed next to last so that he hears the answers of all the others before him. The confederates unanimously choose the wrong line each time and surprisingly the real subject follows suit just like in an information cascade and chooses the same wrong one. When questioned later, subjects revealed that they thought they were choosing the wrong one but imitated the others based on the information they saw. Social conformity is also fragile similar to information cascades. When the experiment was changed and the subject had just one other person that spoke up and chose the correct answer instead of going along with the herd, the percentage of the conformity drastically reduced.

Flow of information is a potent theme in today’s world. On one hand, it is the gateway to an aware and well-informed society while on the other; it can lead to socially non-optimal situations and in the extreme case, have dangerous repercussions. One example of such a negative impact is the spread of suicide epidemics. A paper, http://cpi.kagoshima-u.ac.jp/occasional/vol-36/33-42.pdf, by Anthropologist Donald H. Rubinstein discusses the surge in suicide rates among young males in Micronesia. Between 1960 and 1989, there were more suicides in this sparse archipelago than anywhere else in the world. Teenagers committed suicides for trivial reasons like being yelled at by parents, not having a graduation gown, or because a girlfriend had been seen with another boy. It is claimed that the trigger for this unfortunate series of events was the following incident. In November 1966, the well-known son of one of Ebeye’s (a Micronesian island) wealthiest families hanged himself when he could not decide between his two girlfriends, having fathered a child with each. This event caused considerable commotion in the society. Three days later, a 22-year old struggling with marital difficulties committed suicide. For an island that had not heard of suicide in the last twelve years, two suicides in the same week served as a catalyst for an epidemic. Over the next twenty years, the suicide rate among teenage males in Micronesia peaked at 160 per 100,000 (the comparable rate in the United States is 22 per 100,000). While quoting this incident in The Tipping Point, Malcolm Gladwell states that ‘suicide can be contagious’. Studies have found that immediately after media coverage of suicide accounts, suicide rates jump. This is particularly true when a celebrity is involved. For example, Marilyn Monroe’s suicide was followed by a temporary 12 percent increase in the national suicide rate in the US. A similar wave of suicides was observed in Cuba in the1970s. The island’s people were emotionally shattered when President Fidel Castro’s plans for a record 10 million ton sugar harvest failed. After that catastrophe, Cuba’s suicide rate nearly tripled, from 8 per 100,000 people in 1969 to 23.2 per 100,000 in 1982.

Such alarming rates highlight the magnitude of impact that people’s reactions and decisions can have. The fact that such trends are noticeable in groups as big as nations raises disturbing questions concerning the risks of sharing information. Knowing that someone embraced suicide as the solution to his/her problems invites other depressed people to do the same. Under the overarching theme of making ideas commonplace and acceptable, flow of information forces otherwise rational human beings to fall into the trap of information cascades. A morbid extension of this trend can be seen in young children who have been found to attempt suicides in a ‘spirit of imitation and experimental play’. An 11 year old boy in Micronesia attempted to hang himself while ‘trying out suicide’. More recently (in 2004), a convict Dhananjoy Chatterjee was hanged in India; the first execution of the country since 1995 (http://news.bbc.co.uk/1/hi/world/south_asia/3597316.stm). After this incident, three children died while enacting his execution. One of them was a 12 year old girl who was showing her younger brother how Dhananjoy was executed. Such incidents can be attributed to media overkill, to which young children are particularly susceptible but the potential threat of uncontrolled information flow cannot be ignored. In a broader sense, any attempt to provide an explanation for a social problem of this complexity, particularly one that cuts across boundaries of maturity, race and gender, always runs the risk of over-simplification. Yet, there are reasons to believe that in the real world, usefulness of sharing information is invariably ambiguous.

http://www.cs.umanitoba.ca/~arindam/pubs/MiM07_aina.pdf

Peer-to-peer (P2P) networking is the primary method by which pirated music is disseminated.  To combat this epidemic, the Recording Industry Association of America (RIAA) has begun to seed corrupt files that track the file downloader’s IP address so that he may be prosecuted. 

The similarity between file sharing via P2P networks and the information cascades we discussed in class is that when one hosts a file, others will see that he has downloaded the file and will be more likely to download that file.  But this is problematic because there is a high potential for spreading corrupt files, especially if the RIAA wishes to flood the P2P network with “users” hosting the said files. To combat this, some popular P2P networks such as Kazaa have begun to implement file-rating systems, which are meant to speak of the file’s integrity.  But the problem with their rating system’s implementation is that each user’s input has the same weight on a file’s rating.  This is obviously silly because there is no way of knowing the reviewer’s credibility.  It could be the case that the RIAA hires people to assign high ratings to corrupted files.

To improve on this method of rating, the authors of the paper have created a system in which “As the stories (files) spread, each node makes an accept, reject, and forward decisions based on the credibility measure which is computed from the estimates of the originating node’s trust and the particular story’s trust.”  To estimate the node’s trust level, Baye’s Theorem, which we discussed in class, is used. 

It is clear that a rating system that uses conditional probability will be far superior to one that does not.  Because we have a way of knowing the reviewer’s credibility along with his review of the file, by Baye’s Rule, it is more likely that a file with a high rating is of good integrity than if we were to just use the rating system that Kazaa uses.

The idea of information cascading can be placed in many different human situations. This can be seen by all the examples given in class (ie: fashion, juries, and entertainment). However, with all these human examples, many people can make the argument that all it takes is one strong willed individual to break the cascade. Sometimes the best examples of social phenomena exist in animals. This article discusses the biological habit of circular mills found in army ants. Circular mills occur when a group of food foragers separate from all the other foragers. Usually the ants march in a line. However, if a group separates, the ants will begin to form a circle where the lead ant will follow the end ant. The ants will continue going around in circles until they exhaust themselves and die.  Interestingly enough, this may have been a result of the evolution of ants.  Ants have been around for millions of years and have always gathered in groups.  This evolved into the army ants we see today, where all ants follow the ant in front of them.

Many times the fact that humans have conscious minds prevents us from analyzing basic properties of our behavior.  This is where analysis of animal behavior is useful.  Where a human may stop the cascade, an animal my follow the cascade to it’s death.  What has worked for millions of years must be right.  Also in the case of ants, it may be even more true since ants are well known for their colony- like behavior:  the individual is not important.  Humans have a different mind set.  Most humans, value their ideas.  Some of the most well known discoveries of man were found through the strong will of one man to ignore the information cascade.  For instance, Galileo was constantly insulted by others prominent members of society to give up his beliefs on a heliocentric world, even after he had already made significant advances to other areas of science.  All it takes is one man  to break the cascade.

The phrase “The Long Tail” was first coined by Chris Anderson in an October 2004 article “The Long Tail” to describe the business and economic model of internet companies like Amazon and Netflix whose sales revenue from selling only a few copies of millions of items was comparable to sales from selling millions of copies each of the few most popular items. The striking feature about this new “long tail economy” was that a significant portion of business sales resulted from items that wouldn’t even be sold by traditional brick-and-mortar stores found in the physical world.

Anderson suggested that the collective sales from products that sell only in small volumes (and that would never be sold by a brick-and-mortar store) could exceed the sales from the most popular products. Anderson extended his work into a book “The Long Tail: Why the Future of Business Is Selling Less of More.” The research article “From Niches to Riches: Anatomy of the Long Tail” states that about 30% to 40% of Amazon’s book sales are in books that wouldn’t be found in a typical brick-and-mortar store; while a typical brick-and-mortar store might stocks between 40,000-100,000 titles, Amazon sells nearly 3 million titles. The article examines both supply side and demand side drivers of the long tail phenomenon.

On the supply side, internet businesses like Amazon can utilize central warehousing facilities to sell pretty much everything whereas traditional stores are limited by shelf space and therefore find it feasible to stock only popular items that sell in large volumes. Anderson stated in his article that one of the constraints for businesses in the physical world (like movie theaters, DVD rental stores etc.) is the need to find local audiences. As an example, he mentioned the “plight of Bollywood in America”, despite their being 1.7 million Indians in the US, the top-rated film Lagaan “opened on just two screens, and it was one of only a handful of Indian films to get any US distribution at all. In the tyranny of physical space, an audience too thinly spread is the same as no audience at all.”

On the demand side, search engines, recommendations, and content-based advertising help consumers find products in the long tail. The internet has helped in connecting supply and demand of niche products in this long tail economy.

Amazon’s Associate program is a form of content-based advertising that attempts to take advantage of the long tail phenomenon; website owners that sign up for this program place links to Amazon products on their website and receive a percentage of the sales if their ads result in a purchase on Amazon. This drives traffic to products in the long tail because the multitude of websites that cater to small niche communities direct traffic to products that are highly relevant for the specific community. Google AdSense similarly places the most relevant ads on a webpage in accordance with the content of the page. Content-based advertising is thus a salient feature of the long-tail economy.

Google gets into the political arena.

http://www.latimes.com/business/la-fi-googltics25mar25,1,1251038.story?coll=la-headlines-business

http://www.wired.com/politics/law/news/2004/05/63557

We saw in class that Google sells certain keywords to merchants such as Ithaca Calendar Clocks and shopping website like eBay and Amazon. However, sellers and traders are not the only ones buying Google “AdWords”, politicians are seeing the vast reach of internet advertising and jumping in. Back in 2004, a group of bloggers pushed Kerry’s campaign website to the top of the search results for the keyword “waffles”. Capitalizing on this internet prank, the Kerry campaign purchased a variety of keywords on Google and linked them to the Kerry campaign site. These words ranged from the mentioned “waffles” to words like “miserable failure” (referring to a Google-bombing involving George Bush) and “George W. Bush”.

Today, a Google search for the keyword “election 2008” brings up the sponsored link to Barack Obama’s campaign website. As a business Google has realize the potential profit to be made by selling campaign advertisement and has began to actively reach out to campaigns. Just recently, Google established a “political sales team” and sent three of the team’s members to deliver a seminar about using Google’s products for political purposes. Although this by itself is not very surprising, the contents of the seminar were. In addition to lecturing about AdWords, Google also delivered a speech about what videos resonate on its newly acquired YouTube website and how to propel websites up the search engine rankings.

From the various examples cited in class, we have seen that information cascades are a very real, and in some cases dangerous, phenomenon. The danger, of course, comes from the fact that in a cascade, people willfully ignore any signal that the choice they are about to make may be incorrect. This conformism, although beneficial in a few cases, is especially damaging in fields which require a fair amount of skepticism. In this post, I will discuss an article studying how statements about molecular interactions that can be found in biological texts vary over time. The question to ask is this: to what extent are such declarations based on blind faith to what has been said before?

Read the rest of this entry »