Slashdot Mirror
Detecting Patterns in Complex Social Networks
Roland Piquepaille writes "So-called social networking is very popular these days, as show the proliferation of services like Friendster, Orkut and dozens of others. But do the companies behind these services have any idea of what is hidden inside their complicated networks? When these networks reach a size of millions of users, it's not an easy task. A researcher at the University of Michigan is trying to help, with a new method for uncovering patterns in complicated networks, from football conferences to food webs. This overview contains more details and references about this non-traditional method. It also includes a spectacular representation of the Internet and another image showing a food web at Little Rock Lake."
I don't want to downplay the possible significance, but if you are focusing on the "clumps" (what disparate entities have in common) isn't this akin to taking slices out of a data warehouse? Aligning everything along a single axis?
In the future, I would want to not be isolated from my friends in the Space Station.
I think if the internet was studied as a social network, on might find that someone like Janet Jackson was the core of society :-)
It's hard enough to remember my opinions, never mind the reasons for them..
But do the companies behind these services have any idea of what is hidden inside their complicated networks?
I have often wondered this about Slashdot itself. It would appear to me that Slashdot would provide an ideal means to mine data on complex interactions that may have implications for anything from database design to network load analysis or perhaps the results may even apply to the modeling of biological systems. The owners of Slashdot would be missing something big if they were not examining Slashdot very carefully.
Mapping the Internet only has so many applications, but if one really wanted to make an obscene amount of money, figuring out how to model systems is where it would be.
Visit Jonesblog and say hello.
I wonder if this will improve search results? All the fake porn sites will be lumped together, thus, hopefully, taking them out of regular, useful searches.
EVERYDAY IS CATURDAY
In this image..o rks/schoo l.gif
http://www-personal.umich.edu/~mejn/netw
The little single dots on the left..
you have to feel bad for them..
and all the "fringe" people.. they are visibly shown on the fringe..
kind of interesting..
anime+manga together at last.. in real time.
We see and understand patterns based on the amount of data we can digest (which has gone much further with computers). Knowing that you could always be one data set off defining a pattern makes you wonder if chaos exists at all, hence the replacement of words like chaos with words like "complex".
From football conferences to food webs: U-M researcher uncovers patterns in complicated networks
SEATTLE---The world is full of complicated networks that scientists would like to better understand---human social systems, for example, or food webs in nature. But discerning patterns of organization in such vast, complex systems is no easy task.
"The structure of those networks can tell you quite a lot about how the systems work, but they're far too big to analyze by just putting dots on a piece of paper and drawing lines to connect them," said Mark Newman, an assistant professor of physics and complex systems at the University of Michigan.
One challenge in making sense of a large network is finding clumps---or communities---of members that have something in common, such as Web pages that are all about the same topic, people that socialize together or animals that eat the same kind of food. Newman and collaborator Michelle Girvan, a postdoctoral fellow at the Santa Fe Institute in Santa Fe, New Mexico, have developed a new method for finding communities that reveals a lot about the structure of large, complex networks. Newman will discuss the method and its applications Feb. 15 at the annual meeting of the American Association for the Advancement of Science in Seattle.
"The way most people have approached the problem is to look for the clumps themselves---to look for things that are joined together strongly," said Newman. "We decided to approach it from the other end," by searching out and then eliminating the links that join clumps together. "When we remove those from the network, what we're left with is the clumps."
The researchers tested their method on several networks for which the structure was already known---college football conferences, for example. In college football, teams in the same conference face off more frequently than teams in different conferences. When inter-conference games do occur, they're more likely to be between teams that are geographically close together than between teams that are far apart. Plugging in information on frequency of games between pairs of teams in the 2000 regular season, Newman and Girvan tested their method to see if it could correctly sort the colleges into conferences. "There were a few cases where it made mistakes, but it got well over 90 percent of them right," said Newman. "It gave us the structure we were expecting, so that was encouraging."
Newman and Girvan---and other researchers who've learned about their work---have gone on to apply the technique to systems where the structure is not as well understood, looking at everything from networks of Spanish language web logs to communities of early jazz musicians to a food web of marine organisms living in Chesapeake Bay.
"Networks and other systems that we study are becoming increasingly large and complicated these days," said Newman. "New methods like this help us to make sense of what we see and to understand better how things work."
###
For more information:
Mark Newman -- http://www-personal.umich.edu/~mejn/
American Association for the Advancement of Science -- http://www.aaas.org/
Santa Fe Institute -- http://www.santafe.edu/
The blue node (left center) in this diagram was gettin' some action!
TK
Social network analysis has been around for years in social science, so I don't see what is new here. And before anyone complains, yes, these nice pictures are also far from new.
The researchers tested their method on several networks for which the structure was already known---college football conferences, for example. In college football, teams in the same conference face off more frequently than teams in different conferences. When inter-conference games do occur, they're more likely to be between teams that are geographically close together than between teams that are far apart. Plugging in information on frequency of games between pairs of teams in the 2000 regular season, Newman and Girvan tested their method to see if it could correctly sort the colleges into conferences. "There were a few cases where it made mistakes, but it got well over 90 percent of them right," said Newman. "It gave us the structure we were expecting, so that was encouraging."
Finally, something that can help me understand the divisions in the NHL. I've been confused ever since they got rid of Smythe, Norris, and all the rest...
Stop by my site where I write about ERP systems & more
The uses for this software are astounding. It is, essentially, a breed of software designed to recognize and manipulate social class systems.
... imagine that ... a means of actually targetting campaigns and capers directly to the primary delivery mechanisms of word of mouth among a large group. This software can give you that.
... put this in the hands of the right (wrong?) people, and we could see social revolutions targetted and executed with such blinding accuracy and predictability that most of us simply won't know what hit us ...
... maybe its time to unplug.
Imagine a system which tells you, easily enough, who the 'most popular person for subject ___Y___' is, in your neighborhood? Target a campaign of computer-buying to only -3- folks in an area, and end up blanketing the entire region with tuber-like memes...
PR agencies could use this data to identify the core 'gossip leaders', the ones who have massive impact on multiple peers, and then they could target only those people with their campaigns
There are numerous religious theories, also, on the strengths of individuals and groups and the effect that these social connections have on a movement
This is the danger zone. The moment we start using computers to do qualitative analysis of social dynamics, and then using the data for commercial/religious/nefarious purposes, well
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
I have an idea. Phone books of mobile phones form another kind of network. Imagine, A has number of B in his/her phone book. B has number of C. E knows both A and B. Chances are, most of GSM users in Latvia are nodes of this network. But this network can be fragmented as well. I think we could study interesting things about society this way.
:)
We have 7-digit phone numbers and two mobile networks here in Latvia. Data can be stored this way:
6787026 -> 9131415
9131415 -> 5956564
etc...
All we need is one hashtable (or MySQL table) and data collection interface
The problem is more complicated, and you touch on one of the main weaknesses of any system where reputation and feedback in involved.
One aspect of the problem is the granularity by which relationships are defined. In many of the sites there is only one state: "friend or non friend". The real world encompases a number of shades and types, from business acquaintance to personal friend, intimate lover, etc.
Another aspect is the incentive to "game" these systems by increasing your friend count. This inevitably leads people loosening their interpretation such that they increase their visibile friend count. If the number if friends you were linked to was not public, there would be less of this (but you can't do that without breaking some of the functionality of the sites)
People have talked about "winning" at friendster or tribe or orkut - but there is no "winning" in these systems, as there should not be competition.
Last, there is no method for verification of any status between peers. Can you "prove" that so and so is really a friend?
There are others, but these are the main three, and not likely to be solved or addressed any time soon.
I wonder whether they'll finally be able to (dis)prove the hypothesis that everybody knows everybody else within six (or however many) degrees of separation.
Then again, most people will probably have a connection to Nigeria due to the certain organ-lengthening drug that they are so famous for.
I remember the first maps of the Internet showed that certain nodes concentrate power in terms of the number of connections they make. Google, perhaps.
A quick reading on Zipf's Law shows that many natural systems (and many artificial ones that obey similar laws of construction and equilibrium) observe 'power rules' where the distribution of power is inverse to the number of entities at any level.
Surprising that earthquakes, cities, businesses, follow the same rules. And yet quite meaningless in any direct sense because we can't manipulate these rules, only observe them.
Human social networks also follow rules that I suspect are quite simple and possibly similar to Zipf's Law. For instance, a person can only maintain a finite number of contacts (technology may increase this number but it remains finite at any given time). Any new contact coming in displaces an existing contact. So a single person's contact list will follow a power law: twice as many contacts used half as often, ten times as many contacts used a tenth as often...
Mapping a contact network would need to take the importance of each contact into account. I may have my grandmother in my list, but I speak to her once a year. My accountant - every week. My wife - twice a day. My girlfriend - every hour.
Next: the differences between individuals in terms of how much time/skill they invest in networking. Gender differences... women do this much more and better than men, in general. Age differences... younger men do it less well than older men. Wealth differences... richer people do more networking, I'd suspect, until a certain point when they start to delegate it. Very poor people do very little networking.
So, the network is not a flat map. It's got two dimensions for the lines, but each line has a thickness, and each node (individual) has a size.
Finally, I'd suspect that the network also maps power in terms of social success. Those people with the most powerful networks (a recursive definition: the networks which involve the most powerful people) will also be the most successful socially / financially.
But they may not be the happiest.
Large scale networks have limitations because real relationships are complex. The notion of A-is-a-friend-of-B or A-trusts-B is too simplistic for large scale networks. These connectivity relationships are not transitive in real-life (A-trusts-B & B-trusts-C does not imply A-trusts-C)
Rather, the network needs some form of role-based assertion or qualification of the relationship. I know friends that I like to go hiking with, but that I disagree with politically. I know people that I do trust to recommend software, but don't trust to recommend a restaurant. And if I trust person B to recommend software, I would probably only trust that person B to recommend another person C in a limited set of domains (like software or technical issues). Thus the real relationship is more like person-A-trusts-person-B-for-role-C.
Such a scheme of role-defined relationships could be self-organizing or predefined. The self-organizing approach would look for disjoint clusters of members in a network or use semantic analysis of the messages passed between people to infer a set of role-clusters. Predefined relationship might be OK, but could become unwieldly if the network creators force people to answer a long multiple-choice test about every relationship.
Two wrongs don't make a right, but three lefts do.
Well, on Slashdot, I get fans because people see and like what I post. (Except for one guy, I think he's just trying to max out his friends list.) I set friends based on whether I like and appreciate what they say, and would like to be reminded that I have them set as "friends" whenever they say something I don't necessarily agree with. It helps me consider other points of view.
Granted, its a set of small steps towards understanding the opposing point of view, but it does help broaden my horizons.
It's actually a very useful system.
tasks(723) drafts(105) languages(484) examples(29106)
Why is it that many /.'ers are so concerned with their privacy in some cases, such as pay pal, ebay, etc, and yet have no problem giving another company their contact info and the contact info of everyone they know? It seems that digitizing social networks (ala friendster) really opens you up for privacy abuse. These companies could, frankly, really mess up your life if they decided to do such a thing, or if a hacker broke in and did such a thing.
Check out the "highschool friendships" diagram.
I think I was the yellow dot on the far left.
In all matters of opinion, our adversaries are insane. -Oscar Wilde
I have been thinking about concept clumps - kind of similar to social clumps and cluster - but relating things that are based around similar ideas of information that they are trying to convey.
:)
Similar in the way that grokker clumps navigable areas together, it would be interesting to instead clump things together based on the relations of the meaning of the information they contain.
For example, lets say that you are reading an article on any given site. You would be able to highlight a phrase, a word or a sentance, then look that term up in context. This is different than simply googling the term in that you are looking for the context of the term as opposed to a concrete definition.
so if you were reading an article regarding the legal take over of a company by intel, you would be able to easily search for articles writen that involve intel in any other litigation, with results containing intel involved in purchases or sales of companies and their technologies coming to the top of the list...
obviously there is a lot more in this required to accomplish it - so Ill just stop here before giving it all away.
The main point being that this type of searching is easily applicable to understanding relationships in social networks as far as identifying how common intrests are shared.
The clustering of attractions and dislikes to profile trends and personalities in any given demographic are made especially easy in systems such as friendster and orkut. By having people OPT-IN to the deepest marketing database available and provide you with all the details of not only the things they like (under the guise of sharing yourself with the others in the community) AND showing you what other people they are connected with who share common interests is one of the biggest social hijacks ever.
Just when you thought marketing was a dead science that is too transparent to have any real impact, social networks arise to provide marketing data on an astounding level.
[don tin foil hat]
Just wait till they are able to correlate all this info with DNA profiles
Not that this is bad per se, but it is a fact taht this info will be the next gold standard in market research where marketing will move to a social promotion system.
I think that the goal here is the promotion of product will largely come from people advertising their likes of a product through their profiles and communications with friends online.
It will be very easy for a group of people to communicate things (it already is) that are of interest to their social networks. Like on person telling the other 65,000 friends they have how they jsut experienced product Y, and that everyone should try it....
interstingly, will we see fakesters made specifically to spam the other friends with testimonial like adverts for products they are trying to introduce to a specific demographic?
And his name is Kevin Bacon.
Unfortunately, then only pattern in my social network is the singleton pattern.
- the wiring of it (topology) can gives a lot of insight on how it works and can even explain some emerging side effects.
- it evolves with time - new connexions are made between nodes everyday, and we observe self-optimization.
- the information that is communicated within the network itself is also pretty important. Actually, this is not only the tracer from which we derive its topology and its evolution, but also the very meaning of it.
There is something way too similar about social networks, internet and the brain that really troubles me.Furthermore, read a few books on emergence (like Kevin Kelly's "Out of Control"). Might as well also pick up and read Wolfram's "A New Kind of Science"...
I have said it before and I will say it again: Taken together, the knowledge within these three books could very well lead to some amazing breakthroughs in many of the sciences, in particular cognitive sciences and genetics. Even if some of the theories prove to be wrong, I think there is enough there to be a springboard for someone else - please read and decide for yourself!
Reason is the Path to God - Anon