As a network engineer for a major financial trading company I've some experience in this area. I've also served as a network engineer for several companies in various fields (Internet Service Provider, Professional Services Vendor, Extremely Large Retail (Borders... I'll miss you.). In my experience traffic monitoring becomes a key requirement of any efficient & secure organization and a key responsibility of any qualified network engineer.
Depending on context traffic monitoring has several definitions. You (or your boss) appears to be headed in the direction of security and/or packet intercept. This is one of those projects that is rarely implemented well. Furthermore there are major legal and privacy concerns. Before you proceed further, I recommend you receive written confirmation from your employer that his employees (or family in this case) is notified of the scope and depth of monitoring. In my opinion if you do so without this confirmation, you are morally and professionally just as responsible for any abuses that may occur.
Let's begin with some of the options that you have available to you.
SNMP - The most basic network monitoring tool, supported by most devices out there. For example, a Cisco router or firewall is polled by a SNMP monitoring application, showing interface usage as a function of packets per second or total throughput in both directions. Not really what you want to do here but any discussion of "traffic monitoring" should start here.
Netflow - Netflow is set up in a similar manner. A Netflow supporting device is configured to send a record of traffic conversations to a collector and/or analyzer. This could be a router, switch or firewall. This begins to provide some of the information that you are looking for. Flows are packets matched with the same source, destination and ports. Netflow provides valuable information for this reason. What ports are in use? What are my most common destinations? Who is my bandwidth hog? An analyzer might also include DNS look ups as a feature, so a Facebook destination address shows up as Facebook's DNS in a reporting chart or export spreadsheet.
To go any deeper than that, your looking at packet intercept, which can be done in a few different ways.
Hardware:
I'm assuming that you don't have a Cisco 6500 or Nexus 7000, so simply buying a $30,000 packet intercept blade and sliding it in is out of reach. You appear to be much more familiar with software (and comfortable with those options) so I won't try to steer you away from that. I'm only going to briefly cover your hardware choices. These may or may not provide you with the information your looking for. For example, depending on the application even the internal messaging component you mentioned could be encrypted and the information gibberish.
Firewall - The simplest and easiest "appliance" you can buy is a next generation firewall. Such as a model sold by Sonic Wall. The TZ Network Security Appliance Series has a lot of useful features, including DNS intercept, filtering, packet intercept, built in netflow collector & analyzer, etc.. I haven't used the packet intercept features myself, so I can't tell you exactly what information can be accessed or in what format.
Specialized Appliance - An appliance specialized for packet intercept and analysis, other than the Cisco packet intercept models, I haven't used anything else so I won't mislead you with guesses or half truths. I will say that generally these are going to relatively large financial investments.
Software:
Proxy - Maybe your cheapest and/or best bet. Implementing a web proxy on a server (such as the open source Squid project) should give you most of the information you are looking for. DNS, content analysis, packet intercept and "scamming protection". At Borders, each of our stores ran a Squid proxy server for internal traffic, and public traffic went through a pair of McAfee proxy appliances (oh how I hated them).
I work at a internet service provider, configuring and setting up integrated voice and data network equipment. Honestly, the Healthcare IT I've worked with have been the most incompetent fools I've ever known. The head of IT for a major hospital for example didn't even know how to change IP routes on his internal routers when the hospital decided to switch to us. For days he said it was our problem. Eventually, we got to management and I think he and half of his department got fired.
While I was a Junior/Senior at Everett High School, (Lansing, Michigan), I built a tunneling scanning electron microscope. We originally followed/used a kit from the University of Muenster in Germany that I had learned about from Slashdot. Unfortunately, the documentation sucked, the circuit board was etched incorrectly and there was a design error. Furthermore, the control software was written is visual basic and was nothing more then a toy.
With the help of a electrical engineering group at Michigan State University we overcame the problems and I decided to modify the original design to use GXSM, a powerful open source electron microscope software package that is Linux only. This required adding a sranger digital signal processing board and stepping up the input/output voltages for the piezo crystals. Amazingly, almost all the work was done by myself or fellow students, MSU only guided us in understanding the circuit diagrams, making small adjustments, fixing the errors in the plans and designing/building the stepping circuits for my modifications.
I have some really great memories, spending all day in the basement lab I had set up, eating pizza while skipping all my classes with permission from the principal, "accidentally" burning my long time enemy with the soldering iron, ripping a chunk of my finger off jumping a network wiring cage to connect the main computer to the internet.
Working with the electronics and science was very interesting, but the most valuable experience came from lobbying for the funding from local government, assembling a team of fellow students to work on the project and starting a Nanotechnology elective class to actually use the damn thing. Eventually, former State Senator Virg Bernero (now Mayor of Lansing, Michigan) convinced BioPort (the company that makes the Anthrax vaccine) to provide the majority of the funds.
The project eventually inspired local university and government leaders (I wouldn't stop bugging them;)) to support accelerated Nanotechnology development and commercialization while also encouraging applied and basic research. Michigan State University and the surrounding universities are home to world class researchers and students working on Nanotechnology and Nano-Biotechnology. It has been decided that it is time the state began to leverage that asset to create a bright 21st century future for our citizens.
I'm 19 years old, and thanks to the Slashdot article "build your own electron microscope" I've actually become something I'm proud of. I've built a tunneling scanning electron microscope, lobbied for funding and government support, founded a Nanotechnology class at Everett High School with help from a amazing science teacher who now is inspiring the class to even greater things while developing a soon to be accredited curriculum, hired as a contract consultant by a company in silicon valley, been sent overseas, all expenses paid to a nanotube conference in Japan by the same company and I now work at M.S.U. as the only employee in a new Nanotechnology supporting office at the college of Engineering. (There is also some other stuff I'm not allowed to speak of.)
I've met very important people from NASA's JPL, IBM, Oxford, Harvard and founders/pioneers of Nanotechnology.
In my free time, I lobby for the creation of a Michigan Institute of Nanotechnology, which will become the center of Nanotechnology in the state, facilitating the cooperation of private industry, research, academia and government to create jobs, businesses, breakthroughs and secure a portion of the world economy for ourselves. It already has a extremely wide and powerful base of support.
I don't know about you, but where I come from (Michigan) we require more than just blog posts to make such dangerous accusations. Perhaps you should consider modifying your links to point to more direct and reputable sources of information?
Make no mistake, local government is aware of the OSS community. For example, I work in a State Senator's office, while our Senator isn't exactly technologically savvy, our policy guy is. He reads slashdot every day and we occassionally discuss open source. The politicians can't work their magic without community support, an internet base just dosen't cut it. Also, having Novell or Redhat knocking on your door, asking for your support and donating some campaign contributions couldn't hurt either.
Really, if you want government support of open source, at least on the local level, schedule an appointment with your State Senator and Representitive, bring a few like minded friends. Most likely you'll be asked to go out to lunch or coffee with the official and before you know it you'll be chatting like old friends. It's what politicians do. Just most people never think they will be given the time of day. All they need is an indictation that within the district their are supporters willing to dedicate their time to help push through a initative, bill or amendment.
It really isn't that hard.
Just expect to have to baby-sit the thing for few years before you see results.
Your sarcasm needs work, or else you are seriously mistaken. A "semi-hostile" takeover of Wikimedia by an established dictionary entity would mean only one thing, they are trying to eliminate the free competition.
To assume that Britannica would release it's content or try to improve the Wiki is rather naive.
That, or I am a socialist with a grudge against anyone who seeks profit at the expense of the free exchange of information.
What can we possibly do to turn back the tide against the vicious spread of misinformation mongering? Surely, there must be something, a website, a petition, a campaign. I'm tired of just sitting on my hands and shrugging my shoulders. I'm not calling for a holy war here, but surely there must be some way to stop this kind of absolute crap.
Will parents ever learn to take responsibility for raising there own kids instead of trying to create the perfect enviroment for them to raise themselves?
Anyone who wants to censor violence from entertainment media because they can't take the time to teach their children right from wrong is bullshit.
Games arn't games anymore, they are still fun, but the time of innocence is long past.
When I play a violent war game, when I blow someone up or stab them.
I know one thing for sure, wars should be avoided at all costs.
This, is what video games teach us, the real horror of being there. That's something the watered down text books of our under funded public schools can't teach us.
Hillary Clinton won't become president, that much is for sure, she just alienated most of the 49 percent of democrats that would have voted for her.
Well, let's see, they provided weapons, military training and aid to the American Colonists in the Revolutionary War. They developed the most heavily armored and gunned tanks during the early German Blitz, one French Char B1-Bis held up an entire German Division for an entire day. One little short frenchie with a bad attitude almost conquered the entire world, twice.
They've developed nuclear weapons, were one of the original founders of the European Union, who's Euro continues to dominate the American Dollar. They were one of the first modern countries to pick on the buzzword "Democracy" long before a bunch of colonists got pissed at their King's latest tax law.
Oh, did I mention numerous American, Australian and British courts have upheld the same reverse engineering proof of concept rulings?
You Sir, are an uneducated bigot.
(Note: I am not anti-American, I'm just hitting him where it hurts.:))
This was a completely irresponsible repost of a story less then 24 hours old. What the hell is up with the mods? I mean, come on! It's still on yesterday's news!
This is the first rehash of a story to actually make me lose my temper with the mods.
I can't figure out what Gigabye's roadmap is. I mean, the dual 6600GTs on a single card came out of no where. Now there is the dual PCI express board coming out that allows any two Video Cards to run in parellel, it's not SLI. Now they come out with this.
I dunno what they have in mind, but they sure are stiring things up a bit, but arn't they risking alienating nVidia with these "almost" SLI competetor alternatives?
I never thought about it that way before, I always kind of assumed that in the process of colonizing the stars we would become the kind of people you describe.
We really need something like this to justify our space program to the masses and to wake people up. There is more to life then just cheeseburgers and scoring that awsome IT job.
Human destiny lies with the stars, simply because eventually the Planet Earth will no longer be able to support human life, soon then later at our current population and resource expenditure.
On a long enough timeline, survival probability always drops to zero...
Sure they are still a monopoly but competition is competition. The only way Microsoft can really dethrone google is if they come out with a better internet search engine. If we get a improved system and outlook search, all the better.
I really hope that this gives Linux the kick in the pants it needs for someone to come up with better system search solutions. Find is absolutely terrible in my humble opinion, especially it's tendency to freeze up when you stop a search.
Lack of metadata search makes baby Linus cry.
Bring me browser wars! Bring me os wars! Bring me search wars! These are the only kind of conflicts in which the consumer benefits, so we might as well encourage them!
Re:Electricity IS Civilization
on
Port-A-Nuke
·
· Score: 1
Well I assume whereever the thing goes it's going to be supervised by the United States or U.N., if it's abused we will simply take it away and maybe steamroll the dictatorship into oblivion.
Electricity IS Civilization
on
Port-A-Nuke
·
· Score: 2, Interesting
Developing countries, national crisis areas, there is practically no limit for something like this. I don't see it being easily abused either. Power is civilization and civilization is generally a good thing.:p
I seem to remember reading that some organization was setting up servers on abandoned oil rigs in international waters for just such a purpose. I don't know what happened to them. Something about a giant squid maybe?
Anyone remember that show? Some guy took over as the chief of the Washington DC Capital district and enacted major changes. The police department used a huge real time statistics tracking system and displayed it on a huge display. I think the idea was kind of revolutionary to the average joe like me; but I don't know if police were using a system like that before then.
I wouldn't mind seeing systems like this implimentated in say, elected public offices to keep track of opinion areas, ethnic densities, crime rates, poll results, average pay. etc. To help them keep better tabs on what they need to improve and how to vote on what bill.
Oh, did I mention I plan on making a run or two for public office?:D
I want to play Halo 2, Half-Life 2 and Doom 3. To be able to play those games, I would be required to be an at least $390 video card and about $200 worth of ram. Heck, throw in a new $250 processor and a $120 motherboard while your at it.
Or I could just buy a $190 xbox and Playstation 2.
It's simple numbers.
I'm comfortable running a low end linux computer and a microsoft xbox for my gaming needs.
It's like Microsoft decided to play nice and gave us one of the main aspects of windows (games) without all the other os crap involved.
Really these NEA are like a universal strip tease rather then big rocks of death. Imagine parking a couple of these things in orbit, we could mine it out and turn it into a space station. We could even use it as to start a human colony in orbit. If it was slightly bigger that is.:/
Slashdot Mirror
Greetings,
As a network engineer for a major financial trading company I've some experience in this area. I've also served as a network engineer for several companies in various fields (Internet Service Provider, Professional Services Vendor, Extremely Large Retail (Borders... I'll miss you.). In my experience traffic monitoring becomes a key requirement of any efficient & secure organization and a key responsibility of any qualified network engineer.
Depending on context traffic monitoring has several definitions. You (or your boss) appears to be headed in the direction of security and/or packet intercept. This is one of those projects that is rarely implemented well. Furthermore there are major legal and privacy concerns. Before you proceed further, I recommend you receive written confirmation from your employer that his employees (or family in this case) is notified of the scope and depth of monitoring. In my opinion if you do so without this confirmation, you are morally and professionally just as responsible for any abuses that may occur.
Let's begin with some of the options that you have available to you.
SNMP - The most basic network monitoring tool, supported by most devices out there. For example, a Cisco router or firewall is polled by a SNMP monitoring application, showing interface usage as a function of packets per second or total throughput in both directions. Not really what you want to do here but any discussion of "traffic monitoring" should start here.
Netflow - Netflow is set up in a similar manner. A Netflow supporting device is configured to send a record of traffic conversations to a collector and/or analyzer. This could be a router, switch or firewall. This begins to provide some of the information that you are looking for. Flows are packets matched with the same source, destination and ports. Netflow provides valuable information for this reason. What ports are in use? What are my most common destinations? Who is my bandwidth hog? An analyzer might also include DNS look ups as a feature, so a Facebook destination address shows up as Facebook's DNS in a reporting chart or export spreadsheet.
To go any deeper than that, your looking at packet intercept, which can be done in a few different ways.
Hardware:
I'm assuming that you don't have a Cisco 6500 or Nexus 7000, so simply buying a $30,000 packet intercept blade and sliding it in is out of reach. You appear to be much more familiar with software (and comfortable with those options) so I won't try to steer you away from that. I'm only going to briefly cover your hardware choices. These may or may not provide you with the information your looking for. For example, depending on the application even the internal messaging component you mentioned could be encrypted and the information gibberish.
Firewall - The simplest and easiest "appliance" you can buy is a next generation firewall. Such as a model sold by Sonic Wall. The TZ Network Security Appliance Series has a lot of useful features, including DNS intercept, filtering, packet intercept, built in netflow collector & analyzer, etc.. I haven't used the packet intercept features myself, so I can't tell you exactly what information can be accessed or in what format.
Specialized Appliance - An appliance specialized for packet intercept and analysis, other than the Cisco packet intercept models, I haven't used anything else so I won't mislead you with guesses or half truths. I will say that generally these are going to relatively large financial investments.
Software:
Proxy - Maybe your cheapest and/or best bet. Implementing a web proxy on a server (such as the open source Squid project) should give you most of the information you are looking for. DNS, content analysis, packet intercept and "scamming protection". At Borders, each of our stores ran a Squid proxy server for internal traffic, and public traffic went through a pair of McAfee proxy appliances (oh how I hated them).
I work at a internet service provider, configuring and setting up integrated voice and data network equipment. Honestly, the Healthcare IT I've worked with have been the most incompetent fools I've ever known. The head of IT for a major hospital for example didn't even know how to change IP routes on his internal routers when the hospital decided to switch to us. For days he said it was our problem. Eventually, we got to management and I think he and half of his department got fired.
I'm in the process of building a new computer for video games.
What is the capability like for PC games? How is the performance? Is DirectX10 ready yet? How about drivers?
I order the operating system Friday. Should I go with XP or Vista?
While I was a Junior/Senior at Everett High School, (Lansing, Michigan), I built a tunneling scanning electron microscope. We originally followed/used a kit from the University of Muenster in Germany that I had learned about from Slashdot. Unfortunately, the documentation sucked, the circuit board was etched incorrectly and there was a design error. Furthermore, the control software was written is visual basic and was nothing more then a toy.
;)) to support accelerated Nanotechnology development and commercialization while also encouraging applied and basic research. Michigan State University and the surrounding universities are home to world class researchers and students working on Nanotechnology and Nano-Biotechnology. It has been decided that it is time the state began to leverage that asset to create a bright 21st century future for our citizens.
With the help of a electrical engineering group at Michigan State University we overcame the problems and I decided to modify the original design to use GXSM, a powerful open source electron microscope software package that is Linux only. This required adding a sranger digital signal processing board and stepping up the input/output voltages for the piezo crystals. Amazingly, almost all the work was done by myself or fellow students, MSU only guided us in understanding the circuit diagrams, making small adjustments, fixing the errors in the plans and designing/building the stepping circuits for my modifications.
I have some really great memories, spending all day in the basement lab I had set up, eating pizza while skipping all my classes with permission from the principal, "accidentally" burning my long time enemy with the soldering iron, ripping a chunk of my finger off jumping a network wiring cage to connect the main computer to the internet.
Working with the electronics and science was very interesting, but the most valuable experience came from lobbying for the funding from local government, assembling a team of fellow students to work on the project and starting a Nanotechnology elective class to actually use the damn thing. Eventually, former State Senator Virg Bernero (now Mayor of Lansing, Michigan) convinced BioPort (the company that makes the Anthrax vaccine) to provide the majority of the funds.
The project eventually inspired local university and government leaders (I wouldn't stop bugging them
I'm 19 years old, and thanks to the Slashdot article "build your own electron microscope" I've actually become something I'm proud of. I've built a tunneling scanning electron microscope, lobbied for funding and government support, founded a Nanotechnology class at Everett High School with help from a amazing science teacher who now is inspiring the class to even greater things while developing a soon to be accredited curriculum, hired as a contract consultant by a company in silicon valley, been sent overseas, all expenses paid to a nanotube conference in Japan by the same company and I now work at M.S.U. as the only employee in a new Nanotechnology supporting office at the college of Engineering. (There is also some other stuff I'm not allowed to speak of.)
I've met very important people from NASA's JPL, IBM, Oxford, Harvard and founders/pioneers of Nanotechnology.
In my free time, I lobby for the creation of a Michigan Institute of Nanotechnology, which will become the center of Nanotechnology in the state, facilitating the cooperation of private industry, research, academia and government to create jobs, businesses, breakthroughs and secure a portion of the world economy for ourselves. It already has a extremely wide and powerful base of support.
Not bad for someone who graduated with a 2.5 GPA.
Me too.
There will be a demo released on steam. ;P
The politcal officer... duh
I don't know about you, but where I come from (Michigan) we require more than just blog posts to make such dangerous accusations. Perhaps you should consider modifying your links to point to more direct and reputable sources of information?
Mod Parent Up, why is he modded -1?
Make no mistake, local government is aware of the OSS community. For example, I work in a State Senator's office, while our Senator isn't exactly technologically savvy, our policy guy is. He reads slashdot every day and we occassionally discuss open source. The politicians can't work their magic without community support, an internet base just dosen't cut it. Also, having Novell or Redhat knocking on your door, asking for your support and donating some campaign contributions couldn't hurt either.
Really, if you want government support of open source, at least on the local level, schedule an appointment with your State Senator and Representitive, bring a few like minded friends. Most likely you'll be asked to go out to lunch or coffee with the official and before you know it you'll be chatting like old friends. It's what politicians do. Just most people never think they will be given the time of day. All they need is an indictation that within the district their are supporters willing to dedicate their time to help push through a initative, bill or amendment.
It really isn't that hard.
Just expect to have to baby-sit the thing for few years before you see results.
Your sarcasm needs work, or else you are seriously mistaken. A "semi-hostile" takeover of Wikimedia by an established dictionary entity would mean only one thing, they are trying to eliminate the free competition.
To assume that Britannica would release it's content or try to improve the Wiki is rather naive.
That, or I am a socialist with a grudge against anyone who seeks profit at the expense of the free exchange of information.
What can we possibly do to turn back the tide against the vicious spread of misinformation mongering? Surely, there must be something, a website, a petition, a campaign. I'm tired of just sitting on my hands and shrugging my shoulders. I'm not calling for a holy war here, but surely there must be some way to stop this kind of absolute crap.
Will parents ever learn to take responsibility for raising there own kids instead of trying to create the perfect enviroment for them to raise themselves?
Anyone who wants to censor violence from entertainment media because they can't take the time to teach their children right from wrong is bullshit.
Games arn't games anymore, they are still fun, but the time of innocence is long past.
When I play a violent war game, when I blow someone up or stab them.
I know one thing for sure, wars should be avoided at all costs.
This, is what video games teach us, the real horror of being there. That's something the watered down text books of our under funded public schools can't teach us.
Hillary Clinton won't become president, that much is for sure, she just alienated most of the 49 percent of democrats that would have voted for her.
Well, let's see, they provided weapons, military training and aid to the American Colonists in the Revolutionary War. They developed the most heavily armored and gunned tanks during the early German Blitz, one French Char B1-Bis held up an entire German Division for an entire day. One little short frenchie with a bad attitude almost conquered the entire world, twice.
:))
They've developed nuclear weapons, were one of the original founders of the European Union, who's Euro continues to dominate the American Dollar. They were one of the first modern countries to pick on the buzzword "Democracy" long before a bunch of colonists got pissed at their King's latest tax law.
Oh, did I mention numerous American, Australian and British courts have upheld the same reverse engineering proof of concept rulings?
You Sir, are an uneducated bigot.
(Note: I am not anti-American, I'm just hitting him where it hurts.
This was a completely irresponsible repost of a story less then 24 hours old. What the hell is up with the mods? I mean, come on! It's still on yesterday's news!
This is the first rehash of a story to actually make me lose my temper with the mods.
"It's a pretty narrowminded, materialistic and typical capitalistic view on things."
WHOA. WHOA. WHOA.
There is another way to view things?
News to me.
I can't figure out what Gigabye's roadmap is. I mean, the dual 6600GTs on a single card came out of no where. Now there is the dual PCI express board coming out that allows any two Video Cards to run in parellel, it's not SLI. Now they come out with this.
I dunno what they have in mind, but they sure are stiring things up a bit, but arn't they risking alienating nVidia with these "almost" SLI competetor alternatives?
I never thought about it that way before, I always kind of assumed that in the process of colonizing the stars we would become the kind of people you describe.
We really need something like this to justify our space program to the masses and to wake people up. There is more to life then just cheeseburgers and scoring that awsome IT job.
Human destiny lies with the stars, simply because eventually the Planet Earth will no longer be able to support human life, soon then later at our current population and resource expenditure.
On a long enough timeline, survival probability always drops to zero...
Sure they are still a monopoly but competition is competition. The only way Microsoft can really dethrone google is if they come out with a better internet search engine. If we get a improved system and outlook search, all the better. I really hope that this gives Linux the kick in the pants it needs for someone to come up with better system search solutions. Find is absolutely terrible in my humble opinion, especially it's tendency to freeze up when you stop a search. Lack of metadata search makes baby Linus cry. Bring me browser wars! Bring me os wars! Bring me search wars! These are the only kind of conflicts in which the consumer benefits, so we might as well encourage them!
Well I assume whereever the thing goes it's going to be supervised by the United States or U.N., if it's abused we will simply take it away and maybe steamroll the dictatorship into oblivion.
Developing countries, national crisis areas, there is practically no limit for something like this. I don't see it being easily abused either. Power is civilization and civilization is generally a good thing. :p
I seem to remember reading that some organization was setting up servers on abandoned oil rigs in international waters for just such a purpose. I don't know what happened to them. Something about a giant squid maybe?
Anyone remember that show? Some guy took over as the chief of the Washington DC Capital district and enacted major changes. The police department used a huge real time statistics tracking system and displayed it on a huge display. I think the idea was kind of revolutionary to the average joe like me; but I don't know if police were using a system like that before then.
:D
I wouldn't mind seeing systems like this implimentated in say, elected public offices to keep track of opinion areas, ethnic densities, crime rates, poll results, average pay. etc. To help them keep better tabs on what they need to improve and how to vote on what bill.
Oh, did I mention I plan on making a run or two for public office?
I want to play Halo 2, Half-Life 2 and Doom 3. To be able to play those games, I would be required to be an at least $390 video card and about $200 worth of ram. Heck, throw in a new $250 processor and a $120 motherboard while your at it.
Or I could just buy a $190 xbox and Playstation 2.
It's simple numbers.
I'm comfortable running a low end linux computer and a microsoft xbox for my gaming needs.
It's like Microsoft decided to play nice and gave us one of the main aspects of windows (games) without all the other os crap involved.
I can't wait for the Xbox 2.
Really these NEA are like a universal strip tease rather then big rocks of death. Imagine parking a couple of these things in orbit, we could mine it out and turn it into a space station. We could even use it as to start a human colony in orbit. If it was slightly bigger that is. :/