It just means we will be doing other IT related stuff.
Right on. People always affraid of jobs disappearing and often forget that there is always new jobs being created. It is called progress. Every major labor saving invention puts people out of job. But it frees them up to do something new.
My last sysadmin job (I'm in bizdev now) was at a brokerage firm. While the solution wasn't implemented before the higher ups pillaged the company, I and the network engineer came up with a way around this issue: use the F5 SSL accelerator to encrypt/decrypt the SSL stream, then use SSH port forwarding to make sure the cleartext data was encrypted between the machines. We never got it into production, but it worked great in the lab
Interesting. The only thing is - is SSH encryption any less computationaly intensive than SSL?
Web servers acts as presentation layer, putting together HTML pages. They contain no business logic they only know how to take a request, as app server for information and render this information. In addition to web servers, presentation can be IVR (phone interface), WAP serverfor mobile phone, or any other user interface. Because app server knows nothing on how to present user data, it could care less what is the presentation layer as long as presentation layer knows how to form a request adn read the responce.
App servers know how to process data - aka business logic. They take a simple request and return data only. They have no idea how to render data in user viewable way. Ideally app servers store no data in memory - cache sometimes, but not store.
DB servers. App servers store all data on DB servers. These now nothing of buseness logic or presentation, just know how to store data and ow to quickly retrieve it.
For real info read up on "3-tier architecture"
It sounds like you need to get a LOT more education before doing something like this. Not to discourage you but you are not going to find much more than basic pointers on Slashdot. You do need to do your own research and read up on theory and play with this yourself. The info is out there. Google is your friend.
What does it mean to not scale "vertically"? When I read that, the only thing that comes to mind is to put the boxes next to each other, not on top of each other. From context I gather that horizontally means extra machines, but what does vertically mean?
Horizontal scaling - adding more machine Vertical scaling - adding more CPU/Memory/etc to existing machines.
For example, a horizontally scaled application may have 20 1u 1cpu servers, a vertically scaled one has a Sun E15k heating up the room.
For "dropping in an extra server when needed without a lot of reconfiguring", what do you mean by "a lot of reconfiguring"? Obviously you need to get the machine, install the os, set up networking, install the web server, setup the web application, point it at the database, etc. How does the application being "stateless" help? I guess, what are some examples of state that an application can have that will make configuring an additional web server difficult?
Reconfiguring the application not the servers. A stateless web server does not store any user state. Meaning that if a user hits web server A for one request, and web server B for another, the user will not know the difference. Also meaning that if you add another server, you do not need to worry about conflicts, sharing data, etc. Stateless servers can be taken offline or brought online without any fuss. They become a commodity appliance and if you need more, you just get more. In realistic terms this means that if you need state for the application (login, etc) you either store the state on the client's machine in a cookie (BAD, all sorts of abuse is possible) or better store an temporary ID in a cookie (or in URL) and store state in App server or (better) DB. A lot of web servers and app servers offer clustering to solve the state issue. While this may or may not work, most of the time it is a marketing hype that rarely lives up to expectations and add extra load. It also violates KISS principle (Keep It Simple Stupid) and will give you more headache than it is worth.
Concerning the pseudo static data regeneration, what if the thing that was being updated was only accessed once every half-hour on average? I am assuming then that generating the page on demand would be better?
Use your brain. The idea is to lower CPU requirement and potential risk from overloading, not just to use a cool trick. Do whatever works best.
I don't really know what you mean by "MAKE YOUR WEB SERVERS STATELESS". I mean, they have to know if a request just came in, where the data is, what time it is etc, and that stuff gives it state. I am assuming you mean something else by stateless but I cannot figure it out.
State implies retained state across MULTIPLE connections/hits. Most application require state, however state does not need to be kept on the web servers and sometimes not even on app servers.
It may or may not be a great idea depending on your situation. For one - the cost of SSL card for F5 is so high, it may be easier to just get extra servers. For another, I work with some banking applications and having data sent cleartext, even on an inside network directly connected to load balancers is NOT a valid option.
However if local security can be ignored and you have the money to spend, F5's offer a nice offload of encryption processing. But then again, so do hardware cards for individual servers.
These are some very basic thoughts on the subject. They may not be 100% right for you, but will get you thinking in the right way:
Rule 1 - Three tier archictecture is popular for a reason - it works. Offload user interface (web) to dedicated boxes, make application itself run on separate boxes and make database separate
Rule 2 - When possible, scale horizontaly not vertically. Make sure your application is as stateless as possible and is capable of you just dropping in an extra server when needed without a lot of reconfiguring. Make sure you can survive a loss of a server without loss of data. Lots of cheap servers will most always work out better (and cheaper) than one big ass box.
Rule 3 - Make as much of your application as static as possible. Even pseudostatic data (something that updates every minute or so) should be made static and have a process re-generating it every minute or so. Not wasting your CPU time to render a menu or something on every hit will add up fast under heavy stress.
Rule 4 - Strip your HTML. For example, some crappier web languages (think ColdFusion) have a tendancy of inserting spaces for every line of code etc. A large application running CF (dont ask) would insert enough spaces to make a simple page hundreds of kb in size. Just turning on "the write to output only on demand" option will drop size of the page to next to nothing. So know what it is that you are producing on output and make sure it is lean. Turning on server side compression solves this better, however adds to CPU requirements. On trully stateless web servers this just mean you need more web servers. So MAKE YOUR WEB SERVERS STATELESS.
Rule 5 - Know how many users your upstream connection can handle (in simplest terms - average size of HTML communication * number of users) and make sure you do not exceed it. Limit your connectivity at load ballancer. Having some users not be able to access your site is better than having ALL users not be able to access your site. Make sure you get plenty of bandwidth to spare. If you are setting up a multi-site presence, make sure your intersite communication is a - not going over same line as incoming and b - has sufficient bandwith and latency to serve the traffic.
Rule 6 - Professional load testing tools cost big bucks. But if you are carefull you can fake it with some open source software. Google it. When testing remember to take into consideration the limitation of your tester system and bandwidth.
Why not have it compute the stamp before you send the mail? You start a new mail window, that least intensive of applications. In the background it calculates the stamp while you type.
Under that system, you could make the stamps as much as a minute. Very few e-mails are written in less than twenty seconds, most take a few minutes. Really short messages go via IM. You still queue it to go after the stamp is ready to deal with the short e-mails, of course.
The reason this will not work is due to the way a typical SMTP connection actually works -
Steps:
1 - User writes email 2 - User sends email to their ISP's SMTP server 3 - The ISP SMTP tranfers message to destination SMTP server 4 - Destination SMTP server delivers mail to destination mailbox 5 - Profit (just kididng)
The checksum check will actually occur at step 3. Destination server will request the checksum from ISP's SMTP server - NOT FROM USERS MACHINE. Which means that the cost to large or even medium sized ISPs will be very significant. This means unless the end user machine will start sending email out directly to destination ISPs (bypassing step 2, a practice some broadband providers block to curb spam bots), this scheme will cost significant amount of money to ISPs in processing power. This also means that what you propose - calculation of the checksum on user machine during writing of the message is impossible.
True solution to the issue (or at least BEGININGS of a solution) should start with authentication of authorized SMTP servers for domains - like what Yahoo/Google/Microsoft & others were trying to do via DNS a few months back. (Whatever happened to that, BTW??)
How so? I can see the increase in bandwidth but it is not all that significant if spammers are not paying for it (as is the case with worm bots, etc). Adding support for retry to the bots would be trivial. Its a temporary solution at best.
Oh, and its not too tough to move someone from a whitelist to the blacklist.
Yes, but it is not any tougher adding them to a blacklist without going through the whole hoopla in the first place.
If the mail never comes back, then the sender is now blacklisted. If the mail does come back, the sender is whitelisted...so this will work until spammers add a retry to the mailers - at which time they are whitelisted.
Joe Sixpack wants to send a mail. If it takes him an hour to parse a key, he's not going to mail his mother anymore.
The general idea is that it will take a relatively small yet significant time to compute. So for example (also random) 30 seconds. Joe Sixpack will not notice 30 second delay on his computer for one email. However Jack Spammer who sends a million emails will need 500,000 minutes to compute the sums. A huge difference.... until you figure out that Joe Sixpack computer's spyware is what actually doing the computing.
There are no drivers required to make TurboNet active (Tivo actually added TurboNet drives to their 3.x software) However ftp/telnet are not enabled by default. The easiest thing to do it is to follow directions for TurboNet install CD - it makes enabling ftp/telnet (as well as assigning a static IP for DHCP chalenged)a heck of a lot easier for average user - not to mention fully automated.
As for extraction, this generally requires extra software. Your best bet is reading a lot of forum postings. There are multiple ways to extract data, look up TyStudio, but there are other tools too.
Lastly, to extract from S1 DTivo, you will need to either disable encryption PRIOR to recording or use a kernel module that will decrypt video for you. See same forum for details.
You mean less more usefull every day? They are cutting capabilities of FUTURE services (NFL restriction, for example, is on LONG DISTANCE TRANSFER of shows), not available on Tivo now.
and call her MythGF, cuz you certainly don't have the real thing.
"MythGF" is probably closer to truth than you think. Spending so much time trying to make MythTV do anything resembling usablity, "RealGF" is getting cranky.
Gmail most certainly does use SSL for viewing mail. I'm using it right now.
Hmm, interesting. I just edited the URL to use https, and sure enough, it is running in SSL. Even though it uses SSL for login in both cases, it will display mail in whatever mode you started the connection with. It appears that it is up to user - good to know and I stand corrected.
https is not cached. if your webmail isnt using ssl then, once again, this is not google's problem.
Well, a quick check shows that neither Yahoo webmail, not GMail use SSL for email (for login, yes, but not for viewing email) - I imagine that Hotmail and other popular webmail providers are same. I understand that your self-run Squirrel installation will have SSL on it (mine does), but that does no good to most webmail users.
Is it really google's problem? I do not think it is. Is it a privacy issue - hell, yeah!!
What he's saying is that a system administrator is supposed to install software, not end users. These aren't Windows95 boxes at your home, these are powerful business computing devices running Windows XP Professional.
Where is your proof? What version of Firefox? Have you reported this to the FF developers?
Proof: None other than what Ad-Aware picked up. I happily clicked on delete and that was it. I do not even remember the name of it, just noticed it was an executable and under firefox plugin directory. I found it to be odd enough to note, but not a big deal.
Better or not it was there, executable plugin, and was downloaded via Firefox without my conscent. Yes, I know that Slashdot policy is "if it is Microsoft blame the company if it is Linux/OSS blame the user". Does not make it always right.
You mean you found more cookies in Firefox because you use it more often?
Surely you got to be kidding.
If you cannot tell a difference between a cookie and actual spyware you should not be posting here. This was some sort of a browser plugin I never heard of (can check what the name of it was) and I did not install anything so I am guessing this went through a website and found some way around the security in mozilla. Furthermore I have Spybot's Resident software running and it did not catch it.
I am running IE and FireFox (using both). Tonight I run AdAware to find spyware in FireFox and not in IE. The more people use Firefox the more Firefox spywhere there will be. Switching to Firefox does not solve the problem in thed long run.
The first "production transistor radio" was actually produced by Raytheon. As part of an experiment, transistors that were mil-spec drop outs were re-used with other circuitry as "drop in" replacements for vacuum tubes. I do not remember how many were produced, but there was a genuine production run.
I believe the article menations the first pocket-sized transistor radio - not the first transistor radio in general.
Slashdot Mirror
It just means we will be doing other IT related stuff.
Right on. People always affraid of jobs disappearing and often forget that there is always new jobs being created. It is called progress. Every major labor saving invention puts people out of job. But it frees them up to do something new.
-Em
My last sysadmin job (I'm in bizdev now) was at a brokerage firm. While the solution wasn't implemented before the higher ups pillaged the company, I and the network engineer came up with a way around this issue: use the F5 SSL accelerator to encrypt/decrypt the SSL stream, then use SSH port forwarding to make sure the cleartext data was encrypted between the machines. We never got it into production, but it worked great in the lab
Interesting. The only thing is - is SSH encryption any less computationaly intensive than SSL?
-Em
Look up "three-tier architecture".
A quick rundown:
Web servers acts as presentation layer, putting together HTML pages. They contain no business logic they only know how to take a request, as app server for information and render this information. In addition to web servers, presentation can be IVR (phone interface), WAP serverfor mobile phone, or any other user interface. Because app server knows nothing on how to present user data, it could care less what is the presentation layer as long as presentation layer knows how to form a request adn read the responce.
App servers know how to process data - aka business logic. They take a simple request and return data only. They have no idea how to render data in user viewable way. Ideally app servers store no data in memory - cache sometimes, but not store.
DB servers. App servers store all data on DB servers. These now nothing of buseness logic or presentation, just know how to store data and ow to quickly retrieve it.
For real info read up on "3-tier architecture"
It sounds like you need to get a LOT more education before doing something like this. Not to discourage you but you are not going to find much more than basic pointers on Slashdot. You do need to do your own research and read up on theory and play with this yourself. The info is out there. Google is your friend.
HTH
-Em
What does it mean to not scale "vertically"? When I read that, the only thing that comes to mind is to put the boxes next to each other, not on top of each other. From context I gather that horizontally means extra machines, but what does vertically mean?
Horizontal scaling - adding more machine
Vertical scaling - adding more CPU/Memory/etc to existing machines.
For example, a horizontally scaled application may have 20 1u 1cpu servers, a vertically scaled one has a Sun E15k heating up the room.
For "dropping in an extra server when needed without a lot of reconfiguring", what do you mean by "a lot of reconfiguring"? Obviously you need to get the machine, install the os, set up networking, install the web server, setup the web application, point it at the database, etc. How does the application being "stateless" help? I guess, what are some examples of state that an application can have that will make configuring an additional web server difficult?
Reconfiguring the application not the servers. A stateless web server does not store any user state. Meaning that if a user hits web server A for one request, and web server B for another, the user will not know the difference. Also meaning that if you add another server, you do not need to worry about conflicts, sharing data, etc. Stateless servers can be taken offline or brought online without any fuss. They become a commodity appliance and if you need more, you just get more. In realistic terms this means that if you need state for the application (login, etc) you either store the state on the client's machine in a cookie (BAD, all sorts of abuse is possible) or better store an temporary ID in a cookie (or in URL) and store state in App server or (better) DB. A lot of web servers and app servers offer clustering to solve the state issue. While this may or may not work, most of the time it is a marketing hype that rarely lives up to expectations and add extra load. It also violates KISS principle (Keep It Simple Stupid) and will give you more headache than it is worth.
Concerning the pseudo static data regeneration, what if the thing that was being updated was only accessed once every half-hour on average? I am assuming then that generating the page on demand would be better?
Use your brain. The idea is to lower CPU requirement and potential risk from overloading, not just to use a cool trick. Do whatever works best.
I don't really know what you mean by "MAKE YOUR WEB SERVERS STATELESS". I mean, they have to know if a request just came in, where the data is, what time it is etc, and that stuff gives it state. I am assuming you mean something else by stateless but I cannot figure it out.
State implies retained state across MULTIPLE connections/hits. Most application require state, however state does not need to be kept on the web servers and sometimes not even on app servers.
HTH
-Em
It may or may not be a great idea depending on your situation. For one - the cost of SSL card for F5 is so high, it may be easier to just get extra servers. For another, I work with some banking applications and having data sent cleartext, even on an inside network directly connected to load balancers is NOT a valid option.
However if local security can be ignored and you have the money to spend, F5's offer a nice offload of encryption processing. But then again, so do hardware cards for individual servers.
-Em
These are some very basic thoughts on the subject. They may not be 100% right for you, but will get you thinking in the right way:
Rule 1 - Three tier archictecture is popular for a reason - it works. Offload user interface (web) to dedicated boxes, make application itself run on separate boxes and make database separate
Rule 2 - When possible, scale horizontaly not vertically. Make sure your application is as stateless as possible and is capable of you just dropping in an extra server when needed without a lot of reconfiguring. Make sure you can survive a loss of a server without loss of data. Lots of cheap servers will most always work out better (and cheaper) than one big ass box.
Rule 3 - Make as much of your application as static as possible. Even pseudostatic data (something that updates every minute or so) should be made static and have a process re-generating it every minute or so. Not wasting your CPU time to render a menu or something on every hit will add up fast under heavy stress.
Rule 4 - Strip your HTML. For example, some crappier web languages (think ColdFusion) have a tendancy of inserting spaces for every line of code etc. A large application running CF (dont ask) would insert enough spaces to make a simple page hundreds of kb in size. Just turning on "the write to output only on demand" option will drop size of the page to next to nothing. So know what it is that you are producing on output and make sure it is lean. Turning on server side compression solves this better, however adds to CPU requirements. On trully stateless web servers this just mean you need more web servers. So MAKE YOUR WEB SERVERS STATELESS.
Rule 5 - Know how many users your upstream connection can handle (in simplest terms - average size of HTML communication * number of users) and make sure you do not exceed it. Limit your connectivity at load ballancer. Having some users not be able to access your site is better than having ALL users not be able to access your site. Make sure you get plenty of bandwidth to spare. If you are setting up a multi-site presence, make sure your intersite communication is a - not going over same line as incoming and b - has sufficient bandwith and latency to serve the traffic.
Rule 6 - Professional load testing tools cost big bucks. But if you are carefull you can fake it with some open source software. Google it. When testing remember to take into consideration the limitation of your tester system and bandwidth.
-Em
Why not have it compute the stamp before you send the mail? You start a new mail window, that least intensive of applications. In the background it calculates the stamp while you type.
Under that system, you could make the stamps as much as a minute. Very few e-mails are written in less than twenty seconds, most take a few minutes. Really short messages go via IM. You still queue it to go after the stamp is ready to deal with the short e-mails, of course.
The reason this will not work is due to the way a typical SMTP connection actually works -
Steps:
1 - User writes email
2 - User sends email to their ISP's SMTP server
3 - The ISP SMTP tranfers message to destination SMTP server
4 - Destination SMTP server delivers mail to destination mailbox
5 - Profit (just kididng)
The checksum check will actually occur at step 3. Destination server will request the checksum from ISP's SMTP server - NOT FROM USERS MACHINE. Which means that the cost to large or even medium sized ISPs will be very significant. This means unless the end user machine will start sending email out directly to destination ISPs (bypassing step 2, a practice some broadband providers block to curb spam bots), this scheme will cost significant amount of money to ISPs in processing power. This also means that what you propose - calculation of the checksum on user machine during writing of the message is impossible.
True solution to the issue (or at least BEGININGS of a solution) should start with authentication of authorized SMTP servers for domains - like what Yahoo/Google/Microsoft & others were trying to do via DNS a few months back. (Whatever happened to that, BTW??)
-Em
It kills off all of the owned windows machines
How so? I can see the increase in bandwidth but it is not all that significant if spammers are not paying for it (as is the case with worm bots, etc). Adding support for retry to the bots would be trivial. Its a temporary solution at best.
Oh, and its not too tough to move someone from a whitelist to the blacklist.
Yes, but it is not any tougher adding them to a blacklist without going through the whole hoopla in the first place.
-Em
If the mail never comes back, then the sender is now blacklisted. If the mail does come back, the sender is whitelisted. ..so this will work until spammers add a retry to the mailers - at which time they are whitelisted.
-Em
Joe Sixpack wants to send a mail. If it takes him an hour to parse a key, he's not going to mail his mother anymore.
The general idea is that it will take a relatively small yet significant time to compute. So for example (also random) 30 seconds. Joe Sixpack will not notice 30 second delay on his computer for one email. However Jack Spammer who sends a million emails will need 500,000 minutes to compute the sums. A huge difference.... until you figure out that Joe Sixpack computer's spyware is what actually doing the computing.
-Em
There are no drivers required to make TurboNet active (Tivo actually added TurboNet drives to their 3.x software) However ftp/telnet are not enabled by default. The easiest thing to do it is to follow directions for TurboNet install CD - it makes enabling ftp/telnet (as well as assigning a static IP for DHCP chalenged)a heck of a lot easier for average user - not to mention fully automated.
As for extraction, this generally requires extra software. Your best bet is reading a lot of forum postings. There are multiple ways to extract data, look up TyStudio, but there are other tools too.
Lastly, to extract from S1 DTivo, you will need to either disable encryption PRIOR to recording or use a kernel module that will decrypt video for you. See same forum for details.
You mean less more usefull every day? They are cutting capabilities of FUTURE services (NFL restriction, for example, is on LONG DISTANCE TRANSFER of shows), not available on Tivo now.
Anything you really want to keep should be stuck onto a DVD burner.
TurboNet for S1 (or USB network card for S2) and your show is on your video server where XBox can play it back any time you want.
and call her MythGF, cuz you certainly don't have the real thing.
"MythGF" is probably closer to truth than you think. Spending so much time trying to make MythTV do anything resembling usablity, "RealGF" is getting cranky.
Gmail most certainly does use SSL for viewing mail. I'm using it right now.
Hmm, interesting. I just edited the URL to use https, and sure enough, it is running in SSL. Even though it uses SSL for login in both cases, it will display mail in whatever mode you started the connection with. It appears that it is up to user - good to know and I stand corrected.
-Em
https is not cached. if your webmail isnt using ssl then, once again, this is not google's problem.
Well, a quick check shows that neither Yahoo webmail, not GMail use SSL for email (for login, yes, but not for viewing email) - I imagine that Hotmail and other popular webmail providers are same. I understand that your self-run Squirrel installation will have SSL on it (mine does), but that does no good to most webmail users.
Is it really google's problem? I do not think it is. Is it a privacy issue - hell, yeah!!
-Em
Not if it is also caching the web pages you visited - including webmail.
What he's saying is that a system administrator is supposed to install software, not end users. These aren't Windows95 boxes at your home, these are powerful business computing devices running Windows XP Professional.
Point taken.
Where is your proof? What version of Firefox? Have you reported this to the FF developers?
Proof: None other than what Ad-Aware picked up. I happily clicked on delete and that was it. I do not even remember the name of it, just noticed it was an executable and under firefox plugin directory. I found it to be odd enough to note, but not a big deal.
Version: 0.9.3
NO NOT COOKIES.
Better or not it was there, executable plugin, and was downloaded via Firefox without my conscent. Yes, I know that Slashdot policy is "if it is Microsoft blame the company if it is Linux/OSS blame the user". Does not make it always right.
-Em
You mean you found more cookies in Firefox because you use it more often?
Surely you got to be kidding.
If you cannot tell a difference between a cookie and actual spyware you should not be posting here. This was some sort of a browser plugin I never heard of (can check what the name of it was) and I did not install anything so I am guessing this went through a website and found some way around the security in mozilla. Furthermore I have Spybot's Resident software running and it did not catch it.
-Em
Why is a normal user allowed to install programs in the first place?
Because that computer thing is meant to be USEFUL
I am running IE and FireFox (using both). Tonight I run AdAware to find spyware in FireFox and not in IE. The more people use Firefox the more Firefox spywhere there will be. Switching to Firefox does not solve the problem in thed long run.
The first "production transistor radio" was actually produced by Raytheon. As part of an experiment, transistors that were mil-spec drop outs were re-used with other circuitry as "drop in" replacements for vacuum tubes. I do not remember how many were produced, but there was a genuine production run.
I believe the article menations the first pocket-sized transistor radio - not the first transistor radio in general.
-Em