Slashdot Mirror


User: srealm

srealm's activity in the archive.

Stories
0
Comments
126
First seen
Last seen
Profile
(view on slashdot.org)

Comments · 126

  1. GDDR4 on Samsung to Produce Faster Graphics Memory · · Score: 1

    Missing Floppies?

  2. Re:Define your thread's purposes on Learning High-Availability Server-Side Development? · · Score: 1

    This was on Linux, using C++ - and very little in the way of libraries. We DID of course have to modify some system settings (like max FD's per process, etc). The socket library we used was custom written, and everything was non-blocking.

    We used Berkeley DB for a data repository (read-only, most of this data loaded in memory where we could afford it - or at least partially loaded in memory to speed up accessing). The files were mirrored from a central repository, and we had some Small File Caching ability so that frequently sent files less than a certain size would be loaded into memory and LUA'd out.

    The app WAS serving up P2P files, and even had some logic in there to do things like restrict the bandwidth system-wide. I am currently working on re-writing some of the stuff I wrote for this application in open-source library form (still C++). But that is still a work in progress.

  3. Define your thread's purposes on Learning High-Availability Server-Side Development? · · Score: 5, Informative

    I've worked in multiple extremely super-scaled applications (including ones sustaining 70,000 connections at any one time, 10,000 new connections each minute, and 15,000 concurrent throttled file transfers at any one time - all in one application instance on one machine).

    The biggest problem I have seen is people don't know how to properly define their thread's purpose and requirements, and don't know how to decouple tasks that have in-built latency or avoid thread blocking (and locking).

    For example, often in a high-performance network app, you will have some kind of multiplexor (or more than one) for your connections, so you don't have a thread per connection. But people often make the mistake of doing too much in the multiplexor's thread. The multiplexor should ideally only exist to be able to pull data off the socket, chop it up into packets that make sense, and hand it off to some kind of thread pool to do actual processing. Anything more and your multiplexor can't get back to retrieving the next bit of data fast enough.

    Similarly, when moving data from a multiplexor to a thread pool, you should be a) moving in bulk (lock the queue once, not once per message), AND you should be using the Least Loaded pattern - where each thread in the pool has its OWN queue, and you move the entire batch of messages to the thread that is least loaded, and next time the multiplexor has another batch, it will move it to a different thread because IT is least loaded. Assuming your processing takes longer than the data takes to be split into packets (IT SHOULD!), then all your threads will still be busy, but there will be no lock contention between them, and occasional lock contention ONCE when they get a new batch of messages to process.

    Finally, decouple your I/O-bound processes. Make your I/O bound things (eg. reporting via. socket back to some kind of stats/reporting system) happen in their own thread if they are allowed to block. And make sure your worker threads aren't waiting to give the I/O bound thread data - in this case, a similar pattern to the above in reverse works well - where each thread PUSHING to the I/O bound thread has its own queue, and your I/O bound thread has its own queue, and when it is empty, it just collects the swaps from all the worker queues (or just the next one in a round-robin fashion), so the workers can put data onto those queues at its leisure again, without lock contention with each other.

    Never underestimate the value of your memory - if you are doing something like reporting to a stats/reporting server via. socket, you should implement some kind of Store and Forward system. This is both for integrity (if your app crashes, you still have the data to send), and so you don't blow your memory. This is also true if you are doing SQL inserts to an off-system database server - spool it out to local disk (local solid-state is even better!) and then just have a thread continually reading from disk and doing the inserts - in a thread not touched by anything else. And make sure your SAF uses *CYCLING FILES* that cycle on max size AND time - you don't want to keep appending to a file that can never be erased - and preferably, make that file a memory mapped file. Similarly, when sending data to your end-users, make sure you can overflow the data to disk so you don't have 3mb data sitting in memory for a single client, who happens to be too slow to take it fast enough.

    And last thing, make sure you have architected things in a way that you can simply start up a new instance on another machine, and both machines can work IN TANDEM, allowing you to just throw hardware at the problem once you reach your hardware's limit. I've personally scaled up an app from about 20 machines to over 650 by ensuring the collector could handle multiple collections - and even making sure I could run multiple collectors side-by-side for when the data is too much for one collector to crunch.

    I don't know of any papers on this, but this is my experience writing extremely high performance network apps :)

  4. Huh? on A Cynic Rips Open Source · · Score: 1

    I'm sorry, his argument doesn't make sense. Especially the ending of 'The only people who really USE open source are universities or people with no other option'.

    I work for Large Wall St. Financial (tm). All our core systems are Linux. Lots of Java (now open source) and C++ (using gcc) here. The majority of the C++ code uses boost and a number of other open source products (Berkeley DB etc).

    I'm pretty sure the company COULD afford to beat their vendors over the head and get the price point they want, but the fact is, using Linux systems and these open source products give them two things the vendors so far have not seemed to come through with:
    1. The resulting products are faster.
    2. If there is a problem, they can debug the entire system, even the code they didn't write (the open source).

    These are the top two reasons open source is taking hold, nothing to do with price. The only people switching to open source because of price are governments and such. Most of Corporate America's open source use is simply because the open source product is a better product at this point.

    Thats not to say there is not proprietary software in use too, there are some challenges that open source just doesn't address, or is behind on because the domain is complex and a company that can pay 50 people's full-time salary to work on that specific problem domain are going to make a better product than a few open source guys working on it in their spare time. But any large open source product (with a big contributor base, say a few hundred) will usually end up surpassing any vendor product.

    I call grandstanding.

  5. PHBs? on New Technique for Recycling PCBs · · Score: 2, Funny

    I first read that as 'Recycling PHBs' ... now that is something I would REALLY like to see, because lord knows I need a new boss!

  6. Re:How about we take the easy way out? on The Future of Packaging Software in Linux · · Score: 2, Interesting

    One more thing.

    Make it easy to install software that has DIFFERENT dependencies.

    The classic example I always give is LDAP - you either want it or you don't. However why should I install LDAP because some application decided to link against LDAP? Or conversely, what if the application was NOT linked against LDAP and I want LDAP support in the app (which is available in the app itself).

    Unfortunately, the only real solution to this is source based installs or having a compile for every combination of dependencies. But possibly something that will allow you to specify dependencies, and if you break from the 'blessed' dependencies, still has the ability to compile the base version (with no extra steps required by the user apart from saying 'yes, OK, do the compile'). Without requiring it to recompile ALL packages involved (ie. if the LDAP is coming from some library used by the app you requested to be installed, it should only recompile that library, not everything or the final app (since its using shared code and not using LDAP directly).

    And as an addition to that, there should be a way to 'offshore' said building to a build server (or servers) - that will keep the package once built (with record of its dependencies) - so that if you are deploying on a large scale, you can use the SAME package manager that comes with the default distro on ALL your boxes, and install your package with custom dependencies (triggering a compile), and then when you install that package with the same dependencies later, it will just pick up the pre-built version, just as if it came from the official pipe.

    These kind of problems are rarely solved at all by packaging systems, and usually installing from a source archive for a single package on a binary distribution is a pain in the neck - especially if you want to actually build from a source archive with ALTERED dependencies. Right now, only completely source-based distributions allow you to change dependencies, but then you need to pay the penalty of building EVERYTING (yes, Gentoo and such have their pre-built packages for large modules like Gnome or KDE, but you still end up building a lot).

    Basically, I'm saying I could not really use a binary packaging system unless it had the ability to fall back to source altering its dependencies (which means the package needs to know HOW to compile disabling and enabling certain things (ie. configure flags), not just how to compile the same way the binary distro was made), and do so seamlessly and without forcing the user to go though 7000 additional steps or have intimate knowledge of how to compile an application. Remember, from the amdin's point of view, they just don't want to install LDAP unnecessarily, they don't specifically want to have to know how to recompile XYZ package to exclude LDAP support, or even if they know how to, want to have to do that for every package that is optionally supporting LDAP using manual procedures to disbale LDAP support.

    This, and the fact that getting 0-day releases for certain packages when a new version comes out (sometimes a new release fixes a specific bug you have been waiting to be fixed) without having to recompile it all myself and break the packaging system or have to get more intimate with the packaging system than I want are the reasons I switched back off of Ubuntu and back to Gentoo. I tried Ubuntu because I wanted to see what life would be like without having to compile all the time - but in the end found I could suffer the compile (especially with things like distcc) to have a system the way I like it without extra crap installed because it happened to be compiled with support I don't need, want and will never use.

    PreZ :)

  7. Re:I'll take two. on Verizon To Pump $18B Into FiOS · · Score: 1

    If you're savvy enough, and can get an IP address tunnelled to you this is no problem.

    I have FiOS, I also happen to have a server colocated elsewhere. I setup a simple GRE tunnel of some IP addresses to my FiOS line (using dyndns so the colocated server can automatically re-point the GRE tunnel if my FiOS connection drops and re-establishes). Works great, and is very fast.

    If you wanted to use a more advanced tunneling solution (ie. a proper client/server tunnel) you could do it without dyndns, just get your system on the FiOS side to reconnect if it loses a connection to the tunnel server. With this methodology you could also do things like add encryption, etc.

    Either way, you can host something on your FiOS if you try hard enough, even without using alternate ports (as a previous poster suggested). Of course, the disadvantage of the tunneling approach is that a) your throughput on that IP is limited to the speed at which the server tunneling you an IP address can transfer data, and b) you are guarenteed to add whatever the latency is between you and your tunnel server to every connection on that IP address.

  8. Re:On empires.... on Mono Blocked from MS Conference · · Score: 1

    So Microsoft should start moving its employees to other offices and paying for relocation instead of firing them when it finds out they're pedophiles?

    Seriously though, The RC church is not the best example really. Remember, at one time, christianity ment catholocism. Every denomination of christianity that exists today is therefore a splinter off of RC. Which makes christianity the most fractured religion there is, since a good proportion of denominations disagree violently with others that are technically supposed to be the SAME religion.

    Some how, I don't think MS could survive thousands of other companies (some large, some small) fracturing off of its own corporate body, selling the same product, but changing a few bells and whistles to distinguish themselves. Plus you seem to forget that the RC church right now is probably the least relevent it has been in well over 1000 years. Yes, there was a time when the RC church said something, and all of europe followed for fear of their soul. However the RC membership, as a percentage of the population, has been in decline for some time.

    The other thing you have to remember is that a very large part of the Roman Empire's fall was a failure of leadership, some bad decisions (such as not waiting for the rest of the army before launching a decisive battle) and complacency (ie. allowing the military to fall into a boarder protection role and losing its ability to work as an effective fighting force). Plus of course, it had overstreached itself to what it could maintain.

    Your view is also flawed since you also don't take into account what the Roman empire did for the places it conquered. Sure, it might not be fun being conquered, however generally it dragged the people living in those places into a new era, and once the people accepted roman taxes, the quality of life increased dramatically for the most part with the introduction of modern facilities and concepts (for example, the benefits of running water). Similarly, when the romans left, the population was both again plunged back about 1000 years socially/technologically, and left a prime target for an invading force (Britain is a prime example).

    That said, I'm not sure I'd consider MS the roman empire OR the RC church.

  9. Re:Free Boxes on FedEx Cracks Down on Box Furniture, Citing DMCA · · Score: 1

    Personally, I use UPS for toilet use ...

    After all, even their advertising campeign says 'what can brown do for you'...

  10. Company team building exercises ... on Offshoring to a Ship in International Waters · · Score: 1

    "All employees are required to enroll in one of our two offered team building courses.

    'Endurance Swimming', to ensure you can get back to the shore in case of this office sinking.

    or

    'How to be an effective boat anchor', for those who are out of breath after the 30ft walk to the coffee machine.

  11. Simply not far enough ... on Offshoring to a Ship in International Waters · · Score: 1

    Didn't anyone bother telling them that the US has terrirorial waters of 12 nautical miles, and an exclusive exconomic zone of 200 nauticle miles?

    http://www.encyclopedia.com/html/w1/waters-t.asp

  12. oops? on Bill Gates to Receive Honorary UK Knighthood · · Score: 1

    I dunno, the Queen is getting on now days, what do you think the likelyhood of her coming over faint at the exact moment she is knighting him, and collapsing on the sword (putting more pressure on it than she intended), and accidentally decapitaing poor ol' Bill?

    For all you nay sayers, leave me alone with my dreams!

  13. Re:VNC?? on Which VNC Software Is Best? · · Score: 1

    Ve are ze Vepublicans, ve hef veys of meking you vote!

  14. Re:No syntax files for c++ yet on JOE Hits 3.0 · · Score: 1

    I would hope you also contributed them back to the joe team :)

    Incietnally, I just set *.cpp (and *.C and *.cxx) to use my c.jsf file, the differences are minor anyway.

  15. Re:Bad logic on CPA Googles For His Name, Sues Google For Libel · · Score: 1

    Theres a HUGE difference between civil and criminal law.

    If you gave child porn with a disclaimer, you could still be convicted in a CRIMINAL court, but not sued.

  16. Re:Does TCO include the cost of virus attack ?? on Energy Company Refutes Windows TCO Claims · · Score: 1

    Actually, there is a HUGE difference.

    In most cases, mail is opened (and even attachments executed) as a user. The windows security model is poor, and an application can actually get Administrator rights without too much trouble.

    On *nix, the limitation of damage for an executed virus is usually anything that user has write permissions to. On windows, you can pretty much count the whole system as given over entirely to the virus.

    This is even besides the situation whereby for the most part, running windows as a non-administrator (ignoring the poor security model) is too restrictive for somoene to do it on their own machine, so most users set their own account up as an administrator. As opposed to *nix where you can install/run apps just fine as a standard user (in your home directory), and if you want to install something system wide, you 'su' - you don't need to log completely out, and then log-in again just to install an app (which is too much effort for most users).

    Plus, as previously mentioned, windows has a habit of automatically doing stuff (executing scripts to show 'web-enabled' content, auto-running whats on a CD, etc). You have to explicitly disable this behavior on windows machines -- and most people just don't know how.

  17. Re:only Democrats believe that: on MPAA, RIAA Seek Permanent Antitrust Exemption · · Score: 1
    If you legalize drugs, less people will use them.

    I encourage you to look at the drug statistics in The Netherlands vs. the USA (proportionate to the population of course, to keep the statistics fair).

    You can solve all crime problems by taking guns away from people who obtain them legally.
    As a democrat, I don't believe in this - however I DO believe that ALL firearms should be much harder to get, licensed NATION WIDE, subject to scrutany (ie. finding out WHY they need a gun), and stiff criminal penalties be enforced for improper storage of a firearm and ammunition (up to and including being automatically part of any crime committed with the weapon if it was improperly stored - which includes keeping the ammunition and the firearm in the same place).

    Spotted owls have more rights than children.
    Again, I don't agree - but that doesnt mean I think that because they're non-human, their habitats should be taken willy-nilly or they should be allowed to driven into extinsion. EVERY animal diserves the right to at least survive.

    All adolescents have sex, so it's no use teaching abstinence.
    Absolutely not - however it is also foolhardy to think that all adolescents will take these teachings to heart, and practice it - therefore giving them the tools they need to help them stop their small mistakes as adolescents balooning into life-changing mistakes (babies, STDs, etc) can't hurt either.

    It is the government's obligation to provide everyone with jobs, healthcare, bread, circuses, and to protect them from their own stupidity.
    I agree only with healthcare and bread. Its part of that 'life, liberty, and persuit of happiness' thing. Healthcare and bread help ensure they have life, and are able-bodied enough to persue happiness, liberty is achieved by not stripping people of their civil rights.

    Perjury is a crime ... unless you are the president.
    Perjury is a crime regardless. Nobody is questioning whether Clinton lied. However it should NOT be something that distracts the country, and the president from running the country. Sure, he should do the community service or pay the fine the court imposes, and move on. Did you notice while Clinton was in office, most major issues could not actually hit the front pages because the republicans were too interested in his sex life?

    The solution to every problem involves raising taxes.
    Again I disagree -- however one is for certain, the solution to everyone's problem is NOT increasing the country's national debt, cutting funding for emergency services, or becoming Wyett Erp and spending many billions fighting a war that the rest of the world said they would help you fight (and bare the financial burdon of) *IF* you could actually back up your claims of wrongdoing with something other than hot air.

    The words "separation of church and state" are in the constitution.
    You're correct - what the constitution ACTUALLY says is "Congress shall pass no law respecting an establishment of religion, or prohibiting the free exercise thereof." This has been expanded by CASE LAW to be the full fledged separation of church of state - and remember, MOST of the laws in this country are actually fleshed out, expanded upon, or even created based on case law. It can be overturned, but as it stands, the law is basically interpereted as "the government can not be shown to favour any religion over another", which is why things like the Ten Commandments have been ordered removed from courthouses - it is a governmental building, and that is obviously showing favouritism to Christianity. A re-creation of the arc of the covernet, or the Torah or the Koran, or a statue of Buddah would have all been removed under similar pretext.

    Atheism is not a religion and zero is not a number.
    By the definition of religion, Atheism is NOT a religion - it is the staunch rejection

  18. FUD! on iTunes: Don't Leave Home With Them · · Score: 1

    Bah, the license only says you can't BUY the music from iTunes outside the US.

    Later in the license, they even give you explicit license to Burn and Export your music. (emphasis mine)

    Theres also nothing about taking you're music overseas, just that you buy any new stuff while overseas.

  19. Sympathy on MPAA to Launch Anti-Piracy Commercials · · Score: 1

    I'm sympathetic to these artists ...

    So I'll give them payment the same way I'd give a bum on the street charity - I'll send them some food.

    Hey! At least I know it won't be used to buy drugs and alcohol ... at least, not ouside of prison ;)

  20. Kevin Mitnick on Meet the DoJ's 'Anti-Piracy' Lawyers · · Score: 3, Interesting

    When Kevin Mitnick was tried, the 'damages' listed in his trial were calculated simply by taking the arbitary value that DEC, et al. attributed to the software he copied, and adding them all together (regardless of any actual damages suffered). This (and being the first big cybercrime case that the Justice Dept. wanted to come out strong against) was a primary motivation in the overly severe (which has been admitted post-trial) sentance handed down. Combined with this, unprecidented media hype to Kevin being held without even a bail hearing pre-trial.

    How has the Justice Department's methods of assigning damages, investigation tecniques, and tech savvy (ie. telling the difference between hype and fact) improved since the Mitnick case? And would Mitnick have been tried as severely if he was being tried by today's Justice Department as opposed to the 'wild west' of the computing era?

  21. New campaign on Law Professor Examines SCO Case · · Score: 2, Funny

    We need a new campaign.

    How about "Hell no! I won't SCO!" :)

  22. Re:Well, will only make me stop shop on U.S. E-Commerce Sites To Collect EU VAT · · Score: 1

    Ever heard of 'taxation without representation'?

    Something to remember, the consumer never actually pays the tax, the vendor does. They increase the price, and show you how much tax is involved in the price so that both you don't think they're too expensive, and to explain differences between the price advertised and the price charged, however the vendor is the one paying the tax (which is required by law).

    Therefore, the same issues that apply for inter-state taxes, apply to inter-country taxes, but even worse. Taxes exist as a governments way of saying "Because you have a right to be involved in how government is run, by choosing who shall govern you in elections, we reserve the right to tax you to pay for your decisions." Thats fair. However, when purchasing something from a foreign state or country, you don't have that representation (at the state level, it depends if its a federal or state tax), so what is the basis for this right to tax you?

    Before you go off about competition, I understand the whole competition argument, thats what import duties are all about. They add to the cost of an item to stop people from doing exactly whats mentioned above - buying overseas where its cheaper because of no taxation (because of no representation).

    Apart from this, this whole thing sounds like a logistics nightmare. Its already a nightmare with US companies paying inter-state taxes, organising how the money is sent, etc. Internationally, this would be a disaster. Especially since, how the heck are they going to enforce it? They can't just send federal police around to arrest people for tax evasion, because they have no authority to do so. They could try and convince the US government to do so, but the US government would probably tell them to piss off, and be in their rights to do so.

    Not only this, it starts an arms race. Now, because the EU is forcing US companies to pay VAT for EU citizens (remember, the company pays the VAT, not the citizen (its a tax on the sale, not on the citizen)), now the US will demand EU companies start paying US sales tax, which if you recall, is done STATE BY STATE (not even for the entire country, it'd be like paying provincial tax foe every province in a particular EU country).

    Where does it end? Why can't they just enforce the measure already put in place to handle this exact problem, import duties.

  23. Re:And now, a translation... on Gentoo Reviewed · · Score: 1

    "Let's face it, Gentoo is the future."
    "OK, so no serious business is going to even consider Gentoo in the near future, and even with proper support and QA in place, it'll still eat up far too much of a company's valuable time. But this guy I met on #animepr0n is now using it, so it must be growing!"


    Thats very amusing. Where I work, we now have 12 Gentoo Linux systems (excluding desktops) and will be converting the other 13 linux machines to Gentoo in the near future.

    Gentoo has one huge advantage for business in this regard. Although it takes longer to do the initial setup, maintanence is MUCH easier. Remember, you don't need to compile everything on every machine, you CAN compile it once (make a binary package in the process), and then just distro the binary package you made (which is as simple as supplying a '-b' argument to the emerge command) to all the other systems, and emerge it onto them too. I don't think a sysadmin in the world could fuck that up.

    The biggest advantage though, is that we don't need the next version of bsd or (insert binary distro here). We don't need to wait for them to release updates for various software packages, we don't have dependancy hell when we want to upgrade any individual part of the system, etc.

    Lets review that procedure:
    emerge -b mypackage
    for x in $all_my_machines; do
    scp /usr/portage/packages/All/mypackage.tbz2 $x:/usr/portage/packages
    done
    Then on each machine:
    emerge mypackage

    (assuming you have set PKGDIR to /usr/porage/packages on each of these machines).

    Sound kind of like .RPM or .DEB? Well, it is kindof, except the package (no matter WHICH package it is, even glibc) is optimized for your hardware, you don't need to download it to each machine, and you don't have to go through dependancy hell.

  24. Re:Only on older planes on Wireless Computing and Airplanes? · · Score: 1

    I wouldnt call all planes since they introduced 'fly-by-wire' systems 'new'. Old and new are relative, when I said old, I ment post-hydrolic, pre-properly-shielded-wiring planes. Are you trying to tell me that every plane built since fly-by-wire control systems were installed are properly shielded from cell phone effects? I don't think so. Infact, its only planes made in the last few years that have wiring that is shielded against cell phone inteferance.

    And the pilot sounded more pissed off than joking. Nobody flubbs a landing that bad, I'm talking sudden jerks in a direction we werent supposed to go. No, the story is not immagination.

  25. Re:Only on older planes on Wireless Computing and Airplanes? · · Score: 1

    I was landing in Italy, actually.