Actually with DSL there's more overhead for small packets. Typically, for DSL the providers use PPPoE with a LLC/SNAP header. LLC/SNAP adds a 10 byte header (rfc1483), Ethernet adds a 14 byte header, PPPoE adds a 6 byte header, and PPP adds a 2 byte header, adding an extra 32 bytes per packet.
In addition, DSL runs over ATM. ATM chops up packets into cells. Each cell has a 5 byte header and a 48 byte payload, except for the last cell, which can only have a 40 byte payload due to an 8 byte trailer.
Needless to say, there's a fair amount of overhead for small packets with DSL. I know this because I write code to forward and terminate DSL PPPoE sessions.
Cisco's new software should indeed be much more secure, being built on top of QNX rather than their home-grown kernel. This will significantly improve their memory protection and make the system much more robust.
Good luck. Where I work we legally have access to Cisco IOS, although we're very strict and only a handful of engineers have the permissions to access it (me being one of them). The code is very clean and when I've browsed it looking to see if there's any exploits, I have thus far come up empty. The code does not look like the Microsoft code I've seen, which tends to be overly complex IMO. That's not to say we don't find bugs in Cisco's code, but generally it's very high quality.
Less code does not equal faster code. You can usually get the best performance increases by using a better algorithm. For example, if you're doing a lot of random adding and deleting of entries, a hash table will be much faster than a linked list. This will have the greatest impact.
Other things that can help are doing your own memory management at times (i.e. freelists) since that will be faster than malloc/new, and will have less memory overhead. Also, design your storage to your data. If you know you'll allocate up to 64K of an item, and the item is small, allocate an array of 64K of them and maintain a freelist. This will use a lot less memory than dynamically allocating each item and will result in better locality.
I write code in the embedded space, where memory usage and performance are both equally important. Usually getting the last ounce of performance out of the compiler doesn't make much difference.
A good real-world example is that I replaced the malloc code provided by a popular embedded OS with DLMalloc, which glibc is based. The dlmalloc code is *much* more complicated, and the code path is much longer, but due to much better algorithms, operations that took an hour with the old simple malloc dropped down to 3 minutes. It went from exponential to linear time.
Also as long as the state doesn't have to pay for hospital stays, rehabilitation, or long term care for those who injure themselves. Even Jessie Ventura discovered that the seatbelt law in his state was good because it saved the state a lot of money, just like motorcycle helmet laws. The law's not just protecting others, but protecting the taxpayers from having to deal with people's stupidity.
I guess the alternative would be that if a person wasn't protecting themselves and hurt themselves, then the state would not in any way pay for a person's recovery and insurance companies wouldn't be required either. Then a person who hurts themself would either die (since a hospital wouldn't be required to treat them in the emergency room), fend for themselves, or be a burdon on their family and friends.
Video is NOT RGB, so 1920x1080i is only the luminance portion. HD is broadcast in 422 format using YUV instead of RGB. This means that the intensity information is at twice the resolution as the color information.
At least here in the US, here's the back of an envelope calculation:
Typically, with 8 bits, 2 pixels are represented by 4 bytes:
YUYV
This is more like 16 bits per pixel.
Y is the luminance (brightness), U and V represent color and saturation.
1920x1080i*30*16 = 995,328,000 bits/second or 124,416,000 bytes/second uncompressed.
Also, it's interlaced, a frame is 2 fields. 60 fields/second = 30 frames/second.
The color resolution for HD is 960x1080.
I think a more common format will be 720p, while lower resolution, it is progressive instead of interlaced and will tend to look better, although with current technology 1080i can be deinterlaced fairly well, especially if the source is film (which is only 24 frames/second so they use 3:2 pulldown, which means they show 3 fields of one film frame, and 2 fields of the other. Personally I wish they used a higher frame rate for film.
Other video formats are also available. What I described above is referred to as 4:2:2. There are other formats which further reduce the color information, both vertically and horizontally.
There may not be a lot of people in wheelchairs, but I'm sure a high percentage of elderly people appreciate the ramps on places like street corners, as do cyclists. Go watch an elderly person try and get around and maybe you'll understand. I'm also sure that when you get to be that age and arthritic you'll appreciate it as well.
Using computation won't work
on
Gates on Spam
·
· Score: 4, Insightful
The problem with requiring computation cycles is that you need to deal with a lot of older computers. I have friends with old Pentium-based computers, some of whom cannot afford a nice new P4 system.
Also, what happens to all these web-based email accounts like Yahoo or Microsoft's Hotmail? I guess they'll need to spend a lot of money adding processing power for their users to send email.
What's to stop someone from making hardware to do the processing? It shouldn't be too difficult to implement an FPGA or an ASIC that could do the processing much faster. I imagine it wouldn't take too long for PCI boards to come out to offload the processing for large mail servers, then spammers with money could just buy the board to offload the processing.
-Aaron
Mailing lists
on
Gates on Spam
·
· Score: 5, Interesting
I can't see how this could work. Any spam-prevention measure must also have some provision to deal with legitimate mailing lists. Some mailing lists can be quite busy and have thousands of members.
Also, Gate's method has a lot of flaws, security being only one of them. For example, how will you deal with all the various different operating systems and embedded hardware that send email? For example, my Netgear firewall box periodically sends me emails of logs or alerts.
Also, you can't easily change the way email is done because its use is so widespread.
Making it computationally based has a number of major flaws.
1. How do you deal with the wide range of computer performance? For example, my mail server is a Pentium II, which is more than adequate for my needs, or my firewall, which is a 50MHz StrongARM processor?
2. How would you allow others to use your computer to make computations? This opens up some serious security considerations, not to mention the fact that there's a wide range of processors and operating systems that would need to be supported. I won't run Seti@Home because the last time I ran it it crashed my mail server after over 200 days of uptime. I don't know what it was about Seti, but it would always immediately crash my server.
3. You would need to make everyone agree to do this. The Internet is international.
A better way would be to strongly encourage ISPs to block spammers and give them the tools to go after them. An ISP should be able to charge the hell out of a spammer on their network and encouraged to do so.
Why not give the backbones the power to cut off major spam sources and provide financial incentives to do so?
There's lots of other methods that could be used. If you make life completely miserable for spammers, they'll stop. If there's no profit, they'll stop.
If our stupid congress critters would do something right for a change, like California's anti-spam law that was blocked by the Washington idiots, then we'd have a lot more power to go after the spammers.
I'm a former attbi and now comcast user. They transitioned me all right, but I'm pissed off I lost my main email address to some godawful long address. Because of that, I still use my attbi address and when that's gone I'll switch to my own domain name, which I'm already doing to some extent. I'm sick of one ISP buying out another and losing my email address in the process. First @Home->Attbi, then Attbi->Comcast.
I'm pissed off because I have an attbi account (now Comcast) which I still use. It seems that I must have pissed off a spammer (easy, since I use Spamcop to report them) and they're forging the from address as my address for some of their spams and I get the bounces:-/ Worse, someone keeps spewing out viruses forging the from address as mine and the antivirus software keeps sending me nastygrams. This is not possible since I don't run Windows.
I've done this with a cheap UPS I bought for $5. The UPS was of very low capacity with a couple of dead batteries. There was plenty of room inside the unit for larger batteries, so I added a couple of significantly larger capacity SLA batteries. So far it's worked great. Of course I don't run much load off of the UPS. It's basically just powering my ReplayTV and cable box.
I think generally it's probably OK to put in larger capacity batteries as long as you keep the load on the UPS well below what it's designed for. Otherwise I'd worry about burning out the inverter since they're not designed for running for long periods of time.
Given that it's GPL, there's nothing stopping you or anyone else from porting the X QT to Windows. There are many other GNU Unix apps that haven't been ported either. If by free you expect Trolltech to release their non-free Windows version under GPL, it's their product. They can do whatever they want with it.
I can definitely say that Can-Spam is not working. I saw a definite increase in the amount of spam I receive per day. My employer is also trying to deal with it, but so far without a lot of success. With around 500 employees, we're receiving on the order of 300,000 emails per day, most of which are spam.
I like the idea of a bounty on spammers. They should legalize bounty hunting of spammers, where a licensed bounty hunter sues a spammer at the request of the spam recipient. Bounty hunters could compete by offering different percentages of their winnings. If we did this, I imagine spam would dry up in this country real fast.
Besides going after the spammer, they should also be able to sue the company paying the spammer to spam. Of course if it's a joe-job then the spammer should get fined triple damages, with the extra damages going to the victim.
I'm getting hit right now. Some spammer is using one of my email addresses as his from address to mass-spam a virus. I've received literally hundreds of bounced emails, unfortunately most of which do not have enough information to track down the perpetrator. I know I've probably pissed off a lot of spammers because I typically report any spam I receive immediately through Spamcop. It's a bit of work, but anything to cost a spammer money and hurt a spammer is a good thing.
How about we convince the department of Homeland Security that all the random words at the end of many new spam messages are actually code used by Al Quada and other terrorist organizations? (not far from the truth if you consider spammers terrorist).
Although currently ethanol production is expensive and produces little, if any, additional energy than what is required to create it, the reactor could change this.
Think what would happen if all the farm equipment were powered by a fuel-cell and ethanol instead of diesel. This would significantly increase efficiency in the production.
However, the cost of the production could be quite high in the long run. I imagine a lot of water, fertilizer and pesticide is needed to grow the corn, at least with current large-scale farming techniques.
I think this can only become feasible if current farming practices change significantly.
1. Use less, or better yet, no pesticide. 2. Replace all the farm equipment and transportation equipment with fuel-cell powered equipment to improve efficiency. 3. Reduce the amount of water required for growing the corn. 4. Practice farming techniques that improve the soil over time.
I also doubt that we could grow enough corn to provide enough ethanol if everything ran on it. Also, a drought or other major disaster would also have a much greater impact. Perhaps this could be combined with another technique I read about recently where someone found a method of converting most organic waste into high-quality petroleum by combining it with water under high temperature and pressure. It was estimated that if we did this we could supply half of the oil requirement of the US and completely eliminate our dependency on the middle east for oil.
In my day we didn't even have spam, and we liked it! You kids and your fancy smancy spam and filters and whatnot have no idea of the difficulties before spam. Hell, if we wanted to find out about penis enlargement pills we had to go out hiking through the snow uphill to search for them. And we liked it!
What about mailing lists? I'm on numerous mailing lists, some of which have thousands of users. How do you combat spam and not stop legitimate mass mailings?
Also, as far as charging people for sending email, what about all the hijacked machines out there? Granted, it would certainly give an incentive for people to patch their machines. Finally, all ISPs would have to approve it. If one ISP, to attract customers, says unlimited free email, they might get a lot of people who legitimately send a lot of email as well as spammers. What about foreign countries? How would you get countries like China, Korea, or Russia to do this?
I think the only technical solutions are white lists and some form of user authentication. However, authentication can also cause problems, especially if an ISP blocks outbound port 25 connections. For example, I have several domain names and I use various email addresses in order to block spam. I have one email address where I never receive spam because I only use it for personal communications or emails to people I know for sure will not spam me. I never use it for mailing lists since spammers frequently spider web sites that archive messages. I also never publish it on my web site.
However, no matter what email address or domain I use for sending mail, I have my mail server forward everything through my ISP's mail server.
If my ISP decided to only allow the email address they assigned me and they blocked outbound port 25 then I could not send any email with a different from and/or reply-to address. I also send email with my work address when working from home.
I think if something like California's spam law went into effect nationwide, it could do a lot to stop spammers. Even though most spam appears to come from China or other foreign countries, most of the companies sending the spam are right here in the US.
Looking at some other VxWorks development sites it is apparent that the file system code is not the most robust. For those of you that don't know embedded systems, it is actually quite common to not have any file system at all. Wind River in fact has strong cautions about using FAT because it's not robust and there's all sorts of reports about DOS having problems and having to perform a chkdsk. VxWorks does not have any tools like chkdsk built-in either. About all you can do when there's filesystem corruption is to reformat.
Also, running out of memory in VxWorks is a critical problem. Think of it as a single program with a fixed total amount of memory. "system" calls are basically not much more than local function calls. If you run out of memory, it is highly likely that a function call will die.
The flash file system, from what I can see, appears to be rather limited and quite primitive compared to other Flash file systems like JFFS and JFFS2. The default configuration is limited to a maximum size of 40MB, but can be expanded up to 128MB, I believe you can also have only a limited number of files.
Embedded programming is rather different than writing for a desktop OS. Your application *must not crash* for any reason, must not have any memory leaks, and must be able to handle all boundary conditions. Also, in VxWorks, a task is like a thread. All memory is globally visible. Any function can call any other function.
You tend to do things a bit differently. You may have a lot more error checking code in the released product when stability is required, and a lot more code to recover if there's a problem. You also tend to write a lot of code for tracing events and debugging and for something like the rovers you'd leave that code in to debug problems remotely.
Filesystems are not one of VxWorks strong points, but then again, many embedded applications don't need much, if any, filesystem. There is no root filesystem either.
Think of VxWorks as the Linux kernel and all tasks as linked-in modules. It can load additional modules similar to how Linux loads loadable modules. In fact, like Linux, they're basically just object files.
Take a look at QNX. I'm not very familiar with many other RTOSes. RTEMS is another interesting one I looked at (is open source). If you do decide to go with VxWorks, make sure you get the source code. A word of warning, though, VxWorks TCP/IP support isn't all that great and performance is mediocre. It's based on an ancient BSD stack.
QNX has a lot of reliability features built in that are not present in VxWorks, Linux, or RTEMS. It can take periodic snapshots of a task and recover if a task dies.
Also, field debugging of VxWorks can be a real pain. It does not generate a core file you can debug later.
Timesys has very good real time support, but the context switching time is on the order of 10us depending on the processor, QNX is better.
QNX is more more like a unix environment with real threads and processes. QNX relies mostly on effecient message passing between tasks. QNX also has much better TCP/IP support than VxWorks, including support for IPv6. From what I can tell, development and debugging in QNX is much easier than VxWorks. QNX does not have as many priority levels as VxWorks. QNX Neutrino supports 64 priorities. VxWorks supports 256. Timesys Linux can support thousands of priorities.
QNX also has built-in support for distributed processing as long as there is an interconnect. It also supports SMP (VxWorks does not).
VxWorks, on the other hand, is like a single task with many threads. A "task" in VxWorks is more like a thread in that all memory is globally accessable. Any function can call any other function.
I havn't worked with QNX or Timesys personally. I've spent the last 4 years developing for VxWorks Tornado II. The group where I work dealing with OS related issues has more or less decided to use either QNX or Timesys for our next generation software. Timesys looks good from the tools perspective and the wide Linux experience out there, plus the fact that currently our product runs on a mixture of Unix and VxWorks, plus many of the drivers we need are available for Linux and would need to be ported to QNX. Some of the key software we are using has been ported to QNX, though, and the reliability of QNX is also of interest. QNX has also won some nice design wins like Cisco. Since we make routing equipment we tend to pay attention to things like that.
Also, beware that Wind River's support isn't very good from our experience. It basically boils down to "what support?" even with a support contract. Wind River has layed off a lot of people lately and I'm sure that hasn't helped the company. They've been bleeding money like crazy and are hurting pretty bad. They've been losing market share to other embedded RTOSes and Linux. Hell, Wind River has ported their development tools to support embedded Linux (I wonder why?).
As far as real-time Linux is concerned, I looked at several of them. RT-Linux requires that RT tasks talk to the RT kernel and use a separate API and drivers than the Linux tasks. It's basically Linux running on top of a RT kernel. Monta Vista does not provide hard real-time, and when I last spoke with them they had no solution for priority inversion. Timesys modified the stock Linux kernel so RT tasks don't need to use a separate set of APIs from non-RT tasks and Timesys can guarantee timing, both for the CPU and networking. For soft real-time, Monta Vista seems to be pretty popular. The people I spoke with who used it said it was quite reliable but couldn't comment on support since they never needed any.
While I would agree with you that avoiding malloc and preallocating memory is the way to go, but it is not always possible. In my case, we are using various 3rd party libraries, and changing them to use static memory allocation would be prohibitave. In at least one case, the third party library source code was not available. Also, in many cases the dynamic nature of some algorithms requires dynamic memory management. You cannot statically allocate everything, especially in a limited memory environment.
I know we're not the only ones to have been burned by Wind River's malloc. I know several major companies that also had to replace Wind River's code.
As far as being able to dynamically replace code, VxWorks isn't alone in that. Numerous other RTOSes out there can do the same thing, including QNX. QNX even supports the concept of a hot standby process to take over if the main process dies.
To give you an idea about how Wind River's malloc works, they keep a sorted linked list of fragments from the smallest to the largest. When you try and allocate a block, it walks the linked list until it finds a block large enough. Likewise, when you free a block it checks if it can coalesc the block with a neighboring block. It then goes through the linked list looking for a slot to insert the free block.
Yes, VxWorks may have been around since the 80's, but that's part of the problem too and it is showing its age. In the 80s embedded processors typically did not have MMUs. Now MMUs are quite common in the more powerful embedded processors.
You say you can't have low latency and memory protection? QNX proves that you can. It is low latency and *very* robust. If your driver dies, no problem, restart it. Timesys Linux also has a very low latency, although not as low as QNX. Timesys also has an interesting feature where you can guarantee CPU and networking resources. I can schedule a task to be guaranteed 5.8ms of execution every 8.3ms and it will guarantee that that task will get the CPU time allotted to it with the desired resolution. This is without increasing the system tick rate (usually 10ms). Timesys can also schedule a task to be higher priority than an interrupt. I'm not as familiar with QNXs scheduler, but it's also quite flexible from what I've heard.
As far as FAT, it is not a robust filesystem. It never has been. If the FAT gets corrupted or a directory entry gets corrupted it's difficult to recover. Other than possibly having 2 copies of the FAT cluster table, any corruption can be difficult to repair. If the FAT table gets corrupted, which table is corrupt and which is not? If a directory entry gets corrupted, it can be impossible to fix. For flash memory, unless you are using a device with special wear-leveling, FAT is about the worst choice since any file write that changes the size of a file requires a write to the directory entry and possibly the FAT table. If the table gets corrupted and you don't run a repair operation (which often ends up leaving orphaned files as lost clusters), the file system can happily corrupt itself to death.
Why do you think every time DOS/Windows9x/ME crashed it had to repair the disk with scandisk? FAT is a poorly designed file system that was originally designed for 160K floppies and scales poorly. FAT32 is an improvement, but it's still not very robust. For flash, something like Linux's journalling flash file system 2 (JFFS2). More information on VxWorks file system support can be found here.
If anyone saw my earlier posts on VxWorks they would see I am not at all surprised about the problems NASA is having.
As someone with first-hand experience with VxWorks let me say that VxWorks' memory handling code sucks. Their malloc implementation has got to be the worst one ever designed. It fragments horribly and when fragmented has unusable performance. A malloc call can take many milliseconds when memory gets fragmented. Our box used to crash due to the fragmentation. I replaced Wind River's code with DLMalloc and that fixed the memory issue. We went from many tens of thousands of fragments to only a few dozen. Our reliability significantly increased as well after dumping Wind Rivers brain-dead malloc code. BTW, glibc uses a variant of Doug Lea's malloc code, so it's been widely tested.
Furthermore, in VxWorks there is no way to identify what process malloced a block of memory. There is no memory protection either (think DOS). If a task has a memory leak, VxWorks does not provide any method of tracking down the culprit. I had to add that support to the DL Malloc code I ported so we could find memory leaks and general memory usage by task.
Since malloc is critical to the working of VxWorks, if you run out of memory you are basically completely dead. Often the only way to recover is to wait for the hardware watchdog timer to kick in and reboot the system.
If a task dies in VxWorks with a bad pointer there is no way to recover other than reboot. The OS will not clean up after a task (i.e. free memory, close files, release semaphores).
As far as flash goes, VxWorks supports the FAT16 file system. As you know, FAT16 sucks. It only supports a limited number of files in the root directory. It's relatively easy to corrupt, and when corrupt it tends to corrupt itself even worse. There is no wear leveling support. If the FAT table or root directory gets corrupted, you're screwed. VxWorks is even worse since there arn't tools to fix a corrupt file system.
VxWorks is not a scalable OS. The OS gets slower as the number of tasks increases. Realtime support sucks. Although it has support for things like priority inheritance to prevent priority inversion, the best guaranteed realtime latency is half the system tick rate (the tick rate is usually 10ms).
Also remember that unlike open source operating systems, the source code to VxWorks is not available unless you pay some major $$$. Without the source you're basically working blind.
VxWorks is an old RTOS, and its age is definitely showing. It is not a robust OS.
As far as turning VxWorks into a firewall, you'll need to write all your own code. The VxWorks TCP/IP stack is an archaic vulnerable version of the BSD stack. TCP sequence number guessing is trivial. There is no built-in support for firewall support, NAT, or anything else. I have heard many many complaints about the VxWorks networking code. Although the box I'm working on is a router and broadband remote access server, we don't use the VxWorks TCP/Ip stack much. I am sure the VxWorks stack is vulnerable to many of the current DOS attacks as well.
Some of you might say why not use VxWorks AE, which adds memory protection. Think hacked-in memory protection far worse than Windows 95. It's slow, very buggy, and poorly supported. We tried AE and after many months we dumped it and went back to the non-AE version. Very few companies actually went with AE.
Today there are many other choices for an RTOS that are better than VxWorks. For our next major project we're looking at both QNX and TimeSys Linux. QNX is a true microkernel design with the core being around 70K. Every driver is protected and can be restarted if it crashes. I think you can even buy a medical grade version of QNX. TimeSys Linux is also pretty cool, with excellent real-time support and all the advantages of Linux. For something like the Mars rover, QNX would be better due to the limited amount of memory and greater robustness.
I can tell you that AE is in many ways WORSE than the standard VxWorks. It has a lot more bugs and is quite a bit slower. Think of regular VxWorks with memory protection hacked in, not designed in from the ground up.
As a VxWorks programmer for the last 5 years, I can honestly say VxWorks is a PoS that is losing market share at a tremendous rate to the likes of embedded Linux and QNX. Wind River decided to spend tons of money buying add-in products like Routerware instead of improving their RTOS. It was a huge waste of money and now they're paying for it. They're losing money hand over fist and have had a lot of layoffs lately. They were good at one time, but they have fallen far behind the curve now in embedded RTOS design, especially for complex systems.
VxWorks comes with support for a FAT flash file system, a completely broken malloc implementation, an ancient BSD TCP/IP stack, poor RT support, no memory protection, and no way to clean up after a task that dies. Not only that, it usually costs a fortune, but I've heard they're willing to sell it very cheap now because they're desparate.
I looked into embedded Linux for our next generation hardware and software and Timesys appears to have a very nice solution with hard real-time support. The kernel is fully preemptable using semaphores instead of spinlocks and has priority inversion support. They also offer resource reservations, so I can say "I want this task to be guaranteed 5.73ms of execution time every 9.8ms" where after 5.73ms the task either gives up the CPU entirely, or else changes to a non-RT priority to not starve other tasks. It's really quite clever. Not only that, unlike RT-Linux there isn't a separate API for RT vs non-RT tasks. Monta Vista Linux is soft real-time. It cannot guarantee context switching time, nor does it deal with priority inversion. In RT, priority inversion can be a major problem (see the first Mars rover for an example).
For an example of priority inversion say you have 3 tasks, a low priority, medium priority, and a high priority. The low priority task acquires a mutex semaphore to protect a critical section and starts processing. It is interrupted by a medium priority task. Meanwhile, a high-priority task unblocks and attempts to grab the mutex. The high priority task will block until the medium priority task blocks so that the low priority task can release the semaphore. A common solution is priority inheritance. With priority inheritance, as soon as the high priority task attempts to acquire the mutex semaphore, the low priority task has its priority bumped to that of the high priority task until it releases the semaphore. In this way, the low priority task will interrupt the medium priority task so that the high priority task won't have to wait as long.
QNX is also a very good alternative. Very fast context switching and extremely robust memory protection. I think with QNX you can even buy a license suitable for use in medical devices (i.e. you absolutely cannot afford to have the OS crash for any reason).
I've heard rumours that Wind River is dropping AE since nobody is using it. After our experience, I pity whoever tries it.
Also, unless you get the source to VxWorks, which usually costs a lot of $$$, debugging is a complete nightmare, especially when you hit a bug in the Wind River code (and there's a lot of them). Hell, they couldn't even implement malloc right!
Wind River is coming out with version 6 of VxWorks, but it is basically an enhanced version of AE. I'm not holding my breath.
Slashdot Mirror
Actually with DSL there's more overhead for small packets. Typically, for DSL the providers use PPPoE with a LLC/SNAP header. LLC/SNAP adds a 10 byte header (rfc1483), Ethernet adds a 14 byte header, PPPoE adds a 6 byte header, and PPP adds a 2 byte header, adding an extra 32 bytes per packet.
In addition, DSL runs over ATM. ATM chops up packets into cells. Each cell has a 5 byte header and a 48 byte payload, except for the last cell, which can only have a 40 byte payload due to an 8 byte trailer.
Needless to say, there's a fair amount of overhead for small packets with DSL. I know this because I write code to forward and terminate DSL PPPoE sessions.
Cisco's new software should indeed be much more secure, being built on top of QNX rather than their home-grown kernel. This will significantly improve their memory protection and make the system much more robust.
Good luck. Where I work we legally have access to Cisco IOS, although we're very strict and only a handful of engineers have the permissions to access it (me being one of them). The code is very clean and when I've browsed it looking to see if there's any exploits, I have thus far come up empty. The code does not look like the Microsoft code I've seen, which tends to be overly complex IMO. That's not to say we don't find bugs in Cisco's code, but generally it's very high quality.
Less code does not equal faster code. You can usually get the best performance increases by using a better algorithm. For example, if you're doing a lot of random adding and deleting of entries, a hash table will be much faster than a linked list. This will have the greatest impact.
Other things that can help are doing your own memory management at times (i.e. freelists) since that will be faster than malloc/new, and will have less memory overhead. Also, design your storage to your data. If you know you'll allocate up to 64K of an item, and the item is small, allocate an array of 64K of them and maintain a freelist. This will use a lot less memory than dynamically allocating each item and will result in better locality.
I write code in the embedded space, where memory usage and performance are both equally important. Usually getting the last ounce of performance out of the compiler doesn't make much difference.
A good real-world example is that I replaced the malloc code provided by a popular embedded OS with DLMalloc, which glibc is based. The dlmalloc code is *much* more complicated, and the code path is much longer, but due to much better algorithms, operations that took an hour with the old simple malloc dropped down to 3 minutes. It went from exponential to linear time.
-Aaron
Also as long as the state doesn't have to pay for hospital stays, rehabilitation, or long term care for those who injure themselves. Even Jessie Ventura discovered that the seatbelt law in his state was good because it saved the state a lot of money, just like motorcycle helmet laws. The law's not just protecting others, but protecting the taxpayers from having to deal with people's stupidity.
I guess the alternative would be that if a person wasn't protecting themselves and hurt themselves, then the state would not in any way pay for a person's recovery and insurance companies wouldn't be required either. Then a person who hurts themself would either die (since a hospital wouldn't be required to treat them in the emergency room), fend for themselves, or be a burdon on their family and friends.
Video is NOT RGB, so 1920x1080i is only the luminance portion. HD is broadcast in 422 format using YUV instead of RGB. This means that the intensity information is at twice the resolution as the color information.
At least here in the US, here's the back of an envelope calculation:
Typically, with 8 bits, 2 pixels are represented by 4 bytes:
YUYV
This is more like 16 bits per pixel.
Y is the luminance (brightness), U and V represent color and saturation.
1920x1080i*30*16 = 995,328,000 bits/second or 124,416,000 bytes/second uncompressed.
Also, it's interlaced, a frame is 2 fields. 60 fields/second = 30 frames/second.
The color resolution for HD is 960x1080.
I think a more common format will be 720p, while lower resolution, it is progressive instead of interlaced and will tend to look better, although with current technology 1080i can be deinterlaced fairly well, especially if the source is film (which is only 24 frames/second so they use 3:2 pulldown, which means they show 3 fields of one film frame, and 2 fields of the other. Personally I wish they used a higher frame rate for film.
Other video formats are also available. What I described above is referred to as 4:2:2. There are other formats which further reduce the color information, both vertically and horizontally.
Framemaker can do these types of things, sadly Adobe only released a beta version for Linux and pulled it.
There may not be a lot of people in wheelchairs, but I'm sure a high percentage of elderly people appreciate the ramps on places like street corners, as do cyclists. Go watch an elderly person try and get around and maybe you'll understand. I'm also sure that when you get to be that age and arthritic you'll appreciate it as well.
The problem with requiring computation cycles is that you need to deal with a lot of older computers. I have friends with old Pentium-based computers, some of whom cannot afford a nice new P4 system.
Also, what happens to all these web-based email accounts like Yahoo or Microsoft's Hotmail? I guess they'll need to spend a lot of money adding processing power for their users to send email.
What's to stop someone from making hardware to do the processing? It shouldn't be too difficult to implement an FPGA or an ASIC that could do the processing much faster. I imagine it wouldn't take too long for PCI boards to come out to offload the processing for large mail servers, then spammers with money could just buy the board to offload the processing.
-Aaron
I can't see how this could work. Any spam-prevention measure must also have some provision to deal with legitimate mailing lists. Some mailing lists can be quite busy and have thousands of members.
Also, Gate's method has a lot of flaws, security being only one of them. For example, how will you deal with all the various different operating systems and embedded hardware that send email? For example, my Netgear firewall box periodically sends me emails of logs or alerts.
Also, you can't easily change the way email is done because its use is so widespread.
Making it computationally based has a number of major flaws.
1. How do you deal with the wide range of computer performance? For example, my mail server is a Pentium II, which is more than adequate for my needs, or my firewall, which is a 50MHz StrongARM processor?
2. How would you allow others to use your computer to make computations? This opens up some serious security considerations, not to mention the fact that there's a wide range of processors and operating systems that would need to be supported. I won't run Seti@Home because the last time I ran it it crashed my mail server after over 200 days of uptime. I don't know what it was about Seti, but it would always immediately crash my server.
3. You would need to make everyone agree to do this. The Internet is international.
A better way would be to strongly encourage ISPs to block spammers and give them the tools to go after them. An ISP should be able to charge the hell out of a spammer on their network and encouraged to do so.
Why not give the backbones the power to cut off major spam sources and provide financial incentives to do so?
There's lots of other methods that could be used. If you make life completely miserable for spammers, they'll stop. If there's no profit, they'll stop.
If our stupid congress critters would do something right for a change, like California's anti-spam law that was blocked by the Washington idiots, then we'd have a lot more power to go after the spammers.
I'm a former attbi and now comcast user. They transitioned me all right, but I'm pissed off I lost my main email address to some godawful long address. Because of that, I still use my attbi address and when that's gone I'll switch to my own domain name, which I'm already doing to some extent. I'm sick of one ISP buying out another and losing my email address in the process. First @Home->Attbi, then Attbi->Comcast.
-Aaron
I'm pissed off because I have an attbi account (now Comcast) which I still use. It seems that I must have pissed off a spammer (easy, since I use Spamcop to report them) and they're forging the from address as my address for some of their spams and I get the bounces :-/ Worse, someone keeps spewing out viruses forging the from address as mine and the antivirus software keeps sending me nastygrams. This is not possible since I don't run Windows.
I've done this with a cheap UPS I bought for $5. The UPS was of very low capacity with a couple of dead batteries. There was plenty of room inside the unit for larger batteries, so I added a couple of significantly larger capacity SLA batteries. So far it's worked great. Of course I don't run much load off of the UPS. It's basically just powering my ReplayTV and cable box.
I think generally it's probably OK to put in larger capacity batteries as long as you keep the load on the UPS well below what it's designed for. Otherwise I'd worry about burning out the inverter since they're not designed for running for long periods of time.
-Aaron
Given that it's GPL, there's nothing stopping you or anyone else from porting the X QT to Windows. There are many other GNU Unix apps that haven't been ported either. If by free you expect Trolltech to release their non-free Windows version under GPL, it's their product. They can do whatever they want with it.
I can definitely say that Can-Spam is not working. I saw a definite increase in the amount of spam I receive per day. My employer is also trying to deal with it, but so far without a lot of success. With around 500 employees, we're receiving on the order of 300,000 emails per day, most of which are spam. I like the idea of a bounty on spammers. They should legalize bounty hunting of spammers, where a licensed bounty hunter sues a spammer at the request of the spam recipient. Bounty hunters could compete by offering different percentages of their winnings. If we did this, I imagine spam would dry up in this country real fast. Besides going after the spammer, they should also be able to sue the company paying the spammer to spam. Of course if it's a joe-job then the spammer should get fined triple damages, with the extra damages going to the victim. I'm getting hit right now. Some spammer is using one of my email addresses as his from address to mass-spam a virus. I've received literally hundreds of bounced emails, unfortunately most of which do not have enough information to track down the perpetrator. I know I've probably pissed off a lot of spammers because I typically report any spam I receive immediately through Spamcop. It's a bit of work, but anything to cost a spammer money and hurt a spammer is a good thing.
How about we convince the department of Homeland Security that all the random words at the end of many new spam messages are actually code used by Al Quada and other terrorist organizations? (not far from the truth if you consider spammers terrorist).
Although currently ethanol production is expensive and produces little, if any, additional energy than what is required to create it, the reactor could change this.
Think what would happen if all the farm equipment were powered by a fuel-cell and ethanol instead of diesel. This would significantly increase efficiency in the production.
However, the cost of the production could be quite high in the long run. I imagine a lot of water, fertilizer and pesticide is needed to grow the corn, at least with current large-scale farming techniques.
I think this can only become feasible if current farming practices change significantly.
1. Use less, or better yet, no pesticide.
2. Replace all the farm equipment and transportation equipment with fuel-cell powered equipment to improve efficiency.
3. Reduce the amount of water required for growing the corn.
4. Practice farming techniques that improve the soil over time.
I also doubt that we could grow enough corn to provide enough ethanol if everything ran on it. Also, a drought or other major disaster would also have a much greater impact. Perhaps this could be combined with another technique I read about recently where someone found a method of converting most organic waste into high-quality petroleum by combining it with water under high temperature and pressure. It was estimated that if we did this we could supply half of the oil requirement of the US and completely eliminate our dependency on the middle east for oil.
In my day we didn't even have spam, and we liked it! You kids and your fancy smancy spam and filters and whatnot have no idea of the difficulties before spam. Hell, if we wanted to find out about penis enlargement pills we had to go out hiking through the snow uphill to search for them. And we liked it!
What about mailing lists? I'm on numerous mailing lists, some of which have thousands of users. How do you combat spam and not stop legitimate mass mailings?
Also, as far as charging people for sending email, what about all the hijacked machines out there? Granted, it would certainly give an incentive for people to patch their machines. Finally, all ISPs would have to approve it. If one ISP, to attract customers, says unlimited free email, they might get a lot of people who legitimately send a lot of email as well as spammers. What about foreign countries? How would you get countries like China, Korea, or Russia to do this?
I think the only technical solutions are white lists and some form of user authentication. However, authentication can also cause problems, especially if an ISP blocks outbound port 25 connections. For example, I have several domain names and I use various email addresses in order to block spam. I have one email address where I never receive spam because I only use it for personal communications or emails to people I know for sure will not spam me. I never use it for mailing lists since spammers frequently spider web sites that archive messages. I also never publish it on my web site.
However, no matter what email address or domain I use for sending mail, I have my mail server forward everything through my ISP's mail server.
If my ISP decided to only allow the email address they assigned me and they blocked outbound port 25 then I could not send any email with a different from and/or reply-to address. I also send email with my work address when working from home.
I think if something like California's spam law went into effect nationwide, it could do a lot to stop spammers. Even though most spam appears to come from China or other foreign countries, most of the companies sending the spam are right here in the US.
-Aaron
Looking at some other VxWorks development sites it is apparent that the file system code is not the most robust. For those of you that don't know embedded systems, it is actually quite common to not have any file system at all. Wind River in fact has strong cautions about using FAT because it's not robust and there's all sorts of reports about DOS having problems and having to perform a chkdsk. VxWorks does not have any tools like chkdsk built-in either. About all you can do when there's filesystem corruption is to reformat.
Also, running out of memory in VxWorks is a critical problem. Think of it as a single program with a fixed total amount of memory. "system" calls are basically not much more than local function calls. If you run out of memory, it is highly likely that a function call will die.
The flash file system, from what I can see, appears to be rather limited and quite primitive compared to other Flash file systems like JFFS and JFFS2. The default configuration is limited to a maximum size of 40MB, but can be expanded up to 128MB, I believe you can also have only a limited number of files.
Embedded programming is rather different than writing for a desktop OS. Your application *must not crash* for any reason, must not have any memory leaks, and must be able to handle all boundary conditions.
Also, in VxWorks, a task is like a thread. All memory is globally visible. Any function can call any other function.
You tend to do things a bit differently. You may have a lot more error checking code in the released product when stability is required, and a lot more code to recover if there's a problem. You also tend to write a lot of code for tracing events and debugging and for something like the rovers you'd leave that code in to debug problems remotely.
Filesystems are not one of VxWorks strong points, but then again, many embedded applications don't need much, if any, filesystem. There is no root filesystem either.
Think of VxWorks as the Linux kernel and all tasks as linked-in modules. It can load additional modules similar to how Linux loads loadable modules. In fact, like Linux, they're basically just object files.
I also found a good FAQ here. -Aaron
Take a look at QNX. I'm not very familiar with many other RTOSes. RTEMS is another interesting one I looked at (is open source). If you do decide to go with VxWorks, make sure you get the source code. A word of warning, though, VxWorks TCP/IP support isn't all that great and performance is mediocre. It's based on an ancient BSD stack.
QNX has a lot of reliability features built in that are not present in VxWorks, Linux, or RTEMS. It can take periodic snapshots of a task and recover if a task dies.
Also, field debugging of VxWorks can be a real pain. It does not generate a core file you can debug later.
Timesys has very good real time support, but the context switching time is on the order of 10us depending on the processor, QNX is better.
QNX is more more like a unix environment with real threads and processes. QNX relies mostly on effecient message passing between tasks. QNX also has much better TCP/IP support than VxWorks, including support for IPv6. From what I can tell, development and debugging in QNX is much easier than VxWorks. QNX does not have as many priority levels as VxWorks. QNX Neutrino supports 64 priorities. VxWorks supports 256. Timesys Linux can support thousands of priorities.
QNX also has built-in support for distributed processing as long as there is an interconnect. It also supports SMP (VxWorks does not).
VxWorks, on the other hand, is like a single task with many threads. A "task" in VxWorks is more like a thread in that all memory is globally accessable. Any function can call any other function.
I havn't worked with QNX or Timesys personally. I've spent the last 4 years developing for VxWorks Tornado II. The group where I work dealing with OS related issues has more or less decided to use either QNX or Timesys for our next generation software. Timesys looks good from the tools perspective and the wide Linux experience out there, plus the fact that currently our product runs on a mixture of Unix and VxWorks, plus many of the drivers we need are available for Linux and would need to be ported to QNX. Some of the key software we are using has been ported to QNX, though, and the reliability of QNX is also of interest. QNX has also won some nice design wins like Cisco. Since we make routing equipment we tend to pay attention to things like that.
Also, beware that Wind River's support isn't very good from our experience. It basically boils down to "what support?" even with a support contract. Wind River has layed off a lot of people lately and I'm sure that hasn't helped the company. They've been bleeding money like crazy and are hurting pretty bad. They've been losing market share to other embedded RTOSes and Linux. Hell, Wind River has ported their development tools to support embedded Linux (I wonder why?).
As far as real-time Linux is concerned, I looked at several of them. RT-Linux requires that RT tasks talk to the RT kernel and use a separate API and drivers than the Linux tasks. It's basically Linux running on top of a RT kernel. Monta Vista does not provide hard real-time, and when I last spoke with them they had no solution for priority inversion. Timesys modified the stock Linux kernel so RT tasks don't need to use a separate set of APIs from non-RT tasks and Timesys can guarantee timing, both for the CPU and networking. For soft real-time, Monta Vista seems to be pretty popular. The people I spoke with who used it said it was quite reliable but couldn't comment on support since they never needed any.
-Aaron
I know we're not the only ones to have been burned by Wind River's malloc. I know several major companies that also had to replace Wind River's code.
As far as being able to dynamically replace code, VxWorks isn't alone in that. Numerous other RTOSes out there can do the same thing, including QNX. QNX even supports the concept of a hot standby process to take over if the main process dies.
To give you an idea about how Wind River's malloc works, they keep a sorted linked list of fragments from the smallest to the largest. When you try and allocate a block, it walks the linked list until it finds a block large enough. Likewise, when you free a block it checks if it can coalesc the block with a neighboring block. It then goes through the linked list looking for a slot to insert the free block.
Yes, VxWorks may have been around since the 80's, but that's part of the problem too and it is showing its age. In the 80s embedded processors typically did not have MMUs. Now MMUs are quite common in the more powerful embedded processors.
You say you can't have low latency and memory protection? QNX proves that you can. It is low latency and *very* robust. If your driver dies, no problem, restart it. Timesys Linux also has a very low latency, although not as low as QNX. Timesys also has an interesting feature where you can guarantee CPU and networking resources. I can schedule a task to be guaranteed 5.8ms of execution every 8.3ms and it will guarantee that that task will get the CPU time allotted to it with the desired resolution. This is without increasing the system tick rate (usually 10ms). Timesys can also schedule a task to be higher priority than an interrupt. I'm not as familiar with QNXs scheduler, but it's also quite flexible from what I've heard.
As far as FAT, it is not a robust filesystem. It never has been. If the FAT gets corrupted or a directory entry gets corrupted it's difficult to recover. Other than possibly having 2 copies of the FAT cluster table, any corruption can be difficult to repair. If the FAT table gets corrupted, which table is corrupt and which is not? If a directory entry gets corrupted, it can be impossible to fix. For flash memory, unless you are using a device with special wear-leveling, FAT is about the worst choice since any file write that changes the size of a file requires a write to the directory entry and possibly the FAT table. If the table gets corrupted and you don't run a repair operation (which often ends up leaving orphaned files as lost clusters), the file system can happily corrupt itself to death. Why do you think every time DOS/Windows9x/ME crashed it had to repair the disk with scandisk? FAT is a poorly designed file system that was originally designed for 160K floppies and scales poorly. FAT32 is an improvement, but it's still not very robust. For flash, something like Linux's journalling flash file system 2 (JFFS2). More information on VxWorks file system support can be found here.
Basic VxWorks information can be found http://www.slac.stanford.edu/exp/glast/flight/docs /VxWorks_2.2/vxworks/guide/.
If anyone saw my earlier posts on VxWorks they would see I am not at all surprised about the problems NASA is having.
As someone with first-hand experience with VxWorks let me say that VxWorks' memory handling code sucks. Their malloc implementation has got to be the worst one ever designed. It fragments horribly and when fragmented has unusable performance. A malloc call can take many milliseconds when memory gets fragmented. Our box used to crash due to the fragmentation. I replaced Wind River's code with DLMalloc and that fixed the memory issue. We went from many tens of thousands of fragments to only a few dozen. Our reliability significantly increased as well after dumping Wind Rivers brain-dead malloc code. BTW, glibc uses a variant of Doug Lea's malloc code, so it's been widely tested.
Furthermore, in VxWorks there is no way to identify what process malloced a block of memory. There is no memory protection either (think DOS). If a task has a memory leak, VxWorks does not provide any method of tracking down the culprit. I had to add that support to the DL Malloc code I ported so we could find memory leaks and general memory usage by task.
Since malloc is critical to the working of VxWorks, if you run out of memory you are basically completely dead. Often the only way to recover is to wait for the hardware watchdog timer to kick in and reboot the system.
If a task dies in VxWorks with a bad pointer there is no way to recover other than reboot. The OS will not clean up after a task (i.e. free memory, close files, release semaphores).
As far as flash goes, VxWorks supports the FAT16 file system. As you know, FAT16 sucks. It only supports a limited number of files in the root directory. It's relatively easy to corrupt, and when corrupt it tends to corrupt itself even worse. There is no wear leveling support. If the FAT table or root directory gets corrupted, you're screwed. VxWorks is even worse since there arn't tools to fix a corrupt file system.
VxWorks is not a scalable OS. The OS gets slower as the number of tasks increases. Realtime support sucks. Although it has support for things like priority inheritance to prevent priority inversion, the best guaranteed realtime latency is half the system tick rate (the tick rate is usually 10ms).
Also remember that unlike open source operating systems, the source code to VxWorks is not available unless you pay some major $$$. Without the source you're basically working blind.
VxWorks is an old RTOS, and its age is definitely showing. It is not a robust OS.
As far as turning VxWorks into a firewall, you'll need to write all your own code. The VxWorks TCP/IP stack is an archaic vulnerable version of the BSD stack. TCP sequence number guessing is trivial. There is no built-in support for firewall support, NAT, or anything else. I have heard many many complaints about the VxWorks networking code. Although the box I'm working on is a router and broadband remote access server, we don't use the VxWorks TCP/Ip stack much. I am sure the VxWorks stack is vulnerable to many of the current DOS attacks as well.
Some of you might say why not use VxWorks AE, which adds memory protection. Think hacked-in memory protection far worse than Windows 95. It's slow, very buggy, and poorly supported. We tried AE and after many months we dumped it and went back to the non-AE version. Very few companies actually went with AE.
Today there are many other choices for an RTOS that are better than VxWorks. For our next major project we're looking at both QNX and TimeSys Linux. QNX is a true microkernel design with the core being around 70K. Every driver is protected and can be restarted if it crashes. I think you can even buy a medical grade version of QNX. TimeSys Linux is also pretty cool, with excellent real-time support and all the advantages of Linux. For something like the Mars rover, QNX would be better due to the limited amount of memory and greater robustness.
Wi
I can tell you that AE is in many ways WORSE than the standard VxWorks. It has a lot more bugs and is quite a bit slower. Think of regular VxWorks with memory protection hacked in, not designed in from the ground up.
As a VxWorks programmer for the last 5 years, I can honestly say VxWorks is a PoS that is losing market share at a tremendous rate to the likes of embedded Linux and QNX. Wind River decided to spend tons of money buying add-in products like Routerware instead of improving their RTOS. It was a huge waste of money and now they're paying for it. They're losing money hand over fist and have had a lot of layoffs lately. They were good at one time, but they have fallen far behind the curve now in embedded RTOS design, especially for complex systems.
VxWorks comes with support for a FAT flash file system, a completely broken malloc implementation, an ancient BSD TCP/IP stack, poor RT support, no memory protection, and no way to clean up after a task that dies. Not only that, it usually costs a fortune, but I've heard they're willing to sell it very cheap now because they're desparate.
I looked into embedded Linux for our next generation hardware and software and Timesys appears to have a very nice solution with hard real-time support. The kernel is fully preemptable using semaphores instead of spinlocks and has priority inversion support. They also offer resource reservations, so I can say "I want this task to be guaranteed 5.73ms of execution time every 9.8ms" where after 5.73ms the task either gives up the CPU entirely, or else changes to a non-RT priority to not starve other tasks. It's really quite clever. Not only that, unlike RT-Linux there isn't a separate API for RT vs non-RT tasks. Monta Vista Linux is soft real-time. It cannot guarantee context switching time, nor does it deal with priority inversion. In RT, priority inversion can be a major problem (see the first Mars rover for an example).
For an example of priority inversion say you have 3 tasks, a low priority, medium priority, and a high priority. The low priority task acquires a mutex semaphore to protect a critical section and starts processing. It is interrupted by a medium priority task. Meanwhile, a high-priority task unblocks and attempts to grab the mutex. The high priority task will block until the medium priority task blocks so that the low priority task can release the semaphore. A common solution is priority inheritance. With priority inheritance, as soon as the high priority task attempts to acquire the mutex semaphore, the low priority task has its priority bumped to that of the high priority task until it releases the semaphore. In this way, the low priority task will interrupt the medium priority task so that the high priority task won't have to wait as long.
QNX is also a very good alternative. Very fast context switching and extremely robust memory protection. I think with QNX you can even buy a license suitable for use in medical devices (i.e. you absolutely cannot afford to have the OS crash for any reason).
I've heard rumours that Wind River is dropping AE since nobody is using it. After our experience, I pity whoever tries it.
Also, unless you get the source to VxWorks, which usually costs a lot of $$$, debugging is a complete nightmare, especially when you hit a bug in the Wind River code (and there's a lot of them). Hell, they couldn't even implement malloc right!
Wind River is coming out with version 6 of VxWorks, but it is basically an enhanced version of AE. I'm not holding my breath.
-Aaron