Counterfeit Cisco Gear Showing Up In US

spazimodo writes to point out a Network World report on the growing problem of counterfeit networking equipment. The article surveys the whole grey-market phenomenon, which is by no means limited to Cisco gear — they just happen to be its biggest target. From the article: "Thirty cards turned out to be counterfeit... Despite repeated calls and e-mails to his supplier, Atec Group, the issue was not resolved... How did a registered Cisco reseller (also a platinum Network Appliance partner and gold partner to Microsoft and Symantec) acquire the counterfeit [WAN interface cards] in the first place?... Phony network equipment [has] been quietly creeping into sales and distribution channels since early 2004... Counterfeit gear has become a big problem that could put networks — and health and safety — at risk. 'Nobody wants to say they've got counterfeit gear inside their enterprises that can all of a sudden stop working. But it's all over the place, just like pirated software is everywhere,' says Sharon Mills, director of IT procurement organization Caucus."

Comments

Just FUD?

[LiquidCoooled]

This all smells of FUD.

What he didn't know was that phoney network equipment had been quietly creeping into sales and distribution channels since early 2004, when manufacturers began seeing more returns, faster mean-time between failures and higher failure rates,

Isn't this the same period we have seen bad caps making equipment randomly fail, batteries which blow up, hard drives not being hard enough and dead pixel nightmares for all different companies?

Is it not more likely that this is just another symptom of too much, too quickly and they should just improve their quality control and testing regimes?

Sure, the cards might have been resold, but they are branded cisco items bearing the entire cisco interface and functionality - somehow I doubt outright fake chipsets and devices like this can be produced by anyone other than cisco themselves.

The article manages to totally skip highlighting a single specific case of fake hardware, the nearest being a raid on a hardware repair centre where officials from a group of agencies pounced.

Reports in the San Francisco Chronicle made it appear at first like an immigration raid, as 12 illegal immigrants (11 from Mexico and one from Colombia) were taken away. But that wouldn't explain the presence of so many agencies, including the FBI, the U.S. Immigration and Customs Enforcement, the U.S. Postal Service and the Rapid Enforcement Allied Computer Team, which investigates large-scale, high-tech piracy and counterfeit cases.

Just because a group of people from different departments turns up does not justify the argument, there could be any number of reasons.
If it was directly related to fake hardware, don't you think cisco would be highlighting the fact a little clearer than supposition?

They just want to scare people into paying top dollar from the top tier people.
I have no problem with this, but it seems like an underhanded way to say it.

Re:Just FUD?

[superskippy]

I work for an ISP in the UK, and we've bought fake Cisco interface cards in the past (although it was before I started working there), that we're labeled as genuine.

So this stuff definitely does exist.

Re:Just FUD?

[Rice-Pudding]

Sure, the cards might have been resold, but they are branded cisco items bearing the entire cisco interface and functionality - somehow I doubt outright fake chipsets and devices like this can be produced by anyone other than cisco themselves.

Whether or not this is what happened in this particular case, I don't know. But in general, the issue is not that someone has taken the time to reverse-engineer a complete product and produce it again from the ground up. The "fake" hardware likely comes from any combination of several places:

• Chip vendors often have huge inventories of chips that failed testing, but are otherwise marginally functional. Some of these chips could be branded and sold by an unscrupulous factory (hehe, that sounds funny :-) as legitimate parts. Or more likely, they can be sold to the guy in the next point:
• Factories in 3rd-world or offshore countries (cheap labour) can and do produce legitimate hardware items for some of the big-name companies, of which Cisco is one. That is, they produce the legit hardware by day. After hours, or for a portion of the day, they can use the exact same process, exact same tooling, etc. to produce the knock-offs. These are then distributed through some other means.
• Finally, the contract manufacturers (factories in the above point) will have many products that failed QA, but a marginally functional. These also can get sold as counterfeit gear. So the reverse-engineering is not so much the issue (although I am sure there is some degree of that). But as another poster mentioned, if you have the expertise to completely reverse-engineer something and reproduce it, you should go into business yourself selling a competing product that is much cheaper:-)

Work great

[Anonymous Coward]

those Gears work nicely here. BTW first po$%&$&R/&A98908 NO CARRIER

If they can make something good enough for counter

[Sinryc]

If they can make something that people will think is good enough to be a Cisco product, they should go legit and sell cheaply. I mean it would be genius of them

Photography gear

[dedazo]

The 'grey market' for cameras, lenses and other accessories is also huge, especially now with the wild proliferation of digital cameras, although it used to be smaller in scale in the days of film SLRs.

Even reputable shops like Adorama will sell you 'grey' prosumer Nikon digital SLRs for example. The difference is the lack of a US-actionable warranty and funky things like manuals in Turkish and whatnot... but other than that the gear is largely the same (be careful who you buy from anyway!). These things typically go for about 10% less than the 'straight' ones.

I've bought a couple of high-end Canon lenses this way and I haven't been burned yet, but I probably won't be doing it anymore. Too much risk.

Counterfit vs. Legit

[JakiChan]

My understanding is that a vendor is contracted to produce, say, 100,000 cards for Cisco. They make 100,000 and then another 100,000 more (say without the Cisco logo or whatever) and sell the extra ones on the pirate market. It's not like it's totally hacked together - this is gear off of the same production line. They may sub in some cheaper components.

Now would I knowingly use pirate gear in my production network? No. But when I was building a lab at home and needed 20 WIC-1Ts I was sure glad I could get them on eBay in bulk. Probably not legit but I wasn't planning on putting my home lab under Smartnet.

not quite as bad...

[User 956]

This isn't as bad as when pirates pirated an entire company: NEC. Yeah, they had fake buildings, fake manufacturing facilities, fake executives, everything.

Well for a start

[LWATCDR]

Don't build stuff in China.

To be blunt Cisco and 3Com build stuff in china because it is cheap. The people that build the stuff can pick up a little extra money selling the gerbers , firmware, and document ion to the counterfeiters.

This is the price price for doing business in China and other very cheap countries.

What will really become expensive is when these companies can take what they have learned building stuff for Cisco and 3Com and then compete with them directly.

You can pay now or you can pay later.

Re:If they can make something good enough for coun

[Rosco P. Coltrane]

If they can make something that people will think is good enough to be a Cisco product, they should go legit and sell cheaply. I mean it would be genius of them

You miss the point : people who make counterfeit products pay peanuts to manufacture the fake goods, and sell them with a huge markup because the goods are branded with the logo of a company that makes expensive stuff. If they went legit and sold Cisco-compatible equipment under the SuperCrapola brand, instead of selling illegal Cisco-compatible equiment under the Cisco brand, they'd be a lot poorer.

It should be EASY to track.

[khasim]

These are physical items. It's not like software.

You buy them from a store. The store has to have them on hand or order them. Either way, since the store you're buying them from did not make them, shipment will be required.

So just keep following each shipment back until you find the company that manufactured the parts or the company that "cannot find their records".

There, problem solved.

Re:It should be EASY to track.

[Anonymous Coward]

They are easy to track up to a point. I work for a large network equipment vendor who is constantly targeted by counterfeit equipment. Although we can track the origin up to a point, it usually ends up leading to some shady manufacturers or criminal enterprises. In one case I was involved in, some legitimate cards were sent from an authorized manufacturer out the "front door" but in the "back door" they were receiving counterfeit ones and shipping them along with the good ones. One time a truck was HIJACKED in Asia and the good cards were swapped with counterfeit ones and sold into the channel that way. Who hijacked the truck, who stood to profit (or how) and who made the fake cards we don't know, but when the customer received a bunch of them, they looked smelled and felt like real cards, they just didn't work. The OS didn't even recognize them and couldn't even load the drivers. Posting Anonymously for good reason.

I'm in no danger

[antifoidulus]

I know a genuine Sysco 4507 when I see one!

Cisco RAM Trick

[mahesh_gharat]

One of the Cisco vendor in my area used to replace the original RAM chips from new Cisco routers before shipping. They used to replace those RAM chips with made in taiwan RAM chips which were dirt cheap (1/5th or lesser in price). Then this vendor used to sell those original RAM chips, that they earlier removed from Cisco routers to other customers at higher rate. PROFIT.

How do I know this?
The guy who use to work there, was my college mate during my Computer Science graduation days. You can still find all of us drinking beers on Weekends at near by joint. ;-)

not just FUD - shortcuts by sub-sub contractors

[caesar-auf-nihil]

I'm not surprised by this - I'm seeing it more often with supposedly fire safe parts with the "UL" tag on them. Since so many electronic parts/appliances now have such very tight profit margins, the following happens:

Primary original equipment manufacturer (OEM) subcontracts out to a cheaper source to make some profit on the part.
Secondary part supplier, also hit with tight margins, subcontracts to local supplier/small business to make the part.
Tertiary part manufacturer, also hit with tight margins but glad to have the business uses off-spec parts, or in the case of flame retardant rated plastics, dilutes the specified plastic with non-flame retarded plastic to get the parts made on time, and cheaply.

There has been an increase in the parts that have UL tags "failing" random pulled fire tests that UL makes by going into stores and randomly pulling consumer goods off the shelves. So I'm not surprised that this is happening in other areas as well when all sorts of quality control go out the window since the OEM can't directly supervise the secondary and tertiary suppliers, and they won't know the part is off-spec until they get the failed test. Once the tertiary vendor has made the part once, they usually have all the molds and other expensive equipment to start making knock-offs, especially in areas with poor law enforcement.

It's apparently a life-threatening problem!

[rickkas7]

FTA: "What if it wasn't a bank subnet that went offline because of a faulty card in the router? What if it were an air-traffic control network instead?" van de Gohm asks. "This is no different than counterfeit medicine in the pharmaceutical industry. And it's potentially just as life-threatening."

If the air traffic control system can go down because of a single faulty card in a router, fake or not, I'm thinking I want to avoid planes, and look up a lot more than I do now.

Re:Don't say I didn't warn you

[Trillan]

Hey, that was my thought!

Its not like Cisco has a manufacturing plant...

[Lanboy]

They send thier chipsets and engineering specs to an outside company (flextronics) just like all the other vendors. I imagine that with ISO9001 certifcation making every detail of label placement and branding a documented aspect of the manufacturing process, the details on how to build a card can fit on a USB drive, and be sent to taiwan or china for the incredible markup Cisco enjoys. I would further assume that the failure rate off the assembly line is about the same as the real production runs, its just a matter of who is going to bother QAing parts that are conterfeit.

For that matter the cards that don't meet vendor QA are a likely source of these counterfeits.

Keep in mind, the markup on flash and dram memory that is essentially identical to off the shelf memory is intense, and back when I cared about how much the crap cost, I would skimp on the gen-u-wine cisco memory or pix interface cards myself. I wouldn't want to buy a conterfeit DS3 blade though...

The scary thought is that if Chineese plants are going to slap together a counterfeit router, how hard would it be to add wiretap capability. THE YELLOW IT PERIL!!!

False Confidence In Non-Counterfeit

[aldheorte]

"Nobody wants to say they've got counterfeit gear inside their enterprises that can all of a sudden stop working."

That sentence reads the same if you remove "counterfeit". Hardware and software that can all of a sudden stop working is a fact of life, regardless of manufacturer.

The use of logos to indicate that a piece of hardware is genuinely from another company when it is not is unethical and should be stopped, but this argument is simply a scare tactic attempting to disguise the real interest, which is that of the manufacturer whose logo is on the product and is angry they did not derive any revenue from the sale. Otherwise, they could care less. From a consumer standpoint, safety is found in redundancy and contingency planning, not trusting that the logo of any one manufacturer on an item means it will not suddenly stop working. I do not blame the manufacturer for wanting in on the sale, but tell it straight, don't childishly trot out the bogeyman to get sympathy,.

Folex or illegal production?

[Kadin2048]

Did you examine or keep any of the fake ones around?

I'm really curious to see a "fake" one right next to an "authentic" Cisco part. Are they duplicates? Or just some other network card that they stamped a phoney Cisco logo on?

It would make a pretty big difference. In the latter case, they're nothing more than counterfeits, like the fake Rolexes that you can get from guys in Battery Park.

But if they're actual Cisco parts, being sold "unauthorized" (perhaps the factory they're outsourcing the assembly to decided to run an extra production shift or something, make a little money on the side), then the situation could be a lot different.

So which is it? A fake Rolex that actually has a $0.25 quartz movement inside? Or the real deal in terms of functionality and hardware, being made somehow without Cisco's approval and without going through their distribution chain?

Re:Folex or illegal production?

[tkrotchko]

Years ago we purchased a Cisco Ethernet interface and paid some outrageous amount. Like... 4 figures. It was a standard PCI Ethernet card with no ID on it. Except the board had an FCC number on it. We checked, and it turned out to be a cheap Ethernet card that was readily available for about $25 anywhere. The only difference was there was no manufacturer identified.

Now, I don't know if this was a special case, but surely somebody figured out that some of these parts are generic parts and is selling them with phony Cisco papers and making a tidy profit.

Overproduction?

[Kadin2048]

I've heard stories that a lot of the off-brand clothing and shoes that you can buy in Asia are actually produced in the same factories that make name-brand stuff. At the end of the day, after finishing a run of $US_BRAND, they'll bring in the third-shifters and run another production cycle and just not put the logos on. (And depending on who you ask, use lower quality raw materials, etc. etc.)

I wonder if the contract electronics assemblers are doing similar stuff? Seems like it would be pretty easy. If you're assembling network cards for Cisco, you know where all the parts are coming from, and how to put them together. Chances are, all the parts suppliers are also going to be Chinese; not too difficult to call them up and request an extra 1,000 widgets, and just pay for it out-of-pocket. Then you just keep assembling parts until the supplies are exhausted, package up whatever you've promised to deliver to the foreign company (Cisco), and sell the remainder to a local distributor who makes sure they disappear into basically untraceable Asian markets.

As foreign companies outsource more and more of not only the production and assembly, but also the supply-chain-management and procurement functions to "one stop shops," this becomes easier and easier. There are plenty of companies who would be happy to manufacture your widget for you, and handle all the parts sourcing -- allowing Western companies to avoid all the unpleasantness that sometimes involves. But that means there's very little way to verify whether the company is ordering more components than are actually needed to complete the run. In fact, it's nearly impossible -- without intimate knowledge of the part's defect rate and of manufacturing errors, you have no idea how many extra parts need to be ordered. Are they buying 5% more ICs than necessary because they know the factory tends to produce crummy ones (but is still the cheapest available), and are looking out for you? Or are they padding the order so they can overproduce and sell the excess on the side?

Like you, I have little sympathy for American companies who get bitten by this. If they wanted control over the manufacturing process, they could keep it here in the States. If counterfeiting is what happens when you outsource everything to a country with cheap labor and little respect for foreign intellectual property, you made your bed and now you can sleep in it.

Most Cisco hardware not ASIC accelerated

[anti-NAT]

You generally have to go above the 7000 series to get ASIC accelerated forwarding. As an example, the specifications of a Broadcom BCM1250 read remarkably like the specifications of a Cisco 7301, because that's what's inside one.

show ver

on the router shows the CPU model number, and

show controller <blah>

will show you the current register values, which you can then look up in the BCM1250 reference manual.