RIAA's 'Expert' Witness Testimony Now Online

NewYorkCountryLawyer writes "The online community now has an opportunity to see the fruits of its labor. Back in December, the Slashdot ('What Questions Would You Ask an RIAA Expert?') and Groklaw ('Another Lawyer Would Like to Pick Your Brain, Please') communities were asked for their input on possible questions to pose to the RIAA's 'expert'. Dr. Doug Jacobson of Iowa State University, was scheduled to be deposed in February in UMG v. Lindor, for the first time in any RIAA case. Ms. Lindor's lawyers were flooded with about 1400 responses. The deposition of Dr. Jacobson went forward on February 23, 2007, and the transcript is now available online (pdf) (ascii). Ray Beckerman, one of Ms. Lindor's attorneys, had this comment: 'We are deeply grateful to the community for reviewing our request, for giving us thoughts and ideas, and for reviewing other readers' responses. Now I ask the tech community to review this all-important transcript, and bear witness to the shoddy investigation and junk science upon which the RIAA has based its litigation war against the people. The computer scientists among you will be astounded that the RIAA has been permitted to burden our court system with cases based upon such arrant and careless nonsense.'"

Comments

One quick thought about licensure

[Raul654]

I saw something in the transcript that I wanted to point out before anyone else here criticizes Jacobson on it:

Q. By what body are you certified as an engineer?
A. By no professional society.
Q. No professional society? Is there any organization that has certified you as an engineer?
A. No.
Q. Are you part of any peer regulatory body?
A. I don't quite understand what you mean by --
Q. Are you part of any body the members of which are peer-regulated?
A. Can you give me an example of what you are --
Q. A lawyer, an architect, an accountant. I thought an engineer had to be certified by a peer-regulated body.
A. To be called a professional engineer they do.
Q. So are you not a professional engineer?
A. I do not have a PE license.

Based on his Jacobson's research page. It looks like Jacob's, a professor "on the faculty of Electrical and Computer Engineering", is a computer engineer. Given that, the above statement is totally understandable As a computer engineer myself, I can say that it is *EXTREMELY* rare for a computer engineer to be a licensed PE. (Not a single computer engineering professor in my University is). PE's are common in engineering professions where somebody needs to sign off on the final product - civil engineering especially, and mechanical engineering to a lesser extent.

Re:One quick thought about licensure

[Yvanhoe]

I would also say that I don't really understand the tone of the /. post here. I have read half of the 143 pages and I must say Jacobson has made patient and correct statements all the way of the interview. It must have been really frustrating explaining how MAC and IP address work to a lawyer.

Re:One quick thought about licensure

[ClosedSource]

I think there are a lot of organizations that would love to take your money and it might be good PR to join one, but I don't think it proves anything about your abilities.

Re:One quick thought about licensure

[NewYorkCountryLawyer]

It's not a question of how patient he was, or how frustrating it was for him, or how ignorant I am of technical things. It's a question of a man purporting to giving "expert" opinions which are not based on any verifiable methodology worthy of being used in a court of law to support someone's claim against another person for tens of thousands of dollars.

You shouldn't be feeling sorry for him, you should feel sorry for his thousands of victims.

He had a choice of whether to accept an assignment he was not qualified to do, or to perform the assignment in a shoddy and unworkmanlike manner, printing out sloppy imprecise opinions by rote inculpating innocent people. He also could have chosen to spend more than 45 minutes on the assignment, and to have done some verifying and testing and probing, in which event perhaps he would not have found himself opining that there was copyright infringement in each and every case in which he was called upon to testify.

His victims were given no choice.

If you read the deposition along with the written opinions he has given (exhibits 15 and 16 listed here), you will see that he has repeatedly stated things in his written opinion that he has no support for. And make no mistake.... the RIAA has repeatedly used those "expert" opinions to convince the judge that they had evidence of a copyright infringement by the defendant when in fact they did not.

And by the way, experts who know what they're talking about have no problem explaining themselves to lawyers, judges, jurors, or anyone else.

It's experts who are phonies, who haven't done their homework, and who don't have proper backup for their opinions, who have a problem with that.

One quick thought about expert witnesses.

[Anonymous Coward]

"Q. Are you part of any peer regulatory body?
A. I don't quite understand what you mean by --"

A professor is part of a "peer-regulated" body. He may not be able to call himself an engineer, but that doesn't mean he's not an expert.

Some "expert"!

[Coopjust]

This guy comes to the conclusion that it was the defendant's computer, even though there is no evidence from hard drive forensics, and he says there is no wireless router since the IP was registered to the house.

Also, he kept no records of the forensic analysis, and he is always trying to pin the idea that an IP address is a computer, even though it's obvious he's avoiding or twisting questions, even to someone who isn't so technically inclined.

Re:Some "expert"!

[tftp]

To me it's crystal clear that they observed someone's Kazaa traffic, but when they snatched the HDD it was some other computer. The reason for that is not some outlandish NAT or Kazaa hack, but simply an IP address confusion (either a true collision, or a wrong DHCP log at Verizon - not that they care.)

Re:Some "expert"!

[geoskd]

Also, he kept no records of the forensic analysis, and he is always trying to pin the idea that an IP address is a computer, even though it's obvious he's avoiding or twisting questions, even to someone who isn't so technically inclined.

I feel bad for the guy. Yes, he sold his soul to the mafiaa, but internetworking is difficult enough to explain to someone with some background in IT. This deposition is exactly the same kind of thing you would get if a lawyer had to explain tax law to a computer engineer, with the added benefit that the Q/A process is an exceptuionally difficult way to go about educating someone on how this crap actualy works. The long and the short is that The guy can demonstrate that the machine that was running KaZaa thought that its IP address and the IP address of the network connection were identical. This shows that either KaZaa was running on a machine that was *not* behind a NAT, or someone went to great lengths to convince KaZaa that it wasn't behind a NAT and have it work correctly. The net result is that it is reasonable to say that the computer that had that IP address was the *only* device connected through that particular Cable Modem / DSL line at that particular time. If it was behind a NAT, KaZaa would have showed a primary IP of 141.155.57.198, and the host IP of something like 192.168.1.100, or somesuch. Thus when he says that an IP address uniquely identifies a computer, in this case it does. He tried very hard not to say that it is always true because it isn't. That is why the lawyer (who clearly doesn't understand internetworking, but had a list of "gothchas") couldn't pin him down to anything. Otherwise, the only real glaring omission that should have been added is that some routers have *multiple* MAC address' one for each port. (modern routers only have one cause each connection can safely assume that it won't be rerouted back to the same router, but some early routers had a unique MAC for each port, before someone discovered that it was a waste of good MAC's)

-=Geoskd

Re:Some "expert"!

[Ungrounded Lightning]

An expert who ignores that there is a subnet mask that gives you a full 4th octet under a single IP either hasn't ever worked with networking, or is not aware of the knowledge they are shelling out to first year students in technical institutes;

The record doesn't show anything like that.

One of the few things he did right was determine that the IP address was assigned to the computer, that NAT wasn't in use. The tool he used does this by extracting and displaying both the "from" IP address on the packet and a copy of the interface's IP address that KaZaA helpfully records in the data part of at least one of the packets of the exchange. This eliminates NAT on routers and wireless access points.

Since the connection was a dialup with a DHCP-assigned dynamic IP address, it would have a single IP address - which eliminates multi-address subnets. The combination of that with "no NAT" eliminates wireless access points and multi-computer home networks. (The computer that dialed up COULD be NATting and forwarding for others, but it WAS the one that ran the KaZaA client.)

But it doesn't eliminate the possibility that the IP was actually assigned to the defendant. There are a lot of ways that could happen. For instance: Maybe the clocks were off between the ISP's logger and the tool that captured the IP address of the "pirate publisher". Maybe the ISP's logs weren't high enough resolution and there was a logon-logoff event. Maybe somebody typoed the IP address somewhere. And a bunch of other possibilities. The MAC address wasn't recorded (or recordable remotely) so they don't have a unique identifier of the computer's wireless card, and even if they did it's possible to hack 'em.

Given that there's no sign of a KaZaA client or music files on the captured hard drive, it seems likely that th identification of the defendant's computer from the ISP's logs and the IP capturing tool output was somehow in error, and they got the wrong victim.

Re:Some "expert"!

[Anonymous Coward]

If this was indeed a dial up account, there is also the possibility that someone stole the username and password and dialed up from another computer entirely, possibly dialing up from a list of multiple stolen names and passwords randomly to keep from being detected. I didn't see anywhere in the testimony that a dialup account was used, I just saw that there was no router. Unless the defense can come up with an expert to contradict the testimony on the ip address under a nat (do any versions of kazaa or kazaa lite fake subnets?) then they should not even bring this up, if the plaintiff does it looks like a red herring, focus on the fact that three pieces of data were collected, pieces 1 and 2 were necessary to id the plaintiff (if either is wrong, plaintiff's account, nothing more, was incorrectly identified) and neitehr were verified or documented nor can they be publicly verified. The third piece of evidence, the only collected evidence fromthe defendant herself, contradicts the first two. The expert assumes the first two pieces are correct, while the third is incorrect although he admits there is no evidence it was erased, reformatted, or tampered with. Funny how a paid witness automatically assumes another paid witness and a subpoenad isp are correct yet law enforcement (I assume they obtained the hard drive) was incorrect.

Also, just because an IP is identified (and watch how defensive the expert gets, they're messing with his bread and butter, if he can't sell his service to the riaa he loses income, and his service is convincing jurors that an ip address is a defendant, which it is not) does not mean the defendant is identified. Another computer could be plugged into a modem (assuming it is a cable or dsl modem) another person could sign into a dial up account if that is the case, expert admits he cannot id mac addresses, only isp. then of course all the issues with the screenshot and verizon's determination. Make big points about how this method had never been verified or checked for accuracy, it was not. Good example, it was scientific knowledge that leaches cured diseases even though it was never verified, but every leach salesman insisted it was science at the time (and the plaintiff objects, OUCH!)

THen there is the issue of validity, how accurate is his method? Has it been tested against mac spoofing, ip spoofing, where is the data? Just because he's a "really smart guy" doesn't mean we should trust him without evidence. Has he tested his method against every version of kazaa and klite out there? Has he tested it against tor (does he know what tor is) and ip spoofing software? Does he have any data to prove someone far more clever than him did not make him look like an idiot? If not does that make a reasonable doubt that he is wrong? And if he has no indication as to how accurate his method is, what says his method is not 1%, 20%, 50%, or even 100% wrong? Let him say something based on opinion, and slap him, he's a scientist for christ sakes, where's the data? How can you expect any reasonable person to accept data from an unproven scietntific method that he refuses to let get verified, oh that's right, this is a source of income for you, don't want to mess with that.

Do the same to the company that took the screenshot, if they want their evidence presented, they have to demonstrate methods and reliability, otherwise demand it get thrown out. This is the 21st century, we do not deal with witchcraft and hocus pocus, how does it work? Finally, get to the fact that there is no collaborating evidence of this other than two "experts", both paid to testify by the plaintiff, neither able to collaborate their findings or validate their methods, claim this person, who has been documented to have no computer skills, whose hard drive had absolutely no evidence and no evidence of tampering, did something illegal. And their claims are not specific scientific claims, but they are biased, unjustified assumptions based on highly suspect data (they keep saying they identified a computer when all they ide

Re:Some "expert"!

[alita69]

He has no proof that NAT was not in use. He says flat out he doesn't even know how the defendant's computer was set up. There isn't a think in his testimony that truly proves no NAT. He says Windows was set to use DHCP, which is true of most home machines behind a NAT box. He claims that the packets examined not having private IP addresses proves there wasn't a NAT. Well, no, not even close. They weren't captured from the defendant's computer, so they had already passed through the NAT box when captured, which means the private IPs would have been replaced by the public IP from Verizon.

This guy is shoveling bullshit, and does a lot of dancing around questions that would open credibility holes in the RIAA cases.

IPV6

[Nom du Keyboard]

There's a spot down in there where the RIAA expert refers to IPV6, and this refers to 2004. That alone should get him laughed out of the tech community.

Not to mention that he maintains he can trace the IP address back to a specific ISP account and computer (emphasis mine). Unless he's a Peeping Tom with a web-cam in the defendant's house, the RIAA should be demanding their money back from him.

Oh, and then there's the place where he maintains that at the time the computer was imaged many months afterwards, that there was no wireless router in use at that time Media Sentry "discovered" this "infringer". Is there a log that keeps records of every IP address you've ever connected with?

And I have to laugh at how he refers to "registered" computers. I thought he was talking about gun registration, or some such thing. I've never heard of my own computer being "registered" to anything. Is this another invented RIAA term, like "Media Distribution System"? Has anyone else ever referred to KaZaA, or any other P2P program, as an MDS? Ray, you can't be letting the RIAA frame the terms of the debate to ignorant Judges.

And don't miss the parts where he says he didn't actually document any of his findings because there was nothing to find, however, you should go through your own copy of the disc to verify my Registry findings that no wireless router was in place. He's supposed to be the expert, and he wants the defense to replicate his findings in the Registry??? Are there any registry experts here? Probably a few, but not many. But he assures us it's there.

Biggest thing is that he says that no KaZaA was present, nor any infringing music files. The only way the RIAA can respond is you sent us the wrong hard drive. No question that the person in question might have actually been innocent. RIAA -- You Bastards!

Glad to know that we helped, Ray! Keep fighting the good fight!

Re:Damn

[NewYorkCountryLawyer]

I think many of his students will be appalled at the actual contents of his testimony.

For example, he teaches a course in "Information Warfare", the entire thrust of which is that the internet is dangerous and insecure in the extreme. He teaches students all about the infinite numbers of vulnerabilities.

Then he testifies that he forms an opinion in 45 minutes based upon some printouts from an investigator who pulled down some screenshots from the internet.... with no verification whatsoever.

And that he's give about 200 such opinions. And so far, 200 out of 200 concluded, without reservation, that there was indeed copyright infringement.

What kind of grade would he issue to a student who handed in work like that?

This testimony fails a basic test for evidence

[grandpa-geek]

IANAL, but I understand that there are standards for admissibility of scientific evidence, and the questions quoted below (and several that follow) cover them. The most recent ruling is called "Daubert."

Whatever this witness has to say based on his methods is useless because the methods have not been generally accepted and/or there are no peer reviews or tests of the methods' accuracy/reliability and no known level of accuracy/reliability.

Q. Has your method of determining from
the MediaSentry materials whether a particular
computer has been used for uploading or downloading
copyrighted works been tested by any testing body?

A. Not that I have submitted.
Q. Do you know anyone else that is using
your method, other than you?
A. Not that I'm aware of.
Q. Has your method of determining
through the MediaSentry materials whether a
particular computer has been used for uploading or
downloading copyrighted works been subjected to any
form of peer review?
A. Not that I'm aware of.
Q. Has your method of determining from
the MediaSentry materials whether a computer has
been used for uploading or downloading copyrighted
works been published?
A. No.
Q. Is there a known rate of error for
your method?
A. No.
Q. Is there a potential rate of error?
MR. GABRIEL: Object to the form.
A. I guess there is always a potential
of an error.
Q. Do you know of a rate of error?
A. To my process, no.

Q. Are there any standards and controls
over what you have done?
A. No.
Q. Have your methods been generally
accepted in the scientific community?
A. The process has not been vetted
through the scientific community.

Re:PE software engineers

[NMerriam]

Requiring PE involvement in the software world might work to put some kind of (very welcome) brake on the reckless development practices that many companies follow, but given the added cost and added legal responsibility, I suspect it'd just end up decimating the domestic software labor pool and pricing custom software out of the reach of all but a few companies. Substantially more programs get built during any given year than bridges, after all.

True, but a lot more "things" get built than bridges, and most things don't require a PE. The software world has no reason to be different -- PEs could be required for some only segments or applications. You don't need an engineer to sign off on your homemade bookshelves, and you wouldn't need one to sign off on your shareware CD catalog program. But Red Hat and Microsoft might very well be expected to provide a certification that the kernel or cryptographic subsystem they provide are built to certain accepted development and code reviewing standards.

Re:Relevance of the registry for DHCP

[Dachannien]

It could also be distressingly misleading if, for example, file sharing was taking place on that IP address when it was assigned to someone else, and shortly thereafter, the computer being examined had received that IP address and successfully re-requested it every time after that.

Standards for Evidence?

[Proudrooster]

Wow! I just finished reading the ASCII transcript and would be embarassed to bring this case. Just looking at the following facts:

• The "expert" did about 45 minutes worth of work and produced no evidence to support the allegations and produced almost no documentation.
• The "expert" does not fully understand how the software that gathered the evidence functions
• The "expert" does not know if the information he received from the ISP (Verizon/3rd Party) is accurate.
• The "expert" does not know if the clocks were synchronized between the evidence gatherers and the ISP.
• The "expert" can not identify which computer is involved in the allegations.
• The "expert" can not identity what physical person is involved in the allegations.
• The "expert" understands the Internet is insecure and computers can be taken over and remote controlled.
• The "expert" understands there are several methods which could have mistakenly identified the accused, e.g. "ip spoofing".
• The "expert" either lied under oath or is not really an expert when he said he could not make certain determinations about a computer based soley on the harddrive. He stated he could not tell if the computer had a "wireless network card" by looking soley at the registry without the computer that the registry came from. Huh???? Hint to the "expert", look for "WLAN" in the Registry, double hint, WLAN='Wireless LAN'.
• The "expert" could not demonstrate that the files uploaded/downloaded were copyrighted material and simply had a screen shot of some filenames and ip addresses from a 3rd party.
• The "expert" acknowledged that screenshots could be faked.
• The "expert" acknowledged that public IP addresses can change often and could be spoofed

This entire case hinges on screenshots, mystery analysis software "encase", a questionable expert, and an IP address obtained from an ISP. The evidence in this case doesn't even make it to the standard of "hearsay" not to mention the fact that the plaintiff lawyer appears to be highly inexperienced with Turets syndrome and keeps blurting "Objection to form."

I suspect that if one were to dig deeper into the so-called evidence, one would learn that information obtained from Verizon is prone to error, and that the procedures for generating the screenshots from KaZaa are based on assumptions which are prone to error and probably performed by monkeys. I want to read the deposition from the "dude/monkey" who took the screenshots, please post that one next.

If I were the lawyer for the defendant, I would already be filing my motion for dismissal "with prejudice" with the award of reasonable lawyer fees for having brought a case without any evidence.

Are there any standards for evidence? Is a printout obtained via supoena really a standard for evidence? If so, I can prove anything you like and as a bonus, I even have a professional certification. :)

a joke

[acidrain]

A scientist, an engineer and a programmer are on a road trip. Their car goes out of control on a steep hill and they barely make it to the bottom alive.

The scientist tries to calculate the distance to the nearest repair shop, the engineer suggests checking the wiring and brake pads, and the programmer suggests driving to the top and seeing if it happens again.

My point? Programmers and engineers are different. The best way to solve their problems is different. I trust this CTO more because he doesn't have engineering certification. In the same way a person with a music degree is less specialized as a programmer.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:Objection, your honor!

[danrik]

I am sorry, but I have completed four years of undergrad and three years of a PhD program and have never even heard the words mentioned in my education.

As an outside observer reading a transcript, I can infer their meaning from their roots and context, even if I didn't come from a family of attorneys. As someone being paid to defend an untenable posision, in the high pressure situation of a deposition (and make no mistake, having been deposed before, depositions are quite intimidating), I can see why someone didn't make an obvious leap.

A good attorney does not need to make an already hostile witness any more hostile by being a pretentious ass, not that this at all questions the validity of the line of questioning.

Re:so sad

[mamer-retrogamer]

Perhaps you should go back to stealing. It'll cost you less (jail) time and money if you get caught shoplifting a physical CD than if you are accused of making an unauthorized copy of it.

Re:Damn

[violet16]

I'm not especially techy, but it seems that the general opinion here is much harsher on Jacobson than is really warranted. Obviously most of us here think he's on the wrong side of an important fight, but we need to actually address what he says, not dismiss him because we think he sucks.

The on-topic +5 posts here seem very biased to me. They are insulting towards Jacobsen but fail to identify anything like an actual error in anything he says. The general opinion as to why he's wrong seems to be (a) the RIAA could have faked their screenshots, (b) the application could have been custom-hacked to lie about its private IP address, (c) Jacobson doesn't know exactly how the sniffer technology works. Which is all true. But it's quite unlikely that the RIAA is faking up screenshots so they can accuse completely random people of illegal file sharing, or that the accused custom-hacked their Kazaa client, or that the sniffer tech is totally bogus.

If you're accused of illegal file sharing and you're innocent, I'd imagine plausible reasons why are:
(a) They identified the infringer's IP address correctly but are mistaken in thinking it was assigned to you during the relevant time window; or
(b) The infringement did take place on your IP address but you have an unsecured network (ideally a wireless router) and god knows who did it; or
(c) The infringement did take place on your computer but several people use that and who knows which of them did it.

Unless Verizon screwed up, (a) seems out. And despite what Ray seems hell-bent on establishing, so does (b), given the public IP/private IP match. That strongly suggests it was indeed a single computer with a direct connection to the internet. Now, I know it's not 100% proof. But it seems to be quite likely, and I'd think it certainly sounds plausible to a judge.

Now please correct me if and where I'm wrong! Can we actually find something Jacobson said that's plainly wrong, and not just possibly wrong under unlikely circumstances?

Re:Pretentious? Hardly. Never took Latin?

[LarsG]

Quidquid latine dictum sit, altum sonatur.

Not trying to be a troll here, but why is knowledge of latin often seen as a requirement for intellectualism?

Re:Respect

[OnlineAlias]

I am simply shocked that no one stated that the type of card that is currently being used is stored in the registry, and he could have simply looked. All of this "was it wireless" "was it not wireless" could have been blown out of the water. Also, the guy kept referring to MAC addresses being transmitted in the packet, they aren't. They are transmitted in the frame. IP knows nothing of MAC addresses. There were so many flaws in his testimony I was simply dying while reading it...I so wanted to be there to tug on someone's shoulder and say "WAIT, he just said IPV4, now he said IPV6! and "if behind NAT, the addresses are irrelevant, and DO NOT have to be RFC1918 compliant private, they can be any address one chooses". ARG...

Re:Damn

[NewYorkCountryLawyer]

They're not biased, violet.... They're just pointing out the glaring technical deficiencies and fallacies in Dr. Jacobson's "opinions", and the absence of any proper methodology used by him in arriving at them. I'm not especially techy either, Violet, but this thread is really one for the techies. People like you and I need to step aside and let the tech community vet Dr. Jacobson's "methods". He himself admits he has never published them, or tested them, in any way. We need to let the tech community do its work.

Re:Damn

[Compholio]

(a) They identified the infringer's IP address correctly but are mistaken in thinking it was assigned to you during the relevant time window; or

This is more complicated than you make it out to be - just because your ISP assigns you an IP address doesn't mean you have to use it. You can very easily spoof someone else's IP, and if you were up to something inappropriate (say, a huge file sharer) you might want to do that. Before you move on and say "but then you can check the MAC address", you can change your MAC address too - it's not that hard.

One of the networks I connect to regularly registers and tracks computers based on MAC address, where you must fill out a form to use the internet. In order to not have to fill out the damn form every time I'm using a new computer, or switch from using a hard-line to the wireless, I've registered the address DE:AD:CA:FE:BA:BE. So, I can easily change the MAC address of whatever computer I'm using to an already registered MAC. However, that MAC is "mine" and is personally registered to me - if someone on the same network wanted to do something illicit then they could easily dump the DHCP or ARP traffic on the network and randomly pick someone else's MAC address. You can do this with your home ISP as well, it's like identity theft for computers and it's not hard at all.

Re:PE software engineers

[ClosedSource]

"What does the fact that two bytes having 65,536 states have to do with complexity? That's irrelevnat to the complexity of the system; any software system can (and should) be broken down into smaller functional modules that can be further broken down into smaller chunks etc"

Sure, there is decomposition, but the number of correct states remains high compared to physical systems. However, since nearly all software has bugs, there are other states the software can assume that are totally unknown. So the number of theoretically possible states (65,536 states in the case of 2 bytes) determines the worst-case complexity (based on one of several accepted meanings of the word "complexity").

"If we are going to have a pissing contest with number of variables, the fact that the real world is not discrete like logic leads to a much greater number of possible positions for physical things.."

Actually the fact that a physical system at the macro level isn't discrete is why it's not as complex as software. Two positions that are close together in the physical world are nearly indistinguishable in their effect on a system. In software, however, changing a single bit can result in radically different effects. So while the number of states of objects within a physical system might be quite large, the number of states that result in different system behavior are much smaller. So the number of system states of a physical system are typically less than the number of system states in software.