Slashdot Mirror
Hacker Exposes Evidence of Widespread Grade Tampering In India
Okian Warrior writes "Hackaday has a fascinating story about Indian college student Debarghya Das: 'The ISC national examination, taken by 65,000 12th graders in India, is vitally important for each student's future: a few points determines which university will accept you and which will reject you. One of [Debraghya]'s friends asked if it was possible to see ISC grades before they were posted. [Debraghya] was able to download the exam records of nearly every student that took the test. Looking at the data, he also found evidence these grades were changed on a massive scale."
Sometimes you have to do the needful to get into the school you want.
This would be true in the US and the UK, and India doesn't even match up to those "high" standards. He'll be in jail because someone with power will be embarrassed by this.
More for the discussion of statistics than for the really sad excuse for security on those pages..
He tried to kill me with a forklift!
this is the type of coding that you get in India stuff done on the cheap and likely to coded to spec with no thinking about how bad of a idea this is.
The test results were manipulated. There are missing scores (from 1-100) on a test taken by 150,000 students. That is not possible. They have been bumped up to passing. The graphs show jagged peaks separated by gaps rather than a curve. Unless his data is incomplete or has been manipulate, there is no reasonable explanation for the jagged charts.
Nothing I hear about education fraud in India surprises me since one of my Indian coworkers explained how people "buy" degrees from Indian universities.
University employees can be bribed to create the records for an entire curriculum, spanning multiple years of attendance. This record is indistinguishable from a valid one and generates a real diploma. The University will confirm education because "it's in the system".
I think he said it cost about $3000 USD or so for a Masters degree.
It definitely does not represent standardization to a score of 100. It's not an even distribution of peaks. It is pushed up above the failing mark, and there is no gap from 94-100. Furthermore, all the different tests in different subjects show the same gaps. This is not reasonable at all.
What does "ls -l" do? Please describe below.
That kind of thing. So, I'm not surprised if institutions are manipulating test scores. India is more about the perception of computer savvy developers than the reality of it.
No sigs in BETA. Beta SUCKS.
The Indian system of education doesn't work like that. Here's a post I made on another forum: You can theoretically attain all marks in the 0-100 range because there is no scaling up. Each paper has components that together total upto a 100. For example, there could be 10 1-mark questions, 15 2-mark questions, 4 3-mark questions, 3 4-mark questions and 6 6-mark questions. Each question can be graded to a fraction of it's worth. So you can get 1.5 on a 2-mark question, 0.5 on a 3-mark question, etc. Thus theoretically, all possible combinations of scores are possible. The absence of certain scores is evidence of tampering. SOURCE: I appeared for the CBSE exams last year. The system is similar, though not the same.
There is nothing in the article that indicates caste has anything to do with it. Most of the discussion suggested that the cause may have been to "bump" almost-passing grades to passing grades (and presumably other achievement tiers as well).
"Here Lies Philip J. Fry, named for his uncle, to carry on his spirit"
Back in late 2009 and early 2010 I was scraping jail inmate registry records for Scott and Dakota County, MN. This was simply a script which incremented the ID numbers by one several times a day and put them out into a CSV. I uploaded these to Google Docs and had Docs Widgets build simple charts based on those data for a rolling ~6 month window of inmates.
As I started looking deeper into the data I started noticing I had ages lower than 18. Odd I thought but sure enough, Scott County was including their juvenile records in the data mixed with the adults even though it wasn't shown on their public website.
I contacted the County and they fixed the bug (you can read about that here: http://www.lazylightning.org/scott-county-quickly-fixes-juvenile-jail-roster-issue) but I was still surprised at the relative lack of security for juvenile records:
It's surprising how lax security is anywhere and to the poster elsewhere in this thread that said this is what you get when you outsource to India, this particular web stuff was not performed with outsourced talent so that comment was nothing short of asinine.
1. Teachers have to ensure that their class marks have a certain average and median before they submit them. There can't be too many failures either.
2. Teachers know not to give a grade of 49 if the pass is 50 since the student will argue to get that missing point. If you want to be safer, just don't give out anything in the forties.
3. If a test gives letter grades, that equates to a particular number. A = 85, A- = 83, and so on. In that case, no one gets an 84, ever.
"We are here on Earth to fart around. Don't let anybody tell you any different!" -- Kurt Vonnegut
Are you trying to mock educational standards by pretending to be someone who failed statistics?
Poisson distributions have to do with frequency of repeatable events over time. You meant Gaussian or Normal distribution.
You should read the original http://deedy.quora.com/Hacking-into-the-Indian-Education-System. The missing scores are in extremely suspicious positions. For example, there are no scores of 32,33 and 34, and the minimum pass grade if 35. That looks pretty close to a bump to get people to pass. This doesn't look like someone not understanding the grading system. It looks like manipulation. Frankly, speaking as someone who does a fair bit of grading, yes one can get weird distributions from legitimate adjustments, but they don't look like this.
Technically, in a caste system, you're not allowed to move up except in very narrow circumstances. You're not actually allowed to move at all - up or down. You can be the most brilliant person on the planet, but if you were born to an untouchable in India, well, no one would listen to you.
More likely though, it would be done by people from higher castes because they have a certain image to maintain.
Remember, in Asia, this all derived from the old school British system where exams basically set you on your path through life - basically the final exams at the end of high school was The Final Exam(tm). Score well, and you'd go to university. Score not-so-well, you got to a second-rate college. Score less and you're a lowly tradesperson. Score even worse and you're an unskilled labourer.
So in general, it's an extremely high-stress period where teens would basically be locked in their rooms spending all the time studying because it really is it - no chance to take it over (well, I suppose there are certain humanitarian reasons they allow), and it basically determines your future.
Likewise, for anything with this much pressure on it, people succumb to the human condition - suicide is common, both before and after the exame. Cheating is as well - and many elaborate cheating machines have been conjured up over the years - this isn't your own hide-a-cheat-sheet scale - this is full on tiny 2-way radios and other mechanisms. And of course, hacking of grades to improve one's score.
Interestingly, I think in China one district is forcing all test-takers through a very sensitive metal detector and forcing them to strip - just one step below forcing test-takers to be stark naked during testing. The metal detector is extremely sensitive and basically won't allow anything metal in.
That's how serious the test is, and how serious everyone takes it.
For all its flaws, the modern American system is generally better and more "available" (and even the modern British education system isn't as strict). I'm not entirely sure that letting one test determine your future is entirely wise, and it's one reason why a lot of students travel abroad to study. Some do it because they scored well and got prestigious international study scholarships from their country, but others do it because they couldn't get in, and studying abroad is an option for those that do not pass.
I've read the original. Whether your policy can produce that sort of distribution depends entirely on what the policy is, no?
As an example of a system that produces exactly this sort of pattern, at my university the pass grade was 50%. Anyone who scored at least 45% but less than 50% in the exam could apply to sit a supplementary exam a few weeks later. The supplementary exam score would then be your final score, but the maximum mark available in the supplementary exam is 50%. If this results in you scoring 50% then the subject is recorded as a "conceded pass". You can only take one conceded pass in a year and many degree programs also limit how many conceded passes you can count towards your degree.
It's a system that lets you have another go if you had a bad day in an exam and, yes, in many subjects it produces this pattern of no-one receiving 45, 46, 47, 48 or 49 and a big lump of people receiving 50.
Slashdot - News for Nerds, Stuff that Matters, in ISO-8859-1 Has just realised that beta makes this signature redundant
Why don't you just read the fucking article instead of trying to come up with your own wackjob explanation? He quite clearly explains it:
One of the most common critiques of my theory was this - maybe there were questions with only 3 or 4 mark intervals in all subjects making certain marks mathematically unattainable. My counterargument? All numbers from 94 to 100 are attainable and have been attained. What does this mean? It means that increments of 1 to 6 are attainable. By extension, all numbers from 0 to 100 are achievable.
Let me give you an example. If 99 and 98 were definitely achievable with deductions of 1 and 2 respectively, this means one of two cases - there is a question A worth 1 mark that made 99 occur, and a question B worth 2 maks that made 98 occur, which meant getting A and B both wrong would mean 97 could occur. Case 2 - Question A was worth 1 mark, and question B was worth 1 mark too. The 99 got A wrong, and the 98 got A and B wrong. By this logic, if 97 were not possible, it would mean that there is no other question of 1 mark in the examination or that nobody got a 2 point question wrong and question A or B.
Basically, because 99, 98 and 97 were all attained, then any increment of 1, 2 or 3 points should be possible. The fact that nobody got 80% in any subject in the entire country points to widespread tampering.
Help I am stuck in a signature factory!
The criticism seems rather pedantic. I'm the last one to defend the barely-reading, never-correcting, link-to-blog-post-instead-of-actual-article, duplicate-posting slasheditors, but the fact is:
1) the server has a place where you put in a code and, i'd guess, a passcode. He looked at the code, determined the data was being drawn by a simple java query to an unsecured text file. Did he get the data the way it was intended? CERTAINLY not. Did he essentially 'break in' through what was relatively tissue-thin (derived from obscurity only, really) security? Yes, I'd say he did. So yes, in MOST people's definitions, he 'hacked' their shitty website.
2) WTF are you talking about? Every school system in existence ADJUSTS grades on standardized tests? Proof? The guy discovered that something like of the passing scores (everything > 35), like 40% of the possible scores NEVER showed up. Ie, nobody *ever* got a score of 82, 84, 91, or 93, while 94-100 was regularly distributed. Mathematical anomaly? Maybe. But that seems unlikely with a massive test, and multiple added scores that this is possible.
I think what he discovered was a ridiculously insecure web service, and a list of grade scores that have suspiciously regular omissions.
So "hacked" and "possible grade tampering" seems pretty spot on.
-Styopa
The examples in parent post are wrong.
"Breaking and entering" requires physical trespass. There is no trespass involved when using the GET method, which is part of a standard and open protocol, to request a web page, which in this case is unencrypted and easily read by anyone who asks for it.
The "bait car" analogy fails miserably. There is no property theft involved in what was described by TFA since nobody was deprived of use of anything. In the general case, "intellectual property" is not physical property and courts need to recognize the differences.
If anyone needs a physical analog of what this fellow has done, it is like this:
Imagine that for reasons unknown, the New York City Board of Education recorded the student ids and test scores as graffiti on all the park benches in Central Park. Where any passer-by could read them. Each student was directed to the bench where their data was recorded (in indelible magic marker), and the BoE patted itself on the back for having found a way to make use of all those benches. Then this guy comes along and develops an efficient way to go from bench to bench to bench... Data on the Internet, accessible without any protection to anyone who had or could construct the URL, is as freely available as any graffiti written on a park bench.
Questions should begin with why the India agency responsible for handling this data put up these web pages without involving anyone who had a year or more of training in information management techniques. They certainly had persons on staff who would have avoided making the JavaScript so readily accessible, and there should have been some kind of password scheme so that only the student would be able to access his own scores. Why were their in house experts not involved? It is as if those who were delegated to build the web site did not want to involve anyone who knew enough about data management that they would become suspicious about it being manipulated.
I think there is more than enough evidence here that something is very corrupt in the India education system. Even if the data obtained had not been so obviously altered, the grossly amateur handling of highly personal information stinks to high heaven.
Will
Kinda like yours, except that you likely know even less about the test than he does.
If this had happened in the usa
Something very similar to this did happen in the USA, from some time in the 1980s until around 1995. It involved a government forestry agency, and the database they had to track logging, replanting, spraying, road building, and other commercial forest management activities.
I became involved about 1993 when I was hired by an eco-activist group who had used FOIA to obtain a digital copy of a detail report of the entire forestry database for the region. My task was to develop one-off perl scripts to extract the data from the report format and build a Paradox database that could be queried to see if the forestry records indicated any violations of the laws to protect spotted owl habitat. This was straightforward work: as I recall the hardest part was staying awake when doing the validation cross-checking. (I also dislike reconciling my checking account with the bank statement.)
But what I discovered was that the forestry database was full of crap. You cannot harvest a 20 year old stand of timber from a parcel that had been clear cut just three years earlier; you cannot harvest anything from a parcel before the access road to it is completed. A big portion of the database lacked self-consistency. Years later, I learned that the consultant that the forestry agency had hired to develop and maintain the database had been convicted of fraud, and that there had been a shake-up in the management of that agency. (Since the database records were crap, the eco-activists chose not use it in their spotted owl fight. Instead a new, and appropriate, attack on the managerial competency of the forestry agency was launched, I believe by persuading one of the State Representatives to demand an investigation.)
I do not think that computer fraud on this scale is likely to happen in the USA now, because I think every manager of any kind of any large government database is well aware that he needs to cover his ass by having his stuff validated by Information Management. However the news indicates this kind of fraud is happening in some small towns, and some of the smaller departments of cities-- places where there is still no easy access to information management professionals, where decisions involving database management have to be made by persons without a background in the subject.
Will
Correct; typical example could come from counting, then plotting, discrete data. Number of children in a family, doors on a car...
Note that whilst you might expect a normal distribution, with events (exam results) distributed evenly but randomly about the mean, the fact the the guy found something that certainly looks non-normal, (he did not do normality tests, but having looked at his results, I don't think he needed to), does not itself prove that the results were altered.
Imagine a 'perfect' exam, where the expected (average) result for the student population was 50 out of 100, or 50%
Now imagine an (equally unlikely) 'perfect' candidate population.
If you plotted the exam results, you could expect the population to be centered on a mean result of 50, with half the scores higher, half lower.
If you had a (really getting unlikely now) 'perfect' education system, there would be a low standard deviation in your data, let's say 2%
If the results could be modelled with the Gauss curve, then 99.73% of your distribution would be at +/- 3 sigma (standard deviations) from the mean.
So lowest expected score of 50-2*3=46, with highest of 56.
Of course, candidate abilities could be much more varied than this, so sigma could be anything...5%, 10%
Anyway, getting to the point, if the mean of a what you *might* be expecting to be a Gauss / Normal curve is shifted sufficiently towards a 'hard' limit, (in our example, you cannot score less than 0%, or more than 100%, so both are 'hard' limits, or 'boundaries'), then the data (example results) do tend naturally to 'pile up' against the limit. (Think of a snow plough pushing snow aganist a wall - it's go nowhere to go, except up).
Thus you get a non-normal distribution, (typically better modelled with a lognormal or Weibull curve, not Poisson).
But WHAT can cause the mean to shift? For this example:
- Either the exam is "too easy", or
- The students are all very good (yeah, same thing,really), or
- The marking system is biased.
I'll leave you to draw your own conclusions on that one, but I've personally found that in India, (as in other places, including the USA), a little cash can go a long way...
But that was not the most compelling evidence of bias; that would be the very strange 'missing' data points, (especially close to critical scores such as the 35 pass. /endoldstatsbore
Possibilities:
- There is a national cheating conspiracy ...or....
- The test score is not based on assigning a value to each question and adding up those values.
For example, the test could simply be scored as such:
All answers correct: Score 100
Miss one question: Score 99
Miss two questions: 98
Three questions: 97
Four: 96
Five: 94
Six: 92
etc etc
Miss 20 questions: 35
Miss 21 questions: 31
etc etc.
The author makes the ASSUMPTION that the score of the test must be the sum of the value of the questions answered correctly. There is no basis for that assumption. The fact that certain values are not present, and the values 34, 33 and 32 are not present, are likely by design (i.e. don't make people feel like they just missed passing.)
All the author has shown is that India is apparently doing a very poor job teaching critical thinking skills (as evidenced by the author's inability to exercise critical thinking skills.)
paintball