Absolutely. Break the problem down smaller and smaller and throw away any presumptions before you start. So many programmers come in with the "oh it can't be a problem with X" when approaching a problem. Well, if that's your starting point and it turns out to be X then you're gonna chase your tail for hours.
Amen to this. I went to college in the late 90s and was blown away by the people who saw a CS degree as a good way to make money without having any actual interest in the field. They spent four years bitching about not getting trained on the flavor of the week technology because they couldn't extrapolate how things they learned in language A could be applied to language B with minimal work. Those of us who actually wanted to know *how* things worked and what was underlying whatever keyword / checkbox was "critical" on a resume have done pretty well but I would be shocked if half of the people who went into it for a quick paycheck were still in the industry.
The giant tech bubble bursting about the time I graduated played a roll in sending me to grad school but it was a hell of a filter on the somewhat capable, get rich quick students.
When it comes to hiring, I don't care how old you are. In fact, I'd rather have the less frazzled older guy working 40 hours a week than the guy fresh out of school working 60 but producing 30 because he's going back and fighting his own design decisions every other day. Keep updating your skills. If that new language or framework really looks promising, great, do some side or personal project in it. Be prepared to explain why the new is or isn't better than the old. Don't blindly embrace the new and don't blindly hold onto the old just because you know it. Show me you can use your brain since that's kind of a requirement for these sorts of jobs. Too many people couldn't debug their way out of a paper bag. Try not to limit yourself based on your prejudices. I'll never love working in javascript, but I know I need to be competent. Don't try and bluff your way through talking about something you know literally nothing about. I'd rather hear "I know the name but I've never worked with it" than watch you make an ass of yourself explaining something I understand and you googled in the waiting area. If you're enthusiastic, bright, clearly know what you're doing with at least tangential skills, and demonstrate that you can pick up new things then I can overlook some mismatch of training. I mean, come on, if you're *really* good at almost any procedural language you'll have 90% of what's required in most of the rest. These are not all special and unique snowflakes. Show me you can *think*.
Ugh, sorry, had some flashbacks to days interviewing new hires.
Its those failures that will terrify lay people. When the computer does something that most people could not understand and the experts say "sure, there was a chance this could happen" bad new rules and regulations get introduced. I think we need to make sure there are fail safes wrapping the decisions of any critical NNs to try and constrain the errors, but I'm not sure that's any easier of a task.
Uh, it's simple.... No, it's not easy. But it's absolutely knowable and testable.
I agree that it's completely doable, but the poster I replied to was stating that the programmer who wrote the algorithm must understand how it's making decisions and that only the less skilled maintenance coders would be confused. That's simply not true. I know people who could write a neural net from a reasonable spec but doing the steps you described above would blow their minds. I'd also argue that a NN with even a few layers of nodes can get complex fast enough that what you're proposing would result in a document the size of a novel and still not capture all the nuances.
I really appreciate your point that
Getting any useful info out of that will be an issue though. You may find out that somewhere deep in your neural net it's looking for a seemingly random pattern of contrast or checking against some strange distance/angle.
If the net is using some seemingly random pattern that's where you can get some bizarre (to human thinking) failures. We tend to understand when something goes wrong in a way we can comprehend. If the seemingly random pattern the computer finds happens to call a slightly obscured "stop sign" a "no u-turn" sign that would be incomprehensible to a human, but might make perfect sense to the NN.
This all isn't to say that you can't reduce the odds of this sort of problem to such a small number that it's meaningless especially in comparison to human error. Still, when crap like this happens it makes the news and gets blown all out of proportion, so expect "the sky is falling" stories to follow any uncertainty AI behavior.
I completely agree that simulation and training are the solution and that the bar to beat humans at driving is pretty low. That doesn't make it any less of a nasty task to figure out WTF the neural net is actually basing decisions on or make it any more understandable to the programmer who wrote it. I'd gladly give up my vehicle for a well tested self driving car. I'd still like the option to drive sometimes, but the normal day-to-day is just a dangerous waste of time.
Based on this statement I'm guessing you've never worked with statistically based machine learning. Take a "simple" artificial neural network trained to do classification. The person who wrote the algorithm knows how samples from the training set are presented to the network, i.e. what features hit the first layer. The author also knows how data propagates through the network (i.e. a value is propagated to the next layer along the edges connected to a previous layer's node) and even how the weighting on different edges connecting the nodes are updated based on classification failures.
Once that network is trained it may spit out correct answers time and time again, but the author who knows the algorithms inside and out doesn't know exactly how the network decides that it's looking at a lunar crater and not a volcano. Not knowing those details means that it is incredibly hard to define how the trained AI will fail when faced with an unexpected input.
There's the problem: if you have a trained AI and not some sort of expert system based on a collection of human knowledge it's nearly impossible to say how it will handle the unexpected near-garbage input.
I hope the security is better than I expect it to be. Roving hotspots with mediocre credentials could make for some interesting future problems. If someone comes up with a reliable way to crack the current wireless encryption standards any time in the next 10 years some of these vehicles will still be on the road. At least with an uplink they can theoretically update the firmware, but given the examples of just about every company I've dealt with, especially companies that make "smart" anything, I'd be surprised if that happened.
True, but as a portion of my payment all those things total just under 20%. Even if taxes went up 20%, and people would be up in arms about that, it's still less than a 4% change in my monthly payment. These BC fluctuations are a whole different ball game.
It's also definitely colored my view on trusting a third-party to continue to maintain services on my behalf that are not locally-hosted on my own equipment. After all, I don't really know when they'll yank the rug out from under me.
This is incredibly important and I wish more people would kind of come to terms with this issue. I remember a while back when a javascript library was changed and broke a bunch of online applications. For a deployed project there is really no reason you should be pulling live from someone else's server/repository. Host it on your own server and periodically snap everything forward after testing that no one you rely on broke something you need. At the same time, make sure you actually DO test and move forward as things are updated, but if your app breaks because someone changed a library then the customer see the egg on your face, not theirs.
Bullshit. The company that can cure a major cancer type will make money hand over fist for years. Given society's emphasis on short term profits the executives and investors will happily walk away with billions before any patents run out. There will also always be something else to cure especially since almost every cancer is different. if you cure cancer 'a' then someone might live long enough to get cancer type 'b' and pay you again.
Pretty similar. Also similar to tantrums thrown by the other side 4 and 8 years ago and again 12 years ago on the dem side. There be drama queens everywhere. My favorite was probably anti-Obamacare threats to move to Canada. Someone didn't think that one through.
It's not just the newer models. There are a lot of older Trump voters forwarding around the same garbage. I'd love to see some sort of "that's bullshit" flagging, but no one would ever trust it from a single source. You'd need at least several competing sites. Maybe facebook could just add those flags w links to external sites about the "fact". Then there's at least a slim chance of impartial flagging?
The polls were off by a fairly normal amount. People treated a 3-4% majority off he popular vote as an absolute prediction of victory. That's garbage. Polling sucked as much as it often sucks. An error of 2ish% has no consequence if the polls show you 10 point up. If you're only in 3-4% that's a completely different story.
I'm completely with you up till the statisticians needing to get new jobs. Silver gave about a 1/3 chance of trump winning. The polls were off by only a couple points and 538 (mostly Silver) pointed out a number of times that the election was still in the normal polling error range. In fact the polls were closer here than in other recent events (brexit for one). I don't know how much clearer he could have been that this wasn't a done deal. It seems like most of the complaints people are leveling at why polling failed were things specifically called out before the election: must model states not just popular vote, states are correlated, polling errors are commonly in the couple point ballpark, etc.
A 1/3 chance is a pretty big chance especially for an outcome that supporters see as such a catastrophe. That's two rounds in a revolver for Russian roulette. Not a good bet.
Honestly I'd go to a Trump rally just to see the show but I'd never vote for the man. How many people show up isn't a perfect reflection of voting preference. I wouldn't bother going to a Clinton rally because I already know enough about her to know I don't really want to vote for her either. Luckily(?) if my anti-trump vote matters here in South Carolina he's already lost the country in a landslide.
The poster is referring to risk compensation. Multiple studies show that with the introduction of abs brakes people follow more closely. Similar things are observed with seat belts and some people argue w bike helmets. Check out the examples section of the Wikipedia article: https://en.m.wikipedia.org/wik...
Interestingly people may actually be pretty good at the risk evaluation since the fatal accidents are still reduced even w the increase in risky behaviors.
Check out my description of the FDA software approval process a couple comments up in this thread, but in our case, no the FDA does not run the software directly. You provide reams of documentation regarding tests you've run or had third parties run, documentation showing that you can trace failures back to faults, and documentation demonstrating that you've thought of the various ways the device can fail. I don't know how the regulation of a blood testing device would differ, but I bet it would be similar.
I've written code for an FDA approved class 2 device. In this case class 2 means roughly that the device is used for making medical decisions but not in direct contact w the patient. A failure could mislead a physician causing serious harm, but the device can't harm the patient directly. The code actually was the product since the approved "device" was a software program. We had to submit documentation that showed the device was safe when used by the targeted user, in our case a trained physician. We also submitted a lot of paperwork showing how the product was developed w solid tracability to all design decisions for everything from the software to the packaging to the marketing materials. Probably the most important part of those docs was a large table that had to be reviewed at every meeting showing that if the risk of a certain failure was X and the event caused by that failure had severity Y and their combination was above a threshold that we did something to mitigate the risk.
All that documentation ran for hundreds of pages and thankfully my tracability was mostly pointing at source control and saying, "you can see when i did what and why based on the timestamps and what ticket or design requirement it was connected to". Beyond that we had very large data studies showing that the product worked as we claimed. Those consisted of running previously recorded clinical data through our system and showing that with all these real-world recordings we did what we claimed. This counted as real-world since the software was designed to be run against recorded patient data.
At no point along the way did the FDA actually RUN our software or test it independently. The onus of that was on us and we had to submit extensive documentation about that testing. Audits consisted of the FDA representative coming to the company, reading reams of documentation, randomly pulling records that we claimed we had to prove we REALLY did what we said, and speaking with employees to confirm that they had some idea of what they were doing and what documentation they had to produce.
Honestly I'm not sure how else the system could run given that it takes so many incredibly specialized people to reasonably test a device. I don't really see any way the FDA could have a staff capable of doing that without becoming an absolute behemoth and slowing the approval process even more (it took years when you include the testing, verification, and documentation). There's room for improvement, but it's already no walk in the park. The FDA is pretty much making sure that if you claim a product does X that you have data to back that up and then if something goes wrong that you can trace the fault back to the source and know what happened.
Slashdot Mirror
Absolutely. Break the problem down smaller and smaller and throw away any presumptions before you start. So many programmers come in with the "oh it can't be a problem with X" when approaching a problem. Well, if that's your starting point and it turns out to be X then you're gonna chase your tail for hours.
Amen to this. I went to college in the late 90s and was blown away by the people who saw a CS degree as a good way to make money without having any actual interest in the field. They spent four years bitching about not getting trained on the flavor of the week technology because they couldn't extrapolate how things they learned in language A could be applied to language B with minimal work. Those of us who actually wanted to know *how* things worked and what was underlying whatever keyword / checkbox was "critical" on a resume have done pretty well but I would be shocked if half of the people who went into it for a quick paycheck were still in the industry.
The giant tech bubble bursting about the time I graduated played a roll in sending me to grad school but it was a hell of a filter on the somewhat capable, get rich quick students.
When it comes to hiring, I don't care how old you are. In fact, I'd rather have the less frazzled older guy working 40 hours a week than the guy fresh out of school working 60 but producing 30 because he's going back and fighting his own design decisions every other day. Keep updating your skills. If that new language or framework really looks promising, great, do some side or personal project in it. Be prepared to explain why the new is or isn't better than the old. Don't blindly embrace the new and don't blindly hold onto the old just because you know it. Show me you can use your brain since that's kind of a requirement for these sorts of jobs. Too many people couldn't debug their way out of a paper bag. Try not to limit yourself based on your prejudices. I'll never love working in javascript, but I know I need to be competent. Don't try and bluff your way through talking about something you know literally nothing about. I'd rather hear "I know the name but I've never worked with it" than watch you make an ass of yourself explaining something I understand and you googled in the waiting area. If you're enthusiastic, bright, clearly know what you're doing with at least tangential skills, and demonstrate that you can pick up new things then I can overlook some mismatch of training. I mean, come on, if you're *really* good at almost any procedural language you'll have 90% of what's required in most of the rest. These are not all special and unique snowflakes. Show me you can *think*.
Ugh, sorry, had some flashbacks to days interviewing new hires.
Its those failures that will terrify lay people. When the computer does something that most people could not understand and the experts say "sure, there was a chance this could happen" bad new rules and regulations get introduced. I think we need to make sure there are fail safes wrapping the decisions of any critical NNs to try and constrain the errors, but I'm not sure that's any easier of a task.
Uh, it's simple .... No, it's not easy. But it's absolutely knowable and testable.
I agree that it's completely doable, but the poster I replied to was stating that the programmer who wrote the algorithm must understand how it's making decisions and that only the less skilled maintenance coders would be confused. That's simply not true. I know people who could write a neural net from a reasonable spec but doing the steps you described above would blow their minds. I'd also argue that a NN with even a few layers of nodes can get complex fast enough that what you're proposing would result in a document the size of a novel and still not capture all the nuances.
I really appreciate your point that
Getting any useful info out of that will be an issue though. You may find out that somewhere deep in your neural net it's looking for a seemingly random pattern of contrast or checking against some strange distance/angle.
If the net is using some seemingly random pattern that's where you can get some bizarre (to human thinking) failures. We tend to understand when something goes wrong in a way we can comprehend. If the seemingly random pattern the computer finds happens to call a slightly obscured "stop sign" a "no u-turn" sign that would be incomprehensible to a human, but might make perfect sense to the NN.
This all isn't to say that you can't reduce the odds of this sort of problem to such a small number that it's meaningless especially in comparison to human error. Still, when crap like this happens it makes the news and gets blown all out of proportion, so expect "the sky is falling" stories to follow any uncertainty AI behavior.
I completely agree that simulation and training are the solution and that the bar to beat humans at driving is pretty low. That doesn't make it any less of a nasty task to figure out WTF the neural net is actually basing decisions on or make it any more understandable to the programmer who wrote it. I'd gladly give up my vehicle for a well tested self driving car. I'd still like the option to drive sometimes, but the normal day-to-day is just a dangerous waste of time.
Based on this statement I'm guessing you've never worked with statistically based machine learning. Take a "simple" artificial neural network trained to do classification. The person who wrote the algorithm knows how samples from the training set are presented to the network, i.e. what features hit the first layer. The author also knows how data propagates through the network (i.e. a value is propagated to the next layer along the edges connected to a previous layer's node) and even how the weighting on different edges connecting the nodes are updated based on classification failures.
Once that network is trained it may spit out correct answers time and time again, but the author who knows the algorithms inside and out doesn't know exactly how the network decides that it's looking at a lunar crater and not a volcano. Not knowing those details means that it is incredibly hard to define how the trained AI will fail when faced with an unexpected input.
There's the problem: if you have a trained AI and not some sort of expert system based on a collection of human knowledge it's nearly impossible to say how it will handle the unexpected near-garbage input.
I hope the security is better than I expect it to be. Roving hotspots with mediocre credentials could make for some interesting future problems. If someone comes up with a reliable way to crack the current wireless encryption standards any time in the next 10 years some of these vehicles will still be on the road. At least with an uplink they can theoretically update the firmware, but given the examples of just about every company I've dealt with, especially companies that make "smart" anything, I'd be surprised if that happened.
True, but as a portion of my payment all those things total just under 20%. Even if taxes went up 20%, and people would be up in arms about that, it's still less than a 4% change in my monthly payment. These BC fluctuations are a whole different ball game.
It's also definitely colored my view on trusting a third-party to continue to maintain services on my behalf that are not locally-hosted on my own equipment. After all, I don't really know when they'll yank the rug out from under me.
This is incredibly important and I wish more people would kind of come to terms with this issue. I remember a while back when a javascript library was changed and broke a bunch of online applications. For a deployed project there is really no reason you should be pulling live from someone else's server/repository. Host it on your own server and periodically snap everything forward after testing that no one you rely on broke something you need. At the same time, make sure you actually DO test and move forward as things are updated, but if your app breaks because someone changed a library then the customer see the egg on your face, not theirs.
That reasoning leads to a "but I was just following orders" excuse and that really shouldn't fly for anything significantly unethical.
Bullshit. The company that can cure a major cancer type will make money hand over fist for years. Given society's emphasis on short term profits the executives and investors will happily walk away with billions before any patents run out. There will also always be something else to cure especially since almost every cancer is different. if you cure cancer 'a' then someone might live long enough to get cancer type 'b' and pay you again.
Pretty similar. Also similar to tantrums thrown by the other side 4 and 8 years ago and again 12 years ago on the dem side. There be drama queens everywhere. My favorite was probably anti-Obamacare threats to move to Canada. Someone didn't think that one through.
Welcome to 2016, 70 year olds are not unusual on facebook. They see what their grandkids are doing as well as forward around conspiracy theories.
It's not just the newer models. There are a lot of older Trump voters forwarding around the same garbage. I'd love to see some sort of "that's bullshit" flagging, but no one would ever trust it from a single source. You'd need at least several competing sites. Maybe facebook could just add those flags w links to external sites about the "fact". Then there's at least a slim chance of impartial flagging?
The polls were off by a fairly normal amount. People treated a 3-4% majority off he popular vote as an absolute prediction of victory. That's garbage. Polling sucked as much as it often sucks. An error of 2ish% has no consequence if the polls show you 10 point up. If you're only in 3-4% that's a completely different story.
I'm completely with you up till the statisticians needing to get new jobs. Silver gave about a 1/3 chance of trump winning. The polls were off by only a couple points and 538 (mostly Silver) pointed out a number of times that the election was still in the normal polling error range. In fact the polls were closer here than in other recent events (brexit for one). I don't know how much clearer he could have been that this wasn't a done deal. It seems like most of the complaints people are leveling at why polling failed were things specifically called out before the election: must model states not just popular vote, states are correlated, polling errors are commonly in the couple point ballpark, etc.
A 1/3 chance is a pretty big chance especially for an outcome that supporters see as such a catastrophe. That's two rounds in a revolver for Russian roulette. Not a good bet.
Btw you know we actually have medical leaches and use them after surgeries like finger reattachment?
OK, I take a dump in the backseat
The land could easily be worth millions and the farmer have little to nothing else. It's not the 40 acre hick, but a few hundred acres isn't crazy.
Honestly I'd go to a Trump rally just to see the show but I'd never vote for the man. How many people show up isn't a perfect reflection of voting preference. I wouldn't bother going to a Clinton rally because I already know enough about her to know I don't really want to vote for her either. Luckily(?) if my anti-trump vote matters here in South Carolina he's already lost the country in a landslide.
Based on the assorted crap those agencies have been caught doing I'm not sure you can really say anyone controls them very well.
The poster is referring to risk compensation. Multiple studies show that with the introduction of abs brakes people follow more closely. Similar things are observed with seat belts and some people argue w bike helmets. Check out the examples section of the Wikipedia article: https://en.m.wikipedia.org/wik...
Interestingly people may actually be pretty good at the risk evaluation since the fatal accidents are still reduced even w the increase in risky behaviors.
Type 2 is the one associated with sugar consumption/obesity. Type 1 is thought to be closer to an autoimmune disorder.
Check out my description of the FDA software approval process a couple comments up in this thread, but in our case, no the FDA does not run the software directly. You provide reams of documentation regarding tests you've run or had third parties run, documentation showing that you can trace failures back to faults, and documentation demonstrating that you've thought of the various ways the device can fail. I don't know how the regulation of a blood testing device would differ, but I bet it would be similar.
I've written code for an FDA approved class 2 device. In this case class 2 means roughly that the device is used for making medical decisions but not in direct contact w the patient. A failure could mislead a physician causing serious harm, but the device can't harm the patient directly. The code actually was the product since the approved "device" was a software program. We had to submit documentation that showed the device was safe when used by the targeted user, in our case a trained physician. We also submitted a lot of paperwork showing how the product was developed w solid tracability to all design decisions for everything from the software to the packaging to the marketing materials. Probably the most important part of those docs was a large table that had to be reviewed at every meeting showing that if the risk of a certain failure was X and the event caused by that failure had severity Y and their combination was above a threshold that we did something to mitigate the risk.
All that documentation ran for hundreds of pages and thankfully my tracability was mostly pointing at source control and saying, "you can see when i did what and why based on the timestamps and what ticket or design requirement it was connected to". Beyond that we had very large data studies showing that the product worked as we claimed. Those consisted of running previously recorded clinical data through our system and showing that with all these real-world recordings we did what we claimed. This counted as real-world since the software was designed to be run against recorded patient data.
At no point along the way did the FDA actually RUN our software or test it independently. The onus of that was on us and we had to submit extensive documentation about that testing. Audits consisted of the FDA representative coming to the company, reading reams of documentation, randomly pulling records that we claimed we had to prove we REALLY did what we said, and speaking with employees to confirm that they had some idea of what they were doing and what documentation they had to produce.
Honestly I'm not sure how else the system could run given that it takes so many incredibly specialized people to reasonably test a device. I don't really see any way the FDA could have a staff capable of doing that without becoming an absolute behemoth and slowing the approval process even more (it took years when you include the testing, verification, and documentation). There's room for improvement, but it's already no walk in the park. The FDA is pretty much making sure that if you claim a product does X that you have data to back that up and then if something goes wrong that you can trace the fault back to the source and know what happened.