Slashdot Mirror
Carmack's Armadillo Aerospace Rocket Crashes and Burns
mcgrew (sm62704) writes "New Scientist is reporting that John Carmack's 'Armadillo Aerospace' has suffered a large setback in the Northrop Grumman Lunar Lander Challenge after one of its two main rockets crashed and burned. 'During the test, Texel lifted off and hovered without incident, then descended again and touched the ground. But it then rose again unexpectedly and began accelerating upward. "Crap, it's going to fly into the crane, I need to kill it," Carmack recalls thinking. He hit the manual shutdown switch, turning off the vehicle's engine in mid-flight. Texel was about 6 metres above the ground and fell like a stone. One of its fuel tanks broke open when it hit the ground, spewing fuel that ignited and engulfed the vehicle in flames. "It made a fireball that would make any Hollywood movie proud," Carmack says.' No one was hurt in the crash, but the vehicle was destroyed."
Gravity will make you it's bitch!
"It is a miracle that curiosity survives formal education." -Albert Einstein
On one hand what they were working on was completely destroyed, on the other the explosion was AWESOME!
"I use a Mac because I'm just better than you are."
...anisotropic filtering.
My blog
I think it was Doomed from the start.
This is a good thing. It proves (again and again) that new technology is never perfect. Just think, no computer program is ever completely bug free the first time it's compiled. The first car is never perfect... There are always bugs in any system. The point is that the safety mechanisms in the system worked well (after all, acording to the inputs of the lander, it was falling). As with any "accident", there are many failures that lead up to those incidents. That's the price of achievement. Nobody was hurt, so learn, build bigger and build better. If you learn from it, it wasn't a "mistake"...
If a man isn't willing to take some risk for his opinions, either his opinions are no good or he's no good
Always ensure you have enough HP to survive the landing or an invulnerability artifact when performing a rocket jump.
Unpleasantries.
It was a bad weekend for Armadillo. We set out to put some flights on Texel, the backup Quad vehicle, and it didn't go so well. We have video that we will be releasing, but Matt had to leave for Germany the next day, so it won't be digitized for a week and a half.
We started out with a normal 90 second elevated / tethered hover test, but we ran into a problem with the actuator power. We initially thought it was a bad main power switch, but it turned out to be the lithium-polymer battery pack cutoff circuit incorrectly shutting down at 16 amps of load instead of 40. This was a new battery pack ( www.batteryspace.com HPL-8059156-4S-WR), and it had passed all the individual actuator checks, but when the igniter started firing with both high amp NOS solenoids, the battery shut down (went to 0.3 volts indicated) after one second and stayed there until it was physically disconnected. Russ made a fairly heroic field repair, cutting open the battery pack and wiring around the protection circuit while sitting on top of the rocket. The total time spent on this after three attempts was 90 minutes, and enough lox had boiled off that the vehicle hit lox depletion at 60 seconds of flight. We got a few good data points from this: the batteries need to be checked at full current load, with vents open we boil off about two pounds of lox a minute, and lox-depletion runs are benign, if a little flamey.
For the second flight we were going to do a ground liftoff (still tethered for runaway protection) to test the automatic ground contact engine shutoff code. We have had several reasons to want to automate this: We get a fair bit of bounce on touchdown, because the engine is essentially keeping the vehicle weightless during the terminal descent. A computer controlled shutdown would be at least a half second faster than my manual punching of the shutdown when I visually see ground contact. The quads will just safely bounce around on the ground a bit if the engine just goes to idle and doesn't shut down, but the module, with the gimbal below the CG, will try to tip itself over when a landing leg becomes a pivot point, so there is extra incentive to get it shut off fast. You can see that in our XPC '05 vehicle flight. We also need to handle the case of the vehicle landing in a situation where I can't shut the engine off promptly, either because there was a telemetry problem, or when we are doing high altitude flights, it lands out of direct sight. There is a separate shutdownTime parameter that will keep it from sitting there at idle for ten minutes, but a telemetry abort could still have it on the ground and cooking for the better part of 220 seconds. We could still shut the flight safety fuel valve, which would result in just idle level lox pouring out of the engine, but that has its own problems.
I have been very hesitant to put in ground contact shutoff code, because shutting the engine down for some incorrect reason would be catastrophic, and I would feel awful if that ever happened. We had some switch based ground contact sensors on the old VDR, but they never got tested. We have concluded that the landing jolt, as seen by the IMU accelerometers, is a good enough ground contact signal. There is always the worry that combustion instability, or a nozzle ejection event, might trigger the signal level, so there are additional guards about it only functioning when you are within three meters of the ground (we must leave some slop for uneven terrain or GPS innacuracy) and trying to descend.
We loaded up again, being very thankful that we now pack three six-packs of helium for each test trip after we were forced to cancel the second flight on a previous test session due to insufficient helium after troubleshooting a problem forced a repressurization on the first flight. Liftoff and hover was fine, and at the 45 second mark (no sense pushing it on a ground liftoff), I had it come in for a landing. It hit the ground, and I saw it bounce back up. My first thought was "That didn't seem to help at all".
Sometimes it's best to just let stupid people be stupid.
It took me a while to get the hang of rocket jumping, too. Keep at it!
And to think, they want us all to ride in these things commercially....
Actually, this is exactly why John and company will be successful. The biggest problem with modern aerospace is "paralysis by analysis". They're so afraid of crashing anything that they have to produce (sometimes literally) millions of pages of documentation before they actually put something into the air.
Armadillo learns by *doing*, not just by creating paper studies. When they're ready to put humans in space, you can bet that their ships will have had hundreds of test flights.
Sometimes it's best to just let stupid people be stupid.
From John's post to the Amateur Rocketry list :
We have video that we will be releasing, but Matt had to leave for Germany the next day, so it won't be digitized for a week and a half.
So, it's coming, just not released yet.
XenoPhage
Technological Musings
So they're not there yet. Big deal. Armadillo's attitude to safety is that it's ok to risk the vehicle in testing, as long as people aren't at risk. They do a *very* fast development cycle, and they don't pretend to be able to find every problem through analysis -- which means some of them get found the hard way. That's a *good* thing for safety, not a bad thing. You *can't* find every problem through analysis, even if your budget is 5 orders of magnitude larger than Carmack's and you try.
Carmack's approach is to treat the vehicle as a developmental test platform, and that involves a certain level of risk to the vehicle and acceptance of that risk. The result, however, is that he learns things a *lot* faster than he otherwise might, and as a result the entire development program is faster and cheaper, counting the cost of the lost vehicles.
When Carmack shifts the vehicle from developmental status to operational testing status and then to operational status, I'd be happy to trust him when he says it's safe. It's unfair to criticize him for being unsafe now -- crashing the vehicle wasn't a safety risk!
When can we buy parts of the wreckage on ebay?
He should fly the rocket from a first person perspective.
This is my opinion. To make sure you don't steal it, it's covered by the DMCA.
Failed? I think not. Just so you're aware, Armadillo was the only team last year to even attempt the lunar lander prize, and except for some bad luck, would have walked away with it.
This year, there may be a few other challengers, but I think John and company will walk away with it. John and his team have taken this challenge in directions that the "big guys" have never tried, and it's working.
We'll see! Only 65 days left!
XenoPhage
Technological Musings
Carmack, not Romero... Carmack had nothing to do with Daikatana...
XenoPhage
Technological Musings
M-Must've hit the Tarmack pretty hard? I guess? Did I do it right?
I'm with you. I'm with you 110%. This is EXACTLY why I have long opposed private spaceflight. Long story short, profit = cut corners = death. We saw it at the composites factory, and we'll see more of it. Private interests just do not have the long term perspective necessary to take the appropriate caution to prevent deaths. This is why space colonization should always be a government function.
Would NASA cut corners like this and end up killing someone? Hell no.
Had Carmack's rocket killed someone (or many people), he would would have been stopped by "paralysis by lawsuit-ysis". Ignoring the huge dangers of rocketry by cutting corners during design may be cheaper in the short run, but as soon as real human lives are lost because of it, you can bet your ass they are going to have to spend more time and money testing their designs "on paper".
Ahh, I'm bummed out now. I was really looking forward to seeing them at the X-Prize Cup in October. They were expected to claim the prize (for the level 1 lander challenge), as they had already completed flights matching the profile on their own, and just had to repeat it a the cup for it to be official. I don't know if they'll have enough time to rebuild the craft in time for the event.
Id say so.
Had Carmack's rocket killed someone (or many people), he would would have been stopped by "paralysis by lawsuit-ysis". Ignoring the huge dangers of rocketry by cutting corners during design may be cheaper in the short run, but as soon as real human lives are lost because of it, you can bet your ass they are going to have to spend more time and money testing their designs "on paper".
The point isn't "cutting corners", the point is learning by testing and learning with actual hardware, rather than testing with paper. No one was in any danger at any point during this test. You would have a point if you could claim they were cutting corners in *safety culture*, but they're not. They're not strapping people into test vehicles. There is no human risk here at all.
Sometimes it's best to just let stupid people be stupid.
Or, put another way... within 20 years of the Wright Brothers the airplane industry had far better safety records than the space industry does after 50 years.
Engineering is the art of compromise.
When I was a kid, I remeber the coming of the 747 (I paid attention since my father was a commercial pilot). Many ppl swore up one side and down the other, that this was a NIGHTMARE in the making. They said that they would never go because it would crash all the time killing more ppl than were in my town (small town). Their were so many cowards and small thinkers. Fortunately, Boeing pushed it, built it, and now, it is the major largest craft going.
Another group thought that we had no business going to the moon and swore that LLM would simply sink into the moon. I suspect that these same ppl believe that we never went.
Just so that you know, Carmack and his rocket are real.
I prefer the "u" in honour as it seems to be missing these days.
Let's see.....
NASA death toll = 10 (3 Gemini astronauts, plus one space shuttle full)
Armadillo death toll = 0
You, sir, are a buffoon.
I have no opinion on Carmack one way or another, but tagging this story with 'haha' and 'hesnorocketscientist' seems a tad mean.
So he's a game designer dabbling in space exploration. It's not like he ran a bicycle shop or something. Now *there's* a logical starting point for a career in aeronautics!
"Only two things are infinite, the universe and human stupidity, and I'm not sure about the former."
It is more for the competition. After that, you can bet that it will be changed. Though, I have to say, I have been think about the GPS units. It strikes me that if we are going to explore the moon and mars, we should be developing cheap GPS sats. to send there.
I prefer the "u" in honour as it seems to be missing these days.
This was how soviet was testing their rockets, by trial and error. They launched a prototype and then they looked how it flew and why it blew up if it did. Saw a documentary about soviet rocket engineering and in it some nasa guys said it was one of the mayor reasons why soviet was greatly ahead of USA in rockets.
blog posting said the guy who does the videos is out for a week and a half, but they will be posting it - presumably to their website.
Why would a candidate for a mock lunar lander be designed to depend on GPS? There won't be GPS service on the moon in the foreseeable future.
John goes on about the use of GPS in the control for acceleration for a bit. Understanding that where you have no nearby reference points, such as in space, this may be a good solution. At the same time, you usually don't have anything nearby to worry about crashing into (such as the ground). Although GPS can be very accurate, it often takes more datapoints that can be obtained in a very short timeframe to get that accuracy.
I wonder if there's a reason why they aren't using some means of LASER or RADAR rangefinding when in close proximity to landing for obtaining positioning (altitude) and velocity/acceleration information. The update rate could easily be several orders of magnitude faster than GPS could ever provide...especially since you need two position reports from GPS to find velocity and three to determine acceleration.
Texel was one of two essentially identical vehicles that Armadillo put together last year for the Lunar Lander Challenge. The other is Pixel, which is the one they actually flew last year (and that had a good shot at winning) at the LLC level 1 event. Pixel is still flightworthy. This crash of Texel doesn't take them out of the LLC race, although it will lower their chances of success; it is going to make them much more cautious about banging Pixel up ahead of the next LLC competition and therefore they'll get flight less testing in.
They're also working on a set of new vehicles they call Modules, of which I gather they have one essentially complete and five in production.
With reasonable men I will reason; with humane men I will plead; but to tyrants I will give no quarter. -- William Lloyd
Shame, I was already to invest in his new company, dubbed the "Union Aerospace Corporation"
You mean even safer than a huge orange fireball?
I don't know, that's a pretty high bar.
I'm hesitant to criticize a group that is breaking so much new ground so quickly, but this sounds like some really amateurish mistakes when it comes to electrical engineering. Basically they added new sensors to detect when the craft impacts the ground. The computer monitoring the sensors was expecting a signal of a certain strength to indicate it had touched the ground, however the value the computer was expecting was higher than what the sensors could physically produce. So it sounds like they either engineered the electronics wrong making it impossible for the sensor to produce a meaningful response, or they misread the sensor datasheet which resulted in flawed software.
Now it's one thing to make an engineering mistake, but it couldn't have taken them an hour to rig up a simple test rig they that they could drop onto the ground, or tap with a mallet, or something similarly simple, to see if the computer could register a landing.
I just can't imagine strapping something new onto an entire rocket assembly, going to all the risk and expense to actually launch the thing and fly it around, hoping that all the new circuitry and software will work perfectly the first time.
It makes me wonder about the whole process NASA has in place with these contests. Even if a craft can meet various flight goals, does it result in anything of worth to NASA? For example, take a piece of software. Say there is this program that really does something impressive (game engines come to mind). So you take a look at the source, and find it is a total and complete mess. Maybe it is full of memory leaks and other bugs, so it just can perform a specific task right, but given other scenarios it crashes. Maybe the code is insecure, or is not scalable, or cannot be extended, or is not maintainable, or is not portable to other platforms. Any of those things could practically render the sources useless. But yet the program does a specific task and does it really well. For some reason I feel that NASA is going to end up with crafts with similar engineering caveats.
Dan East
Better known as 318230.
Uh, no. If you paid any attention to Carmack's progress *at* *all*, you would know that's completely false. He posts more details about his rocket than any other group out there, including crash videos (this isn't his first), detailed technical designs, photographs of the interesting guts of the engine, lessons learned, decisions taken and not taken, and far more. Let's see you build a rocket that sets new performance records in your spare time before you criticize too harshly because one of their tech crew is busy working on his real job.
The batteries themselves are fine for the most part. The trouble is, any commercial lithium pack you buy will have some VERY complicated built-in controller hardware. The controller hardware and its associated sensors (temperature, current, voltage) is generally buggy as all hell, and/or designed to perform ridiculously-conservatively by vendors who don't want to be blamed for battery fires.
What Carmack just discovered is that "fail safe" means one thing when you're designing a battery pack for a laptop, and another thing entirely when the application is a rocket control system.
In this case the controller saw a condition it didn't like, possibly a transient or spurious one, and opened the circuit at 16 amps instead of 40. Chances are, the battery itself was fine.
This evolutionary path is yet to be walked down.
You know what, this is one area where I prefer intelligent design!
(I know, I know, I have sacrificed my principles for a cheap joke..)
"A nation that forgets its past is doomed to repeat it." - Churchill
So if you die building and testing rockets for the government, that's noble -- but if you die building and testing rockets for a private company, that's ignoble.
Bullshit.
Advancing the state of the art is a noble cause no matter who pays the bills -- whether it's the taxpayers as a whole or a few millionaires who want to go on expensive vacations, working on spaceflight is a just and honorable vocation. To the extent that this research -- whatever the immediate funding source -- helps to bring down the cost of launching payloads into orbit in the long term or leads to the use of less expensive, reusable launch vehicles, the people involved in it are doing something they can legitimately decide is an activity worth risking death over.
Legislatively restricting spaceflight to governments in the name of protecting those people who may otherwise voluntarily choose to work in a field which they know has more risk than some desk job is an example of the worst sort of "mother-knows-best" nanny state bullshit governance. You can have your safe office job if you want it -- but don't you presume to speak for my interests when you lobby against letting me choose to work on something more interesting and useful to humanity as a whole than 99% of the population has any opportunity to be a part of.
Exploration for profit has a long and proud history -- what do you think brought Columbus out of Spain? The profit motive makes the work itself no less worthy of respect.
Well, the fact that he's released basically everything in the past and says he will release this video once it's ready to release counts for... everything.
Boeing mostly loses a couple of $100-plus-million satellites due to a leaking valve and we have a couple of relatively small press releases from them and the DOD about the valve issue. They all time things and restrict things up down and sideways.
John gives us good images and in many cases video, within short times of the accident, and a technical description of the failure and root causes that's first-class. Within a couple of days.
It takes a special kind of petty mind to see malign intent in a one-week delay because someone's on a trip...
And to think, they want us all to ride in these things commercially....
:P
:P
Actually if you want something that is more akin to the airline industry, a "space tourism/transport" type thing with certified pilots and strict regulations regarding maitenance schedules and such, then you should be looking at Scaled Composites and Virgin Galactic. They are the ones trying to turn this into industry.
Armadillo is more like the cowboy spaceman notion of Star Wars or Firefly, where a few skilled amateurs operate and maintain their own space craft much like a street racer would maintain their own car, cruising around space as they please. Not very surprising that cowboy space craft made by even wealthy amateurs in their spare time is a more distant dream than large corporate conglomerates creating vessels for space tourism.
Carmack's work is fantastic from that standpoint. The amount of knowledge he is creating for the amateur rocketry field is astounding. And yes, the amateur's way is going to involve more explosions than the careful, highly-financed corporate way. Of course the amateurs are careful too, knowing their way is more explodey and not wanting to die more than anyone else. This is why Armadillio haven't actually had anyone sit on one of these things while they're testing it.
Given that, I don't think he's asking anyone else to ride on it any time soon.
On another note, I have been very impressed with the safety methodologies of Scaled Composites. It's epitomized to me by the design of Space Ship One which causes it to automatically (as in aerodynamically) orient itself correctly for reentry, even if it begins reentry in the worst possible orientation. Burt Rutan -- who man years ago (X15 project?) saw a test pilot die in reentry due to incorrect orientation -- is very much into the "safety by inherent design" and I applaud that.
The enemies of Democracy are
Uh, if they gave a flying fudge about bad publicity John Carmack wouldn't have posted a blog entry detailing all the bad assumptions and decisions that caused the crash. They've already deliberately put out all the bad PR you could ask for; what more is a video going to do to them?
Why do you think Armadillo needs good PR anyway? They are not a commercial venture. And the X-Prize Cup that they are competing for doesn't consider your success/failures before the actual prize attempt either, much less your PR spin on such, they only care about success or failure in an official attempt.
The word "Translation" in your post should be translated as "Blindfolded Rectal Extraction".
The enemies of Democracy are
For instance, one expects that an upgraded OS would include all of the features in the current OS, plus some additional ones. Instead, one winds up with Vista.
XenoPhage
Technological Musings
You see, we're all volunteers at Armadillo, and therefore all have day jobs. My day job required me to come to Germany for the Leipzig Games Convention to promote things entirely unrelated to Armadillo. This is the job that actually provides a salary, so it kind of takes precedence over Armadillo sometimes.
Could I have left the video with the others so that they could capture it and get it up on the web page? Well, no one else on the team has any experience with that -- their expertise is in software design, electronics, manufacturing, welding, etc. So I'd have had to train them to do it. And again, why would I do that when, as I understood it, John's not going to post the video till the next update anyway?
But you go and believe whatever you want. Just know that we WILL post the video when I get back.
Matthew Ross
Armadillo Aerospace
"Crap, it's going to fly into the crane, I need to kill it," Carmack recalls thinking. He fired his railgun into the vehicle several times before grabbing a nearby quad damage and finishing it off with a rocket. "It made a fireball that would make any Hollywood movie proud," Carmack says.
YES!