It is easy to use superlatives like "safest rocket ever designed". But a lot of stuff can go wrong - fast - when you're sitting on a big pile of potential explosives that under other circumstances you'd keep a safe distance from. Until they actually achieve it, you shouldn't throw the superlatives around.
I will be the first to applaud if SpaceX achieves this goal. I am certain that they want to achieve this goal. But when you look at previous launch vehicles from existing agencies that had over 100 launches, their launch failure rate vary from 1.4% (the Space Shuttle) to 14% (US Atlas) with the Russian Soyuz and European Ariane both coming in somewhere around 5%. SpaceX would have to improve on existing rocket designs by several orders of magnitude just to get to a pretty crappy safety record.
So far SpaceX has had 8 launches with 5 consecutive successes. The initial failures were clearly part of the learning process. But their current string of successes does not provide any statistical evidence that they will prove to be safer in the long run than even the worst major rocket program. Sure they plan to be safe. But safety is based on seeing what happens, not what they planned to have happen.
In this light it is worth reviewing http://science.ksc.nasa.gov/shuttle/missions/51-l/docs/roger... to see how safe people thought that the Space Shuttle would be. Initial estimates of the safety of a launch went from 1/100 to 1/100,000 with the riskier estimates coming from lower level engineers and the safer ones coming from upper management. Whenever you see numbers in the press, it is guaranteed that they represent the view from the top. We should therefore assume that they will prove to be shockingly optimistic until there is concrete data.
Here's why it's likely the manned Dragon capsule will be one of the safest manned spacecraft in history.
First, the Falcon 9 so far has a 100% success rate. The rocket engines are designed to be able to handle having a stainless steel nut pass through the fuel lines without shutting down. More so the vehicle is designed to be able to withstand an engine exploding, even at launch, without damaging the other engines and without preventing the launcher from reaching orbit. And the vehicle has already demonstrated the ability to safely abort on the pad after all the engines have been started.
Second, the launch escape system for the Dragon will be far more capable than anything that's been developed before. Instead of using a separate, heavy solid fueled tractor rocket like Apollo or Soyuz an integrated liquid fueled rocket system will be used. This will provide full escape coverage from the launch pad all the way to orbit, which no manned launcher in history has had. Additionally, the escape rocket will have a high level of redundancy.
Third, the advanced thermal protective system, simple capsule design, and robust drogue and parachute system make atmospheric reentry a lot less risky than, say, the space shuttle or even Soyuz capsules.
Fourth, before a human ever gets in a SpaceX capsule the launch escape system will already have been demonstrated on unmanned launches. By then the confidence in the system should be enormously high.
It's always possible to miscalculate or ignore risk, but SpaceX is doing a tremendous job to increase the reliability of their launcher and to make their manned capsule fundamentally safe by design.
> Here's why it's likely the manned Dragon capsule will be one of the safest manned spacecraft in history.
Lets not forget that the two shuttle accidents were as much about a human factors failure, as a systems failure. Make no mistake about it, If a company like SpaceX ever got complacent about safety, it is game over. They will not get a second chance. Remember McDonald-Douglas, and how many DC-10 accidents, that basically forced the company out of the airliner business.
The Space Shuttle was a far less reliable system from a fundamental design perspective. The size of the wings, the nature of the thermal protection system, the insulating foam and the use of a super-cryogen as fuel, the side-by-side staging, the use of solid fueled boosters on a manned launcher, etc. Engineering estimates show that we were incredibly, incredibly lucky to have the Shuttle safety record we did, the vehicle was a death-trap. We almost lost Atlantis in 1988 and Discovery in 2005 to the same thing we lost Columbia to, and we almost lost several shuttles in the early 80s to the same fate that doomed Challenger.
Of course if an organization ever gets complacent about safety then things can go downhill, but in the case of SpaceX they have a fundamentally more robust vehicle design, a system with more and safer abort scenarios, and every possible reason to maintain a high degree of vigilance about safety (if for no other reason than their company's image and financial bottom line).
Saying "we almost lost Atlantis" only confirms the point that they could have mitigated the foam lost before Columbia, and they could have fixed the O-Rings before Challenger. Not investigating those incidents properly, is a human factors failure.
I do agree that design decisions like side-by-side staging of SRBs wasn't the safest choice. Safer abort scenarios is definitely a feature of Dragon.
One problem i see with spacex is their propulsive landing feature. What if things go wrong at that point, like a software or a sensor problem causing it to descend at a speed too high/not realizing it approaches the surface/just hovering and expending the propelland quickly? If landing on land, they don't have any backup, and astronauts are doomed, falling from even very low altitude (30-50ft) will be fatal.
To some degree that's the case. The Falcon 9/Dragon do very much get a benefit from modern technology. Better machine tools, high-speed digital communications and onboard computers, better material science, the ability to run computer simulations of different flight profiles or to model the operation of a rocket engine, etc. However, in another sense the Falcon 9/Dragon represent very primitive designs. Indeed, 1950s or 1960s designs even. A simple cylindrical 2 stage LOX/Kerosene booster in combination with a simple frustrum shaped manned capsule. These designs have heritage going all the way back to project Mercury.
Yeah, and guns are vastly more reliable than rockets or missiles, too. Sometimes old and simple is the way to have high reliability, sacrificing performance.
Good design does magically appear as time goes by but is a result of a sound design and testing process. Fortunately, I am sure SpaceX is aware of this so they will most likely get a much better track record than the space shuttle.
The B-52 is still a heavily used part of the US heavy bomber fleet and current plans are to keep it in service until 2040 and perhaps beyond (which is longer than the B-1b is planned to be in service).
I wouldn't say the DC-10 problems of the 1970s forced McDonnell-Douglas out of the airliner business. They continued in business into the 1990s before being acquired by Boeing. The DC-10 itself was in passenger service until just a few years ago, and is still flown in freighter and military configurations.
The DC-10's legacy is really one of bad timing more than anything. Its direct competitor, the L-1011 suffered a very short production run for many of the same reasons, none of which involved safety.
The 1980s ushered in a new era of twin engined widebody aircraft and the phasing out of the flight engineer. These two trijets were consequently a difficult sell, although the MD-11 was competitive with the 747 briefly in the early to mid 90s.
The last DC-10 airliner was made in the 80s. The 747 is still in production. People aren't buying tickets to fly FedEx so the safety perception isn't as big an issue.
Statistically the 747 is 6x safer than the DC10, and its figures are distorted by the Canary islands collision that had nothing to do with airworthiness.
> More so the vehicle is designed to be able to withstand an engine exploding, even at launch, without damaging the other engines and without preventing the launcher from reaching orbit.
That is absolutely incredible. Gives new meaning to the phrase "over-engineered" (and not in a bad way).
Do you know off the top of your head what's the failure rate for a generic launch? How much is it saying that the Falcon is at 100% so far?
The shuttle could not pass the tests that SpaceX will demonstrate. Post-Challenger, it was substantially improved, but still had survivability gaps and questions whether the some of the abort modes were truly survivable (especially under the more severe conditions). http://en.wikipedia.org/wiki/Space_Shuttle_abort_modes#Post-...
While the SRBs were burning, the shuttle could not abort at all. This is the ascent period when the Challenger was lost.
After the SRBs detached, there were several options to abort, all of them assuming the shuttle was mostly functional. For the first four shuttle launches, the shuttle had ejection seats so the astronauts (two, anyway :-O) could escape a crippled shuttle. http://en.wikipedia.org/wiki/Space_Shuttle_abort_modes
There is a very important distinction between "launch failure" and "exploded rocket." MERELY failing to launch is safe. I am normally failing to launch while sitting in my chair. If they can lick the "exploded rocket" issue by substituting it with a mere "failure to launch" then that is great.
Ignore the launch statistics. Only one thing matters: SpaceX is hiring the top grads of the top aerospace engineering schools, and no one else even comes close. There's a Michigan professor who said every year he keeps track of his top "rock star" students to see where they end up, and the last couple of years almost all of them have gone to work at SpaceX. We're talking the top 1% of aerospace engineers of the top schools for aerospace: MIT, Caltech, Stanford, Purdue, Michigan, etc. etc. Elon is getting them all. That is why he will succeed, and you can take it to the bank.
Why does the safety of manned space flights get so much attention? Last time I checked the AIDS or malaria weren't cured. Perhaps those interested in fatality rates should divert their attention to Africa and give SpaceX more breathing space. It's not like there's going to be a shortage of astronauts.
Elon Musk: There must be some ability to experiment to advance the state of the art. In the early days of aviation there was a great deal of experimentation and a high death rate. We don't want that — the public would not be accepting — but by the same token we can't have a situation where no deaths are ever allowed, because that would put innovation in a coffin too.http://www.latimes.com/news/opinion/commentary/la-oe-0801-mo...
> It is easy to use superlatives like "safest rocket ever designed". But a lot of stuff can go wrong - fast - when you're sitting on a big pile of potential explosives that under other circumstances you'd keep a safe distance from. Until they actually achieve it, you shouldn't throw the superlatives around.
This claim revolves more around the fact that the rocket can do a powered abort at any time, I think. It's purely liquid fueled, with full ability to shut off and restart the engines. They're also designing the capsule to land on rockets, meaning it won't need to aim at a runway or water - any flat area will do.
So what you're telling me is that they have a more sophisticated system than previous ones with sophisticated backups. When it comes to reliability sophisticated can go either way - it can be better or can just give more things than can go wrong. For instance what happens on your vertical landing if a sensor malfunction causes the software to decide that the rocket is not vertical unless it is at a 20% tilt? (Don't laugh - based on the description that Elon gave in an interview, the failure of their third launch was due to a sensor that concluded they were in high in the atmosphere while they were still on the ground. Bad things happened when it tried to initiate rocket separation on the launch pad.)
Until you have actual experience with the weird stuff that goes wrong in the field, you do not really know what is going to go wrong. And the more complicated the system is, the more true this fact is.
Incidentally I had the opportunity at one point to ask the first software developer ever hired at SpaceX whether he would trust his own software. His answer was enlightening, I wouldn't want to put my life in the hands of someone who trusted his own software, so I'm going to say no.
There is always SOMETHING that can go wrong. You want the person who created your safety system to be a paranoid freak. And you have to pray that they didn't miss something. You hope and pray for the best, but until you have actual data, you don't know how it is going to turn out.
It's actually not more sophisticated, if anything it's simpler.
Here's how the Falcon 9 + manned Dragon would abort: the capsule would separate from the launcher as the escape rockets fired, meanwhile the launcher's engines would be shut down (cutting off thrust). The capsule would then coast through the atmosphere until it slowed down then deploy its parachutes when it had slowed down enough and float to the surface.
Here's one abort mode for the space Shuttle: the Shuttle continues to fly until the SRBs have burned out (there's little choice about that) and jettisoned, then the orbiter rotates 180 degrees so that it is thrusting against its direction of travel, the SSMEs continue to run until the forward momentum is cancelled and the vehicle is headed back toward the launch site, then the SSMEs are shut down and the ET jettisoned, the orbiter then glides to an unpowered landing at the KSC landing strip.
Or, how about this one: if the Shuttle's SSMEs cut out during flight then the orbiter rides along until the SRBs burn out, then they and the ET are jettisoned, the orbiter glides a bit until it slows down, then a hatch is opened and a rail is extended out of the hatch, the crew then individually bail out of the vehicle as it is traveling several hundred mph by sliding along the rail (which allows them to avoid hitting the wing) and deploy their individual parachutes, they land in the ocean and are recovered by a search-and-rescue team.
That's complicated, the Dragon/Falcon is utter simplicity.
> So what you're telling me is that they have a more sophisticated system than previous ones with sophisticated backups.
Given that the shuttle's SRBs were "once they start, they don't stop until they're empty", sometimes more complicated is a good thing.
> Until you have actual experience with the weird stuff that goes wrong in the field, you do not really know what is going to go wrong. And the more complicated the system is, the more true this fact is.
I'm fairly certain SpaceX has been employing people with actual experience.
> Incidentally I had the opportunity at one point to ask the first software developer ever hired at SpaceX whether he would trust his own software. His answer was enlightening, I wouldn't want to put my life in the hands of someone who trusted his own software, so I'm going to say no.
That's why they probably have more than one developer plus QA team.
> There is always SOMETHING that can go wrong. You want the person who created your safety system to be a paranoid freak. And you have to pray that they didn't miss something. You hope and pray for the best, but until you have actual data, you don't know how it is going to turn out.
True, certainly, but the same argument can be used on cars. Things still go wrong, but we can also safely say they're safer than a car from the 1970s.
You're wrong that the same argument can be used on cars. Because one thing that we have with cars is lots and lots of real data from the field. We don't say that cars today are safer than cars from the 70s because we have a list of safety features. We can say it because we have actual accident statistics and casualty rates.
Yet even with cars, even with the fact that they have experienced designers, when you build a new car model and put it out, you don't always get it right. You don't know what you did wrong. Which is why we regularly see safety recalls issued on cars - even from the best companies - for everything from software updates to changing the floor mat.
Until you see a model in use in the real world you simply don't know. Even if you have experience on a related model, there is the possibility of something new going wrong. Something you would have never thought of. Like the gas pedal getting stuck on a floor mat.
You take into account everything you know about. You run every feasible test that you think is useful. You run tests in simulators, on isolated components, on the whole model. You review and double review your work. You set up safety checklists to verify everything on the spot. But until you actually see the accident data, you don't really know whether you missed something important.
This is true for cars. It is true for airplanes. It is true for rockets. But the critical differences are that rockets are inherently more dangerous, it isn't feasible to do the same number of tests on them, and we don't get nearly as much accident data.
Option 1: Ride in a car with a single button labeled "press to start engines, you will not be able to stop until they run out of fuel". This car has been driven about a hundred times.
Option 2: Ride in a car with a "start engine" button and a "stop engine" button, plus a throttle. This car has been driven several times plus extensive testing on the ground.
"Option 2 is a safer setup" seems like a fair statement.
The shuttle also contains design compromises caused by congress splitting up contracting. E.g. The o-rings are there to seal connections that an ideal design wouldn't have. The pieces joined were assembled in different congressional districts.
I wouldn't want to put my life in the hands of someone who trusted his own software, so I'm going to say no.
I like that. There are a million ways you can answer a given interview question that will cost you the job, but there are also a few ways you can make me end an interview and make you an offer on the spot. Under the right circumstances, that answer would be one of those.
And you have to pray that they didn't miss something.
> (Don't laugh - based on the description that Elon gave in an interview, the failure of their third launch was due to a sensor that concluded they were in high in the atmosphere while they were still on the ground.
Source? If you're talking about F1-003, you're incorrect. If you're talking about something else, I'm interested.
And yes, I had remembered it wrong from when I watched it a few weeks ago. I remembered the bit about dropping below sea level pressure, but not the rest of the cascade.
> Until you have actual experience with the weird stuff that goes wrong in the field, you do not really know what is going to go wrong. And the more complicated the system is, the more true this fact is.
I have to say, watching the Dragon rocket abort the launch after the engines were firing and before the rocket left the pad was one of the more impressive things I've seen.
> It is easy to use superlatives like "safest rocket ever designed"
Well... Considering its predecessor, the shuttle, was one of the least safe rockets ever designed, it's not very hard to improve upon. Use multiple engines, avoid solid rockets (at least the ones that cannot be throttled down), mount your heatshield where nothing crashes on it, keep it simple...
Those comparisons make no sense and are just plain wrong.
First point: When looking at safety we care about injuries and deaths, not the launch failure rate. A broken parked car might not be reliable but it certainly is safe. Don’t confuse reliability and safety.
The two are certainly related – but not one and the same. The biggest problem with the Space Shuttle, for example, is that there is no plan B (most of the time). If you have a launch abort system you can decrease the reliability of your rocket (and thus change the failure rate for the worse) while still being just as safe.
Second point: It just makes no sense to compare manned and unmanned launch systems. Bringing Atlas and Ariane in the mix just makes no sense at all. You can’t meaningfully compare the safety between those launch systems and the Shuttle, if only because it’s impossible for humans to be injured or killed in those rockets.
On a deaths per humans brought to space metric the Shuttle is a horrible performer and very unsafe.
You are right, I was confusing reliability and safety.
That said, we currently have little reliability data on various SpaceX technologies, and no safety data at all. They are doing everything they should do to make those numbers good, but until those numbers come in we won't know what to think of them.
Sure, for a billion per flight you can more than double check even the most abstruse piece of that nightmare construction. Wings for instance, which we all know are super important for a spacecraft.
This is a pretty important observation. I will applaud SpaceX for their efforts as well, but there is a massive difference between transporting cargo into space and human beings.
SpaceX claims their transport is cheaper but they have not claimed it is safer. If they are really aiming for the human transportation business I wonder if the true cost of their rocket ships will be much more on par with existing vehicles. I'm sure they have taken advantage of some optimizations and efficiencies in new technologies and materials. But I also believe one of the reasons the other rockets are so much more expensive is not because they are relying on old technology, but having to satisfy much more stringent government regulations regarding safety, testing, and quality control. The ATLAS V has had 100% Mission Success and that does not come cheaply.
From TFA: "This is a decisive milestone in human spaceflight and sets an exciting course for the next phase of American space exploration," said SpaceX CEO and Chief Designer Elon Musk. "SpaceX, along with our partners at NASA, will continue to push the boundaries of space technology to develop the safest, most advanced crew vehicle ever flown."
So yes, he is claiming it is the safest, and I think he has good reason to, as your sibling comments explain.
'to develop the safest' is the future tense. It seems nitpicky but I believe Elon is trying to communicate that safety of the crew will be a priority.
If you read some of the Congressional debate on the commercial crew program you will see that some people question whether or not a commercial enterprise will put the safety of the crew ahead of their own profits, and even the survival of their own company. Their argument is that only a government agency with a mandate for safety can be assured of making that choice even when that results in escalating costs.
It was pointed out in counter arguments that the Challenger blew up in part because NASA launched it over the objections of the commercial provider of the solid rocket boosters.
My personal bias is that while we lost a lot of people doing barnstorming stunts in Aviation, their willingness to put their life on the line allowed things to happen faster than they might otherwise. I am not sure if I could objectively reason to that bias though. I did get a chance to talk to the folks at Scale Composites just after they did their X-prize winning flight and they felt that they were over regulated in the name of crew safety. Nobody objected to regulations to protect the safety of the people down range or nearby, it was constraints on the crew that chafed. A tech remarked that he would not be surprised if they were asked to put an ADA compliant bathroom in Spaceship One at the next review.
I'm not sure what SpaceX's "failures" were. Did the spacecraft crash and explode? Was this the Falcon 9 or the Falcon 1?
Historical safety is indicative of actual safety but not definitive, and not all you should be interested in. The shuttle was almost certainly less safe than its statistics indicate. If every launch delay of more than one day were considered a failure the shuttle's success rate would be very poor indeed.
Another question is how safety statistics are calculated. E.g. The overall safety of cars, measured by deaths per passenger mile, say, reflects many things, not just the engineering of cars themselves (in like manner, NASA's cautious use of the shuttle did much to ameliorate its excessively fraught design). On the other hand, crash safety ratings are prospective -- a five star crash rating won't save you colliding with a semi trailer, driving off a cliff, or into a lake, or having an accident at 120mph, or having your fuel tank explode.
There was an interesting discussion of actual car safety per model in the New Yorker a few years back, with the Toyota Avalon on top, and the VW Jetta second. Who drives a car and how it is driven turn out to be more important than crumple zones.
1. Exploded 25 seconds after launch, rocket parts are still washing ashore
2. Successful transition to second stage, but didn't deploy its payload properly. 1st stage has not been recovered
3. 1st stage engine was 'more efficient than expected' and continued to burn after separation of 2nd, bumping into it. I got to see a video of this at a conference, very sad. But impressive that the 1st stage was able to generate thrust (pressure in its fuel tank was rediculously low). Mostly I was relieved because my satellite was almost on that launch, and now still has a chance of going up.
I wouldn't consider a launch delay/abort a failure in any sense, especially not due to design or related to safety. Every rocket is designed to work within certain parameters, and when those aren't satisfied, the launch is aborted. This can be due to weather (completely out of human control) or malfunction (not the designer's fault). In the case of a design error, operators usually don't know what faults to look for, and then there is a failure.
You bring up an important distinction, though. There is a difference between the safety of a design, and the safety record of its implementations. In the quote in question, Musk promises it will be the "safest rocket ever designed" which doesn't mean that he's promising the safest record, or that it will have the same 100% success rate as other rockets, but that it will be designed to be safe for the astronauts. Already it has a safer design than the shuttle, since it is using liquid rather than solid fuel. That means a launch can be aborted mid-flight, and the crew brought down safely somehow. The shuttle required a man on the ground to be ready to hit a 'kill' switch.
Edit: turns out that the top comment here wasn't actually quoting anything when he put "safest rocket ever designed" in quotes. Musk said "safest, most advanced crew vehicle ever flown," which I suppose could be interpreted as a promise about its as-flown safety record. I still don't see the harm in him saying that, since he's also said that he accepts a non-zero risk of casualties. It's not like the families of dead astronauts are going to sue him over this quote.
Safety isn't as much determined by your aspirations, as it is by your record. 37% failure is an acceptable milestone considering all the SpaceX has achieved in its short lifetime, but I doubt would satisfy NASA's safety regulations for human spaceflight.
Not counting their initial failures, SpaceX has had a 100% success rate. And each rocket was so significantly different from the previous rocket[1], that it's more fair to say they had 3 rocket models with 100% failure, rather than one model with 60% failure rate. And the Falcon 9 has a 100% success rate. And they were awarded this contract, so obviously NASA believes they can achieve the desired reliability and safety for the rocket that will be used under the contract (not F1).
I'm not affiliated with SpaceX, nor trying to be a fanboi, but I believe you are distorting things.
[1] Which did earn them a lot of criticism from the industry, actually
That's not what I'm saying at all. Every shuttle was designed the same. Every F1 that failed had a different design, and the company's policies and practices changed drastically over that time as well. You can't compare it to an established program with intermittent failures.
Suspect you may be confusing the state of 'being safe', with the measurement of 'being safe'. Empirical methods are not the only way to quantify something.
Either world view relies on
"Just trust me, I know"
or
"The numbers never lie"
I'm curious to know how this compares to other new aircraft testing, since there are already plenty of commercial companies in that space. Admittedly, spacecraft is a a bit different from normal aircraft, but something going horribly wrong in either situation still means almost certain death.
It's a whole nuther ballgame for commercial airliners, for example. They undergo hundreds of test flights and all manner of subsystem tests before getting certification.
Of course rockets are single use. Most rocket designs will never fly even half the number of test program flights a commercial airliner has to undergo. The cost would be (hohoho) astronomical.
A lot of the reliability estimates of spacecraft are really high speculative, and that's a very polite euphemism. I know personally a few people at Nasa involved in human spaceflight, and they say, in unguarded moments at the end of a long day, things like 'well... what the hell does 'man rated' actually mean anyway? does anyone know?'. No one does. Obviously to say 'it will be 99.9% reliable' is very odd - who is going to pay for the several thousand tests required to make a statement like that with any confidence? So yes, take all talk of safety, especially quantitatively, with a pinch of salt.
Interestingly the Skylon (reusable) Spaceplane is down to be qualified to the same standards as a commercial airliner. That might yield some data.
Actually, the largest cost is building a whole new structure. With a fully reusable rocket (as SpaceX is going to use), the only costs are fuel and maintenance, which is more expensive than an airplane (~$50k per flight, IIRC), but negligible compared to the initial cost of designing and building the structure.
Could you imagine if airplanes were single use? There would no way Southwest could achieve a 45-minute turnaround time
And reliability numbers for man rated parts don't come out of nowhere. They come from extensive simulations and tests, which are then extrapolated. It's not the same guarantee as running thousands of end-to-end missions, but it's better than you imply.
> With a fully reusable rocket (as SpaceX is going to use)
This is the sort of thing where you need to be careful with what you infer, the the sort of thing that often causes engineers to be overconfident in performance and reliability estimates.In reality, they're doing some very initial experiments in vertical landing with a view towards exploring reusability. That is different to your implication that reusability is a done deal.
> They come from extensive simulations and tests, which are then extrapolated. It's not the same guarantee as running thousands of end-to-end missions, but it's better than you imply.
And that's the problem. Notice you're talking about man rated 'parts' and I'm very deliberately not. Many of the mission failures or anomalies in launch vehicles so far have come from parts that work fine on the bench as individual subsystems. It's the lack of full-scale, realistic tests of complete systems that cause problems. There's just not the money for it nowadays. For example, Orion's crew vehicle had budgeted 2 aeroplane parachute drop tests. Apollo's landing module had over 230. Interestingly, they recorded anomalies on over 210 of those.
As for simulations, well one of the catch-phrases in the rocket engine business is 'plumbing never leaks in simulations'.
As for extrapolation, as a datapoint related to a field I have worked in (parachutes for space systems), quite a few of the high profile parachute failures were colloquially summarised as 'they extrapolated without a license'. All the Mars landers the USA have landed so far have used disc-gap-band parachutes of the same design and size that were explored in a set of very expensive and extensive tests performed at high altitudes for the 70s Viking Lander. It's called the 'viking box' and people at JPL know you do not just 'extrapolate' out of it because they've seen what happens when smart, well intentioned engineers do. That's why they called it a box :)
Going back to simulation for a moment, I am familiar with the state of the art of parachute simulation (and fluid-structure interaction simulation in general), and so are they people in charge of the space missions, and that's why they stick to the Viking box. We can barely match that viking data in sims, let alone start wondering out of it into unexplored territory.
Finally a little anecdote from Charles 'Chuck' Lowry, the guy who designed the apollo landing systems, about testing. On Apollo 15 reentry, one of the 3 parachutes failed, the first and only recorded failure of an apollo chute during operations. It was traced back to being because the landing module thrusters had vented their fuel out before landing, but this had ignited on the still hot nozzles on the way out, causing a load of burning fuel to go fly up into the chute and destroy it. Thank god, he said, that it only caught the one and not a second one, else it could have ended very badly.
The parachute system tested perfectly, and the thrusters performed admirably during their entire qualification program and all previous flights. But the combination of these two systems, under real conditions, interacted in such that the consequences were a significant risk to life. 'You ain't tested it till you've tested it', he said.
HN is full of similar examples of outages of things like AWS due to an interaction of failures of parts, systems, and bob the technician not putting the circuit breaks back in exactly the right place after routine maintenance. The space of possible failures rises exponentially with the number of parts, when you consider all the ways they can interact. It's a hard problem to solve and the people at the top are under no illusions about the reliability numbers, they're made for congress and journalists.
It's the lack of full-scale, realistic tests of complete systems that cause problems. There's just not the money for it nowadays. For example, Orion's crew vehicle had budgeted 2 aeroplane parachute drop tests. Apollo's landing module had over 230. Interestingly, they recorded anomalies on over 210 of those.
I would argue that modern designs that learn from the mistakes of past designs require less testing. Maybe it takes 230 drops to understand the aerodynamics of a falling capsule, but once the knowledge is obtained, it only takes 2 to verify a new capsule works as good as the old one.
As for simulations, well one of the catch-phrases in the rocket engine business is 'plumbing never leaks in simulations'.
So why not simulate leaky plumbing? Computer modeling has come a very long way over time.
'You ain't tested it till you've tested it', he said.... The space of possible failures rises exponentially with the number of parts, when you consider all the ways they can interact.
True, but each new failure requires a series of events more complex than previous failures. You're using a bunch of examples of old failures to imply that new designs will fail in the same way, when the reality is that new designs have the benefit of learning from every single previous failure, and every subsequent failure further increases the reliability of the system.
I'm reminded of Asimov's essay, The Relativity of Wrong[0]. Despite your experience in the field, it seems you're too eager to assume that every new idea can be just as wrong as the previous one. Sure, new space vehicle designs like those from SpaceX may fail in ways we couldn't predict, but that is completely different from saying that they'll be less safe or less reliable than their predecessors, or that they have to fail in all the same ways as their predecessors first in order to prove their success.
First of all, what do you think you're arguing against? The argument isn't that SpaceX can't do better than existing designs. It isn't that they aren't trying. It isn't that they aren't going around this the right way. It is simply that we do not have data right now to have any real idea how well they actually will do.
Do you have anything to counteract that?
As for the rest, do you have any idea how ridiculous your position is? Based on a misinterpretation of a popular essay about basic science, you conclude that experimental data is less important now than previously. And you're doing this when arguing with an expert in parachute design who is well aware of the current limits of simulations, and several examples of what has happened when actual engineers tried to extrapolate from past designs and models to predict what would happen with a future design.
Furthermore you're doing this with willful ignorance of the fact that every area of technology where people actually achieve high reliability, it is done by people who place a lot of emphasis on actual data from experiment. Simulations are a supplement, not a replacement for that.
Finally your claim, new designs have the benefit of learning from every single previous failure is plain wrong. Anyone who studies this stuff will tell you that people keep making the same types of boneheaded mistakes over and over again. And, people being people, it is hard for us to recognize when we've made that particular type of error again. Therefore we create procedures to automatically catch errors that our organization has proven to have a tendency to make. Those procedures need to include live tests. Furthermore our expectation should be that we will continue to screw up in similar ways to what we have done before, and not that we've learned from the past and now only make more exotic errors.
All of that said, let me repeat. The people working at SpaceX absolutely know this. They seem to be on course to potentially do better than has been done in the past. But until they accumulate an accident record, we won't know how well they've done. (And at this point their designs are in sufficient flux that it will be years before we really can establish a good baseline.)
First of all, what do you think you're arguing against? The argument isn't that SpaceX can't do better than existing designs. It isn't that they aren't trying. It isn't that they aren't going around this the right way. It is simply that we do not have data right now to have any real idea how well they actually will do.
It seems we were using a different vocabulary and/or arguing along orthogonal axes. My initial impression of your and ballooney's comments was one of excessive pessimism, presumably to temper what you perceived as excessive optimism.
I'm just an interested layman trying to keep people from giving up on the idea of eventually sending people to Mars, because darn it, I really want to go ;), and I'm willing to accept "extensively simulated and unit tested with a few successful integration/flight tests" as good enough for me.
> So why not simulate leaky plumbing? Computer modeling has come a very long way over time.
You're not even close! There is no value in nebulous hand-waving statements about 'a long way'- what does that even mean? Did you not read the rest of my message which had very specific examples of how simulation isn't there yet?
And as for simulating combustion (when fuel leaks onto a hot pipe, say), academia are only just scratching the surface of simulating things like combustion instability in very toy problems, where they deliberately induce some perturbation x on a flow y and sample they system at some frequency that will just about tell them if there's a limit cycle going on. This is still so far away from actually being able to simulate a burning rocket engine properly.
Now of course I must encourage you to stop being so literal. It's not like rocket engineers say 'oh no you can't do simulation with rocket engines, because you can't model leaks properly'. That's preposterous and you're the first person I've ever come across who has inferred it so. What it means is that real actual hardware is very much not like a computer program where you can test something against all inputs and be deterministic about how it will respond because a computer is a comparatively simple, discrete thing. It's a vastly different problem to simulate a rocket engine. This saying speaks to the fact that you can't simulate every paramater of something like a rocket engine - it's just not computationally feasible, and there are plenty of people working on these problems who are familiar with the state of the art of estimation techniques too. It's hard regardless. Oh what I wouldn't give for a real world version of Haskell's QuickCheck!
Here's the thing about simulation. It's not, as you might imagine it is, a little local copy of the universe in your pc where you just arrange all the bits at t0 and say 'ok go!' and come back and see that it's worked so your design is fine. Instead you the engineer make the rules and propagate the system through your rules for a bit. If you haven't thought of a scenario, it's unlikely that your simulation will be able to show it. There are not a whole bunch of hidden states.
Now you can do universe-in-your-pc type simulation which produces very realistic looking results, but to simulate something as complicated as a rocket in flight would probably take longer than the age of the universe per second. And there are still lots of assumptions there.
> You're using a bunch of examples of old failures to imply that new designs will fail in the same way, when the reality is that new designs have the benefit of learning from every single previous failure,
This is the sweetest and most endearingly optimistic thing I've read all day. I imagine you ride into work on a unicorn. There's some validity to what you're saying, of course people say 'ok won't try doing it that way' but like with your simulation comment, I think you just don't know the reality of how these things actually work in practice.
I promise you there are still lots of Fuel Slosh Failures (an interaction of the control law and the fluid dynamics of the fuel tank causing the failure of Falcon 1 2nd flight) out there in the wild that you don't pick up till you actually fly the damn thing, despite I'm sure very thorough simulated control systems in computers on the ground by people who have a deep understanding of control theory. Static test fires never picked up the pogo effect ( http://en.wikipedia.org/wiki/Pogo_oscillation ) which blew up a few rockets and caused engine shutdowns in others. None of these are witchcraft, in that engineers perfectly understand them once they've seen them. My point is that you can't come up with them and swat them in advance in every case because there are just too many possible ways things can perniciously interact to cause you problems.
Likewise, 'every single previous failure' is not that many in rocketry, because there have not been that many rockets. It's not like rocket X blew up because it's failure mode Y was The Failure Mode for rocket X. Rocket X probably took with it to the grave several other possible failure modes, it was just failure mode Y that got there first.
The shuttle flew hundreds of time before that bit of foam broke off and put a hole in the wing's leading edge. We might never know that that particular bit of foam was a Loss of Life waiting to happen if something else had blown up the shuttle sooner. I went to the talk, one of the best talks I've seen in the whole of my career in engineering, by one of the lead investigators of the Columbia disaster. He handed around 2 identical bits of foam, about an inch diameter and 2 inches long. They looked a bit like that dense styrofoam you use in roof insulation. Anyway, these were the insulation foam on the shuttle fuel tank. He then showed us a 200,000fps video of their pneumatic cannon firing these samples at a bit of carbon carbon composite of the sort used in the leading edge of the shuttle wing. The first sample collided and then disintegrated into a cloud of dust, leaving the wing edge unharmed. The second sample collided with the wind edge and punched a huge whole straight through. The whole audience gasped. It turns out the 2nd type of foam was 'trivially' different in some small way, that no one thought would be an issue at design time, but that was the composition of the bit of the foam that broke off and put a hole in the shuttle.
Now, that whole audience was an audience of engineers, and we were all shocked. We all knew that if we were asked to simulate it, we'd say 'well, it's a homogenous foam. this kind of density. this kind of young's modulus. this kind of poisson ratio. this kind of hardness. ok that'll do' and simulated with it. But these 2 kinds of foam were basically identical in all these respects, yet their behaviour was vastly and tragically different. Unless you simulate down to the sort of molecular level, simulations just don't show you this stuff.
So I understand as an outsider [I am making an assumption that you are from your understanding, apologies if you are not] why you might think that 'surely' simulation 'should' be able to be good enough 'nowadays' what with Moore's Law and MCMC and so on. But really honestly no, not to the point you're going to catch the kind of outliers that cause problems.
P.S. I'm not arguing either way on whether or not SpaceX will be safer or not than some marker. I'm sure they'll have among the safest launch vehicles ever flown. But my points so far have been 1) Beware people putting numbers or otherwise strong claims on reliability and 2) I've been trying to kick the tyres of the mental tools and reasoning people from a software background bring to bear when trying to understand things like space hardware. There is a lot more to it.
> P.S. I'm not arguing either way on whether or not SpaceX will be safer or not than some marker. I'm sure they'll have among the safest launch vehicles ever flown. But my points so far have been 1) Beware people putting numbers or otherwise strong claims on reliability and 2) I've been trying to kick the tyres of the mental tools and reasoning people from a software background bring to bear when trying to understand things like space hardware. There is a lot more to it.
Now this I definitely agree with. I may have misunderstood your initial comment, but I was only trying to argue generally against the idea that Elon Musk is somehow out of line for saying that F9/Dragon will be the "safest, most advanced crew vehicle ever flown" because what else do you expect him to say? "We'll make it, uh, as safe as we can, I guess. Safety's a difficult concept, and rocket science is hard, y'know?"
You and I both have experience in space hardware (my guess is that you have more experience than I), so we both know how rediculous it is to put a number on things, but in this thread, we were arguing the same position from different directions. You against the sentiment of "of course it will be safe" and me against "they can't possibly know how safe it will be." Does that make sense?
> your implication that reusability is a done deal.
The plan is full reusability, except for a few parts that are lost as each stage separates from the rest, and fuel. There is no plan B. http://www.transterrestrial.com/?p=27574
And re: testing, I misspoke. By "part" I didn't mean component, I meant to say "system" but had in mind testing each part of the mission and contingency plan (in addition to end-to-end tests). I, and the engineers at SpaceX, are aware that most failures come from interactions between systems. And by "extrapolate", I didn't mean extrapolate from one part to the entire system, but from N tests to the N+1th test (because when you test as you fly and fly as you test, the actual mission is just another test).
Also, you conflate 'safety' with 'complete system success.' Apollo 15 had a safe, successful reentry, despite the single parachute failure, because there was designed-in redundancy. You can expect similar from SpaceX.
But you are right in that assigning any sort of reliability number is rediculous. On my project, we have analyzed every possible failure scenario we can think of, and come up with contingency plans on top of contingency plans, until we get to the point where so many things would have to be wrong in order for our plan to be used that it's not worth the effort, and we would think on our feet at that point. But still, nobody has bothered putting out a percentage chance of mission success. And nobody has asked the launch vehicle for a percentage chance of correctly inserting us into our desired orbit.
Actually one of the design goals for SpaceX is to develop reusable rockets. According to Elon in an ideal world, this would reduce the cost of a launch by something like a factor of 100. Elon claims that a realistic target is to reduce it by a factor of 10.
Of course as soon as you design a rocket to be reusable, the task of making sure it is still safe after a dozen or a hundred launches becomes much harder.
But I am not a fan of using as yet unproven superlatives. They will have a failure rate. They will have a fatality rate. The only question is what that rate will be. And right now we have no useful data on that.
I will be the first to applaud if SpaceX achieves this goal. I am certain that they want to achieve this goal. But when you look at previous launch vehicles from existing agencies that had over 100 launches, their launch failure rate vary from 1.4% (the Space Shuttle) to 14% (US Atlas) with the Russian Soyuz and European Ariane both coming in somewhere around 5%. SpaceX would have to improve on existing rocket designs by several orders of magnitude just to get to a pretty crappy safety record.
So far SpaceX has had 8 launches with 5 consecutive successes. The initial failures were clearly part of the learning process. But their current string of successes does not provide any statistical evidence that they will prove to be safer in the long run than even the worst major rocket program. Sure they plan to be safe. But safety is based on seeing what happens, not what they planned to have happen.
In this light it is worth reviewing http://science.ksc.nasa.gov/shuttle/missions/51-l/docs/roger... to see how safe people thought that the Space Shuttle would be. Initial estimates of the safety of a launch went from 1/100 to 1/100,000 with the riskier estimates coming from lower level engineers and the safer ones coming from upper management. Whenever you see numbers in the press, it is guaranteed that they represent the view from the top. We should therefore assume that they will prove to be shockingly optimistic until there is concrete data.