Busy today

I was going to write a follow on post about electricity metering, but I was too busy today. I coded the GUI for the test system controller using Python/Tkinter, and was stuck for a while trying to figure out how to put a message to the user temporarily over the main display while the system is converting the acquired data to csv format, a process that can take several seconds.

The problem was that the screen doesn't refresh until the callback that starts the process is complete. Eventually I found out how to generate an 'alarm' after a specified delay which can be attached to another callback. So the first callback puts up the message and starts a separate thread that converts the data, then exits to allow the main window to refresh. Then the alarm occurs after about 1 second and the alarm callback blocks until the convert process is complete, then closes the message. Simple!

Are there any better ways? (apart from not using Tkinter)

Smart Metering

I was surprised to find that it took Google to develop a method of actually allowing the consumer to see their electicity use and payment in real time. Naively I had assumed that this is what smart metering is about when I first heard the term.

However it is not clear if they intend to fulfil the second part of my very own 'Smart Metering Vision' by providing the means for consumers to set appliances to automatically turn off (and alert the owner) when electricity rates, changing in real time, reach a 'cutoff point'.

Surely, this is the only way to allow a free market for electricity to exist. The user decides in advance how much they are willing to pay, and it is up to the utilities to try and provide sufficient power to maximise their profits in a competitive environment. To me it seems a much better solution than brownout inducing price caps, or having the utility arbitrarily switch off your heater/aircon/dryer.

I have some more thoughts on electricity metering and how to integrate with a pricing structure that promotes clean energy without selective subsidies or high prices across the board. But they will have to wait until another day, maybe tomorrow. ;-)

Problem with testing new turbine

We encountered a problem related to our new turbopump turbine. The turbine spins up a lot slower than we expected. This apparently has a ripple on effect to other parts of the test system in a way that I don't fully understand. Various flow rates and orifice sizes will have to be adjusted. I hope it doesn't delay us too much. I'll provide more updates as information comes to light.

Spilling Tests

We spent much of today out in the cold doing spilling tests. We got some odd results from some of the flowmeters, so we'll investigate those, and if it turns out that there are no serious anomalies, we'll be go for a full engine firing.

That NASA Management Video

I recently became aware of this video on NASA management problems (via zdnet and npr). It was apparently was put together by astronaut Andy Thomas and was acted out by actual NASA employees. According to Hobbyspace it's been known about for a while now.

If the issues acted out in the video are representative of NASA, it raises concerns about NASA's ability to complete large projects on time and on budget that are much more serious than just the usual problems of large organizations.

Specifically, my concern is this: suppose an organization with the management problems outlined in the video takes on a large project. This project is then divided into separate sub-projects with each sub-project corresponding to a subsystem that is specified with a list of requirements strictly based on a set of top level requirements that represent the goals of the system as a whole. The subsystems are then in turn broken down and more requirements are derived for them.

As a simple example, suppose there are 10 subsystems, each with its own project and management team. Each team is working hard to ensure that the subsystem meets its requirements, but no more than that. Because the people in the organization are dedicated and competent, 9 out of 10 projects result in a subsystem that meets the requirements, however one unlucky project team happens to get the subsystem with unknown problems, that couldn't have been predicted at the start of the project. They miss the requirements goals by about 30%. Now, because all the other teams were only trying to meet the requirements, not exceed them, and only managed to exceed them by a few percent at most, there are no easy ways to make up for the loss in performance or capability. It may be possible to ask some of the other teams to do a redesign, but the other teams, not wanting to make it seem that their initial design was flawed, all vouch for their own designs, based on the unarguable fact that their current design meets the requirements.

The only alternative is to push the problem up into the next higher level of integration, where problems with other sub-systems meeting their requirements are starting to accumulate and cause serious performance/cost/performance issues. Eventually, it is found that the entire system design is flawed as measured against the requirements, forcing either costly design changes or reducing the performance requirements to an obtainable level.

Short of deconstructing the entire organization, I can think of two possible mitigation measures that may be taken to protect against the above scenario. The first method is just to ensure that all performance requirements have generous margins. This involves giving the design team a set of performance requirements that are tighter than strictly necessary, with the option of loosening them later on, however it does somewhat corrupt the whole concept of having the requirements fully derived from the top level, and unless communication with the organization is very good, the individual teams may not have a clear idea of how critical the requirement actually is.

The second is to form 'tiger teams' that actively go out and search for bad designs, and correct them. I suppose this could be quite effective, and could also be used as a formidable 'stick' to threaten low level managers with if their team's design is suboptimal, given the ego bruising that would occur if one's own design were overhauled. However, I doubt that tiger teams are often used this way, although I could be wrong. And even if it were the case, a manager might just think it easier to weather the storm, when the tiger team members come knocking on his/her door, judging the risk of being 'caught' with a poor design preferable to a pre-emptive design change making it look like the team got it wrong from the start.

In any case this approach could have the unwanted, serious side effect of fostering a herd mentality amongst the staff, who learn never to stand out from the crowd for fear of being picked out by a bunch of domain experts hunting for a reason to justify their jobs. If done badly, a tiger team approach could be disastrous for employee morale.

Ultimately the best solution is 'organizational redundancy', by which I mean utilizing multiple organizations with differing management cultures and design methodologies, and allocating work strictly on the basis of past performance. Clark Lindsey commented in his blog entry about this video, saying "The only way I see NASA overcoming this problem is to move product development out to commercial companies and letting them compete to build the products". If this were the case, the US taxpayer could have a lot more confidence that their money is being used effectively, and the USA would have a much more vibrant and viable space program.

Unfortunately, given the political realities, an outsourced space program seems unlikely, in which case we can only hope that either the problems shown in the video are exaggerated, or that somehow NASA can reform itself. If the video is being used at management retreats, at least they have gotten to the stage of acknowledging the problem. The next stage is treatment, but I guess we won't know the results until the next big government mandated project. Personally I don't think the US taxpayer will want to wait that long.

Will Ares I and V be NASA's last launch vehicles?

Toaster from scratch

Here is an interesting project: Thomas Thwaites, who is studying at the Royal College of the Arts in London, has embarked on the quest to build a toaster from scratch, using only raw materials such as iron ore and crude oil.

Though his reasons in no part mention space colonization, his work could have some valuable lessons for those of us wondering what are the minimum necessary equipment, resources and manpower necessary for a group of people to survive and thrive in a colony mostly separate from the mass produced resources available here on earth. Even at this early stage he has learned how to smelt iron ore using only a domestic microwave oven - a handy skill for an off earth colonist.

Some people may disagree with me, but I'm guessing that the first off-earth settlers will be pretty much self sufficient initially. Any supplies delivered on an ongoing basis are likely be limited to lightweight, high value goods, such as roll/fold up solar panels, fuel cell membranes, precision mechanical components, semiconductor devices and medical supplies. Therefore, before they can head out, we will need a clear idea of the infrastructure the settlers will need to maintain their colony at a given minimum mass or value transfer rate from earth.

A lot more work needs to be done in physically building and testing the needed infrastructure for an off-earth colony, and the Toaster Project is a step in the right direction! I would like to see a lot more of this kind of thing, and to get our heads around the problem, I suggest it could be done as a kind of competitive event where several teams compete to see who can reproduce various important technologies from scratch the fastest, based on a set of resources that are themselves reproducible. Possible technologies could range from basic ones such as a smelting facility and forge, glass or glue and fabric, to more complex tech such as a pressure chamber, liquid pump, insulated electrical wire, oxygen/hydrogen production or a functioning mining drill. (I think it would make a great TV show!) Ideally, equipment provided for the competition will also be a goal technology for the competitors, to demonstrate that the colony will be perpetually self sustaining with the given equipment set.

Let me know if this idea or a variant is already in existence.

Gas Generator Tests

Here are some pictures from our recent gas generator tests. We were pretty happy with the result. The picture is a hotlink to our website which doesn't seem to have the news release in the English version.



btw, we have the entire test stand set up inside a temporary tent structure right now to protect ourselves and the engine against rain and snow over winter.

Thoughts on a universal Launch Escape System

I'm completely unqualified to provide any serious analysis of this subject but it is my hope that someone else more qualified may pick it up and run with it...

I get the impression that a Launch Escape System (LES) on a crewed launch vehicle is not considered the most fundamental consideration of safety. By that I mean that risk analyses tend to calculate the risk of loss of the entire vehicle, and then tack on the failure rate of the LES as an extra factor. I'm sure I've seen the probability of loss of crew calculated like this a number of times.

My question is " Can the reliability of the LES, given loss of vehicle be drastically improved?"

I suspect the answer is 'yes' for a number of reasons:

-Identified Vehicle failure modes as related to LES mostly seem to assume the worst possible scenario, eg an explosion at maximum velocity. But many failure modes may be quite benign from the point of view of the LES, eg drifting out of the correct flight path or loss of adequate thrust.

-Not many LESs have been designed. A quick online search that inevitably led to Wikipedia revealed only two that have been or will be used in operational systems, the Ares-I Launch Abort System, and the Apollo LES. There must surely be great scope for iterative improvements.

-As far as I know, an LES has never been designed as an individual project with the explicit goal of saving the crew regardless of how the rocket underneath behaves.

-If great advances in crash safety can be made in motor car racing, it is reasonable to hope that some of the same principles are transferrable to launch systems.

-The next generation of fighter aircraft, such as the Eurofighter Typhoon have extremely capable ejection systems that are intelligent enough to adjust their behaviour for the situation, to maximise the likelihood of survival for the pilot, eg by dropping them down more quickly from high altitude using a drogue shute.

Suppose NASA were to make development of a new 'safe' LES that could be fitted to either Falcon 9 or EELV's, or possibly other rockets a priority, the whole method of crewed launch vehicle development could change. With such a prototype LES in hand, NASA could deliberately search for unproven launch vehicles for use in an iterative test regime. This would make a great partnership between NASA and private companies, although admittedly each organization would have almost diametrically opposed goals during the test process. NASA would be happy for catastrophic failures to occur in order to test their system under realistic conditions, while the private entity offering the launch vehicle would be trying to ensure NASA doesn't get to test their LES too thoroughly! A nice problem to have would be where the unproven launch vehicles are found to be too reliable to meet the test objectives, forcing NASA to deliberately rig the launches to induce failures - something that hasn't been done previously to my knowledge.

Australia Bushfires

By now most people will have learned of the tragic events of last Saturday in Australia. Over the course of a few hours, over 300,000 hectares of land was destroyed, 10001831 houses were burned down, an estimated 200 to 300 people were killed and 5000 were left homeless. The state affected was my home state of Victoria, and many of the places destroyed we have visited in the past, including the tourist town of Marysville, which is was just a couple of hours scenic drive from our previous home in Nunawading. It used to be one of my favorite day trip destinations. Fortunately I don't have any friends or family that were directly affected, as far as I know, although I did make a call to a couple in Bendigo we know, who were ok.
It is a strange feeling being overseas while all this happened. I didn't find out until Sunday night when my mum called to give me the news. Being so disconnected from the events despite the amazingly comprehensive online coverage yields a sense of unreality to it all.
The reason for such utter destruction was an unusual confluence of circumstances. The previous week, the state of Victoria had been subject to a heatwave with temperatures soaring to 43 to 44 deg. C in Melbourne. (109.4 to 111.2F). Then on the day the fires happened, the temperature reached a never before recorded 46.4C (115.5F) along with gale force winds. In Kinglake, one of the worst hit areas, a house surrounded by 16 hectares of almost completely bare paddock burned down, killing the two occupants even though they were actively trying to save the house, with working pumps connected to a full dam. Survivors describe the locations near the bushfire fronts as raining embers.

My thoughts and prayers are with the victims and their families.

Gearing up for another engine firing test

Almost exactly the same time of year after last year's test we are gearing up for another one. This time things should work a lot more smoothly. We are much better at data acquisition, procedures, etc.

My own contribution is the 'CEMORE' (CHASE-10 Engine MOnitor-REgulator) engine controller. Last year I was using a serial interface to shift a small amount of data to the PC from the FPGA. Now I'm using the PXA272 microprocessor on the development board, to provide an Ethernet link to the PC. This is providing a data transfer rate of 256K/sec with our current hardware, compared with 1K/sec of unreliable transmission, previously. The better setup allows the CEMORE to be our basic DAQ system, with the NI DAQ system used as backup on some data channels. For our first test last year, the setup was the other way round, with the NI system as the primary.

Other enhancements include the capability to rapidly modify the 'cyclogram', the sequence of actuator controls during the firing, to allow fast turnaround during test attempts, and other improvements for increased reconfigurability.

I'm Baaaaack

After a hiatus of nearly a year, I've decided to restart this blog. I'm still here in Korea at the same place, pretty much doing the same thing, so what's new, and why did I stop posting for so long and then suddenly start over?



Well, there were a number of reasons I stopped posting. Certainly I was just too busy, what with studying part time and working and trying to have enough time for family. But as well as that, I have to admit I was discouraged after my company's first attempt at firing the new engine. There was too much time pressure and I felt that marketing commitments had taken priority over the actual engineering, which ultimately severely limited our marketing potential since we were unable to collect the data during the test that would have shown conclusively whether the test had been a success. Nevertheless the company was able to attract significant investment, just by being able to produce a good display of smoke and fire on the demonstration day, which just goes to show! (By the way, although we didn't get good data, we recorded the firing on video, and there is evidence to indicate that the firing was successful, but the mixture ratio was out, resulting in an almost clear, invisible flame - at least that is what I have been told)



Another reason was that I was unable to decide what the blog should be about. There are plenty of really good blogs that post regular news stories, or do detailed in depth analysis of space issues. I certainly have a different perspective based on my location and background but that didn't translate to new topics, so I started to feel that either I had to find new topics and a new focus for the blog, or to fully immerse myself in the nitty gritty of ongoing space related news and discussion in order to provide the quality I expected of myself.



Lastly, I found some of my own ideas in flux, and I wanted to be clear about my own opinions before commenting about someone else's.



So why start posting again and what am I going to write about anyway? Blogs such as Hobbyspace and Transterrestrial Musings offer respectively space related comprehensive online news coverage and detailed in depth analysis and opinion, and there is no point replicating that. Instead I will endeavour to offer a variety of items, such as news about my company, ideas and commentary on space related policy, bits of personal news trivia as it comes up, some news and analysis on Korea, my country of residence, some minor technical stuff, and some daft ideas that come to me from time to time. I have thought of having a different topic for each day of the week, but considering I was posting less than once a month before quitting for almost a year, this could be considered a little ambitious, maybe? Generally, I want to post stuff that might matter to someone else, expands the reader's horizons and perhaps makes their day better.



There is one other topic I want to cover, which is a large part of my reason for restarting. Lately there has been some discussion about motivations and reasons for going into space and the role of government and private industry/ventures. I want to weigh in on this discussion with my own perspective and some associated ideas on space policy, which, while being at least partially in sync with the opinions of the better known commentators, have a different focus that might just have an impact with some readers. You will find out what I mean if you keep reading and I keep posting :)



Finally, I want to maintain this blog in the spirit of a (web) log, not an authoritative text. I doubt that I will have time to furnish every claim and argument with references, although ideally I would like to. I'm likely to make mistakes and change my opinions over a period of time, or even stop posting altogether for a while if things gets too busy. That's just the way life is.