The following article offers a nice introduction to some design techniques that may be used to create more reliable operating systems.

Nevertheless, it is interesting to note that microkernels long discarded as unacceptable because of their lower performance compared with monolithic kernels might be making a comeback due to their potentially higher reliability, which many people now regard as more important than performance. The wheel of reincarnation has turned.

Can We Make Operating Systems Reliable and Secure? by Andrew S. Tanenbaum