When we think about the source of discourse, it can be boiled down to the clashing of models of reality. If you believe the notion that there exists a universal truth, then every being capable of modeling that truth will have developed their own unique system for modeling it. Systems that model truth are equipped with an adjustment mechanism – a necessary tool – to allow the system to update itself in light of new evidence. Discourse amongst humans is a beautiful thing. Belief systems can be hard to change because of the costs associated with changing them not to mention biases that prevent systems from properly updating when they should. Discourse is a necessary function for human models of reality to hash it out and together determine what is most likely to be representative of the truth.

This brings us to ethics. When people declare something as “right” or “wrong”, what does that mean? Where does the idea of an ethic arise? Imagine for a moment a grassy field. You’re standing on one side of this large grassy field and opposite is the base of a mountain. You’ve walked along your side of this grassy field and have seen several tracks which you assume leads to the base of the mountain. These paths obviously differ as they each snake their way through the grassy landscape. Assume for a moment that the path you take towards the mountain can be labelled as either right or wrong. Now then, how do we label each path and why? The problem here is obvious. Differing truth models can and will evaluate each path and come up with some value of whether that path is the right or wrong path to take towards the mountain. Two truth modelers will undoubtedly evaluate the same path differently. The important thing to note is that the state of the paths are independent of their labels. Evaluating a path as good or bad does not change the path itself and a path is simply just a path. Some may lead up to and climb the mountain, others might end prior to reaching the base and still others may circle the base or turn downwards into the earth to form a tunnel of some sort and end up in a time warp.

Morality is a human concept and does not exist outside of our minds. Or does it? What makes this interesting is that if we say that morality is a universal truth, then it might be possibly to formalize and harness the all-knowing truth modeler – a model which will be the oracle for truth. On the other hand, if it’s inherently human, then we might come up with a universal truth modeler by modeling all collective truth models and finding common agreement. However, the strategy of collection and aggregation pose its own problems because they remove the details of its constituents which may be important for the process – universal truth modeler – which started the aggregation in the first place. Thus, maybe there are subsets of morality that exist and cannot be coerced to a single universal model. Each model – moral scope – may exist perfectly as they are and we lose whatever essence they carry individually if we try to subsume them in some greater model. We can think of them as atomic.

Some people are trying to formalize what is good and bad – human ethics – and give them to machines to help ensure our survival when the Singularity hits. We are in a tough situation as a species, because if we cannot use mathematics – undoubtedly one of mankind’s greatest achievements and language for truth – to formalize, how can we hope to ensure our survival as a species if there are too many loopholes for our extinction? An interesting read is the paper Corrigibility which discusses ideas and issues on successfully shutting down artificial intelligence. My first thought on this is that, here we are as a species trying to create something which has the potential to be more powerful than us, but yet we want to be able to maintain and exert control over it. It’s a double blade that cuts itself when you create a power you can’t harness. We all know the mantra, power corrupts and absolute power corrupts absolutely. Now then, it seems we should be thinking about how we can create beneficial systems, but that puts us somewhat back at square one. What is beneficial?