[ivan.moony]

I don't know how much are you guys into programming, but I'll try to explain it anyway. There are two major branches of programming: imperative and functional. They differ on how they calculate information.

Imperative branch is all about utilizing side effects, meaning we store and change states in different memory locations. We assign a value to a variable, and then we possibly reassign other values to the same variable, and we do it with a bunch of variables, while changing flow of execution that says what assignment to which variable will going to be next. Some of the most popular imperative languages are Java, Python and C.

Functional paradigm is all about avoiding side effects, meaning a variable can't be reassigned once its value is assigned in its scope of assignment. It is based on function compositions, where function parameters are input and function result is output. Some pretty mean compositions can be made in functional programming, including case branching and loop implementation by recursive function calls. All the things we can do in imperative programming we can do in functional programming also. Functional programming is of interest to AI community because it is more suited for tasks such are theorem proving, out of which we can implement answering yes/no questions; who, what, where and what property questions; how, why and under which circumstances questions. Functional programming languages are not so popular outside of academic circles, while one of the most popular functional languages is Haskell.

I imagine human brain as a very complex function composition where at input end we pass parameters from our input senses, calculate pass through function composition, canonically calculating intermediate results, where at the output end we have linked our limbs. Our output can also be read by our input, forming an infinite loop, being run inside our minds from the day we are born, to the day we will die.

On the opposite side of the human brain is the Universe. The Universe could also be a machine with its input and output, if you like, where its input is linked to our limbs, and its output is linked to our eyes, ears, touch sensors and so on. So we would have two kinds of machines, one is the Universe and the other is a set of living beings being linked to the Universe.

I think this could be a solid ground for basing AGI machine as a part of the Universe being linked to the outside Universe, simulating intelligent behavior. My opinion is that functional programming is more suitable than imperative programming for building an AGI.

[infurl]

Actually, there are three major branches of programming languages. The third one is Declarative programming (like Prolog, SQL and OWL) and is by far the most important class of languages nowadays. Does that affect your theory much?

[ivan.moony]

Actually, there are three major branches of programming languages. The third one is Declarative programming (like Prolog, SQL and OWL) and is by far the most important class of languages nowadays. Does that affect your theory much?

Not much. Actually there are a lot of programming paradigms. Some paradigms apply to general domain languages (you can do everything with them), and some of them apply to domain specific languages (DSL - cover only a part of programming domain like SQL). Sometimes you can mix several paradigms into a single language.

I'm trying to promote functional programming. By Curry-Howard correspondence, logical programming and functional programming should come down to the same thing, but I find functional way of programming more natural and closer to the way humans think.

[Zero]

Yes, "functional" belongs to the "declarative" family.

Neurons can certainly be seen as functions. This is a very interesting path to AGI.

Don't forget that at output, there are not only limbs, but also several mental functionalities, like the abilities to focus willingly on something, to swap current tasks, to abandon tasks, to open a new temporary task, to learn something, to speak to oneself, to choose to interpret input in a certain way (look at this cloud, it looks like a sheep), ...etc. A lot of "mental actions" that we can choose to perform.

I'll use Racket since that's the language I'm currently swimming in.

So imagine a lot a functional neurons. Feurons. Each feuron is an object with the following properties:
- the computation function (a Racket function, like "+" or something more elaborated).
- a list of "follower" feurons, that receive this feuron output as part of their input
- a "backlink" list of followed feurons, from which this feuron takes its own inputs.

Imagine there is a big list of computation functions available, say 200 user-coded functions. This list represents a list of all possible types of feurons.

Now, I would go this way.

Start with an initial wiring schema, like Korrelan's connectome. Then go step by step.

At each step, do this:

First do your math, so you know the output of every feuron.

Then try to find similarities between results. For example, notice that feuron #2554 and feuron #9810 currently have similar results. You can also search for similarities between current results and results obtained a few steps ago.

For each found similarity, something has to be done. But it's still a little blurry in my mind for now. It's about creating new links. Perhaps Korrelan could come in and tell us what to do at this point.

Then, natural selection needs to happen. You create a lot of new feurons, arbitrarily connected to existing one (which ones?), with computation functions randomly chosen among our 200 functions list. At the same time, you need to let die feurons that are useless (which is related to the similarity thing: useless feurons seems to be completly out of the system, computing non-sense stuff).

Then you do your next step.

This is all very approximative, but I think the answer lies somewhere in that direction.

What do you think guys?


EDIT:

From the wikipedia article on Hebbian theory:

The theory is often summarized by Siegrid Löwel's phrase: "Cells that fire together, wire together." However, this summary should not be taken literally. Hebb emphasized that cell A needs to "take part in firing" cell B, and such causality can only occur if cell A fires just before, not at the same time as, cell B. This important aspect of causation in Hebb's work foreshadowed what is now known about spike-timing-dependent plasticity, which requires temporal precedence.

What needs to be found is a function that takes a cell A's output as input and that outputs the same output than another cell B. Perhaps.

Also, each type of feuron should come with its own linking machinery: specific ways to link feurons of this type to other feurons. Because it really depends on what this type of feuron eats and produces.

[ivan.moony]

Don't forget that at output, there are not only limbs, but also several mental functionalities, like the abilities to focus willingly on something, to swap current tasks, to abandon tasks, to open a new temporary task, to learn something, to speak to oneself, to choose to interpret input in a certain way (look at this cloud, it looks like a sheep), ...etc. A lot of "mental actions" that we can choose to perform.

I see mental actions such are free will, focus attention and forming knowledge as changing function compositions, i.e. changing formations that connect between input and output. At the same time, there can be a lot of input-output connections, and by passing parameters to the input, we can actually choose which path to use to perform actions, regarding to those parameters. Sometimes it is necessary to carry on some mutable data through several input-output cycles. In those cases, in each cycle we can output a (modified) data directly to the next cycle input, thus carrying on or advancing the data appearance, until the final result appears. For example, we can pass a topic of conversation through multiple hear/say cycles, and that would influence forming of our answers.

All of this seems so natural once you dive into functional programming paradigm. Anything could be done in each cycle, from making a foot step to making a step in solving an algebra expression. By predicting which way chaining different steps lead us, we can plan what combination of actions to perform to get the machine to a certain state.

[ivan.moony]

I've been experimenting with a chatbot implementation in my imaginary functional programming language. Basically, it works by looping over the same function that outputs a sentence regarding to previous input. Input also can change "topic" and "memory" variable in which it can store any variable and pick input expectation or output according to these. "topic" and "memory" are passed to each looping call to the main function.

Functional programming can't have side effects, so it can't regularly store memory variables like in imperative programming. That is what is making them "pure", as a very desirable property of functional programming languages. By avoiding side effects, the code gets more readable and more predictable. In pure languages, we can only pass parameters to functions and get their results. To simulate a variable state changes, it is possible to pass some state as a parameter, and then return a modified copy of it, which in turn we can recursively pass again to the same function, then again returning it over and over again, changed from step to step.

There are a lot of talks pointing to "monads" as a neat way of simulating state changes, but I never understood the concept very clearly. Rather, I use a common sense functional constructions for simulating states, and I don't miss anything that bothers me. Perhaps I'm constructing monads all the time without knowing it? 

[Zero]

I can't help you with monads, sorry 

There's something bothering me with functional programming. The very first thing you find in the Racket guide, is how to define stuff. Definitions are an obvious need, since programming in such a language is all about defining what things are. Well... definitions are side-effects, aren't they? When you define something, you bind an identifier (in a namespace) to a definition, which means you're modifying something outside of the "define" form. After the definition, the world is different than before. In other words, the Racket guide tells me "using set! to modify values is wrong", but definition are actually set!s in disguise, since you're modifying the initial value of an identifier (undefined) to something else. Now without definition, how would you program? I must be misguided somewhere.

Also, I had trouble figuring out how I was gonna do real stuff with Racket without the "procedural" element, until I found the thread thing. Then it made sense. Threads speaking to each others, waiting for messages, reacting, ...that's more like it!

[ivan.moony]

I don't find myself as an expert on functional programming, but I have an opinion on that subject. I think that definitions are compile time thing, before actual running the code. Once that definitions are compiled and set-up, runtime value operating machinery comes into place, with or without side effects. I think we need to differentiate compile-time step from run-time execution to wrap our heads around the subject.

[Zero]

I understand your point, but that would mean there's no room for meta-prog in functional paradigm? A program would be a static giant clockwork, that couldn't modify itself?

[ivan.moony]

Good point. I don't know, maybe it is about internal compiler and external programming stuff. When you consider a function as a whole, it internally also holds states, states that say how many parameters are applied during currying, so the compiler knows what next parameter to expect. Internally, we can expect all kinds of unpure tricks because, after all it all ends up as a state changing machine code. But externally, we can use the intent magic.

I guess it is about compiler and outer code. Someone built a compiler and boxed it, extracting all the pure calls outside the compiler, so we get the desired pure properties during a programming session.

[infurl]

Lisp-like languages (in fact many languages) implement functions as first class objects meaning that function definitions and references can be passed as arguments to other functions. Functions such as APPLY, EVAL and LAMBDA (and many others) support meta-programming in Common Lisp. You write a function that returns generated code to be evaluated. No side effects required.

[Zero]

Without side-effects, you can change what's inside eval (the code you want to evaluate), but not the eval call itself.

I imagine human brain as a very complex function composition

There's still something about time... Brain's functions are more like a flow, not like instantaneous calculus.

[Korrelan]

https://en.wikipedia.org/wiki/Liquid_state_machine

https://en.wikipedia.org/wiki/Echo_state_network

https://en.wikipedia.org/wiki/Self-organizing_map

https://en.wikipedia.org/wiki/Convolutional_neural_network

https://en.wikipedia.org/wiki/Spiking_neural_network

https://en.wikipedia.org/wiki/Recurrent_neural_network

https://en.wikipedia.org/wiki/Stochastic_neural_network

https://en.wikipedia.org/wiki/Probabilistic_neural_network

https://en.wikipedia.org/wiki/Hopfield_network

https://en.wikipedia.org/wiki/Neural_oscillation

https://en.wikipedia.org/wiki/Stone%E2%80%93Weierstrass_theorem

 

[Zero]

So, yeah. Flows.
It's abit discouraging, isn't it?

[ivan.moony]

I imagine human brain as a very complex function composition

There's still something about time... Brain's functions are more like a flow, not like instantaneous calculus.

I think time is about side effects.

[edit]
And there is the System Theory. Roughly, a system is something that has an input and an output. Also, the output can be stitched to the input, and considering some termination rule, we can iterate the system until it reaches some balanced state. Systems Theory is being taught at cybernetics university and I thought someone might find it interesting.