Neural networks and Markov: a new potential, with a single problem. in AI Programming

I have a new idea, where Markov chains build sentences, for which the connections are chosen by neural networks. The max number of normalizable connections for each Markov node is 128. The problem here is how to find out how to form the reply, where the input is the sentence that came *before*. To retrieve it, I need a neural network that gets the next node from the current reply node, AND the input sequence. Seq2seq networks are not an option.

My possible solution would be to convert a phrase into a number using of an encoder neural network, e.g. with the phrase "Hello, my dear son!" we iterate from inputs [''Hello", 0] → A, where A is the output of the neural network. Then we do it again, but with the word "my", so that ["my", A] → A + B. And so on, until we convert that phrase to A + B + C + D — where the plus sign isn't a sum, but some sort of joining, that goes on inside the neural network.

That number is then passed into a decoder neural network, such that [0, A + B + C + D] → [N₁, A + B + C + D], and [N₁, A + B + C + D] → [N₂, A + B + C + D], ..., [0, A + B + C + D]. Nₙ is denormalized into the word that corresponds to the nth node that follows the node Nₙ₋₁

What about you? Any better solutions or suggestions? :)

6 Comments | Started December 04, 2017, 04:21:16 pm


Listing States / Processes / EventActs in AI Programming

Hi guys,

Could you please help me make a complete list of every thing that can exist, or occur, in a mind. Every item should fall in one of these 3 categories:
- state
- process
- event/act

EDIT: The list currently looks like this.

   type of thoughts
      logical argument
      logical assertion
      mental image
         perceptual component
      thought experiment
   content of thoughts
         bayesian network
            cause consequence chain
            know how to
      goal-to-current path
         state event state
         path criteria
   thought frames
      active frame
      suspended frame
   thought frame handling
      pattern recognition
      pattern label creation
      unexpected event noticing
      abductive reasoning
      analogical reasoning
      deductive reasoning
      inductive reasoning
      moral reasoning
      probabilistic reasoning
      decision making
         goal-to-current pathfinding
         path comparing
      problem solving
      context targeting
      choosing role set
      role dispatching
      scene space creation
      scene element evocation
      scene evolution
      open thought frame
      switch thought frame
      close thought frame
      find goal-to-current path
      link new percept to set of percepts

It's far from complete!

14 Comments | Started November 29, 2017, 09:53:03 am


Ultra Hal 7 - News Update in UltraHal

After passing through Alpha testing it has recently gone into Beta and shouldn't be as such much longer before going to RC (Release Candidate), then Final.
Although I am not privy to a public release date, I would have to think it to be within a relatively short time frame.

Most testing has gone quite well and the new Hal 7 will have a lot of really nice and productive features.


1 Comment | Started December 11, 2017, 02:48:22 pm


What's everyone up to ? in General Chat

Been a bit quiet lately, just wondering what people are up to at the moment...  Working on an exciting project ?  Sunbathing ?  On your holidays ?

I guess as usual for this time of year we are out and about in Real Life more often.  I've had a few nice days out already this summer and looking forward to a few more.  I've been working on my photography skills, mostly plants and nature.  I'm thinking of making myself a personal website and turning some of them into wallpapers and things.

Anyways, keep in touch :)

917 Comments | Started July 13, 2009, 02:53:30 pm


Reading a neural network’s mind in Robotics News

Reading a neural network’s mind
11 December 2017, 4:59 am

Neural networks, which learn to perform computational tasks by analyzing huge sets of training data, have been responsible for the most impressive recent advances in artificial intelligence, including speech-recognition and automatic-translation systems.

During training, however, a neural net continually adjusts its internal settings in ways that even its creators can’t interpret. Much recent work in computer science has focused on clever techniques for determining just how neural nets do what they do.

In several recent papers, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Qatar Computing Research Institute have used a recently developed interpretive technique, which had been applied in other areas, to analyze neural networks trained to do machine translation and speech recognition.

They find empirical support for some common intuitions about how the networks probably work. For example, the systems seem to concentrate on lower-level tasks, such as sound recognition or part-of-speech recognition, before moving on to higher-level tasks, such as transcription or semantic interpretation.

But the researchers also find a surprising omission in the type of data the translation network considers, and they show that correcting that omission improves the network’s performance. The improvement is modest, but it points toward the possibility that analysis of neural networks could help improve the accuracy of artificial intelligence systems.

“In machine translation, historically, there was sort of a pyramid with different layers,” says Jim Glass, a CSAIL senior research scientist who worked on the project with Yonatan Belinkov, an MIT graduate student in electrical engineering and computer science. “At the lowest level there was the word, the surface forms, and the top of the pyramid was some kind of interlingual representation, and you’d have different layers where you were doing syntax, semantics. This was a very abstract notion, but the idea was the higher up you went in the pyramid, the easier it would be to translate to a new language, and then you’d go down again. So part of what Yonatan is doing is trying to figure out what aspects of this notion are being encoded in the network.”

The work on machine translation was presented recently in two papers at the International Joint Conference on Natural Language Processing. On one, Belinkov is first author, and Glass is senior author, and on the other, Belinkov is a co-author. On both, they’re joined by researchers from the Qatar Computing Research Institute (QCRI), including Lluís Màrquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and Stephan Vogel. Belinkov and Glass are sole authors on the paper analyzing speech recognition systems, which Belinkov presented at the Neural Information Processing Symposium last week.

Leveling down

Neural nets are so named because they roughly approximate the structure of the human brain. Typically, they’re arranged into layers, and each layer consists of many simple processing units — nodes — each of which is connected to several nodes in the layers above and below. Data are fed into the lowest layer, whose nodes process it and pass it to the next layer. The connections between layers have different “weights,” which determine how much the output of any one node figures into the calculation performed by the next.

During training, the weights between nodes are constantly readjusted. After the network is trained, its creators can determine the weights of all the connections, but with thousands or even millions of nodes, and even more connections between them, deducing what algorithm those weights encode is nigh impossible.

The MIT and QCRI researchers’ technique consists of taking a trained network and using the output of each of its layers, in response to individual training examples, to train another neural network to perform a particular task. This enables them to determine what task each layer is optimized for.

In the case of the speech recognition network, Belinkov and Glass used individual layers’ outputs to train a system to identify “phones,” distinct phonetic units particular to a spoken language. The “t” sounds in the words “tea,” “tree,” and “but,” for instance, might be classified as separate phones, but a speech recognition system has to transcribe all of them using the letter “t.” And indeed, Belinkov and Glass found that lower levels of the network were better at recognizing phones than higher levels, where, presumably, the distinction is less important.

Similarly, in an earlier paper, presented last summer at the Annual Meeting of the Association for Computational Linguistics, Glass, Belinkov, and their QCRI colleagues showed that the lower levels of a machine-translation network were particularly good at recognizing parts of speech and morphology — features such as tense, number, and conjugation.

Making meaning

But in the new paper, they show that higher levels of the network are better at something called semantic tagging. As Belinkov explains, a part-of-speech tagger will recognize that “herself” is a pronoun, but the meaning of that pronoun — its semantic sense — is very different in the sentences “she bought the book herself” and “she herself bought the book.” A semantic tagger would assign different tags to those two instances of “herself,” just as a machine translation system might find different translations for them in a given target language.

The best-performing machine-translation networks use so-called encoding-decoding models, so the MIT and QCRI researchers’ network uses it as well. In such systems, the input, in the source language, passes through several layers of the network — known as the encoder — to produce a vector, a string of numbers that somehow represent the semantic content of the input. That vector passes through several more layers of the network — the decoder — to yield a translation in the target language.

Although the encoder and decoder are trained together, they can be thought of as separate networks. The researchers discovered that, curiously, the lower layers of the encoder are good at distinguishing morphology, but the higher layers of the decoder are not. So Belinkov and the QCRI researchers retrained the network, scoring its performance according to not only accuracy of translation but also analysis of morphology in the target language. In essence, they forced the decoder to get better at distinguishing morphology.

Using this technique, they retrained the network to translate English into German and found that its accuracy increased by 3 percent. That’s not an overwhelming improvement, but it’s an indication that looking under the hood of neural networks could be more than an academic exercise.

Source: MIT News - CSAIL - Robotics - Computer Science and Artificial Intelligence Laboratory (CSAIL) - Robots - Artificial intelligence

Reprinted with permission of MIT News : MIT News homepage

Use the link at the top of the story to get to the original article.

Started December 11, 2017, 12:03:40 pm


Inner self dashboard in AI Programming

Imagine a program that would be able to reproduce each function of a human brain. We would still need an execution model for it to work as a whole.

I propose an "inner self" dashboard, like the one in the picture, as central point of the system.

Instead of trying to create an execution model, we expose the program's states, processes and events through a virtual dashboard.

Making it visual helps figuring things out.

On the top, we have the Activity tabs. We need it to be able to switch activities, suspend them, go back to them, ...etc.

Right in the middle, there's the Main work zone. This is what contains the mental stuff we're working on right now, whatever it is.

On the left side, a Navigation panel lets us choose what we see in the Main work zone. The navigation panel only shows relevant/related stuff.

On the right side, there's a Tool box containing everything we need to act upon what's in the Main work zone. Again, it only shows relevant tools.

When interesting events occur, they pop up in the Notifications zone, below the Tool box.

Now, this dashboard would actually be "used" by a second program which thus would "drive" the first one. I guess it would be a neural net, with deep learning and a maximize pleasure / minimize pain goal. The first program (the one that can be driven through the dashboard) has sensors to feel what the second one does and report it, so we have consciousness. If learning and understanding new things is a source of pleasure, we'll have a nice little AGI.

Started December 11, 2017, 11:22:38 am


Programming language designed specifically for AGI in General AI Discussion

It's Xmas soon, how about making a wishlist!

Can you imagine a programming language designed specifically for Artificial General Intelligence... What features should it have? Would it be compiled or interpreted, or both at the same time? Dynamic or static typing? Strong or weak? What paradigmS? What syntax? What semantics? What ecosystem? What code-sharing system?

Don't be shy, you can ask for anything, like an entire DeepNN running in GPU defined as easily as a javascript one-line function, and access to NLP tools from python, all in the same place!

What's a conscious mind developper's dream language?

EDIT: and don't tell me "C", unreality  ;)

38 Comments | Started November 16, 2017, 02:56:14 pm

Don Patrick

A free summarizer browser add-on for Chrome and Firefox in General Software Talk

I created a browser add-on that summarises news and blog articles, with a particular focus on filtering out nonsense (as often featured in A.I. articles). You can find the links for Chrome and Firefox at the start of this article wherein I also explain my methods:

Below is an example. Try it out at your leisure  :)
I would also appreciate ratings or reviews, as that would help raise the add-on's rank in the webstore so more people can find it. I think the internet would be a better place if we could all fast-forward through clickbait.

3 Comments | Started December 10, 2017, 11:36:35 am


Slogen generation via AI (academic research) in General Project Discussion

Hi guys,

currently I am working on a University project in Europe where we plan to use AI for a task to test it.
The task is the following: based on a few event slogens and mottos as inputs we would like an AI mechanism to think, create what the next year slogen, motto could be for an event.
For example: inputs: 2014: ABCD, 2015: FGHD: 2016: EEIUK, 2017: UZHTG,
 output: 2018: ?

How could we solve this issue in a fast and new way? What do think? Is there a website where we could just type in the input and we than we get the result (output)?

Thank you very much your precious time!


2 Comments | Started December 10, 2017, 07:06:16 am


Farming Robots? in AI News

Not the two legged type with the team of horses plowing the fields.
This is much better.

Farming Robots

Started December 10, 2017, 01:27:53 am
What are the main techniques for the development of a good chatbot ?

What are the main techniques for the development of a good chatbot ? in Articles

Chatbots act as one of the most useful and one of the most reliable technological helpers for those, who own ecommerce websites and other similar resources. However, a pretty important problem here is the fact, that people might not know, which technologies it will be better to use in order to achieve the needed goals. Thus, in today’s article you may get an opportunity to become more familiar with the most important principles of the chatbot building.

Oct 12, 2017, 01:31:00 am

Kweri in Chatbots - English

Kweri asks you questions of brilliance and stupidity. Provide correct answers to win. Type ‘Y’ for yes and ‘N’ for no!


FB Messenger






Oct 12, 2017, 01:24:37 am
The Conversational Interface: Talking to Smart Devices

The Conversational Interface: Talking to Smart Devices in Books

This book provides a comprehensive introduction to the conversational interface, which is becoming the main mode of interaction with virtual personal assistants, smart devices, various types of wearables, and social robots. The book consists of four parts: Part I presents the background to conversational interfaces, examining past and present work on spoken language interaction with computers; Part II covers the various technologies that are required to build a conversational interface along with practical chapters and exercises using open source tools; Part III looks at interactions with smart devices, wearables, and robots, and then goes on to discusses the role of emotion and personality in the conversational interface; Part IV examines methods for evaluating conversational interfaces and discusses future directions. 

Aug 17, 2017, 02:51:19 am
Explained: Neural networks

Explained: Neural networks in Articles

In the past 10 years, the best-performing artificial-intelligence systems — such as the speech recognizers on smartphones or Google’s latest automatic translator — have resulted from a technique called “deep learning.”

Deep learning is in fact a new name for an approach to artificial intelligence called neural networks, which have been going in and out of fashion for more than 70 years.

Jul 26, 2017, 23:42:33 pm
It's Alive

It's Alive in Chatbots - English

[Messenger] Enjoy making your bot with our user-friendly interface. No coding skills necessary. Publish your bot in a click.

Once LIVE on your Facebook Page, it is integrated within the “Messages” of your page. This means your bot is allowed (or not) to interact and answer people that contact you through the private “Messages” feature of your Facebook Page, or directly through the Messenger App. You can view all the conversations directly in your Facebook account. This also needs that no one needs to download an app and messages are directly sent as notifications to your users.

Jul 11, 2017, 17:18:27 pm
Star Wars: The Last Jedi

Star Wars: The Last Jedi in Robots in Movies

Star Wars: The Last Jedi (also known as Star Wars: Episode VIII – The Last Jedi) is an upcoming American epic space opera film written and directed by Rian Johnson. It is the second film in the Star Wars sequel trilogy, following Star Wars: The Force Awakens (2015).

Having taken her first steps into a larger world, Rey continues her epic journey with Finn, Poe and Luke Skywalker in the next chapter of the saga.

Release date : December 2017

Jul 10, 2017, 10:39:45 am
Alien: Covenant

Alien: Covenant in Robots in Movies

In 2104 the colonization ship Covenant is bound for a remote planet, Origae-6, with two thousand colonists and a thousand human embryos onboard. The ship is monitored by Walter, a newer synthetic physically resembling the earlier David model, albeit with some modifications. A stellar neutrino burst damages the ship, killing some of the colonists. Walter orders the ship's computer to wake the crew from stasis, but the ship's captain, Jake Branson, dies when his stasis pod malfunctions. While repairing the ship, the crew picks up a radio transmission from a nearby unknown planet, dubbed by Ricks as "planet number 4". Against the objections of Daniels, Branson's widow, now-Captain Oram decides to investigate.

Jul 08, 2017, 05:52:25 am
Black Eyed Peas - Imma Be Rocking That Body

Black Eyed Peas - Imma Be Rocking That Body in Video

For the robots of course...

Jul 05, 2017, 22:02:31 pm

Winnie in Assistants

[Messenger] The Chatbot That Helps You Launch Your Website.

Jul 04, 2017, 23:56:00 pm