[Zero]

How do you imagine an intelligent user interface? I'm talking about what happens on the screen.

People who are not comfortable with computers often don't use several windows simultaneously, and are surprised when I show them how to split screen. So I tend to think that fullscreen activities would be the most natural way of doing it. Just like smartphones.

Now what's the "intelligent" in "intelligent interface"... probably something that "gets used to you" somehow, but if it's just about showing the most often used tools & links, well it has been done already, and it's not particularly smart. So how do we get further?

Maybe initiative would make a difference. The capacity to take the lead, so to speak. Instead of an interface that waits for you to choose what you want, this would be an interface that actively tries to propose new stuff / previously unseen information / previously undone activities.

I don't know, I'm just thinking it while I'm typing it.
What's your take?

[ivan.moony]

I imagine a voice control. You have a conversation with the computer, and it occasionally pops up what you want to see. In the same time, a regular UI is responding to mouse and keyboard if you don't want to use voice.

Regular UI is good, and although it is verbose, it may be the best way to communicate with computer. Just like trying to use math symbols versus explaining in words the same math symbols. Voice command may be more approachable to novice users, but once they get into the stunt, I think regular UI is more suitable.

Those are some of my thoughts about combining the current technology with NN conversational agent. But probably there are some unexplored possibilities using just mouse/keyboard.

[WriterOfMinds]

I'm thinking a really exceptional help system. If you asked it "How do I do this?" with the software, it could take over the cursor temporarily and walk you through the process. So no more need to look up how-to articles or video tutorials on YouTube.

What Ivan is proposing goes further, I think: not so much a help system to teach people how to use the existing commands and menus, but rather, a further layer of abstraction that accepts commands directly in natural language. I think that's useful too - it could become the next step past GUIs in "ease of use" evolution.

[infurl]

I would like an intelligent agent to behave like another team member. I work as part of a team and all our interactions are via instant messaging, email, and journal entries so we can maintain an audit trail of what we do and what we know and when we know it. We use voice and video conferencing tools rarely and never for anything important. If I couldn't distinguish between the software agent and another team member I would be happy to work with it.

[frankinstien]

Welcome to my nightmare... Microsoft's speech-to-text software of yesteryear was trash but the newer version is on par with Cortona and is excellent! Problem, it only works with UWP. UWP is a bastardized .net Com wrapper nightmare. Because UWP is based on com interfaces that were the thing way back when and still are for hardcore C++ programmers they became utterly useless when literally 15 years of reflection has been implemented with .NET.

In any case, there's still a problem with voice commands and it just gets frustrating when you have to prefix every command with "Cortona", or "Computer" or whatever name you want to call your agent it becomes mind-numbing after just several utterances. Also, I can type faster than I can talk and when you master an app you learn its shortcuts, and the keyboard screams much faster than using a mouse.

Now have you noticed that when you get that aha moment, you have this notion of understanding but nothing has been translated into words yet. Yet when you need to translate that notion or idea it's almost instant as to how it turns into a well grammatically formatted sentence. Literally very low latency between thought and serialization into words. So, I'm thinking that Musk is on a streek and a mind to machine interface is much better than some smart agent trying to guess your intention, which is hard even for human beings. But a mind-machine interface means thoughts can get translated into sequences of actions, code, and paragraphs if not pages of words. I mean just thinking of ideas and seeing them appear on your word doc into well-formatted grammatically correct sentences will make you soo much more productive and the feedback or impressions you get back from this kind of interface is immediate. Also, think how more productive you can be with images, where you're just thinking about what something should look like and a 3D model just appears where you can manipulate, and rotate it.

The better interface is the one that makes you much more productive, if not addicted to working with it, where the latency is milliseconds to see your thoughts turn into results...

[MagnusWootton]

I imagine a voice control. You have a conversation with the computer, and it occasionally pops up what you want to see. In the same time, a regular UI is responding to mouse and keyboard if you don't want to use voice.

Regular UI is good, and although it is verbose, it may be the best way to communicate with computer. Just like trying to use math symbols versus explaining in words the same math symbols. Voice command may be more approachable to novice users, but once they get into the stunt, I think regular UI is more suitable.

Those are some of my thoughts about combining the current technology with NN conversational agent. But probably there are some unexplored possibilities using just mouse/keyboard.

with the keyboard and mouse there, i never really felt the need for voice control on the computer,  but I think if u have a robot it then hits a new level of usefulness, and I think that might actually hit mainstream if we ever get fully automatic bot businesses going.

[Don Patrick]

I consider Iron Man's Jarvis an intelligent interface in the sense that it anticipates the user's next steps. In that vain, I've been upgrading my AI program with predictive skills that have it suggest to do something for you, or offer tools for your presumed goal.



As a verbal interface it could possibly get as annoying as Clippy, but let's say we take the other approach: Toss in a neural network and let it monitor your computer habits. Now, you've clicked "Start", you're hovering the mouse over "games", and you've been playing Tomb Raider every other day around this hour. Wouldn't it be helpful if it tossed up a shortcut directly to that game? We're still asked to manually make desktop shortcuts, or only get a list of "most recently used". Predictive shortcuts would be intelligent and useful.