Hey guys, what's up?
So, I realized that for a long time I've been looking for a single middleware that would be able to host the many shapes intelligence can take, when in fact there can't be such a thing, unless you're ok to end up doing intractable things. I was wrong, this is not what I need. I need a common blackboard, rather than a common middleware. Like a central database, all in-memory. Except different parts of the program would be interested by different aspects of the content, which calls for a NoSQL-like solution.
NoSQL goes like: you set up views (https://docs.couchdb.org/en/stable/ddocs/views/intro.html), and whenever you throw data at it, the db updates views to keep em ready for you. Instead of running 1 query over 100K entries (at query-time), you rather check 1 datum against 100 views (at update-time). Ok it's more complicated but you get the point. Very fast.
I first looked at PouchDB's Github's Issue section. Then I decided to roll my own.
I did a first version, all small. Then I made another version, even smaller and a lot more powerful, by extending Javascript's Map base class. It's the very core of what makes NoSQL cool (to me, at least), once you drop all the fancy. Of course I don't have revs, but I don't need it in my use case. Here:
// Sensitive Maps
class SMap extends Map {
sensors = {};
sense(id) {
return this.sensors[id].members;
}
removeSensor(id) {
delete this.sensors[id];
}
sensor(id, validator) {
this.sensors[id] = { validator, members: new Set() };
this.forEach((value, key) => {
if (validator(key, value))
this.sensors[id].members.add(key);
});
}
set(...args) {
for (let s in this.sensors)
if (this.sensors[s].validator(args[0], args[1]))
this.sensors[s].members.add(args[0]);
else
this.sensors[s].members.delete(args[0]);
return super.set(...args);
}
delete(...args) {
for (let s in this.sensors)
this.sensors[s].members.delete(args[0]);
return super.delete(...args);
}
}
You'd use it like this:
let smap = new SMap();
smap.set('i1', 'foo');
smap.sensor('strings', (k,v) => typeof v == 'string');
smap.set('i2', 'bar');
smap.set('i3', 4);
console.log(smap.sense('strings'));
// => Set(2) { 'i1', 'i2' }
The initial need was to assemble 2 of my previous experiments: the consnet, and the more recent belief-based programming language. Now with Smaps, I think I can assemble pretty much anything, it's like a super-tiny glue.
:P
The speed doesn't depend on the size of the db, it only depends on the number of changes made to it. The implementation could be faster I guess.
This one should be a bit faster (avoiding a few useless sensing).
// Sensitive Maps
class SMap extends Map {
sensors = {};
updated = new Set();
deleted = new Set();
constructor(content) {
super();
if (content)
for (let [k, v] of content) this.set(k, v);
}
sense(id) {
if (this.updated.size || this.deleted.size) {
for (let s in this.sensors) {
for (let key of this.updated)
if (this.sensors[s].validator(key, this.get(key)))
this.sensors[s].members.add(key);
else
this.sensors[s].members.delete(key);
for (let key of this.deleted)
this.sensors[s].members.delete(key);
}
this.updated.clear();
this.deleted.clear();
}
return this.sensors[id].members;
}
removeSensor(id) {
delete this.sensors[id];
}
sensor(id, validator) {
this.sensors[id] = { validator, members: new Set() };
this.forEach((value, key) => {
if (validator(key, value))
this.sensors[id].members.add(key);
});
}
set(...args) {
this.updated.add(args[0]);
this.deleted.delete(args[0]);
return super.set(...args);
}
delete(...args) {
this.deleted.add(args[0]);
this.updated.delete(args[0]);
return super.delete(...args);
}
}
Edit: faster again.
Edit: added a constructor so we can clone the dbase.
Here is the new version.
// Sensitive Maps
class SMap extends Map {
sensors = {};
constructor(content) {
super();
if (content)
for (let [k, v] of content) this.set(k, v);
}
sense(id) {
return this.sensors[id].solution;
}
removeSensor(id) {
delete this.sensors[id];
}
sensor(id, initial, reducer) {
this.sensors[id] = { reducer, solution: initial };
this.forEach((value, key) => {
this.sensors[id].solution =
this.sensors[id].reducer(
this.sensors[id].solution,
key,
value
)
});
}
touch(key, previous) {
for (let s in this.sensors)
this.sensors[s].solution =
this.sensors[s].reducer(
this.sensors[s].solution,
key,
this.get(key),
previous
);
}
set(...args) {
let previous = this.get(args[0]);
let result = super.set(...args);
this.touch(args[0], previous);
return result;
}
delete(...args) {
let previous = this.get(args[0]);
let result = super.delete(...args);
this.touch(args[0], previous);
return result;
}
}
Example usage.
let smap = new SMap();
smap.set('i1', "foo");
smap.set('i2', "bar");
smap.sensor("string counter", 0, function(solution, key, newValue, oldValue) {
if (typeof oldValue != "string" && typeof newValue == "string") ++solution;
if (typeof oldValue == "string" && typeof newValue != "string") --solution;
return solution;
})
smap.sensor("string selector", new Set(), function(solution, key, newValue, oldValue) {
if (typeof newValue == "string")
solution.add(key);
else
solution.delete(key);
return solution;
});
smap.set('i3', "baz");
console.log("counter:", smap.sense("string counter"));
smap.set('i3', 4);
console.log("counter:", smap.sense("string counter"));
console.log("selector:", smap.sense("string selector"));
// counter: 3
// counter: 2
// selector: Set(2) { 'i1', 'i2' }
Edit: corrected a wrong behavior (when overwriting an existing sensor).
Edit: another bug died today.
Edit: added oldValue argument.
Edit: yet another bug.