[AndyGoode]

How does the brain store images? This is a question that has perplexed me for a long time. The basic problem is that images are so extremely complex, especially from scenes of the real world where there exist shadows and imperfections, that geometry won't work as a representational method, unless there exists some new branch of geometry that hasn't been invented or popularized yet. [There does exist a branch of math called 'shape theory' but it's pretty stupid and impractical, in my opinion.]

A related question, one that I saw in a book on AI, was how humans can 'zoom in' on a memory of a complex or large object like a map or building, to whatever resolution they want, like remembering the exact location of a rock along the side of a road located in the middle of an entire country with which they are familiar, or the location of a dent in a piece of furniture in a house where they lived. The range of zoom in these cases must be many orders of magnitude. Also, we humans don't suddenly reach a point where the smallest objects visualized appear to our brains as pixels, which implies our brains aren't storing images in pixelized formats like JPEG or GIF or BMP.

Another related question is how brains even represent the shape of a well-known mathematical curve. For example, how can we recognize the difference between a parabola versus a catenary at a glance, or the difference between a dampened sine wave and a Bessel function at a glance?





I did a little search on how curves are stored on computers to get a better feel for the topic. That alone is a huge topic. Graphical computer games, for example, often use quadtrees...

Quadtree Explanation
Dec 3, 2014
MrHeyheyhey27


...and SVG image files use functions that allow indefinite zoom due to storing curves as mathematical formulas...

A beginners guide to SVG | Part One: The Why, What, and How
Jun 20, 2018
Kevin Powell


...and Bezier curves are cool, smooth curves with infinite resolution since they too are based on math formulas...

Bezier Curves
Oct 14, 2014
0612 TV w/ NERDfirst


...and computerized road maps use some form of smoothing, maybe splines, between their stored data points...

http://roadcurvature.com/technology/

...so are our brains using mathematics like that used in SVG format or Bezier curves?

It also seems to me that computer trees would be useful for storing images, especially in the way that JPG images compress image data so that they don't have to store as many pixels as a BMP file, where the leaves of the tree represent arbitrarily-sized regions, which varies slightly from quadtrees that have pre-defined leaf sizes, but so far I haven't been able to find any references to such work.

----------

(p. 7)
      Introduction

What is shape? What is form? To say that two objects have the same shape has an
intuitively obvious, but very imprecise meaning. As was pointed out by Lord and
Wilson in their preface to [69], a mathematics of form description and analysis is
greatly needed. This monograph summarises a theory that may help towards this
goal, at least in the study of irregular shapes.
   The majority of the techniques of geometric pattern description require that the
objects being studied be smooth and fairly regular. Methods from differential
geometry, for instance, require smoothness whereas algebraic topological methods
require that the object, whether a physical or an abstract one, may be built up from
cells or simplices (see Spanier [96]). Increasingly, these methods have been applied,
to varying extents, to diverse practical problems of shape, and pattern description
and analysis (see Faux and Pratt [39] and Gasson [46]).
   Naturally occurring objects are rarely smooth and, as the work on fractals has
shown, are by their very nature irregular. Within the abstract setting also, objects
frequently occur that can be arbitrarily irregular, for instance closed bounded subsets
of an Euclidean space. Is it possible to extend methods used in the study of smooth
or regular geometric objects to such as these? With regard to the methods of algebraic
topology, the answer is positive and the resulting theory is known as shape theory.

Cordier, J.-M. and T. Porter. 1989. Shape Theory: Categorical Methods of Approximation. New York: John Wiley & Sons.

[LOCKSUIT]

"we humans don't suddenly reach a point where the smallest objects visualized appear to our brains as pixels, which implies our brains aren't storing images in pixelized formats like JPEG or GIF or BMP."

But I do see pixels when I look at a cat down the end of the road. My vision is pixels, but what i c is features. I may see a little yellow blob with a spike on its back, which is from the cat, but looks like a horn shape maybe, it is the lowest features, like lines a balls.

[goaty]

As humans, we can see pixels if we want,  we don't even have to see whats actually there at all for that matter.

But a robot needs a definite method for doing it,  any way under the sun you could try.
Doing image classification in 2d (like yolo), or 3d (like Kinect) is possible,  you can use a point cloud, or you can use triangles made from corners.  Its a lot like how they make video games different ways, doing computer vision.

[HS]

We definitely don't see what's there. It seems like we see with associations, eg, triggered memory, since logic is part of memory, this is also triggers "everyday" logic. This is all melted at the edges and fitted into a unified view which makes "everyday" sense to us.

[goaty]

That's fricken amazing,  but that last one doesn't seem to be true,  but the others are.

[LOCKSUIT]

Looks like nearby context decides what something is recognized as, its rotation, size, location, color, or even if its a cat but truly a dog, even spins if see spirals. What we think we see, is based on the WHOLE context bag of features, the dog, the brick wall, the milk, the babe. That's why commercials sell cars with hot muffin cakes next to em.

I better start doing that.

[goaty]

Ive got my computer vision thing right,  but I haven't really thought about the reality of it to it,  most of its artificially the same,  its all just a bunch of dots connecting each other, Im finding you need very little information to describe the most of things,  but when it comes to the point where the robot has no response to stimuli,  that's where the gaps are... what its missing.   that visual show, is to do with consciousness, something I never study much at all, because there is more pertinent things to do, getting this thing running at all.

[AndyGoode]

We definitely don't see what's there. It seems like we see with associations, eg, triggered memory, since logic is part of memory, this is also triggers "everyday" logic. This is all melted at the edges and fitted into a unified view which makes "everyday" sense to us.

Nice examples of optical illusions, but that doesn't address the issue of the final memory storage scheme. The eye and associated visual areas of the brain use heuristics as a way to speed up processing at the cost of accuracy and objectivity, but even assuming that the ensuing illusions get stored in their inaccurate manner, how are they physically stored, using biological neurons? I think you hit on the answer--somehow it's done with associations instead of pixels or math. That narrows it down, but how exactly can neural associations create an image? Last night a possible answer hit me, but it's a little disturbing, and I might want to write an article about it before disclosing it.

[lordjakian]

I enjoy the supposition that memories are impressions.....lots of es sounds secures the way....An impression of events.

But as for how the brain does it.......no good.

The brain will have an idea....not  a full idea, but enough of an idea with enought knowledge about the feedback.....able to create the impression.......but your arm is missing.

[HS]

but that doesn't address the issue of the final memory storage scheme.

Not sure how to investigate this, but Ivan linked a cool article on fractals:

Is consciousness a fractal?

I also googled it: https://www.google.com/search?client=ubuntu&channel=fs&q=is+consciousness+a+fractal%3F&ie=utf-8&oe=utf-8

Seems like it's worth a look and a think.

[AndyGoode]

but Ivan linked a cool article on fractals:

I don't like fractals. They look like they'd be sharp to the touch.

[HS]

Fair enough.