[8pla.net]

In a large non-relational system, without unique indices, duplicates may creep in:

Code

[
	{
		"what": "Salary",
		"who": [
			{
				"first": "John",
				 "last": "Doe",
				 "paid": "1500.00"
			}
		],
		"when": "Friday"
	},
		{
		"what": "Salary",
		"who": [
			{
				"first": "John",
				 "last": "Doe",
				 "paid": "1500.00"
			}
		],
		"when": "Friday"
	}
]

[8pla.net]

Yet unique indices are machine generated when the JSON gets decoded:

Code

array (
  0 => 
  array(
     'what' => 'Salary',
     'who' => 
    array (
      0 => 
      array(
         'first' => 'John',
         'last' => 'Doe',
         'paid' => '1500.00',
      ),
    ),
     'when' => 'Friday',
  ),
  1 => 
  array(
     'what' => 'Salary',
     'who' => 
    array (
      0 => 
      array(
         'first' => 'John',
         'last' => 'Doe',
         'paid' => '1500.00',
      ),
    ),
     'when' => 'Friday',
  ),
)

Now unique indices tell them apart.

[Zero]

Well it seems fair: duplicates happen in real life. It's all about the resolution, the precision of data. In very low resolution, say 8x8 pic, one human really looks like another one. They're duplicate. But if you get better resolution, say 64x64, you start seeing little details, that help you distinguish humans. Same goes here. "John Doe" and "John Doe" look like duplicates, just because we don't have enough details. If you add their age in the dataset for example, maybe you can distinguish them, and tell who's who.

[8pla.net]

"duplicates happen in real life", Zero said.
 Yes they do, but they can cause problems
 such as the Y2K bug which could not tell
 the difference between years 2000 & 1900.

[spydaz]

No lag at all. Mitsuku usually processes and responds to any input under half a second even when dealing with hundreds of users at once. She has to search and process over 300,000 categories and the responses are extremely quick.

This has also been a major problem, Response time!

Is it a programming issue(refactor/Simplify) or data searching issue?

How do you reduce the response time?

I constantly debate whether to move the Processing and data to the cloud to have some grid computing power?