Crash Crash Crash Crash Crash
Today I tried to fix the query by reducing the amount of data used within the database. You can view it if your are in the Arup githup Repo. Essentially it is flipped around and attempts to reduce the amount of strained with repetitive searches in the beacon detection database (800k documents).
Before, it used to find all the unique detections and apply a search for through the detection database. The proble with this is that it attempts to search each and every collection meaning that if it had 5 beacons in the database it would check 800,000 X 5. Therefore we find the right detections by using a
$group, a simmilar methord but only displaying the correct detections. It then applies the check with the beacons database to assign the correct detections.
There are still a few bugs left that needs to be sorted out and check the quality when compared agaisnt the past. It also still breaks with the 800k database, so its probally the size since its about 12gigs.